A robot visual positioning method based on feature matching

Abstract

The present invention discloses a robot visual positioning method based on feature matching, comprising the following steps: collecting and saving color images of robot motion scenes, preprocessing the color images of robot motion scenes, obtaining image sequence data that is cropped and subtracted from the mean; calculating the characteristic values ​​of all color images; in a set motion space, the robot uses a photo taken with a monocular camera as a query image, crops the image to be queried and subtracts the mean value of the query image in step S1, and calculates the characteristic value of the query image; compares the characteristic value of the query image with the characteristic value of the color image, finds the two images with the closest distance and their spatial coordinates, and obtains the spatial position of the robot at this time. The present invention avoids extracting artificially designed feature points in the use of only image positioning, greatly reduces hardware costs, greatly reduces workload, and improves analysis accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of robot visual positioning, and in particular relates to a robot visual positioning method based on feature matching. Background Art

[0002] In the research of autonomous mobile robot related technologies, navigation technology is an important core technology, which plays a key role in the autonomous movement and decision-making of robots. The premise for mobile robots to realize autonomous navigation in the working environment is to realize their own positioning. Positioning is particularly important for mobile robots to realize path planning, obstacle avoidance and other functions. The positioning functions of mobile robots include GPS, WIFI, NFC, Bluetooth, laser sensors, visual sensors, etc. When mobile robots use WIFI, NFC, Bluetooth and other devices for positioning, auxiliary equipment needs to be installed in the environment in advance, which will cause changes in the environment; laser sensors can achieve higher positioning accuracy, but the equipment is expensive and the cost is high; visual sensors are relatively cheap and have lower environmental requirements. At present, most robot positioning methods use multiple sensors, and the robot vision positioning method that only uses a single purpose is not involved. Summary of the invention

[0003] Purpose of the invention: In order to overcome the problems in the prior art that most existing robot positioning methods use multiple sensors, are costly and difficult to implement, a robot visual positioning method based on feature matching is provided.

[0004] Technical solution: To achieve the above purpose, the present invention provides a robot visual positioning method based on feature matching, comprising the following steps:

[0005] S1: Collect and save the color image of the robot motion scene, pre-process the color image of the robot motion scene, and obtain image sequence data that is cropped and mean-subtracted;

[0006] S2: Calculate the feature values ​​of all color images based on the image sequence data;

[0007] S3: In the set motion space, the robot uses the photo taken by the monocular camera as the query image, crops the image to be queried and subtracts the mean value of the query image in step S1, and calculates the feature value of the query image;

[0008] S4: Compare the eigenvalues ​​of the query image with the eigenvalues ​​of the color image, find the two images with the closest distance and their spatial coordinates, and obtain the spatial position of the robot at this time.

[0009] Furthermore, the preprocessing of the color image in step S1 includes the following process:

[0010] A1: Annotate the spatial position of the color image of the robot motion scene;

[0011] A2: cropping the color image obtained in step A1, obtaining and saving the cropped color image;

[0012] A3: Calculate the image mean of the cropped color image, subtract the mean from all the cropped color images, and obtain and save the preprocessed color image.

[0013] Furthermore, in step A1, the following formula is used to mark the spatial position:

[0014] L(n)=(x n y n z n )

[0015] Among them, L(n) is the labeled spatial information, x n ,y n and z n They are the x, y and z coordinates of the robot in the nth frame of color image.

[0016] Furthermore, in step A2, the color image is cropped by regular framing, specifically:

[0017] The upper left corner vertex of the color image is taken as the coordinate origin (0, 0), the horizontal direction is the x-axis, and the vertical direction is the y-axis to establish a plane image coordinate system;

[0018] Then, the coordinates of the upper left corner (0, 0), the upper right corner (224, 0), the lower left corner (0, -224) and the lower right corner (224, -224) are used as the boundary range of the rectangular frame to determine the cropping position of the image, and the cropped color image is obtained and saved.

[0019] Furthermore, the calculation process of the image mean in step A3 is:

[0020] Calculate the mean of the three color channels of the color image red R, green G and blue B respectively:

[0021]

[0022]

[0023]

[0024] Among them, R mean , G mean and B mean are the mean values ​​of the three color channels, red R, green G, and blue B. ni is the value of the red R channel of the i-th pixel in the n-th frame image, Gni is the value of the green G channel of the i-th pixel in the n-th frame image, B ni It is the value of the blue B channel of the ith pixel in the nth frame image.

[0025] Furthermore, the mean value is subtracted from the cropped color image in step A3, and the red R, green G and blue B color channels of all pixels in the color image are subtracted from the mean value in the order from top to bottom and from left to right. mean , G mean and B mean , get and save the image after subtracting the mean:

[0026] R1 ni =R ni -R mean

[0027] G1 ni =G ni -G mean

[0028] B1 ni =B ni -B mean

[0029] Among them, R1 ni G1 is the new red R channel value obtained by subtracting the red R channel mean from the red R channel of the i-th pixel of the n-th frame color image; ni B1 is the new green G color channel value obtained by subtracting the green G channel mean from the green G channel of the i-th pixel in the n-th frame color image; ni The new blue B channel value is obtained by subtracting the blue B channel mean from the blue B channel of the i-th pixel in the n-th frame color image.

[0030] Furthermore, the calculation formula of the characteristic value of the color image in step S2 is:

[0031]

[0032] The characteristic value of each frame of the image is obtained by calculating the values ​​of the three color channels of each pixel of each frame of the color image, and P(n) is the characteristic value of the nth frame of the color image.

[0033] Furthermore, the step S3 is specifically as follows:

[0034] R1 i =R i -R mean

[0035] G1 i =G i -Gmean

[0036] B1 i =B i -B mean

[0037] Among them, R i , G i and B i are the values ​​of the three color channels of red R, green G, and blue B at the i-th pixel of the query image, R mean , G mean and B mean are the mean values ​​of the three color channels, red R, green G, and blue B. ni , G ni and B ni are the values ​​of the three color channels of red R, green G and blue B of the i-th pixel after subtracting the mean from the query image, R1 i , G1 i and B1 i are the values ​​of the three color channels of red R, green G, and blue B of the i-th pixel in the query image minus the mean;

[0038] The feature value of the query image is obtained by the following formula:

[0039]

[0040] Among them, P is the eigenvalue of the image.

[0041] Furthermore, the specific expression of step S4 is as follows:

[0042] E(n)=|PP(n)|

[0043]

[0044] Where E(n) is the absolute value of the difference between the query image and the n-th frame image, and P(n) is the feature value of the n-th frame image;

[0045] L min1 =(x mn1 y mn1 z mn1 )

[0046] L min2 =(x mn2 y mn2 z mn2 )

[0047] Among them, the two smallest values ​​of E(n) min1 , E min2 and its corresponding sequence image and its spatial position Lmin1 and L min2 ;x mn1 ,y mn1 , z mn1 is the spatial position L min1 The coordinates on the x, y, and z axes, x mn2 ,y mn2 , z mn2 is the spatial position L min2 Coordinates on the x, y, and z axes;

[0048] L=(xyz)

[0049]

[0050] Wherein, L is the spatial position of the query image, and the midpoint of the corresponding spatial positions of the two frames of images with the smallest eigenvalue difference is taken as the spatial coordinate of the query image.

[0051] The present invention provides a robot visual positioning method based on feature matching, which avoids extracting artificially designed feature points in positioning using only images, greatly reduces hardware costs, greatly reduces workload, and improves analysis accuracy. The present invention has good practical application value and engineering significance in improving robot positioning accuracy, improving the calculation efficiency of robot positioning, and reducing the cost of robot positioning equipment.

[0052] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0053] (1) The robot can achieve relatively high positioning accuracy by relying on a monocular camera, solving the problem that the monocular camera does not have spatial scale;

[0054] (2) Reduced robot sensor costs and algorithm difficulty;

[0055] (3) Reduced computational complexity and requirements for robot hardware;

[0056] (4) The robot has fast positioning speed, high accuracy and strong resistance to background interference. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of the workflow of a robot visual positioning method based on feature matching provided by an embodiment of the present invention;

[0058] Figure 2 An image of a mobile robot using monocular vision positioning accuracy error comparison in a working environment provided by an embodiment of the present invention;

[0059] Figure 3 This is an actual effect image of a positioning trajectory of a mobile robot in an indoor working environment provided by an embodiment of the present invention.

[0060] Figure 4 This is an actual effect image 1 of a mobile robot autonomously positioning and moving in an indoor environment provided by an embodiment of the present invention.

[0061] Figure 5 This is a second actual effect image of a mobile robot autonomously positioning and moving in an indoor environment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0063] The present invention provides a robot visual positioning method based on feature matching, as shown in the figure, which includes the following steps:

[0064] S1: Collect and save the color image of the robot motion scene, pre-process the color image of the robot motion scene, and obtain image sequence data that is cropped and mean-subtracted;

[0065] S2: Calculate the feature values ​​of all color images based on the image sequence data;

[0066] S3: In the set motion space, the robot uses the photo taken by the monocular camera as the query image, crops the image to be queried and subtracts the mean value of the query image in step S1, and calculates the feature value of the query image;

[0067] S4: Compare the eigenvalues ​​of the query image with the eigenvalues ​​of the color image, find the two images with the closest distance and their spatial coordinates, and obtain the spatial position of the robot at this time.

[0068] The preprocessing of the color image in step S1 includes the following steps:

[0069] A1: Annotate the spatial position of the color image of the robot motion scene;

[0070] A2: cropping the color image obtained in step A1, obtaining and saving the cropped color image;

[0071] A3: Calculate the image mean of the cropped color image, subtract the mean from all the cropped color images, and obtain and save the preprocessed color image.

[0072] In step A1, the following formula is used to mark the spatial position:

[0073] L(n)=(x n y n z n )

[0074] Among them, L(n) is the labeled spatial information, xn ,y n and z n They are the x, y and z coordinates of the robot in the nth frame of color image.

[0075] In step A2, the color image is cropped by regular framing, specifically:

[0076] The upper left corner vertex of the color image is taken as the coordinate origin (0, 0), the horizontal direction is the x-axis, and the vertical direction is the y-axis to establish a plane image coordinate system;

[0077] Then, the coordinates of the upper left corner (0, 0), the upper right corner (224, 0), the lower left corner (0, -224) and the lower right corner (224, -224) are used as the boundary range of the rectangular frame to determine the cropping position of the image, and the cropped color image with a size of 224×224 is obtained and saved.

[0078] The calculation process of the image mean in step A3 is:

[0079] Calculate the mean of the three color channels of the color image red R, green G and blue B respectively:

[0080]

[0081]

[0082]

[0083] Among them, R mean , G mean and B mean are the mean values ​​of the three color channels, red R, green G, and blue B. ni is the value of the red R channel of the i-th pixel in the n-th frame image, G ni is the value of the green G channel of the i-th pixel in the n-th frame image, B ni It is the value of the blue B channel of the ith pixel in the nth frame image.

[0084] The mean value is subtracted from the cropped color image in step A3. In the order from top to bottom and from left to right, the three color channels of red R, green G and blue B of all pixels in the color image are subtracted from R mean , G mean and B mean , get and save the image after subtracting the mean:

[0085] R1 ni =R ni -R mean

[0086] G1 ni=G ni -G mean

[0087] B1 ni =B ni -B mean

[0088] Among them, R1 ni G1 is the new red R channel value obtained by subtracting the red R channel mean from the red R channel of the i-th pixel of the n-th frame color image; ni B1 is the new green G color channel value obtained by subtracting the green G channel mean from the green G channel of the i-th pixel in the n-th frame color image; ni The new blue B channel value is obtained by subtracting the blue B channel mean from the blue B channel of the i-th pixel in the n-th frame color image.

[0089] The calculation formula of the characteristic value of the color image in step S2 is:

[0090]

[0091] The characteristic value of each frame of the image is obtained by calculating the values ​​of the three color channels of each pixel of each frame of the color image, and P(n) is the characteristic value of the nth frame of the color image.

[0092] Step S3 is specifically as follows:

[0093] R1 i =R i -R mean

[0094] G1 i =G i -G mean

[0095] B1 i =B i -B mean

[0096] Among them, R i , G i and B i are the values ​​of the three color channels of red R, green G, and blue B at the i-th pixel of the query image, R mean , G mean and B mean are the mean values ​​of the three color channels, red R, green G, and blue B. ni , G ni and B ni are the values ​​of the three color channels of red R, green G and blue B of the i-th pixel after subtracting the mean from the query image, R1 i , G1i and B1 i are the values ​​of the three color channels of red R, green G, and blue B of the i-th pixel in the query image minus the mean;

[0097] The feature value of the query image is obtained by the following formula:

[0098]

[0099] Among them, P is the eigenvalue of the image.

[0100] The specific expression of step S4 is as follows:

[0101] E(n)=|PP(n)|

[0102]

[0103] Where E(n) is the absolute value of the difference between the query image and the n-th frame image, and P(n) is the feature value of the n-th frame image;

[0104] L min 1=(x mn1 y mn1 z mn1 )

[0105] L min 2=(x mn2 y mn2 z mn2 )

[0106] Among them, the two smallest values ​​of E(n) min 1.E min 2 and its corresponding sequence image and its spatial position L min 1 and L min 2;x mn1 ,y mn1 , z mn1 is the spatial position L min 1 is the coordinate on the x, y, and z axes, x mn2 ,y mn2 , z mn2 is the spatial position L min 2 coordinates on the x, y, and z axes;

[0107] L=(xyz)

[0108]

[0109] Wherein, L is the spatial position of the query image, and the midpoint of the corresponding spatial positions of the two frames of images with the smallest eigenvalue difference is taken as the spatial coordinate of the query image.

[0110] In this embodiment, the above method is used to locate the robot's spatial position. The specific positioning accuracy error is as follows: Figure 2 The actual positioning trajectory and application effect are shown in Figure 3-Figure 5 shown.

[0111] In addition, this embodiment further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the robot visual positioning method based on feature matching provided by the present invention. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0112] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

Claims

1. A robot visual positioning method based on feature matching, characterized in that: The steps include: S1: Collect and save the color image of the robot motion scene, pre-process the color image of the robot motion scene, and obtain image sequence data that is cropped and mean-subtracted; S2: Calculate the feature values ​​of all color images based on the image sequence data; S3: In the set motion space, the robot uses the photo taken by the monocular camera as the query image, crops the query image and subtracts the mean of the color channel, and calculates the eigenvalue of the query image; S4: Compare the eigenvalues ​​of the query image with the eigenvalues ​​of the color image, find the two images with the closest distance and their spatial coordinates, and obtain the spatial position of the robot at this time.

2. The robot visual positioning method based on feature matching according to claim 1, characterized in that: The preprocessing of the color image in step S1 includes the following process: A1: Annotate the spatial position of the color image of the robot motion scene; A2: cropping the color image obtained in step A1, obtaining and saving the cropped color image; A3: Calculate the image mean of the cropped color image, subtract the mean from all the cropped color images, and obtain and save the preprocessed color image.

3. The robot visual positioning method based on feature matching according to claim 2, characterized in that: In step A1, the following formula is used to mark the spatial position: L(n)=(x n y n z n ) Among them, L(n) is the labeled spatial information, x n ,y n and z n They are the x, y and z coordinates of the robot in the nth frame of color image.

4. The robot visual positioning method based on feature matching according to claim 2, characterized in that: In step A2, the color image is cropped by regular framing, specifically: The upper left corner vertex of the color image is taken as the coordinate origin (0, 0), the horizontal direction is the x-axis, and the vertical direction is the y-axis to establish a plane image coordinate system; Then, the coordinates of the upper left corner (0, 0), the upper right corner (224, 0), the lower left corner (0, -224) and the lower right corner (224, -224) are used as the boundary range of the rectangular frame to determine the cropping position of the image, and the cropped color image is obtained and saved.

5. The robot visual positioning method based on feature matching according to claim 2, characterized in that: The calculation process of the image mean in step A3 is: Calculate the mean of the three color channels of the color image red R, green G and blue B respectively: Among them, R mean , G mean and B mean are the mean values ​​of the three color channels, red R, green G, and blue B. ni is the value of the red R channel of the i-th pixel in the n-th frame image, G ni is the value of the green G channel of the i-th pixel in the n-th frame image, B ni It is the value of the blue B channel of the ith pixel in the nth frame image.

6. The robot visual positioning method based on feature matching according to claim 2, characterized in that: The mean value is subtracted from the cropped color image in step A3. In the order from top to bottom and from left to right, the three color channels of red R, green G and blue B of all pixels in the color image are subtracted from R mean , G mean and B mean , get and save the image after subtracting the mean: R1 ni =R ni -R mean G1 ni =G ni -G mean B1 ni =B ni -B mean Among them, R1 ni The new red R channel value obtained by subtracting the red R channel mean from the red R channel of the i-th pixel of the n-th frame color image; G1 ni B1 is the new green G color channel value obtained by subtracting the green G channel mean from the green G channel of the i-th pixel in the n-th frame color image; ni The new blue B channel value is obtained by subtracting the blue B channel mean from the blue B channel of the i-th pixel in the n-th frame color image.

7. The robot visual positioning method based on feature matching according to claim 1, characterized in that: The calculation formula of the characteristic value of the color image in step S2 is: The characteristic value of each frame of the image is obtained by calculating the values ​​of the three color channels of each pixel of each frame of the color image, and P(n) is the characteristic value of the nth frame of the color image.

8. The robot visual positioning method based on feature matching according to claim 1, characterized in that: The step S3 is specifically as follows: R1 i =R i -R mean G1 i =G i -G mean B1 i =B i -B mean Among them, R i , G i and B i are the values ​​of the three color channels of red R, green G, and blue B at the i-th pixel of the query image, R mean , G mean and B mean are the mean values ​​of the three color channels: red R, green G, and blue B, R1 i , G1 i and B1 i are the values ​​of the three color channels of red R, green G, and blue B of the i-th pixel in the query image minus the mean; The feature value of the query image is obtained by the following formula: Where P is the feature value of the query image.

9. The robot visual positioning method based on feature matching according to claim 1, characterized in that: The specific expression of step S4 is as follows: E(n)=|PP(n)| Wherein, E(n) is the absolute value of the difference between the eigenvalues ​​of the query image and the n-th frame color image, and P(n) is the eigenvalue of the n-th frame color image; L min1 =(x mn1 y mn1 z mn1 ) L min2 =(x mn2 y mn2 z mn2 ) Among them, the two smallest values ​​of E(n) min1 , E min2 and its corresponding sequence image and its spatial position L min1 and L min2 ;x mn1 ,y mn1 , z mn1 is the spatial position L min1 The coordinates on the x, y, and z axes, x mn2 ,y mn2 , z mn2 is the spatial position L min2 Coordinates on the x, y, and z axes; Wherein, L is the spatial position of the query image, and the midpoint of the corresponding spatial positions of the two frames of images with the smallest eigenvalue difference is taken as the spatial coordinate of the query image.

Images