Robot positioning method, chip and robot

Abstract

The invention discloses a robot positioning method, a chip and a robot, and the method comprises the steps: S1, enabling the robot to obtain an environment image through an RGB camera, and enabling the robot to obtain a feature point and a description operator in the environment image through a SuperPoint algorithm; s2, the robot matches the acquired environment image with a previous frame of environment image acquired by the RGB camera through the extracted feature points and description operators; and S3, the robot determines the current position of the robot according to the position of the extracted feature point in the environment image and the position of the corresponding feature point in the matched environment image. According to the robot, visual positioning of the robot is achieved through the RGB camera and the feature point extraction algorithm, the cost of the robot is reduced, the energy consumption of the robot during working is reduced, and the working time of the robot is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and particularly relates to a robot positioning method, a chip and a robot. Background Art

[0002] For a robot to achieve intelligence, a basic technology is the ability to position itself and walk, and indoor navigation technology is a key technology among them. Currently, indoor navigation technologies include inertial sensor navigation, laser navigation, visual navigation, radio navigation, etc., and each technology has its own advantages and disadvantages. Inertial sensor navigation uses gyroscopes, odometers, etc. for navigation and positioning, which is inexpensive but has the problem of long-term drift; laser navigation has high precision but is relatively expensive, and its lifespan is also a problem; traditional visual navigation is computationally complex, requires high processor performance, and has high power consumption and price; radio requires multiple fixed radio transmitters, which is inconvenient to apply and is also relatively expensive. The integration of multiple technologies to achieve low cost and high precision is a development direction of robot navigation technology.

[0003] In existing visual robots, the visual robot mainly obtains the depth map of the environmental image through an RGBD depth camera, and then locates based on the depth information in the depth map, resulting in a relatively high cost and power consumption of the visual robot. Summary of the Invention

[0004] To solve the above problems, the present invention provides a robot positioning method, a chip and a robot. The specific technical solutions of the present invention are as follows: A robot positioning method, the method includes the following steps: S1: The robot obtains an environmental image through an RGB camera, and then the robot obtains feature points and description operators in the environmental image through the SuperPoint algorithm; S2: The robot matches the obtained environmental image with the previous frame of environmental image obtained by the RGB camera through the extracted feature points and description operators; S3: The robot determines the current position of the robot according to the position of the extracted feature points in the environmental image and the position of the corresponding feature points in the matched environmental image.

[0005] Further, in step S1, when the robot obtains an environmental image through an RGB camera, it includes the following steps: Before the robot obtains an environmental image through the RGB camera, it first detects the environmental light intensity through an infrared sensing point outside the lens of the RGB camera; if the environmental light intensity is greater than or equal to the set light intensity, the robot controls the daytime filter of the IR-CUT dual filter of the RGB camera to work, and then obtains the environmental image through the RGB camera; if the environmental light intensity is less than the set light intensity, the robot controls the night filter and fill light of the IR-CUT dual filter of the RGB camera to work, and then obtains the environmental image through the RGB camera.

[0006] Further, when the robot obtains the environmental image through the RGB camera, it obtains one frame of the environmental image every set time, and calculates the current position of the robot based on the current frame of the environmental image and the previous frame of the environmental image.

[0007] Further, in step S1, the robot obtains the feature points and descriptor operators in the environmental image through the SuperPoint algorithm, including the following steps: The robot inputs the environmental image obtained through the RGB camera into the shared encoding network, and the shared encoding network performs convolutional processing and pooling processing on the environmental image to obtain the tensor of the environmental image; The robot inputs the tensor of the environmental image into the feature point decoding network, and the feature point decoding network performs convolutional processing on the tensor of the environmental image, and then performs data organization processing on the tensor of the environmental image to obtain the feature point probability of the pixel points of the environmental image; The robot inputs the tensor of the environmental image into the feature point decoding network, and the feature point decoding network first performs convolutional processing on the tensor of the environmental image, and then performs linear interpolation processing and normalization processing on the tensor of the environmental image to obtain the descriptor operator of the environmental image.

[0008] Further, in step S2, the robot matches the obtained environmental image with the previous frame of the environmental image obtained by the RGB camera through the extracted feature points and descriptor operators, including the following steps: The robot extracts specific points and descriptor operators from the previous frame of the environmental image obtained by the RGB camera; The robot determines the corresponding descriptor operator according to the feature point probability of the pixel points of the environmental image; The robot performs similarity calculation on the corresponding descriptor operators to obtain a similarity matrix; The robot performs augmentation processing on the similarity matrix, and then obtains the optimal assignment of the augmented lower similarity matrix; The robot performs summation calculation on the optimal assignment of the obtained similarity matrix to remove the unmatched feature points and obtain the matched feature points.

[0009] Further, in step S3, the robot determines its current position according to the positions of the extracted feature points in the environmental image and the positions of the corresponding feature points in the matched environmental image, including the following steps: The robot determines the pixel coordinates of the extracted feature points in the environmental image according to the positions of the extracted feature points in the environmental image; The robot converts the pixel coordinates of the feature points in the environmental image into robot coordinates through a coordinate transformation formula; The robot determines the pixel coordinates of the corresponding feature points in the matched environmental image according to the positions of the corresponding feature points in the matched environmental image; The robot converts the pixel coordinates of the feature points in the matched environmental image into robot coordinates through a coordinate transformation formula; The robot respectively uses the feature points in the environmental image and the robot coordinates of the feature points in the matched environmental image as the coordinate origin to construct a feature point coordinate system and determines the feature coordinates of the robot in this coordinate system; The robot determines its position in the current walking map according to the coordinate changes of the robot in the feature point coordinate system.

[0010] Further, after the robot determines its position in the current walking map, the robot performs object recognition on the acquired environmental image, including the following steps: The robot performs grid processing on the acquired environmental image and divides the environmental image into N*N grids; The robot detects the grids of the environmental image to obtain 2N prediction boxes and the confidence of each prediction box; The robot performs target classification on the 2N prediction boxes according to the confidence of the prediction boxes; The robot performs non-maximum suppression calculation on the classified prediction boxes to obtain the recognition result; where N is a natural number greater than 1.

[0011] Further, the robot marks the recognized object in the walking map, including the following steps: After the robot performs object recognition on the acquired environmental image, according to the settings of the recognition model, it names the recognized object and marks the position and name of the object in the walking map.

[0012] A chip with a built-in control program configured to execute the above robot positioning method.

[0013] A robot, the robot includes a main control chip, an RGB camera and a fill light, the main control chip is the above chip, and the RGB camera includes an infrared sensing point and an IR-CUT dual filter.

[0014] Compared with the existing technology, the beneficial effects of the present invention are as follows: The robot described in this application realizes visual positioning of the robot through an RGB camera and a feature point extraction algorithm, which not only reduces the cost of the robot, but also reduces the energy consumption during the operation of the robot and improves the working time of the robot; The robot uses the SuperPoint algorithm to extract specific points, which can effectively resist the interference of different illuminations during the day and at night. Description of the Drawings

[0015] Figure 1 It is a schematic flow chart of a robot positioning method in an embodiment of the present invention. Embodiment

[0016] The embodiments of the present invention will be described in detail below. The described embodiments are illustrated in the accompanying drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0017] The technical solution and its beneficial effects of the present invention will be made clearer and more definite by further describing the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention, but should not be construed as limiting the present invention.

[0018] As Figure 1 shown, a robot positioning method includes the following steps: S1: The robot obtains an environmental image through an RGB camera (a visible light camera sensor is an imager that collects visible light (400 - 700 nm) and converts it into an electrical signal, and then organizes this information to render images and video streams), and then the robot obtains feature points and description operators in the environmental image through the SuperPoint (Self-Supervised Interest Point Detection and Description) algorithm. S2: The robot matches the obtained environmental image with the previous frame of environmental image obtained by the RGB camera through the extracted feature points and description operators; S3: The robot determines its current position according to the positions of the extracted feature points in the environmental image and the positions of the corresponding feature points in the matched environmental image. The robot described in this application realizes visual positioning of the robot through an RGB camera and a feature point extraction algorithm, which not only reduces the cost of the robot, but also reduces the energy consumption during the operation of the robot and increases the working time of the robot; the robot realizes the extraction of specific points through the SuperPoint algorithm, which can effectively resist the interference of different illuminations during the day and at night.

[0019] As one of the embodiments, in step S1, the robot obtains an environmental image through an RGB camera, including the following steps: Before the robot obtains an environmental image through the RGB camera, it first detects the environmental light intensity through the infrared sensing points outside the lens of the RGB camera. If the environmental light intensity is greater than or equal to the set light intensity, the robot controls the daytime filter of the IR-CUT dual filter of the RGB camera to work, and then obtains an environmental image through the RGB camera. If the environmental light intensity is less than the set light intensity, the robot controls the night filter and fill light of the IR-CUT dual filter of the RGB camera to work, and then obtains an environmental image through the RGB camera. The IR-CUT dual filter means that a set of filters is built into the lens group of the RGB camera. When the infrared sensing points outside the lens detect the change in the intensity of the light, the built-in IR-CUT automatic switching filter can automatically switch accordingly according to the intensity of the external light, so that the image reaches the best effect. That is to say, in the daytime or at night, the dual filter can automatically switch the filter, so that the best imaging effect can be obtained both in the daytime and at night. With the help of the IR-CUT dual filter and fill light, the robot can obtain a clear environmental image in the night environment, enabling the robot to avoid obstacles in the night environment, which has high practicability.

[0020] As one of the embodiments, when the robot obtains an environmental image through the RGB camera, it obtains a frame of environmental image every set time, and calculates the current position of the robot according to the current frame of environmental image and the previous frame of environmental image. The robot calculates the current position of the robot through the differential frame method, and the accuracy is relatively high.

[0021] As one of the embodiments, in step S1, the robot obtains feature points and descriptor operators in the environmental image through the SuperPoint algorithm, including the following steps: The robot inputs the environmental image obtained through the RGB camera into a shared encoding network, and the shared encoding network performs convolutional processing and pooling processing on the environmental image to obtain a tensor of the environmental image; The robot inputs the tensor of the environmental image into a feature point decoding network, and the feature point decoding network performs convolutional processing on the tensor of the environmental image, and then performs data organization processing on the tensor of the environmental image to obtain the feature point probability of the pixel points of the environmental image; The robot inputs the tensor of the environmental image into a feature point decoding network, and the feature point decoding network first performs convolutional processing on the tensor of the environmental image, and then performs linear interpolation processing and normalization processing on the tensor of the environmental image to obtain the descriptor operator of the environmental image.

[0022] As one of the embodiments, in step S2, the robot matches the acquired environmental image with the previous-frame environmental image acquired by the RGB camera by using the extracted feature points and descriptor operators, including the following steps: The robot extracts specific points and descriptor operators from the previous-frame environmental image acquired by the RGB camera; The robot determines the corresponding descriptor operators according to the feature point probabilities of the pixel points of the environmental image; The robot calculates the similarity of the corresponding descriptor operators to obtain a similarity matrix; The robot performs augmentation processing on the similarity matrix, and then obtains the optimal assignment of the augmented lower similarity matrix; The robot performs a summation calculation on the optimal assignment of the obtained similarity matrix to remove the mismatched feature points and obtain the matched feature points.

[0023] As one of the embodiments, in step S3, the robot determines its current position according to the positions of the extracted feature points in the environmental image and the positions of the corresponding feature points in the matched environmental image, including the following steps: The robot determines the pixel coordinates of the extracted feature points in the environmental image according to the positions of the extracted feature points in the environmental image; The robot converts the pixel coordinates of the feature points in the environmental image into robot coordinates through a coordinate transformation formula; The robot determines the pixel coordinates of the corresponding feature points in the matched environmental image according to the positions of the corresponding feature points in the matched environmental image; The robot converts the pixel coordinates of the feature points in the matched environmental image into robot coordinates through a coordinate transformation formula; The robot uses the feature points in the environmental image and the feature points in the matched environmental image as the coordinate origins respectively to construct a feature point coordinate system, and determines the feature coordinates of the robot in this coordinate system; The robot determines its position in the current walking map according to the coordinate changes of the robot in the feature point coordinate system.

[0024] As one of the embodiments, after the robot determines its position in the current walking map, the robot performs object recognition on the acquired environmental image, including the following steps: The robot performs grid processing on the acquired environmental image and divides the environmental image into N*N grids; The robot detects the grids of the environmental image to obtain 2N prediction boxes and the confidence of each prediction box; The robot performs target classification on the 2N prediction boxes according to the confidence of the prediction boxes; The robot performs non-maximum suppression calculation on the classified prediction boxes to obtain the recognition result; where N is a natural number greater than 1.

[0025] As one of the embodiments, the robot marks the recognized object in the walking map, including the following steps: After the robot performs object recognition on the acquired environmental image, according to the settings of the recognition model, it names the recognized object and marks the position and name of the object in the walking map.

[0026] A chip with a built-in control program configured to execute the above robot positioning method.

[0027] A robot, the robot includes a main control chip, an RGB camera and a fill light, the main control chip is the above-mentioned chip, and the RGB camera includes an infrared sensing point and an IR-CUT dual filter.

[0028] In the description of the specification, the description with reference to terms such as "in one embodiment", "preferably", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. The schematic expression of the above terms in this specification does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. The connection manner described in the description of the specification has obvious effects and practical effectiveness.

[0029] Through the description of the above structure and principle, those skilled in the art should understand that the present invention is not limited to the above specific implementation manners, and the improvements and substitutions using the well-known technologies in the art on the basis of the present invention all fall within the protection scope of the present invention, which should be defined by each claim.

Claims

1. A robot positioning method, characterized in that, The method includes the following steps: S1: The robot obtains an environmental image through an RGB camera, and then the robot obtains feature points and descriptor operators in the environmental image through the SuperPoint algorithm; S2: The robot matches the obtained environmental image with the previous environmental image obtained by the RGB camera through the extracted feature points and descriptor operators; S3: The robot determines its current position according to the positions of the extracted feature points in the environmental image and the positions of the corresponding feature points in the matched environmental image.

2. The robot positioning method according to claim 1, wherein In step S1, when the robot obtains an environmental image through an RGB camera, it includes the following steps: Before the robot obtains an environmental image through the RGB camera, it first detects the environmental light intensity through the infrared sensing points outside the lens of the RGB camera; If the environmental light intensity is greater than or equal to the set light intensity, the robot controls the daytime filter of the IR-CUT dual filter of the RGB camera to work, and then obtains the environmental image through the RGB camera; If the environmental light intensity is less than the set light intensity, the robot controls the night filter and fill light of the IR-CUT dual filter of the RGB camera to work, and then obtains the environmental image through the RGB camera.

3. The robotic positioning method according to claim 2, wherein When the robot obtains an environmental image through the RGB camera, it obtains one frame of environmental image every set time, and calculates the current position of the robot according to the current frame of environmental image and the previous frame of environmental image.

4. A robot positioning method according to claim 1, characterized in that, In step S1, when the robot obtains feature points and descriptor operators in the environmental image through the SuperPoint algorithm, it includes the following steps: The robot inputs the environmental image obtained through the RGB camera into a shared encoding network, and the shared encoding network performs convolutional processing and pooling processing on the environmental image to obtain a tensor of the environmental image; The robot inputs the tensor of the environmental image into a feature point decoding network, and the feature point decoding network performs convolutional processing on the tensor of the environmental image, and then performs data organization processing on the tensor of the environmental image to obtain the feature point probability of the pixel points of the environmental image; The robot inputs the tensor of the environmental image into a feature point decoding network, and the feature point decoding network first performs convolutional processing on the tensor of the environmental image, and then performs linear interpolation processing and normalization processing on the tensor of the environmental image to obtain the descriptor operator of the environmental image.

5. A robot positioning method according to claim 4, characterized in that, In step S2, when the robot matches the obtained environmental image with the previous environmental image obtained by the RGB camera through the extracted feature points and descriptor operators, it includes the following steps: The robot extracts specific points and descriptor operators from the previous environmental image obtained by the RGB camera; The robot determines the corresponding descriptor operator according to the feature point probability of the pixel points of the environmental image; The robot performs similarity calculation on the corresponding descriptor operators to obtain a similarity matrix; The robot performs augmentation processing on the similarity matrix, and then obtains the optimal assignment of the augmented lower similarity matrix; The robot performs summation calculation on the optimal assignment of the obtained similarity matrix to remove the mismatched feature points and obtain the matched feature points.

6. A robot positioning method according to claim 5, characterized in that, In step S3, the robot determines its current position based on the positions of the extracted feature points in the environmental image and the positions of the corresponding feature points in the matched environmental image, including the following steps: The robot determines the pixel coordinates of the extracted feature points in the environmental image according to the positions of the extracted feature points in the environmental image; The robot converts the pixel coordinates of the feature points in the environmental image into robot coordinates through a coordinate transformation formula; The robot determines the pixel coordinates of the feature points in the matched environmental image according to the positions of the corresponding feature points in the matched environmental image; The robot converts the pixel coordinates of the feature points in the matched environmental image into robot coordinates through a coordinate transformation formula; The robot respectively uses the feature points in the environmental image and the robot coordinates of the feature points in the matched environmental image as the coordinate origin, constructs a feature point coordinate system, and determines the feature coordinates of the robot in this coordinate system; The robot determines its position in the current walking map according to the coordinate changes of the robot in the feature point coordinate system.

7. A robot positioning method according to claim 6, characterized in that After the robot determines its position in the current walking map, the robot performs object recognition on the acquired environmental image, including the following steps: The robot performs grid processing on the acquired environmental image, dividing the environmental image into N*N grids; The robot detects the grids of the environmental image to obtain 2N prediction boxes and the confidence of each prediction box; The robot performs target classification on the 2N prediction boxes according to the confidence of the prediction boxes; The robot performs non-maximum suppression calculation on the classified prediction boxes to obtain the recognition result; Wherein, N is a natural number greater than 1.

8. A robot positioning method according to claim 7, characterized in that, The robot marks the recognized object in the walking map, including the following steps: After the robot performs object recognition on the acquired environmental image, according to the settings of the recognition model, it names the recognized object and marks the position and name of the object in the walking map.

9. A chip with a built-in control program, characterized in that, This program is configured to execute the robot positioning method described in any one of claims 1 to 8.

10. A robot, characterized in that, The robot includes a main control chip, an RGB camera and a fill light. The main control chip is the chip described in claim 9. The RGB camera includes an infrared sensing point and an IR-CUT dual filter.

Images