Indoor Positioning Method, Device, Equipment and Storage Medium for Autonomous Mobile Robot

Abstract

The present invention discloses an indoor positioning method for an autonomous mobile robot. The method includes: when it is determined that the visual assistance positioning condition is satisfied, controlling the autonomous mobile robot to acquire a ceiling acquisition image, and generating a position descriptor according to the visual features detected in the ceiling acquisition image; searching for a target visual feature road sign that matches the position descriptor in a pre-established visual feature map; and positioning the autonomous mobile robot according to the target image feature pose in the target visual feature road sign and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image. Through the technical solution of the present invention, it is possible to achieve global positioning of the autonomous mobile robot relying on a pre-established indoor ceiling map, improve the success rate and accuracy of indoor positioning of the autonomous mobile robot, and the technical solution of the present invention does not require pre-deploying positioning tags on the ceiling, reducing the positioning cost.

Description

Technical Field

[0001] The present invention relates to the field of aircraft assembly and manufacturing, and particularly to an indoor positioning method, device, equipment and storage medium for an autonomous mobile robot. Background Art

[0002] An autonomous mobile robot is an integrated system that combines functions such as environmental perception, dynamic decision-making and planning, behavior control and execution. Compared with the previous generations of mobile robots that relied on magnetic strips or QR codes for positioning and navigation, the latest generation of autonomous mobile robots do not need to rely on magnetic strips or QR codes for positioning and navigation, and have the capabilities of environmental perception, autonomous decision-making and control. They can dynamically plan paths and avoid obstacles autonomously according to the on-site situation, and are currently the cutting-edge mobile robots in technology.

[0003] In the prior art, the autonomous mobile robots applied indoors generally adopt the following two methods: 1) The method of using a single-line lidar for mapping and navigation positioning. This navigation and positioning method is based on a two-dimensional map generated by the single-line lidar slam algorithm. This two-dimensional map describes the obstacle information in the plane where the single-line lidar scanning plane is located. During positioning, the obstacle contour in the plane scanned by the lidar is compared with the map, and the position of the robot is obtained through algorithms such as adaptive Monte Carlo localization. 2) The method of presetting tags on the ceiling. This method is a method of presetting positioning tags on the ceiling and realizing positioning by collecting the images of the positioning tags through the robot camera.

[0004] In the process of the inventors implementing the present invention, it is found that the prior art has the following defects: For method 1), due to the needs of production and life indoors, the positions of the obstacles used as positioning road signs near the movement path of the autonomous mobile robot often change, which does not match the positioning road sign situation when generating the map during scanning, resulting in the failure of the robot's positioning based on the obstacle map and the inability to realize the indoor positioning of the autonomous mobile robot. For method 2), it is necessary to preset positioning tags with special patterns on the ceiling, which requires relatively high installation and maintenance costs. Summary of the Invention

[0005] The present invention provides an indoor positioning method, device, equipment and storage medium for an autonomous mobile robot to solve the problems of the failure of indoor positioning of the existing autonomous mobile robot due to the change of positioning road signs and the relatively high installation and maintenance costs of the positioning system.

[0006] According to one aspect of the present invention, an indoor positioning method for an autonomous mobile robot is provided. The method includes:

[0007] When it is determined that the visual assistance positioning condition is satisfied, control the autonomous mobile robot to acquire a ceiling acquisition image, and generate a position descriptor according to the visual features detected in the ceiling acquisition image;

[0008] In the pre-established visual feature map, search for the target visual feature road signs that match the position descriptor;

[0009] Among them, the visual feature road signs include an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor;

[0010] According to the target image feature pose in the target visual feature road sign and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image, position the autonomous mobile robot.

[0011] According to another aspect of the present invention, there is provided an indoor positioning device for an autonomous mobile robot, and the device includes:

[0012] A descriptor generation module, configured to control the autonomous mobile robot to acquire a ceiling acquisition image when it is determined that the visual assistance positioning condition is satisfied, and generate a position descriptor according to the visual features detected in the ceiling acquisition image;

[0013] A road sign search module, configured to search for the target visual feature road signs that match the position descriptor in the pre-established visual feature map;

[0014] Among them, the visual feature road signs include an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor;

[0015] A positioning module, configured to position the autonomous mobile robot according to the target image feature pose in the target visual feature road sign and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image.

[0016] According to another aspect of the present invention, there is provided an electronic device, and the electronic device includes:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the indoor positioning method of the autonomous mobile robot according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the indoor positioning method of the autonomous mobile robot according to any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention obtains a ceiling acquisition image when visual assistance positioning conditions are met, detects visual features in the image to obtain a position descriptor, then searches for a target visual feature landmark matching the position descriptor in a visual feature map, and finally realizes the positioning of the autonomous mobile robot according to the obtained target visual feature landmark and the current shooting pose information when the autonomous mobile robot obtains the ceiling acquisition image. This solves the problems of the failure of indoor positioning of the autonomous mobile robot caused by the change of positioning landmarks during the indoor positioning process of the autonomous mobile robot, and the high installation and maintenance costs of the positioning system, realizes the global positioning of the autonomous mobile robot relying on a pre-established indoor ceiling map, improves the success rate and accuracy of indoor positioning of the autonomous mobile robot, and reduces the positioning cost.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0024] Figure 1 is a flowchart of an indoor positioning method for an autonomous mobile robot according to Embodiment 1 of the present invention;

[0025] Figure 2 is a flowchart of an indoor positioning method for an autonomous mobile robot according to Embodiment 2 of the present invention;

[0026] Figure 3 is a schematic structural diagram of an indoor positioning device for an autonomous mobile robot according to Embodiment 3 of the present invention;

[0027] Figure 4 It is a schematic structural diagram of an electronic device for implementing the indoor positioning method of the autonomous mobile robot according to the embodiments of the present invention. Detailed implementation manners

[0028] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 For Embodiment 1 of the present invention, a flowchart of an indoor positioning method for an autonomous mobile robot is provided. This embodiment is applicable to the situation where an autonomous mobile robot needs to perform positioning during operation in an indoor environment. This method can be executed by the indoor positioning device of the autonomous mobile robot. The indoor positioning device of the autonomous mobile robot can be implemented in the form of hardware and / or software, and the indoor positioning device of the autonomous mobile robot can be configured in an autonomous mobile robot with positioning functions. As Figure 1 shown, the method includes:

[0032] S110. When it is determined that the visual assistance positioning condition is met, control the autonomous mobile robot to acquire a ceiling acquisition image, and generate a position descriptor according to the visual features detected in the ceiling acquisition image.

[0033] Among them, the autonomous mobile robot can be a robot with binocular cameras installed on the top, and a single-line lidar and an inertial measurement unit (IMU) required for the single-line lidar slam algorithm retained, and has autonomous movement performance.

[0034] In this embodiment, the visual feature can be the pipeline on the indoor ceiling or the architectural feature of the steel structure with image corners and corner vertices; it is easy to understand that the pipeline on the indoor ceiling or the architectural feature of the steel structure with image corners and corner vertices remains unchanged for a long time. Therefore, the visual map established by the method of the present application based on the long-term unchanged visual features has the characteristic of long-term stability, and solves the problem of the failure of indoor positioning of the autonomous mobile robot caused by the change of positioning road signs in the indoor positioning process of the existing technology.

[0035] Optionally, control the autonomous mobile robot to capture the ceiling acquisition image through the camera installed on the top.

[0036] Among them, the camera can be a binocular camera; further, the binocular camera can perform absolute measurement of the distance of the target object, and give necessary warnings or braking according to the change of distance information for any type of obstacle.

[0037] Specifically, determining that the visual assistance positioning condition is satisfied includes:

[0038] When the lidar device of the autonomous mobile robot is in a positioning failure state, or when it is determined that the current indoor scene is an indoor scene that requires visual assistance positioning, it is determined that the visual assistance positioning condition is satisfied; and / or

[0039] Detect visual features in the ceiling acquisition image through at least one of the feature point detection algorithm, the speeded-up robust features algorithm, and the feature descriptor extraction algorithm.

[0040] Among them, the feature point detection algorithm can be: generating an image scale space, detecting local extreme points in the scale space, and then accurately positioning the local extreme points by removing low-contrast points and edge response points; further, the speeded-up robust features algorithm can be: processing the image through Gaussian filters with continuous different scales, and detecting scale-invariant feature points in the image through the difference of Gaussians for accurate positioning; further, the feature descriptor extraction algorithm can be: accurately positioning feature points through the Hamming distance and the exclusive OR operation between bits.

[0041] Optionally, according to the detection position of the visual feature in the ceiling acquisition image, extract a plurality of pixel points around the visual feature in the ceiling acquisition image;

[0042] Generate a position descriptor based on the visual feature and the multiple pixel points.

[0043] In this embodiment, after detecting the visual feature in the image collected on the ceiling, the visual feature and multiple pixel points around it are extracted, and based on the corresponding algorithm, the visual feature and multiple pixel points around it are calculated and transformed into a quantitative metric, and the quantitative metric is used as the position descriptor of the visual feature; further, the corresponding algorithm may be: corresponding to the algorithm used in the step of detecting the visual feature in the ceiling image, that is, if the feature point detection algorithm is used to detect the visual feature of the ceiling image, then the feature point detection algorithm is correspondingly used to calculate the position descriptor.

[0044] S120. Search for a target visual feature landmark that matches the position descriptor in the pre-established visual feature map.

[0045] Wherein, the visual feature map includes: the ceiling image and the visual feature landmark.

[0046] Further, the visual feature landmark includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor.

[0047] In this embodiment, the autonomous mobile robot captures a ceiling image through a camera, detects a visual feature from the ceiling image, after detecting the visual feature, extracts it together with the surrounding pixels, calculates a position descriptor, and then matches the position descriptor with the image feature descriptor in the visual feature map. If the match is successful, the visual feature landmark corresponding to the image feature descriptor is used as the visual feature landmark of the ceiling image, and the image feature pose in the visual feature landmark is extracted as the image feature pose of the current autonomous mobile robot.

[0048] Wherein, the image feature pose includes: the pose information of the autonomous mobile robot when generating the image feature descriptor, the shooting angle information, and the pixel information of the camera, etc.

[0049] S130. Locate the autonomous mobile robot according to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot obtains the ceiling image.

[0050] Further, obtain the target standard position information and target standard shooting pose information included in the target image feature pose;

[0051] Calculate the current position information corresponding to the autonomous mobile robot according to the target standard position information, the target standard shooting pose information, the current shooting pose information, and the shooting parameters of the camera in the autonomous mobile robot.

[0052] In this embodiment, when it is determined that the visual assistance positioning condition is satisfied, the autonomous mobile robot captures a ceiling image, then detects visual features in the image. After the visual features are detected, they and their surrounding pixels are extracted from the image, and a descriptor is calculated. Then, according to the descriptor of the visual feature landmark in the visual feature map, a search and match are performed to obtain the visual feature landmark that matches the image feature. Since the visual feature landmark is unique, it can be considered that the feature detected in the image is the projection of the visual feature landmark in the map in this image. Further, in addition to the visual feature descriptor of the landmark, the visual feature landmark in the visual feature map also includes the three-dimensional spatial coordinates of the landmark in space. Subsequently, according to the two-dimensional pixel coordinate positions of the visual feature in the left and right images of the binocular camera, the binocular disparity ranging algorithm is used to calculate the three-dimensional spatial coordinates of the object feature corresponding to the visual feature in the camera three-dimensional coordinate system. Further, for a monocular camera, the pose transformation of the two camera coordinate systems of the binocular camera can be regarded as the two frames of images before and after, and the method similar to that of the binocular can be used to obtain it. Finally, according to the three-dimensional spatial coordinates of multiple feature landmarks in the world coordinate system and the coordinates in the camera three-dimensional coordinate system, through coordinate transformation, the pose of the camera coordinate system in the world coordinate system is obtained. According to the pose change relationship between the camera coordinate system and the robot coordinate system, the pose of the robot coordinate system in the world coordinate system is obtained to achieve the final positioning.

[0053] The technical solution of the embodiment of the present invention obtains a ceiling acquisition image when the visual assistance positioning condition is satisfied, detects the visual features in the image to obtain a position descriptor, then searches for the target visual feature landmark that matches the position descriptor in the visual feature map, and finally realizes the positioning of the autonomous mobile robot according to the obtained target visual feature landmark and the current shooting pose information when the autonomous mobile robot obtains the ceiling acquisition image, solves the problems of the failure of the indoor positioning of the autonomous mobile robot caused by the change of the positioning landmark and the high installation and maintenance costs of the positioning system during the indoor positioning process of the autonomous mobile robot, realizes the global positioning of the autonomous mobile robot relying on the pre-established indoor ceiling map, improves the success rate and accuracy of the indoor positioning of the autonomous mobile robot, and reduces the positioning cost.

[0054] Embodiment 2

[0055] Figure 2The following is a flowchart of an indoor positioning method for an autonomous mobile robot provided in the second embodiment of the present invention. This embodiment is a supplement to the above embodiment. Specifically, before searching for target visual feature road signs that match the position descriptor in the pre-established visual feature map, it further includes: controlling the autonomous mobile robot to move indoors, and during the movement, synchronously collecting lidar data, standard ceiling images, and standard shooting pose information corresponding to each standard ceiling image respectively; determining standard position information corresponding to multiple lidar key frames according to the lidar data; screening to obtain standard ceiling images corresponding to each standard position information respectively according to the time stamps of each lidar key frame and the shooting time points of each standard ceiling image; constructing multiple visual feature road signs according to the standard ceiling images corresponding to each standard position information respectively and the standard shooting pose information corresponding to each standard ceiling image; and adding the multiple visual feature road signs to the visual feature map respectively.

[0056] Correspondingly, as Figure 2 shown, the method includes:

[0057] S210. Control the autonomous mobile robot to move indoors, and during the movement, synchronously collect lidar data, standard ceiling images, and standard shooting pose information corresponding to each standard ceiling image respectively.

[0058] Among them, the lidar data may be lidar key frame data, lidar key frame acquisition time stamps, and relevant data required for the algorithm applicable to generating the position descriptor in S260.

[0059] Among them, the size and pixels of the standard ceiling image can be determined by the shooting parameters of the camera and set manually.

[0060] S220. Determine standard position information corresponding to multiple lidar key frames according to the lidar data.

[0061] In this embodiment, based on the lidar slam algorithm, the multiple lidar key frames are calculated on the basis of S210 to obtain the current pose of the autonomous mobile robot at the time stamp of the lidar key frame acquisition, and the time stamp of the current lidar key frame is recorded, which together with the current pose constitutes the standard position information corresponding to the current lidar key.

[0062] S230. Screen to obtain standard ceiling images corresponding to each standard position information respectively according to the time stamps of each lidar key frame and the shooting time points of each standard ceiling image.

[0063] In this embodiment, exemplarily, based on S220, a set of ceiling images that meet the preset requirements for the time interval from the time stamps of each laser key frame are matched in each ceiling image; wherein, the time interval can be set manually and adjusted. Exemplarily, the time interval can be 0.1 second. Further, the laser key frames corresponding to the time stamps of the laser key frames are matched, and the standard position information corresponding to each laser key frame is obtained. Further, based on the time stamps of the laser key frames, the standard position information is matched with the pre-standard ceiling images to obtain the standard ceiling images corresponding to each standard position information respectively.

[0064] S240. According to the standard ceiling images corresponding to each standard position information respectively, and the standard shooting pose information corresponding to each standard ceiling image, a plurality of visual feature road signs are constructed.

[0065] Optionally, based on the standard position information and the standard shooting pose information corresponding to the standard ceiling image, a mapping between the plurality of standard position information and the standard shooting pose information is constructed as a plurality of visual feature road signs.

[0066] In this embodiment, specifically, based on S230, the standard ceiling images that have been matched with the standard position information are subjected to visual feature detection. Among them, the visual feature detection can adopt at least one of a feature point detection algorithm, a speeded-up robust features (SURF) algorithm, and a feature descriptor extraction algorithm. Further, after the visual features are detected, they are jointly extracted with the surrounding pixels to calculate an image feature descriptor, and then the standard position information of the autonomous mobile robot is matched with the visual image feature descriptor. If the match is successful, the mapping of the corresponding standard position information and the standard shooting pose information is used as the visual feature road sign of the ceiling image.

[0067] S250. The plurality of visual feature road signs are respectively added to the visual feature map.

[0068] Among them, the visual feature map includes multiple groups of visual feature road signs and the ceiling map of the current indoor environment.

[0069] S260. When it is determined that the visual assistance positioning condition is met, the autonomous mobile robot is controlled to acquire a ceiling acquisition image, and a position descriptor is generated according to the visual features detected in the ceiling acquisition image.

[0070] S270. In the pre-established visual feature map, a target visual feature road sign that matches the position descriptor is searched for.

[0071] Among them, the visual feature landmark includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and the standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor.

[0072] S280. Position the autonomous mobile robot according to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image.

[0073] The technical solution of the embodiment of the present invention acquires a ceiling acquisition image when the visual assistance positioning condition is met, detects the visual features in the image to obtain a position descriptor, constructs a plurality of visual feature landmarks through the indoor movement of the autonomous mobile robot and the operation of collecting data, and adds the visual feature landmarks to the visual feature map respectively to complete the construction of the visual feature map. Then, search for the target visual feature landmark matching the position descriptor in the visual feature map. Finally, according to the obtained target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image, the global positioning of the autonomous mobile robot is realized relying on the pre-established indoor ceiling map, the success rate and accuracy of the indoor positioning of the autonomous mobile robot are improved, and the positioning cost is reduced.

[0074] Embodiment III

[0075] Figure 3 It is a schematic structural diagram of an indoor positioning device for an autonomous mobile robot provided in Embodiment III of the present invention. As Figure 3 shown, the device includes:

[0076] A descriptor generation module 310, configured to control the autonomous mobile robot to acquire a ceiling acquisition image when it is determined that the visual assistance positioning condition is met, and generate a position descriptor according to the visual features detected in the ceiling acquisition image.

[0077] A landmark search module 320, configured to search for a target visual feature landmark matching the position descriptor in a pre-established visual feature map.

[0078] Among them, the visual feature landmark includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and the standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor;

[0079] A positioning module 330, configured to position the autonomous mobile robot according to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image.

[0080] In the technical solution of the embodiment of the present invention, when the visual assistance positioning condition is satisfied, a ceiling acquisition image is obtained, visual features in the image are detected to obtain a position descriptor, then a target visual feature road sign matching the position descriptor is searched in the visual feature map, and finally, based on the obtained target visual feature road sign and the current shooting pose information of the autonomous mobile robot when acquiring the ceiling acquisition image, the positioning of the autonomous mobile robot is realized. Relying on the pre-established indoor ceiling map for global positioning of the autonomous mobile robot improves the success rate and accuracy of indoor positioning of the autonomous mobile robot and reduces the positioning cost.

[0081] Based on the above embodiments, the descriptor generation module 310 may include:

[0082] An image acquisition unit, configured to control the autonomous mobile robot to capture a ceiling acquisition image through a camera disposed at the top.

[0083] Based on the above embodiments, the descriptor generation module 310 may further include:

[0084] A feature extraction unit, configured to extract a plurality of pixel points around the visual feature in the ceiling acquisition image according to the detection position of the visual feature in the ceiling acquisition image;

[0085] A descriptor acquisition unit, configured to generate a position descriptor according to the visual feature and the plurality of pixel points.

[0086] Based on the above embodiments, the road sign search module 320 may include:

[0087] An indoor information acquisition unit, configured to control the autonomous mobile robot to move indoors, and during the movement, synchronously acquire lidar data, standard ceiling images, and standard shooting pose information corresponding to each standard ceiling image respectively;

[0088] A standard position information determination unit, configured to determine standard position information corresponding to a plurality of lidar key frames according to the lidar data;

[0089] An image screening unit, configured to screen out standard ceiling images corresponding to each standard position information respectively according to the time stamp of each lidar key frame and the shooting time point of each standard ceiling image;

[0090] A visual feature road sign construction unit, configured to construct a plurality of visual feature road signs according to the standard ceiling images corresponding to each standard position information respectively and the standard shooting pose information corresponding to each standard ceiling image;

[0091] A visual feature map building unit, configured to add the multiple visual feature road signs to the visual feature map respectively.

[0092] Based on the above embodiments, the image screening unit may include:

[0093] A mapping construction unit, configured to construct a mapping between multiple standard position information and standard shooting pose information based on the standard position information and standard shooting pose information corresponding to the standard ceiling image, as multiple visual feature road signs.

[0094] Based on the above embodiments, the positioning module 330 may include:

[0095] A pose information acquisition unit, configured to acquire the target standard position information and target standard shooting pose information included in the target image feature pose;

[0096] A position information calculation unit, configured to calculate the current position information corresponding to the autonomous mobile robot according to the target standard position information, the target standard shooting pose information, the current shooting pose information, and the shooting parameters of the camera in the autonomous mobile robot.

[0097] Based on the above embodiments, the descriptor generation module 310 may further include:

[0098] A visual feature detection unit, configured to determine that the visual assistance positioning condition is satisfied when the lidar device of the autonomous mobile robot is in a positioning failure state, or when it is determined that the current indoor scene is an indoor scene that requires visual assistance positioning; and / or

[0099] Detect visual features in the ceiling acquisition image through at least one of a feature point detection algorithm, a speeded-up robust features algorithm, and a feature descriptor extraction algorithm.

[0100] The indoor positioning device of the autonomous mobile robot provided by the embodiments of the present invention can execute the indoor positioning method of the autonomous mobile robot provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0101] Embodiment 4

[0102] Figure 4The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0103] As Figure 4 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0104] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the indoor positioning method of an autonomous mobile robot.

[0106] Specifically, the method includes:

[0107] When it is determined that the visual - assisted positioning condition is met, control the autonomous mobile robot to acquire a ceiling acquisition image, and generate a position descriptor based on the visual features detected in the ceiling acquisition image;

[0108] In a pre - established visual feature map, search for a target visual feature landmark that matches the position descriptor;

[0109] Among them, the visual feature landmark includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and standard shooting pose information of the autonomous mobile robot when the image feature descriptor is generated;

[0110] According to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image, position the autonomous mobile robot.

[0111] In some embodiments, the indoor positioning method of the autonomous mobile robot can be implemented as a computer program, which is tangibly contained in a computer - readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via ROM12 and / or communication unit 19. When the computer program is loaded into RAM13 and executed by the processor 11, one or more steps of the indoor positioning method of the autonomous mobile robot described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the indoor positioning method of the autonomous mobile robot by any other suitable means (e.g., by means of firmware).

[0112] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field - programmable gate arrays (FPGA), application - specific integrated circuits (ASIC), application - specific standard products (ASSP), system - on - chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor. The programmable processor can be a dedicated or general - purpose programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] A computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0115] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0116] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0117] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact via a communication network. The client-server relationship is created by computer programs that run on respective computers and have a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0118] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0119] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An indoor positioning method for an autonomous mobile robot, characterized in that, Including: When it is determined that the visual assistance positioning condition is satisfied, controlling the autonomous mobile robot to acquire a ceiling acquisition image, and generating a position descriptor according to the visual features detected in the ceiling acquisition image; Searching for a target visual feature landmark that matches the position descriptor in a pre-established visual feature map; Wherein, the visual feature landmark includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and the standard shooting pose information of the autonomous mobile robot when the image feature descriptor is generated; Positioning the autonomous mobile robot according to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image.

2. The method according to claim 1, wherein Controlling the autonomous mobile robot to acquire a ceiling acquisition image includes: Controlling the autonomous mobile robot to capture a ceiling acquisition image through a camera arranged at the top.

3. The method according to claim 1, characterized in that, Generating a position descriptor according to the visual features detected in the ceiling acquisition image includes: Extracting a plurality of pixel points around the visual feature in the ceiling acquisition image according to the detection position of the visual feature in the ceiling acquisition image; Generating a position descriptor according to the visual feature and the plurality of pixel points.

4. The method according to any one of claims 1 to 3, characterized in that, Before searching for a target visual feature landmark that matches the position descriptor in a pre-established visual feature map, it further includes: Controlling the autonomous mobile robot to move indoors, and during the movement, synchronously collecting lidar data, standard ceiling images, and standard shooting pose information corresponding to each standard ceiling image respectively; Determining the standard position information corresponding to a plurality of lidar key frames according to the lidar data; Filtering to obtain a standard ceiling image corresponding to each standard position information according to the time stamps of each lidar key frame and the shooting time points of each standard ceiling image; Constructing a plurality of visual feature landmarks according to the standard ceiling image corresponding to each standard position information and the standard shooting pose information corresponding to each standard ceiling image; Adding the plurality of visual feature landmarks to the visual feature map respectively.

5. The method according to claim 4, wherein Constructing a plurality of visual feature landmarks according to the standard ceiling image corresponding to each standard position information and the standard shooting pose information corresponding to each standard ceiling image includes: Based on the standard position information and the standard shooting pose information corresponding to the standard ceiling image, constructing a mapping between the plurality of standard position information and the standard shooting pose information as a plurality of visual feature landmarks.

6. The method according to claim 1, wherein Positioning the autonomous mobile robot according to the target image feature pose in the target visual feature landmark and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image includes: Obtaining the target standard position information and the target standard shooting pose information included in the target image feature pose; Calculate the current position information corresponding to the autonomous mobile robot according to the target standard position information, the target standard shooting pose information, the current shooting pose information, and the shooting parameters of the camera in the autonomous mobile robot.

7. The method according to claim 1, wherein Determine that the visual assistance positioning condition is satisfied, including: When the lidar device of the autonomous mobile robot is in a positioning failure state, or when it is determined that the current indoor scene is an indoor scene that requires visual assistance positioning, determine that the visual assistance positioning condition is satisfied; and / or Detect visual features in the ceiling acquisition image through at least one of a feature point detection algorithm, a speeded-up robust features algorithm, and a feature descriptor extraction algorithm.

8. An indoor positioning device for an autonomous mobile robot, characterized in that, Include: A descriptor generation module, configured to control the autonomous mobile robot to acquire a ceiling acquisition image when it is determined that the visual assistance positioning condition is satisfied, and generate a position descriptor according to the visual features detected in the ceiling acquisition image; A road sign search module, configured to search for a target visual feature road sign that matches the position descriptor in a pre-established visual feature map; Wherein, the visual feature road sign includes an image feature descriptor and an image feature pose. The image feature descriptor is used to compare with the position descriptor, and the image feature pose is used to describe the standard position information and the standard shooting pose information of the autonomous mobile robot when generating the image feature descriptor; A positioning module, configured to position the autonomous mobile robot according to the target image feature pose in the target visual feature road sign and the current shooting pose information when the autonomous mobile robot acquires the ceiling acquisition image.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the indoor positioning method of the autonomous mobile robot according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to implement the indoor positioning method of the autonomous mobile robot according to any one of claims 1-7 when executed.

Images