Robot self-localization methods, devices and computer-readable media

Abstract

This application provides a robot self-localization method, device, and computer-readable medium. This scheme can acquire sensor observation information of the current frame, and based on the sensor observation information of the current frame and map information, obtain the robot's first pose information. Simultaneously, it can acquire sensor observation information from multiple frames containing the current frame, stitch these multiple frames together to generate a current sub-image of the sensor observation information, and based on the current sub-image and map information, obtain the robot's second pose information. Then, it performs pose fusion based on the first pose information and the second pose information to generate the final pose information. This enables more accurate localization in environments with similar characteristics or frequent dynamic changes, improving the robustness of localization.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot self-localization method, device and computer-readable medium. Background Technology

[0002] Robot self-localization has always been one of the most concerning and challenging problems in the robotics industry and academia. A common solution for LiDAR-based robot self-localization methods is to obtain the current pose information based on the current single-frame sensor observations, using the Adaptive Monte Carlo Localization (AMCL) algorithm and prior pose information from the previous frame. However, this approach has limitations. In environments with similar layouts (such as long corridors) or those with frequent dynamic changes (such as numerous dynamic obstacles), the localization accuracy is low, and robustness is severely challenged. This is a key issue limiting the long-term unattended operation of robots in real-world environments. Summary of the Invention

[0003] One objective of this application is to provide a robot self-localization method, device, and computer-readable medium to address the problem of insufficient localization accuracy in existing technologies when the environment is similar or dynamically changing frequently.

[0004] To achieve the above objectives, this application provides a robot self-localization method, the method comprising:

[0005] Obtain sensor observation information for the current frame;

[0006] Based on the sensor observation information of the current frame and combined with map information, the robot's first pose information is obtained;

[0007] Obtain multi-frame sensor observation information including the current frame;

[0008] The multi-frame sensor observation information is stitched together to generate a current sub-image based on the sensor observation information.

[0009] Based on the current sub-graph and combined with map information, the robot's second pose information is obtained;

[0010] Pose fusion is performed based on the first pose information and the second pose information to generate the final pose information.

[0011] Furthermore, the method also includes:

[0012] Before processing the sensor observation information, the sensor observation information is filtered to remove noise information.

[0013] Further, based on the sensor observation information of the current frame and combined with map information, the robot's first pose information is obtained, including:

[0014] Based on the sensor observation information of the current frame and combined with map information, a particle filter algorithm is used to obtain the robot's first pose information.

[0015] Furthermore, based on the sensor observation information of the current frame and combined with map information, a particle filter algorithm is used to obtain the robot's first pose information, including:

[0016] Identify the sampling particles on the map;

[0017] Update the position of the sampled particles using robot motion information;

[0018] The particle weight is calculated by using the similarity between the sensor observation information of the current frame and the feature information of the sampled particle's location;

[0019] The particle distribution corresponding to the current frame is obtained based on the particle weights;

[0020] Based on the particle distribution, the sampled particles are clustered, and the mean pose of the largest cluster of particles is calculated.

[0021] The first pose information is determined by matching the mean pose, the sensor observation information of the current frame, and the map information.

[0022] Further, the sampled particles are identified on the map, including:

[0023] Using a random sampling strategy, sampled particles are re-determined on the map based on the particle weights of the previous frame.

[0024] Further, based on the mean pose, the sensor observation information of the current frame, and the map information, the first pose information is determined by matching, including:

[0025] Based on the mean pose, the sensor observation information of the current frame, and the map information, the first pose information is determined by using an iterative nearest-point algorithm for matching.

[0026] Further, acquiring multi-frame sensor observation information including the current frame includes:

[0027] A fixed-size buffer is used to store the current frame and the sensor observation information of the previous multiple frames.

[0028] Furthermore, pose fusion is performed based on the first pose information and the second pose information to generate the final pose information, including:

[0029] Pose fusion is performed based on the first pose information and the second pose information, and the final pose information is generated by using extended Kalman filter, particle filter or particle swarm optimization algorithm.

[0030] Based on another aspect of this application, a robot self-localization device is also provided, the device including a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device performs the robot self-localization method.

[0031] In addition, this application also provides a computer-readable medium having computer-readable instructions stored thereon, which can be executed by a processor to implement the robot self-localization method.

[0032] Compared to existing technologies, the solution provided in this application can acquire sensor observation information of the current frame, and obtain the robot's first pose information based on the sensor observation information of the current frame and map information. Simultaneously, it can acquire multi-frame sensor observation information containing the current frame, stitch these multi-frame sensor observation information together to generate a current sub-graph of sensor observation information, and obtain the robot's second pose information based on the current sub-graph and map information. Then, it performs pose fusion based on the first pose information and the second pose information to generate the final pose information. Since the multi-frame sensor observation information containing the current frame is sensor observation information about historical time windows, stitching these sensor observation information into a sub-graph can accumulate observation information over a certain time dimension. This can be used to filter out potential dynamic obstacles and improve positioning accuracy in similar environments. Therefore, by achieving self-localization based on the stitched sub-graph and then combining it with the single-frame self-localization result to obtain the robot's final pose information, positioning becomes more accurate and robust when the environment is similar or dynamic changes are frequent. Attached Figure Description

[0033] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0034] Figure 1 A flowchart illustrating a robot self-localization method provided in an embodiment of this application;

[0035] Figure 2 This is a schematic diagram illustrating the specific processing steps of self-localization using the particle filter algorithm in the embodiments of this application;

[0036] Figure 3 This is a schematic diagram illustrating the processing principle of robot self-localization using the solution of this application embodiment;

[0037] The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0040] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0041] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0042] This application provides a robot self-localization method. This method acquires sensor observation information from the current frame and, based on this information and map information, obtains the robot's first pose information. Simultaneously, it acquires multi-frame sensor observation information containing the current frame, stitches these multi-frame observations together to generate a current sub-graph of sensor observation information, and, based on this sub-graph and map information, obtains the robot's second pose information. Finally, it fuses the first and second pose information to generate the final pose information. Since the multi-frame sensor observation information containing the current frame is related to a historical time window, stitching these observations together to form a sub-graph accumulates observation information over a certain time dimension. This information can be used to filter out potential dynamic obstacles and improve localization accuracy in similar environments. Therefore, by achieving self-localization based on the stitched sub-graph and then combining it with the single-frame self-localization result to obtain the robot's final pose information, localization becomes more accurate and robust when environments are similar or dynamic changes are frequent.

[0043] Figure 1 The present application illustrates a robot self-localization method according to an embodiment, which may include at least the following processing steps:

[0044] Step S101: Obtain sensor observation information for the current frame.

[0045] The sensor observation information refers to relevant information that can be used for robot self-localization. It is generally obtained by processing sensor data collected by sensors on the robot. For example, the sensor observation information can be obtained by processing data collected by sensors such as odometers and laser rangefinders installed on the robot. Specifically, it can include displacement angle changes over a preset time period and the measured distance to obstacles at the corresponding angles.

[0046] In some embodiments of this application, to further improve the accuracy of subsequent processing, the sensor observation information can be filtered before processing to remove noise. For example, in practical scenarios, isolated point filtering or window-based mean filtering can be used. Isolated point filtering removes isolated points from the sensor observation information, while window-based mean filtering is based on observation continuity; both can remove noise from the sensor observation information, thereby improving the accuracy of subsequent processing based on the sensor observation information.

[0047] Step S102: Based on the sensor observation information of the current frame and combined with the map information, obtain the robot's first pose information.

[0048] In this process, a particle filter algorithm can be used to achieve robot self-localization. Specifically, based on the sensor observation information of the current frame and combined with map information, a particle filter algorithm is used to obtain the robot's first pose information. In the embodiment of this application, the specific processing procedure for self-localization using the particle filter algorithm can be as follows: Figure 2 As shown, the processing steps include the following:

[0049] First, sampling particles can be determined on the map. Here, the map refers to the robot's current motion environment. When determining sampling particles, if it is the first time to determine sampling particles, a random sampling strategy can be used to randomly generate sampling particles on the map. If it is not the first time to determine sampling particles, resampling can be performed based on the particle weight w(t-1) corresponding to the previous frame to regenerate sampling particles.

[0050] After identifying the sampled particles, the robot's motion information can be used to update the positions of the sampled particles. For example, if the robot moves forward n centimeters, the positions of the sampled particles can be updated based on the robot's motion information of moving forward n centimeters, updating forward n centimeters from the current position of each sampled particle.

[0051] Then, the particle weight can be calculated using the similarity between the sensor observation information of the current frame and the feature information of the sampling particle's location. Here, the feature information of the sampling particle's location refers to the sensor observation information of that location on the map. The similarity between the sensor observation information of the current frame and the feature information of the sampling particle's location represents the degree of similarity between the robot's current location and the sampling particle's location. Therefore, the particle weight can be calculated; the higher the similarity, the higher the particle weight.

[0052] After determining the particle weights, the particle distribution corresponding to the current frame can be obtained based on the particle weights. The sampled particles are then clustered based on the particle distribution, and the mean pose of the particles in the largest cluster is calculated. The largest cluster is the cluster with the highest weighted average among the clustering results, and the particles in the largest cluster are the particles contained within that cluster. Therefore, the mean pose calculated based on the particles in the largest cluster is closest to the robot's true pose.

[0053] Finally, the first pose information can be determined by matching the mean pose, the sensor observation information of the current frame, and the map information. Specifically, in determining the first pose information, an Iterative Closest Point (ICP) algorithm can be used, that is, matching the mean pose, the sensor observation information of the current frame, and the map information using the Iterative Closest Point algorithm to determine the first pose information. Thus, the robot's current pose information is determined based on the sensor observation information of a single frame.

[0054] Step S103: Obtain multi-frame sensor observation information including the current frame. Here, multi-frame sensor observation information including the current frame refers to sensor observation information of the current frame and several frames preceding it, i.e., multi-frame sensor observation information within the historical time window.

[0055] In some embodiments of this application, a fixed-size cache space can be used to store the current frame and the sensor observation information of the previous multiple frames. This can limit memory consumption, avoid excessive memory consumption affecting other processing, and avoid accuracy problems caused by the continuous accumulation of errors.

[0056] Step S104: The multi-frame sensor observation information is stitched together to generate a current submap based on the sensor observation information. In a real-world scenario, the generation of the current submap can be triggered by a positioning request, thereby completing the stitching of multi-frame data.

[0057] Step S105: Based on the current sub-image and combined with map information, obtain the robot's second pose information. Since the stitched current sub-image contains multi-frame sensor observation information from historical time windows, obtaining the robot's second pose information based on the current sub-image can be used to filter out potential dynamic obstacles and improve positioning accuracy in similar environments.

[0058] For example, if a single-frame laser observation can only detect a distance of up to 20m, while a submap obtained by stitching together multiple frames of sensor observation information can contain data up to 100m, when a long corridor with high environmental similarity is 50m long, the similarity of the first pose information obtained from multiple single-frame sensor observation information will be high. If localization is achieved based solely on the first pose information, the self-localization accuracy will be low. However, since the stitched submap can cover 100m, it can include the entire long corridor and sensor observation information outside the long corridor. Even if there are some similar frames, it will not lead to a decrease in self-localization accuracy.

[0059] In situations with numerous dynamic obstacles, obtaining the first pose information based solely on a single frame of sensor observations will result in significant discrepancies between the sensor observations and the actual map information at the robot's location, making it difficult to obtain accurate localization results. However, since the submap is composed of multiple frames of sensor observations, dynamic obstacle information can be removed based on consecutive frames, thereby improving the situation where accurate localization is not possible.

[0060] It should be understood by those skilled in the art that the numbers between the above steps are only used to distinguish different processing steps, and their numerical order is unrelated to the actual execution order and processing logic of the steps, and is not used to limit the execution order between steps. For example, steps S101 to S102 in the embodiments of this application can be executed synchronously with S103 to S105, and after obtaining their respective processing results, the subsequent step S106 is executed.

[0061] Step S106: Perform pose fusion based on the first pose information and the second pose information to generate the final pose information. During fusion, an Extended Kalman Filter (EKF), Particle Filter (PF), or Particle Swarm Optimization (PSO) algorithm can be used. That is, pose fusion is performed based on the first pose information and the second pose information, and the final pose information is generated using an EKF, Particle Filter, or PSO algorithm.

[0062] Since the multi-frame sensor observation information containing the current frame is sensor observation information about historical time windows, by stitching these sensor observation information into a sub-graph, observation information in a certain time dimension can be accumulated. This can be used to filter out potential dynamic obstacles and improve the positioning accuracy in similar environments. Therefore, by achieving self-localization based on the stitched sub-graph and combining it with the single-frame self-localization result to obtain the robot's final pose information, the positioning can be more accurate and robust when the environment is similar or the dynamic changes are frequent.

[0063] Figure 3 The present application illustrates the processing principle for robot self-localization using the scheme of this embodiment. The following core modules can be used to implement the relevant functions during the processing:

[0064] 1) Data Acquisition Module: Primarily used to acquire sensor observation information such as laser data and odometer data, providing data support for the operation of other modules. It mainly consists of an odometer module and a radar acquisition and filtering module. The odometer module acquires odometer data, while the radar acquisition and filtering module acquires laser data from the lidar and uses correlation filtering algorithms to remove unnecessary noise.

[0065] 2) Autonomous Localization Module: Based on sensor observations of the current frame and map information, the module uses a matching algorithm to obtain the first pose information, enabling the robot to know its location in the environment in real time. The autonomous localization uses a particle filtering algorithm: Specifically, for each particle, a random sampling strategy is first used to regenerate sampled particles based on the particle weights of the previous frame. The position of the sampled particles is updated using robot motion information obtained from the sensors. Then, the particle weights are calculated based on the similarity between the sensor observations and the feature information at the sampled particle's location to obtain the current particle distribution. The sampled particles are then clustered to obtain the mean pose of the largest cluster. Finally, ICP matching is performed using the mean pose, the sensor observations of the current frame, and the map used for localization to obtain the matching degree and the first pose information after precise matching.

[0066] 3) Submap Localization Module: This module uses a fixed-size cache to store multiple frames of sensor observation information according to certain constraints. A localization request triggers the generation of the current submap, which involves stitching together multiple frames of sensor observation information and removing potential dynamic obstacles. After obtaining the submap, it combines it with map information and uses a self-localization algorithm to obtain the second pose information.

[0067] 4) Positioning fusion module: The first pose information and the second pose information are fused using algorithms such as EKF, PF or PSO to obtain the final pose information.

[0068] Furthermore, this application also provides a robot self-localization device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the aforementioned robot self-localization method.

[0069] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a processing unit, it performs the functions defined in the methods of this application.

[0070] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0071] In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0072] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0073] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0074] In another aspect, embodiments of this application also provide a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The aforementioned computer-readable medium carries one or more computer-readable instructions, which may be executed by a processor to implement the steps of the methods and / or technical solutions of the various embodiments of this application.

[0075] Furthermore, this application also provides a computer program stored in a computer device, which enables the computer device to execute the robot motion behavior control method.

[0076] It should be noted that this application can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0077] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. A robot self-localization method, characterized in that, The method includes: Obtain sensor observation information for the current frame; Based on the sensor observation information of the current frame and combined with map information, the robot's first pose information is obtained; Obtain multi-frame sensor observation information containing the current frame, wherein the multi-frame sensor observation information containing the current frame refers to the sensor observation information of the current frame and several frames preceding the current frame. The multi-frame sensor observation information is stitched together to generate a current sub-image based on the sensor observation information. Based on the current sub-graph and combined with map information, the robot's second pose information is obtained; Pose fusion is performed based on the first pose information and the second pose information to generate the final pose information.

2. The method according to claim 1, characterized in that, The method further includes: Before processing the sensor observation information, the sensor observation information is filtered to remove noise information.

3. The method according to claim 1, characterized in that, Based on the sensor observation information of the current frame and combined with map information, the robot's first pose information is obtained, including: Based on the sensor observation information of the current frame and combined with map information, a particle filter algorithm is used to obtain the robot's first pose information.

4. The method according to claim 3, characterized in that, Based on the sensor observation information of the current frame and combined with map information, a particle filter algorithm is used to obtain the robot's first pose information, including: Identify the sampling particles on the map; Update the position of the sampled particles using robot motion information; The particle weight is calculated by using the similarity between the sensor observation information of the current frame and the feature information of the sampled particle's location; The particle distribution corresponding to the current frame is obtained based on the particle weights; Based on the particle distribution, the sampled particles are clustered, and the mean pose of the largest cluster of particles is calculated. The first pose information is determined by matching the mean pose, the sensor observation information of the current frame, and the map information.

5. The method according to claim 4, characterized in that, Identify the sampled particles on the map, including: Using a random sampling strategy, sampled particles are re-determined on the map based on the particle weights of the previous frame.

6. The method according to claim 4, characterized in that, Based on the mean pose, the sensor observation information of the current frame, and the map information, the first pose information is determined by matching, including: Based on the mean pose, the sensor observation information of the current frame, and the map information, the first pose information is determined by using an iterative nearest-point algorithm for matching.

7. The method according to claim 1, characterized in that, Obtaining multi-frame sensor observation information containing the current frame includes: A fixed-size buffer is used to store the current frame and the sensor observation information of the previous multiple frames.

8. The method according to claim 1, characterized in that, Pose fusion is performed based on the first pose information and the second pose information to generate the final pose information, including: Pose fusion is performed based on the first pose information and the second pose information, and the final pose information is generated by using extended Kalman filter, particle filter or particle swarm optimization algorithm.

9. A robot self-positioning device, characterized in that, The device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device performs the method of any one of claims 1 to 8.

10. A computer-readable medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 8.

Images