Robot repositioning method and robot
By acquiring robot motion measurement data and terrain height data to generate local terrain descriptors, and using a terrain descriptor index library for indexing, the problem of lawnmower robots failing to relocate in complex environments was solved, achieving higher positioning accuracy and stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANKER INNOVATIONS TECH CO LTD
- Filing Date
- 2026-03-25
- Publication Date
- 2026-06-26
Smart Images
Figure CN122281902A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robot relocation technology, and in particular to a robot relocation method and a robot. Background Technology
[0002] With the development of artificial intelligence technology, robots are being used more and more widely in daily life. For example, lawn mowing robots use vehicle-type mobile platforms to perform lawn mowing tasks, and their autonomous navigation and trajectory tracking depend heavily on positioning accuracy.
[0003] In related technologies, lawnmower robots need to relocate when their current position is uncertain during autonomous operation. This is typically achieved by extracting image feature data of key locations within the lawn area using visual sensors and then relocating based on this data. However, this method may result in relocation failure for the lawnmower robot. Summary of the Invention
[0004] Therefore, it is necessary to provide a robot relocation method and a robot to address the aforementioned technical problems.
[0005] In a first aspect, this application provides a robot relocalization method, the method comprising:
[0006] Acquire the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area;
[0007] Based on the current motion measurement data, terrain image data and terrain height data of the current operating location, a local terrain descriptor for the current operating location is generated;
[0008] Based on the local terrain descriptor of the current operating location, an index is performed in the preset terrain descriptor index library to determine the target area corresponding to the current operating location; the terrain descriptor index library includes the mapping relationship between multiple sets of terrain descriptors and areas in the operating area;
[0009] The robot is relocalized based on the target area and current motion measurement data.
[0010] In one embodiment, a local terrain descriptor for the current operating location is generated based on current motion measurement data, terrain image data of the current operating location, and terrain height data, including:
[0011] Based on the current motion measurement data, terrain image data and terrain height data of the current operating location, a local height sub-map of the current operating location is generated;
[0012] The local terrain descriptor for the current operating location is determined based on the local elevation submap.
[0013] In one embodiment, the current motion measurement data includes acceleration information. Based on the current motion measurement data, terrain image data of the current operating location, and terrain height data, a local height sub-map of the current operating location is generated, including:
[0014] The current motion measurement data, terrain image data of the current operating location, and terrain height data are time-stamp aligned.
[0015] Based on the current motion measurement data, the gravity direction vector is determined, and combined with the terrain image data, the timestamp-aligned terrain height data is mapped to the gravity reference frame to obtain the point cloud data of the current running position;
[0016] Based on the point cloud data, the robot's current running position is used as the center point, and the point cloud data is extracted with a preset distance as the radius to generate a local height sub-map.
[0017] In one embodiment, determining the local terrain descriptor of the current operating location based on the local elevation submap includes:
[0018] The local height submap is projected onto a two-dimensional plane along the direction of gravity, and the two-dimensional plane is divided into multiple grids according to a preset resolution;
[0019] Based on the height data of the point cloud falling within each grid, calculate the mean height and variance of each grid.
[0020] A local terrain descriptor is generated based on the terrain height data of the current operating location, the mean height and the variance of each grid; the local terrain descriptor also includes at least one of the slope histogram features generated based on the height gradient of each grid and the curvature features based on the point neighborhood.
[0021] In one embodiment, the method further includes:
[0022] If the robot relocalizes successfully, the terrain description sub-index is updated using the local height sub-map of the robot's current location, employing either weighted smoothing or exponential smoothing strategies.
[0023] In one embodiment, the construction process of the terrain description sub-index library includes:
[0024] The system acquires optimized poses at multiple locations, terrain image data at multiple locations, and terrain height data at multiple locations within the work area, obtained through simultaneous localization and mapping (SLAM) while the robot is navigating the work area.
[0025] Based on the optimized poses of multiple locations in the work area, the terrain image data of multiple locations in the work area, and the terrain height data of multiple locations in the work area, a global terrain elevation map of the work area is generated.
[0026] The global terrain elevation map is divided into multiple regions, and the terrain descriptor corresponding to each region is calculated;
[0027] A terrain descriptor index library is established based on each region and its corresponding terrain descriptor.
[0028] In one embodiment, the terrain descriptor index library also includes robot candidate pose data corresponding to each region; based on the local terrain descriptor of the current running position, an index is performed in the preset terrain descriptor index library to determine the target region corresponding to the current running position, including:
[0029] Based on the local terrain descriptor of the current operating location, index the terrain descriptor index library to determine at least one candidate region;
[0030] The initial candidate regions are determined by screening based on the similarity score and confidence level of each candidate region; the confidence level of each candidate region is used to characterize the dispersion of point cloud data in the candidate region.
[0031] Based on the inter-regional similarity between any two initial candidate regions, anchoring candidate regions are determined, and candidate pose data and local elevation segments corresponding to each anchoring candidate region are obtained from the terrain description sub-index library.
[0032] The target region is determined based on the candidate pose data corresponding to each anchoring candidate region and the current motion measurement data.
[0033] In one embodiment, determining the target region based on the candidate pose data corresponding to each anchoring candidate region and the current motion measurement data includes:
[0034] The candidate pose data corresponding to each anchor candidate region is combined with the current motion measurement data to calculate the pose and generate the initial pose value corresponding to each anchor candidate region.
[0035] Based on the initial pose values and local elevation sub-maps corresponding to each anchoring candidate region, fine registration is performed on local elevation segments at different resolutions to determine the target region.
[0036] In one embodiment, the robot is relocalized based on the target area and current motion measurement data, including:
[0037] Based on the candidate pose data corresponding to the target area, pose combination calculation is performed with the current motion measurement data to generate the target pose.
[0038] The target pose is used as the pose data for robot relocalization.
[0039] Secondly, this application also provides a robot, including a main body, a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the robot relocation methods described in the first aspect.
[0040] In one embodiment, the robot includes a vision sensor and an inertial measurement sensor.
[0041] The aforementioned robot relocalization method and robot first acquire the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area. Then, based on the current motion measurement data, terrain image data, and terrain height data, a local terrain descriptor for the current operating position is generated. Next, based on the local terrain descriptor, an index is performed in a pre-defined terrain descriptor index library to determine the target area corresponding to the current operating position. The terrain descriptor index library includes multiple sets of mapping relationships between terrain descriptors and areas within the work area. Finally, the robot is relocalized based on the target area and the current motion measurement data. By acquiring the robot's current motion measurement data, terrain image data, and terrain height data to generate a local terrain descriptor for the current operating position, and then relocalizing the robot based on the local terrain descriptor matching, the accuracy of robot relocalization is improved because the terrain information changes less compared to the environmental appearance information. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a schematic diagram of the internal structure of the robot in one embodiment;
[0044] Figure 2 This is a flowchart illustrating a robot relocation method in one embodiment;
[0045] Figure 3 This is a flowchart illustrating the steps involved in constructing a terrain description sub-index library in one embodiment.
[0046] Figure 4 This is a flowchart illustrating the steps of generating a local terrain descriptor in one embodiment;
[0047] Figure 5 This is a flowchart illustrating the steps for generating a local height submap in one embodiment;
[0048] Figure 6 This is a flowchart illustrating the local terrain descriptor generation step in another embodiment;
[0049] Figure 7 This is a flowchart illustrating the target region determination steps in one embodiment;
[0050] Figure 8 This is a flowchart illustrating the target region determination step in another embodiment;
[0051] Figure 9 This is a flowchart illustrating the pose data determination steps for relocation in one embodiment.
[0052] Figure 10 This is a flowchart illustrating the robot relocation method in another embodiment;
[0053] Figure 11 This is a structural block diagram of a robot relocation device in one embodiment. Detailed Implementation
[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0055] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0056] The robot relocation method provided in this application embodiment can be applied to, for example, Figure 1 The illustrated outdoor service robot includes a processor, memory, and network interface connected via a system bus. The processor provides computational and control capabilities. The memory comprises non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to the robot's relocation method. The network interface allows the robot to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a robot relocation method.
[0057] The aforementioned outdoor service robots include, but are not limited to, different types of robots such as lawnmowers, fertilizers, seeders, leaf sweepers, and snow removal robots. These outdoor service robots may also include various types of sensors, such as vision sensors, inertial measurement sensors, ultrasonic sensors, and infrared sensors.
[0058] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0059] In one exemplary embodiment, such as Figure 2 As shown, a robot relocalization method is provided, which is applied to... Figure 1 Taking an outdoor service robot as an example, the explanation includes the following steps 201 to 204. Wherein:
[0060] 201. Obtain the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area.
[0061] The robot's current motion measurement data can be its current acceleration and angular velocity data. This motion measurement data can be detected by sensors installed at the front of the robot's body. These sensors can be inertial measurement sensors, encoders, lidar, or visual odometry, among other types.
[0062] For example, the work area is the area where the robot works, the current running position is the robot's current location, the terrain image data of the current running position is used to describe the image information of the current running position, and the terrain height data of the current running position is used to describe the height information of the current running position. The height information can be used to determine the terrain information of the current running position, such as slope, height, curvature, etc. The terrain image data and terrain height data can be detected by a vision sensor capable of simultaneously outputting color images and depth information, such as an RGB-D camera.
[0063] When a robot restarts, fails to track, or loses its pose, it needs to be relocalized to ensure that it can perceive its position and posture in the work area and continue to perform subsequent navigation tasks. When outdoor service robots work outdoors, the outdoor environment changes frequently. Taking lawnmower robots as an example, the height, density, color, and texture of the lawn in the work area change frequently with the season, climate, and mowing behavior. Furthermore, during operation, the lawn is mowed, leading to changes in height and boundaries, or different lawn areas are watered or shaded, resulting in different behaviors. Relocalization methods that rely solely on visual appearance are difficult to adapt to these long-term, cumulative changes, easily leading to relocalization failure.
[0064] However, although the visual appearance of the lawn surface and the height of the topsoil are prone to change, the topographic geometry of the underlying working area (such as the overall elevation undulation, slope, curvature and other spatial geometric features of the ground) can remain highly stable over a long period of time.
[0065] Therefore, this application embodiment generates a local terrain descriptor by acquiring terrain height data of the current operating location, transforming the relocation benchmark from a variable two-dimensional visual appearance into stable three-dimensional terrain geometric features, thereby significantly improving the accuracy and long-term robustness of robot relocation in complex and variable outdoor environments.
[0066] Therefore, by acquiring terrain image data and terrain height data of the robot's current operating position, and further obtaining terrain information of the current operating position, and using the terrain information as a reference for relocalization, the impact of changes in the appearance of the working area image can be reduced during the robot relocalization process.
[0067] 202. Based on the current motion measurement data, the terrain image data and terrain height data of the current operating location, generate a local terrain descriptor for the current operating location.
[0068] In this embodiment, when repositioning is required, the robot moves in place or around a preset radius and collects terrain image data and terrain height data of the current running position over a period of time. The collection frequency and number of collection frames can be determined according to the sensor performance and the robot's movement speed, so that the collected terrain image data and terrain height data can cover the current running position.
[0069] For example, the collected multi-frame terrain height data is preprocessed by removing noise information from the data through median filtering or bilateral filtering. Then, the holes in the terrain height data are filled by interpolation or a neighborhood-based hole-filling algorithm. Finally, obvious outliers in the terrain height data are removed to obtain a high-quality terrain height data sequence that has undergone noise suppression, hole filling and outlier removal.
[0070] For example, a local terrain descriptor is used to characterize terrain information at the robot's current operating location.
[0071] In some embodiments, terrain information such as height, slope, curvature and other geometric features of the current operating location can be calculated based on current motion measurement data, terrain image data and terrain height data. The geometric features are then concatenated to obtain a high-dimensional vector, thereby obtaining a local terrain descriptor.
[0072] In some embodiments, when the robot has sufficient computing power, a pre-trained convolutional neural network can be used to process the terrain image data and terrain height data. Features are extracted through the convolutional neural network, fused through a feature layer, and then reduced in dimensionality by a fully connected layer to output a local terrain descriptor representing the current operating position. Optionally, the local terrain descriptor obtained by the convolutional neural network has the same vector format as the local terrain descriptor calculated based on terrain information; a suitable method can be selected to generate the local terrain descriptor according to the actual situation.
[0073] 203. Based on the local terrain descriptor of the current operating location, index the preset terrain descriptor index library to determine the target area corresponding to the current operating location.
[0074] The terrain descriptor index library includes mapping relationships between multiple sets of terrain descriptors and regions within the work area. This index library is constructed during the mapping phase of the work area. A robot traverses the work area using a SLAM (Simultaneous Localization and Mapping) trajectory to collect terrain image data, terrain height data, and pose data. The work area is then divided into multiple regions, and terrain descriptors are calculated for each region based on its terrain image data, terrain height data, and pose data, thus obtaining the correspondence between multiple sets of terrain descriptors and regions within the work area.
[0075] After determining the local terrain descriptor in step 202 above, the local terrain descriptor is used as an index value and indexed in the preset terrain descriptor index library to determine the area corresponding to the terrain descriptor that matches the local terrain descriptor of the current operating position as the target area, that is, to determine the location information of the current operating position in the working area.
[0076] In some embodiments, a similarity search is performed in the terrain descriptor index based on the local terrain descriptor to select N candidate regions with high similarity, and the target region is determined based on the N candidate regions.
[0077] 204. Based on the target area and current motion measurement data, the robot is relocalized.
[0078] For example, the initial pose of the robot for relocalization is determined by combining the terrain features of the target area, the corresponding pose data, and the current motion measurement data.
[0079] The aforementioned robot relocalization method first acquires the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area. Then, based on the current motion measurement data, terrain image data, and terrain height data, a local terrain descriptor for the current operating position is generated. Next, based on the local terrain descriptor, an index is retrieved from a pre-defined terrain descriptor index library to determine the target area corresponding to the current operating position. The terrain descriptor index library includes mapping relationships between multiple sets of terrain descriptors and areas within the work area. Finally, the robot is relocalized based on the target area and the current motion measurement data. By acquiring the robot's current motion measurement data, terrain image data, and terrain height data to generate a local terrain descriptor for the current operating position, and then relocalizing the robot based on the matching of the local terrain descriptor, the accuracy of robot relocalization is improved because the terrain information changes less compared to the environmental appearance information.
[0080] In the embodiments of this application, the construction process of the terrain description sub-index library in the above-mentioned mapping process is as follows: Figure 3 As shown, the steps 301 to 304 may be included:
[0081] 301. Obtain the optimized pose of the robot at multiple locations, the terrain image data of multiple locations in the work area, and the terrain height data of multiple locations in the work area obtained by synchronous localization and mapping when the robot is cruising in the work area.
[0082] For example, the robot cruises within the work area using a SLAM trajectory, collecting optimized poses, terrain image data, and terrain height data at multiple locations within the work area, and records the acquisition time of each data frame.
[0083] The terrain height data from multiple locations are preprocessed according to the preprocessing method described in the above steps to obtain a terrain height data sequence.
[0084] 302. Generate a global terrain elevation map of the work area based on the optimized poses of multiple locations in the work area, the terrain image data of multiple locations in the work area, and the terrain height data of multiple locations in the work area.
[0085] The optimized pose, terrain image data, and terrain height data of multiple locations in the multi-frame operation area are time-stamp aligned to correct the temporal differences caused by different sensors, so that the optimized pose, terrain image data, and terrain height data correspond one-to-one in time.
[0086] In some embodiments, the relative pose change of adjacent frames of pose data is estimated by short-time pre-integration. Based on the relative pose change, the terrain image data and terrain height data are mapped from the camera coordinate system to the gravity reference system by coordinate transformation, thereby maintaining the consistency of the ground direction between different frames, that is, ensuring that the direction of gravity is consistent in each frame of data, eliminating the tilt or distortion of terrain image data and terrain height data caused by pose change, and providing accurate basic data for subsequent terrain fusion to generate a global terrain elevation map.
[0087] Point cloud data is obtained by mapping terrain image data and terrain height data to a gravity reference frame. In this way, pose estimation and point cloud alignment are performed with the assistance of terrain image data to construct a global terrain elevation map.
[0088] 303 divides the global terrain elevation map into multiple regions and calculates the terrain descriptor for each region.
[0089] For example, within the world coordinate system, a two-dimensional grid with a preset resolution is established. Point cloud data falling into each grid cell is statistically analyzed. The mean height and variance of each grid cell are calculated based on the terrain height data of the point cloud data. The formula for calculating the mean height is the sum of all terrain height data within the grid cell divided by the number of point cloud data points. The variance of height is calculated based on the mean height. If no point cloud data exists in the grid, it is marked as a hole. The preset resolution can be divided according to the working area, such as 1m × 1m; this embodiment does not impose such a limitation.
[0090] The global terrain elevation map is divided into multiple local area units according to a preset size. Each local area unit includes several grids, and a terrain descriptor is calculated for each local area unit.
[0091] For example, for each local region unit, based on the terrain height data of the grid units included in the local region unit, a slope histogram based on the height gradient is calculated for the local region unit, the curvature of the local region unit is calculated based on the curvature estimation algorithm based on the point neighborhood, the void ratio is calculated based on the void grids of the local region unit and the total number of grids, the height mean and height variance of each grid included in the local region unit are statistically analyzed, and a multi-dimensional vector is obtained by splicing the above statistical information as the terrain descriptor of the local region unit.
[0092] In one embodiment, the method further includes normalizing and standardizing the dimensions of the terrain descriptor for each local region unit to obtain a standardized terrain descriptor. The preset size and preset resolution can be set according to the actual operating area, and this application does not impose any restrictions on them.
[0093] 304. Based on each region and its corresponding terrain descriptor, establish a terrain descriptor index library.
[0094] Each standardized terrain descriptor is used as an index, and an index entry is formed by the corresponding region, the center pose of each region, the keyframe identifier, and the frame time, and stored. The center pose is the pose data corresponding to the point cloud data at the center of the region, and the keyframe identifier is the identifier information of the frame corresponding to the point cloud data at the center of the region. Based on all regions obtained from the operational area division and the multiple index entries consisting of the corresponding terrain descriptors, a terrain descriptor index library is constructed.
[0095] For example, the KD-tree index structure is used to index each terrain descriptor, and the local region center pose, keyframe identifier and frame time corresponding to the terrain descriptor are written into the index entry and persistently stored.
[0096] In the above embodiments, a global terrain elevation map is generated by acquiring terrain data of the work area, and a terrain description sub-index library is established based on the global terrain elevation map. Feature retrieval can be performed in the terrain description sub-index library based on the current terrain features to determine the current position and realize the robot's relocalization.
[0097] In one embodiment, the process of generating a local terrain descriptor in step 202 above is as follows: Figure 4 As shown, it may include steps 401 and 402:
[0098] 401. Based on the current motion measurement data, the terrain image data of the current operating location, and the terrain height data, generate a local height sub-map of the current operating location.
[0099] When the robot needs to relocalize, short-term data acquisition is initiated. It moves in place or around a preset radius to collect current motion measurement data, terrain image data of the current operating position, and terrain height data. The steps for generating a local height submap are as follows: Figure 5 As shown, it may include:
[0100] 501. Perform timestamp alignment processing on the current motion measurement data, the terrain image data of the current operating location, and the terrain height data.
[0101] As described in the above embodiments, the current motion measurement data, the terrain image data of the current operating position, and the terrain height data are timestamped to ensure that the multi-source sensor data are synchronized in the time dimension, so as to achieve a one-to-one correspondence between the current motion measurement data, the terrain image data, and the terrain height data in the time dimension.
[0102] 502. Based on the current motion measurement data, determine the gravity direction vector, and combine it with the terrain image data to map the timestamp-aligned terrain height data to the gravity reference frame to obtain the point cloud data of the current running position.
[0103] In some embodiments, similar to the construction method of the global terrain elevation map described above, the relative pose change of adjacent frames of pose data is estimated by short-time pre-integration to determine the gravity direction vector. Combined with the terrain image data, the timestamp-aligned terrain height data is mapped from the camera coordinate system to the gravity reference system by coordinate transformation to obtain the point cloud data of the current running position.
[0104] 503. Based on the point cloud data, take the robot's current running position as the center point and use a preset distance as the radius to extract the point cloud data to generate a local height sub-map.
[0105] In some embodiments, a preset shape is defined as the coverage area of a local height submap, with the robot's current operating position as the center point and a preset distance as the radius. The preset shape can be square, circular, or other shapes. Based on the point cloud data falling within this coverage area, point cloud data is extracted to generate a local height submap.
[0106] Optionally, to reduce the impact of sensor noise and measurement anomalies, hole repair algorithms based on neighborhood interpolation can be used to repair holes in the local height submap and fill in missing data. Alternatively, low-pass smoothing of the local height submap can be performed using filters to remove noise.
[0107] 402, Determine the local terrain descriptor of the current operating location based on the local elevation submap.
[0108] In some embodiments, the method of determining the local terrain descriptor of the current operating location based on the local elevation submap is similar to the method of determining the terrain descriptor corresponding to each local region in the global terrain elevation map described above. For example... Figure 6 As shown, steps 601 to 603 may be included:
[0109] 601. Project the local height submap along the direction of gravity onto a two-dimensional plane, and divide the two-dimensional plane into multiple grids according to a preset resolution.
[0110] The local height submap is projected onto a two-dimensional plane along the direction of gravity. Using the same preset resolution as in the above embodiments, the local height submap of the two-dimensional plane is divided into multiple grids, and the point cloud data falling into each grid cell is counted.
[0111] 602. Based on the height data of the point cloud falling within each grid, calculate the mean height and variance of each grid.
[0112] The mean height and variance of each grid cell are calculated based on the terrain height data of the point cloud data in each grid.
[0113] 603. Generate a local terrain descriptor based on the terrain height data of the current operating location, the mean height and variance of each grid.
[0114] The local terrain descriptor also includes at least one of the slope histogram features generated based on the height gradient of each grid and the curvature features based on the point neighborhood.
[0115] In one embodiment, based on the terrain height data of the grids included in the local height submap, a slope histogram based on the height gradient is calculated for the local height submap. The curvature of the local height submap is calculated based on the curvature estimation algorithm of the point neighborhood to obtain curvature features. The void rate is calculated based on the voided grids and the total number of grids. The mean height and variance of the height of each grid included in the local height submap are statistically analyzed. A multidimensional vector is obtained by splicing the above statistical information as the local terrain descriptor of the local height submap.
[0116] The local terrain descriptor is normalized and its dimensions are standardized to obtain the standardized local terrain descriptor.
[0117] In the above embodiments, a local terrain descriptor is generated by obtaining the terrain information of the current location, and the terrain information is used as the reference for positioning to avoid the influence of changes in image appearance.
[0118] In embodiments of this application, the method further includes: if the robot relocalization is successful, updating the terrain description sub-index library by using a weighted smoothing or exponential smoothing strategy based on the local height sub-map of the robot's current running position.
[0119] If the robot relocalizes successfully, the information of the successfully matched target region in the terrain description sub-index is updated based on the local height submap of the current operating location. For example, the point cloud data of the target region is updated using a weighted smoothing or exponential smoothing strategy based on the point cloud data of the local height submap. Accordingly, the statistical information such as the height mean and height variance of the corresponding grid in the target region is updated, and the hole rate, center pose, and other information of the target data are calculated based on the updated point cloud data.
[0120] In some embodiments, the terrain descriptor index library also includes the confidence scores of each terrain descriptor. The confidence score is used to characterize the dispersion of point cloud data in the region. The more concentrated the point cloud data, the higher the confidence score; the more dispersed the point cloud data, the lower the confidence score. Therefore, the updated confidence score is calculated based on the updated point cloud data.
[0121] In the above embodiments, when the robot relocalizes successfully, the terrain description sub-index library is updated based on the local height sub-map of the current operating location to reflect the slow evolution of the environment and maintain the stability of historical information.
[0122] In embodiments of this application, the terrain descriptor index library also includes robot candidate pose data corresponding to each region, where the candidate pose data is also the center pose data of that region; the step of determining the target region corresponding to the current running position by indexing the local terrain descriptor of the current running position in the preset terrain descriptor index library is as follows. Figure 7 As shown, it may include:
[0123] 701. Based on the local terrain descriptor of the current operating location, index the terrain descriptor index library to determine at least one candidate region.
[0124] Using the vector of the local terrain descriptor at the current operating location as an index, a search is performed in the terrain descriptor index library. The similarity between the local terrain descriptor and each terrain descriptor in the index library is calculated. Cosine similarity or Euclidean distance can be used as similarity metrics. Based on the similarity, at least one region with the smallest distance, i.e. the highest similarity, is selected as a candidate region.
[0125] In some embodiments, the similarity calculation between local terrain descriptors and various terrain descriptors can be performed in parallel to improve retrieval speed.
[0126] 702. Based on the similarity score and confidence level of each candidate region, the initial candidate regions are determined.
[0127] The confidence score of each candidate region is used to characterize the dispersion of point cloud data in the candidate region. The initial candidate regions are determined by screening based on the similarity score and confidence score of each candidate region.
[0128] The confidence level of each candidate region can be calculated based on the density of the point cloud data of each candidate region. Based on the similarity score and confidence level of each candidate region, the candidate regions with low similarity or low confidence are eliminated, and the candidate regions with similarity scores greater than the first preset threshold and confidence levels greater than the second preset threshold are selected as the initial candidate regions.
[0129] 703. Based on the inter-regional similarity between any two initial candidate regions in each initial candidate region, determine the anchoring candidate region, and obtain the candidate pose data and local elevation segments corresponding to each anchoring candidate region from the terrain description sub-index library.
[0130] By calculating the inter-regional similarity between each initial screening candidate region, redundant initial screening candidate regions, i.e., those with high similarity, are filtered out. From these high-similarity candidate regions, those with lower confidence are removed to avoid redundant verification, resulting in anchored candidate regions, i.e., high-quality candidate regions.
[0131] Optionally, based on each anchoring candidate region, the candidate pose data of each anchoring candidate region, the terrain image segment of the candidate region, the point cloud data included in the candidate region, etc. are determined from the terrain description sub-index library. The terrain image segment of the candidate region is also a local elevation segment.
[0132] 704. Determine the target region based on the candidate pose data corresponding to each anchoring candidate region and the current motion measurement data.
[0133] In some embodiments, such as Figure 8 As shown, the steps for determining the target region based on each anchoring candidate region may include:
[0134] 801. Combine the candidate pose data corresponding to each anchor candidate region with the current motion measurement data to calculate the pose and generate the initial pose value corresponding to each anchor candidate region.
[0135] For example, the current motion measurement data is processed by short-time pre-integration to obtain the relative pose change. The relative pose change is then combined with each candidate pose data to calculate the initial pose value corresponding to each anchoring candidate region. The pose combination calculation can be a weighted summation process.
[0136] In some embodiments, if the robot also includes data collected by an odometer or wheel speedometer, the pose can be combined and calculated based on the relative pose change, each candidate pose data, odometer data, and wheel speedometer data to obtain the initial pose value corresponding to each anchoring candidate region.
[0137] 802. Based on the initial pose values and local height sub-maps corresponding to each anchoring candidate region, fine registration is performed on local elevation segments at different resolutions to determine the target region.
[0138] The global terrain elevation map is obtained by the robot during its navigation within the work area using simultaneous localization and mapping (SMR). The mapping process described in the above embodiment yields a high-precision global terrain elevation map of the work area. Downsampling or blurring this high-precision global terrain elevation map results in a low-resolution global terrain elevation map. Local elevation segments of different resolutions are obtained by dividing the global terrain elevation map into segments of different resolutions.
[0139] For example, fine registration is performed on the initial pose values corresponding to each anchoring candidate region. Based on the initial pose values and local height sub-maps corresponding to each anchoring candidate region, a quick alignment is first performed at the level of a low-resolution global terrain elevation map to verify the rationality of each initial pose value, and obviously deviated initial pose values are eliminated to achieve rapid screening.
[0140] Then, fine-tuning is performed by executing an iterative nearest-point algorithm on the point cloud level or a high-resolution global terrain elevation map. This process integrates inlier selection and robust weight updates to resist noise. Optionally, inlier selection can be performed using RANSAC (Random Sample Consensus). By randomly selecting several spatial points, a transformation matrix is calculated based on the correspondence between the spatial points and the global terrain elevation map. Inliers are those that coincide with points on the global terrain elevation map under this transformation matrix, provided the error is less than an error threshold. This process is repeated iteratively to obtain the transformation matrix and the set of inliers that have the largest number of inliers.
[0141] Finally, by comparing the registration residuals and inlier percentages of each initial pose value on a high-resolution global terrain elevation map, a preset number of initial pose values with high registration confidence are selected, provided that the configuration residual is less than a preset configuration residual threshold and the inlier percentage is greater than a preset inlier percentage threshold. Then, the selected preset number of initial pose values are compared with the robot pose at the previous moment. Based on temporal rationality, the consistency between each initial pose value and the robot pose at the previous moment is judged, meaning that each initial pose value meets the temporal continuity requirement with the robot pose at the previous moment. Simultaneously, based on historical observation overlap, the anchoring candidate region corresponding to the initial pose value with the highest consistency and historical observation overlap is selected as the target region. The historical observation overlap is used to evaluate the visual or geometric similarity between the candidate initial pose value and the scene traversed by the robot. It can be quantified by calculating the matching point density or grid overlap after projecting the point cloud of the current local elevation submap onto the historical point cloud map.
[0142] If the configured residual and interior point ratio do not meet the requirements of the preset configuration residual threshold and preset memory ratio threshold, the registration will fail, meaning the target area cannot be determined.
[0143] In the above embodiments, candidate regions are determined by indexing, anchor candidate regions are further screened and determined, and then fine-tuning is performed to determine the target region. By using a coarse-to-fine screening method, search efficiency is significantly improved while reducing false matches, while ensuring accuracy.
[0144] In the embodiments of this application, after the index matches the target region, the robot is repositioned as follows: Figure 9 As shown, it includes:
[0145] 901. Based on the candidate pose data corresponding to the target area, the pose is combined with the current motion measurement data to generate the target pose.
[0146] The relative pose change is obtained by performing short-time pre-integration on the current motion measurement data. The relative pose change is then combined with the candidate pose data corresponding to the target region to calculate the pose. Finally, the pose is transformed into the robot coordinate system to obtain the target pose.
[0147] 902, use the target pose as the pose data for robot relocalization.
[0148] The target pose is published to the robot's SLAM relocalization interface. The SLAM system receives the target pose and uses it as the pose data for the current frame to achieve robot relocalization.
[0149] In some embodiments, if the above relocation process does not match the target area, such as configuration failure, the step of obtaining the local elevation submap is returned. The spatial range of the data collection can be expanded and the collection time increased during the process of re-obtaining the local elevation submap, or the search range can be expanded during the indexing of the local terrain descriptor to improve the probability of successful matching.
[0150] Optionally, the probability of successful target region matching can be improved by acquiring the visual appearance descriptor of the current operating position and indexing it in parallel with the local terrain descriptor. Alternatively, the number of anchor candidate regions can be increased by adjusting the values of the first and second preset thresholds, or other judgment thresholds, thereby expanding the scope of fine registration and improving the probability of successful matching. Another option is to control the robot to enter a manual or remote intervention mode and complete the relocalization process according to the control commands input manually or remotely. In this way, the accuracy of robot relocalization can be improved through different methods, further enhancing the robustness of the robot during operation.
[0151] In the embodiments of this application, such as Figure 10 As shown, a robot relocalization method is provided, including:
[0152] 1001. Obtain the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area.
[0153] 1002. Based on the current motion measurement data, the terrain image data and terrain height data of the current operating position, generate a local height sub-map of the current operating position.
[0154] 1003, Determine the local terrain descriptor of the current operating position based on the local elevation submap.
[0155] 1004. Based on the local terrain descriptor of the current operating location, index the terrain descriptor index library to determine at least one candidate region.
[0156] 1005. Based on the similarity score and confidence level of each candidate region, the initial candidate regions are determined.
[0157] 1006. Based on the inter-regional similarity between any two preliminary candidate regions in each of the preliminary candidate regions, determine the anchoring candidate region, and obtain the candidate pose data and local elevation segments corresponding to each anchoring candidate region from the terrain description sub-index library.
[0158] 1007. Combine the candidate pose data corresponding to each anchor candidate region with the current motion measurement data to calculate the pose and generate the initial pose value corresponding to each anchor candidate region.
[0159] 1008. Based on the initial pose values corresponding to each of the anchoring candidate regions and the local height sub-map, fine registration is performed on local elevation segments at different resolutions to determine the target region.
[0160] 1009. Based on the candidate pose data corresponding to the target area, the pose is combined with the current motion measurement data to generate the target pose.
[0161] 1010, use the target pose as the pose data for robot relocalization.
[0162] In the above embodiments, the pose data for robot relocalization can be determined by simple indexing using local terrain descriptors, resulting in higher matching efficiency.
[0163] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0164] Based on the same inventive concept, this application also provides a robot relocation device for implementing the robot relocation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more robot relocation device embodiments provided below can be found in the limitations of the robot relocation method described above, and will not be repeated here.
[0165] In one exemplary embodiment, such as Figure 11 As shown, a robot relocalization device is provided, comprising: an acquisition module 1101, a generation module 1102, an indexing module 1103, and a relocalization module 1104, wherein:
[0166] The acquisition module 1101 is used to acquire the robot's current motion measurement data, as well as the terrain image data and terrain height data of the robot's current running position in the work area;
[0167] The generation module 1102 is used to generate a local terrain descriptor for the current running position based on the current motion measurement data, the terrain image data and terrain height data of the current running position;
[0168] The index module 1103 is used to index the local terrain descriptor of the current operating location in a preset terrain descriptor index library to determine the target area corresponding to the current operating location; the terrain descriptor index library includes the mapping relationship between multiple sets of terrain descriptors and areas in the operating area;
[0169] The relocalization module 1104 is used to relocalize the robot based on the target area and current motion measurement data.
[0170] In one embodiment, the generation module 1102 is specifically used to generate a local elevation sub-map of the current operating position based on the current motion measurement data, the terrain image data of the current operating position, and the terrain elevation data; and to determine a local terrain descriptor of the current operating position based on the local elevation sub-map.
[0171] In one embodiment, the current motion measurement data includes acceleration information. The generation module 1102 is specifically used to perform timestamp alignment processing on the current motion measurement data, the terrain image data of the current running position, and the terrain height data; determine the gravity direction vector based on the current motion measurement data, and combine it with the terrain image data to map the timestamp-aligned terrain height data to the gravity reference frame to obtain the point cloud data of the current running position; based on the point cloud data, take the robot's current running position as the center point and use a preset distance as the radius to extract the point cloud data to generate a local height sub-map.
[0172] In one embodiment, the generation module 1102 is specifically used to project a local height submap onto a two-dimensional plane along the direction of gravity, and divide the two-dimensional plane into multiple grids according to a preset resolution; calculate the mean height and variance of each grid based on the height data of the point cloud falling into each grid; generate a local terrain descriptor based on the terrain height data of the current running position, the mean height and variance of each grid; the local terrain descriptor also includes at least one of the slope histogram feature generated based on the height gradient of each grid and the curvature feature based on the point neighborhood.
[0173] In one embodiment, the apparatus further includes an update module for updating the terrain description sub-index library by using a weighted smoothing or exponential smoothing strategy, based on a local height submap of the robot's current operating position, if the robot relocalization is successful.
[0174] In one embodiment, the device further includes a construction module for acquiring optimized poses of multiple locations obtained by simultaneous localization and mapping (SLAM) during robot navigation in the work area, terrain image data of multiple locations in the work area, and terrain height data of multiple locations in the work area; generating a global terrain elevation map of the work area based on the optimized poses of multiple locations in the work area, the terrain image data of multiple locations in the work area, and the terrain height data of multiple locations in the work area; dividing the global terrain elevation map into multiple regions and calculating the terrain descriptor corresponding to each region; and establishing a terrain descriptor index library based on each region and the terrain descriptor corresponding to each region.
[0175] In one embodiment, the terrain descriptor index library also includes robot candidate pose data corresponding to each region; the indexing module 1103 is specifically used to index the terrain descriptor index library based on the local terrain descriptor of the current running position to determine at least one candidate region; to filter based on the similarity score and confidence of each candidate region to determine the initial candidate region; the confidence of each candidate region is used to characterize the dispersion of point cloud data in the candidate region; to determine the anchoring candidate region based on the inter-regional similarity between any two initial candidate regions, and to obtain the candidate pose data and local elevation segments corresponding to each anchoring candidate region from the terrain descriptor index library; and to determine the target region based on the candidate pose data corresponding to each anchoring candidate region and the current motion measurement data.
[0176] In one embodiment, the indexing module 1103 is specifically used to perform pose combination calculations with the current motion measurement data for each anchoring candidate region to generate initial pose values for each anchoring candidate region; and to perform fine registration in local elevation segments of different resolutions based on the initial pose values and local elevation sub-maps for each anchoring candidate region to determine the target region.
[0177] In one embodiment, the relocalization module 1104 is specifically used to perform pose combination calculation based on the candidate pose data corresponding to the target region and the current motion measurement data to generate the target pose; and to use the target pose as the pose data for robot relocalization.
[0178] Each module in the aforementioned robot repositioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0179] In one exemplary embodiment, a robot is provided, including a body, a memory, and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.
[0180] In one exemplary embodiment, the robot includes a vision sensor and an inertial measurement sensor, both of which are disposed at the front of the main body of the robot.
[0181] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above method embodiments.
[0182] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above method embodiments.
[0183] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the above method embodiments.
[0184] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0185] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0186] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0187] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A robot relocalization method, characterized in that, The method includes: Acquire the robot's current motion measurement data, as well as terrain image data and terrain height data of the robot's current operating position in the work area; Based on the current motion measurement data, the terrain image data and terrain height data of the current operating location, a local terrain descriptor for the current operating location is generated; Based on the local terrain descriptor of the current operating location, an index is performed in a preset terrain descriptor index library to determine the target area corresponding to the current operating location; the terrain descriptor index library includes multiple sets of mapping relationships between terrain descriptors and areas in the operating area; The robot is repositioned based on the target area and the current motion measurement data.
2. The method according to claim 1, characterized in that, The step of generating a local terrain descriptor for the current operating location based on the current motion measurement data, the terrain image data of the current operating location, and the terrain height data includes: Based on the current motion measurement data, the terrain image data and terrain height data of the current operating position, a local height sub-map of the current operating position is generated; The local terrain descriptor for the current operating location is determined based on the local elevation submap.
3. The method according to claim 2, characterized in that, The current motion measurement data includes acceleration information. Generating a local elevation sub-map of the current operating position based on the current motion measurement data, the terrain image data of the current operating position, and the terrain elevation data includes: The current motion measurement data, the terrain image data of the current operating location, and the terrain height data are time-stamp aligned. Based on the current motion measurement data, the gravity direction vector is determined, and combined with the terrain image data, the timestamp-aligned terrain height data is mapped to the gravity reference system to obtain the point cloud data of the current running position; Based on the point cloud data, the robot's current running position is used as the center point, and the point cloud data is extracted with a preset distance as the radius to generate the local height sub-map.
4. The method according to claim 2, characterized in that, The step of determining the local terrain descriptor of the current operating position based on the local elevation submap includes: The local height submap is projected onto a two-dimensional plane along the direction of gravity, and the two-dimensional plane is divided into multiple grids according to a preset resolution; Based on the height data of the point cloud falling within each of the grids, calculate the mean height and variance of each grid. The local terrain descriptor is generated based on the terrain height data of the current operating location, the mean height and the variance of each grid; the local terrain descriptor also includes at least one of the slope histogram feature generated based on the height gradient of each grid and the curvature feature based on the point neighborhood.
5. The method according to claim 2, characterized in that, The method further includes: If the robot relocalization is successful, the terrain description sub-index is updated using the local height sub-map of the robot's current operating position through a weighted smoothing or exponential smoothing strategy.
6. The method according to any one of claims 1 to 5, characterized in that, The construction process of the terrain description sub-index library includes: The system acquires optimized poses of multiple locations obtained by simultaneous localization and mapping (SLAM) during the robot's navigation in the work area, terrain image data of multiple locations in the work area, and terrain height data of multiple locations in the work area. Based on the optimized poses of multiple locations in the work area, the terrain image data of multiple locations in the work area, and the terrain height data of multiple locations in the work area, a global terrain elevation map of the work area is generated. The global terrain elevation map is divided into multiple regions, and the terrain descriptor corresponding to each region is calculated; A terrain descriptor index library is established based on each region and its corresponding terrain descriptor.
7. The method according to any one of claims 2 to 5, characterized in that, The terrain descriptor index library also includes candidate robot pose data for each region; the step of indexing the target region corresponding to the current running position in the preset terrain descriptor index library based on the local terrain descriptor of the current running position includes: Based on the local terrain descriptor of the current operating location, index the terrain descriptor index library to determine at least one candidate region; The candidate regions are selected by filtering based on their similarity scores and confidence levels; the confidence level of each candidate region is used to characterize the dispersion of point cloud data in the candidate regions. Based on the inter-regional similarity between any two preliminary candidate regions, anchoring candidate regions are determined, and candidate pose data and local elevation segments corresponding to each anchoring candidate region are obtained from the terrain description sub-index library. The target region is determined based on the candidate pose data corresponding to each anchoring candidate region and the current motion measurement data.
8. The method according to claim 7, characterized in that, The step of determining the target region based on the candidate pose data corresponding to each of the anchoring candidate regions and the current motion measurement data includes: The candidate pose data corresponding to each anchor candidate region is combined with the current motion measurement data to calculate the pose, thereby generating the initial pose value corresponding to each anchor candidate region. Based on the initial pose values corresponding to each of the anchoring candidate regions and the local elevation sub-map, fine registration is performed on the local elevation segments at different resolutions to determine the target region.
9. The method according to claim 8, characterized in that, The relocalization of the robot based on the target area and the current motion measurement data includes: Based on the candidate pose data corresponding to the target region, pose combination calculation is performed with the current motion measurement data to generate the target pose. The target pose is used as the pose data for robot relocalization.
10. A robot comprising a main body, a memory, and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.
11. The robot according to claim 10, characterized in that, The robot includes vision sensors and inertial measurement sensors.