Method and device for updating an environmental map used for robot self-position estimation
The hybrid approach of using static maps with centralized batch-optimized SLAM updates addresses map degradation issues in robot navigation, ensuring efficient and accurate positioning without operational interruptions.
Patent Information
- Application Number
- JP2023555663
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-03-15
- Filing Date
- 2022-03-11
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2042-03-11
AI Technical Summary
Existing robot navigation systems face challenges with map degradation due to errors in simultaneous localization and mapping (SLAM), leading to position estimation drift and the need for costly and disruptive manual remapping, which interrupts operations.
A hybrid approach where robots use static maps for localization, and updates are performed by a centralized computer facility based on logged data from multiple robots, allowing batch-optimized SLAM to maintain high-quality maps without interrupting operations.
This method ensures high-quality map updates are performed efficiently and automatically, reducing operational disruptions and maintaining accurate robot positioning, even with environmental changes.
Smart Images

Figure 0007582504000006 
Figure 0007582504000007 
Figure 0007582504000008
Abstract
Description
[Technical field]
[0001] The methods and apparatus disclosed herein provide for the use of a static map of the environment that is used by the robot to perform localization while operating within the environment, and further provides for updating the map on an ongoing basis based on batch processing of logged data collected from the robot. [Background technology]
[0002] A robot performing a task in an environment such as a warehouse or factory floor, for example, moves from one "goal" to the next, with each goal having a corresponding location in the environment. Autonomous navigation by a robot within an environment to assist in the performance of a task requires the robot to maintain an accurate understanding of its position and orientation within the environment, which is called "localization." The phase corresponding to the robot's performance of a task is called the "task phase."
[0003] A robot may, for example, sense its immediate surroundings using one or more types of sensors and compare what it senses to a map of the environment to determine its most likely current location and orientation. Such a map may be created using sensor data from a CAD-based floor plan of the environment or in a manual mapping phase performed prior to the task phase.
[0004] A technique called "simultaneous localization and mapping," or SLAM, allows a robot to create a map of its environment and continually refine that map as it detects differences between what its sensors sense about its current position and orientation and what it should sense about its position and orientation according to the map.
[0005] Maps that are updated by a robot during task execution, such as is done using traditional SLAM, are referred to as "online" or "dynamic" maps to indicate that the robot continually updates its map as it uses it to navigate the environment for task execution. The online map used by each robot degrades over time due to errors inherent in the SLAM-based map update process. Map degradation manifests, for example, as drift of important landmarks that the robot uses for localization.
[0006] When the initial map is significantly different from the current environment, traditionally the only solution is to remap the environment and use the resulting map for the robot to navigate in the environment. However, manual mapping has drawbacks, such as the difficulty of acquiring the map's "modality" and the time and cost involved in manually remapping the environment. Such costs include the penalty of taking the robots that rely on the map offline during the remapping process. Summary of the Invention
[0007] The map update technique provides the performance advantages afforded by an autonomous robot using a good quality static map for autonomous navigation within an environment, while providing the convenience of automatic updates. In particular, one or more robots log data while autonomously navigating within an environment according to the static map, and a computing facility, such as a centralized server, collects the logged data as historical log data and performs a map update process using the historical log data. Such operations allow the static map to be updated periodically or as needed based on actual observational data collected from the environment by the robot, while allowing richer, more time-consuming updates to be performed by the computing facility without the need to take the robot offline for computations.
[0008] One embodiment includes a method executed by a computing facility. The method includes monitoring localization quality for one or more robots performing autonomous navigation within an environment according to a static map of the environment. The localization quality experienced by any particular robot at any particular time depends on the extent to which relevant portions of the static map match local environmental features sensed by the robot. The method further includes triggering an update procedure in response to the monitored localization quality for the one or more robots falling below a defined threshold. The update procedure includes outputting map updates for the static map for transmission to the one or more robots. The map updates are calculated from past log data based on the one or more robots reporting log data while operating in the environment. The log data from any particular robot includes sensor data used by the particular robot to perform localization for autonomous navigation within the environment and one or both of corresponding odometry data and corresponding localization data used in the localization process performed by the particular robot.
[0009] Another embodiment comprises a computing facility having a communications interface and a processing circuit operatively associated with the communications interface (i.e., for sending and receiving information using the communications interface). The processing circuit is configured to monitor localization quality for one or more robots performing autonomous navigation within an environment according to a static map of the environment. The processing circuit is further configured to trigger the aforementioned update procedure in response to the monitored localization quality for the one or more robots falling below a defined threshold.
[0010] Of course, the present invention is not limited to the above features and advantages, and those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a block diagram of an embodiment of a system comprising one or more robots and a computing facility. [Diagram 2] FIG. 1 is a block diagram of an embodiment of a computing facility implementing a map update function to update a map of an environment, and a robot implementing a localization function that uses a static copy of the map for autonomous navigation within the environment. [Diagram 3] FIG. 2 is a logic flow diagram of one embodiment of a method for updating a map. [Figure 4] FIG. 2 is a logic flow diagram of one embodiment of a method for a robot to perform localization and transmit log data. [Diagram 5] FIG. 13 is a logic flow diagram for one embodiment of a conditional map update. [Figure 6] FIG. 13 is a logic flow diagram for one embodiment of conditional map development. [Figure 7] 1 is a plot of localization quality as a function of deviation between the environment and a static map of the environment used by the robot to localize itself within the environment. [Figure 8] FIG. 1 is a logic flow diagram of one embodiment of a method of operation by a system comprising a computing facility for updating a map and individual robots using a static copy of the map to perform self-localization while operating within an environment represented by the map. [Figure 9] FIG. 4 is a state diagram of one embodiment of a method for updating a map in response to specific input from a user. [Figure 10] FIG. 1 is a block diagram of a robot, also referred to as an end unit or an autonomous intelligent vehicle (AIV), and functional units or processing modules and associated data in a computer facility, also referred to as a computer server. [Figure 11A] FIG. 11 is a logic flow diagram detailing one embodiment of the Full SLAM BOMU introduced in FIG. [Figure 11B] FIG. 11 is a logic flow diagram detailing one embodiment of the Full SLAM BOMU introduced in FIG. [Figure 12] 1 is a plot of an example particle state during self-localization. [Figure 13]FIG. 1 is a block diagram of an example of a multimodal environment. [Figure 14] FIG. 2 is a block diagram of an example of an updated environment map. [Figure 15] FIG. 2 is a logic flow diagram of a method of operation by a computing device according to one embodiment. [Figure 16] FIG. 2 is a logic flow diagram of a method of operation by a computing device according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] FIG. 1 is a block diagram of an embodiment of a system 8 comprising a computer facility 10, also referred to as a "computer server." A map 12 of an environment 14 is used by one or more robots 16 to perform localization while operating within the environment 14. Each robot 16 uses a static copy of the map 12 to perform localization. As used herein, the term "localization" refers to operations performed by a robot 16 to determine its pose within the environment 14, and the term "pose" refers at least to the position of the robot in an applicable coordinate system or frame of reference. "Pose" may also refer to the orientation of the robot 16 within the environment 14. An exemplary "pose" includes two-dimensional or three-dimensional coordinate values and one or more angle values that indicate the heading direction of the robot 16.
[0013] Certain aspects of map updating of particular interest herein benefit from having multiple robots 16 operating within the environment 14. Correspondingly, Figure 1 provides an example of a "fleet" of three robots 16-1, 16-2, and 16-3 operating in the environment 14. However, the techniques disclosed herein apply to fleets of robots 16 ranging from a single robot 16 to tens or hundreds of robots 16, or even more.
[0014] Similar flexibility applies with respect to the environment 14 and the robots 16 operating within the environment 14. Non-limiting examples include a warehouse or factory floor as the environment 14 and wheeled or tracked autonomous vehicles as the robots 16. However, the robots 16 may also be unmanned aerial vehicles (UAVs) operating within the airspace of the environment 14, such as drones carrying payloads in open or closed airspace. Due to the range of robot types contemplated, the robots 16 are also referred to as "end units" or autonomous intelligent vehicles (AIVs). Furthermore, the individual robots 16 that make up a "fleet" operating within the environment 14 need not be homogenous, and there may be two or more types of robots 16 with different configurations, capabilities, and task responsibilities.
[0015] Regardless of the type of robot, the commonality of operation is that each robot 16 operating within the environment performs autonomous navigation within the environment 14 by performing localization using a static copy of the map 12 of the environment 14. In contrast to each robot 16 performing simultaneous localization and mapping, or "SLAM," for autonomous navigation, each robot 16 performs localization using the map 12 but does not update its copy of the map 12 during "live" navigation within the environment 14. Instead, the computing equipment 10 performs map updates on an ad-hoc basis, either conditionally or on a trigger basis, i.e., on demand.
[0016] Employing a hybrid approach in which the robot 16 does not update its copy of the map 12, but instead operates on a static or fixed copy of the map 12, with map updates being performed by the computing equipment 10, the robot 16 can be provided with an arbitrarily high quality map. For example, the map 12 can be initially determined using an offline process in which the robot 16 or other equipment involved in developing the map 12 of the environment 14 is not obligated to perform other tasks or jobs, and multiple "passes" or iterations may be used to develop an initial version of the map 12. In this way, the robot 16 benefits from having a higher quality map than if it were developed individually using SLAM.
[0017] Of course, environment 14 may change slowly or rapidly, i.e., over time, or may "cycle" through different configurations depending on the time of day, day, or week, etc. The cyclical or recurrent changes in environment 14 are sometimes referred to as the "mode" of environment 14.
[0018] Thus, the state of the environment 14 at any location and any time may deviate from the map 12 currently being used by the robot 16. Such deviations affect the "quality" of localization by an individual robot 16 using the map 12 to navigate within the environment 14. As used herein, "quality" measures or represents a level of confidence associated with the ability of the robot 16 to determine the most probable pose at any given time during live operation of the robot 16 within the environment 14. As the map 12 becomes "stale" in light of changes occurring within the environment 16, the quality of localization deteriorates, at least within the affected areas of the environment 14.
[0019] A degradation in localization quality impacts the ability of the robot to perform assigned tasks and also represents a risk to the safety of people sharing the environment 14 with the robot 16, and to the robot 16 and other equipment within the environment 14. Thus, an important advantage of the hybrid localization and map updating contemplated herein is the conditional or on-demand updating of the map 12 by the computing equipment 10, with the advantage of processing map updates on a "batch" basis, without the need to take the robot 16 offline while the map 12 is being updated.
[0020] The hybrid approach allows the robots 16 to operate with the benefit of a high quality map 12, and allows the computing facility 10 to update or refresh its map 12 while the robots 16 continue to operate on their respective current static copies of the map 12. Only minimal interruptions to ongoing operations by the individual robots 16 occur in connection with pushing updates to the map 12 to the robots 16. Furthermore, because the map updates determined by the computing facility 10 are based on the computing facility 10 receiving log data generated by the individual robots 16 as they move within the environment 14 using their respective static copies of the map 12, changes in the environment 14 can be accurately tracked or accounted for.
[0021] In a sense, the hybrid approach can be understood as offloading the task of map updates to an “offline” component such that map updates of any quality are determined while the robot operation is in progress, to allow for the possibility of the robot operation being stalled if the quality of any one or more of the robot's 16 localization estimates deteriorates below a defined quality threshold.
[0022] In one or more embodiments of the hybrid localization and map updating approach, the robot 16 has a wired or wireless link 18 to the computing equipment 10 or other components, such as a local network / proxy 20, that provides an indirect path for sending robot-generated log data 22 to the computing equipment 10 for use in determining map updates. As an example of a wired version of the link 18, a charging station (not shown) provides Ethernet or powerline-based communication between the robot 16 and the computing equipment 10 or proxy 20 while the robot 16 is charging. In other embodiments, the link 18 is wireless, such as a Wireless Local Area Network (WLAM) link or a Wide Area Network (WAN) link.
[0023] Regardless of the implementation details of the link 18, the computer equipment 10 is configured to "push" or "distribute" map updates 24 (illustratively "MUI") to the robots 16 directly or indirectly, for example, via a proxy 20. Such pushes may be performed automatically according to timed criteria or other conditional triggers, or may be performed on-demand, such as under the control of a human operator. The human operator may also provide the final control decision as to whether or when the computer equipment 10 pushes the MUI 24 to the robots 16. As used herein, the MUI 24 includes, for example, the latest updated version of the map 12, or change data that allows each robot 16 to update its static copy of the map 12 to the latest version.
[0024] As a simple example, there may be a "current" version of map 12 provided to each robot 16, and each robot 16 uses a static copy of that current version of map 12 to perform its localization. When each robot 16 receives the MUI 24, the robot 16 replaces or otherwise updates its local static copy of map 12 held in memory for use in resumed localization operations. An individual robot 16 may temporarily stop operation to update its local static copy of map 12 with the received MUI 24, and then resume operation using the updated version of map 12.
[0025] The computing facility 10 obtains the log data 22 generated by the individual robots 16 and uses the log data 22 from the robots 16 to perform its own version of SLAM. The log data 22 from each robot 16 includes at least sensor data logged by the robot 16. The sensor data may include content useful for map annotation. In an exemplary case, the sensor data may include motor current and torque sensed values associated with the robot 16 moving autonomously within the environment 14 via a drive system. The sensor data logged by the robot 16 may comprise environmental sensing data that the robot 16 uses to sense its immediate surroundings and movements within the environment 14 for the purpose of performing self-localization. Such sensor data may include, for example, any one or more of camera data, ultrasonic sensor data, and LIDAR or laser scanner data obtained from planar or volumetric laser scanning performed by the robot 16 to obtain a depth map or point cloud indicative of the physical features of the surroundings within the environment 16.
[0026] In one or more embodiments, the logged data includes various types of sensor data, including sensor data used for self-localization, and further includes localization process results. Thus, unless otherwise clear from the surrounding context or explained otherwise, the term "logged data" herein is understood to mean one or more types of sensor data logged by the robot 16 while operating in the environment 14. Such data may include odometry data, and the logged data in one or more embodiments may further include localization process results, such as data generated by a Monte Carlo localization (MCL) or other localization algorithm running on the robot 16. In at least one example, the robot 16 uses a particle filter-based localization algorithm, and the logged data may include particles generated by the particle filter algorithm.
[0027] As previously mentioned, the performance of SLAM by the computing equipment 10 does not impact the robots 16, which can continue their "normal" or "live" operation in the environment 14 using the current version of the map 12. That is, the robots 16 can continue to operate while the computing equipment 10 collects log data 22 from the individual robots 16 and performs map updates.
[0028] Further, the SLAM performed by the computer facility 10 is distinguished from conventional SLAM used in autonomous vehicles in several ways. First, the computer facility 10 is essentially "playing back" and supplementing localization operations previously performed by the robot 16, rather than performing live localization. Second, the SLAM process is applied to batches of log data 22, which may cover multiple time periods with different environmental characteristics operating between those time periods, which may result from multiple robots 16 viewing the same or overlapping parts of the environment 14 from different perspectives, resulting in different systematic errors affecting each localization process. Correspondingly, in one or more embodiments, the SLAM performed by the computer facility 10 is a batch-optimized SLAM that takes into account or adjusts for the heterogeneity of the log data 22 that is batched for processing.
[0029] 1 includes implementation details for a representative one of the computer equipment 10 and robot 16 according to one or more embodiments. Components included in the computer equipment 10 include a communication interface 30, a processing circuit 32, and a storage 34 for storing, for example, data 36 and the map 12. The computer equipment 10 may also include a user interface 38 or may be coupled to a user interface via the communication interface 30, such as by being communicatively coupled to a PC or other device that provides the user interface.
[0030] The communications interface 30 comprises a network or computer data interface, such as an Ethernet interface. Additionally or alternatively, the communications interface 30 comprises wireless communications circuitry (radio frequency transceiver) to provide wireless communications, e.g., for a Wi-Fi or other WLAM connection. Broadly speaking, the communications interface 30 is operable to output messages or other signals generated by the processing circuitry 32 and to input captured / received messages or other received signals to the processing circuitry 32.
[0031] The processing circuitry 32 comprises, for example, one or more microprocessors or digital signal processors or other digital processing circuits. In at least one embodiment, the processing circuitry 32 is comprised of one or more microprocessors specially adapted to perform the operations described herein for the computer equipment 10 based at least in part on the execution of computer program instructions from one or more computer programs stored in the storage 34. To that end, the storage 34 comprises one or more types of computer readable media, such as a mixture of volatile and non-volatile storage. Examples include Random Access Memory (RAM) as working memory for program execution and FLASH, EEPROM, or Solid-State Disk (SSD) for long-term storage of the involved computer programs and any supporting configuration data 36. Such data includes, for example, log data 22 recorded by the robot 16 and accumulated for batch processing by the computer equipment 10.
[0032] In an exemplary embodiment, each robot 16 includes a drive system 40, an odometry system 42, an environmental sensing system 44, and a robot control system 46. The robot control system 46 may include or be associated with storage 48. The details of these components or subsystems vary depending on the design of the robot 16 and its intended purpose and capabilities. For example, the details of the drive system 40, the odometry system 42, and the environmental sensing system 44 may vary widely, such as for a wheeled or tracked robot intended for autonomous locomotion on the ground, or for an aerial robot intended for autonomous flight in airspace. In this regard, at least the odometry system 42, also referred to as a dead-reckoning system, may be used to measure or estimate the degree or range of movement by the robot 16 relative to some reference position.
[0033] For a wheeled or tracked robot, the exemplary embodiment of drive system 40 includes one or more drive wheels and one or more corresponding drive motors, and odometry system 42 includes one or more rotary encoders or other sensing mechanisms used to sense speed and distance traveled. For an aerial robot, the exemplary embodiment of drive system 40 includes one or more propellers or rotors and one or more corresponding drive motors, and odometry system 42 includes airspeed, altitude, and other sensors.
[0034] With regard to the environmental sensing system 44, in an exemplary embodiment, it comprises one or more LIDAR or RADAR sensors, or both, and may additionally or alternatively comprise one or more cameras for machine vision-based sensing. In one embodiment or scenario, the robot 16 transports materials, subassemblies, or finished products in a production environment and navigates through a building or defined workspace that participates as the environment 14 using a static copy of the map 12 of the environment. For example, the robot 16 in such a case may comprise LIDAR sensors, cameras, ultrasonic sensors, or some mix thereof, to detect obstacles such as walls, pallets, machines, other robots, etc. in the environment 14. Correspondingly, the robot control system 46 performs path planning that assists in performing one or more assigned tasks or jobs. Path planning relies on the robot 16 determining where it is in the environment 14 at any given moment using data generated by its odometry system 42, its environmental sensing system 44, and a local static copy of the map 12 of the environment 14.
[0035] 2 depicts an exemplary arrangement in which the computing equipment 10 implements a map update function 50 and the robot 16 implements a localization function 52 that determines the location of the robot 16 within the environment 14 during operation of the robot 16. Again, there may be multiple robots 16, each implementing a localization function 52 that the robot 16 uses to localize itself within the environment.
[0036] Data recorded by the robot 16 as logged data 22 may include, for example, sensor data used by the robot 16 to perform localization for autonomous navigation within the environment 14, and corresponding odometry data and / or corresponding localization data used in the localization process performed by the robot 16. Such data may be generated or otherwise used by the robot 16 to localize the robot 16 within the environment 14 in real-time or near real-time while the robot 16 performs (operates) an assigned or programmed task within the environment 14 during a task phase.
[0037] As previously mentioned, one or more robots 16 operating within the environment 14 report log data 22, for example on a continuous or periodic basis, that is used by the computing equipment 10 as historical log data 23 for determining map updates. In Figure 2, the log data 22 collected from one or more robots 16 is described as "historical log data 23" to indicate that it may include data collected over an extended period of time, or over multiple time intervals, or from multiple robots 22.
[0038] Thus, the historical log data 23 used by the computer facility 10 may include a potentially large batch of data including multiple sets of log data 22 corresponding to multiple reports from one or more robots 16. Any reference to "data available at the time" in the context of the historical log data 23 shall be understood to refer to data available to the computer facility 10 at the particular time of interest, i.e., data reported from the robots 16. Also, in Fig. 2, "historical map data 25" is described to indicate the fact that the static map 12 in the computer facility 10 may be an updated version of an older copy of the static map, for example, an updated map generated by a subsequent map update performed by the computer facility 10. The historical map data 25 may also be referred to as historical map update information.
[0039] 3 illustrates one embodiment of a method 300 performed by the computer facility 10. The method 300 includes the computer facility 10 receiving (block 302) log data 22 generated by individual robots 16 operating within the environment 14. The method 300 further includes a step or action (304) of the computer facility 10 using the received log data 22 to perform an update of the map 12. Again, the computer facility 10 may receive the log data 22 directly from the robots 16 or indirectly via one or more intermediate media or accessible databases. Additionally, the log data 22 may be periodically transmitted by each robot 16, for example, whenever the robot 16 has logged a threshold amount of data from the robot's 16's continued operation within the environment 14. The computer facility 10 then aggregates or otherwise batches the log data 22 from one or more time periods and / or one or more robots 16 together to obtain data batches that are processed to determine the map updates.
[0040] An optional further step or action in the method 300 includes the computing equipment 10 sending (block 306) the map update information (MUI 24 in FIG. 1 ) or outputting the MUI 24 for distribution to the robot 16. Such actions may be performed autonomously by the computing equipment 10 or may be gated or controlled, subject to a human operator providing the final decision or permission to push the MUI 24 to the robot 16.
[0041] 4 illustrates a method 400 of operation by a robot 16 according to one or more embodiments. The method 400 complements the method 300 in that it provides logged data 22 for use by the computing equipment 10 in updating the map 12 that the robot 16 uses to localize itself within the environment 14 represented by the map 12. Steps or operations of the method 400 include the robot 16 performing a localization within the environment 14 (block 402) and the robot 16 logging data from one or more of its sensors (block 404). Additionally, the method 400 includes the robot 16 transmitting the logged data 22 for use in updating the map 12 (block 406).
[0042] 5 illustrates a method 500 of operation by the computer equipment 10 according to one or more embodiments of "conditional" map updates in the computer equipment 10. The method 500 includes the computer equipment 10 determining whether an update trigger has occurred or is otherwise satisfied (block 502). If not (NO from block 502), the computer equipment continues to monitor for conditional fulfillment of the trigger as part of performing other ongoing processing (block 504), such as collecting or compiling log data 22 from individual robots 16 generated during operation in the environment 14.
[0043] If the update trigger occurs or is otherwise satisfied (YES from block 502), the method 500 continues with the computer equipment 10 performing an update of the map 12 (block 506). As previously described, the computer equipment 10 in one or more embodiments pushes updates to the robots 16 on its own, while in other embodiments the computer equipment 10 follows human input as to whether updates are pushed to the robots 16. Furthermore, in one or more embodiments the computer equipment 10 may follow pushing map updates to the robots 16 until it detects that the localization quality of one or more robots 16 falls below a defined quality threshold. Upon detecting such a drop, the computer equipment 10 in one or more embodiments pushes the last map update determined by it. In other embodiments the computer equipment 10 does not perform map updates until it detects a drop in localization quality.
[0044] 6 illustrates one example of such operations by describing a method 600 performed by the computer facility 10. The method 600 includes the computer facility 10 collecting log data 22 from individual robots 16 operating in the environment 14 (block 602) and performing map updates as a "background" operation (block 604). As used herein, "background" refers to the fact that updates to the map 12 may be performed while one or more robots 16 using the map 12 are operating in the environment 14.
[0045] However, rather than blindly pushing an updated version of the map 12, the computing facility 10 determines whether the existing map quality is good (block 606). Herein, this is determined by whether the localization quality being achieved by the individual robots 16 using their current local static copies of the map 12 is sufficient. If so (YES from block 606), processing returns to the data collection and map update operations (blocks 602 and 604).
[0046] If the computing equipment 10 detects that the existing map quality is not good (NO from block 606), it pushes the map update to the robots 16 (block 608), but in at least one embodiment requires a confirmatory input from a human operator to perform the push. Pushing the map update includes sending the latest results from the most recent execution of the map update in block 604, or sending information derived from those latest results. For example, the computing equipment 10 may send an updated version of the map 12 as the MUI 24, or may send change data (delta information) that each robot 16 uses to update its local static copy of the map 12 to the latest version.
[0047] FIG. 7 shows an example plot of “amount” of change in environment 14 against localization accuracy (i.e., the difference between the estimated and true pose of the robot). “Change” refers to the difference between the characteristics of environment 14 sensed by robot 16 for self-localization and the characteristics of environment 14 represented in the local static copy of map 12 used by robot 16. Localization accuracy remains flat up to a certain threshold amount of change, depicted as “X%” in the figure. Further change beyond the threshold results in degraded localization accuracy. While localization accuracy is not a generally measurable quantity (outside of experiments in controlled conditions), localization quality is an indication of localization accuracy. Poor localization quality correlates with the probability of poor localization accuracy (area beyond threshold-X%). Up to this threshold, localization quality degrades with the amount of change in the environment, but localization accuracy does not degrade significantly.
[0048] This behavior is achieved because so long as a fair percentage of the map 12 in use by the robot 16 is matched with observed environmental features, the position and pose of the robot 16 within the environment will be accurately estimated. Once matching falls below the aforementioned threshold, the degradation of instantaneous localization accuracy due to uncertainty in the robot's pose will accelerate significantly, potentially resulting in the robot 16 switching between correct and incorrect pose determinations within a given time window, causing it to become "lost" in the environment 14.
[0049] Using the depicted labels, the illustrated "A" accuracy level is a baseline or starting accuracy level, and the "B" accuracy level represents a threshold of accuracy degradation corresponding to an equivalent decrease in quality level that becomes a decision point for whether a map update is required for the robots 16 operating in the environment. In one embodiment, if the computing equipment 10 determines that the localization quality for any one of the robots 16 falls below the B threshold, then the MUI 24 needs to be pushed to the robot 16. A second lower threshold, shown in the figures as "C", may be used as a decision point to declare or impose an emergency stop for all robots 16, or at least for any individual robot 16 whose assessed localization quality falls below the C threshold.
[0050] In the broadest sense, the localization quality is a metric that is calculated independently and individually by all robots 16. In one or more embodiments, each robot 16 uses the localization quality metric as a condition for logging sensor data and possibly other data as logged data 22 to be used in subsequent map updates by the computer equipment 10. For example, each robot 16 decides whether to log data or not depending on whether its localization quality is below a defined threshold. There is also an average / minimum localization quality for the entire fleet robots that the computer equipment 10 estimates from the individual localization qualities determined by the robots 16. This summary or overall localization quality is used by the computer equipment 10 as a trigger condition for performing a map update based on logged data 22 from the robots 16.
[0051] 8 illustrates a process or operational framework 800 that assumes that each robot 16 in the robot fleet has been configured with a static copy of the map 12, such as by using a manual update process. The map 12 may have one or more layered "update areas" that represent areas of expected map change. Examples of such areas include loading docks, material staging areas, areas where other mobile machinery is located, etc.
[0052] The "task phase" of operation (block 802) refers to the normal autonomous operation of the robot 16 (or multiple robots 16) within the environment 14, e.g., the robot 16 autonomously navigating the environment 14 in the performance of one or more material transportation jobs. The task phase relies on the robot 16 being configured with a local, static copy of the map 12.
[0053] In one or more embodiments, the robot 16 performing a task phase performs self-localization within the environment 14 using on-board Monte Carlo localization (MCL). The MCL process uses a current static copy of the map 12 held in on-board memory to estimate the pose of the robot 16 and calculates a localization quality metric that indicates the proportion of sensor data that corresponds to the static copy of the map 12 being used by the robot 16. Additionally, the robot 16 logs the sensor data to an archive file. This logging behavior is gated by two criteria: (1) the robot must be located in one of the predefined update regions for the map 12; and (2) the localization quality of the robot 16 must be within a certain threshold range.
[0054] The decision point of item (2) above is represented in the workflow of FIG. 8 as “Is the localization quality good?” (YES, from block 804). If the localization quality is high, there may be incremental / transient changes in the environment 14, but the local static copy of the map 12 held in the robot's memory continues to represent the environment 14 well enough. As a result, the robot 16 collects sensor data, shown in FIG. 8 as “collect background data” (block 806), which may be valuable for observing incremental changes. Conversely, if the localization quality is low, this means that the static copy of the map 12 held by the robot 16 does not accurately represent the actual environment, and a map update is likely to be beneficial.
[0055] A subsequent decision is made, represented in the workflow as "Localization quality has degraded but still OK?" (YES from block 808). The localization score is compared to a second threshold. If the localization quality has degraded but is still sufficient for the robot 16 to complete the task, the robot 16 may continue to operate normally while maintaining background data collection (block 810). It may also begin the time-consuming process of computing a map update with data collected so far from the robot 16 so that a higher quality map is available as soon as possible.
[0056] Also, if the localization score is low enough to impair the operation of the fleet (i.e., the robot 16 will soon be lost and stopped), the robot localization quality will be poor (YES from block 812). From here, a map update is required to return the fleet of robots 16 to operation. The localization of the robot 16 is turned off, and the data logging and provision of the last map for localization is also turned off (i.e., all these components are turned off). However, most importantly, the map update process is triggered. The computer facility 10 uses the available localization data 22 to determine the MUI 24, and a corresponding quality check of the map update may be performed (block 814).
[0057] In particular, the map update algorithm takes as input all of the logged data 22 collected thus far, including observations of incremental changes in the environment, and uses a map update process to calculate a new version of the map 12 for the fleet of robots 16 that includes the detected changes. In at least one embodiment, the map update process is a "multi-modal, multi-unit batch optimization" update process, referring to taking into account or otherwise adjusting for the logged data 22 collected for multiple modes of the environment 14, the fact that the logged data 22 may include data from different robots 16, and the fact that the logged data 22 is "offline" and processed in batches following the live localization operation that resulted in the data's generation. The update process is sometimes described as "BOMU," standing for "batch optimized map update."
[0058] In one or more embodiments, BOMU is a batch-optimized SLAM algorithm. "Traditional" SLAM performs localization and mapping simultaneously while the robot navigates in the environment, and may be able to maintain a certain level of localization quality despite changes in the environment. However, the batch-optimized SLAM disclosed herein continuously achieves high quality of static mapping that is robust against the risk of losing localization completely and map updating collapsing, a risk present in traditional SLAM that performs localization and map updates together. The nature of continuous SLAM also makes it impossible for the robot 16 to determine an accurate target point and perform the corresponding path planning actions.
[0059] In further contrast to batch-optimized SLAM, continuous SLAM typically only looks at a small (running) time window of the last sensor data to make a decision to filter the observation data. As such, continuous SLAM is vulnerable to noise and dynamic artifacts that may be erroneously included in the map. Furthermore, continuous SLAM induces drift of landmark positions in the map with respect to the true measurable coordinate system of the world the robot resides in. The continuous SLAM process also has much more stringent computational requirements than the batch-optimized SLAM described herein, since continuous SLAM is responsible for providing high-frequency pose updates to the robot 16 during the live operation of the robot 16 within the environment 14.
[0060] In batch optimized SLAM, the robot 16 performs real-time or near real-time position estimation, but the responsibility of map updates is "offloaded" to the computing equipment 10. The computing equipment 10 is thus free to perform longer or more sophisticated filtering of the data and to consider multiple data sets from multiple robots 16 for multiple modes of the environment 14. By performing batch-based SLAM in this manner offline in the computing equipment 10, more complex or rigorous SLAM processing is possible, which can produce a higher quality set of map updates than would be achievable within any robot 16 performing continuous SLAM as known in the art.
[0061] Returning to the framework depicted in Figure 8, the quality of the new map generated by the computing facility 10 using BOMU, i.e., the quality of the updated version of the map 12, is evaluated in block 816. In one or more embodiments, the evaluation is both programmatic (i.e., a measure of reliability determined by the computing facility 10) and subjective (i.e., by a human operator). This map calculation and evaluation can be performed as a background process without interrupting normal operation of the robot 16 to stop production, at least if such operation is performed before the localization quality degrades to some critical threshold.
[0062] If the quality is found to be sufficient (YES from block 816), the computing facility 10 pushes the MUI 24 to the fleet of robots 16 (block 818), which then use the MUI 24 to update their local static copies of the map 12. However, if the assessed quality of the newly updated map is insufficient (NO from block 816), a manual remapping of the environment 14 is required (block 820).
[0063] Responsibility for executing the workflow is shared between the robots 16 and the computer equipment 10. In one embodiment, the computer equipment 10 continuously downloads archive logs of log data 22 from the individual robots 16 in the fleet (the output of background data collection operations). The computer equipment 10 tracks when the robots 16 enter and leave any of the predefined updater regions, and what localization scores they report for those regions. The computer equipment 10 can choose to call BOMU for map updates when the filtered average (or minimum) localization score falls below a certain threshold. Alternatively, the computer equipment 10 repeatedly calls BOMU, for example at set intervals, but simply stores the results as long as the localization score is high. Calling BOMU continuously is an implementation-dependent detail, but is desirable to make it more computationally efficient, as each update process needs to operate on a smaller data set, i.e., a smaller batch of log data 22, and incorporate smaller changes. It should be noted that while the presented map update process is well suited for mobile industrial robots, the technique can also be applied to other frameworks such as drone networks, cellular networks with shared visual positioning systems (VPS), etc. Thus, the robot 16 can be broadly understood as a kind of end unit that provides log data 22.
[0064] In one or more embodiments, the map update framework disclosed herein uses a particle filter-based SLAM framework to enable the computing facility 10 to update the map 12 using log data 22 generated during operation of task phases of one or more robots 16 within the environment 14 represented by the map 12. The framework generates map updates that identify incremental and periodic changes in the environment and provide the most accurate map for the deployed fleet of robots 16 up until the time of generation of the map update using sensor observations recorded by the robots 16 operating within the environment 14. These sensor observations are captured in the log data 22 provided by each individual robot 16.
[0065] Understanding of the benefits derived from the disclosed technology is aided by viewing map updating as an "offline" or "parallel" process, meaning that the data processing and calculations performed to arrive at an updated version of the map 12 are performed in the computing facility 10, while the involved robots 16 continue to operate in the environment 14 using their current copies of the map 12. Because of this arrangement, contemplated map updates can take into account more data from multiple robots 16, over a longer time frame, using more sophisticated filtering and processing, as compared to conventional SLAM processing on a single robot.
[0066] Map regeneration and / or pushing of the latest map updates to the robot 16 occurs, for example, in response to the average / minimum localization quality dropping beyond a certain threshold. Each of the individual localization quality values is achieved by the individual robots 16 while operating within the environment 14 using their static copies of the map 12. Updating their local static copies of the map 12 based on the latest map updates determined by the computing facility 10 resets the localization quality substantially back to that of the newly generated offline map. Furthermore, these map updates are generated using log data 22 that the robot 16 generates while operating within the environment 14, rather than from a laborious manual process. As the robot 16 is expected to operate in a similar space further down the line, incorporating its observations into an updated version of the map 12 improves the robot's localization performance.
[0067] The overall map update framework can be viewed as a hybrid partitioning, where the robot 16 generates log data 22 during its “normal” operation within the environment 14, and the computing equipment 10 aggregates or otherwise collects the log data 22 for batch-based processing to determine the latest map updates.
[0068] Changes in the environment may be incremental. A typical example is when pallets or boxes move incrementally on the horizon for short or long periods of time with poor repeatability (these are usually moved manually and approximately aligned with floor markers). The localization performance of the robots 16 in the fleet may begin to degrade if the incremental changes become too large to ignore during the localization process performed by the individual robots 16 operating in the affected area. Using a map update process, the robot 16 can use structures to accommodate incremental changes built into the map update to keep the robot 16 well localized when these changes are not dramatic. Typically, one can expect the map update process to tolerate more than 100% changes to a fixed base map if the incremental changes between revisits to observe the changes are less than 30%.
[0069] Also, changes in the environment can be periodic. Such changes are typical of a warehouse or shipping center where, for example, in the morning (e.g., 9:00 a.m.), boxes are being prepared at the dock for shipping. By the afternoon (e.g., 4:00 p.m.), the shipping trucks are loaded, and the map of the environment 14 looks completely different than it did in the morning. The next day, the boxes are returned to the dock at 9:00 a.m. Such periodic map changes typically occur daily, or according to some other pattern. Similar long-term map change behavior and history can occur for gates, doors, and other structures in the environment 14. The map update process described herein can generate a multi-modal map representation, where each mode represents a characteristic configuration of the environment 14, and can characterize periodic changes in the map for optimal position estimation over the horizon throughout the day and longer. Thus, modality is an important attribute in characterizing environmental changes with respect to the map 12.
[0070] In other scenarios, the changes in the environment 14 may be totally dynamic. In these scenarios, the changing structure dominates the scene. An example of this domination occurs when the only structural features the robot 16 observes in a "dead zone" are movable objects (such as pallets in a warehouse), while a local static copy of the map 12 that the robot 16 uses for self-localization is created without any movable objects in the relevant area or region of the map 12. Traditionally, repeated remapping is required to localize the robot in such conditions. However, the map update process disclosed herein can operate from features from dynamic objects (movable objects) and thus can characterize such changes as part of the map. Thus, the resulting localization is robust to such changes as long as the changes are not dramatic and at least a minimum number of obstacles retain their pose between map updates.
[0071] As previously mentioned, the map update process takes the form of a decoupled process (i.e., updates are separated from the ongoing normal operation of the individual robots 16) and is driven by the BOMU executed by the computing facility 10. In at least one embodiment, the BOMU is a batch-optimized SLAM algorithm that uses logged data 22 from one or more robots 16 to generate an updated version of the map 12. In at least one embodiment, the logged data 22 from each robot 16 includes data from a Monte Carlo Localization (MCL) function executed on the robot 16, as well as environmental sensing data and odometry data. Data from the MCL function includes, for example, pose data calculated in dependence on the environmental sensing data and odometry data.
[0072] As used herein, the MCL function determines the location of the robot within the map frame. The MCL function can be substituted with other types of localization algorithms depending on the particular design and operational requirements being performed. In at least one embodiment, the self-localization in the robot 16 is a particle filter-based MCL and the batch optimization SLAM implemented by the computer facility 10 is a particle filter-based SLAM algorithm. In at least one embodiment, the particle filter-based SLAM algorithm implemented by the computer facility 10 is a Rao-Blackwellized particle filter algorithm. The localization operations in the robot 16 and the map update process in the computer facility 10 are not limited to the use of particle filters, although the particle filter-based implementation provides certain performance and quality advantages in at least some scenarios.
[0073] 9 is a state diagram of a map update process according to one embodiment. An "end unit" refers to essentially any type of robot 16 that uses a local static copy of the map 12 for self-localization within the environment 14 represented by the map 12. As can be seen, when the end unit's localization quality (or equivalently expressed herein as "location accuracy") degrades to such an extent that the float's operation is stopped or in imminent danger of being stopped, the computer equipment 10 invokes a map update from the BOMU. The computer equipment 10 runs the BOMU using a log file (log data 22) relating to the time and area within the environment 14 where the degradation in localization quality occurred.
[0074] Consider an example where two robots 16, denoted robots A and B, are operating in a three room environment. The corresponding map 12 has three regions, one in each room. In this example, during time period "t0", robot A is exclusively in region 1 and robot B is exclusively in region 3. During time period "t1", robot A is in regions 1 and 2, and robot B is in regions 3 and 2. Robot A repeatedly achieves a localization score of 0.5 and robot B achieves a localization score of 1.0. Thus, the average localization score for the first region is 0.5 (only due to robot A) and the average localization score for the third region is 1.0 (only due to robot B). However, the average localization score for region 2 may be calculated to be 0.75 (the time-weighted average of robots A and B). The set of log files for each region of the map 12 is composed of log data 22 from the robots 16 located in that region. It should be noted, for example, that a map update for region 2 does not include files from time period "t0".
[0075] Map updates may be triggered using the localization quality average or using individual localization qualities (scores) for each robot 16. As previously mentioned, the logged data 22 from each robot 16 includes at least sensor data logged by the robot 16 from one or more types of sensors used by the robot 16. The sensor data may include odometry data used by the robot 16 in localizing itself. Additionally, the logged data 22 may include localization process results, e.g., output generated by the robot's localization algorithm using the sensor data and odometry data.
[0076] The map update process performed to generate a new version of the map 12 can take time, however, as previously mentioned, one advantage is that the robots 16 can continue normal operation in the environment 14 using their current copies of the map 12 while the computing facility 10 determines the map updates.
[0077] Once the map update is complete and an updated version of the map 12 is generated (e.g., a new map file is generated), the results may be pushed to the robot 16 or, in at least one embodiment, presented to a human operator to evaluate whether the updated version of the map 12 is of sufficient quality. In this case, the operator has four options: (1) the operator determines that the map quality is insufficient for deployment to the fleet and rejects the map in favor of performing a manual remapping process; (2) the operator determines that the map quality is insufficient for deployment to the fleet and rejects the map, with the fleet continuing to operate and a new map being generated with newer log data 22; (3) the operator determines that the map quality is sufficient but does not contain enough information to justify the fleet stopping to send a map update, meaning that the changes to the map 12 are deferred to the next update; or (4) the operator determines that the map quality is sufficient and the changes contained therein require immediate transmission to the fleet. That is, the fleet of robots 16 is stopped, all robots 16 are updated with a new version of the map 12 , and then resume performing tasks within the environment 14 using the updated version of the map 12 .
[0078] 10 is a functional systems diagram of a framework in one embodiment for supporting the map update process. The "Sensor Interface and Log" block contains the robot software responsible for collecting relevant sensor and localization state data from real-time localization (which primarily drives the functionality of the robot 16 during task phase operations), writing it to a log file, compressing it, and making it available for collection by the computing equipment 10 through the communication interface.
[0079] The real-time localization block (function) of the exemplary robot 16 represents a localization algorithm that provides a pose of the robot 16 relative to a static copy of the map 12 that is being used by the robot 16 to localize itself within the environment 14 represented by the map 12. In one or more embodiments, the robot 16 implements this functionality using MCL functions. In at least one embodiment, the pose determinations, as well as internal algorithm state information, constitute all or part of the logged data 22 that is provided to the computing equipment 10 for performing the map update process.
[0080] The "Server Interface" block in the computing equipment 10 represents functionality that interacts with the underlying network stack running on the computing equipment 10 or on an associated computing system that functions as a "fleet server" responsible for dispatching robots 16 for the performance of tasks. In either case, the computing equipment 10 receives or obtains log data 22 directly or indirectly from the robots 16 and pushes MUI 24 generated by the Full SLAM BOMU directly or indirectly to the robots 16.
[0081] The "Localization / Map Quality Detection Interface" block in the computer facility 10 determines the average / minimum localization quality from the values for each individual robot 16 and its position in the map 12, and in one or more embodiments uses such determination results to check whether a map update is required.
[0082] The “Scan, Interface for Collecting Sensor Logs” block in the computer equipment 10 represents one or more functions implemented in the computer equipment 10, whereby the computer equipment 10 is operable to acquire, organize, and process log data 22 from the robot 16.
[0083] The block “Map Update / Push Interface” in the computer facility 10 represents the function that is called to push the MUI 24 to the robot 16 .
[0084] The "User Selection / Acceptance" block in the computer equipment 10 represents functionality in the computer equipment 10 whereby the computer equipment allows user intervention to accept or reject updates to the map 12. The user can also select the modality of the generated map for deployment. This functionality also allows the user to select the map update area and / or to ignore redundant modes.
[0085] The "Multimodal Full SLAM BOMU" block in the computing facility represents the core map update functionality according to one or more embodiments. In at least one embodiment, this block is implemented as a Rao-Blackwellized particle filter with global optimization that performs a full SLAM update to determine a new version of the map 12 from currently available log data 22. This block presents a number of distinctions over traditional SLAM, even traditional SLAM based on particle filters.
[0086] First, the blocks are configured to run a version of full SLAM configured for multimodal map generation and map management, providing the following types of updates: (a) a single map update, where only one map mode is generated, reflecting the assumption that changes in the environment 14 are permanently embedded and do not change periodically according to the requirements of the application (this operation can accommodate incremental changes); (b) a multimodal map update, where the map update process generates multiple updated versions of the map 12, where each "version" represents a different mode of the environment 14. In this way, the MUI 24 pushed to any given robot 16 may include a multimodal update, meaning that the robot 16 receives updated versions of the map 12 for multiple modes of the environment 14. The local static copy of the map 12 updated by the robot 16 using the MUI 24 includes two or more multimodal maps, where the robot 16 uses the modal map that corresponds to the relevant environment mode (e.g., time of day, day of week, etc.).
[0087] Moreover, the BOMU implemented by the computing facility 10 provides further points of distinction, such as: (1) being operable to save and load the internal particle state (including the pose along the robot trajectory and the probabilistic occupancy grid of each particle) to an existing map file, thereby preserving the map update state across runs; and (2) implementing a hybrid process that includes (i) a separated SLAM process, where localization occurs online within the robot 16 and map updates occur offline (i.e., outside the robot 16 and in parallel or independent of the robot 16 performing ongoing task operations in the environment 14), and (ii) a combined SLAM process, where both occur simultaneously in an offline process. In a conventional SLAM pipeline, the world coordinate system (associated with the particles) of the current best map can deviate significantly from that of the base map. This is due to the inherent nature of SLAM, where errors or drifts in localization appear as deviations of map coordinates from world coordinates, which results in further deviations or drifts in the estimate of localization from world coordinates. To counter this, MCL particles can be used. However, MCL particles are generally noisy and bouncy, which prevents smooth trajectory estimation, which is important for clean map updates. Moreover, since the robot in the task phase is usually operating close to its maximum speed, the state estimates of MCL particles are sparse and contain a wide range of instantaneous / transient errors. Thus, a "map using known pose" approach is infeasible. For this reason, we use a hybrid motion model for SLAM-based map updates that combines cloned MCL particles (which have low drift and world coordinate errors, but are noisy and bouncy) and independently evolving particles (which are smooth and locally accurate, but have drift).
[0088] Such BOMU functionality appears in Figures 11A and 11B, which together represent an example of a batch optimization SLAM implementation of BOMU in a computing facility 10. Notable functional blocks appear in Figures 11A and 11B, outlining a particle filter-based map update process supported by a global optimization of estimated trajectories from each robot 16 to generate an updated version of the map 12. The functionality of the core map update blocks in the context of the full SLAM pipeline is described below. It should be noted that in terms of software design, the framework can be integrated to run on a computing facility 10 running as a separate map updater server on an independent PC / IPC, Edge instance, or in the cloud, allowing for scalability and flexibility in the deployment of the framework.
[0089] A particularly noteworthy feature of the BOMU in this embodiment is the particle evolution model that depends on the motion model. While the motion model for particle evolution in a typical particle filter SLAM pipeline uses an odometry-based motion model, in BOMU, a hybrid approach is used. In this specification, a novel hybrid of a "separate map update process" and a "joint simultaneous localization and map update process" is adopted as the state estimation process. The "joint process" estimates the localization step and the map update step iteratively and simultaneously offline, while the "unjoint process" separates the localization step and the map update step, and the robot 16 performs localization using the MCL particle filter in the online phase as part of the task phase operation of the robot 16, and the map update is performed in the offline phase in the computer facility 10. In effect, this approach is neither an online SLAM version nor an offline SLAM version, but a hybrid map update process, where only the joint process is simultaneous.
[0090] Thus, particle evolution follows either an independent (coupled process; has its own resampling / resetting process) or parallel (uncoupled process; cloning of MCL particle evolution) fate. As mentioned before, the hybrid evolution model (and corresponding kinematic model) is the key to handle map updates using logged data 22 from localization conditions of various task phases while maintaining the global consistency and world-frame groundedness of the map 12 being updated. While cloned particles keep particle diversity low, independent particles increase diversity, and particle management keeps a balance between the two while constraining the particles. This model is suitable for single robot map update processing while also being applicable to multiple robots.
[0091] Traditionally, online SLAM particles distribute the error between the localization and map update processes, and every drift in the localization causes the generated map to deviate from the world coordinates, resulting in compounding errors and drifting of fixed localization estimates, such as goals, which are crucial in industrial environments. Such goals need to be located with high fidelity. On the other hand, MCL particles, which refer to a fixed world coordinate map, are ideal for map updates, as they do not drift and can maintain fixed landmarks, such as goals. However, all errors in the process (from sensor noise or driving errors) are reflected in the localization. MCL particles and their corresponding localization estimates can be noisy and variable, increasing the incidence of sudden and potentially irrecoverable localization losses or degradations, hindering the map update process. Therefore, some independent evolution of the particles with optimization of particle state based on sensor observations (as is done with SLAM particles) is necessary.
[0092] Another important deviation from the standard single-robot online SLAM pipeline is the introduction of a "Trajectory Closure Module" in the BOMU used by the computing facility 10. This module is responsible for managing sensor streams from multiple robots 16 that overlap in terms of observations (and observation times) and may or may not be inconsistent. These inconsistencies can be due to noise in the localization estimates (as a function of differences in sensor noise models at each robot 16 and, more importantly, MCL state noise / errors from non-optimal scan matching). Here, "scan matching" refers to the degree of correlation between the sensed environment as represented by the robots' sensor data and the local static copy of the map 12 used by the robots 16.
[0093] Simply put, scan matching is an assessment of how well the robot's sensor data at any given moment matches a portion of map 12. For example, the robot's sensor data may include a "point cloud" obtained from a 2D or 3D LIDAR scan that allows the robot 16 to detect obstacles and openings, such as wall and door features, and these detected features are compared to the configuration of features captured in the robot's local static copy of map 12.
[0094] Also, discrepancies in the log data 22 from each robot 16 arise from changes in the environment 14 over time. The map update process is heavily weighted in favor of recursion (in observations, i.e., the latest changes must have a large impact on the output map compared to older observations), which is characteristic of gradual and globally dynamic changes. It is therefore desirable to apply the sensor logs from multiple robots 16 to the map update process in the order (time) of their observations. However, it is also desirable to maintain trajectory consistency (and support loop closure). For this reason, it is desirable to preserve the evolution of particles over time steps of sensor data from a single robot. These requirements are therefore complementary.
[0095] Also, the frequency in the observations is an important consideration, but this is explicitly handled by the Hidden Markov Model (HMM) module. The trajectory closure module handles discrepancies in time and noise variance and fuses the observations by computing the boundaries of the particle evolution process to close the trajectory. However, a trajectory can only be closed at a stable point where no potential loop closure or particle divergence occurs. This is achieved by determining the deviation between the bound and unbound states (along with the peak localization score and modality count reported by the unbound state) and allowing trajectory closure when the two sets of states match. Given a trajectory closure corresponding to a particle state from the observations of one robot, observations from a second robot that overlap in time and observation sequence are used to update the map along this trajectory before closing this trajectory and switching to the observation sequence from the other robot. Finally, the observation sequence from the first robot that is later in time than the first trajectory is incorporated into the map update process.
[0096] Thus, trajectory closure maintains the consistency of observation trajectories by closing trajectories from multiple robots based on certain constraints, while ensuring causality and recursion of map update operations. In effect, the trajectory closure module balances "trajectory consistency" against "recursion / causality" of observations.
[0097] Some of the considerations detailed in the previous paragraph in terms of inconsistencies between map update observations (which can potentially result in artifacts such as broken linear structures (e.g., walls)) are further mitigated by the multi-robot state fusion / global optimization and map refinement modules. For the global optimization process, relative pose factors can be used to smooth divergent pixel occupancy observations from multiple robots 16 with a non-linear optimization process to converge position estimates of the divergent observations. Additionally, the map refinement process can improve the consistency of observations from multiple robots by refining pixel occupancies into line segments and filling in discontinuities or gaps in the observations. These modules are described in more detail below as part of an integrated system.
[0098] Initialize the map and system state
[0099] The basic assumption in the case of a typical or conventional online SLAM pipeline is an unknown world with a fixed origin that provides the start of the online mapping process. Conversely, the envisaged map update requires that the SLAM-based update process (and therefore the SLAM state) be initialized with respect to a known representation (map) of the world already deployed on the robot 16 for the task phase operation (with localization). Thus, the BOMU in the computing facility 10 includes a module or function responsible for converting the representation of the world (metric map) into an occupancy grid in the SLAM coordinate space. It should be noted that while the map (of the world space) is usually represented in a plot coordinate system, the occupancy grid uses a binned image coordinate system (thus flipping the vertical axis and relocating the origin). Furthermore, the occupancy grid used in the SLAM pipeline maintains a symmetric axis span for efficient memory management. The map initialization process converts the probabilistic free space grid map (fixed base map) generated by a typical SLAM-based mapper into an occupancy grid and uses this map to initialize the map states of all particles of the state estimation filter. This module also reinitializes the system state variables that hold the state of the SLAM process between instances of map updates.
[0100] Initialization and cloning of particle states
[0101] As mentioned before, the map updater uses a hybrid motion model for SLAM-based map updates. This module, along with the particle evolution module, is the biggest deviation from a typical SLAM pipeline. In addition to the best estimated pose of the robot defined by the MCL localization in the task phase, the hybrid model consumes the MCL non-coupled localization particle state. An interface is built into the MCL in the robot 16 to support this operation, and a state logger records this information in a log file for processing by the computer facility 10. It should be noted that this includes all variables related to the MCL particle state (mainly the robot's estimated pose (position and orientation) for each particle, best among particles, particle distribution information, peaks, localization scores, likelihoods, probabilities, laser scan data used in particle fit calculations, etc.). This module loads and initializes the hybrid particle filter. Corresponding to the coupled and non-coupled update processes, the motion model for particle evolution uses an independent odometry-based motion model (similar to online SLAM) and a parallel evolution model, respectively.
[0102] MCL particles are much more numerous (typically several thousand since the process of computing MCL particle states involves only validation of localization states, which is cheap and therefore can be computed with few resources), but these particles are tightly clustered around the odometry estimate, proving to be suitable for precision navigation states. They also lack diversity due to aggressive resampling. On the other hand, independent SLAM particles are fewer in number (typically several tens since the process of computing SLAM particle states involves expensive optimization of localization / pose followed by map updates), but allow for a larger spread and deviation (to accommodate errors in localization and jumps in map coordinate updates).
[0103] Furthermore, a null model is used to handle certain use cases, as described below. Specifically, the best scan-synchronized particle estimate from the MCL is cloned into the particles of the parallel evolution model, along with its probability score and likelihood. Similarly, the limited best set of resampled particles that form the peaks of the MCL particle distribution are cloned into additional particles of the parallel evolution model. Finally, this is augmented by additional particles that correspond to the peaks of the resampled MCL, but are distributed further away from the central peak. This is done to improve particle diversity. As can be seen from FIG. 12, the diversity and style of distribution of MCL particles in typical movements is lower with a fixed map (precision must be considered). This is especially true in cases where relocalization would be successful without the loop-closing movement. The odometry-based motion model helps to avoid such situations through independent evolution and a coupled localization and map update process. It uses the best estimate of the robot's pose by the MCL (scan-matching estimate or odometry interpolation) simultaneously with the coupled motion model to advance the particles. It is also seen that the MCL particles exhibit discrete jumps.
[0104] Referring to FIG. 12, a simplified diagram of particle evolution during task phase position estimation in the robot 16 is depicted. Leading particles are weighted according to probability, but in this diagram no attempt is made to depict the probability weighting, nor the actual number of particles. Trailing particles represent the state of the particles after scan matching correction, and the peaks in the distribution are resampled peaks. The diagram also shows the interpolated odometry corrected best pose. Panel A shows the trajectory of the robot 16 moving from top to bottom in a 20m x 20m space in dashed lines. Panels B and C show a "zoomed-in" view of the particle state. Note that (i) leading particles lag behind trailing particles, and the particle spread is largely oriented in the direction of motion during translational motion (top half of plot); and (ii) the particle spread is more circular with ambiguous orientation due to poor rotational odometry, but is not lagging (center of plot). (iii) The distribution of particles throughout both movements is typically unimodal and not diverse due to active resampling. (iv) The reported best pose becomes the particle's pose by interpolation. All or part of the particle data depicted in Figure 12 is recorded in the robot's 16 log data 22 as "position estimate data" and reported to the computing facility 10 for use in its map update operations.
[0105] Particle evolution and management for map updates
[0106] As mentioned above, the evolution of particles following the independent model uses an odometry-based coupled motion model. Furthermore, this motion model is modulated by a null model for a specific use case. During sections of the robot trajectory along a featureless corridor (environment with no bounded heading), the trailing particles (after observation, i.e., scan matching) may diverge significantly along the direction of motion and in azimuth to this direction. Also, in sections where there are no structures on one side of the robot, the particles may diverge from the side with observable structures. To limit such evolution, a null motion model is used and odometry is applied directly to particles whose covariance state is frozen to the state just before encountering such environmental structures. In general, the evolution and management (resampling / killing / resetting) operations of particles following the independent evolution model are similar to a general particle filter SLAM pipeline. In the case of the parallel evolution model, the particles copy the evolution and management operations of the MCL particles and also copy the resampling operations. The result is a healthy pool of particles at each generation or iteration step that is both diverse (to accommodate sensor and actuation limitations) and accurate (to preserve the true world coordinate reference). New position estimates for independently evolved particles are computed by the coupled process (independent evolution model), while the position estimates for MCL particles are copied by the uncoupled process for clone particles. Each particle in both the coupled and uncoupled pools then updates its map. The map update operation prioritizes rechecking (via weight parameters) and clearing occupied cells to accommodate incremental and global dynamic changes in the task phase. This differs from the map update process in typical SLAM pipelines, which retains cell occupancy observations as long as possible while clearing unoccupied cells with low likelihood. This aggressive updating of map cells ensures that the updated map is close to the current environment state. The final map for deployment is selected from the maps updated and stored by each particle based on a best particle decision, similar to traditional online SLAM. Furthermore, only stable and persistent features are added to the updated map.These are features observed over several frames / beams, are statistically significant, and survive recursion-based vanishing of change points.
[0107] Orbital closure
[0108] As mentioned before, the trajectory closure module maintains the consistency of observation trajectories by closing trajectories from multiple robots based on certain constraints while ensuring causality and repeatability of map update operations. The two elements of trajectory consistency and causality and repeatability are contrasting goals. The trajectory closure module is primarily responsible for managing sensor observations for updates (from multiple robots) that may conflict in time. It manages these conflicts by splitting the particle evolution process into discrete chunks. In other words, the trajectory defined by the sequence of SLAM particle evolution steps and corresponding localization and map update steps is finalized and terminated (thus preserving trajectory consistency) before potentially overlapping trajectories (in space and time) from other robots and their corresponding observations are processed by the particle filter for map updates. Once trajectory closure is completed for the second robot, the process is handed back to the remainder of the trajectory of the first robot or to the third robot for updates (thus preserving causality) based on which robot's raw readings are earlier. "Causality" means that earlier sensor observations are applied to the map update process before applying later sensor observations, so that the map update process results in a map that is closer to the state of the environment from the current time rather than some time in the past. It is necessary to perform closure to ensure that changes in the map in time are applied to the map update process in a "causal" way, and that any desired "recursive" behavior in the map update is maintained. Thus, violations of "causality" and "recursiveness" are minimized.
[0109] Maintaining the "recursion" priority is essential to handle gradual and globally dynamic changes to the environment. Periodic changes (as characterized by "frequency" behavior) are handled by the transition probabilities of the HMM. The trajectory closure process is significantly different from traditional continuous / online SLAM, which processes all trajectories sequentially. Thus, in traditional on-line SLAM, sensor observations from the first robot (ordered by "causality") are applied to update the map along the robot's trajectory. This is followed by sequential updates using trajectory data from the second robot, followed by trajectory data from the third robot, and so on. Thus, such traditional SLAM methods violate the criteria of "causality" and "recursion" since the data read from the second robot is applied to the map update process after the data read from the first robot. However, "trajectory consistency" is preserved.
[0110] Besides temporal mismatch, discrepancies in environment occupancy observations can arise from errors / noise in position estimation that vary as a function of differences in sensor noise models, and non-optimal scan matching (formulated as an optimization problem that does not have a closed-form solution), resulting in smeared and unconfident map updates. Frequent orbit closures and switching sensor streams are important to mitigate artifacts from both error sources.
[0111] The question remains, how are trajectory closure instances determined? The trajectory closure module identifies closure instances by computing the boundaries of the particle evolution process. We show that if the particle is multimodal, the trajectory cannot be closed and loop closure may occur within the span of the trajectory further along in time as successive readings in the trajectory are processed. We also show that if the particle evolution is divergent (the distribution of particles is spread out), the trajectory cannot be closed and resampling or switching the best particle index may be necessary. Thus, this module tracks the deviation between the bound and unbound states as reported by the unbound state, the overall diversity of all particles (more diversity indicates a less ideal state), large entropy changes between iterations (i.e., more change in the map per map update step), and the length of time since resampling (the longer the elapsed time, the more stable the trajectory is and the more suitable it is for closure), along with the localization score (the worse the localization score is due to a larger mismatch between the observed environment and the map, the lower the ideality of closure), and the peak modality count (more modes indicate a larger deviation of the possible true assumptions, the lower the ideality of closure), and determines a stable point of convergence of the two sets of states and overall particle agreement. If enough time has passed and these necessary conditions are met, trajectory closure is initiated.
[0112] It should be noted that these conditions ensure "trajectory consistency and completeness" (longer trajectories have more consistency). As mentioned before, this is in contrast to "causality and repeatability" which is maintained when trajectories are shorter. There is therefore a trade-off between the two, and by ensuring that trajectory closure and switching only occurs when the aforementioned conditions are met, an appropriate balance between the two is maintained. The trajectory closure module is also responsible for terminating a trajectory when the robot leaves a sector tagged as a "map update region" (an area that can be specified by the user to limit the area of the map where changes are expected and need to be updated).
[0113] Multi-robot state fusion, trajectory reordering and global optimization
[0114] While trajectory closure can largely mitigate artifacts that are primarily due to inconsistent "causal" observations between the robots, inaccurate MCL position estimation, system errors between the robot sensors, and time synchronization errors can cause blurring artifacts in the generated maps. The probabilistic nature of the map representation and the probabilistic map cleaning and map refinement stages can reduce these errors to some extent, but significant errors can still occur, especially during high speed robot movements. To this extent, a global optimization process based on factor graph optimization is used to handle inconsistent observations.
[0115] Global pose graph optimization employing factor graphs and operating on data from a single robot is popular (e.g., GT-SAM). It helps to handle and resolve different estimates of pose measured by wheel encoders, IMUs, and laser observations. A factor graph is a representation of a Bayesian network or a hidden Markov model (HMM) over multiple pose steps along the robot's trajectory. We propose a novel approach and application of such factor graphs by extending their structure and resolving mismatches between multiple robots.
[0116] As in traditional factor graphs, each node is a conditional density. The Markov chain encodes the previous robot state p(x1) and the transition probabilities p(x2|x1), p(x3|x2), and p(x4|x3) defined by the odometry, while the measurements z t (e.g., measurements from a laser scanner) are expressed as a function of the conditional density p(z t |x t ) state x tGiven known measurements z'1,z'2,z'3,z'4, the robot's true pose-hidden state sequence (x1,x2,x3,x4) is estimated to maximize the posterior probability p(x1,x2,x3,x4|z1=z'1,z2=z'2,z3=z'3,z4=z'4). The measurement variables z1,z2,z3,z4 (z1 is the characteristic measurement z 11 and z 12 , the posterior probability is proportional to the product of six factors, three of which come from the Markov chain, and three likelihood factors,
number
number
[0117] Now, in the extended structure, there are additional observations from a second robot (or robots), which are expressed in terms of their pose hidden state variables (y1,y2,y3,y4). The non-matching occupied pixel clusters from the multiple robot observations are added as factors in the factor graph formulation in the form of the relative pose factor of the second robot (shown in grey) and the closest trajectory pose variable of the first robot's trajectory. Thus, state variable y1 is connected to variables x1,x2 through a relative pose factor based on the triangular difference of the measurements z1,z2 of the two robots. The updated posterior probability is then:
number
[0118] The maximum a posteriori (MAP) estimation or estimator estimates the pose parameters with the highest density of a posteriori probabilities. Assuming independence between the factors, this is expressed as the product,
number
number
[0119] This factor graph representation is solved as a pose optimization problem that optimizes / smooths only the pose of the robot, and the final map is estimated from the smoothed trajectory. Optimization over the pose graph requires two components. The first component is the ability to estimate the factors (implemented with linearized factors) and the geovians / partial derivatives over the SE(2) pose group. The second component is a nonlinear optimization process (implemented with the Levenberg-Marquardt algorithm) that maximizes the posterior density. The result is reduced discrepancies between observations and less blurring of structures in the final map.
[0120] Multimodal map generation using periodic behavior (stationary distribution and mixing time estimation)
[0121] As mentioned above, in a typical industrial environment, it is expected that multiple maps will be required to capture the environmental characteristics at different points in time. This is due to the inherently cyclic nature of the changes that are typical in industrial environments. Therefore, multiple map representation modes are essential. These map modes emerge based on the underlying structure in the data observations. To capture these modalities, the map update relies on the innovative use of Hidden Markov Models (HMMs) to represent the temporal characteristics of each occupied grid cell. Traditionally, the use of HMMs in SLAM has been in online localization and mapping processes to the extent of tracking deviations from a base map. Thus, one difference in the use of HMMs described herein is that they are used by the robot 16 in performing self-localization, offline, after generating maps of multiple modes. During the envisioned map update process within the SLAM pipeline implemented in the computer facility 10, the HMM tracks the map occupancy distribution. When the distributions converge periodically (as the robot re-observes the modes in the multi-modal changes of the environment), a "stationary distribution" is generated. That is, the distribution of each cell of the map converges to a particular probability with trivial changes in the probability due to additional observations in the time window. Note that in this scenario, the observations are windowed (or truncated) when computing the distribution.
[0122] Thus, stationary distributions represent cases where the environment looks similar to what was previously observed. Observations of any stationary distribution may occur multiple times. There may also be multiple observed stationary distributions. The number of stationary distributions defines the modality of the map representation. In this case, the stationary distributions are calculated from transition probabilities (obtained from stochastic transitions between occupied and free space). While memory usage during the construction of an updated map may be high due to the storage requirements of the HMM cell distributions, this can be kept low in practice by culling out non-stationary distributions, including past distributions that fit the current model of the stationary distributions, and all other observations that do not represent significant changes in the environment.
[0123] In addition to identifying stationary distributions, we also identify the "mixing time" of the HMM. Mixing time is defined (in this context) as the time required to converge from one stationary distribution to another. It is expressed in the HMM as the number of steps required for each occupied cell to move from the first stationary distribution to the next. These steps include when the cell is observed (depending on the robot's position in the environment) and when it is not observed. We compare two cell distributions to determine whether they are similar enough to be classified as stationary (regularly observed) distributions, and use the total variation distance to estimate the completion of the transition from the first stationary distribution to the next stationary distribution. The mixing time is the minimum time that minimizes the distance between the two stationary distributions. The mixing time is also used to sift through the past history of the cell distributions to minimize memory usage. Additionally, the time between observations of the same stationary distribution is tracked.
[0124] From these stationary distributions of map cell occupancies and their corresponding mixing times, modes of the map representation are generated and timing priorities for the deployment of the modes are determined. Note that the timing priorities (based on the mixing times) indicate the recommended timings for the localization to deploy the map modes (during the localization of the task phase with a given multimodal map). It is also possible for the localization of the task phase to localize the robot based on all available modes and determine the best mode to use at any given time. Such an implementation is computationally expensive but is more robust to changing periodicities of the map environment changes. Also, the total number of map modes may be limited to improve computational efficiency.
[0125] An example of a case where a multimodal map representation based on timing priorities is useful is shown in FIG. 13. This example shows the case of a warehouse loading dock. From the analysis of the scanner observations of the environment 14, it can be seen that there are three map modes corresponding to a closed loading dock roll-up door (mode A in FIG. 13), an open door (mode B in FIG. 13), and an additional mode where the open loading dock roll-up door leads to the view of a parked transport truck (mode C in FIG. 13). This (third) mode is typically observed in sufficient time and with good periodicity by one or more robots 16 in a typical warehouse with a large amount of cargo entering and leaving every day, but is less deterministic than the other two because the cargo may be observed progressively occluding the model's field of view during the loading / unloading process. Once these modes are calculated, they are stored between runs of the SLAM pipeline to allow further map expansion. The multimodal HMM modeling implemented by the computer facility 10 for BOMU allows the detection of these modes.
[0126] System state freeze
[0127] The system state free module is responsible for finalizing the SLAM map update process and logging / serializing the state of the update process in terms of the probabilistic occupancy grid data and trajectory data of the particles. However, considering the hybrid particle evolution model, the reinitialization of the SLAM map update process does not require the complete state of all particles. Preserving the state of the probabilistic occupancy grid and HMM is necessary to ensure continuity between the map update invocations and to correctly model the "recursion" effect. The trajectory data is useful for performing loop closure operations in the reinitialization of the map update process during the robot motion in the task phase when moving through large open areas without features (especially when using manual or line / path following systems). The trajectory data is also useful for performing additional global optimizations and for inserting / editing manual landmarks (such as goals) in post-processing steps after the map update.
[0128] Map landmarks and metadata conversion
[0129] The map landmark and metadata transformation module is responsible for preserving and transforming the landmarks and other points of interest present in the original version of the map 12 and used in the task phase operations into the new updated map. The transformation is straightforward since the world coordinate systems of the current version of the map 12 and the updated version of the map 12 are preserved (after transformation to the SLAM occupancy grid coordinate system) for most use cases. However, goals and other traversable landmarks located in occupied grid cells that are currently considered occupied are removed from the final updated version of the map 12. This removal ensures that there are no unreachable goals or any possible collisions when the updated map 12 is deployed to the robot 16. Furthermore, in exceptional cases where a section within an occupied grid cell has been deformed (stretched), the affected landmarks are updated appropriately (by computing a projective trajectory transformation) as a result of computing a better estimate for the metric world map position as part of the map update process.
[0130] Similarly, in addition to goals, other landmarks for docks, sectors, zones, etc. are also converted into the final map (as shown in FIG. 14). It should be noted that map regions around docks, goals, or other areas where robots or other mobile units (palletizers, pallet jacks, forklifts, etc.) may be parked for extended periods of time may also be added to the map. At this point, the user is expected to clean the map 12 of such structures before deploying it to the robot 16.
[0131] More specifically, FIG. 14 shows an example of the results of BOMU with landmark transformation. An initial or current version of the map 12 appears on the left, with the goals and docks set for the robot's task phase operation. On the right, an updated version of the map 12 is displayed, with updates determined using (historical) log data 22 collected from the robot 16 during normal live operation of the robot 16 in the environment 14 using a previous version of the map 12. During such task phase operation (in this specific example), the robot 16 patrols between three goals while observing additional structures that surround the robot's operating space and are positioned at various orientations. These are progressively modified during the task phase operation. The updated map on the right shows that these structures (hatched) have been inserted into the map space. Sections of the map not observed by the robot during the task phase operation retain the original map occupancy information. Note that the locations of the goals and docks have been transformed into the updated map, and the non-traversable goal (G2) has been removed from the updated map.
[0132] With the above examples in mind, this specification broadly discloses a method and apparatus for autonomously and centrally updating a map during a task phase of a mobile robot's deployment. The method, in one or more embodiments, includes any one of the following steps: incorporating dynamic changes (incremental, periodic, and globally dynamic) into a fixed map; automatically extracting modalities (counts) of dynamic change distributions from sensor logs and map transition operations as implemented using an innovative HMM-based stationarity estimation process; fusing states from task phase operations and offline computations for a hybrid state estimation resulting in an updated map, which is designed as a novel hybrid of a “decoupled map update process” and a “coupled simultaneous localization and map update process” (wherein the “coupled process” iteratively and simultaneously estimates localization and map updates offline, whereas the decoupled process separates localization and map updates and performs localization in an online phase as part of the robot's task phase operations (implemented as an MCL particle filter, with map updates occurring in the offline phase)); fusing sensor streams from multiple robots whose observations (specifically, observation times) may overlap and potentially be inconsistent (such operations rely on an innovative trajectory closure module); and resolving incompatibilities between sensor observations from multiple robots through grid cell landmark estimation and global optimization.
[0133] According to some embodiments herein, the map update process includes a system 8 comprising a computer facility 10, such as a computer server, and one or more robots 16. The robots 16 contribute to the proposed map update process by performing self-localization within the environment 14 using a local static copy of the map 12 of the environment 14. Log data 22 generated by each robot 16 while operating within the environment 14 is provided to the computer facility 10. The data includes, for example, particles generated by each robot using a particle filter-based Monte Carlo localization (MCL) function and the sensor data used to generate those particles. Advantageously, the computer facility 10 uses this historical log data 22 to execute a SLAM algorithm that is optimized for batch processing of collected data (and taking into account sensing / localization errors between the robots 16), changes in the mode of the environment 14, and different trajectories of multiple robots 16 traversing the same map area.
[0134] Since the computational and memory burden for the map updates is on the computing facility 10 and the computations for the map updates are performed by the computing facility 10 while the robots 16 can continue their normal task operations within the environment 14, the contemplated approach is not subject to the complexity and time constraints that apply in the conventional case where simultaneous localization and mapping must be performed in real-time or near real-time as each individual robot operates within the environment 14. As a result, by using log data 22 spanning a longer period of time or relating to different modes of the environment 14 and by using / recalibrating log data 22 from multiple robots 16, the quality of the map updates can be significantly improved, thereby mitigating robot-specific errors in sensing and odometry.
[0135] It should be noted that modifications and other embodiments of the disclosed invention will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings.
[0136] Among the notable items disclosed herein are the following: 1. A method and apparatus for autonomously and centrally updating maps during the task phase of a mobile robot's deployment. 2. Methods and procedures for incorporating dynamic changes (incremental, cyclical, and globally dynamic) into a fixed map. 3. Method and procedure for automatically extracting the modality (counts) of the distribution of dynamic changes from sensor logs and map transition behavior. This is implemented as a novel HMM-based stationarity estimation process. 4. A method and procedure for hybrid state estimation that fuses states from task phase operations and offline computations to result in an updated map. It is designed as a new hybrid of the "decoupled map update process" and the "combined simultaneous localization and map update process." The "combined process" iteratively and simultaneously estimates localization and map updates offline, while the "decoupled process" separates localization and map updates, with localization occurring in the online phase as part of the robot's task phase operations implemented as an MCL particle filter, and map updates occurring in the offline phase. 5. Methods and procedures for fusing sensor streams from multiple robots with overlapping and potentially conflicting observations (especially in time), which are designed as a novel track closure module. 6. Methods and procedures for resolving incompatibilities between sensor observations from multiple robots through grid cell landmark estimation and global optimization.
[0137] Embodiment
[0138] 1. A computer system comprising: (a) a communications interface 30; and (b) a processing circuit 32, The computer equipment 10 includes a processing circuit 32 configured to (i) receive, via the communication interface 30, log data 22 generated by individual robots 16 that use a static copy of the map 12 to localize themselves within the environment 14, and (ii) execute a SLAM (simultaneous localization and mapping) algorithm using the received log data 22 to update the map 12.
[0139] 2. The computer equipment 10 of embodiment 1, wherein the processing circuit (32) is configured to perform updates on a trigger basis.
[0140] 3. The computer equipment 10 of embodiment 2, wherein the trigger basis is a periodic basis, and the processing circuit 32 is configured to repeatedly perform the update according to a defined interval.
[0141] 4. The computer equipment 10 of embodiment 3, wherein the processing circuitry 32 is configured to adjust the defined interval depending on the extent or nature of map changes determined by the computer equipment 10 in one or more of the repeated updates.
[0142] 5. The computer equipment 10 of embodiment 2, wherein the trigger basis is map quality based, and the processing circuit 32 is configured to perform the update in response to determining that the quality of the map 12 has deteriorated below a threshold.
[0143] 6. A computer equipment 10 as described in any of embodiments 1-5, wherein the processing circuitry 32 is configured to generate an alert indicating the execution of an update or to indicate the availability of an updated version of the map 12 as a result of the update.
[0144] 7. A computer equipment 10 as described in any of embodiments 1-6, wherein the processing circuitry 32 is configured to initiate transmission of map update information to individual robots 16 for use by each robot 16 in updating a local static copy of the map 12, the map update information including an updated version of the map 12 or change data indicating an update to the map 12.
[0145] 8. The computer equipment 10 of embodiment 7, wherein the processing circuit 32 is configured to initiate transmission of the map update information in response to the computer equipment 10 receiving an input signal indicating authorization to initiate transmission.
[0146] 9. The computer equipment 10 according to any one of embodiments 1-8, wherein the communication interface 30 is a wireless communication interface configured to receive the log data 22 wirelessly.
[0147] 10. A computer facility 10 according to any of embodiments 1-9, wherein the log data 22 from each robot 16 includes sensor data from one or more types of sensors used by the robot 16.
[0148] 11. The computer equipment of embodiment 10, wherein the sensor data includes odometry data.
[0149] 12. A computer facility 10 as described in embodiment 10 or 11, wherein the logged data 22 from each robot 16 includes particle filter data generated by each robot 16 using a Monte Carlo localization (MCL) function.
[0150] 13. The computer equipment 10 of embodiment 12, wherein the SLAM algorithm executed by the computer equipment 10 is a particle filter-based SLAM algorithm that uses particles contained in particle filter data from individual robots 16 to determine map update information (MUI) 24 for the map 12.
[0151] 14. A computer facility 10 as described in embodiment 13, wherein the particle filter data from each robot 16 includes particles, each particle representing a possible pose of the robot 16 within the environment 14.
[0152] 15. The computer equipment 10 of any of embodiments 10-14, wherein the processing circuitry 32 is configured to accumulate the received log data according to a defined accumulation interval or volume threshold to form a data batch, and the processing circuitry is further configured to update the map 12 by executing a SLAM algorithm on the data batch.
[0153] 16. A method 300 of operation by a computing facility 10, the method including (a) receiving logged data 22 generated by individual robots 16 operating within an environment 14 (block 302), and (b) updating a map 12 by executing a simultaneous localization and mapping (SLAM) algorithm using the received sensor and logged data 22 (block 304).
[0154] 17. The method 300 of embodiment 16, further comprising performing updates on a trigger basis.
[0155] 18. The method 300 according to embodiment 17, wherein the trigger basis is a periodic basis, and the method 300 includes repeatedly performing the update according to a defined interval.
[0156] 19. The method 300 of embodiment 18, further comprising adjusting the defined interval depending on the extent or nature of map changes determined by the computing equipment 10 in one or more of the repeated updates.
[0157] 20. The method 300 of embodiment 17, wherein the trigger basis is map quality based, and the method 300 includes performing the update in response to determining that the quality of the map 12 has deteriorated below a threshold.
[0158] 21. The method 300 of any of embodiments 16-20, further comprising generating an alert indicating the performance of the update or indicating the availability of an updated version of the map 12 as a result of the update.
[0159] 22. A method 300 according to any of embodiments 16-21, further comprising initiating transmission of map update information 24 to individual robots 16 for use by each robot 16 in updating a local static copy of the map 12, the map update information (MUI) 24 including an updated version of the map 12 or change data indicating an update to the map 12.
[0160] 23. The method 300 of embodiment 22, wherein transmission of the map update information is initiated in response to the computer equipment 10 receiving an input signal indicating authorization to initiate transmission.
[0161] 24. The method 300 of any of embodiments 16-22, including wirelessly receiving the log data 22.
[0162] 25. A method 300 according to any of embodiments 16-24, wherein the log data 22 from each robot 16 includes sensor data obtained via one or more types of sensors of the robot 16, the sensor data including environmental sensing data used by the robot 16 for self-localization in the environment 14, and the log data 22 further includes localization processing results such as particles from a particle filter-based localization algorithm used by the robot 16 for self-localization in the environment 14.
[0163] 26. The method 300 according to embodiment 25, wherein the SLAM algorithm executed by the computer equipment 10 is a particle filter-based SLAM algorithm that determines an updated version of the map 12 using particles that constitute the particle filter data contained in the log data 22 from the individual robots 16.
[0164] 27. A method 300 according to any of embodiments 25-27, comprising accumulating the received log data 22 to form a data batch according to a defined accumulation interval or volume threshold, and the processing circuitry 32 is further configured to update the map 12 by running a SLAM algorithm on the data batch.
[0165] 28. A robot 16 comprising: (a) a drive system 40 configured to move the robot 16 within an environment 14; (b) an odometry system 42 configured to track the movement of the robot 16 within the environment 14 in a dead-reckoning based manner; (c) an environmental sensing system 44 configured to sense characteristics of the environment 14; and (d) a robot control system 46. The robot control system 46 is configured to (i) perform localization of the robot 16 while the robot 16 is operating within the environment 14, the localization being performed using a static copy of a map 12 of the environment 14 held in a memory 48 of the robot 16; (ii) record data generated by one or more types of sensors of the robot 16 during operation of the robot 16 in the environment 14 as log data 22; and (iii) transmit the log data 22 via a communication link 18 to a remote computer facility 10 or its proxy 20 for use by the computer facility 10 in updating the map 12.
[0166] 29. The robot 16 described in embodiment 28, wherein the robot control system 46 is further configured to: (a) receive map update information (MUI) 24 including an updated version of the map 12 or change data indicating an update to the map 12, the map update information being generated by a remote computing facility using log data 22 transmitted by the robot 16 or such as log data transmitted by one or more other robots 16 operating in the environment 14; (b) determine from the map update information an updated local static copy of the map 12; and (c) perform a further self-localization while operating in the environment 14, the further self-localization being performed by the robot 16 using the updated local static copy of the map 12.
[0167] 30. A robot (16) as described in embodiment 28 or 29, wherein the robot control system (46) is configured to perform self-localization, and further performs self-localization using a Monte Calro localization (MCL) function based on particle filtering.
[0168] 31. A robot described in any of embodiments 28-30, wherein the self-localization is based on the robot control system 46 executing an algorithm based on a particle filter, and the log data 22 recorded by the robot control system 46 includes at least environmental sensing data obtained by the environmental sensing system 44 of the robot 16 to perform self-localization of the robot 16 in the environment 14, and the log data 22 further includes at least one of particle filter data generated by the particle filter algorithm and odometry data obtained from the odometry system 42 and used by the robot control system 46 to perform self-localization of the robot 16 in the environment 14, and the environmental sensing data includes sensor data from one or more types of sensors included in the environmental sensing system 44.
[0169] 32. The robot 16 of embodiment 31, wherein the particle filter data includes particles.
[0170] 33. A method 400 performed by a robot 16, comprising: (a) performing self-localization of the robot 16 using a static copy of a map 12 of the environment 14 held in the memory 48 of the robot 16 while the robot 16 is operating within the environment 14 (block 402); (b) recording data generated from one or more types of sensors of the robot 16 while the robot 16 is operating in the environment 14 as logged data 22 (block 404); and (c) transmitting the logged data 22 via a communication link 18 to a remote computer facility 10 or its proxy 20 for use by the computer facility 10 in updating the map 12 (block 406).
[0171] Figure 15 illustrates a method 1500 of operation of the computer equipment 10 according to one embodiment. The method 1500 can be understood as a combination of the operations illustrated in Figures 3, 5 and 6, or a variation or extension of such operations.
[0172] The method 1500 includes the computing facility 10 monitoring (block 1502) localization quality of one or more robots 16 performing autonomous navigation in the environment 14 according to a static map 12 of the environment 14. The localization quality experienced by any particular robot 16 at any particular time depends on the extent to which relevant portions of the static map 12 match local environmental features sensed by the robot 16. For example, laser scan data or point cloud data acquired by the robot 16 during a given scan cycle may indicate certain features in the environment 14, such as corners, doorways, etc., and the localization quality depends on the extent to which the static map 12 contains the same or similar features in relation to the position and orientation of the robot 16 relative to the scan data. Broadly speaking, each robot 16 is equipped with one or more types of sensors used to sense its local surroundings within the larger environment 14, said data sometimes being referred to as environmental sensing data.
[0173] The method 1500 further includes the computer facility 10 triggering an update procedure in response to the monitored localization quality of the one or more robots falling below a defined threshold ("YES" from block 1504). Executing the update procedure includes the computer facility 10 outputting map update information of the static map 12 for transmission to the one or more robots 16 (block 1506). The map update information is calculated from historical log data 23 collected based on one or more robots 16 reporting log data 22 while operating in the environment 14, the log data 22 from any particular robot 16 including sensor data used by the particular robot 16 to perform localization estimates for autonomous navigation within the environment 14, and one or both of corresponding odometry data and corresponding localization data generated by a localization process performed by the particular robot 16.
[0174] The update procedure further includes, for example, the computer facility 10 calculating, as part of the update procedure, map update information 24 according to the historical log data 23 then available to the computer facility 10.
[0175] In at least one embodiment, the computer facility 10 further calculates the map update information 24 based on historical update information determined by the computer facility 10 from one or more previous performances of the update procedure by the computer facility 10 .
[0176] The computer facility 10, in one or more embodiments, is configured to iteratively calculate map updates 24, each calculation using historical log data 23 then available to the computer facility 10. The update procedure in such an embodiment outputs the last calculated map update.
[0177] In at least one embodiment, outputting the map update 24 includes conditionally transmitting the map update 24 to one or more robots 16 in response to receiving user input indicating an acceptable map quality of the map update 24. Conditionally outputting the map update 24 includes, for example, outputting graphical information for display to a user of the computer equipment 10, the graphical information being a graphical depiction of the static map 12 updated in accordance with the map update 24, for the user to determine whether the quality of the map update 24 is acceptable or not. Such operations are illustrated in FIG. 16 as operations 1506A, 1506B, and 1506C.
[0178] Outputting the map update information 24 may include transmitting the map update information 24 to one or more robots 16 or transmitting the map update information 24 to a proxy device 20 which forwards the map update information 24 to one or more robots 16 .
[0179] In at least one embodiment, outputting the map update information 24 comprises outputting a prompt indicating that the map update information 24 is available to be sent to one or more robots 16 and, in response to administrative input from a local or remote user of the computing equipment 10, sending the map update information 24 to one or more robots 16 or to a proxy device 20 that forwards the map update information 24 to one or more robots 16.
[0180] One or more embodiments of the method 1500 include restricting data logging by one or more robots 16 to a particular area within the environment. For example, the computer equipment 10 outputs map information identifying the designated area for transmission to the one or more robots 16. The designated area may be user-configurable, such as when the computer equipment receives user input defining the designated area.
[0181] In one or more embodiments, the method 1500 includes recognizing periodic changes in map conditions during calculation of the map updates, which are referred to as map modalities. Each map modality is associated with a corresponding time interval, e.g., time of day. In at least one such embodiment, the map updates 24 include two or more mode maps reflecting different map modalities for use by one or more robots 16 during the corresponding time intervals associated with the map modalities. Thus, in one or more embodiments, a robot updating its respective copy of the static map 12 according to the map updates 24 will have a modality-specific mapping to deploy at the corresponding time.
[0182] The method 1500 may further include the computing equipment 10 computing the map update information 24 as part of a triggered update procedure or periodically, such that the map update information 24 is available for use in the triggered update procedure. Any particular instance of computing the map update information 24 by the computing equipment 10 will depend on the historical log data 23 available to the computing equipment 10 at the time.
[0183] In at least one embodiment in which the historical log data 23 spans one or more intervals of operating time within the environment 14 for one or more robots 16, the method 1500 further includes the computing equipment 10 identifying inconsistencies in localization determinations or feature observations made by each robot 16 with respect to the same region within the environment 14 and resolving the inconsistencies while computing the map update 24. Resolving inconsistencies during computation of the map update may include, for example, resolving inconsistencies between first data from a first robot of the one or more robots 16 and associated second data from a second robot of the one or more robots 16, such as between the first data and the second data, using the most recent data.
[0184] In one or more embodiments, each of the one or more robots 16 autonomously navigates the environment 14 according to the static map 12 using a Monte Carlo localizer (MCL) instantiated on the robot that is particle filter-based and performs localization but does not update the static map. Correspondingly, the method 1500 in at least one such embodiment includes the computer facility 10 performing, for each robot (16) represented in the history log data (23), a combined process performing a facility-activated SLAM process that is particle filter-based and performs localization and updates the static map 12 simultaneously, and calculating the map update information 24 by performing a non-coupled process that clones selected particles from each localization cycle performed by the robot-activated MCL and evolves the selected particles using a motion model that is identical to the motion model used by the robot-activated MCL, but with an additional step of map updating for each particle. The combined process evolves the particles of the facility-activated SLAM process based on the odometry data and corresponding environmental sensing data used in the robot-activated MCL, independent of the particle evolution that occurred in the robot-activated MCL. Such operations, in one or more embodiments, are included in the overall BOMU process described herein.
[0185] In such an embodiment, the method further includes the computer facility 10 comparing the map updates determined via the combined process and the non-combined process to obtain a finalized map update for output as map update information 24. The finalized map update is based on determining the best particle in terms of localization score from a pool of particles including combined particles from the combined process and non-combined particles from the non-combined process.
[0186] The computing equipment 10 according to one or more embodiments includes a communication interface 30 and a processing circuit 32 operatively associated with the communication interface 30. The processing circuit 32 is configured to perform any one or more of the operations described above for the method 1500. For example, the processing circuit 32 is configured to monitor a localization quality of one or more robots (16) performing autonomous navigation within an environment (14) according to a static map (12) of the environment 14. The localization quality experienced by any particular robot 16 at any particular time depends on the degree to which a relevant portion of the static map 12 matches local environmental features sensed by the robot 16. Furthermore, the processing circuit 32 is configured to trigger an update procedure in response to the monitored localization quality of the one or more robots 16 falling below a defined threshold.
[0187] The update procedure includes the processing circuitry 32 outputting map updates 24 for the static map 12 for transmission to the one or more robots 16. The map updates 24 are calculated from historical log data 23 based on the one or more robots 16 reporting log data 22 during operation in the environment 14. The log data 22 from any particular robot 16 includes sensor data used by the particular robot 16 for use in performing localization for autonomous navigation within the environment 14, along with corresponding odometry data and / or corresponding localization data used in the localization process performed by the particular robot 16.
[0188] With the above examples in mind, it is intended that the disclosed invention not be limited to the specific embodiments disclosed, and that modifications and other embodiments are included within the scope of the present disclosure. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
Claims
1. A method (1500) performed by a computer facility (10), comprising: monitoring (1502) localization quality for one or more robots (16) performing autonomous navigation within an environment (14) according to a static map (12) of the environment (14), wherein the localization quality achieved by any particular robot (16) at any particular time depends on the degree to which relevant portions of the static map (12) match local environmental features sensed by the robot (16); triggering (1504) an update procedure in response to the monitored localization quality of the one or more robots falling below a defined threshold, the update procedure including outputting (1506) map update information (24) for the static map (12) for transmission to the one or more robots (16), the map update information (24) being calculated using historical log data (23) collected from one or more robots (16) reporting log data (22) during operation in the environment (14), the log data (22) from any particular robot (16) including sensor data used by the particular robot to perform localization estimates for autonomous navigation within the environment (14), and corresponding odometry data and / or corresponding localization data used in a localization process performed by the particular robot (16); Including, The method (1500), wherein the update procedure further includes, as part of the update procedure, the computer facility (10) iteratively calculating the map update information (24) while the robot is online using historical log data (23) available at that time, the update procedure outputting the final calculated map update information (24).
2. 2. The method (1500) of claim 1, wherein the computer facility (10) further calculates map update information (24) based on historical map update information (25) from one or more previous executions of the update procedure by the computer facility (10).
3. 3. The method (1500) of claim 1 or 2, wherein outputting the map update information (24) comprises outputting the map update information (24) for transmission to the one or more robots (16) in response to receiving supervisory input from a user of the computer equipment (10).
4. The method (1500) of any one of claims 1 to 3, wherein outputting the map update information (24) comprises transmitting the map update information (24) to the one or more robots (16) or transmitting the map update information (24) to a proxy device (20) that forwards the map update information (24) to the one or more robots (16).
5. The method (1500) of any one of claims 1 to 4, further comprising limiting the log data (22) by the one or more robots (16) to a particular area within the environment (14) by outputting map information identifying the particular area for transmission to the one or more robots (16).
6. 6. The method (1500) of claim 5, further comprising receiving a user input specifying the particular area for which updates of the static map are permitted.
7. A method (1500) according to any one of claims 1 to 6, comprising calculating the map update information (24), where calculating the map update information (24) comprises recognizing periodic changes in map state, referred to as map modalities, each map modality reflecting a repeating state of the environment (14) and having a corresponding time interval associated therewith.
8. 8. The method (1500) of claim 7, wherein the map update information (24) comprises two or more modal maps reflecting different map modalities for use by the one or more robots (16) during the corresponding time intervals associated with the map modalities.
9. A method (1500) according to any one of claims 1 to 8, wherein the historical log data (23) spans one or more intervals of operating time within the environment (14) for the one or more robots (16), the method (1500) further comprising identifying inconsistencies in localization determinations or feature observations made by each robot (16) for the same region within the environment (14) and resolving the inconsistencies during calculation of the map update information (24).
10. 10. The method (1500) of claim 9, wherein resolving the discrepancies during calculation of the map update information (24) includes resolving the discrepancies between first data from a first robot of the one or more robots (16) and associated second data from a second robot of the one or more robots (16) using up-to-date data regarding the first data and the associated second data.
11. each of the one or more robots (16) performs autonomous navigation in the environment according to the static map using a robot-activated Monte Carlo localization (MCL) that is particle filter-based and performs localization but does not update the static map (12); The method (1500) may further comprise: The computer facility (10) calculates, for each robot (16) represented in the history log data (23), a coupled process in which the computer facility (10) executes an equipment-initiated SLAM process that is particle filter based and performs simultaneous localization and updating of the static map (12), the coupled process evolving particles of the equipment-initiated SLAM process based on odometry data and corresponding environmental sensing data used in the robot-initiated MCL, independent of particle evolution caused by the robot-initiated MCL; performing a non-coupled process of cloning selected particles from each localization cycle performed by the robot-activated MCL and evolving the selected particles using a motion model identical to the motion model used by the robot-activated MCL but with an additional step of updating a map for each particle; and comparing the map updates determined via the combining process and the decombining process and determining the best particle in terms of localization score from a pool of particles including combined particles from the combining process and uncombined particles from the decombining process to obtain a final map update for output as the map update information (24); calculating said map update information (24) by The method (1500) according to any one of claims 1 to 10, comprising:
12. A communication interface (30); a processing circuit (32) operatively associated with said communication interface (30); A computer facility comprising: The processing circuit (32) monitoring localization quality for one or more robots (16) performing autonomous navigation within an environment (14) according to a static map (12) of the environment (14), the localization quality achieved by any particular robot (16) at any particular time being dependent on the degree to which relevant portions of the static map (12) match local environmental features sensed by the robot (16); triggering an update procedure in response to the monitored localization quality of the one or more robots falling below a defined threshold, the update procedure including outputting map update information (24) for the static map (12) for transmission to the one or more robots (16), the map update information (24) being calculated using historical log data (23) collected from one or more robots (16) reporting log data (22) during operation in the environment (14), the log data (22) from any particular robot (16) including sensor data used by the particular robot (16) to perform localization for autonomous navigation within the environment (14), and corresponding odometry data and / or corresponding localization data used in a localization process performed by the particular robot (16); The processing circuitry (32) is further configured to, as part of the update procedure, cause the computer facility (10) to iteratively calculate the map update information (24) while the robot is online using historical log data (23) currently available, the update procedure outputting a final calculated map update information (24). Computer equipment (10).
13. 13. The computer facility (10) of claim 12, wherein the processing circuitry (32) is further configured to calculate the map update information (24) based on historical map update information (15) and historical log data from one or more previous executions of the update procedure by the processing circuitry (32).
14. 14. The computer equipment (10) of claim 12 or 13, wherein the processing circuitry (32) is configured to output the map update information (24) for transmission to the one or more robots (16) in response to receiving supervisory input from a user of the computer equipment.
15. The computer equipment (10) of any one of claims 12 to 14, wherein the processing circuitry (32) is configured to output the map update information (24) by transmitting it via the communication interface (30) to the one or more robots (16) or to transmit the map update information (24) to a proxy device (20) that forwards it to the one or more robots (16).
16. The computer equipment (10) of any one of claims 12 to 15, wherein the processing circuitry (32) is configured to limit the log data (22) by the one or more robots (16) to the particular area within the environment (14) by outputting map information identifying the particular area for transmission to the one or more robots (16).
17. 17. The computer facility (10) of claim 16, wherein the processing circuitry (32) is configured to receive a user input defining the particular area within which updates of the static map are permitted.
18. 18. The computer equipment (10) of any one of claims 12 to 17, wherein the processing circuitry (32) is configured to calculate the map update information (24), and as part of calculating the map update information (24), the processing circuitry (32) is configured to recognize periodic changes in map state, referred to as map modalities, each map modality reflecting a repeating state of the environment (14) and having a corresponding time interval associated therewith.
19. 20. The computer equipment (10) of claim 18, wherein the map update information (24) comprises two or more modal maps reflecting different map modalities for use by the one or more robots (16) during the corresponding time intervals associated with the map modalities.
20. The history log data (23) spans one or more intervals of operating time within the environment (14) for the one or more robots (16), and the processing circuitry (32) is configured to identify inconsistencies in localization determinations or feature observations made by each robot (16) for the same region within the environment (14) and to resolve the inconsistencies during calculation of the map update information (24). A computer installation (10) according to any one of claims 12 to 19.
21. 21. The computer facility (10) of claim 20, wherein to resolve a discrepancy between first data from a first robot of the one or more robots (16) and associated second data from a second robot of the one or more robots (16) during calculation of the map update information (24), the processing circuitry (32) is configured to resolve the discrepancy using the most recent data with respect to the first data and the associated second data.
22. each of the one or more robots (16) performs autonomous navigation in the environment according to the static map (12) using a robot-activated Monte Carlo localization (MCL) that is particle filter-based and performs localization but does not update the static map (12); The processing circuit (32) further comprises: For each robot (16) represented in the history log data (23): a coupled process in which the computer facility (10) executes an equipment-initiated SLAM process that is particle filter based and performs simultaneous localization and updating of the static map (12), the coupled process evolving particles of the equipment-initiated SLAM process based on odometry data and corresponding environmental sensing data used in the robot-initiated MCL, independent of particle evolution caused by the robot-initiated MCL; performing a non-coupled process of cloning selected particles from each localization cycle performed by the robot-activated MCL and evolving the selected particles using a motion model identical to the motion model used by the robot-activated MCL but with an additional step of updating a map for each particle; and comparing the map updates determined via the combining process and the decombining process and determining the best particle in terms of localization score from a pool of particles including combined particles from the combining process and uncombined particles from the decombining process to obtain a final map update for output as the map update information (24); configured to calculate the map update information (24) by A computer installation (10) according to any one of claims 12 to 21.
Citation Information
Patent Citations
Autonomous moving body and environment map update device
JP2018017826A
System and method for utilizing non-local information to constrain robot services
JP2019003602A
Mobile body and mobile body system
JP2019175137A
Systems and methods for providing vehicle awareness
JP2019510240A
Position estimation device
JP2021018640A
