Automatic mapping by mobile robots

Abstract

The automated mapping system defines the exploration behavior of a mobile robot in a physical environment according to multi-objective optimization, in which the robot's exploration behavior depends on simultaneously considering two or more search objectives, such as speed, coverage, and consistency. In at least one embodiment, the automated mapping system is user-customizable via a user interface that allows the user to prioritize and / or select the search objectives considered in the multi-objective optimization, among other things. The automated mapping system in one or more embodiments incorporates advantageous formulations such as loop closure prediction for consistent mapping, kinematic prioritization for smooth trajectory generation, local refinement for improving map coverage, a time-based blacklisting table for managing search objectives, a 2D attraction layer and recovery behavior for efficient path planning, and a recommendation system that allows the operator / user to refine subsequent search runs, particularly for remapping.

Description

[Technical Field]

[0001] The present disclosure relates generally to mobile robots, and more particularly to methods and apparatus for automatic mapping of a physical environment by a mobile robot. [Background technology]

[0002] Mapping for industrial mobile robots typically requires a skilled human operator to guide the robot around the physical environment of interest and collect data from sensors on the robot. A map is then constructed offline on a computer using SLAM techniques. The map is further refined offline by the skilled operator to omit "no-go" areas or to correct areas that were incorrectly mapped by the offline process. Once the map is created, it is not updated until the environment changes enough to require remapping.

[0003] A robot navigating an environment represented by a map creates a path plan using set start and end goals defined in the map, and performs the path planning using a localization algorithm that allows the robot to determine its position and orientation as it moves through the environment. SLAM is not performed continuously because it is not always possible to distinguish which feature points are static and which are not, which can result in errors in the performed plan.

[0004] The commercial home vacuum market includes robotic vacuum cleaners with automatic mapping capabilities. See, for example, U.S. Patent No. 9,188,983. Such systems use a real-time SLAM algorithm to search for a "frontier," which refers to the boundary between known (mapped) and unknown (unmapped) areas, and then attempt to map the unmapped areas. Unmapped areas are prioritized based on the largest frontier boundary. The mapping process is complete when no frontiers exist that are larger than some specified limit. Users can proactively configure "no-go" zones using panels or other techniques, such as those found in U.S. Patent No. 9,854,737. Using the resulting map, the robot can ensure that it can completely cover the space enclosed by the map. In this situation, precise following of a specified path is less important. While the basic frontier approach works well in small spaces such as homes, it has practical limitations in larger spaces, such as factories or other industrial or commercial environments. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Patent No. 9,188,983 [Patent Document 2] U.S. Patent No. 9,854,737 Summary of the Invention

[0006] The mobile robot operates as an automated mapping system, and the mobile robot implements an automated map generator that defines the robot's exploration behavior in a physical environment according to multi-objective optimization. Multi-objective optimization relies on simultaneously considering two or more search objectives. In at least one embodiment, the automated mapping system is user-customizable via a user interface that allows the user, among other things, to prioritize and / or select the search objectives to be considered in the multi-objective optimization.

[0007] One or more embodiments include a method for automated mapping by a mobile robot, the method including the robot using a simultaneous online localization and mapping (SLAM) algorithm to identify a next frontier to explore from among two or more candidate frontiers in a map generated by the mobile robot. The identification is based on the robot performing multi-objective optimization for each candidate frontier, simultaneously evaluating a set of two or more search objectives, and determining an attractiveness of the corresponding candidate frontier as a next search target. Each of the two or more search objectives is a different search objective. The method further includes the robot planning a global path to a global waypoint on the next frontier and navigating to the global waypoint according to the global path, the global path taking into account collision avoidance and path parameters determined by local path planning performed based on environmental sensors on the robot.

[0008] One or more embodiments include a computer-implemented method for customizing exploration behavior of a mobile robot based on displaying a user interface corresponding to a physical environment to be mapped by the mobile robot, wherein an automated map generator drives the mobile robot to explore the physical environment based on selecting a frontier for exploration according to a multi-objective optimization that considers two or more search objectives, each having an objective type. For example, a first search objective emphasizes exploration for mapping accuracy, while a second search objective emphasizes exploration for mapping speed. More broadly, different types of objectives, expressed as objective functions, represent differently emphasized exploration, such as more or less aggressive exploration with corresponding trade-offs in higher or lower risk of the robot getting lost, or more or less time-consuming exploration with corresponding trade-offs in the completeness or accuracy of the resulting environmental map.

[0009] Multi-objective optimization advantageously considers and balances competing search objectives. In one or more embodiments, multi-objective optimization allows a user to provide input to select which search objectives are considered in the multi-objective optimization or to set priorities for resolving optimization trade-offs between two or more search objectives that are subject to simultaneous optimization.

[0010] In one or more embodiments, the automated mapping system, in response to receiving corresponding user input via a user interface, performs any one or more of: constraining the search space and mapping region; prioritizing the mapping region; selecting termination criteria and bounding strategies; selecting search strategies, constraints, and parameters; selecting static trajectories; customizing the relative weights or priorities of multiple search objectives evaluated by the mobile robot in a multi-objective optimization that controls the mobile robot's exploration behavior; and specifying safety constraints.

[0011] Another embodiment includes a computer-implemented method for providing live feedback and exploration process control for a mobile robot implementing an automatic map generator as described above, including visualizing the robot-generated map, obstacles, robot position, planned global and local paths, cost map and frontiers, frontier priority, and blacklisting status, providing user input controls for changing the frontier priority and blacklisting status, aborting or stopping the exploration process if the robot performs an unsafe maneuver, and rolling back a variable number of trajectories in case of incorrect mapping.

[0012] A mobile robot according to one or more embodiments includes a drive system configured to move the mobile robot within a physical environment, one or more sensors configured to detect obstacles within the physical environment within corresponding detection ranges, and a processing circuit.

[0013] The processing circuitry is configured to use an online SLAM algorithm to identify a next frontier to explore from among two or more candidate frontiers in a map generated by the mobile robot. The identification is based on the processing circuit performing multi-objective optimization that simultaneously evaluates a set of two or more search objectives and, for each candidate frontier, determining an attractiveness of the corresponding candidate frontier as a next exploration target, where each of the two or more search objectives is a different search objective. The processing circuitry is further configured to plan a global path to a global waypoint on the next frontier and navigate to the global waypoint according to the global path while taking into account collision avoidance and path parameters determined by local path planning performed based on environmental sensors on the robot.

[0014] Of course, the present invention is not limited to the above features and advantages. Those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 2 is a block diagram of data and control flow in an automatic map generator according to one or more embodiments. [Figure 2] FIG. 2 is a block diagram illustrating the relative operational timing between the various modules or functions that make up the automatic map generator, in accordance with one or more embodiments. [Figure 3] FIG. 2 is a block diagram illustrating a simplified system diagram of an automatic map generator configuration module according to one or more embodiments. [Figure 4A] FIG. 4 is a block diagram showing further details of the system diagram shown in FIG. 3. [Figure 4B] FIG. 4 is a block diagram illustrating additional details of the system diagram shown in FIG. 3. [Figure 5] FIG. 1 is a block diagram of a generative adversarial network (GAN) used for loop-closing prediction, according to one or more embodiments. [Figure 6] FIG. 2 is a block diagram illustrating the search state and user interface of the automatic map creator in accordance with one or more embodiments. [Figure 7] FIG. 10 is a block diagram illustrating a user interface for customizing the behavior of the automatic map generator, according to one or more embodiments. [Figure 8] FIG. 10 is a block diagram illustrating an automatic map creator feedback interface in accordance with one or more embodiments. [Figure 9] FIG. 1 is a block diagram of a mobile robot according to one or more embodiments. [Figure 10] FIG. 1 is a block diagram illustrating exemplary implementation details of a mobile robot, according to one or more embodiments. [Figure 11] FIG. 1 is a block diagram illustrating exemplary implementation details of a mobile robot, according to one or more embodiments. [Figure 12] FIG. 1 is a block diagram illustrating exemplary implementation details of a mobile robot, according to one or more embodiments. [Figure 13] FIG. 1 is a block diagram illustrating exemplary implementation details of a mobile robot, according to one or more embodiments. [Figure 14]FIG. 1 is a block diagram illustrating exemplary implementation details of a mobile robot, according to one or more embodiments. [Figure 15] FIG. 1 is a block diagram illustrating a mobile robot and associated computer system according to one or more embodiments. [Figure 16] FIG. 10 illustrates exemplary tuning parameters for configuring automatic mapping behavior, according to one or more embodiments. [Figure 17] FIG. 1 is a logic flow diagram illustrating a method of operation by a mobile robot for automated mapping, according to one or more embodiments. [Figure 18] FIG. 18 is a logic flow diagram illustrating a detailed example of the method shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] This specification describes a method and apparatus for automated mapping by a mobile robot. An "automated map generator" according to the disclosed technology replaces the tedious manual mapping of large industrial environments prior to robot deployment. In particular, the automated map generator provides an automated process that reduces the time required to map an environment while minimizing human involvement. When used without limitation, the term "automated map generator" refers to the functionality implemented in the disclosed technology, with the understanding that the functionality is implemented by a processing and control system onboard the mobile robot that performs the mapping operations.

[0017] "Discovery framework" and "automapper" are interchangeable terms in this disclosure, while related terms such as "discovery suite" refer to subsystems or component functions encompassed by the automapper.

[0018] The mobile robot implements an automatic map builder that plans and executes paths through an environment, targeting unmapped areas, while simultaneously estimating the mobile robot's position using SLAM techniques and building a map representing the environment. A notable aspect of the automatic map builder is that it operates according to multi-objective optimization, in which the mobile robot selects a search goal for exploring a physical environment based on simultaneously evaluating a set of two or more search objectives for each frontier that are candidates for selection as the next search goal.

[0019] Each search objective is represented as an objective function whose value is to be minimized or maximized. One or more of the objective functions may be in conflict with one or more other objective functions, such that multi-objective optimization involves a mobile robot determining an optimal tradeoff or balance between multiple search objectives. For example, there may be one or more speed-related objectives that emphasize mapping speed in the search behavior of the mobile robot. However, mapping speed may be in conflict with mapping accuracy and / or mapping completeness, and these objectives are represented by respective objective functions considered in the multi-objective optimization. Additional exemplary objectives include those related to smoothness of motion by the mobile robot, or those related to robustness, e.g., tolerance to loss of localization, battery life, etc.

[0020] In at least one embodiment, the mobile robot includes a memory or other storage device from which it reads user configuration values ​​or data to adjust the multi-objective optimization of the mobile robot.

[0021] Thus, in one or more embodiments, the automated mapping system disclosed herein comprises a mobile robot that instantiates and executes an example of an automated map generator, allowing for human intervention when necessary to establish parameters such as targets and / or definitions of no-go areas and emergency stops. By retaining such human-configured map elements, the system also greatly simplifies remapping a dynamically changing environment (without affecting the human-configured map elements) while the robot is in operation. The system also finds potential use in target discovery. An exemplary system may further include supporting logic available via a PC or other interface device for a user to input configuration parameter values ​​for adjusting the automated map generator.

[0022] Although achieving Level 5 autonomy, as defined by the Society of Automotive Engineers (SAE), is possible with an automatic map generator, a less complex and less expensive implementation of the automatic map generator provides Level 2 autonomy, which has distinct advantages. In such situations, the automatic map generator supports user monitoring of the automatic mapping to enable the execution of an "abort" for potentially dangerous navigation behavior during sensor or decision system failure.

[0023] The automated map generator achieves a significant reduction in mapping time compared to manual mapping by maximizing map quality, coverage, completeness, and localization accuracy while selecting the shortest route and maintaining the greatest distance from structures in the environment according to the detection capabilities of the sensors used by the mobile robot for mapping. A human operator working in the same area would be unable to determine the optimal route that reduces time while improving localization performance and map quality without feedback from the robot, which is generally unavailable in manual mapping processes. Furthermore, the movement speed of a robot operating according to the disclosed automated map generator is much faster than the movement speed of a human operator who moves around with the robot for mapping operations. This speed reduces the man-hours required for mapping. Furthermore, because the operator's visual or remote monitoring of the mapping robot via line of sight is usually sufficient, the automated map generator significantly reduces the operator's workload.

[0024] Exemplary embodiments provide a mobile robot with an automatic map generator, which provides advantages in situations requiring precise navigation in large spaces or improved flexibility for remapping. As used herein, the term "mobile robot" has a broad meaning, unless the context requires otherwise, and encompasses a wide range of wheeled and tracked vehicles, such as industrial robots or autonomous vehicles, and further encompasses drones and other aerial vehicles, as well as autonomous marine vessels.

[0025] Among the various benefits of the automated map generator, one key advantage is its use of a multi-objective optimizer that maximizes map coverage while minimizing the time required for mapping. For example, the multi-objective optimizer, which includes specific processing and computational operations implemented by the execution of corresponding computer program instructions, maximizes global map accuracy by prioritizing a robot path through frontiers that enable loop closure, i.e., by registering the current view with reliably identified landmarks in the environment.

[0026] The techniques implemented in the multi-objective optimizer improve on the current state-of-the-art, focusing on frontier-based mapping that prioritizes the largest frontier to maximize coverage, but without multi-objective optimization that exploits trade-offs between various timely search options. For example, in one or more embodiments, the multi-objective optimization embodied in the disclosed automated map generator considers more than one objective function simultaneously, with each objective function implementing a different search goal, such as a more comprehensive search or a fastest search. Simultaneous optimization balances trade-offs, for example, between mapping time and global map accuracy. In at least one embodiment, the automated map generator incorporates or supports a user interface that exposes specific parameters of the optimization algorithm, thus providing a user with an advantageous mechanism for “tuning” the optimization behavior of a mobile robot implementing the automated map generator. For example, a user can adjust mapping speed or mapping accuracy, or adjust for more or less aggressive searching. Generally, embodiments of the automated map generator provide a user with an advantageous ability to balance various trade-offs made by the robot during mapping.

[0027] The term "search suite" refers to a constituent function or subsystem contained within an overall automated map creation or search framework, such that the search suite determines global waypoints for navigation as a multi-objective optimization problem that seeks to maximize map quality, coverage, and localization score while minimizing the mapping distance covered and the time it takes to complete the search process. These sets of objectives are conflicting and involve making trade-offs, which are managed by a multi-objective optimizer. How the optimizer balances the trade-offs is, at least in part, "tunable" by the user to suit specific needs or requirements.

[0028] The search suite influences path planning to enforce loop closure during the search process by robots implementing the optimizer. Loop closure is enforced by predicting how the robot will reach landmarks as it navigates through unmapped frontiers (hereafter referred to as "predictive mapping"). This approach not only ensures accuracy through loop closure, but also prevents the robot from becoming lost (i.e., losing its localization).

[0029] The exploration suite uses kinematics prioritization that places a lower priority on paths that are more likely to require deviation from the robot's current heading, thereby minimizing the probability of changes in the direction of the executed path. This behavior provides the advantage, among other things, that the resulting path the robot follows will be smoother, exhibiting more predictable behavior around humans. Furthermore, smoother trajectories improve battery performance and extend the mechanical life of the motors and drive systems on the robot.

[0030] The robot's trajectory to the optimized waypoints is further subjected to a local refinement process that refines the trajectory by adding waypoints that allow the robot to capture local map structure from different viewpoints, thereby improving local map coverage.

[0031] Additionally, the search suite in one or more embodiments uses a time-based blacklisting table to reject targets deemed unreachable by the global planner. Unlike traditional blacklisting tables that permanently store these targets, the time-based blacklisting table assigns expiration times to table entries.

[0032] Another notable aspect of the exploration suite is the use of a global planning component that pushes the robot near visible structures during mapping, which utilizes a 2D attraction layer, as described later in this specification.

[0033] The search suite also uses a local planner that uses recovery behaviors, including "cost map" modifications, and local recovery behaviors, including backtracking, spins, and U-turns, that allow the robot to push itself out of the way when it gets stuck.

[0034] The automated mapping system is customizable in one or more embodiments, allowing the user to limit and prioritize map areas, manage trajectories, search strategies, and specify safety considerations. Additionally, in at least one embodiment, a recommendation function provides search analysis, evaluation, and feedback in the form of a recommendation system that can be used by the operator to improve and fine-tune current and future search and remapping runs.

[0035] The basic functionality of an automated map generator according to one or more embodiments is illustrated in Figures 1 and 2. In an exemplary context, the robot being used for automated mapping according to the disclosed technology is a differential-drive industrial mobile robot equipped with one or more two-dimensional (2D) laser scanners and / or sonar ranging devices, and the robot's processing system has access to odometry information via, for example, wheel encoders and an inertial measurement unit (IMU). It is also assumed that the robot builds and uses a 2D occupancy map for self-localization. However, the disclosed technology is directly applicable to or extendable to other types of robots, including unmanned aerial vehicles (UAVs), unmanned surface vehicles (USVs), unmanned ground vehicles (UGVs), and the like, with or without additional sensors such as three-dimensional (3D) lidar and vision cameras.

[0036] In one or more embodiments, the exploration framework includes: (a) a module that interacts with sensors on the mobile robot; (b) a search suite that performs multi-objective optimization to find best estimates of a search location and a next best waypoint (i.e., where to search and how to search) based on user-specified constraints; (c) a SLAM module that generates an online map using sensor data based on the searched locations; (d) a global path planner that creates a path to the search location identified by the search suite; (e) a local path planner that calculates parameterized local paths while avoiding obstacles; and (f) a motion controller that converts command positions to velocities. In at least one embodiment, the exploration framework further includes a user interface with options to control one or more aspects of the exploration process. For example, the user interface accepts user input, and exposed functions included in the exploration framework adjust the values ​​of one or more configuration parameters that affect the exploration behavior in response to the user input received via the user interface.

[0037] 1 illustrates an exemplary search framework at a high level, showing exemplary data and control flows through various blocks, where each block can be understood as a function or collection of related functions implemented by the execution of computer program instructions by, for example, one or more microprocessors or other digital processing circuitry specifically adapted to operate as an automated map generator.

[0038] For example, data from sensors onboard a mobile robot, including optical data (primarily laser scanning) and odometry, is streamed to a simultaneous localization and mapping (SLAM) module that generates a map of the environment and simultaneously estimates the robot's position within the map. During this process, an automatic map generator running on the robot performs a loop-closing optimization process at regular intervals based on modifications of the previously mapped region.

[0039] The map from SLAM is then input into a search suite. Based on a set of search parameters, which in one or more embodiments are user-adjustable, the search suite uses a multi-objective global optimization process to calculate optimal targets, or waypoints, from which to perform the search. Identifying and navigating to waypoints can be understood as elements of the search process performed by the mobile robot according to a search framework instantiated by the robot's processing circuitry.

[0040] The search targets / waypoints are then sent to a global path planner, which generates a path to each target. If a valid path cannot be found or is unavailable because all calculated paths are later determined to be feasible by the local planner (as described below), the global planner blacklists the current target and signals the search suite to fetch the next target for planning. Using localization information from the SLAM module, the global path planner can also determine whether a search target has been reached. If so, it signals the search suite to send a new target to search next.

[0041] The global path is then sent to a local path planner, which parameterizes local segments of the global path into a form suitable for operation based on the robot's current position, while simultaneously avoiding obstacles. The required local poses along the local path are then sent to a motion controller, which converts these positions into velocities for the robot's differential drive motors to operate.

[0042] The local planner recalculates a local path if it cannot identify a collision-free path or if obstacles in the robot's vicinity repeatedly halt the robot's execution of its planned path. There is a limit to the number of local paths the local planner can calculate and retry. If this limit is reached, i.e., if all local paths are invalid, the local planner signals the global planner to recalculate a global path. It is possible that all of these recalculated global paths are infeasible.

[0043] 2 is an exemplary timing diagram. The timing diagram is a simplified representation of the relative operational timing between various blocks or modules of the search framework, including periodic synchronous triggers or signals as well as situation-specific asynchronous triggers in an exemplary situation.

[0044] A system-level description of the exploration framework, according to one or more exemplary embodiments, is shown with respect to the notable constituent modules of the exploration framework in Figure 3. The search suite is shown in the center of Figure 3, and Figures 4A and 4B provide a more detailed view, according to one or more embodiments. The supporting algorithms and system components that enable the exploration framework are also shown in Figures 3, 4A, and 4B. As previously defined, the supporting algorithms and system components include industrial mobile robot hardware with sensors, global and local path planning modules, a motion (velocity) controller, SLAM, and a user interface.

[0045] The following sections describe each of the modules shown in Figures 4A and 4B.

[0046] Search Suite - Dynamic Trajectory Generation

[0047] The search suite supports dynamic trajectory generation to operate with minimal user intervention, as well as static trajectory generation with user input. The search suite's dynamic trajectory generation module performs a multi-objective optimization, and the solution determines the next global waypoint to travel to, along with the associated direction of travel. This waypoint information (position variables) is subjected to a local refinement process that determines local positions and orientations. The local refinement process also works independently to refine the local trajectory between global waypoints. The output of the multi-objective global optimization and local refinement (targets / waypoints and associated directions) is used by the global planner to construct a trajectory.

[0048] Dynamic Trajectory Generation - Wide Area Waypoint Estimation

[0049] This section describes an approach that formulates the search in terms of multi-objective optimization with specific multiple objectives to optimize. The global waypoints selected by the multi-objective optimization may be optimal or may be along a non-dominated / Pareto-optimal front. That is, a unique solution may be obtained, or multiple equally optimal solutions may be obtained, each of which performs well for one objective function considered in the multi-objective optimization, but may perform poorly for one or more other objective functions considered. Multi-objective optimization according to one or more embodiments of the automated map generator consists of several factors and corresponding variables for which optimization is performed using multiple objective functions. The multi-objective optimization problem can be framed in terms of the following equation:

number

number

[0050] where f i(x) is the respective objective function to be minimized, and X is the feasible set of decision vectors. Again, each objective function corresponds to a respective robot behavior or attribute, such as mapping speed, mapping accuracy, power efficiency, trajectory smoothness, etc.

[0051] These objectives are linearly reweighted to obtain a scalarized objective, which leads to a further simplified single-objective optimization. The scalarized form of the optimization with respect to the multi-objective function is given by:

number

[0052] where w i are the scalarization weights. These weights are set equal to 1 at the beginning of the optimization process. The weights are then scaled by a priority coefficient p i The weights can be customized by the user for specific use cases. i (x) can also be subject to additional constraints that can be used to simplify the scalarized optimization function.

[0053] The constraints on the objective function can be expressed as follows:

number

[0054] ε i,l and ε i,u denotes the minimum and maximum bounds of objective i.

[0055] In principle, because the solution space (including frontier objectives / waypoints, as discussed in the next section) is discrete, it is possible to solve such systems (i.e., find an index to the next frontier to explore, as discussed in the next section) using a brute force approach (i.e., by evaluating the objective at every point in the solution space) if the list of frontier objectives is not large. In large environments, the list may be large enough to make solution search difficult. In such cases, approximate solutions in the form of mixed-integer linear programming (where some of the objectives may be binary, some integer, and others real) can be obtained.

[0056] These approximate solutions may or may not produce optimal solutions across all objective functions, i.e., they may not produce a global minimum of the scalarized (also called mixed) objective functions. Multi-objective optimization may identify more than one index. Using the approximate solutions produced by the solver, the frontier objective / waypoint index with the closest distance to the robot's current position is identified and used in the next stage of processing. Scalarized forms with mixed objectives are solved using branch and bounds, along with gradient descent of local minima at the bounds. For details on such solvers, see, for example, Deb, K. (2014), "Multi-objective Optimization," (pp. 403-449), Springer, Boston, MA. This provides a simple and fast approach to solving the problem.

[0057] This multi-objective optimization can also be solved using alternative solvers, including approximate solvers. More complex approaches, such as hierarchical or lexicographical approaches, multi-objective decomposition, and hybrid approaches, supported by general solvers, can also be used. Solvers that support evolutionary algorithms, including particle swarm optimization, can also be used to solve such systems.

[0058] Wide Area Waypoint Estimation - Solution Space

[0059] The solution to the above multi-objective optimization problem (i.e., x∈X) is defined in terms of waypoints, or goal points, placed on the frontier. In other words, the overall optimization goal is to select one of the list of frontier goals. We now discuss frontiers.

[0060] Frontiers are used to identify potential targets for exploration. Frontiers are established as pixel neighborhoods of the boundary between known (unoccupied) and unknown (unscanned) space in the current map generated by SLAM (see the SLAM section for map generation). Here, the term "known" corresponds to previous optical sensor observations of either an obstacle at a specified location in space or a vacant / unoccupied location through which the sensor's scan line passed before detecting an obstacle behind it. These locations form cells in the occupancy map. "Unknown" locations correspond to cells in the occupancy map where no obstacle has been detected or where the sensor line has not previously traced an obstacle. The goal of exploration is to navigate to these frontiers to visit unknown regions, thereby adding them to the map. All frontiers so identified are added to a frontier list. Depth discontinuities determined by jumps in sensor measurements are classified as "lifts." Such lifts are also added to the list of frontiers to be processed at any given time.

[0061] These isolated frontier boundary neighborhoods are then characterized by a centroid location that is used as the target location for global path planning. Each frontier also has an associated approach direction, which is chosen as the normal to the frontier so as to extend beyond the frontier and divide the unknown region into approximately two equal halves. A blacklisting table is incorporated to prevent the robot from repeatedly attempting to explore a frontier target when no path to the target exists.

[0062] After repeated local path planning failures prevent the robot from reaching a certain frontier, the frontier is blacklisted for a period of time, during which the robot is free to explore other frontiers. Exploring other nearby frontiers is expected to potentially clear the cost map or add paths that allow the robot to reach a previously blacklisted frontier.

[0063] In one or more embodiments, the automatic map generator operates to cause the robot to retry a blacklisted frontier when a timeout on the blacklist expires and the frontier is removed from the blacklist. During the retry, the robot may or may not approach the originally blacklisted frontier along the same original route, depending on the robot's current location.

[0064] The selection of which frontier target / waypoint to explore next is determined by the output of the multi-objective optimization.

[0065] Wide-area waypoint estimation - optimization objectives

[0066] In one or more exemplary embodiments, various objective functions f to be optimized are i(x) includes: (1) distance-to-target objective, (2) searchability at target objective, (3) map coverage objective, (4) map consistency objective, and (5) kinematic priority objective. Note that these objectives are functions of the frontier targets / waypoints and are normalized to contribute equally to the optimization (framed as minimization). Furthermore, the objective to be maximized is expressed as the negated objective function in the minimization problem.

[0067] Distance-to-target objective: The closer the frontier target, the faster the robot can arrive, resulting in a more rapid search. For any given target, the shortest distance to the target is calculated using Dijkstra's shortest path algorithm on the current map. Additionally, for every turn along the path (i.e., deviation from a straight path), the time it takes to make that turn (based on the robot's maximum angular velocity) is estimated, converted to a distance offset, and added to the objective function. As shown in equation (4), it is multiplied by the maximum linear velocity and added as a distance offset. This ensures that the distance metric also represents the time it takes to reach the selected target. The calculated distance is then normalized using the expected maximum value for the given space. This maximum can be, for example, a user-determined best guess from a rough estimate of the diameter or diagonal distance of the boundary space to be searched, or a value calculated as the mathematical expectation of the maximum distance to the target value of previous runs in the space, as reported by the search analysis. The distance-to-target objective is to be minimized.

number

[0068] where minDist is the Dijkstra's minimum distance between the centroid xc of the frontier x and the current position of the robot r, defined in terms of 2D pixel coordinates (x, y), ν is the maximum linear velocity of the robot, and δθ i is the change in heading along the shortest path, and ω is the maximum angular velocity of the robot.

[0069] Searchability at Target: There are two sub-objectives that fall under the "Searchability at Target" objective: "Frontier Span" and "Frontier Adjacency." These sub-objectives also serve to increase the search coverage of any given environment.

[0070] Frontier Span: For a given frontier, the span of the frontier specifies how large it is and how much space can be explored by visiting it. The span is calculated as the number of pixels along the length of the frontier multiplied by the cell / pixel distance (Manhattan distance) between the pixels corresponding to the two ends of the frontier in the map's occupancy grid (the pixels from the start and end of each frontier are determined from the bounding box of the frontier pixels). Such a metric is expected to capture the extent of the unknown area beyond the frontier while ignoring non-smooth feature points along the length of the frontier. In the early stages of mapping, frontiers that are closer and have a wider area are preferred. Similar to the "distance to target" objective, this objective is normalized by a large expected value. This objective should be maximized. Because the multi-objective optimization problem is framed as a minimization problem, the negation of this objective is used as the objective function.

number

[0071] Here, l(x i ,y i ) is the indicator function of surrounding pixel i (of the frontier) evaluated over the length of the frontier, and BB(x s ,y s ), BB(x e ,y e ) are the start and end points of the bounding box of the frontier x:BB(x).

[0072] Frontier Adjacency: Within a given region, there may be multiple frontiers that can potentially merge or lead to multiple unknown regions that merge once one of the frontiers is explored. Thus, proximity to other frontiers also corresponds to the searchability of the target. For a given target / waypoint, this metric is calculated as the shortest path search between frontier targets using a Rapid Searching Random Tree (RRT*) for all other frontier targets in the immediate vicinity of the selected frontier target. The choice of RRT* here (as opposed to the traditional Dijkstra for wide-area planning) allows the possibility of calculating distances through unknown space. The normalized distance objective is the one to be minimized.

number

[0073] where N(x) is the set of frontiers that are near frontier x, i.e., frontiers that are within a specified maximum distance from frontier x. All distances, including minDist, are calculated using RRT* minimum distance, and if p is a pixel in frontier x, p is a pixel in frontier n, and p and q are the closest pixels to each other among all pixels in x and n, respectively, then x p , n q are the positions of pixel p and pixel q, respectively.

[0074] Map Coverage: The map coverage objective attempts to improve the map coverage (i.e., the proportion of the area that has been mapped relative to the total area to be mapped). Improving map coverage depends on the availability of floor plans, user sketches, previous versions of CAD data, or maps of the environment in a previous state that have since been modified, and may or may not be possible in a particular environment. Therefore, this objective is optional. In the case of automatic remapping of a previously mapped environment, the previous map can be reused for this purpose. In such cases, a graph-based discretization of the search space is performed, i.e., the entire environment map is represented as a graph. Topological refinement methods on graphs, such as tessellation, graph, and tree pruning, are applied to reduce the complexity of the graph and simplify its representation in terms of the graph nodes that form the waypoints. To maintain the topology of the space, fewer nodes are generated in large areas and more nodes are generated in smaller, compact, and highly structured areas. Each node is associated with the area that it "covers." From the list of graph nodes in the unknown region, the "node area" metric corresponding to the closest node of each frontier objective is used as the map coverage metric. Nodes with larger node areas are preferred. This normalized coverage objective is the one to be maximized.

number

[0075] where NA(m) is the node area covered by graph node m. This node m is located in the graph at the frontier centroid x c The graph G(u) to which this node m belongs is a subgraph representing the unknown space in the graph representation of the entire map.

[0076] Map Consistency: The map consistency objective attempts to improve the consistency of the map, i.e., its reproducibility between successive measurements generated by SLAM at each time step. A measure of map quality is the reproducibility of the generated map. The SLAM process measures map consistency in terms of an entropy metric. Consistently high entropy indicates low consistency, i.e., the map representation is significantly different and inconsistent between time instances. Entropy can be lowered by visiting known regions of the map. The localization score is also a related metric, improving as entropy decreases and decreasing as entropy increases. Therefore, the trade-off here can be explained in terms of the exploration-exploitation paradigm. There are three sub-objectives, together classified as "map consistency." These objectives are gated by a binarized version of the entropy metric E (entropy higher than a threshold activates this objective, i.e., the gating priority of this objective i in the multi-objective optimization is p). i = 1, while entropy below the threshold deactivates this objective, and p i =0).

number

[0077] where p(x i ) is the position x i is the occupancy probability of cell i in map M in

[0078] This objective can be thought of as including three sub-objectives: structural adjacency, exploitation, and predictive closure.

[0079] For the structure proximity sub-objective, frontier targets that are adjacent to physical structures (and not in open areas) are preferred. More visible structures help provide better localization for the robot and improve map consistency and visibility. A metric derived from the distance transform (normalized) at the target waypoint on the current binarized, value-inverted SLAM occupancy map (targets with nearby structures have lower distances in this metric) serves as the objective to be minimized.

number

[0080] where distXform(x) is the map grid point-frontier centroid x c is the distance transformation of

[0081] According to the "exploitation" sub-objective, when entropy drops, frontiers (more exploitation, less exploration) that require the robot to move through known territory and observe are favored. This "exploitation" sub-objective is characterized as a binary objective that positively reweights goals farther than a minimum threshold distance, as opposed to goals closer than this threshold. Note that this objective must be maximized, as opposed to the distance-to-goal objective.

number

[0082] where I(x) is the indicator function, x c is the centroid of the frontier x, r is the current robot pose, and D1 is the distance threshold.

[0083] The third sub-objective, predictive closure, targets identifying frontiers that, if the robot moves through the identified frontier, could potentially close a loop, returning to an already visited area with stable landmarks. In other words, the goal is to select frontiers that result in SLAM loop closure. Such loop closure can improve the quality and consistency of the map by providing a correction for drift in the robot's position estimate during the mapping process. The "Predictive Mapping" section of this specification provides details of the prediction process required to identify such a path. Frontiers that are predicted to trigger loop closure are given a negative distance metric, and frontiers that are not predicted to trigger loop closure are given a non-negative distance metric. This objective should be minimized.

number

[0084] where I(x) is the indicator function, x c is the centroid of the frontier x, D2 is the distance threshold, and m q is the location of cell q in the submap M(o) of occupied cells of map M obtained using the current SLAM occupancy map of the environment, where cell q is closest to the frontier centroid, a distance metric calculated on M augmented with the predicted map using Dijkstra's shortest distance (for known / predicted space) and RRT* (for unknown space).

[0085] The kinematics prioritization objective attempts to select frontier targets that result in smoother trajectories for the robot to follow. Kinematics prioritization depends on the robot's drive system. While the effects of changes in direction or speed can be minimized for omnidirectional drive systems (with Mecanum wheels), differential, car-type, 4WD, synchronized drive robots, robots with tracks, and skid / slip maneuvers place significant stress on the motors whenever a change in direction, speed, or acceleration occurs. Selecting a smoother trajectory that aligns with the momentum direction on a continuous field in the robot's local neighborhood can reduce mechanical stress. The resulting path may also require less power. The metric to characterize this objective is the absolute deviation of the direction, measured in 2D, from the robot's current position and heading to the frontier position along the shortest path to the frontier's center of gravity. The scan matching module from SLAM returns a gradient that can also be used for this estimation. The normalized objective is the one to be minimized.

number

[0086] Here, δθ i =θ i+1 -θ i is the change in the travel direction along the shortest path for each step i, θ0 is the initial travel direction, and θ r is the robot's direction of travel at the time of calculation, and θ f is the final direction of travel, and θ x is the direction of travel at the frontier centroid that should be achieved.

[0087] These multiple objectives can be further customized or disabled by the user for a particular use case. As previously mentioned, multi-objective optimization computes a solution, i.e., a frontier x (specified by an index), that minimizes a scalarized form of the objective.

[0088] Dynamic Trajectory Generation - Local Refinement

[0089] As previously mentioned, the position variables obtained from the global waypoint estimation are further refined by local processes that determine local positions and orientations. These local refinement processes are also performed independently at regular intervals to refine the local trajectories between the global waypoints. The timing relationship between the global multi-objective optimization process and the local refinement process is also shown in Figure 2. The exact nature of the refinement in the exemplary embodiment is described below.

[0090] The local refinement process helps to improve the completeness of the local map and is accomplished by two modules: the scan counting module and the next best view module.

[0091] Scan Counting: The Scan Counting module ensures that each structural component located in the map is estimated with sufficient confidence in the observation. In other words, the Scan Counting module ensures that there are a sufficient number of field of view rays to either hit a cell in the map and identify it as an obstacle, or pass through a cell and identify it as an open cell. If the threshold criteria are not met, additional waypoints and directions are sent to the Global Planner to obtain additional scans before continuing at that global waypoint.

[0092] Suboptimal View: The suboptimal view module ensures that the local structure is completely mapped. Local consistency of the map structure is therefore enabled by creating waypoints, paths, and directions that facilitate local consistency. An example of such a trajectory is a circular movement around an object to complete its visual description. This circular movement is performed by creating a convex hull of the visible structure under consideration and moving around the convex hull along a detour path, such that the path distance does not exceed the maximum deviation from the global path to the current frontier target at the deviation point. As the search progresses, the convex hull is updated, and the map is updated using SLAM. If the maximum deviation is reached, a waypoint is generated that returns the robot from the global path to the frontier. On the other hand, if the search around the convex hull is complete, the robot is already back on the global path. Similar to scan counting, these additional waypoints and directions around the convex hull are sent to the global planner, which continues processing the current global waypoint once the visual description of the local structure is complete.

[0093] Search Suite - Static Trajectory Generation

[0094] Several static, user-defined waypoint generation behaviors are also provided as options for trajectory generation. These static, user-defined waypoint generation behaviors include: (1) predefined, user-specified landmarks and / or trajectories that the robot should visit / follow while only avoiding local obstacles; (2) wall following by maintaining a standoff distance from the starting wall and succeeding by looping through interior regions (away from the wall) with only odometry guiding the robot—this behavior is suitable for mapping very sparse, featureless environments; and (3) a spin behavior for extremely rapid mapping of small rooms with minimal clutter.

[0095] Such static trajectories can be used in place of dynamic trajectory generation in certain use cases, and the targets / waypoints generated from these user-defined trajectories are sent to the global planning component for further processing.

[0096] Discovery Suite - Predictive Mapping

[0097] Predictive mapping has been proposed as a technique to enforce the "map consistency" objective, where the goal is to be able to predict what the environment will look like beyond a selected frontier, so that it is possible to determine whether loop closure can be triggered by taking a path through that frontier and finding a route back to a previously observed section of the environment with stable and reliable landmarks.

[0098] When operating in industrial spaces or other large environments, robots are likely to encounter similar or repetitive structures. This situation is particularly true for warehouse aisles and docks. A generative adversarial network (GAN) with generative and discriminative components is used to train on scenes in previously visited environments and predict the next possible structural shape that will complete the map in the robot's path. For example, a predicted shape in a warehouse may generate a structure with a loopback path at the end of an aisle. The generator generates map sections using the SLAM instantaneous map and previous sections of the map during the training phase, and the discriminator is trained to identify, in an unsupervised manner, whether such a generated map differs from the previous full map. This architecture is shown in Figure 4.

[0099] With successive rounds of backpropagation training, the generator becomes better at creating maps that the classifier cannot flag as different from previous maps, while the classifier becomes better at flagging such maps. During inference, the SLAM instantaneous maps are used to generate predicted maps beyond the frontier. GAN predictions of likely loop closures that allow for structures in specific directions downstream from the robot's current position (determined by finding the shortest path to known map points using Dijkstra) are used as the objective of the optimization problem when loop closures mitigate declining localization scores or increasing entropy. GAN prediction directions that do not result in loop closures are ignored in such situations.

[0100] Note that the training process of a GAN can be offline (based on previous maps in a similar domain / environment, such as an industrial warehouse or supermarket) and is performed online, potentially requiring domain adaptation to the specific use-case environment. Domain adaptation requires additional computation. Inference, on the other hand, is computationally inexpensive due to the local nature of predictions. Furthermore, computational complexity is kept low when training and inference are performed using a simple binarized representation of the SLAM-generated maps.

[0101] This novel approach is referred to in this disclosure as "predictive mapping," and as previously mentioned, the predictive mapping module aids in loop closure. If the entropy exceeds a threshold, indicating a potential for increased localization error or error uncertainty, an inference phase is applied to enable the robot to track a path that leads to loop closure, at which point such errors can be reduced or eliminated from the system. This mechanism is also illustrated in FIG. 1, with corresponding timing behavior shown in FIG. 2, and FIG. 5 shows an implementation in an exemplary GAN configuration for loop closure prediction, i.e., the predictive mapping module.

[0102] Search Suite - Stopping Criteria

[0103] Various stopping criteria for the exploration performed by the automatic map creator are also provided. An exemplary set of such criteria includes any one or more of: physical boundaries set a priori by the user, virtual boundaries set a priori by the user using a user interface, spatial scaffolding (user marking of boundary limits for the exploration), parameterized definitions of frontiers and scaffold structures, frontier size limits (minimum and maximum values), exploration time limits, drift distance limits (distance from the base that the robot can travel), threshold limits (whether the robot can navigate to a new room through a potential doorway), and source-sink behavior (stopping exploration when the robot reaches a sink target that is a landmark target such as a dock or an image target).

[0104] Wide-area route planning

[0105] The key component of the global path planning module is a shortest path estimator based on the Dijkstra algorithm, which is used here to find the shortest path toward specified goals or waypoints while avoiding obstacles in the SLAM map. Details of how these goals are specified are explained in the search suite section. Unlike the distance calculation in the search suite, which uses Dijkstra algorithms on SLAM-generated maps, the global path planning module uses additional constraints for shortest path calculation specified in the form of cost maps. While calculating the global path, it uses four layers of cost maps projected onto a single 2D cost map to characterize obstacles that need to be considered. For an example of a common cost map, see Wang, X., Mizukami, Y., Tada, M., & Matsuno, F. (2021), “Navigation of a mobile robot in a dynamic environment using a point cloud map,” Artificial Life and Robotics, 26(1), 10-20.

[0106] The first of the four layers is a 2D "static" occupancy grid layer created from the SLAM-generated map (which is also dynamically updated as part of the mapping operation). Obstacles in this layer are implicit and must be avoided by the global path planner.

[0107] The second layer is a "dynamic" 3D voxel grid layer created from point cloud data from multiple ranging and other landmark sensors placed at different heights on the robot. This dynamic layer is periodically cleared to allow for transients in the scene content, such as a human walking through the environment, while taking into account sensor characteristics such as reliability, frame rate, and noise performance.

[0108] The third layer is a 2D "inflation" layer that grades the distance to obstacles on the curve. In other words, a higher cost is associated with cells in the cost map that are closer to the obstacle than cells that are farther away. This forces the robot to stay away from obstacles while allowing it to approach them when it cannot find a minimum-cost path with a large standoff, or clearance. In such cases, only high-cost paths with small clearances are available, and they are executed slowly by the local planner, reducing the chance of collisions and e-stops.

[0109] The final layer is a 2D "attractor" layer that allows the robot to "hugging" obstacles during navigation (for wall-following behavior). The attractor layer prevents the robot from getting lost in large open areas, selectively acts on large obstacle structures such as walls, and forces the robot to stay within the bounds of the environment. The visibility of the current environment and the correspondence between the known SLAM map and the current environment are estimated by the map entropy metric. If the map entropy metric has a high value, i.e., indicating a lack of correspondence between the observed environment and the current state of the map, the global planner uses the 2D attractor layer to reweight the cost map to select a path that prevents or reduces complete loss of localization during the exploration process.

[0110] The static layer is the most important component for global path planning, while the dynamic layer is most important for local path planning behavior. The inflation layer allows the robot to approach obstacles at a slower speed, thereby allowing for suboptimal paths when an optimal path cannot be found. The attractor layer ensures that the robot is not completely lost during the search process. Thus, the inflation and attractor layers work in parallel for large structural obstacles in the environment.

[0111] Local Path Planning

[0112] The local path planning module is the component responsible for generating potentially kinematically feasible behavior while avoiding collisions. The local path planning module performs calculations within a limited spatial range around the robot's current position and establishes a local trajectory using a polynomial model-based particle simulation. The goal of the local planner is to enable the robot to follow a path as close as possible to the global path, while simultaneously avoiding local obstacles, while generating a simplified path that can be actuated by the robot's differential drive motors.

[0113] Two possible conventional algorithms are incorporated here: dynamic window approach (DWA) and trajectory rollout (TR). Generally, TR provides better results by searching because it samples from the set of achievable velocities over the entire forward simulation period, as opposed to DWA, which samples only for one simulation step. Both algorithms work by forward simulating particle trajectories along a polynomial curve and determining whether a collision occurs at each step of the simulation. A search for the most optimal collision-free path (via a cost map) is performed, and such a path is selected as the trajectory for deployment and implemented using a velocity controller.

[0114] Recovery Behavior for Local Path Planning

[0115] To prevent the robot from potentially getting stuck in a cluttered environment, several recovery behaviors are also implemented, including, for example, modifying the cost map and incorporating spin, backtracking, and U-turn behaviors.

[0116] Exemplary cost map modifications include: (1) periodically clearing the dynamic cost map (note that the dynamic cost map layer is most severely affected by transients in the environment (such as a human or group of humans in the robot's path) that may obstruct the robot and prevent it from following a path; as a result, periodically clearing the dynamic cost map serves to periodically remove these transient obstacles from the cost map after they move out of the obstructing field of view), (2) adjusting the inflation layer gradient weights (the inflation layer gradient weights in a local neighborhood are given decreased weights to allow additional paths that allow the robot to maneuver around obstacles, including paths that may bring the robot closer to known obstacles), and (3) disabling the attraction layer (when the robot is maneuvering around an obstacle, the attraction layer is disabled to avoid further motion toward the obstacle).

[0117] Exemplary local planning behaviors include any one or more of the following: (1) a spin behavior, useful when the robot is obstructed in one or more directions but has at least one approach vector from its current position (using the spin behavior, the robot can find an exit direction and exit the obstructed area); (2) a backtracking behavior, useful when there is loss of localization or significant sensor failure—in such cases, the robot rewinds its trajectory, i.e., returns along the path it took to reach its current position to improve localization accuracy, before attempting an alternate route to the goal waypoint that avoids the obstructed area; and (3) a U-turn behavior, which requires the robot to turn in place and drive to reach a previously visited waypoint before continuing toward the goal (while backtracking assumes that the robot wheels are rotated in the opposite direction, the U-turn behavior requires the robot to turn in place, and is suitable for obstacle avoidance over longer distances).

[0118] Combinations of the aforementioned recovery approaches are used in different situations, guided by sensor observations of the current state of the environment. Spin behavior is the most efficient and therefore the most frequently used.

[0119] Motion (Speed) Controller

[0120] A velocity controller is incorporated into the automatic map generator system to convert the path selected by the local path planner into velocity commands. The standard velocity-acceleration-distance relationship is given by:

number

[0121] SLAM for map generation

[0122] The online SLAM backend is used to generate and update maps for representing the current state of the environment. One or more embodiments use a filtering approach rather than a graph-based approach for SLAM, and these approaches are implemented to support the online mapping mode. Furthermore, predictive mapping leverages loop closure in SLAM. Also, occupancy maps generated by SLAM are used to identify frontiers and plan global / local paths. Scan matching in SLAM can also potentially provide gradient directions for kinematic prioritization. Therefore, this module operates continuously and in parallel with the search suite and other planning and actuation modules.

[0123] User Interface

[0124] The automatic mapping system also provides for user interaction in the form of four user interfaces that are active at various points in the execution of the system.

[0125] The four user interfaces include (1) search customization, (2) live feedback and search control, (3) offline map feedback and map editing tools, and (4) an interface for the analytics and recommendation system.

[0126] Search Customization: This interface provides customization of search parameters and constraints before execution of a search run.

[0127] Live Feedback and Search Control: The interface provides the user with live feedback on the search process during search execution, allowing the user to monitor and control the execution.

[0128] Offline Map Feedback and Map Editing Tools: Once a search run is complete, this interface displays the final map to the user and allows the user to edit the final map.

[0129] Analysis and Recommendation System: This interface presents an overview of the search run, provides an analysis and evaluation of key metrics from the search run, and includes a recommendation system that advises the user on customization / adaptation of future search runs.

[0130] FIG. 6 shows a state diagram of the search associated with the aforementioned interfaces.

[0131] User Interface - Customizing Exploration

[0132] The exploration customization interface presents various options that the user can select / adjust before initiating the automatic mapping process. These options include termination criteria for any one or more of: search time limits, wander distance (how far from the base the robot is allowed to move), frontier size limits (minimum and maximum), threshold limits (whether the robot is allowed to navigate into new rooms through potential doorways), spatial scaffolding (user marking of boundary limits for exploration), and source-sink behavior (the robot stops exploring when it reaches a sink target (a landmark target such as a dock or image target)). In other embodiments, additional or alternative criteria may be specified.

[0133] The search customization interface also gives users the option to select different static path behaviors, such as wall following (which succeeds by looping through interior areas using odometry alone) for operation in very sparse and featureless environments, and spin behavior for extremely fast mapping of small rooms with minimal clutter.

[0134] The user interface is also designed to support various parameter tuning options for safely exploring unpredictable real-world environments. A user interface option for fixed trajectories is also presented, along with adaptations for sparsely populated, feature-free environments. From a search strategy perspective, the user interface also presents options such as the minimum frontier dimension to consider, the distance to the frontier, and the frontier priority. Weight and parameter options for multi-objective global optimization and local refinement are also presented. A simplified diagram of an exemplary interface for user customization is shown in Figure 7.

[0135] It should be noted that the "interface" discussed here is realized by the underlying processing circuitry that generates display data, receives user input, etc., and the display may be contained on the robot or may be external, such as contained on a laptop computer or other computing system. The program instructions providing the user interface may be executed partially or wholly on the robot, for example, with an attached laptop or other display system used as a terminal, or an attached computer may execute code to provide the visual display, with the automatic map generator executing complementary computer code to transmit current settings, receive adjusted settings, etc.

[0136] User Interface - Live Feedback and Search Control

[0137] The user feedback interface provides real-time feedback during the automatic mapping process. In addition to providing a current map of the environment in the form of an occupancy grid, the real-time interface displays the robot's current position, obstacles detected by onboard sensing such as laser or sonar, local and global cost maps for path planning, the locally planned path, the globally planned path, and, most importantly, the frontiers that the automatic mapping will explore. These frontiers are ranked by the priority with which they should be visited. Blacklisted frontiers are indicated by a unique color. The interface also provides the option to reorder or increase or decrease the priority with which a frontier is visited (by clicking the corresponding button in the screen display). Furthermore, the user can blacklist or whitelist a frontier by clicking a checkbox. Active / inactive frontiers (manually or automatically blacklisted via the blacklisting table) are also presented to the user. An automatically blacklisted frontier may become active again after the blacklisting period expires.

[0138] Prompts are also available to allow the user to abort exploration when they anticipate dangerous or unsafe operations on the robot's part. Prompts are also useful when the robot enters an area that should not be mapped by the robot or takes a path that should not be navigated. In such cases, setting up spatial scaffolds or barriers is anticipated, at which point the user can resume mapping by selecting the prompt again. Furthermore, if the user determines that the required mapping coverage of the environment has been obtained and is sufficient for navigation during task phase operations, a prompt can be used to terminate the automatic mapping process and complete map generation. A prompt is also provided to roll back the most recent trajectory traveled by the robot for mapping. Similar to the abort prompt, this prompt is useful when the robot enters an area or takes a path through such an environment that should not be present in the final map. The rollback trajectory prompt allows the user to roll back any number of paths starting from the most recent path. During a rollback operation, the robot repeatedly backtracks along the paths to reach the starting positions of these paths. Map sections added during these paths are also removed from the current map. A simplified representation of the user feedback interface is shown in Figure 8.

[0139] User Interface - Offline Map Feedback and Map Editing Tools

[0140] Providing a manual map editing tool through the user interface serves at least two purposes. First, maps generated using an automated map generator can be refined using a manual map editing process. As part of the map editing process, any sections missed by the automated search can be added. The map editing tool allows the user to specify the boundaries of any missed sections. The boundaries are defined in terms of polygon vertices specified by clicking on the map visualization interface. Furthermore, sections of an environment mapped independently by different robots, regardless of whether they overlap, can be merged using the map editing tool. The map editing tool provides options for translating and rotating maps to bring them into the same global coordinate system and aligning various structures within the map. Map merging is also a useful feature for utilizing maps generated at different times. For example, maps corresponding to large environments may be mapped once, while particularly busy rooms may need to be remapped periodically. The map editing process helps overlay more recent versions of rooms on top of an existing base map and align them with existing structures, thus eliminating the need for a complete remapping of the environment.

[0141] Additionally, as improved maps become available, they can be used to evaluate and adjust the performance of the automated map generator. In this situation, the improved maps serve as ground truth against which maps from the automated map generator are compared to determine coverage of the mapped environment. For example, if a bottleneck corresponding to a minimum size is identified in an unexplored area relative to the ground truth (such as the entrance to a narrow hallway), the minimum size of the wavefront to be searched can be reduced.

[0142] User Interface - Analysis and Recommendation Systems

[0143] In addition to the feedback available in the form of compiled maps, in one or more embodiments, the following measurements and state variables are obtained / calculated to generate evaluation criteria, profiles for operation, and statistical measurements for search analysis and optimization as an offline process: Based on these values ​​presented to the user via the user interface, additional search runs in the same or similar environments can be appropriately pre-configured by the user according to the recommendations made for each metric.

[0144] The automated mapping analysis user interface provides access to any one or more of the following metrics, broadly referred to herein as "exploratory analysis," including power usage, power consumption, coverage rate, boundary detection rate, entropy, odometry xy, odometry theta, twist linearity, twist angle, global plan trigger, global plan neighborhood trigger, local plan neighborhood trigger, target xy, control target acceptance delay, command velocity linearity, command velocity angle, control feedback pose xy, control feedback pose theta, and frontier counts. Details of such exploratory analysis are provided below.

[0145] Power Usage - This metric / curve represents power usage versus time / coverage as measured from the robot battery interface throughout a search run. This metric helps minimize battery usage until the search convergence point. The robot may spend a significant amount of time in the region of 90%-100% coverage, so if the user wants to reduce battery usage, they may want to adapt the search convergence / end point (time, distance or frontier distance, and size stopping criteria).

[0146] Power Consumption - Similar to the power usage metric, power consumption is measured from the robot battery interface throughout the exploration run. Prolonged periods of high power consumption (i.e., sudden voltage drops) can have a negative impact on battery life and may need to be avoided. Users may want to appropriately set kinematic prioritization (momentum) weights in the optimization problem to reduce sharp turns and obtain the desired behavior.

[0147] Coverage rate - Coverage rate is the primary evaluation modality for exploration. Coverage rate is the area (m 2 ) / pixels. This is a metric that can be measured over time and against which other tunable parameters can be benchmarked. If the final coverage rate is low, the user may want to relax the stopping criteria. Ideally, coverage should approach 100%.

[0148] Boundary Detection Rate - Boundary detection rate is calculated as the percentage of detected boundary pixels relative to the ground truth map. Boundary detection rate is measured over time and provides a complementary metric to coverage rate. This metric is also desired to be close to 100%.

[0149] Entropy - As mentioned above, entropy increases when unmapped regions are discovered and decreases with mapping. If the robot spends excessive time in exploitation mode (selecting frontiers that require traversing known space) or looping back, the user can increase the entropy threshold to allow faster exploration.

[0150] Odometry x,y - This is recorded using the Robot Odometry Interface. This helps identify the boundaries of the explored space and trends in the time spent by the robot in mapping each region. This is useful for adjusting scaffold variables (extent of space to map).

[0151] Similar to odometry theta (vi). This helps identify a typical robot's tendency to move towards a given search space. It also identifies the number of turns the robot will make while exploring. This can guide the user to modify weights for kinematic prioritization.

[0152] Torsional Linear - The curve of this variable is derived from the commanded velocity. This metric helps ensure that the robot adheres to maximum linear velocity limits, and also identifies typical velocities and the time spent at those velocities, as well as the evolution of the velocity profile in a given space. This can be useful when the user wants to adjust parameters in a space with humans and other dynamic obstacles to ensure a higher degree of safety and predictable behavior.

[0153] Twist Angle - Twist angle ensures that the robot adheres to maximum angular velocity limits while turning, and also helps identify typical speeds and the time spent at these speeds, as well as how the speed profile changes and how often the robot turns in a given space. User adjustments are the same as above, but angular twist provides a better indication of dynamic obstacles than linear twist.

[0154] Regional Plan Triggers - This timing data comes from the regional path planner. This metric, along with the length of each regional plan, helps identify when a new regional path plan is available. This also indicates how far the frontier is. The regional plan trigger should be synchronized with the availability of new frontiers, which helps optimize and keep the number of additional regional plan triggers for each frontier goal low, thus reducing computational costs.

[0155] Global Plan Neighborhood Trigger - This timing data comes from the local path planner. This metric helps identify when a new global plan neighborhood is available (for the local path plan to follow), and indicates the length of the new global plan neighborhood; a longer length is preferable as it allows the robot to move faster, and a messy plan indicates infeasible global planning and re-planning, and therefore a higher computational load on the robot. A messy plan indicates a tight and dynamic space. Users may want to adapt the cost map weights (especially the inflation layer) to allow for tight passes and tight turns (perhaps at lower speeds) in these situations.

[0156] Local Plan Neighborhood Trigger - This timing data is obtained from the local path planner. The local plan neighborhood trigger helps identify when a new local plan neighborhood is available and indicates the length of the local plan neighborhood. Longer lengths are preferable as they allow the robot to move faster, and messy plans indicate infeasible local plans and replans. Similar considerations apply as for global planning, but this variable is more highly affected by dynamic obstacles.

[0157] Targets x,y - This timing data is obtained from the exploration suite. Targets x,y indicate when and where new frontier targets are available and how much time the robot should spend in each section of the environment mapping the frontier (i.e., unexplored area).

[0158] Control Target Acceptance Delay - This timing data is obtained from the motion controller. This metric indicates complex global and local paths, and therefore infeasible planned paths and replans in terms of the number of attempts before a feasible path is found. If this metric becomes too large, the user may want to limit the number of attempts.

[0159] Commanded Velocity Linear - The commanded velocity linear is obtained from the motion controller and specifies the commanded linear velocity sent by the velocity controller to the motor controller, ensuring the robot adheres to the maximum velocity bounds, with cluttered points indicating higher stress on the motors. Users may want to reduce the maximum local planning rate and robot velocity to reduce stress on the robot motors.

[0160] Commanded Velocity Angle - The commanded velocity angle is also obtained from the motion controller and determines the commanded angular velocity sent by the velocity controller to the motor controller, ensuring the robot adheres to the maximum velocity bounds, with cluttered points indicating higher stress on the motor. User adjustments are similar to the linear case.

[0161] Control Feedback Pose x,y - The control feedback pose x,y is obtained from the motor encoder data and identifies the robot pose (position) estimated by the controller and should be close to the odometry x,y. Any difference from the commanded pose may indicate slippage, and the user may want to adjust the SLAM odometry noise model and / or modify floor conditions (if possible) to address errors resulting from such deviations.

[0162] Control Feedback Pose Theta - The control feedback pose theta is obtained from the motor encoder data and specifies the robot pose (heading) estimated by the controller, and should be close to the odometry theta. User corrections are the same as above.

[0163] Frontier Count - This metric is obtained from the search suite. The frontier count gives the number of frontier pixels and identifies when a new frontier is available (by large values) as the robot reaches a new, unexplored room or other space; the typical trend is that it should asymptotically approach 0 over time as the robot's coverage of the environment increases. This variable can be useful to users in determining the maximum frontier size to be explored in a particular environment, and therefore the associated stopping criteria.

[0164] 9 illustrates a mobile robot 10 according to an exemplary embodiment, with its constituent entities or subassemblies including a control system 12 that provides a runtime environment 14 in which an automatic map creator 16 is instantiated and executed. Thus, the robot 10 executing the automatic map creator 16 can be considered an exemplary automatic mapping system, with the automatic map creator 16 configured or operating according to any of the "automatic map creator" embodiments described above.

[0165] The term "runtime environment" refers to a program execution environment provided by the control system 12, e.g., one or more microprocessors and corresponding memory, and may be configured or controlled by an operating system (OS), such as a real-time OS (RTOS), running on the robot 10.

[0166] The robot 10 further comprises a drive system 18, one or more sensor systems 20, one or more interfaces 22, and one or more power systems 24. The control system 12 interfaces or otherwise interacts with these various entities for the overall control and operation of the robot 10. In this regard, the control system 12 may comprise hardwired or programmable circuitry, or a mixture of both. In at least one embodiment, the control system comprises processing circuitry for communicating with or otherwise interacting with the various components onboard the robot 10, such as the drive system 18, the sensor system 20, etc. In at least one example, such processing circuitry comprises or includes one or more microprocessors or other digital processors specially adapted for the execution of computer program instructions stored on a computer-readable medium, along with supporting input / output circuitry.

[0167] 10 illustrates an exemplary embodiment of drive system 18, including one or more motors 30 that power drive wheels 32. Although not explicitly shown in the figure, robot 10 may include motor control circuitry that converts motion commands generated by processing circuitry in control system 12 into corresponding motor drive signals. Such motor control circuitry may be considered as part of control system 12, part of drive system 18, or as a bridge circuit between control system 12 and drive system 18. Drive system 18 further includes one or more encoders 34 that provide odometry feedback from the drive wheels for dead reckoning calculations performed by control system 12.

[0168] FIG. 11 illustrates an exemplary embodiment of a sensor system 20, which includes any one or more LIDAR devices 40, one or more ultrasonic devices 42, or one or more cameras 44. These sensors enable the robot 10 to scan or otherwise sense the physical environment around the robot 10, including obstacles, structures, etc., for automated navigation and mapping within the environment. Thus, the sensors may be referred to as “environmental” sensors. For example, a LIDAR device 40, such as a laser scanner, sweeps a laser pulse across or through a range of angles in a horizontal plane, with the distance of the reflecting object being determined based on the time of flight of the pulse reflection received by the laser scanner. In one or more embodiments, the robot 10 has multiple laser scanners positioned at different heights to more robustly and reliably detect objects in the surrounding physical environment.

[0169] 12 illustrates an example of an interface system 22 including a communications interface circuit 50 for wired or wireless communications. In this exemplary depiction, the communications interface circuit 50 comprises a first transmitter / receiver circuit including a first transmitter circuit 52-2 and a first receiver circuit 54-1 for wired communications, e.g., an Ethernet connection or other computer data connection. In this regard, the communications interface circuit 50 may be understood as providing physical layer transmitter and receiver circuitry, e.g., along with a protocol processor for implementing one or more communications protocol stacks. Alternatively, the communications interface circuit 50 may provide a physical layer signaling connection, with processing circuitry within the control system 12 providing protocol processing.

[0170] In an exemplary embodiment, the transmit circuitry 52-1 and the receive circuitry 54-1 provide a communications link for coupling to a laptop or other external computing device running software that provides access to the user interface customization features provided by the automatic map generator 16.

[0171] In at least one embodiment, the communication interface circuitry 50 includes a wireless communication interface with a second transceiver including a transmit circuitry 52-2 and a receive circuitry 54-2 that interfaces to one or more antennas 58 via an antenna interface circuitry 56. Such circuitry may implement a Wi-Fi or other wireless communication link and be used for command and control of the robot 10 during live operation, and in at least one embodiment, the wireless interface provides communication access for user interface-based customization of the exploration behavior exhibited by the automatic map creator 16.

[0172] The interface system 22 may further include a control panel, such as a touchscreen or other local user interface (UI) mounted on the robot 10, that allows, for example, an operator to check status, input commands, etc. FIG. 13 shows an exemplary local UI 60 along with an effector 62, such as may be attached to the robot arm. Of course, the physical characterization of the robot 10 will depend on its intended purpose, e.g., to the extent that it is not dedicated to automated mapping operations. For example, the robot 10 may be equipped with a loading platform for material transportation, etc.

[0173] 14 shows exemplary implementation details of control system 12. Processing circuitry 70 implements at least a portion of the processing logic that realizes automatic map generator 16, and comprises, for example, one or more microprocessors 72 that include or are associated with memory 74 that stores one or more computer programs 76 and supporting configuration data 78. Computer programs 76 include computer program instructions that, when executed by microprocessor 72, specially adapt microprocessor 72 to perform processing operations that achieve automatic map generator 16.

[0174] Storage device 74 includes one or more types of computer-readable media and provides non-transitory storage of program instructions and data. "Non-transitory" does not necessarily mean permanent or unchanging, but rather refers to storage with at least some degree of persistence; storage device 74 may include volatile memory or a mix of working and non-volatile memory; exemplary circuits, devices, or elements include any one or more of SRAM, DRAM, FLASH, EEPROM, solid-state disks (SSDs), etc.

[0175] Microprocessor 72 interfaces with one or more motor controllers 80 that provide drive signals to motors 30 in drive system 18, and with sensor system 20 via sensor interface 82. Such interfaces may comprise data bus-based interfaces for exchanging sensor control signaling and sensor data that may be preprocessed by sensor system 20. Additionally or alternatively, such interfaces may include discrete analog and / or digital input / output (I / O). Furthermore, the term "microprocessor" has a broad meaning and encompasses various implementations or types, such as microcontrollers, digital signal processors (DSPs), application specific integrated circuits (ASICs) or field programmable gate array (FPGA) cores, system-on-chip (SoC) variants, etc.

[0176] 15 illustrates an exemplary configuration in which the robot 10 implements the automated map creator 16 described herein. An external computer 90 is coupled to a wireless local area network (WLAN) 92, which in turn communicatively couples the external computer 90 to the robot 10.

[0177] The robot configuration software 100 executing on the external computer 90 provides a user interface 102 for user customization of the exploration behavior of the automatic map creator 16, as described above in the user interface section of this specification. Here, the robot configuration software 100 implements the user interface 102 or provides a terminal function whereby a configuration function 104 implemented by the automatic map creator 16 onboard the robot 10 displays the user interface 102 on the external computer 90. In either implementation, the configuration function 104, also referred to as a publishing function, is configured to use modifiable values ​​of the operating parameters driving the exploration behavior of the automatic map creator 16 for user inspection and adjustment. For example, the configuration function 104 uses stored values ​​that are determined, modified, prioritized, or weighted based on signaling received through the user interface or derived from user input received via the user interface. Alternatively, the configuration function 104 retrieves values ​​from a configuration file or data structure loaded into memory or other computer-readable medium included in the mobile robot.

[0178] As an example of user customization, the external computer 90 communicates multi-objective preferences 106 to the automatic map generator 16. These preferences enable / disable each of the multiple optimization objectives described above, or assign relative priorities or weights that control how the multi-objective optimizer performs simultaneous optimization of multiple objectives. Figure 16 illustrates such a configuration, where several search objectives considered in the multi-objective optimizer each have configured weights or priorities that can be selected or adjusted by the user to adjust the search behavior of the automatic map generator 16, for example, to emphasize map consistency or to emphasize search speed.

[0179] Figure 17 illustrates a method 1700 of automatic mapping performed by the mobile robot 10, according to an exemplary embodiment. For example, the processing circuitry 70 of the mobile robot 10 is configured to implement the operations included in Figure 17. Figure 18 illustrates a method 1800, which can be understood to provide exemplary details of method 1700.

[0180] A further method, in one embodiment, includes a method for customizing exploration behavior of a mobile robot performing mapping using the automated map generator described herein. The method includes displaying a user interface corresponding to a physical environment to be mapped by the mobile robot, where the mobile robot instantiates the automated map generator for exploration of the physical environment based on selecting a frontier for exploration according to a multi-objective optimization that considers two or more types of search objectives. The method further includes, in response to receiving corresponding user input via the user interface, performing any one or more of: limiting the search space and mapping region; prioritizing the mapping region; selecting termination criteria and bounding strategies; selecting search strategies, constraints, and parameters; selecting a static trajectory; customizing relative weights or priorities for multiple search objectives; or specifying safety constraints.

[0181] A computer-implemented method for providing live feedback and exploration process control for a mobile robot according to one embodiment includes visualizing a map generated by a mobile robot operating in accordance with the automated map creator details described herein, the visualization depicting any one or more of obstacles, robot position, planned global and local paths, cost maps, and frontiers, frontier priorities, and blacklisting status. The method further includes providing user input controls for changing the frontier priorities and blacklisting status, aborting or stopping the exploration process in response to the robot performing an unsafe maneuver, and rolling back a variable number of trajectories in response to an erroneous mapping.

[0182] A computer-implemented method for map editing according to one embodiment includes providing a user interface that allows a user to complete or modify a map as generated by a mobile robot according to the automated mapping described herein, and providing a user with an option to merge multiple maps generated from multiple search runs by the mobile robot.

[0183] In at least one embodiment, the automated map generator includes a method for computing a search analysis that includes one or more of performing an evaluation of previous search runs, providing an output including feedback to the user in the form of a recommendation system, and providing an output including advice to the user for customizing and adapting future search or remapping runs.

[0184] A computer-implemented user interface method according to one embodiment includes, for a search by a mobile robot according to the automatic mapping described herein, performing one or more of presenting an overview of the search execution by the mobile robot, presenting search analysis and key metrics from the search execution, and recommending fine-tuning of future search execution by the mobile robot.

[0185] In at least one embodiment, the automated map generator described herein is applied to the use case of automatic remapping of changed sections in an environment with respect to a previous map of the environment obtained manually or using the automated map generator. Additionally, in the same or another embodiment, the automated map generator described herein is applied to the use case of exploratory navigation to discover known targets in unknown locations in the environment.

[0186] In particular, modifications and other embodiments of the disclosed invention will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. It is to be understood, therefore, that the invention is not limited to the particular embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the present disclosure. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. 1. A method for automated mapping by a mobile robot, comprising: using a Simultaneous Localization and Mapping (SLAM) algorithm to identify a next frontier to explore from among two or more candidate frontiers in a map generated by the mobile robot, the identification being based on performing multi-objective optimization that simultaneously evaluates a set of two or more search objectives and determining the attractiveness of each candidate frontier as a next search target, each of the two or more search objectives corresponding to a different search behavior; planning a global route to a global waypoint on the next frontier; navigating to the global waypoint according to the global route while taking into account collision avoidance and path parameters determined by local path planning performed based on environmental sensors onboard the mobile robot; A method comprising:

2. 2. The method of claim 1, wherein the set of two or more exploration objectives includes one or more mapping speed objectives and at least one of a mapping coverage objective or a mapping consistency objective, wherein the mapping speed objective causes the exploration behavior of the mobile robot to emphasize mapping speed, the mapping coverage objective causes the exploration behavior of the mobile robot to emphasize mapping completeness, and the mapping consistency objective causes the exploration behavior of the mobile robot to emphasize mapping accuracy and precision.

3. receiving a signaling signal and, in response to the received signaling, determining importance weights that adjust which two or more search objectives from a defined set of search objectives are considered in the multi-objective optimization or modify the prioritization of individual search objectives among the two or more search objectives considered in the multi-objective optimization; The method of claim 1 or 2, further comprising:

4. The method of claim 3 , wherein the signaling is received in connection with exchanging signaling with an external computer that displays a user interface for a user to customize automated mapping behavior by the mobile robot.

5. The method of any one of claims 1 to 4, further comprising applying user-configured weights to at least one of the two or more search objectives to adjust the multi-objective optimization according to user preferences.

6. The two or more search objectives are: a distance-to-target objective indicating the shortest path distance from the current position of the mobile robot to the corresponding candidate frontier; a searchability objective indicating the searchability associated with the corresponding candidate frontier; a map coverage objective indicating the extent to which exploration of the corresponding candidate frontier improves coverage of the physical environment by the mobile robot; a map consistency objective indicating the extent to which exploration of the corresponding candidate frontier leverages previously explored regions of the physical environment; and a kinematic objective indicating trajectory smoothness for moving from the current position of the mobile robot to a corresponding global waypoint defined on the candidate frontier; The method of any one of claims 1 to 5, comprising two or more of:

7. The method of claim 6 , wherein the kinematic objective deprioritizes paths that are likely to require deviation from the mobile robot's current heading.

8. 8. The method of claim 1, wherein performing the multi-objective optimization comprises forming and evaluating a scalarized objective from the two or more objectives for all candidate frontiers.

9. 9. The method of claim 1, wherein one of the search objectives is a map consistency objective that aims to improve the consistency of an online map generated by the mobile robot, and map consistency is measured using an entropy metric that reflects the consistency of the online map over successive time steps of the SLAM algorithm.

10. 10. The method of claim 9, wherein the map consistency objective is represented by multiple sub-objectives, including a structure adjacency sub-objective that prioritizes frontier goals adjacent to visible structures, an exploitation sub-objective that prioritizes movement by the mobile robot through previously visited or known areas of the physical environment, and a predictive closure sub-objective that prioritizes candidate frontiers that are predicted to return the mobile robot to known areas containing stable landmarks.

11. 11. The method of claim 10, wherein the predictive closure sub-objective predicts and enforces the closure of a physical loop by the mobile robot through the physical environment during the exploration process based on predicting what the physical environment will look like beyond a candidate frontier and determining whether loop closure is possible by navigating through the candidate frontier using a generative adversarial network (GAN) with training and exploration phases of behavior.

12. 12. The method of claim 1, further comprising: performing a local refinement process, the local refinement process comprising modifying the trajectory to the global waypoints by adding waypoints, allowing the mobile robot to capture local map structures from various viewpoints to improve the completeness of the local map.

13. 13. The method of any one of claims 1 to 12, further comprising using a time-based blacklisting table to deactivate search targets that do not have a feasible estimated path from the current position of the mobile robot or for which execution of a valid estimated path results in repeated failures, the deactivation ending after a certain time period after which the search target is reconsidered by the mobile robot for searching.

14. 14. The method of claim 1, further comprising applying stopping criteria to exploration by the mobile robot, the stopping criteria comprising one or more of physical boundaries, virtual boundaries, frontier limits, search time, wander distance, threshold limits, or source-sink behavior.

15. 15. The method of any one of claims 1 to 14, further comprising using one or more methods of static trajectory generation including any one or more of using predefined user-specified landmarks, using connecting trajectories, using wall-following behavior, and using spin behavior.

16. 16. The method of claim 1, wherein planning the global route to the global waypoint on the next frontier imposes multiple constraints on shortest path calculation for the global waypoint according to a multi-layer cost map.

17. The multi-layer cost map may be a first layer including a two-dimensional (2D) static occupancy grid created by the mobile robot performing a simultaneous localization and mapping (SLAM) process, wherein object avoidance is mandatory for obstacles registered in the first layer; a second layer including a dynamic three-dimensional (3D) voxel grid created from point cloud data generated using multiple environmental sensors mounted on the mobile robot and positioned at different heights, the second layer being periodically refreshed to account for temporary obstacles within the mobile robot's working envelope; a third layer including a 2D inflation layer that assigns higher costs to navigation through cells closer to obstacles and lower costs to cells further away within the first and second layers of the multi-layer cost map; a fourth layer including a 2D attractor layer that urges the mobile robot to carve a path along walls or other detected structures in the physical environment when robot position estimates and map consistency degrade; 17. The method of claim 16, comprising:

18. 18. The method of claim 16 or 17, wherein the global path planning is further processed by a local planner that performs collision avoidance and generates kinematically feasible paths, and also uses recovery behaviors including one or both of cost map modifications and local recovery behaviors, the local recovery behaviors including one or more of backtracking, spins, and U-turns to push the mobile robot out of a stuck state.

19. A mobile robot, a drive system configured to move the mobile robot within a physical environment; one or more sensors configured to detect obstacles in the physical environment within corresponding detection ranges of the sensors; a processing circuit configured to process sensor data from the one or more sensors and perform the automatic mapping based on controlling the drive system; The processing circuitry comprises: using a Simultaneous Localization and Mapping (SLAM) algorithm to identify a next frontier to explore from among the two or more candidate frontiers, the identification being based on performing multi-objective optimization that simultaneously evaluates a set of two or more search objectives and determining the attractiveness of each of the two or more candidate frontiers in the map generated by the mobile robot as a next search target, each of the two or more search objectives corresponding to a different search behavior; planning a global route to a global waypoint on said next frontier; navigating to the global waypoint according to the global route while taking into account collision avoidance and route parameters determined by local route planning performed based on the one or more sensors; Performs automatic mapping by configuring Mobile robot.

20. 20. The mobile robot of claim 19, wherein the set of two or more exploration objectives includes one or more mapping speed objectives and at least one of a mapping coverage objective or a mapping consistency objective, wherein the mapping speed objective causes the exploration behavior of the mobile robot to emphasize mapping speed, the mapping coverage objective causes the exploration behavior of the mobile robot to emphasize mapping completeness, and the mapping consistency objective causes the exploration behavior of the mobile robot to emphasize mapping accuracy and precision.

21. The processing circuitry is configured to receive signaling, and in response to the received signaling, selecting which two or more search objectives from a defined set of search objectives are considered in the multi-objective optimization; or determining importance weights that modify the prioritization of individual search objectives among the two or more search objectives considered in the multi-objective optimization; 21. The mobile robot of claim 19 or 20, configured to perform at least one of the following:

22. 22. The mobile robot of claim 21, wherein the processing circuitry is configured to exchange signaling with an external computer that displays a user interface for a user to customize automated mapping behavior by the mobile robot, and the received signaling is received in the exchange.

23. 23. The mobile robot of claim 19, wherein the processing circuitry is configured to apply user-configured weights to at least one of the two or more search objectives to adjust the multi-objective optimization according to user preferences.

24. The two or more search objectives are: a distance-to-target objective indicating the shortest path distance from the current position of the mobile robot to the corresponding candidate frontier; a searchability objective indicating the searchability associated with the corresponding candidate frontier; a map coverage objective indicating the extent to which exploring the corresponding candidate frontier improves coverage of the physical environment by the mobile robot; a map consistency objective indicating the extent to which exploration of the corresponding candidate frontier leverages previously explored regions of the physical environment; and a kinematic objective indicating trajectory smoothness for moving from the current position of the mobile robot to a corresponding global waypoint defined on the candidate frontier; The mobile robot according to any one of claims 19 to 23, comprising two or more of:

25. 25. The mobile robot of claim 24, wherein the kinematic objective deprioritizes paths that are likely to require deviation from the mobile robot's current heading.

26. 26. The mobile robot of claim 19, wherein to perform the multi-objective optimization, the processing circuitry is configured to form and evaluate a scalarized objective from the two or more objectives for all candidate frontiers.

27. 27. The mobile robot of claim 19, wherein one of the search objectives is a map consistency objective that aims to improve the consistency of an online map generated by the mobile robot, and map consistency is measured using an entropy metric that reflects the consistency of the online map over successive time steps of the SLAM algorithm.

28. 28. The mobile robot of claim 27, wherein the map consistency objective is represented by multiple sub-objectives, including a structure adjacency sub-objective that prioritizes frontier goals that are adjacent to visible structures, an exploitation sub-objective that prioritizes movement by the mobile robot through previously visited or known areas of the physical environment, and a predictive closure sub-objective that prioritizes candidate frontiers that are predicted to return the mobile robot to known areas containing stable landmarks.

29. 29. The mobile robot of claim 28, wherein the predictive closure sub-objective predicts and enforces the closure of a physical loop by the mobile robot through the environment during the exploration process based on the processing circuitry being configured to predict what the physical environment will look like beyond a candidate frontier and determine whether loop closure is possible by navigating through the candidate frontier using a generative adversarial network (GAN) with training and exploration phases of operation.

30. 30. The mobile robot of claim 19, wherein the processing circuitry is configured to perform a local refinement process, the local refinement process including modifying a trajectory to the global waypoints by adding waypoints, enabling the mobile robot to capture local map structure from various viewpoints to improve the completeness of the local map.

31. 31. The mobile robot of claim 19, wherein the processing circuitry is configured to use a time-based blacklisting table to deactivate search targets that do not have a feasible estimated path from the current position of the mobile robot or for which execution of a valid estimated path results in repeated failures, the deactivation ending after a fixed time period after which the search target is reconsidered for searching.

32. 32. The mobile robot of claim 19, wherein the processing circuitry is configured to apply stopping criteria to exploration by the mobile robot, the stopping criteria including one or more of a physical boundary, a virtual boundary, a frontier limit, a search time, a wandering distance, a threshold limit, or a source-sink behavior.

33. 33. The mobile robot of claim 19, wherein the processing circuitry is configured to perform static trajectory generation including any one or more of using predefined user-specified landmarks, using connecting trajectories, using a wall-following behavior, and using a spin behavior.

34. 34. The mobile robot of claim 19, wherein, for planning the global path to the global waypoint on the next frontier, the processing circuitry is configured to impose multiple constraints on shortest path calculation for the global waypoint according to a multi-layer cost map.

35. The multi-layer cost map may be a first layer including a two-dimensional (2D) static occupancy grid created by the mobile robot performing a simultaneous localization and mapping (SLAM) process, wherein object avoidance is mandatory for obstacles registered in the first layer; a second layer including a dynamic three-dimensional (3D) voxel grid created from point cloud data generated using multiple environmental sensors mounted on the mobile robot and positioned at different heights, the second layer being periodically refreshed to account for temporary obstacles within the mobile robot's working envelope; a third layer including a 2D inflation layer that assigns higher costs to navigation through cells closer to obstacles and lower costs to cells further away within the first and second layers of the multi-layer cost map; a fourth layer including a 2D attractor layer that urges the mobile robot to carve a path along walls or other detected structures in the physical environment when robot position estimates and map consistency degrade; 35. The mobile robot of claim 34, comprising:

36. 36. The mobile robot of claim 34 or 35, wherein the global path planning is further processed by a local planner implemented in the processing circuitry that performs collision avoidance and generates a kinematically feasible path, and also uses recovery behaviors including one or both of cost map modifications and local recovery behaviors, the local recovery behaviors including one or more of backtracking, spins, and U-turns to push the mobile robot out of a stuck state.

Images