Autonomous mapping of mobile robot

Abstract

An autonomous mapping system defines the probing behavior of a mobile robot in a physical environment according to a multi-objective optimization. Through multi-target optimization, the detection behavior of the robot depends on a joint consideration of two or more detection targets, such as speed, coverage, and consistency. In at least one embodiment, the autonomous mapping system is customizable by a user through a user interface that allows the user to prioritize and / or select probe targets considered in multi-target optimization. The autonomous mapping system in one or more embodiments combines a favorable planning manner, for example, loop closure prediction for consistent mapping, kinematic dynamic prioritization for smooth trajectory generation, local refinement for improving mapping coverage, a time-based blacklist table for managing detection targets, a 2D attraction layer and recovery behavior for efficient path planning, and the like. And a recommendation system enabling an operator / user to refine subsequent probing operations, in particular for re-mapping.

Description

Technical Field

[0001] The present disclosure generally relates to mobile robots, and particularly to methods and devices for autonomously mapping a physical environment for a mobile robot. Background Art

[0002] Mapping of industrial mobile robots typically requires a skilled human operator to guide the robot around the physical environment of interest and collect data from sensors on the robot. Then, SLAM technology is used to construct a map offline on a computer. The skilled operator further adjusts the map offline to omit any "forbidden" areas or correct areas that were not properly mapped during the offline process. Once the map is created, it is not updated until the environment has changed significantly and needs to be remapped. Summary of the Invention

[0003] A robot navigating in the environment represented by the map creates a path plan using the start and end goals of the configuration defined in the map, and executes the path plan using a positioning algorithm that allows the robot to determine its position and orientation while moving in the environment. SLAM is not executed continuously because it is not always possible to distinguish which features are static and which are not, so errors may occur during the execution of the plan.

[0004] In the commercial household vacuum cleaner market, there are robot vacuum cleaners equipped with autonomous mapping functions. For example, see U.S. Patent 9,188,983 B2. Such systems use a real-time SLAM algorithm to find "borders", where a "border" refers to the boundary between a known (mapped) and an unknown (unmapped) area, and then attempt to map these borders. The unmapped area with the largest border is given priority. When there are no borders larger than a certain specified limit, the mapping process is completed. The user can pre-configure "no-go" areas using a panel or other technology, for example, see U.S. Patent 9,854,737 B2. The result is a map that can be used by the robot to ensure that the space enclosed by the map is completely covered by the robot. In this case, accurately following the specified path is not important. Although the basic border method is suitable for small spaces such as homes, it has practical limitations in the case of large spaces such as factories or other industrial or commercial environments.

[0005] The mobile robot operates as an autonomous mapping system, and the robot implements an autonomous mapper that defines the exploration behavior of the robot in the physical environment according to multi-objective optimization. The multi-objective optimization depends on the joint consideration of two or more exploration goals. In at least one embodiment, the autonomous mapping system is customizable by the user through a user interface that allows the user to prioritize and / or select the exploration goals considered in the multi-objective optimization.

[0006] One or more embodiments include a method for autonomous mapping by a mobile robot, the method comprising the steps of: the robot identifying a next boundary to be probed from two or more candidate boundaries in a map generated by the mobile robot using an online Simultaneous Localization and Mapping (SLAM) algorithm. The identification is based on the robot solving a multi-objective optimization for each candidate boundary, which jointly evaluates two or more sets of probing objectives to determine the attractiveness of the respective candidate boundary as the next probing target. Each of the two or more probing objectives is a probing objective of a respective type. The method further includes the robot planning a global path to a global waypoint on the next boundary based on collision avoidance and path parameterization determined by local path planning driven by an environmental sensor on the robot, and navigating to the global waypoint according to the global path.

[0007] One or more embodiments include a computer-implemented method for customizing the probing behavior of a mobile robot based on a user interface that displays a physical environment corresponding to the mapping to be performed by the mobile robot. Here, an autonomous mapper drives the mobile robot to probe the physical environment based on selecting a probing boundary according to a multi-objective optimization of two or more probing objectives considering their respective objective types. For example, a first probing objective biases the probing towards mapping accuracy, and a second probing objective biases the probing towards mapping speed. More generally, different types of objectives (represented as objective functions) represent different probing biases, such as more aggressive or less aggressive probing in cases where the risk of the robot getting lost is higher or lower, or making corresponding trade-offs in terms of the integrity or accuracy of the final environmental map, performing more time-consuming or less time-consuming probing.

[0008] The multi-objective optimization provides a favorable consideration and balance for competing probing objectives. In one or more embodiments, the multi-objective optimization allows user input to select which probing objectives to consider in the multi-objective optimization, or to set priorities to resolve the optimization trade-offs between two or more probing objectives being optimized simultaneously.

[0009] In one or more embodiments, the autonomous mapping system, in response to receiving a corresponding user input via the user interface, performs any one or more of the following operations: restricting the search space and mapping area; prioritizing the mapping area; selecting termination criteria and boundary strategies; selecting probing strategies, constraints, and parameters; selecting static trajectories; customizing the relative weights or priorities for multiple probing objectives evaluated by the mobile robot in the multi-objective optimization for controlling the probing behavior of the mobile robot; and specifying safety constraints.

[0010] Another embodiment includes a computer-implemented method for providing real-time feedback and exploration process control of a mobile robot implementing the autonomous mapper as described above. The method includes the steps of: visualizing the map generated by the robot, obstacles, robot position, planned global and local paths, cost map and boundaries, boundary priorities, and blacklist status, and providing user input controls for changing boundary priorities, blacklist status, aborting, or stopping the exploration process when the robot performs an unsafe operation; and rolling back a variable number of trajectories in case of incorrect mapping.

[0011] A mobile robot according to one or more embodiments includes: a drive system configured to move the mobile robot in a physical environment; one or more sensors configured to sense obstacles in the physical environment within respective sensing ranges; and a processing circuit.

[0012] The processing circuit is configured to identify the next boundary to be explored from two or more candidate boundaries in a map generated by the mobile robot using an online SLAM algorithm. The identification is based on the processing circuit solving a multi-objective optimization for each candidate boundary, which jointly evaluates two or more sets of exploration objectives to determine the attractiveness of the respective candidate boundary as the next exploration target. Here, each of the two or more exploration objectives is an exploration objective of its respective type. The processing circuit is further configured to plan a global path to a global waypoint on the next boundary based on collision avoidance and path parameterization determined by local path planning driven by environmental sensors on the robot, and navigate to the global waypoint according to the global path.

[0013] Of course, the present invention is not limited to the above features and advantages. Those of ordinary skill in the art will recognize additional features and advantages after reading the following detailed description and viewing the drawings. Description of the Drawings

[0014] Figure 1 is a block diagram of data and control flows in an autonomous mapper according to one or more embodiments.

[0015] Figure 2 is a block diagram showing the relative operation timing between various component modules or functions of an autonomous mapper according to one or more embodiments.

[0016] Figure 3 is a block diagram showing a simplified system diagram of the component modules of an autonomous mapper according to one or more embodiments.

[0017] Figure 4A is a diagram showing Figure 3 additional details of the system diagram shown in

[0018] Figure 4B is a block diagram showing Figure 3 additional details of the system diagram shown in

[0019] Figure 5 is a block diagram of a generative adversarial network (GAN) for loop closure prediction according to one or more embodiments.

[0020] Figure 6 is a block diagram showing the detection status of an autonomous mapper and the user interface according to one or more embodiments.

[0021] Figure 7 is a block diagram showing the user interface for customizing the behavior of an autonomous mapper according to one or more embodiments.

[0022] Figure 8 is a block diagram showing the feedback interface of an autonomous mapper according to one or more embodiments.

[0023] Figure 9 is a block diagram of a mobile robot according to one or more embodiments.

[0024] Figure 10 is a block diagram showing example implementation details of a mobile robot according to one or more embodiments.

[0025] Figure 11 is a block diagram showing example implementation details of a mobile robot according to one or more embodiments.

[0026] Figure 12 is a block diagram showing example implementation details of a mobile robot according to one or more embodiments.

[0027] Figure 13 is a block diagram showing example implementation details of a mobile robot according to one or more embodiments.

[0028] Figure 14 is a block diagram showing example implementation details of a mobile robot according to one or more embodiments.

[0029] Figure 15 is a block diagram showing a mobile robot and a related computer system according to one or more embodiments.

[0030] Figure 16 is a diagram showing example adjustment parameters for configuring autonomous mapping behavior according to one or more embodiments.

[0031] Figure 17 is a logic flow diagram showing the operating method of a mobile robot for autonomous mapping according to one or more embodiments.

[0032] Figure 18 is a logic flow diagram showing Figure 17 example details of the method shown. Detailed implementation

[0033] This document describes methods and apparatus for autonomous mapping by a mobile robot. The "Autonomous Mapper" according to the disclosed technology replaces the cumbersome manual mapping of large industrial environments before deploying the robot. In particular, the Autonomous Mapper provides an automated process that reduces the time required for environmental mapping while minimizing human intervention. When used without qualification, the term "Autonomous Mapper" refers to the functionality embodied in the disclosed technology, and it is understood that this functionality is implemented by the on-board processing and control system of the mobile robot that performs the mapping operation.

[0034] The terms "Survey Framework" and "Autonomous Mapper" are interchangeable terms in this disclosure, and related terms such as "Survey Kit" refer to subsystems or component functions included in the Autonomous Mapper.

[0035] The mobile robot implements the Autonomous Mapper to plan and execute a path through the environment, aiming at areas where no map has been established, while using SLAM technology to localize its position and construct a map representing the environment. A notable aspect of the Autonomous Mapper is that it operates based on multi-objective optimization. Through multi-objective optimization, the mobile robot jointly evaluates a set of two or more exploration targets relative to each boundary that is a candidate for selection as the next exploration target to select an exploration target for probing the physical environment.

[0036] Each exploration target is represented as an objective function whose value will be minimized or maximized. One or more objective functions may conflict with one or more other objective functions, such that multi-objective optimization involves the mobile robot determining the best trade-off or balance among multiple exploration targets. For example, there may be one or more speed-related objectives that bias the exploration behavior of the mobile robot towards mapping speed. However, mapping speed may conflict with mapping accuracy and / or mapping integrity, where these objectives are represented by the respective objective functions considered in multi-objective optimization. Other example objectives include objectives related to the smoothness of the movement of the mobile robot, or objectives related to robustness, such as resistance to loss of localization, battery life, etc.

[0037] In at least one implementation, the mobile robot includes a memory or other memory from which user-configured values or data are read to adjust its multi-objective optimization.

[0038] Accordingly, in one or more embodiments, an autonomous mapping system as disclosed herein includes a mobile robot that instantiates and runs an example of an autonomous mapper and that allows for human intervention, if necessary, to establish parameters such as goals, and / or define restricted areas and emergency stops. By preserving such manually configured map elements, the system also greatly simplifies remapping of environments that undergo dynamic changes (not affecting these elements) during robot operation. The system also has potential use in object discovery. Example systems may also include support logic provided via a PC or other interface device for a user to input configuration parameter values to adjust the autonomous mapper.

[0039] The autonomous mapper can be used to achieve level 5 autonomy as defined by the Society of Automotive Engineers (SAE), but a less complex and less costly implementation of the autonomous mapper provides significant advantages for level 2 autonomy. In this case, the autonomous mapper supports user monitoring of the autonomous mapping to perform an "abort" on potentially dangerous navigation behavior in the event of sensor or decision system failure.

[0040] Compared to manual mapping, the autonomous mapper significantly reduces mapping time by selecting the shortest path, maintaining the greatest distance from structures in the environment based on the detection capabilities of the sensors used by the mobile robot for mapping, while maximizing map quality, coverage, integrity, and localization accuracy. Without robot feedback, a human operator working in the same area would not be able to determine the optimal path for reducing time while improving localization performance and map quality, which are not typically available in manual mapping steps. Additionally, the operating speed of a robot operating according to the disclosed autonomous mapper operation is much higher than that of a human operator walking around with the robot for mapping operations. Such speed reduces the number of man-hours required for mapping. Further, since operator line-of-sight vision or remote monitoring of the mapping robot is typically sufficient, the autonomous mapper greatly reduces the operator's workload.

[0041] Example embodiments include an autonomous mapper in a mobile robot and provide advantages in situations where precise navigation in large spaces is required or where increased flexibility in remapping is needed. Here, unless otherwise limited in context, the term "mobile robot" has a broad meaning and includes various wheeled and tracked vehicles such as industrial robots or autonomous vehicles, and also includes drones and other aircraft, as well as autonomous vessels.

[0042] Among its various advantages, a key one is that the autonomous mapper uses a multi-objective optimizer. The multi-objective optimizer maximizes the map coverage while minimizing the time required for mapping. For example, the multi-objective optimizer includes specific processing and computational operations implemented by executing corresponding computer program instructions, and also maximizes the accuracy of the global map by prioritizing the paths of the robot through boundaries that enable loop closure, i.e., registering the current view relative to reliably identified landmarks in the environment.

[0043] The technological improvement embodied in the multi-objective optimizer advances the current state of the art. Its core is boundary-based mapping that prioritizes the largest boundaries to maximize coverage, but without the multi-objective optimization that exploits various opportunities to explore the trade-offs between options. For example, in one or more embodiments, the multi-objective optimization embodied in the disclosed autonomous mapper jointly considers more than one objective function, each of which embodies a different exploration goal, such as more comprehensive exploration or the fastest exploration, etc. The joint optimization balances the trade-off between mapping time and global mapping accuracy. In at least one embodiment, the autonomous mapper includes or supports a user interface that discloses certain parameters of the optimization algorithm, thus providing the user with a beneficial mechanism for "tuning" the optimization behavior of the mobile robot implementing the autonomous mapper. For example, the user can adjust the mapping speed or mapping accuracy, or can also adjust more aggressive exploration or less aggressive exploration. Generally speaking, the embodiments of the autonomous mapper provide the user with a beneficial ability to balance the various trade-offs made by the robot during the mapping process.

[0044] The term "exploration suite" refers to the constituent functions or subsystems contained within the overall autonomous mapper or exploration framework. The exploration suite determines the global path points for navigation as a multi-objective optimization problem, seeking to maximize the map quality, coverage, and localization score while minimizing the surveyed mapping distance and the time required to complete the exploration process. These sets of objectives are opposed to each other and involve trade-offs, which are managed by the multi-objective optimizer. How the optimizer balances the trade-offs is "tunable", at least in part, by the user according to their specific needs or requirements.

[0045] The exploration suite affects path planning in a way that forces the robot implementing the optimizer to perform loop closure during the exploration process. Forcing loop closure is achieved by predicting how the robot will reach landmarks when moving along unmapped boundaries (hereinafter referred to as "predicted mapping"). This method not only ensures accuracy through loop closure but also prevents the robot from getting lost (i.e., losing its position).

[0046] The detection suite uses kinematic dynamic prioritization to place low priorities on paths that may require deviating from the robot's current heading, thereby minimizing the likelihood of executing a path direction change. This behavior provides advantages, especially that the path taken by the robot will be smoother and exhibit more predictable behavior around humans. Additionally, smoother trajectories can improve battery performance and extend the mechanical life of the motors and drive systems on the robot.

[0047] The trajectory of the robot to the optimized waypoint also needs to go through a local refinement process, which refines the trajectory by adding waypoints, enabling the robot to capture the local map structure from various viewpoints, thereby increasing the local map coverage.

[0048] In one or more embodiments, the detection suite also uses a time-based blacklist table to reject goals that the global planner deems unachievable. Different from traditional blacklist tables that always remember these goals, the time-based blacklist table assigns an expiration time to the table entries.

[0049] Another notable aspect of the detection suite is the use of a global planner to push the robot near visible structures during the mapping process. The global planner utilizes a 2D attraction layer, as described later in this document.

[0050] The detection suite also uses a local planner that employs recovery behaviors including "cost map" changes and local recovery behaviors including, for example, backtracking, rotation, and U-turns. These local recovery behaviors enable the robot to save itself when it enters a trapped state.

[0051] In one or more embodiments, the autonomous mapping system is customizable, enabling the user to restrict and prioritize mapping areas, manage trajectories, detection strategies, and specify safety considerations. Additionally, in at least one embodiment, a recommendation function provides detection analysis, evaluation, and feedback in the form of a recommendation system that the operator can use to refine and fine-tune current and future detection and remapping runs.

[0052] Figure 1 and Figure 2The basic functions of an autonomous mapper according to one or more embodiments are shown. In an example context, the robot for autonomous mapping according to the disclosed technology is a differential drive industrial mobile robot having one or more two-dimensional (2D) laser scanners and / or sonar ranging devices, and wherein the processing system of the robot can access odometry information, for example, via wheel encoders and an inertial measurement unit (IMU) or similar devices. It is also assumed that the robot constructs and uses a 2D occupancy map for localization. However, the disclosed technology is directly applicable to or scalable to other types of robots, including unmanned aerial vehicles (UAVs), unmanned surface vehicles (USVs), unmanned ground vehicles (UGVs), etc., whether or not there are additional sensors such as three-dimensional (3D) lidar, vision cameras, etc.

[0053] The exploration framework in one or more embodiments includes: (a) a module that interacts with sensors on the mobile robot; (b) an exploration suite that solves a multi-objective optimization problem to derive an optimal estimate of the exploration location and the next best waypoint (i.e., where to explore and how to explore) based on user-specified constraints; (c) a SLAM module that uses sensor data based on the exploration location to generate an online map; (d) a global path planner that creates a path to the exploration location identified by the exploration suite; (e) a local path planner that calculates a parameterized local path while avoiding obstacles; and (f) a motion controller that converts the commanded position to velocity. In at least one embodiment, the exploration framework further includes a user interface that has options for controlling one or more aspects of the exploration process. For example, the user interface accepts user input, and an exposure function included in the exploration framework adjusts the values of one or more configuration parameters that affect the exploration behavior in response to the user input received via the user interface.

[0054] Figure 1 A high-level example exploration framework is shown, and the example data and control flow through the respective blocks are indicated. Here, each block can be understood as a collection of functions or related functions, for example, implemented by executing computer program instructions by one or more microprocessors or other digital processing circuits specifically adapted to operate as an autonomous mapper.

[0055] Data from sensors on the mobile robot, including optical data (mainly laser scans) and odometry, for example, are streamed to a simultaneous localization and mapping (SLAM) module that generates an environmental map while localizing the robot on the map. In this step, the autonomous mapper executed on the robot also periodically runs a loop closure optimization process based on revisiting previously mapped areas.

[0056] Then, the SLAM map is input into the exploration suite. Based on a set of exploration parameters that are user - adjustable in one or more embodiments, the exploration suite uses a multi - target global optimization process to calculate the best targets or waypoints for exploration. Identifying and navigating to waypoints can be understood as elements of an exploration process performed by a detection framework instantiated by a mobile robot's processing circuitry.

[0057] Then, the exploration targets / waypoints are sent to a global path planner that generates paths to each target. If no valid path is available, either because it cannot find a valid path or all calculated paths are later determined to be infeasible by the local planner (as described later), the global planner signals the exploration suite to blacklist the current target and acquire the next target for planning. Using the localization information from the SLAM module, the global path planner can also determine whether an exploration target has been reached. If so, it sends a new target to the exploration suite for the next step of exploration.

[0058] Then, the global path is sent to a local path planner that parameterizes a local segment of the global path based on the robot's current position and forms it into a form suitable for actuation while avoiding obstacles. Then, the desired local poses along the local path are sent to a motion controller that converts these positions to velocities to drive the robot's differential drive motors.

[0059] If a collision - free path cannot be identified, or the robot is repeatedly blocked from executing the planned path by obstacles in its vicinity, the local planner recalculates the local path. There is a limit to the number of local paths that the planner can calculate and retry. If that limit is reached, i.e., all local paths are invalid, the local planner signals the global planner to recalculate the global path. It is also possible that all these recalculated global paths are found to be infeasible.

[0060] Figure 2 An example timing diagram is shown. A timing diagram is a simplified representation of the relative operation timing between the various blocks or modules of the exploration framework, including periodic synchronization triggers or signals, and specific - context asynchronous triggers for an example context.

[0061] A system - level description of the exploration framework according to one or more example embodiments is shown in Figure 3 in terms of its prominent component modules. According to one or more embodiments, the exploration suite is shown to be located in the middle of Figure 3 , Figure 4A and Figure 4B provides a detailed view. The algorithms and system components that support the exploration framework are also in Figure 3 and Figure 4A / Figure 4Bshown in. As described above, these include industrial mobile robot hardware with sensors, global path planning and local path planning modules, motion (velocity) controllers, SLAM, and user interfaces.

[0062] The following sections describe Figure 4A and Figure 4B each of the modules described in

[0063] Probing Kit - Dynamic Trajectory Generation

[0064] The probing kit supports dynamic trajectory generation, operates with minimal user intervention, and also supports static trajectory generation from user input. The dynamic trajectory generation module of the probing kit solves a multi-objective optimization problem, and the solution determines the next global waypoint to travel and the associated direction of travel. This waypoint information (position variable) goes through a local refinement step that determines the local position and orientation. The local refinement step also works independently to refine the local trajectory between global waypoints. The global planner uses the outputs of multi-objective global optimization and local refinement - the goals / waypoints and associated directions - to construct the trajectory.

[0065] Dynamic Trajectory Generation - Global Waypoint Estimation

[0066] This section describes a method for formulating a probe according to multi-objective optimization with specific optimization goals. The global waypoints selected by the multi-objective optimization can be optimal or along the non-dominated / Pareto optimal front. That is, a unique solution or multiple equally optimal solutions can be obtained, where each solution performs well on one of the objective functions considered in the multi-objective optimization but performs poorly on one or more of the other objective functions considered. The multi-objective optimization according to one or more embodiments of the autonomous mapper consists of several factors and corresponding variables, and multiple objective functions are used to optimize over these factors and variables. The multi-objective optimization problem can be formulated according to the following equation:

[0067] [Equation 1]

[0068] min(f 1 (x), f 2 (x), f 3 (x),..., f k (x)) (1)

[0069] [Equation 2]

[0070] subject to x ∈ X

[0071] where, f iThe objective functions \(f(x)\) are the various functions to be minimized, and \(X\) is the set of feasible decision vectors. Similarly, each objective function corresponds to a corresponding robotic behavior or property, such as mapping speed, mapping accuracy, power efficiency, trajectory smoothness, etc.

[0072] These objectives are linearly reweighted to obtain a scalarized objective, leading to a further simplified single-objective optimization. According to its multi-objective function, the scalarized form of the optimization is given by:

[0073] [Equation 3]

[0074]

[0075] Here, \(w\) i is the scalarization weight. At the beginning of the optimization process, these weights are set to be equal to 1. This is further reweighted by a priority factor \(p\) i which depends on specific conditions discussed later. The weights can be customized by the user for specific use cases. The individual objective functions \(f\) i (x) can also be subject to additional constraints, which can be used to simplify the scalarized optimization function.

[0076] The constraints of the objective functions can be expressed as:

[0077] [Equation 4]

[0078] \(\epsilon\) i，l \(\leq f\) i (x) \(\leq \epsilon\) iu for \(i \in \{1, \ldots, k\}\) (3)

[0079] \(\epsilon\) i,l and \(\epsilon\) i,u represent the minimum and maximum bounds of objective \(i\).

[0080] In principle, since the solution space (including the boundary objectives / waypoints discussed in the next section) is discrete, if the list of boundary objectives is not large, a brute-force method (i.e., by evaluating the objectives at all points in the solution space) can be used to solve such a system (i.e., find the boundary indices for probing as described in the next section). In large environments, the list may be large enough that finding a solution becomes tricky. In such cases, an approximate solution in the form of a mixed-integer linear program can be obtained (since some objectives may be binary, some integer, and some real).

[0081] These approximate solutions may or may not produce the optimal solutions for all objective functions, i.e., they may not produce the global minimum of the scalarized (also known as mixed) objective function. Using the approximate solutions generated by the solver, where multi-objective optimization may identify more than one metric, the boundary target / waypoint metric closest to the current position of the robot is identified and used in the next stage of processing. The scalarized form with a mixed objective uses the branch and bound method to solve, while using the gradient descent method to solve the local solution at the boundary. For details of such solvers, see Deb, K. (2014), "Multi-Objective Optimization", (pp. 403 - 449), Springer, Boston, MA. This provides a simple and quick way to solve the problem.

[0082] This multi-objective optimization can also be solved using alternative solvers including approximate solvers. More complex methods can also be used, such as hierarchical or lexicographic methods, multi-objective degradation, and hybrid methods supported by popular solvers. Solvers that support evolutionary algorithms (including particle swarm optimization) can also be used to solve such systems.

[0083] Global Waypoint Estimation - Solution Space

[0084] The solution of the above multi-objective optimization problem (i.e., x ∈ X) is defined according to the waypoints or target points placed on the boundary. In other words, the overall purpose of the optimization is to select one from a series of boundary targets. The boundary cases are introduced below.

[0085] The boundary is used to identify potential detection targets. The boundary is established as the pixel neighborhood of the boundary between the visible (unoccupied) and invisible (unscanned) spaces in the current map generated by SLAM (see the section of this article on SLAM for map generation). Here, the term "visible" corresponds to the observation of an obstacle or a free / unoccupied position at a specified location in space by a previous optical sensor, where the scanning ray from the sensor has passed through this position before detecting an obstacle behind it. These positions form cells in the occupancy map. "Invisible" positions correspond to cells in the occupancy map where no obstacle has been detected, or where no sensor ray has traced an obstacle before. The goal of detection is to navigate to these boundaries, access the invisible areas, and thus add these areas to the map. All these identified boundaries are added to the boundary list. Depth discontinuities determined by sensor measurement jumps are classified as "cracks". Such cracks are also added to the boundary list to be processed at any time.

[0086] Then, these disjoint boundary neighborhoods are characterized by centroid positions, which are used as the target positions for global path planning. Additionally, an approach direction is associated with each boundary. The approach direction is chosen as the normal to the boundary so that it divides the unseen region outside the boundary into approximately equal halves. To prevent the robot from repeatedly attempting to detect boundary targets, a blacklist table is introduced in cases where there is no path to the target.

[0087] After repeated failures in local path planning prevent the robot from reaching a boundary, the boundary is blacklisted for a certain period of time during which the robot is free to detect other boundaries. It is expected that the detection of other nearby boundaries may clear the cost map or increase the paths that can reach the previously blacklisted boundary.

[0088] In one or more embodiments, the autonomous mapper is operable to cause the robot to retry a blacklisted boundary after the blacklist timeout expires and the boundary is removed from the blacklist. During the retry, depending on the robot's current position, the robot may or may not approach the originally blacklisted boundary along the same original route.

[0089] The selection of which boundary target / waypoint to probe next is determined by the output of multi-objective optimization.

[0090] Global Waypoint Estimation - Optimization Objectives

[0091] The various objective functions f i (x) to be optimized in one or more exemplary embodiments include: (1) distance-to-goal objective, (2) target detection potential objective, (3) map coverage objective, (4) Figure 1 consistency objective, and (5) kinematic dynamic priority objective. Note that these objectives are functions of the boundary targets / waypoints and are normalized so that they contribute equally to the optimization (framed as minimization). Additionally, in a minimization problem, an objective to be maximized is represented as a negative objective function.

[0092] Distance to the target: The target of approaching the boundary enables the robot to reach the target faster, thus enabling faster exploration. For any given target, the shortest distance to the target is calculated using Dijktra's shortest path algorithm on the current map. Additionally, for each change in heading along the path (i.e., deviation from a straight path), the time required to make the heading change (based on the maximum angular velocity of the robot) is estimated and converted into a distance offset to be added to the objective function. As shown in Equation (4), it is multiplied by the maximum linear velocity and added as a distance offset. This ensures that the distance metric also represents the time required to reach the selected target. Then, the calculated distance is normalized using the expected maximum value of the given space. This maximum value can be a best guess (determined by the user, for example, based on a rough estimate of the diameter or diagonal distance of the bounded space to be explored, or calculated based on the mathematical expectation of the maximum distance target value from previous runs in the space reported by the exploration analysis). The distance-to-target objective should be minimized:

[0093] [Mathematical formula 5]

[0094]

[0095] where minDist is the shortest distance between the centroid xc of the boundary x and the current position of the robot r, defined according to 2D pixel coordinates (x, y), ν is the maximum linear velocity of the robot, δθ i is the change in heading along the shortest path, and ω is the maximum angular velocity of the robot.

[0096] Target exploration potential: There are two sub-goals classified as "target exploration potential" goals; namely, the "boundary span" sub-goal and the "boundary adjacency" sub-goal. These sub-goals also contribute to increasing the exploration coverage of any specific environment.

[0097] Boundary span: For a given boundary, the span of the boundary specifies how large the boundary is and how much space can be explored by accessing this boundary. The span is calculated as the number of pixels along the length of the boundary multiplied by the cell / pixel distance (Manhattan distance) between the pixels corresponding to the two ends of the boundary in the map occupancy grid (the pixels at each boundary start and end are determined by the bounding box of the boundary pixels). It can be expected that such a metric captures the extent of the area not visible beyond the boundary while ignoring non-smooth features along the length of the boundary. In the initial stage of mapping, boundaries that are closer and provide large-area exploration are preferred. Similar to the "distance to target" goal, this goal is also normalized with a larger expected value. This goal is to be maximized. Since the multi-objective optimization problem is formulated as a minimization problem, the negative value of this goal is used as the objective function.

[0098] [Mathematical formula 6]

[0099] f(x)=-(∑i∈BB(x) I(x i , y i )) * ||BB(x s , y s ) - BB(x e , y e )|| (5)

[0100] where l(x i , y i ) is an indicator function of the peripheral pixel i evaluated on the boundary length, and BB(x s , y s ), BB(x e , y e ) are the starting and ending points of the bounding box of the boundary x: BB(x).

[0101] Boundary adjacency: It is also possible that there are multiple boundaries in a given area. Once one of the boundaries is detected, these boundaries may merge or cause the merging of invisible areas. Therefore, being close to other boundaries also caters to the detection potential of the target. For a given target / waypoint, this metric is calculated using the Rapidly-exploring Random Tree (RRT*) as the shortest path search between boundary targets for each other boundary target near the selected boundary target. Here, the selection of RRT* (as opposed to traditional Dijkstra global planning) allows the calculation of distances through invisible space. The normalized distance target should be minimized.

[0102] [Equation 7]

[0103]

[0104] where N(x) is the set of boundaries near the boundary x, i.e., the boundaries within a specified maximum distance from the boundary x. Here, all distances including minDist are calculated using the RRT* shortest distance, and x p , n q are the positions of the pixel p in the boundary x and the pixel q in the boundary n, where p and q are the nearest pixels among all pixels in x and n respectively.

[0105] Map Coverage: The map coverage objective attempts to increase the coverage of the map (i.e., the ratio of the surveyed area to the total area to be surveyed). This depends on the availability of floor plans, user sketches, previous CADs, or environmental maps in a previous state that have been modified later, which may or may not be possible for a particular environment. Therefore, this objective is optional. In the case of autonomously re-surveying a previously surveyed environment, the previous map can be reused for this purpose. In this case, a graph-based discretization of the search space is performed, i.e., the entire environmental map is represented as a graph. Topological refinement methods such as tessellation, graph, and tree pruning are applied to the graph to reduce its complexity and simplify its representation in terms of graph nodes forming waypoints. Fewer nodes are generated for larger areas, and more nodes are generated in smaller, compact, and highly structured areas to preserve the topology of the space. Each node is associated with the area it "covers". From the list of graph nodes located in the invisible area, the "node area" measure corresponding to the nearest node to each boundary target is used as a measure of the map coverage. Nodes with a larger node area are preferred. This normalized coverage objective is to be maximized.

[0106] [Equation 8]

[0107]

[0108] where NA(m) is the node area covered by graph node m. This node m is the node in the graph that is closest to the centroid xc of the boundary. The graph G(u) to which this node m belongs is a subgraph of the invisible space in the graph representation of the entire map.

[0109] C Figure 1 onsistency: C Figure 1 onsistency objective attempts to increase the consistency of the map, i.e., the repeatability between consecutive measurements generated by SLAM at each time step. The measure of map quality is the repeatability of the generated map. The SLAM process measures the consistency of the map according to an entropy metric. High entropy in consistency indicates poor consistency, i.e., the map representation varies greatly between time instances and is inconsistent. The entropy can be reduced by accessing known or visible areas on the map. The localization score is also a relevant metric, which increases as the entropy decreases and decreases as the entropy increases. Therefore, the trade-off here can be described by the exploration-exploitation paradigm. There are three sub-objectives, all classified as "C Figure 1 onsistency". These objectives are gated by a binary version of the entropy metric E (entropy above the threshold activates the objective, i.e., the gating priority of this objective i in multi-objective optimization. p i = 1, while entropy below the threshold deactivates the objective, i.e., p i = 0).

[0110] [Equation 9]

[0111]

[0112] where p(x i ) is the occupancy probability of cell i of map M at position x i .

[0113] The consistency goal can be considered to include three sub-goals: structural adjacency, exploitation, and predictive closure. Figure 1 For the structural adjacency sub-goal, boundary goals adjacent to physical structures (rather than open areas) are preferred. More visible structures help better localize the robot and improve map consistency and visibility. A metric derived from the distance transform (normalized) of the target waypoints on the current binarized and value-inverted SLAM occupancy map (targets with nearby structures have a lower distance on this metric) is used as the goal to be minimized.

[0114]

[0115] [Equation 10]

[0116] f(x) = distXform(x c ) (9)

[0117] where distXform(x) is the distance transform of the map grid point - boundary centroid x c .

[0118] According to the "exploitation" sub-goal, when entropy decreases, boundaries where the robot needs to cross and observe visible areas (more exploitation, less exploration) are preferred. This is characterized by a binary goal that re-weights targets farther than a minimum threshold distance, unlike targets closer than the minimum threshold. Note that this goal is the opposite of the "distance to target" goal and needs to be maximized.

[0119] [Equation 11]

[0120] f(x) = -I(||x c - r|| > D 1 ) (10)

[0121] where I(x) is an indicator function, x c is the centroid of boundary x, r is the current pose of the robot, and D 1 is the distance threshold.

[0122] The third sub-goal, "predicted closure", aims to identify boundaries through which, if the robot passes, it may close a loop and return to a previously visited area with stable landmarks. In other words, the goal is to select a boundary that leads to SLAM loop closure. This loop closure can then improve the quality and consistency of the map by correcting for drift in the robot's localization during the mapping process. The "predicted mapping" section of this paper details the prediction process required to identify such paths. Boundaries predicted to trigger loop closure are assigned a negative distance metric, while those not predicted do not. This objective should be minimized.

[0123] [Equation 12]

[0124]

[0125] where I(x) is an indicator function, x c is the centroid of boundary x, D 2 is the distance threshold, m q is the position of cell q in the sub-map M(o) of occupied cells of map M obtained using the current SLAM occupancy map, i.e., the one closest to the boundary centroid, the distance metric is computed on M, and the shortest distances using Dijkstra (for visible / predicted space) and RRT* (for invisible space) are added to the predicted map.

[0126] The kinematic dynamic priority objective attempts to select boundary targets such that the robot follows a smoother trajectory. Kinematic dynamic prioritization depends on the robot's drive system. While an omnidirectional drive system (with omni-wheels) is minimally affected by changes in direction or speed, differential, car-type, four-wheel drive, synchronized drive robots, tracked robots, skid / skid-steer place a large amount of stress on the motors every time there is a change in direction, speed, or acceleration. Selecting a smoother trajectory along the direction of momentum on the continuous field within the robot's local neighborhood can reduce mechanical stress. This may also result in paths that require less power. The metric characterizing this objective is the absolute deviation in direction, such as a 2D measurement from the robot's current position and heading to the boundary, along the shortest path to the boundary centroid. The gradient returned by the scan matching module of SLAM can also be used for this estimate. The normalization objective should be minimized.

[0127] [Equation 13]

[0128]

[0129] where δθ i = θ i+1 - θ i is the change in heading along the shortest path for each step i, θ 0 is the initial heading, θ ris the robot heading at the calculation point, θ f is the final heading, θ x is the heading at the centroid of the boundary to be achieved.

[0130] The user can further customize or disable these multiple objectives for specific use cases. As mentioned before, the multi-objective optimization calculation solves for the solution - the boundary x (identified by its index) that minimizes the scalarized form of the objective.

[0131] Dynamic Trajectory Generation - Local Refinement

[0132] As mentioned before, the position variable from the global waypoint estimate is further refined by determining local steps for local position and orientation. This local refinement step also runs independently at regular intervals, refining the local trajectory between global waypoints. The timing relationship between the global multi-objective optimization and the local refinement step is also as Figure 2 shown. The exact nature of the improvements in the example implementation is described below.

[0133] The local refinement step helps improve the integrity of the local map. This is achieved through two modules: the scan count module and the sub-optimal view module.

[0134] Scan Count: The scan count module ensures that each structural component placed in the map has sufficient confidence in the observations. In other words, it ensures that a sufficient number of observation rays hit the cells in the map and identify them as obstacles or pass through obstacles, thus identifying them as free cells. If the threshold criteria are not met, additional waypoints and directions are sent to the global planner to obtain additional scans before continuing with the global waypoints.

[0135] Sub-optimal View: The sub-optimal view module ensures that the local structure is fully mapped. This can be achieved by creating waypoints, paths, and directions that help achieve this for local consistency of the map structure. An example of such a trajectory is moving in a circle around an object to complete its visual description. This is achieved by creating the convex hull of the visible structure under consideration and moving around the convex hull along a diversion path such that the path distance does not exceed the maximum deviation from the global path to the current boundary target at the deviation point. The convex hull is updated as the exploration progresses, and the map is updated by SLAM. If the maximum deviation is reached, waypoints for a global path that returns the robot to the boundary are generated. On the other hand, if the exploration around the convex hull is completed, the robot will have returned to the global path. As with the scan count, these additional waypoints and directions around the convex hull are sent to the global planner, and once the visual description of the local structure is complete, the processing of the current global waypoints continues.

[0136] Probing Kit - Static Trajectory Generation

[0137] Several static user-defined waypoint generation behaviors are also provided as options for trajectory generation. These options include: (1) predefined user-specified landmarks and / or trajectories that the robot needs to visit / follow, only avoiding local obstacles; (2) wall following, maintaining a certain distance from the starting wall and then using only odometry to guide the robot to circle around the internal area (away from the wall) - this behavior is suitable for mapping in extremely sparse and featureless environments; and (3) rotation behavior for quickly mapping the map of a small room with minimal clutter.

[0138] In specific use cases, this static trajectory can be used instead of dynamic trajectory generation. The goals / waypoints generated from these user-defined trajectories are also sent to the global planner for further processing.

[0139] Detection Suite - Predictive Mapping

[0140] Predictive mapping is proposed as a technique to enhance the goal of "consistency". The goal here is to be able to predict how the environment looks beyond the selected boundary, so that it can be determined whether a loop closure can be triggered by passing through the boundary and finding a route back to a previously observed part of the environment with stable and reliable landmarks. Figure 1 When operating in an industrial space or other large environment, the robot is likely to encounter similar or repetitive structures. This is especially true for warehouse aisles and docks. A generative adversarial network (GAN) with a generation and discrimination component is used to train on scenes in the already visited environment and predict the next possible structure shape to complete the map in the robot's path. For example, the predicted shape in a warehouse can result in a structure with a loop path at the end of the aisle. The generator uses the SLAM instantaneous map and the previous part of the map to generate map parts during the training phase, while the discriminator is trained to identify unsupervised whether the generated map is different from the previous complete map. This architecture is shown in Figure 4.

[0141] Through several rounds of backpropagation learning, the generator can better create maps that the discriminator cannot label as different from the map prior, while the discriminator can better label these maps. During the inference process, the SLAM instantaneous map is used to generate a predictive map beyond the boundary. When a loop closure may mitigate a decrease in the localization score or an increase in entropy, the GAN prediction of a possible loop closure enabling structure in a specific direction downstream of the robot's current position (determined by using the shortest path of Dijkstra to known map points) is used as the objective of the optimization problem. In this case, the directions that do not lead to a loop closure in the GAN prediction are ignored.

[0142]

[0143] ​Note that the training process of the GAN can be performed offline (based on previous maps in a similar domain / environment such as an industrial warehouse or supermarket), domain adaptation may be required for specific use case environments, and it can be performed online. This requires additional computation. On the other hand, due to the locality of the prediction, the computational cost of the inference is low. Additionally, when using a simple binary representation of the map generated by SLAM for training and inference, the computational complexity remains low.

[0144] This novel approach is referred to as "predictive mapping" in the present disclosure. As previously mentioned, the predictive mapping module is helpful for loop closure. When the entropy is higher than a threshold and the likelihood of an increase in the localization error or error uncertainty increases, the inference phase is applied so that the robot can track the path leading to loop closure, at which point such errors in the system can be reduced or eliminated. This mechanism is also captured in Figure 1 and the corresponding timing behavior is captured in Figure 2 Figure 5 shows an example of the GAN configuration for loop closure prediction, i.e., the implementation in the predictive mapping module.

[0145] Survey Kit - Stop Criteria

[0146] Various stop criteria for the autonomous mapper to perform a survey are also provided. An example set of such criteria includes any one or more of the following: physically predefined boundaries by the user, virtual boundaries predefined by the user using a user interface, a spatial framework (the boundary limits of the survey marked by the user), parametric definitions of the boundary and framework structures, boundary size limits (minimum and maximum values), survey time limits, roaming distance limits (the distance the robot is allowed to travel), threshold limits (whether the robot is allowed to enter a new room through a potential doorway), and source-to-sink behavior (where the robot stops surveying once it reaches a sink target - a landmark target such as a dock or an image target).

[0147] Global Path Planning

[0148] ​The main component of the global path planning module is a shortest path estimator based on the Dijkstra algorithm, which is used to find the shortest path to a specified goal or waypoint while avoiding obstacles in the SLAM map. The detection suite section discusses the details of how these goals are specified. Different from the distance calculation in the detection suite using Dijkstra on the map generated by SLAM, the global path planner uses additional constraints for the shortest path calculation, specified in the form of a cost map. When calculating the global path, a four-layer cost map projected onto a single 2D cost map is used to characterize the obstacles to be considered. For an example of a general cost map, see Wang, X., Mizukami, Y., Tada, M., & Matsuno, F. (2021), "Navigating a mobile robot in a dynamic environment using a point cloud map", Artificial Life and Robotics, 26(1), 10-20.

[0149] The first layer of the four layers is a two-dimensional "static" occupancy grid layer created based on the map generated by SLAM (which is also dynamically updated during the mapping operation). The obstacles in this layer are absolute, and the global path planner must forcefully avoid the obstacles in this layer.

[0150] The second layer is a "dynamic" 3D voxel grid layer created from the point cloud data of rangefinding and other landmark sensors located at different heights of the robot. This dynamic layer is periodically cleared to allow for transients in the scene content, such as humans walking in the environment, while considering sensor characteristics such as reliability, frame rate, and noise performance.

[0151] The third layer is a 2D "inflation" layer that grades the distance to obstacles on the curve. In other words, the cells in the cost map closer to the obstacles have a higher cost than those farther away from the obstacles. This forces the robot to stay away from the obstacles while allowing it to approach these obstacles when it cannot find the lowest-cost path with a large spacing or gap. In this case, only high-cost paths with a small gap may be available, and these paths are executed at a low speed by the local planner, thus reducing the possibility of collision and emergency stop.

[0152] The last layer is a 2D "attraction" layer that enables the robot to "hug" obstacles during navigation (for the behavior of following walls). The attraction layer prevents the robot from getting lost in open areas and selectively works on large obstacles such as walls while keeping the force of the robot within the range of the environment. The visibility of the current environment and its correspondence with the known SLAM map are estimated through the map entropy metric. When the map entropy metric has a high value - indicating a lack of correspondence between the observed environment and the current state of the map, the global planner uses the 2D attraction layer to reweight the cost map to select paths that prevent or reduce complete loss of localization during detection.

[0153] The static layer is the most important part of global path planning, while the dynamic layer is the most important part of local path planning behavior. The inflation layer enables the robot to approach obstacles at a lower speed and achieve a sub-optimal path when the best path has not been found. The attraction layer ensures that the robot does not get completely lost during the exploration process. Therefore, for large structural obstacles in the environment, the inflation layer and the attraction layer work in parallel.

[0154] Local Path Planning

[0155] The local path planning module is a component responsible for generating potential kinematically dynamically feasible behaviors while avoiding collisions. It performs calculations within a limited space range around the current position of the robot and uses particle simulation based on a polynomial model to establish local trajectories. The goal of the local planner is to generate a simplified path that can be driven by the differential drive motors of the robot, while allowing the robot to follow a path as close as possible to the global path while avoiding any local obstacles.

[0156] Here, two possible traditional algorithms are combined - the Dynamic Window Approach (DWA) and Trajectory Rollout (TR). Generally, TR provides better results in terms of exploration as it samples from the set of achievable velocities during the entire forward simulation, while DWA samples only for one simulation step. Both algorithms work by forward simulating the particle trajectories along polynomial curves and determining whether a collision occurs at each step of the simulation. An optimal collision-free path search (according to the cost map) is performed, and this path is selected as the deployment trajectory and implemented using a velocity controller.

[0157] Recovery Behaviors for Local Path Planning

[0158] Several recovery behaviors are also implemented to prevent the robot from getting stuck in a chaotic environment. These include changing the cost map and examples such as rotational behavior, backtracking, and U-turn behavior.

[0159] Example cost map changes include: (1) Periodically clearing the dynamic cost map (note that the dynamic cost layer is most severely affected by transients in the environment that may impede the robot and prevent it from moving along the path (e.g., a person or a group of people on the robot's path), so periodically clearing the dynamic cost map helps remove them from the cost map once these transient obstacles have left the obstructed field of view); (2) Adjusting the gradient weights of the inflation layer (the gradient weights of the inflation layer in the local vicinity are given decreasing weights to allow additional paths to enable the robot to avoid obstacles, including those that may bring the robot close to known obstacles); and (3) Disabling the attraction layer (when the robot avoids an obstacle, the attraction layer is disabled to avoid further movement towards the obstacle).

[0160] Example local planning behaviors include one or more of the following: (1) Rotation behavior, which is useful when the robot is obstructed in one or more directions but at least one approach vector deviates from the current position (using rotation behavior allows the robot to find the exit direction and leave the obstructed area); (2) Backtracking behavior, which is useful when localization is lost or sensors are severely obstructed - in this case, the robot releases the trajectory, i.e., returns along the path to the current position and attempts to improve its localization accuracy before attempting an alternative route to the target waypoint to avoid the obstructed area; and (3) U-turn behavior, which requires the robot to turn at an appropriate position and drive to a previously visited waypoint before continuing towards its goal (while backtracking assumes the robot's wheels are turning in reverse, the U-turn behavior requires the robot to turn at an appropriate position and this behavior is applicable for long-distance obstacle avoidance).

[0161] Under the guidance of the sensor's observation of the current state of the environment, combinations of the above recovery methods are used in different situations. Rotation behavior is the most effective and thus the most commonly used.

[0162] Motion (Velocity) Controller

[0163] The velocity controller is incorporated into the autonomous mapping system to convert the path selected by the local path planner into velocity commands. Standard velocity-acceleration-distance relationship

[0164] [Equation 14]

[0165]

[0166] where S is the displacement in the trajectory, u is the initial linear velocity of the robot, a is the acceleration of the robot, t 0 and t n are the initial and final times at which the control is applied to dynamically adjust the velocity to reach the desired trajectory point. The linear velocity and rotational velocity are converted into the radial and tangential velocities of the differential drive wheels. The velocity commands are sent to the motor drivers on the robot.

[0167] SLAM for Map Generation

[0168] The online SLAM backend is used to generate and update the map to represent the current state of the environment. One or more embodiments use filtering methods instead of graph-based methods for SLAM, where these methods are implemented to support the online mapping mode. Additionally, predictive mapping utilizes loop closure in SLAM. The occupancy map generated by SLAM is also used for boundary identification and global / local path planning. Scan matching in SLAM may also provide the gradient direction for kinematic dynamic prioritization. Thus, this module runs continuously in parallel with the detection suite and other planning and execution modules.

[0169] User interface

[0170] The autonomous mapping system also provides user interaction in the form of four user interfaces that are active at different times during the execution of the system.

[0171] These interfaces include interfaces for (1) survey customization, (2) real-time feedback and survey control, (3) offline map feedback and map editing tools, and (4) analysis and recommendation systems.

[0172] Survey customization: This interface provides the ability to customize survey parameters and constraints before a survey run is executed.

[0173] Real-time feedback and survey control: This interface provides the user with real-time feedback on the survey process during a survey run and allows the user to monitor and control the run.

[0174] Offline map feedback and map editing tools: Once a survey run is complete, this interface displays the final map to the user and allows the user to edit it.

[0175] Analysis and recommendation system: This interface presents a summary of the survey run, presenting an analysis and evaluation of key metrics during the run. It also includes a recommendation system that advises the user on customizing / adjusting future survey runs.

[0176] Figure 6 A survey status diagram related to the above interfaces is depicted.

[0177] User interface - Survey customization

[0178] The survey customization interface provides various options that the user can select / adjust before the autonomous mapping process begins. These include termination criteria for any one or more of survey time, roaming distance (the distance the robot is allowed to travel), boundary size limits (minimum and maximum), threshold limits (whether the robot is allowed to enter a new room through a potential doorway), spatial frame (the user marks the boundary limits of the survey), and source-to-sink behavior (where the robot stops surveying once it reaches a sink target - a landmark target such as a dock or an image target). Additional or alternative criteria can be specified in other embodiments.

[0179] The survey customization interface also provides the user with options to select various static path behaviors, such as wall following (followed by circling in the interior area using only odometry) in extremely sparse and featureless environments, and rotational behavior to map small rooms extremely quickly while minimizing clutter.

[0180] The user interface is also designed to support various parameter adjustment options to safely detect unpredictable environments in the real world. User interface options for fixed trajectories are also provided and adjusted for sparse featureless environments. In terms of the detection strategy, the user interface also provides options such as the minimum boundary dimension to be considered, the distance to the boundary, the priority of the boundary, etc. In addition, weight and parameter options for multi-objective global optimization and local refinement are proposed. Figure 7 A simplified representation of a user-customized example interface is shown.

[0181] Note that the "interface" discussed is implemented by an underlying processing circuit that generates display data, receives user input, etc. The display can be included on the robot or external, such as included in a laptop or other computing system. The program instructions for providing the user interface can be executed partially or fully on the robot. For example, using a connected laptop or other display system used as a terminal, or the connected computer can run code that provides a visual display, and the autonomous mapper runs complementary computer code that allows it to send current settings, receive adjusted settings, etc.

[0182] User Interface - Real-Time Feedback and Detection Control

[0183] The user feedback interface provides real-time feedback during the autonomous mapping process. In addition to providing the current environmental map in the form of an occupancy grid, the real-time interface also displays the current position of the robot, obstacles detected through on-board sensing (laser, sonar, etc.), local and global cost maps for path planning, the locally planned path, the globally planned path, and most importantly, the boundaries that the autonomous mapping will detect. These boundaries are also ranked in the order of access priority. Boundaries on the blacklist are represented in a unique color. The interface also provides options to reorder or increase / decrease the priority (by clicking the corresponding button in the screen display) for which the boundaries will be accessed. In addition, the user can click on a checkbox to blacklist or whitelist a boundary. Active / inactive boundaries (manually or automatically blacklisted by the blacklist table) are also shown to the user. Automatically blacklisted boundaries may be reactivated after the blacklist period ends.

[0184] When the user anticipates that the robot will perform a dangerous or unsafe operation, a prompt can also be used to abort the exploration. The prompt is also useful when the robot enters an area or follows a path that it should not map or navigate. In this case, spatial brackets or obstacles need to be set up, and at this time, the user can resume mapping by selecting the prompt again. In addition, when the user determines that the necessary mapping coverage of the environment has been obtained and is sufficient for navigation during the mission phase, this prompt can be used to terminate the autonomous mapping process and complete the map generation. A prompt is also provided to roll back the trajectory that the robot has recently traversed for mapping. Similar to the abort prompt, this is useful when the robot enters an area or passes through an environment that should not appear in the final map. The roll-back trajectory prompt allows the user to roll back any number of paths starting from the latest path. During the roll-back operation, the robot retraces along the paths taken, reaching the starting positions of these paths. The map sections added during these paths will also be deleted from the current map. A simple representation of the user feedback interface is as Figure 8 shown.

[0185] User Interface - Offline Map Feedback and Map Editing Tools

[0186] Providing manual map editing tools through the user interface serves at least two purposes. First, the map generated using the autonomous mapper can be refined using the manual map editing process. As part of the map editing process, any parts missed by the autonomous exploration can be added. The map editing tools allow the user to specify the missing boundaries. The boundaries are defined based on the polygon vertices specified by clicking on the map visualization interface. In addition, the map editing tools can be used to merge the parts of the environment mapped independently by different robots, whether there is overlap or not. The map editing tools provide options to pan and rotate the maps to bring them into the same global coordinate system and align the various structures in the maps. Merging maps is also a useful function for leveraging maps generated at different times. For example, while the map corresponding to a large environment is only mapped once, a particularly busy room may require periodic re-mapping. The map editing process helps to overlay the latest version of the room on the existing base map and align it with the pre-existing structures, eliminating the need for a complete re-mapping of the environment.

[0187] In addition, once the refined map is available, it can be used to evaluate and adjust the performance of the autonomous mapper. In this case, the refined map can be used as the ground truth to compare with the map of the autonomous mapper to determine the coverage of the mapped environment. For example, if a bottleneck corresponding to the minimum size is identified in an undetected area (such as a narrow corridor entrance) related to the ground truth, the minimum size of the wavefront to be detected can be reduced.

[0188] User Interface - Analysis and Recommendation System

[0189] In addition to the feedback in the form of maps that can be used for editing, in one or more embodiments, the following measurements and state variables are also obtained / computed to generate evaluation criteria, operation profiles, and statistical measurements for use in probing analysis and optimization as an offline process. Based on these values presented to the user through the user interface, the user can appropriately pre-configure additional probing runs in the same or similar environments according to the suggestions proposed for each metric.

[0190] The autonomous mapping analysis user interface allows access to any one or more of the following metrics, which are widely referred to herein as "probing analysis": power usage, power consumption, coverage rate, boundary detection rate, entropy, odometer x-y, odometer θ, twist linear, twist angle, global planning trigger, global planning neighborhood trigger, local planning neighborhood trigger, target x-y, control target acceptance delay, commanded speed linear, commanded speed angle, control feedback pose x-y, control feedback pose θ, and boundary count. Details of such probing analysis are as follows.

[0191] Power Usage - This metric / curve describes the power consumption measured from the robot's battery interface during a probing run versus time / coverage. This helps to minimize battery usage until probing convergence. If the user wants to reduce battery usage, they may want to adjust the probing convergence / termination point (time, distance, or boundary distance and size stop criteria), as the robot can spend a significant amount of time within 90% to 100% coverage.

[0192] Power Consumption - Similar to the power usage metric, this is measured from the robot's battery interface during a probing run. It may be necessary to avoid high power consumption for long periods (i.e., rapid voltage drop), as this can be harmful to battery life. The user may want to appropriately set the kinematic dynamics prioritization (momentum) weights in the optimization problem to reduce sharp turns and obtain the desired behavior.

[0193] Coverage Rate - The coverage rate is the main evaluation method for probing. The coverage rate is calculated as the percentage of area (m^2) / pixels relative to the ground truth map. The coverage rate is measured over time and is an indicator against which other adjustable parameters can be benchmarked. If the final coverage rate is low, the user may want to relax the stop criteria. Ideally, we want the coverage rate to be close to 100%.

[0194] Boundary Detection Rate - The boundary detection rate is calculated as the percentage of boundary pixels detected relative to the ground truth map. The boundary detection rate is measured over time and provides a supplementary indicator to the coverage rate. We also want this indicator to be close to 100%.

[0195] Entropy - As mentioned before, it increases with the discovery of uncharted areas and decreases with mapping. If the robot spends too much time in the exploration mode (selecting boundaries that require crossing visible space) or turning back, the user can increase the entropy threshold to allow for faster exploration.

[0196] Odometer x, y - This is recorded using the robot odometer interface. This helps determine the boundaries of the explored space and the time trend the robot spends in mapping each area. This is very useful for adjusting the framework variable (the spatial range to be mapped).

[0197] Odometer θ - Similar to (vi). This helps identify the trend of a typical robot going to a given explored space. This also determines the number of turns the robot makes during exploration. This can guide the user to change the weights of kinematic dynamic prioritization.

[0198] Twist linear - The curve of this variable is obtained according to the commanded speed. This helps ensure that the robot adheres to the maximum linear speed limit, also helps identify typical speeds and the time spent at that speed, and the variation of speed distribution in a given space. This may be helpful if the user wants to adjust parameters in a space with humans and other dynamic obstacles to ensure higher safety and predictable behavior.

[0199] Twist angle - This helps ensure that the robot adheres to the maximum angular speed limit when turning, also helps identify typical speeds and the time spent at that speed, and the variation of the speed curve and the frequency of the robot turning in a given space. The user adjustment is the same as above, but angle twist indicates dynamic obstacles more than linear twist.

[0200] Global planning trigger - This timing data is obtained from the global path planner. This helps determine when a new global path plan is available and the length of each global plan. This also indicates how far the boundaries are. These should be synchronized with the new boundary availability, helping to optimize and keep the number of additional global planning triggers for each boundary target low, thus reducing the computational cost.

[0201] Global planning neighborhood trigger - This timing data is obtained from the local path planner. This helps identify when new global planning neighborhoods (for following local path plans) are available and indicates their lengths - longer lengths are preferred as the robot can go further and faster, and cluttered planning indicates that global planning and replanning are not feasible, so the robot has a higher computational load. This indicates that the space is narrow and dynamic. The user may want to adjust the cost map weights (especially the inflation layer) to allow passage through narrow channels and corners in such cases (possibly with deceleration).

[0202] Local Planning Neighborhood Trigger - This timing data is obtained from the local path planner. It helps identify when a new local planning neighborhood is available and indicates its length - a longer length is preferred as the robot can travel further and faster, and a cluttered plan indicates an infeasible local plan and replanning. Similar considerations apply as for global planning, but this variable is more affected by dynamic obstacles.

[0203] Goal x, y - This timing data comes from the sensing suite. It indicates when new boundary goals are available and how much time the robot spends in each part of the environment mapping the boundaries (and uncharted areas).

[0204] Control Goal Acceptance Delay - This timing data is obtained from the motion controller. It indicates that there are complex global and local paths, so the number of infeasible planned paths and replanning attempts before a feasible path is found is unfeasible. If this metric becomes too large, the user may wish to limit the number of attempts.

[0205] Command Velocity Linear - Obtained from the motion controller, this identifies the commanded linear velocity that the velocity controller sends to the motor controller, ensuring the robot adheres to the maximum velocity limit. Cluttered points indicate greater stress on the motor. The user may wish to reduce the maximum local planning rate and the robot's speed to relieve stress on the robot's motor.

[0206] Command Velocity Angular - Also obtained from the motion controller, this identifies the commanded angular velocity that the velocity controller sends to the motor controller, ensuring the robot adheres to the maximum velocity limit. Cluttered points indicate greater stress on the motor. User adjustment is similar to the linear case.

[0207] Control Feedback Pose x, y - Obtained from the motor encoder data, this identifies the pose (position) of the robot estimated by the controller, which needs to be close to the odometry x, y. The difference from the command may indicate slippage. The user may wish to adjust the SLAM odometry noise model or modify the floor conditions (if possible) to account for errors caused by such deviations.

[0208] Control Feedback Pose θ - Obtained from the motor encoder data, this identifies the pose (heading) of the robot estimated by the controller, which needs to be close to the odometry θ. User modification is as described above.

[0209] Boundary Count - This metric is obtained from the sensing suite. It gives the boundary pixel count, determining when new boundaries are available (by large values) when the robot reaches a new uncharted room or other space. As the robot's coverage of the environment increases, the typical trend should gradually approach zero over time. This variable can be used by the user to determine the maximum boundary size to be explored in a specific environment and thus determine the relevant stopping criteria.

[0210] Figure 9 Shown is a mobile robot 10 according to an example embodiment in terms of its constituent entities or sub - components, where the mobile robot 10 includes a control system 12 that provides a runtime environment 14 in which an autonomous mapper 16 is instantiated and executed. Thus, the robot 10 running the autonomous mapper 16 can be regarded as an example autonomous mapping system, where the autonomous mapper 6 is configured or operated according to any of the above - described "autonomous mapper" embodiments.

[0211] The term "runtime environment" refers to a program execution environment provided by the control system 12, such as one or more microprocessors and corresponding memories, and it can be configured or otherwise controlled by an operating system (OS), such as a real - time OS (RTOS) running on the robot 10.

[0212] The robot 10 also includes a drive system 18, one or more sensor systems 20, one or more interfaces 22, and one or more power systems 24. The control system 12 interfaces with or otherwise interacts with these different entities for the overall control and operation of the robot 10. In this regard, the control system 12 includes fixed circuitry or programmatically configured circuitry or a combination of both. In at least one embodiment, the control system includes processing circuitry for communicating with or otherwise interacting with the various constituent entities (such as the drive system 18, sensor system 20, etc.) on the robot 10. In at least one example, such processing circuitry includes or comprises one or more microprocessors or other digital processors that are specifically adapted according to the execution of computer program instructions stored on an on - board computer - readable medium and supported input / output circuitry.

[0213] Figure 10 Shown is an example embodiment of the drive system 18, which includes one or more motors 30 that provide power to drive wheels 32. Although not explicitly shown in the figure, the robot 10 may include motor control circuitry that converts the motion commands generated by the processing circuitry of the control system 12 into corresponding motor drive signals. Such circuitry can be considered part of the control system 12 or part of the drive system 18, or a bridging circuitry between them. The drive system 18 also includes one or more encoders 34 that provide odometry feedback from the drive wheels for dead - reckoning calculations performed by the control system 12.

[0214] Figure 11An example implementation of the sensor system 20 is shown, which includes any one or more of the following: one or more lidar devices 40, one or more ultrasonic devices 42, or one or more cameras 44. These sensors allow the robot 10 to scan or otherwise sense its surrounding physical environment, including obstacles, structures, etc., in order to perform autonomous navigation and mapping in the environment. Therefore, these sensors can be referred to as "environmental" sensors. For example, the lidar device 40, such as a laser scanner, scans laser pulses within a certain angular range in the horizontal plane, and the distance to the reflecting object is determined based on the time of flight of the pulses reflected by the laser scanner. In one or more embodiments, the robot 10 has multiple laser scanners located at different heights for more robust and reliable detection of objects in the surrounding physical environment.

[0215] Figure 12 An example of the interface system 22 is shown, including a communication interface circuit 50 for wired or wireless communication. In the example description, the communication interface circuit 50 includes a first transceiver circuit, which includes a first transmitter circuit 52-2 and a first receiver circuit 54-1 for wired communication, such as an Ethernet connection or other computer data connection. In this regard, the communication interface circuit 50 will be understood to provide a physical layer transmitter and receiver circuit, as well as a protocol processor, such as for implementing one or more communication protocol stacks. Alternatively, the processing circuit within the control system 12 provides protocol processing, and the communication interface circuit 50 provides a physical layer signaling connection.

[0216] In an example implementation, the transmitter circuit 52-1 and the receiver circuit 54-1 provide a communication link for connecting to a laptop or other external computing device running software that provides access to the user interface customization functions provided by the autonomous mapper 16.

[0217] In at least one embodiment, the communication interface circuit 50 includes a wireless communication interface, which includes a second transceiver, and the second transceiver includes a transmitter circuit 52-2 and a receiver circuit 54-2 that are interface-connected to one or more antennas 58 through an antenna interface circuit 56. Such a circuit implements a Wi-Fi or other radio communication link, and it can be used to command and control the robot 10 during real-time operation. In at least one embodiment, the wireless interface provides communication access for customizing the detection behavior exhibited by the user interface-based autonomous mapper 16.

[0218] The interface system 22 may also include a control panel, such as a touch screen on the robot 10 or other local user interface (UI), for example, allowing an operator to check the status, input commands, etc. Figure 13An example partial UI 60 and an effector 62 are shown, which may be mounted on a robotic arm, for example. Of course, the physical details of the robot 10 depend on its intended purpose, e.g., to the extent that it is not dedicated to autonomous mapping operations. For example, the robot 10 may be equipped with a load-bearing platform for material transportation and the like.

[0219] Figure 14 Example implementation details of the control system 12 are shown. The processing circuit 70 implements at least some of the processing logic for implementing the autonomous mapper 16, and it includes, for example, one or more microprocessors 72, and the microprocessor 72 includes a memory 74 that stores one or more computer programs 76 and supports configuration data 78 or is associated with the memory 74. The computer program 76 contains computer program instructions that, when executed by the microprocessor 72, specifically cause them to be adapted to perform the processing operations for implementing the autonomous mapper 16.

[0220] The memory 74 includes one or more types of computer-readable media and provides non-transitory storage of program instructions and data. "Non-transitory" does not necessarily mean permanent or unchanging, but rather means at least some persistence of storage. The memory 74 may include a mixture of volatile or working memory and non-volatile memory, and example circuits, devices, or components include any one or more of SRAM, DRAM, FLASH, EEPROM, solid-state drives (SSDs), etc.

[0221] The microprocessor 72 interfaces with one or more motor controllers 80, and the motor controllers 80 provide drive signals to the motors 30 in the drive system 18 and interface with the sensor system 20 through a sensor interface 82. Such an interface may include a data bus-based interface for exchanging sensor control signaling and sensor data that may be preprocessed by the sensor system 20. Additionally, or alternatively, such an interface includes discrete analog and / or digital input / output (I / O). Furthermore, the term "microprocessor" has a broad meaning and includes a range of implementations or types, such as microcontrollers, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or field-programmable gate array (FPGA) cores, system-on-chip (SoC) variants, etc.

[0222] Figure 15 An example arrangement is shown in which the robot 10 implements the autonomous mapper 16 described herein. An external computer 90 is connected to a wireless LAN (WLAN) 92, and the WLAN 92 in turn communicatively connects the external computer 90 to the robot 10.

[0223] The robot configuration software 100, which executes on an external computer 90, provides a user interface 102 for user customization of the exploration behavior of the autonomous mapper 16, as described in the user interface section of this document above. Here, the robot configuration software 100 either implements the user interface 102 or provides a terminal function through which a configuration function 104 implemented via the autonomous mapper 16 on the robot 10 causes the user interface 102 to be displayed on the external computer 90. In either implementation, the configuration function 104 (also referred to as an exposure function) is configured to use modifiable values of the corresponding operation parameters that drive the exploration behavior of the autonomous mapper 16 for user inspection and adjustment. For example, the configuration function 104 uses stored values that are determined, modified, prioritized, or weighted based on signaling received through the user interface or derived from user input received through the user interface. Alternatively, the configuration function 104 obtains values from a configuration file or data structure loaded into a memory or other computer-readable medium included in the mobile robot.

[0224] As an example of user customization, the external computer 90 passes multi-object preferences 106 to the autonomous mapper 16, where these preferences enable / disable the corresponding ones of the above-mentioned multiple optimization objectives, or otherwise assign relative prioritization or weights to control how the multi-object optimization performs its joint optimization of the multiple objectives. Figure 16 This arrangement is illustrated, where the multiple exploration objectives considered in the multi-object optimizer have corresponding configuration weights or prioritizations that the user can select or adjust to customize the exploration behavior of the autonomous mapper 16 - for example, biasing Figure 1 towards consistency or towards exploration speed.

[0225] Figure 17 An autonomous mapping method 1700 performed by the mobile robot 10 according to an example embodiment is shown. For example, the processing circuit 70 of the mobile robot 10 is configured to implement Figure 17 the operations included therein. Figure 18 A method 1800 is shown, which can be understood to provide example details of the method 1700.

[0226] In one embodiment, a further method includes a method of customizing the exploration behavior of a mobile robot that performs mapping using the autonomous mapper described herein. The method includes displaying a user interface corresponding to the physical environment to be mapped by the mobile robot. Here, the autonomous mapper instantiates an autonomous mapper for exploring the physical environment based on a multi-objective optimization of two or more exploration targets considering their respective target types to select an exploration boundary. The method further includes the steps of: in response to receiving a corresponding user input via the user interface, performing any one or more of the following operations: restricting the search space and mapping area; prioritizing the mapping area; selecting termination criteria and boundary strategies; selecting exploration strategies, constraints, and parameters; selecting a static trajectory; customizing relative weights or priorities for multiple exploration targets; or specifying safety constraints.

[0227] According to one embodiment, a computer-implemented method for providing real-time feedback and exploration process control of a mobile robot includes visualizing a map generated by the mobile robot, which operates according to the autonomous mapper details described herein, and the visualization depicts any one or more of obstacles, robot position, planned global and local paths, cost maps and boundaries, boundary priorities, blacklist status. The method further includes: providing user input controls for changing boundary priorities, blacklist status, aborting or stopping the exploration process in response to the robot performing an unsafe operation, and rolling back a variable number of trajectories in response to incorrect mapping.

[0228] A computer-implemented map editing method according to one embodiment includes providing a user interface that enables a user to complete or correct a map generated by a mobile robot according to the autonomous mapping described herein. The method includes providing the user with options to merge maps generated by the mobile robot from multiple exploration runs.

[0229] In at least one embodiment, the autonomous mapper includes a method of calculating exploration analysis, which includes one or more of the following steps: performing an evaluation of a previous exploration run; providing an output including feedback to the user in the form of a recommendation system; and providing an output to the user including suggestions for customization and adjustment of future exploration or remapping runs.

[0230] A computer-implemented user interface method according to one embodiment includes, with respect to the exploration performed by a mobile robot according to the autonomous mapping described herein, performing one or more of the following operations: presenting a summary of the exploration runs performed by the mobile robot; presenting the exploration analysis and key metrics of the exploration runs; and suggesting fine-tuning of future exploration runs of the mobile robot.

[0231] In at least one embodiment, the autonomous mapper described herein is applied to use cases of automatically remapping changed portions of an environment relative to a previous map of the environment obtained manually or using the autonomous mapper. Additionally, in the same or another embodiment, the autonomous mapper described herein is applied to use cases of exploratory navigation for discovering known targets at unknown locations in an environment.

[0232] It should be noted that those skilled in the art, benefiting from the teachings presented in the above description and the related drawings, will conceive of modifications and other embodiments of the disclosed invention. Accordingly, it is to be understood that the invention is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the present disclosure. Although specific terms may be employed herein, they are used only in a general and descriptive sense and not for purposes of limitation.

Claims

1. A method for autonomous mapping of a mobile robot, the method comprising the following steps: Based on solving a multi-objective optimization that jointly evaluates a set of two or more detection targets to determine the attractiveness of each candidate boundary as the next detection target, using an online simultaneous localization and mapping (SLAM) algorithm to identify the next boundary to be detected from two or more candidate boundaries in a map generated by the mobile robot, each of the two or more detection targets being a corresponding type of detection target corresponding to a corresponding detection behavior; Planning a global path to a global waypoint on the next boundary; And Navigating to the global waypoint according to the global path based on collision avoidance and path parameterization determined by local path planning driven by an environmental sensor on the robot.

2. The method according to claim 1, wherein the set of two or more detection targets includes one or more mapping speed targets and at least one of a mapping coverage target or a mapping consistency target, the mapping speed target biasing the detection behavior of the mobile robot towards mapping speed, the mapping coverage target causing the detection behavior of the mobile robot to be biased towards the integrity of mapping, and the mapping consistency target causing the detection behavior of the mobile robot to be biased towards the correctness and accuracy of mapping.

3. The method according to claim 1 or 2, the method further comprising receiving a signaling, and in response to the received signaling, adjusting which two or more detection targets in the defined set of detection targets considered in the multi-objective optimization, or determining an importance weight that modifies the prioritization of a single detection target among the two or more detection targets considered in the multi-objective optimization.

4. The method according to claim 3, wherein the received signaling is received together with an exchange signaling with an external computer that displays a user interface for user customization of the autonomous mapping behavior of the mobile robot.

5. The method according to any one of claims 1 to 4, the method further comprising applying a user-configured weight to at least one of the two or more detection targets to adjust the multi-objective optimization according to user preferences.

6. The method according to any one of claims 1 to 5, wherein the two or more detection targets include two or more of the following: A distance-to-target target that indicates the shortest path distance from the current position of the mobile robot to a corresponding candidate boundary; A detection potential target that indicates the detection potential associated with the corresponding candidate boundary; A map coverage target that indicates to what extent the detection of the corresponding candidate boundary improves the coverage of the physical environment by the mobile robot; A map consistency target that indicates to what extent the detection of the corresponding candidate boundary makes use of a previously detected area of the physical environment; And A kinematic dynamics target that indicates the trajectory smoothness of moving from the current position of the mobile robot to a global waypoint defined on the corresponding candidate boundary.

7. The method according to claim 6, wherein, the kinematic dynamic goal places a lower priority on paths that tend to require deviation from the current heading of the mobile robot.

8. The method according to any one of claims 1 to 7, wherein, the step of solving the multi-objective optimization includes forming and evaluating a scalarized objective from two or more objectives for all candidate boundaries.

9. The method according to any one of claims 1 to 8, wherein, one of the detection goals is a map consistency goal aimed at improving the consistency of the online map generated by the mobile robot, and wherein the map consistency is measured using an entropy metric reflecting the consistency of the online map over consecutive time steps of the SLAM algorithm.

10. The method according to claim 9, wherein, the map consistency goal is represented by a plurality of sub-goals, the plurality of sub-goals including a structural adjacency sub-goal that preferably adjoins a visible structure, a development sub-goal that preferably traverses a previously visited or visible area of the physical environment by the mobile robot, and a predicted closure sub-goal that preferably estimates candidate boundaries that cause the mobile robot to return to a visible area including stable landmarks.

11. The method according to claim 10, wherein, the predicted closure sub-goal is based on the appearance of the physical environment outside the predicted candidate boundary and uses a generative adversarial network GAN with training and exploration operation phases to determine whether loop closure can be achieved by navigating through the candidate boundary, thereby predicting and forcing the mobile robot to pass through a closed physical loop of the environment during the exploration process.

12. The method according to any one of claims 1 to 11, the method further comprising performing a local refinement process, wherein, the local refinement process includes modifying the trajectory to global waypoints by adding waypoints, enabling the mobile robot to capture local map structures from various viewpoints to improve the integrity of the local map.

13. The method according to any one of claims 1 to 12, the method further comprising using a time-based blacklist table to deactivate detection goals that do not have a valid estimated path from the current position of the mobile robot, or detection goals for which performing an effective estimated path results in repeated failures, the deactivation expiring within a fixed time, after which the mobile robot reconsiders the detection goals for exploration.

14. The method according to any one of claims 1 to 13, the method further comprising applying a stop criterion to the exploration of the mobile robot, the stop criterion including one or more of a physical boundary, a virtual boundary, a boundary limit, a detection time, a roaming distance, a threshold limit, or a source-to-sink behavior.

15. The method according to any one of claims 1 to 14, the method further comprising using one or more static trajectory generation methods, the static trajectory generation methods including any one or more of the following operations: using predefined user-specified landmarks, using connected trajectories, using wall-following behavior, and using rotational behavior.

16. The method according to any one of claims 1 to 15, wherein, based on the multi-layer cost map, the global path pair to the global waypoints on the next boundary imposes multiple constraints on the shortest path calculation with respect to the global waypoints.

17. The method according to claim 16, wherein, the multi-layer cost map includes: a first layer, the first layer includes a two-dimensional (2D) static occupancy grid created by the mobile robot performing simultaneous localization and mapping (SLAM) processing, wherein, for the obstacles registered in the first layer, object avoidance must be performed; a second layer, the second layer includes a dynamic three-dimensional (3D) voxel grid created based on the point cloud data generated by using environmental sensors arranged at different heights on the mobile robot, wherein the second layer is periodically refreshed to consider transient obstacles within the working range of the mobile robot; a third layer, the third layer includes a 2D dilation layer, the 2D dilation layer assigns a higher cost to the navigation of the cells closer to the obstacles in the first two layers of the multi-layer cost map and a lower cost to the farther cells; and a fourth layer, the fourth layer includes a 2D attraction layer, when the robot localization and map consistency deteriorate, the 2D attraction layer biases the mobile robot to travel along the side walls or other detected structures in the physical environment.

18. The method according to claim 16 or 17, wherein, the global path planning is further processed by a local planner, the local planner performs collision avoidance and generates a kinematically dynamically feasible path, and also adopts a recovery behavior including one or both of cost map change and local recovery behavior to free the mobile robot from a trapped state, the local recovery behavior includes one or more of backtracking, rotation, and U-turn.

19. A mobile robot, the mobile robot comprises: a drive system configured to move the mobile robot in a physical environment; one or more sensors configured to sense obstacles in the physical environment within respective sensing ranges; and a processing circuit configured to perform autonomous mapping based on processing sensor data from the one or more sensors and controlling the drive system, wherein the processing circuit performs the autonomous mapping based on being configured to perform the following operations: Based on solving a multi-objective optimization that jointly evaluates a set of two or more detection targets to determine the attractiveness of each candidate boundary as the next detection target, using an online simultaneous localization and mapping (SLAM) algorithm to identify the next boundary to be detected from two or more candidate boundaries in the map generated by the mobile robot, each of the two or more detection targets being a corresponding type of detection target corresponding to a respective detection behavior; Planning a global path to the global waypoints on the next boundary; and Navigating to the global waypoints according to the global path based on collision avoidance and path parameterization determined by local path planning driven by the one or more sensors.

20. The mobile robot according to claim 19, wherein, the set of two or more detection targets includes one or more mapping speed targets and at least one of a mapping coverage target or a mapping consistency target, the mapping speed target biases the detection behavior of the mobile robot towards mapping speed, the mapping coverage target causes the detection behavior of the mobile robot to be biased towards the integrity of mapping, and the mapping consistency target causes the detection behavior of the mobile robot to be biased towards the correctness and accuracy of mapping.

21. The mobile robot according to claim 19 or 20, wherein, the processing circuit is configured to receive signaling and, in response to the received signaling, perform at least one of the following operations: select which two or more detection targets in the defined set of detection targets to consider in the multi-objective optimization; or determine to modify the prioritization importance weights of individual detection targets among the two or more detection targets considered in the multi-objective optimization.

22. The mobile robot according to claim 21, wherein, the processing circuit is configured to exchange signaling with an external computer that displays a user interface for user customization of the autonomous mapping behavior of the mobile robot, and wherein the received signaling is received in the exchange.

23. The mobile robot according to any one of claims 19 to 22, wherein, the processing circuit is configured to apply user-configured weights to at least one of the two or more detection targets to adjust the multi-objective optimization according to user preferences.

24. The mobile robot according to any one of claims 19 to 23, wherein, the two or more detection targets include two or more of the following: a distance-to-target target that indicates the shortest path distance from the current position of the mobile robot to a corresponding candidate boundary; a detection potential target that indicates the detection potential associated with the corresponding candidate boundary; a map coverage target that indicates to what extent the detection of the corresponding candidate boundary improves the mobile robot's coverage of the physical environment; a map consistency target that indicates to what extent the detection of the corresponding candidate boundary makes use of the previously detected area of the physical environment; and a kinematic dynamics target that indicates the trajectory smoothness of moving from the current position of the mobile robot to a global waypoint defined on the corresponding candidate boundary.

25. The mobile robot according to claim 24, wherein, the kinematic dynamics target places a lower priority on paths that tend to require deviation from the current heading of the mobile robot.

26. The mobile robot according to any one of claims 19 to 25, wherein, to solve the multi-objective optimization, the processing circuit is configured to form and evaluate a scalarized objective from two or more objectives for all candidate boundaries.

27. The mobile robot according to any one of claims 19 to 26, wherein, One of the detection goals is a map consistency goal aimed at improving the consistency of the online map generated by the mobile robot, and wherein the map consistency is measured using an entropy metric that reflects the consistency of the online map over consecutive time steps of the SLAM algorithm.

28. The mobile robot according to claim 27, wherein, the map consistency goal is represented by a plurality of sub-goals, the plurality of sub-goals including a structural adjacency sub-goal of a boundary goal preferably adjacent to a visible structure, a development sub-goal of preferably a previously visited area or a visible area through which the mobile robot traverses the physical environment, and a predicted closure sub-goal of a candidate boundary preferably estimated to return the mobile robot to a visible area including a stable landmark.

29. The mobile robot according to claim 28, wherein, the predicted closure sub-goal is based on the processing circuit being configured to predict the appearance of the physical environment outside the candidate boundary and use a generative adversarial network GAN with training and detection operation phases to determine whether loop closure can be achieved by navigating through the candidate boundary, thereby predicting and forcing the mobile robot to pass through a closed physical loop of the environment during the detection process.

30. The mobile robot according to any one of claims 19 to 29, wherein, the processing circuit is configured to perform a local refinement process, wherein the local refinement process includes modifying the trajectory to global waypoints by adding waypoints, enabling the mobile robot to capture local map structures from various viewpoints to improve the integrity of the local map.

31. The mobile robot according to any one of claims 19 to 30, wherein, the processing circuit is configured to use a time-based blacklist table to deactivate detection goals that do not have a valid estimated path from the current position of the mobile robot, or detection goals for which performing a valid estimated path results in repeated failures, the deactivation expiring within a fixed time, after which the mobile robot reconsiders the detection goals for detection.

32. The mobile robot according to any one of claims 19 to 31, wherein, the processing circuit is configured to apply stop criteria to the detection of the mobile robot, the stop criteria including one or more of a physical boundary, a virtual boundary, a boundary limit, a detection time, a roaming distance, a threshold limit, or a source-to-sink behavior.

33. The mobile robot according to any one of claims 19 to 32, wherein, the processing circuit is configured to perform static trajectory generation, the static trajectory generation including any one or more of the following operations: using predefined user-specified landmarks, using connected trajectories, using wall-following behavior, and using rotational behavior.

34. The mobile robot according to any one of claims 19 to 33, wherein, relative to planning the global path to a global waypoint on the next boundary, the processing circuit is configured to impose a plurality of constraints on the shortest path calculation relative to the global waypoint according to a multi-layer cost map.

35. The mobile robot according to claim 34, wherein, the multi-layer cost map includes: The first layer, which includes a two-dimensional (2D) static occupancy grid created by the mobile robot performing simultaneous localization and mapping (SLAM) processing. For obstacles registered in the first layer, object avoidance must be performed. The second layer, which includes a dynamic three-dimensional (3D) voxel grid created based on point cloud data generated using environmental sensors arranged at different heights on the mobile robot. The second layer is periodically refreshed to account for transient obstacles within the working range of the mobile robot. The third layer, which includes a 2D inflation layer that assigns higher costs to navigation of cells closer to obstacles and lower costs to more distant cells in the first two layers of the multi-layer cost map; and The fourth layer, which includes a 2D attraction layer that biases the mobile robot to travel along sidewalls or other detected structures in the physical environment when robot localization and map consistency deteriorate.

36. The mobile robot according to claim 34 or 35, wherein, global path planning is further processed by a local planner implemented by the processing circuit, the local planner performs collision avoidance and generates a kinematically dynamically feasible path, and also employs a recovery behavior including one or both of cost map changes and local recovery behaviors to free the mobile robot from a trapped state, the local recovery behaviors including one or more of backtracking, rotation, and U-turns.

Images