Temporal separation in large-scale image-based positioning.

Abstract

A computer-implemented method and apparatus for generating an image-based positioning model for mobile robot navigation. The method includes the steps of: generating collected data by performing data collection at a plurality of different service locations where a mobile robot swarm can be deployed; dividing the collected data into a plurality of blocks of continuous portions of the collected data; and generating a first image-based positioning model and a second image-based positioning model for a first service location and a second service location, respectively, of the plurality of different service locations, using the collected data. The method further includes the steps of distributing the first image-based positioning model and the second image-based positioning model on a first mobile robot and a second mobile robot, respectively, of the mobile robot swarm, and the first image-based positioning model and the second image-based positioning model are used to navigate the first service location and the second service location, respectively, of the plurality of different service locations.

Description

[Technical field]

[0001] This application claims the benefit of priority to U.S. patent application Ser. No. 63 / 268,792, filed March 2, 2022, which is incorporated by reference in its entirety herein.

[0002] The present disclosure relates generally to robot navigation and robot localization. [Background technology]

[0003] Mobile robot positioning techniques attempt to provide a reliable solution for determining the position a mobile robot may be at any point in time. Estimation of the robot pose (e.g., position and orientation) relative to a coordinate system poses a variety of technical challenges and can be achieved using a robot navigation model. Summary of the Invention [Problem to be solved by the invention]

[0004] Designing and deploying a user-defined model for the operation of a robot in a particular environment may necessitate manually adjusting or "tweaking" many components of the operation model until the user-defined model provides satisfactory performance. Significant technical challenges arise when scaling robot deployments from single instance deployments to production scale across many diverse environments. When the scale of a robot fleet includes a large number of robots (e.g., tens, hundreds, or even thousands of robots), these technical challenges can be considerable and are multiplied as the size of the robot fleet increases. [Brief description of the drawings]

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

[0006] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number indicate the figure number in which that element is first introduced.

[0007] Examples disclosed herein embody an automated robot training and deployment system for an image-based positioning model that is scalable to large robot swarms. Examples disclosed herein embody a number of data collection and data storage strategies or policies for the data used to generate the image-based positioning model.

[0008] For purposes of illustration, a process and system for generating a single model for a given location is described, however, an example of a model generation and deployment pipeline that can be extended to multiple locations is described.

[0009] 1 is a schematic representation of an environment in which multiple mobile robots 104 (e.g., a swarm of service robots) are deployed in a particular location 102 or environment, such as a restaurant, a hospital, or an elderly care facility. Depending on the location, the mobile robots 104 can perform various functions within the location 102. For example, if such a location 102 is a service location, such as a restaurant, the mobile robots 104 can operate to deliver items from a kitchen to tables within the particular restaurant and assist in transporting dishes, trash, etc. from the tables to the kitchen.

[0010] Each mobile robot 104 is communicatively coupled by a network 106 or multiple networks 106 to a cloud service 110 residing on one or more server systems 108 .

[0011] FIG. 2 is a diagram of an exemplary mobile robot 104 that may be deployed within a location 102, such as a cafeteria. The mobile robot 104 has a chassis or housing 202 that houses various components and modules, including a transportation system having wheels or tracks that allows the mobile robot 104 to propel itself within a service location. A power source for the mobile robot 104 may be a rechargeable battery that provides the energy necessary to operate such components and modules. Navigation and perception systems are also housed within the housing 202. The housing 202 supports a number of trays 204 that can hold dishes and other utensils that are transferred from the kitchen to a table and from the table to the kitchen within the location 102. The mobile robot 104 may also have a manipulator (not shown), such as a robotic arm, that allows the robot to manipulate objects or perform tasks such as lifting objects or opening doors.

[0012] The mobile robot 104 may include a number of sensors, including exteroceptive sensors to capture information about the environment or location in which the mobile robot 104 may operate, and proprioceptive sensors to capture information related to the mobile robot 104 itself. The navigation system enables the mobile robot 104 to map its environment and plan a path to reach a destination. The perception system processes the sensor data to recognize objects, detect obstacles, and interpret the operating environment of the mobile robot 104.

[0013] Examples of exteroceptive sensors may include vision sensors (e.g., two-dimensional (2D), three-dimensional (3D), depth, and RGB cameras), light sensors, acoustic sensors (e.g., microphones or ultrasonic sensors), proximity sensors (e.g., infrared (IR) transceivers, ultrasonic sensors, photoresistors), tactile sensors, temperature sensors, search and location sensors (e.g., Global Positioning System (GPS) sensors). Visual odometry and visual Simultaneous Localization And Mapping (SLAM) may help the mobile robot 104 navigate all indoor and outdoor environments where lighting conditions are reasonable and may be maintained in some instances. 3D cameras, depth, and stereo vision cameras provide pose (e.g., position and orientation) information.

[0014] Examples of proprioceptive sensors include inertial sensors (e.g., tilt and acceleration), accelerometers, gyroscopes, magnetometers, compass, wheel encoders, and temperature sensors. An inertial measurement unit (IMU) in the mobile robot 104 can include multiple accelerometers and gyroscopes as well as magnetometers and barometers. The instantaneous pose (e.g., position and orientation), velocity (linear velocity, angular velocity), acceleration (linear acceleration, angular acceleration) and other parameters of the mobile robot 104 can be obtained through the IMU.

[0015] 3 is a block diagram illustrating one aspect of components and modules of the mobile robot 104 according to some examples. The mobile robot 104 includes a robotics open platform 302, a navigation stack 304, and a robotics controller 330.

[0016] The robotics open platform 302 provides various application program interfaces (APIs), which include: ● Device API306 ● Diagnostic API 308 Robotics API 310 ●Data API312 and ●Flock API314 Includes.

[0017] The operation stack 304 is ●Recognition 316 Peer-to-peer (P2P) operation 318 ●Semantic Operation 320 ●Sensor correction 322 ●Sensor processing 324 ● Obstacle Avoidance 326 Includes components that support:

[0018] The ROS execution stack 328 also forms part of the execution stack 304 .

[0019] The robotics controller 330 ●Power management 332 ●Wireless charging 334 ●Device Interface 336 ●Motor Control 338 Includes components that support:

[0020] 4 is a block diagram illustrating another view of components and modules of the mobile robot 104 according to some example embodiments. The mobile robot 104 includes a robotics stack 402 and an application stack 406. The robotics stack 402 in turn includes a perception stack 404 and a navigation stack 304. The application stack 406 provides telemetry 408 and login 410 services for the mobile robot 104.

[0021] 5 shows some example output of the image-based positioning model 604. The generation and maintenance of the image-based positioning model is further discussed in FIGs. 6 and 7 below.

[0022] An image-based positioning model (or image positioning model) can be used by a mobile robot (on which the model is deployed) to navigate an area or environment, such as a service location. One example of an image-based positioning model can receive an image (e.g., an image captured in a particular environment or location) as input and output a set of potential poses or pose estimates. The poses can include candidate locations (e.g., grid cells of configurable size of a grid corresponding to a map of a navigable area). The poses can include a set of (x, y, yaw) values, where x and y are map or world coordinates, and yaw is a yaw angle (corresponding to orientation information). In some examples, the poses can include pitch and roll values ​​(corresponding to orientation information). In some examples, the positioning model can output a set of potential locations, such as grid cells, and predicted x and y offsets within each grid cell along with one or more of the yaw, pitch, and roll values. Each type of prediction (e.g., grid cell, x offset, and y offset, etc.) can be accompanied by a confidence score or probability value that indicates the model's confidence in the prediction.

[0023] 5 shows an example output of an image-based positioning model 604 that takes as input an image (e.g., an image from the front camera of the mobile robot 104) and generates a probability distribution over a set of potential locations represented as grid cells in a grid where each location corresponds to a map of navigable areas. In this example, the model returns a (grid cell, probability) pair list such as [((e,3), 0.5);((b,3), 0.2);((b,4), 0.1);((d,2), 0.1);((d,3), 0.1)]. Thus, in this example, the model can accurately identify the grid cells that correspond to the most likely locations where the mobile robot 104 is located in a given environment.

[0024] 6 is a block diagram illustrating a model system 602 that operates to generate and maintain image-based positioning models 604 that are deployed to multiple mobile robots 104 at one or more locations 102, according to some illustrative examples. The model system 602 can include the following components or modules: ● A data collection and preparation module 606; ● A model training and evaluation module 608; ● a model placement module 610; and ●Model Refresh Module 612

[0025] Further details regarding the operation of such an example module are provided below.

[0026] FIG. 7 is a flow chart illustrating a model workflow 702 for generating, maintaining, and deploying an image-based positioning model according to some examples. The model workflow 702 may be performed by the model system 602 illustrated in FIG. 6 within the context of the environment illustrated in FIG. 1. The workflow 702 begins with data collection by the data collection and preparation module 606 at a location 102, such as a cafeteria (e.g., map metadata, images, pose data, or pose messages, as illustrated in more detail in FIG. 8). The data collection and preparation module 606 employs a data collection strategy that can use sensors of the mobile robot 104 at the location 102 to compile the data (see below for more information on data collection strategies). In some examples, the data collection strategy can also include using sensors separate from the mobile robot 104 that are installed at the location 102 or movably positioned within the location 102 (e.g., manually captured by a user-held sensor). A sufficient amount of data for the location 102 is collected and stored in a cloud repository (e.g., as part of the cloud service 110). The sufficiency of the amount of data collected may be determined objectively based on a number of criteria, including area coverage ratio of a location, volume-based criteria, or data resolution criteria. The data may be stored in a variety of formats (e.g., images may be stored in formats such as JPG, GIF, PNG, etc., map metadata and / or image or pose messages may be stored in formats such as YAML, XML, JSON, etc.).

[0027] The collected data is then preprocessed (or prepared) using operations that convert the collected raw data into a dataset in the form of prepared data that can be consumed by the model training and evaluation module 608. Data preprocessing or preparation operations include subsampling and splitting the data into train / dev / test sets (or train / validation / test sets), as described in more detail in the description of FIG. 8 and FIG. 10. The preprocessed data is then stored (e.g., in a cloud repository) using an appropriate format (e.g., CSV, Parquet, etc.). The preprocessed data is then used to train the image-based positioning model 604 using the preprocessed data and cloud machine learning capabilities (see the "Model Training and Evaluation" section below). The trained model is then stored / stored in a remote repository, such as a cloud repository (e.g., part of the cloud service 110), in an appropriate format and optionally compressed (e.g., a "model.tar.gz" file corresponds to a model stored in a compressed .tar file or tarball, which is a vowel of the file, and compression is achieved using the gzip utility). Other exemplary compression utilities include bzip2, zstd, etc.

[0028] The trained image-based positioning model 604 is then deployed to the mobile robot 104 at the given location 102. In some examples, the trained image-based positioning model is used by an image localizer, which is a Robot Operating System (ROS) node that provides image-based positioning responses to client requests (see FIG. 12 for more information).

[0029] Additional details for the steps of workflow 702 are provided in the subsections below.

[0030] Data Collection Strategy

[0031] The data collection and preparation module 606 employs a data collection strategy that can use the sensors of the mobile robot 104 in the location 102 to gather data including map metadata, images (e.g., image messages), poses (e.g., pose messages containing pose estimates), etc. In some examples, the image and pose messages can be robot operating system (ROS) messages, and the pose messages can all contain position and orientation data.

[0032] The service mobile robot 104 may use a probabilistic localization method to calculate a pose estimate for the service robot in the context of a pre-computed map of the service environment. The service mobile robot 104 may use, for example, an Adaptive Monte Carlo Localization (AMCL) method (e.g., as embodied by an AMCL node in the ROS navigation stack as shown in FIG. 7) using automatically acquired laser scanning data. In some examples, the service mobile robot 104 may use other localization algorithms (e.g., General Monte Carlo Localization (GMCL)) and the same or additional types of automatically acquired service environment data (e.g., mileage data, sonar data).

[0033] The image and pose messages may also be further attributed a timestamp that identifies the day (e.g., date) and time information when the payload information was captured or generated, and a mission identifier that identifies the particular mission or movement during which the payload information was captured. The timestamp and mission identifier information may be used, for example, to perform evaluation, verification, and visualization of the data collection process by the mobile robot 104. The sensors used to collect data may include the sensors previously described in connection with the mobile robot 104, including, for example, multiple RGB cameras, lidar sensors, and radar sensors. While the primary example describes data collection by a mobile robot, alternative or additional collection processes may include fixed sensors, such as cameras, lidar, or radar, located at different points within the environment. Such sensors may capture information over time to generate a more complete picture of the environment.

[0034] In some instances, the data collection strategy employed by the data collection and preparation module 606 at the location 102 may be intentional or passive.

[0035] Intentional data collection may refer to the process of sending the mobile robot 104 to specifically perform a desired mission to collect data (e.g., image messages, pose messages, map metadata, etc.) using various sensors. An advantage of intentional data collection is that it can provide adequate (e.g., more comprehensive and uniform) data coverage for many accessible points on a map.

[0036] Passive data collection may refer to the process of sampling data (e.g., at a configurable rate) while the mobile robot 104 performs normal delivery operations and uploading it to the cloud service 110. In this case, in some instances, data collection can be turned on and off via a config flag. Passive data collection does not require any additional service robot movement at a location and therefore does not disrupt normal diner / client operations.

[0037] The examples may cover both passive and intentional data collection, however, for purposes of explanation, passive data collection will be described below with reference to the example mobile robot 104.

[0038] Data Retention Policy

[0039] When data collection is generally activated, large amounts of data are accumulated by the data collection and preparation module 606. Insufficient data can cause a bottleneck in many machine learning applications, while too much data can be expensive, difficult to handle, and sometimes unnecessary. To address this, a data reduction strategy is implemented by the data collection and preparation module 606 that prevents the amount of data from becoming arbitrarily large while still ensuring that enough data is stored to train new models.

[0040] Exemplary data retention strategies or policies are age-based and volume-based.

[0041] Age-Based Retention Policy: This exemplary policy retains data for a defined period of time, such as one month or six months, after which the data is deleted or archived. For example, a data retention policy may specify that data older than 30 days is deleted from the system.

[0042] Volume-Based Data Retention Policy: This exemplary policy only stores up to a defined amount of data, expressed as a number of images, amount of data, or both (e.g., 5,000-10,000 images or 500MB-1GB of data).

[0043] Because mobile robots 104 at different locations 102 are used at different frequencies (e.g., some restaurants may be busier than others), volume-based storage policies and associated storage operations may be implemented to ensure that a uniform amount of data is stored across multiple locations 102.

[0044] Other exemplary conservation strategies or policies that may be used include the following.

[0045] Rolling Window Retention Policy: In an exemplary policy, the retention period is not fixed, instead data is retained for a rolling window of a certain period, e.g., the last 30 days or the last 90 days. As each window ends, the oldest data is deleted or archived to make room for new data. For example, a rolling window retention policy may specify that only the last 30 days of data will be retained at any given time. This is intended to ensure that only the most recent data is retained, and is particularly useful for applications where data is constantly changing.

[0046] Quality-Based Retention Policy: This exemplary policy retains data based on the quality of the data rather than the volume of the data. For example, data that is highly accurate, complete, or relevant may be prioritized for retention, while data of lower quality may be discarded. This policy may be implemented when the quality of collected data is variable, and the data retention policy may be adjusted according to the data quality metric.

[0047] Hybrid Retention Policy: Under such an exemplary policy, volume, age, and / or quality considerations are taken into account when determining what data to retain. For example, a certain amount of data may be retained, but higher quality data may be retained in preference to lower quality data. Such a hybrid approach may strike a balance between having enough data for model training and ensuring high quality data.

[0048] Adaptive Retention Policy: Under such an exemplary policy, retention may be automatically adjusted based on changes in data volume, age, quality, and / or other relevant factors. For example, if data volume exceeds a certain critical value, the retention policy may be adjusted to reduce the amount of data stored. Similarly, if data quality decreases, the retention policy may be adjusted to preserve higher quality data. This approach may help ensure that data retention policies remain effective over time, even as conditions change.

[0049] Data Preparation

[0050] 8 is a flow chart illustrating some example data preparation flows 802, as may be performed by the data collection and preparation module 606 to generate data stored in the cloud repository 804. Data preparation or pre-processing in the data preparation flow 802 involves converting the raw data collected into a dataset in the form of prepared data 820 that may be directly consumed by the model training and evaluation module 608 in some examples.

[0051] The collected raw data includes map metadata 818, pose data, or pause messages 806, and images (or image messages) 808. As illustrated in Figure 8, the images 808 and pause messages 806 may be collected independently, so a matching operation 810 matches the images 808 with the corresponding pause messages 806 (e.g., based on their timestamps) to generate matching data 814. An image may be matched with a pause message if the respective timestamps are identical or the difference between the respective capture or collection / generation times as calculated based on the timestamps is less than a predetermined time difference (e.g., up to 0.2 or 0.3 seconds).

[0052] In an assignment operation 812, the pre-processing logic of the data collection and preparation module 606 divides the map of the location 102 (e.g., as represented in map metadata 818) into a grid of cells (see FIG. 9 for an example of map 902). Each robot location (e.g., position information or coordinates captured by a pose message 806) is assigned to a specific grid cell (having a configurable size) of the map. The output of the assignment operation 812 is stored as grid assignment data 816. The matching operation 810 and the assignment operation 812 may be performed at the robot mission level, and in some examples may be performed on a batch basis (e.g., daily).

[0053] When raw data (e.g., pose messages 806 and images 808) are collected (e.g., passively) in a natural working environment, certain areas may be passed through by the robot more frequently than other areas, and therefore certain locations may be more prominently represented in the collected data set than other locations (see FIG. 9). For example, in a dining environment, a narrow hallway between the kitchen and dining area may be included as a commonly passed area. To ensure sufficient coverage of the passed environment, the pre-processing logic of the data collection and preparation module 606 may subsample (or resample) the raw data to realign the data distribution so that it is more uniform throughout a navigable space, such as the location 102. In some examples, given a set of (e.g., image, pose) pairs augmented with pose-level lattice cell assignment information, the data collection and preparation module 606 may individually sample from each subset corresponding to a particular lattice cell (e.g., subsample more frequently occurring lattice cells).

[0054] A partitioning operation 822 divides or separates the grid-assigned data 816 for training purposes. For example, the grid-assigned data 816 may be partitioned into a training / validation set 824 (or training / dev set 824) and a test set 826 (or test / evaluation set 826). As previously discussed, the data partitioning logic of the data collection and preparation module 606 (e.g., in partitioning operation 822) may take into account the grid cell allocations to ensure that observed locations are represented in a balanced manner in the training data set (e.g., prepared data 820). The partitioning operation 822 may further seek to ensure that the training / validation set 824 and the test set 826 are not overly similar (e.g., sufficiently independent), as will be further discussed in connection with FIG. 10 .

[0055] In some instances, data augmentation techniques may be used to enhance the data used for training purposes, for example, photometric augmentation techniques (e.g., adjusting the brightness, hue, or contrast of images in a dataset to obtain additional examples that correspond to a given grid cell / pose message) may be used.

[0056] FIG. 9 illustrates a random sample of raw data corresponding to several trips collected by the mobile robot 104 in the context of a map divided into grid cells of configurable size. Percentage labels displayed with the subsets of cells indicate the percentage of data points corresponding to each particular cell. FIG. 9 illustrates that the raw data is not uniformly distributed, e.g., the four cells with the highest proportion of data points cumulatively account for 48% of the collected data in the random sample. Such uneven distribution of data by the mobile robot 104 may occur at locations 102. To ensure a balanced representation of observed locations in the training dataset (e.g., prepared data 820), pre-processing logic of the data collection and preparation module 606 (e.g., in partitioning operation 822) may assign grid cells as described above.

[0057] Temporal Data Binding

[0058] FIG. 10 illustrates some example block-based data partitioning processes 1002 as may be performed by the data collection and preparation module 606 in the partitioning operation 822 .

[0059] A challenge in developing a machine learning (ML) model is to ensure that there is no or reduced systematic dependency between the data used to train the model and the data used to evaluate the model. In the case of a sequence of images (or pose pairs to which images are matched) collected by a mobile robot (e.g., the mobile robot 104), there may be strong dependencies in the form of temporal coupling between images 808 collected at different time intervals (e.g., any given image may often look similar to an image observed one second earlier or later). It may be desirable to resolve such temporal coupling in order to avoid distorted evaluation results.

[0060] In the case of abundant data, an exemplary strategy for removing or reducing such temporal coupling between training / validation (or training / development) data and test / evaluation data can be time-based data partitioning (e.g., date-based time partitioning). Time-based data partitioning, such as date-based data partitioning, refers to using data collected during different missions or different capture periods for the training / validation and test / evaluation data sets. The missions are indicated by a mission identifier associated with the payload (e.g., images, pause messages, etc.). The capture period (e.g., seconds / minutes / hours / days / weeks / months of capture) can be calculated by the data collection and preparation module 606 based on the timestamps associated with the payload (e.g., images, pause messages, etc.). For example, the data collection and preparation module 606 can partition the captured data (e.g., images, matched image-pose data, etc.) such that data captured from a particular period is used in the training / validation set but not the test / evaluation set (or vice versa).

[0061] In addition to such absolute capture periods calculated based on the payload timestamp, the data collection and preparation module 606 can use relative capture periods (e.g., 30 minutes to one month) to implement time-based data segmentation. Such relative capture periods can capture the time difference between a first capture time of a selected first image (or a separately selected first capture time) and a second capture time of a subsequent image. For example, the data collection and preparation module 606 can ensure that images in the training / validation set are captured within a relative period of one hour relative to a predetermined capture time (e.g., the capture time of the earliest captured image included in the training set).

[0062] However, in some instances, time-based data splitting may be difficult to implement when, for example, 1,000 robots are operating in diverse environments, because a day's worth of data in one location may be much smaller or larger than in some other locations, and therefore time-based data splitting may not be optimal for some use cases.

[0063] Yet another exemplary strategy for preventing temporal coupling can be random shuffling (e.g., random reordering) of collected payload data, such as images (or images and matching pause messages), and assignment of such data to a training / validation set 824 and a test set 826. However, random shuffling in some circumstances and when used alone can still lead to an unacceptably large number of identical or similar images in the training / validation and test / evaluation data sets.

[0064] For example, in the case of sparse data, yet another exemplary strategy for dealing with temporal coupling is to split the image stream into blocks of contiguous images (e.g., matched with corresponding pause messages), where the blocks have a fixed and configurable size. Finally, dealing with temporal coupling can be accomplished by combining such exemplary strategies in series or in other manners.

[0065] 10 illustrates an example block-based data partitioning process 1002. First, the collected data stream is partitioned into equal-sized blocks 1004 of consecutively captured images (e.g., image-pose pairs). In some examples, the images may not be captured consecutively, but within a determinable time period or interval. The blocks 1004 are then randomly shuffled and assigned to a training / validation set 824 (or training / dev set 824) and a test set 826.

[0066] If the block size is large enough and two consecutive blocks are assigned to the training / validation set 824 and the test set 826, temporal coupling may be reduced. Larger blocks improve temporal decoupling, but may exacerbate imbalances in data distribution over locations (as reflected in the distribution across grid cells corresponding to the map 902). For example, if a robot is sent to one destination for the first 10 missions and to another destination for the next 10 missions, and the collected data is split into two equal-sized blocks (where the block size corresponds to the number of missions), the first block will have no data collected from the second destination, and vice versa. Thus, in some examples, the block size may be selected empirically, e.g., by varying the block size such that the data distribution is sufficiently balanced over the observed locations (e.g., as reflected in the distribution across grid cells of the grid corresponding to the example map 902) and selecting the largest block size.

[0067] Finally, in some examples, the data collection and preparation module 606 can prepare a separate data set for evaluation or testing purposes, where the separate data set is completely independent of the data used during training. In some examples, such a separate data set can include images and matched pose messages from a different (e.g., more recent) time and / or date. Such a data set may not include all locations that appear in the training data set, but can be more up-to-date, accurate, and independent, and therefore provide a more useful evaluation measure.

[0068] Data Quality Considerations

[0069] In general, supervised machine learning models are only as good as the data used to train them (the rule "garbage in, garbage out" applies). In some instances, the model system 602 can use uncurated data for model training, so data quality issues can be addressed in a variety of ways.

[0070] First, we can assume that unless the robot gets lost with some regularity, the robot's pose (or the pose estimate generated) will be reasonably accurate most of the time.

[0071] Second, if there are particularly difficult spots in the dining area where the pose estimates are likely to be persistently in error, such unreliable data points can be filtered out using the pose covariance values ​​reported by a probabilistic positioning method (e.g., AMCL).

[0072] Finally, because positioning methods such as AMCL provide pose estimates in "ground truth", the pose estimates will inevitably contain some noise. However, the presence of such noise is mitigated by the ML model training setup. Also, some noise in the training data can make model training more robust, as the model is less likely to remember location values ​​that correspond to specific images in the training set, reducing the chance of overfitting.

[0073] Data for many robot configurations

[0074] In some examples, such as a multiple robot setup, data may be collected from multiple robots operating at a single location. A particular robot may not cover all parts of a site (e.g., one robot may only be deployed to one part of a building while other robots are deployed to other parts most of the time). Thus, in some examples, the model system 602 may combine data from different robots and use this for training and / or evaluation.

[0075] Data collected by and received from multiple robots may, in some examples, be combined to generate a single training / validation set 824. A test / evaluation set 826 may be generated using aggregated data from multiple robots, or, in some examples, may be generated using separate data sourced from a single robot.

[0076] Maintaining the data of a single robot exclusively to generate the test / evaluation set 826 can provide several advantages, such as better understanding whether any hardware-related differences in the data negatively impact the model's performance. For example, this can be taken into account if a robot must be replaced in a cafeteria, but the model was trained on data collected before the replacement.

[0077] Model Training and Evaluation

[0078] FIG. 11 is a flowchart illustrating an image-based positioning model development process 1102 according to some examples that may be implemented by the model training and evaluation module 608. For simplicity, this drawing illustrates the flow of model training and evaluation operations for a model in one location, although multiple models 604 for multiple locations are generated and / or updated at various times (see the "model refresh" section below).

[0079] When the collected data is preprocessed by the data collection and preparation module 606 and split into a training / validation set 824 and a test / evaluation set 826, the model training and evaluation module 608 is arranged to train the model 604.

[0080] In a model training operation 1104, the image-based positioning model 604 is trained using the training / validation set 824. The image-based positioning model 604 can use a variety of model architectures and any combination thereof in a variety of training scenarios. For example, the model training and evaluation module 608 can use a transfer learning technique, where the training module starts with a backbone neural network (NN) pre-trained on a large public dataset (e.g., ImageNet), replaces the last K layers (where K is a predetermined constant, e.g., 1 or 2) of the backbone neural network with a set of customized layers, and trains the customized layers using the training / validation set 824 specific to the operation. The trained customized layers are then used to generate location predictions for the input images. In some examples, the backbone neural network can be a vision transformer or an EfficientNet. In some examples, the set of customized layers can include one or more fully connected layers with a softmax layer as an output layer, where the size of the softmax layer corresponds to the number of distinct grid cell IDs for the grid cells of the grid map for a particular location.

[0081] The trained model is then evaluated in a model evaluation operation 1106 using the test set 826. The evaluation results are considered in an evaluation related decision node 1108, which outputs an indication (e.g., a binary, real-valued confidence, or satisfaction score, etc.) of whether the model's performance (e.g., on the test set 826) is considered satisfactory with respect to one or more evaluation measures (e.g., accuracy, precision / recall, AUC, etc.).

[0082] If the model evaluation is satisfactory, the model 604 may be deployed as a production model on the mobile service mobile robot 104. If the model performance is unsatisfactory, in an update operation 1110, the model training and evaluation module 608 may augment or substitute the training / validation set 824 and retrain the model.

[0083] Model placement

[0084] FIG. 12 is a flow chart illustrating some example image-based positioning model placement processes 1202 that may be implemented by the model placement module 610.

[0085] As discussed in the following section (see model refresh), models 604 for different locations will be generated / updated at different times. Also, deployed robots (e.g., mobile robots 104) are not necessarily rebooted periodically, and in the case of multiple robots, not all robots are updated at once. Thus, in some instances, model deployment is done at the robot instance level (although model training can be done at the single location level). Specifically, the model system 602 can train / prepare new models 604 in the cloud (e.g., stored in cloud repository 804 using cloud services 110) regardless of robot operation. To achieve asynchronous model deployment, a particular robot can check for the presence of a new model when it is rebooted. FIG. 12 illustrates an example of such a process.

[0086] The image localizer 1204 is a robot operating system (ROS) node that provides image-based positioning responses to client requests using an image-based positioning model loaded into the memory of the mobile robot 104. The image localizer 1204 is responsible for loading the latest available image-based positioning model from the disk 1208 of the mobile robot 104 into the memory of the mobile robot (see below).

[0087] The model fetcher 1206 is an independent ROS node that checks (e.g., via a decision operation 1214 upon reboot 1212) whether the latest model 604 (e.g., model XYZ) for the current location (e.g., location XYZ) in the cloud cache 804 matches the current model stored on the robot disk 1208. If the latest model 604 for the current location does not match the model stored on disk, the model fetcher 1206 downloads the latest model for the current location from the cloud cache in a download operation 1210 and updates the model 604 stored on disk 1208. Once the model 604 is available on the local robot disk of the mobile robot 104 (as indicated by operation 1216), the image localizer 1204 can upload the available model to the memory of the mobile robot 104.

[0088] In some instances, the model fetcher 1206 can use other policies to trigger a check for whether a new model for the current location can be downloaded from the cloud repository 804. For example, the model fetcher 1206 can periodically (e.g., based on a set time schedule) check for new location-specific models. Alternatively, the model fetcher 1206 can use a conservative policy and check for the availability of new location-specific models only if no new models have been downloaded for a predetermined period of time. Finally, the model fetcher can use such policies in combination with other policies.

[0089] Model Refresh

[0090] FIG. 13 is a flow chart illustrating some example image-based positioning model refresh processes 1302 that may be implemented by the model refresh module 612.

[0091] The layout of a place 102 (e.g., a diner) can change and evolve over time. In some instances, the changes can be seasonal (e.g., holiday decorations or more outdoor seating in warmer months) and permanent (e.g., interior renovations). In either case, such changes will affect the performance of a model 604 that was trained on previous image data. Thus, the model system 602 provides the ability to automatically update the model 604 to prevent the model from becoming stale and degrading in performance.

[0092] For example, model refresh by the model refresh module 612 can be performed proactively based on a schedule (e.g., monthly / weekly) or reactively based on metrics (e.g., when the relocation convergence rate falls below a critical value). Proactive model refresh attempts to keep the models of all locations approximately the same up-to-date, but can be wasteful in some instances (e.g., if some locations change layout / decoration more frequently than others). Reactive model refresh may involve more engineering / design work (e.g., the correct metrics must be collected to accurately reflect model performance), but may be less resource-intensive (e.g., there is no need to update models for locations that have a certain degree of static layout).

[0093] The model refresh process 1302 illustrated in FIG. 13 is a reactive model refresh lifecycle according to some examples. The navigation stack 304 (e.g., part of the robotics stack 402 of the mobile robot 104) reports one or more online model measures 1304 of model performance for the image-based positioning model 604 used by one or more mobile robots 104. As previously described, the online model measures can include a relocation convergence rate (e.g., the rate at which a probabilistic positioning method such as AMCL converges after relocation of the mobile robot 104). The model measures 1304 are stored in a database (e.g., the cloud repository 804). When the performance of the image-based positioning model (e.g., as determined by the model measures 1304) degrades, the model 604 is automatically retrained (e.g., in operation 1306) to generate a new production model 604. The model retraining operation uses the most recently updated training and test data available. Evaluation metrics can include metrics (e.g., accuracy, precision, recall, etc.) used in initial model training and subsequent evaluation.

[0094] In addition to retraining Model 604 using online metrics (e.g., triggering model retraining based on monitoring metrics reported by the runtime stack 304), in some instances, Model 604 can be evaluated offline (in operation 1308) using the most recent collected data uploaded by the data uploader 1310 of the mobile robot 104. The collected data is prepared through the data preparation stream 802, thereby generating new training / validation and test / evaluation sets. The offline evaluation (operation 1308) of the performance of Model 604 can generate the values of the offline metrics using the newly generated test set that includes the most recently collected data. Such offline metrics can be used by the model refresh module 612 to trigger the retraining of Model 604. The offline metrics can include the metrics used in the initial model training / evaluation (e.g., accuracy / precision / recall for random test samples, etc.). As described above, since the model relearning operation can use the most recently collected data from the operational environment, the retrained model does not become obsolete and better adapts to the current environment configuration and state.

[0095] Model Version Management

[0096] FIG. 14 is a flowchart illustrating a model version management process 1402 according to some examples, as may be implemented by the model system 602.

[0097] During the process of model improvement operations, there may be a need to change the model architecture and / or the serving code. The model system 602 periodically refreshes the model (not necessarily involving any changes to the model serving code), and in so doing, can support multiple versions of the same model as well as multiple different model types. However, updating the models for multiple robots simultaneously poses a number of technical challenges related to feasibility and validity.

[0098] If a new type of model 604 is generated in the cloud (e.g., using the cloud service 110), the new type of model 604 may not be used by the mobile robot 104 until a new robotics software version (e.g., that embodies support for that model type) is deployed. Also, even if the mobile robot 104 is equipped with a new version of the robot software, it does not necessarily mean that the new model type must be available at that location.

[0099] Thus, a mechanism for updating model typologies and / or model versions is provided, including support for different other models deployed at different locations 102 and mobile robots 104. For a single robot, new model deployment may be managed manually. However, for a large number of robots (e.g., 1000's) with many different model typologies, automating this process offers technical and operational advantages.

[0100] Some examples handle asynchronous updates through pointers to different locations. For example, in some examples the following path structure can be used to save a model to the cloud:

[0101]

number

[0102] For example, the enumerated paths may indicate that for location 1, versions v1, v2, and v3 of model 1 are available (e.g., for location 1, version v1 of model 1 is stored in the v1 / subdirectory of the location1 / directory). The enumerated paths may also indicate, for example, that for location 1, there are at least two types of models, namely model 2 and model 1.

[0103] On the robot side, the updated code may include fallback logic that allows the robot to handle the presence of multiple model types and / or multiple model versions of multiple model types for a particular location. Examples of such fallback logic are as follows:

[0104]

number

[0105] For example, given a model for a particular location, the fallback logic checks whether a subdirectory corresponding to the most recent version (e.g., v3) exists in the directory corresponding to the particular location. If so, the v3 version of the corresponding model is used as the current model version. If no such path exists, the fallback logic checks whether a subdirectory corresponding to another model version (e.g., v2 or vl) exists to identify the corresponding model version (e.g., v2 or vl).

[0106] FIG. 14 illustrates an example of locations 1, 2, and 3, where different versions of location-related models are stored (e.g., in cloud repository 804) for the different locations (e.g., using some of the repository paths listed above). Location 1 has model versions 1, 2, and 3, location 2 has model versions 1 and 2, and location 3 has model versions 1 and 3. By examining the directory structures (e.g., directory file structures) and storage paths as described above, an example image localizer node for a mobile robot at location 1, 2, or 3 can retrieve (e.g., via a download operation 1210, not shown) the most recent version of a particular type of location-specific model. For example, an image localizer 1204 for a mobile robot 104 at location 1 can retrieve model version 3 for location 1. Meanwhile, an image localizer 1204 for a mobile robot at location 2 can retrieve model version 2 for location 2, and an image localizer 1204 for a mobile robot at location 3 can retrieve model version 3 for location 3.

[0107] 15 is a block diagram 1500 illustrating a software architecture 1504 that may be implemented in any one or more of the devices described herein, in some examples. The software architecture 1504 is supported by hardware, such as a machine 1502 that includes a processor 1520, memory 1526, and / or I / O components 1538. In this example, the software architecture 1504 may be conceptualized as a stack of layers, with each layer providing a particular function, in some examples. The software architecture 1504 includes layers such as an operating system 1512, libraries 1510, frameworks 1508, and applications 1506. Operationally, the applications 1506 invoke API calls 1550 through the software stack and receive messages 1552 in response to the API calls 1550.

[0108] The operating system 1512 manages hardware resources and provides common services. The operating system 1512 includes, for example, a kernel 1514, services 1516, and drivers 1522. The kernel 1514 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 1514 provides memory management, processor management (e.g., scheduling), component management, networking, and security configuration, among other functions. The services 1516 can provide other common services for the other software layers. The drivers 1522 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 1522 can include a display driver, a camera driver, a BLUETOOTH® or BLUETOOTH® Low Energy driver, a flash memory driver, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a WI-FI® driver, an audio driver, and a power management driver.

[0109] Libraries 1510 provide low-level common infrastructure used by applications 1506. Libraries 1510 can include system libraries 1518 (e.g., the C standard library) that provide functionality such as memory allocation functions, string manipulation functions, mathematical functions, etc. The libraries 1510 may also include API libraries 1524 such as a media library (e.g., a library for supporting the display and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (AVC) or H.264, Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG) or JPG, or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework used to render graphical content on a display in two dimensions (2D) and three dimensions (3D)), a database library (e.g., SQLite for providing various relational database functions), a web library (e.g., Web Kit for providing web browsing functions), etc. The libraries 1510 may also include various other libraries 1528 for providing various other APIs to the application 1506.

[0110] The framework 1508 provides a high-level common infrastructure used by the applications 1506. For example, the framework 1508 provides a variety of graphic user interface (GUI) features, high-level resource management, and high-level location services. The framework 1508 may, in some instances, provide a wide range of other APIs that may be used by the applications 1506, some of which may be specific to a particular operating system or platform.

[0111] In some examples, the applications 1506 can include a wide variety of other applications, such as a home application 1536, a contacts application 1530, a browser application 1532, an e-reader application 1534, a location application 1542, a media application 1544, a messaging application 1546, a games application 1548, and a third party application 1540. The applications 1406 are programs that perform programmatically defined functions. In some examples, a variety of programming languages ​​structured in a variety of ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C or assembly language), can be used to generate one or more of the applications 1506. In certain examples, the third party applications 1540 (e.g., applications developed using an ANDROID™ or IOS™ Software Development Kit (SDK) by an entity other than the particular platform vendor) can be mobile software that runs on a mobile operating system, such as IOS™, ANDROID™, WINDOWS® Phone, or other mobile operating systems. In this example, a third party application 1540 can invoke API calls 1550 provided by the operating system 1512 to implement the functionality described herein.

[0112] FIG. 16 is a schematic representation of a machine 1600 on which instructions 1610 (e.g., software, programs, applications, applets, apps, or other executable code) may be executed to cause the machine 1600 to perform any one or more of the methodologies discussed herein, according to some examples. For example, the instructions 1610 may cause the machine 1600 to perform any one or more of the methods described herein. The instructions 1610 transform the general unprogrammed machine 1600 into a specific machine 1600 programmed to perform the functions described and illustrated in the manner described. The machine 1600 may operate as a stand-alone device or may be coupled to other machines (e.g., coupled to a network). In a networked arrangement, the machine 1600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1600 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop, a netbook, a set-top box (STB), an entertainment media system, a cellular phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or differently executing instructions 1610 that specify operations to be performed by the machine 1600. Also, although a single machine 1600 is illustrated, the term "machine" may include a collection of machines that individually or collectively execute instructions 1610 to perform any one or more of the methodologies discussed herein.

[0113] Machine 1600 may include a processor 1604, memory 1606, and I / O components 1602, which may be configured to communicate over a bus 1640. In some examples, processor 1604 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), other processor, or any suitable combination thereof) may include, for example, processor 1608 and processor 1612 that execute instructions 1610. The term "processor" is intended to include multi-core processors that may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although FIG. 16 illustrates multiple processors 1604, machine 1600 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination of these.

[0114] The memory 1606 includes a main memory 1614, a static memory 1616, and a storage unit 1618, all accessible to the processor 1604 via a bus 1640. The main memory 1606, the static memory 1616, and the storage unit 1618 store instructions 1610 that embody any one or more of the methodologies or functions described herein. The instructions 1610 may also reside, wholly or partially, within the main memory 1614, within the static memory 1616, within the storage unit 1618, within the machine-readable medium 1620, within the processor 1604 (e.g., within a processor's cache memory), or any suitable combination thereof during execution by the machine 1600.

[0115] The I / O components 1602 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, or capturing measurements. The specific I / O components 1602 included in a particular machine will vary depending on the type of machine. For example, a portable machine such as a cell phone may include a touch input device or other such input mechanism, while a headless server machine likely will not include such a touch input device. The I / O components 1602 may include many other components not shown in FIG. 16. In various examples, the I / O components 1602 may include an output component 1626 and an input component 1628. The output component 1626 may include a visual component (e.g., a display such as a Plasma Display Panel (PDP), a Light-Emitting Diode (LED) display, a Liquid Crystal Display (LCD), a projector, or a Cathode Ray Tube (CRT)), an audio component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), or other signal generator. The input components 1628 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, touchpad, trackball, joystick, motion sensor, or other pointing instrument), tactile input components (e.g., physical buttons, a touch screen that provides the position and / or force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), etc.

[0116] In further examples, the I / O components 1602 can include a biometric component 1630, a motion component 1632, an environmental component 1634, or a position component 1636, among various other components. For example, the biometric components 1630 include components that detect facial expressions (e.g., hand expressions, facial expressions, vocal expressions, body movements, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), or identify people (e.g., voice identification, retina identification, face identification, fingerprint identification, or brainwave-based identification). The motion components 1632 include acceleration sensor components (e.g., accelerometers), gravity sensor components, and rotation sensor components (e.g., gyroscopes). The environmental components 1634 may include, for example, one or more cameras, light sensor components (e.g., a photometer), temperature sensor components (e.g., one or more thermometers to detect ambient temperature), humidity sensor components, pressure sensor components (e.g., a barometer), acoustic sensor components (e.g., one or more microphones to detect background noise), proximity sensor components (e.g., an infrared sensor to detect surrounding objects), gas sensors (e.g., a gas detection sensor to detect concentrations of harmful gases or measure airborne pollutants for safety purposes), or other components capable of providing an indication, measurement, or signal corresponding to the surrounding physical environment. The position components 1636 may include a location sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer to detect air pressure from which altitude may be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0117] Communications may be embodied using a variety of technologies. The I / O component 1602 further includes a communication component 1638 operable to couple the machine 1600 to the network 1622 or the device 1624 through a respective coupling or connection. For example, the communication component 1638 may include a network interface component or other suitable device for interfacing with the network 1622. In additional examples, the communication component 1638 may include a wired communication component, a wireless communication component, a cellular communication component, a Near Field Communication (NFC) component, a BLUETOOTH component (e.g., BLUETOOTH Low Energy), a flash memory driver, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a WI-FI component, and other communication components for providing communication through other modalities. The device 1624 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled through USB).

[0118] The communication component 1638 may also include components that detect or are operable to detect an identifier. For example, the communication component 1638 may include a Radio Frequency Identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor to detect one-dimensional barcodes such as Universal Product Code (UPC) barcodes, Quick Response (QR) codes, Aztec codes, Data Matrix, Data glyphs, Maxi Codes, PDF417, Ultra Codes, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone to identify tagged audio signals). A variety of information may also be derived through the communication component 1638, such as location through Internet Protocol (IP) geolocation, location through WI-FI signal triangulation, or location through detection of NFC beacon signals that can point to a specific location.

[0119] Various memories (e.g., main memory 1614, static memory 1616, and / or memory of processor 1604) and / or storage unit 1618 can store one or more sets of instructions and data structures (e.g., software) implemented or used in any one or more of the methodologies or functions described herein. When executed by processor 1604, such instructions (e.g., instructions 1610) cause various operations to be performed to implement the disclosed embodiments.

[0120] The command 1610 may be transmitted or received over the network 1622 using a transmission medium through a network interface device (e.g., a network interface component included in the communications component 1638) and using any one of a variety of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, the command 1610 may be transmitted or received over a transmission medium through a connection (e.g., a peer-to-peer connection) to the device 1624.

[0121] 17 is a block diagram illustrating some example machine learning programs 1700. The machine learning programs 1700, also referred to as machine learning algorithms or tools, may be used as part of the systems described herein to perform search and query response related tasks.

[0122] Machine learning is a field of study that gives computers the ability to learn without being explicitly programmed. Machine learning explores the study and construction of algorithms, also referred to herein as tools, that can learn from or be trained using existing data and make predictions about or based on new data. Such machine learning tools operate by building models from example training data 1708 to make data-based predictions or decisions, which are expressed as outputs or scores (e.g., scores 1716). Although examples have been presented with respect to several machine learning tools, the principles presented herein may be applied to other machine learning tools.

[0123] In some examples, different machine learning tools may be used, such as Logistic Regression (LR), Naive-Bayes, Random Forest (RF), Neural Network (NN), Matrix Decomposition and Support Vector Machine (SVM) tools.

[0124] Two common types of problems in machine learning are classification problems and regression problems. Classification problems, also called categorization problems, aim to classify an item into one of many category values ​​(e.g., is this object an apple or an orange?). Regression algorithms aim to quantify some item (e.g., by providing a real-valued value).

[0125] The machine learning program 1700 supports two types of phases: a training phase 1702 and a prediction phase 1704. In the training phase 1702, supervised learning, unsupervised learning, or reinforcement learning can be used. For example, the machine learning program 1700 (1) receives / receives features 1706 (e.g., as structured or labeled data for supervised learning) and / or (2) identifies features 1706 in training data 1708 (e.g., unstructured or unlabeled data for unsupervised learning). In the prediction phase 1704, the machine learning program 1700 uses the features 1706 to analyze query data 1712 to generate results or predictions as examples of ratings 1716.

[0126] In the training phase 1702, feature engineering is used to identify features 1706, which can include identifying useful, discriminatory, and independent features for the effective operation of the machine learning program 1700 in pattern recognition, classification, and regression. In some examples, the training data 1708 includes pre-identified features 1706 and labeled data, which is known data for one or more outcomes. Each of the features 1706 can be a variable or attribute, such as an individual measurable characteristic of a process, article, system, or phenomenon represented in the dataset (e.g., the training data 1708). The features 1706 can also be of different types, such as numeric features, strings, and graphs, and can include one or more of content 1718, concepts 1720, attributes 1722, historical data 1724, and / or user data 1726, for example.

[0127] During the training phase 1702 , the machine learning program 1700 uses training data 1708 to look for correlations between features 1706 that affect a predicted outcome or score 1716 .

[0128] Using the training data 1708 and the identified features 1706, the machine learning program 1700 is trained during the training phase 1702, at machine learning program training 1710. The machine learning program 1700 evaluates the value of the features 1706 by correlation with the training data 1708. The result of the training is a trained machine learning program 1714 (e.g., a trained or learned model).

[0129] Alternatively, the training stage 1702 may involve machine learning where the training data 1708 is structured (e.g., labeled during a preprocessing operation) and the trained machine learning program 1714 implements a relatively simple neural network 1728 that can perform, for example, classification and clustering tasks. In another example, the training stage 1702 may involve deep learning where the training data 1708 is unstructured and the trained machine learning program 1714 implements a deep neural network 1728 that can perform all of the feature extraction and classification / clustering tasks.

[0130] The neural network 1728 generated during the training phase 1702 and embodied in the trained machine learning program 1714 may include a hierarchical (e.g., layered) organization of neurons. For example, neurons (or nodes) may be arranged hierarchically in multiple layers, including an input layer, an output layer, and multiple hidden layers. A layer in the neural network 1728 may have one or multiple neurons, which compute small functions (e.g., activation functions) operationally. For example, if an activation function produces a result that exceeds a certain critical value, an output may be transmitted from that neuron (e.g., a sending neuron) to a connected neuron (e.g., a receiving neuron) in a successive layer. The connections between neurons also have associated weights that define the influence of the input from the sending neuron to the receiving neuron.

[0131] In some examples, the Neural Network 1728 may also be one of a number of different types of neural networks, including, for example, a single layer feed-forward network, an Artificial Neural Network (ANN), a Recurrent Neural Network (RNN), a Symmetrically Connected Neural Network with unsupervised pre-training, a Convolutional Neural Network (CNN), or a Recursive Neural Network (RNN).

[0132] During the prediction stage 1704, a trained machine learning program 1714 is used to perform the evaluation. Query data 1712 is provided as input to the trained machine learning program 1714, which generates an evaluation 1716 as output in response to receiving the query data 1712.

[0133] Turning now to FIG. 18, there is illustrated a diagrammatic representation of a processing environment (1800) including a processor 1802, a processor 1806, and a processor 1808 (eg, a GPU, a CPU, or a combination thereof).

[0134] The processor 1802 is coupled to a power source 1804 and is illustrated as including the following modules (either permanently configured or temporarily instantiated): a data collection and preparation module 606, a model training and evaluation module 608, and a model deployment module 610.

[0135] Example

[0136] 1. A method for generating an image-based localization model for navigation of a mobile robot, comprising: performing data collection at a plurality of different service locations where a mobile robot swarm can be deployed to generate collected data; dividing the collected data into a plurality of blocks of continuous portions of the collected data; using the collected data to generate a first image-based localization model for a first service location among the plurality of different service locations; using the collected data to generate a second image-based localization model for a second service location among the plurality of different service locations; distributing the first image-based localization model on a first mobile robot of the mobile robot swarm - the first mobile robot is disposed at the first service location among the plurality of different service locations, and the first mobile robot navigates the first service location using the first image-based localization model; and distributing the second image-based localization model on a second mobile robot of the mobile robot swarm - the second mobile robot is disposed at a second service location among the plurality of different service locations, and the second mobile robot navigates the second service location using the second image-based localization model.

[0137] 2. The method of any one or more of the preceding examples, wherein the collected data includes image data and the dividing step includes dividing the image data into equal sized blocks of contiguous images.

[0138] 3. In one or more of the above examples of the method, the step of generating a first image-based positioning model includes the step of training, developing and testing the first image-based positioning model using different blocks of the same size in consecutive images.

[0139] 4. In one or more of the above exemplary methods, the method includes a step of shuffling different blocks of the same size in successive images before allocating different blocks of the same size in successive images to training, development, and testing of the image-based positioning model, respectively.

[0140] 5. Any one or more of the methods described above, including randomly allocating different blocks of equal size in successive images to training, development, and testing of the first image-based localization model, respectively.

[0141] 6. In one or more of the above examples, the method includes automatically determining the size by balancing the size of each of the same-sized blocks of consecutive images based on a balanced distribution across grid cells of a map grid of the first service location.

[0142] 7. In one or more of the above examples, the image data includes image timestamps, each timestamp representing a capture period for a corresponding image, and the image timestamps for images in each same-sized block of consecutive images represent the same capture period for all images within the corresponding block, and the method further includes a step of allocating different blocks of the same-sized blocks of consecutive images to training, development and testing of the first image-based positioning model, respectively, based on the capture periods for the images in the same-sized blocks.

[0143] 8. In one or more of the above example methods, the step of allocating different blocks of the same size of consecutive images to each of the training, development and testing of the first image-based positioning model includes the step of allocating one or more blocks of consecutive images for the first capture period to only one of the training, development and testing of the first image-based positioning model.

[0144] 9. In one or more of the above example methods, the image data includes a mission identifier, each image in the image data is associated with a mission identifier, and each image in the same-sized block of consecutive images is associated with a unique mission identifier among a plurality of mission identifiers, and the method further includes a step of allocating different blocks in the same-sized blocks of consecutive images to each of the training, development and testing of the first image-based positioning model based on the mission identifier for the images in the same-sized blocks.

[0145] 10. In one or more of the above example methods, the step of allocating different blocks of the same size in consecutive images to each of the training, development and testing of the first image-based positioning model includes the step of allocating one or more blocks of consecutive images for the first mission identifier to only one of the training, development and testing of the first image-based positioning model.

[0146] 11. In one or more of the above exemplary methods, generating a first image-based positioning model for a first service location of the plurality of different service locations includes: deriving first location data specific to the first service location from the collected data; generating a plurality of online model performance measures based on the first location data associated with a current version of the first image-based positioning model; performing an offline evaluation on the current version of the first image-based positioning model using at least a portion of the first location data; and automatically generating a new version of the first image-based positioning model for the first service location based on the offline evaluation. Other technical features may be apparent to one skilled in the art from the following drawings, description, and claims.

[0147] 12. The method of any one or more of the preceding examples, wherein the plurality of online model performance measures are reported by a navigation stack of the first mobile robot.

[0148] 13. The method of any one or more of the preceding examples, further comprising retraining the image-based positioning model using a plurality of online model performance measures.

[0149] 14. In one or more of the above examples, the method further includes the steps of: performing a restart operation by the first mobile robot; in response to the restart operation, automatically checking a remote repository by the first mobile robot to determine whether a new image-based positioning model has been generated and stored in the remote repository; in response to determining whether a new image-based positioning model has been generated and stored in the remote repository, storing the new image-based positioning model in a local memory by the first mobile robot; and providing an image-based positioning response to the positioning request by the first mobile robot.

[0150] 15. In one or more of the above examples of the method, the step of automatically checking the remote repository to determine whether a new image-based positioning model has been generated and stored in the remote repository includes checking whether a retrained version of the current image-based positioning model has been generated.

[0151] 16. In one or more of the above examples of the method, the step of automatically checking the remote repository to determine whether a new image-based positioning model has been generated and stored in the remote repository includes a step of checking whether a new image-based positioning model type has been generated.

[0152] 17. The method, in any one or more of the above examples, further comprising: maintaining, in a cloud repository, a plurality of image-based positioning model types and a plurality of versions of each of the plurality of image-based positioning model types; and implementing, in a first mobile robot, fallback logic that enables the first mobile robot to use the plurality of image-based positioning model types and the plurality of versions of each of the plurality of image-based positioning model types.

[0153] 18. In one or more of the above examples, the maintaining step includes maintaining a file structure for storing a plurality of image-based positioning model types and a plurality of versions of each of the plurality of image-based positioning model types in a cloud repository.

[0154] 19. In one or more of the above examples, the fallback logic is included in a robotics stack of the first mobile robot and accesses a file structure to access at least one of a plurality of image-based positioning model types or a plurality of versions of each of the plurality of image-based positioning model types in a cloud repository.

[0155] 20. A computing device comprising at least one processor and a memory, the memory storing instructions that, when executed by the at least one processor, configure the device to perform any one of the above example methods.

[0156] 21. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform any one of the methods described above.

[0157] Glossary

[0158] "Carrier signal" refers to any intangible medium capable of storing, encoding, or transmitting instructions for execution by a machine, including digital or analog communication signals or other intangible media for facilitating communication of such instructions. The instructions may be transmitted or received over a network using a transmission medium through a network interface device.

[0159] A "communications network" refers to one or more portions of a network, which may be an ad hoc network, an intranet, an extranet, a Virtual Private Network (VPN), a Local Area Network (LAN), a Wireless LAN (WLAN), a Wide Area Network (WAN), a Wireless WAN (WWAN), a Metropolitan Area Network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a Plain Old Telephone Service (POTS) network, a cellular network, a wireless network, a Wi-Fi network, other types of networks, or a combination of two or more such networks. For example, the network or portions of a network may include a wireless or cellular network, and the connection may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communication (GSM) connection, or other types of cellular or wireless connections.In this example, the combination may embody any of various types of data transmission technologies, such as Single Carrier Radio Transmission Technology (1xRTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation (4G) wireless networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, other technologies defined by various standards setting organizations, other long-distance protocols, or other data transmission technologies.

[0160] A "component" refers to a device, physical entity, or logic having boundaries defined by functions or subroutine calls, branch points, APIs, or other techniques that provide division or modularization of a particular processing or control function. A component can be coupled to other components through interfaces to perform a machine process. A component can be a packaged functional hardware unit designed for use with other components, and can generally be a part of a program that performs a particular function of related functionality. A component can be a software component (e.g., code embodied in a machine-readable medium) or a hardware component. A "hardware component" is a tangible unit that can perform a particular task, and can be configured or arranged in a particular physical manner. For example, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors) can be configured as a hardware component that operates with software (e.g., an application or application portion) to perform a particular task described herein. A hardware component can also be embodied mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include dedicated circuitry or logic that is permanently configured to perform a particular task. A hardware component may be a special purpose processor such as a Field-Programmable Gate Array (FPGA) or an Application-Specific Integrated Circuit (ASIC). A hardware component may also include programmable logic or circuitry that is temporarily configured by software to perform a particular task. For example, a hardware component may include software running on a general purpose processor or other programmable processor.When configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) customized to perform the function for which it is configured, and is no longer a general-purpose processor. Whether a hardware component is embodied mechanically, with permanently configured dedicated circuitry, or with temporarily configured circuitry (e.g., configured by software) may be determined by cost and time considerations. Thus, the phrase "hardware component" (or "hardware embodied component") should be understood to include a tangible entity that is physically configured, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or perform a particular task described herein. When considering an example in which a hardware component is temporarily configured (e.g., programmed), the hardware component need not be configured or instantiated at any one time. For example, if the hardware component includes a general-purpose processor configured by software to be a special-purpose processor, the general-purpose processor may be configured (e.g., including different hardware components) as different special-purpose processors at different times. Accordingly, the software may configure a particular processor or processors to, for example, be particular hardware components in one instance and other hardware components in other instances. Hardware components may provide information to and receive information from other hardware components. Thus, the described hardware components may be considered to be communicatively coupled. In cases where multiple hardware components are simultaneously present, communication may be through signal transmission between two or more of the hardware components (e.g., through appropriate circuits and buses). In instances where multiple hardware components are configured or instantiated at different times, communication between such hardware components may be through, for example, storing and retrieving information in memory structures to which the multiple hardware components have access.For example, a hardware component may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. Additional hardware components may subsequently access the memory device to retrieve and process the stored output. A hardware component may initiate communication with an input or output device and may operate on a resource (e.g., collect information). Various operations of the exemplary methods described herein may be performed, at least in part, by one or more processors that are temporarily or permanently configured (e.g., by software) to perform the associated operations. Such processors, whether temporary or permanent, may constitute processor implementations that operate to perform one or more operations or functions described herein. As used herein, a "processor implementation" refers to a hardware component that is implemented using one or more processors. Similarly, the methods described herein may be implemented, at least in part, by a processor, and a particular processor or processors may be an example of hardware. For example, at least a portion of the operations of the methods described herein may be performed by one or more processors or processor implementations. One or more processors may also operate to facilitate performance of related operations in a "cloud computing" environment or "Software as a Service (SaaS)." For example, at least a portion of the operations may be performed by a group of computers (examples of machines that include the processors), and such operations may be accessed through a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs). Performance of particular operations may be distributed among the processors, and may reside within a single machine as well as located across multiple machines. In some examples, the processor or an embodiment of the processor may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In some examples, the processor or an embodiment of the processor may be distributed across multiple geographic locations.

[0161] "Computer-readable medium" refers to all machine storage media and transmission media. Thus, the term includes all storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.

[0162] "Machine storage medium" refers to a single or multiple storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions, routines, and / or data. This term includes solid state memory, including memory internal or external to a processor, and optical and magnetic media. Specific examples of machine storage mediums, computer storage media, and / or device storage media include, for example, semiconductor memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), EEPROMs (Electrically Erasable Programmable Read-Only Memory), FPGAs, and flash memory devices), magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and non-volatile memory including CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", and "computer storage medium" have the same meaning and may be used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" specifically exclude carrier waves, modulated data signals and other such media, some of which are encompassed by the term "signal media."

[0163] A "module" refers to a function or logic having boundaries defined by subroutine calls, branch points, application program interfaces (APIs), or other techniques that provide division or modularization of a particular processing or control function. A module is typically coupled to other modules through interfaces to perform a machine process. A module may be a packaged functional hardware unit designed for use with other components, and may be a part of a program that performs a particular function, typically related functionality. A module may be a software module (e.g., code embodied in a machine-readable medium) or a hardware module. A "hardware module" is a tangible unit that can perform a particular task, and may be configured or arranged in a particular physical manner. In various examples, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules (e.g., a processor or group of processors) of a computer system may be configured as a hardware module that operates with software (e.g., an application or application portion) to perform particular tasks described herein. In some examples, a hardware module may be embodied mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform a particular task. For example, a hardware module may be a special purpose processor such as an FPGA or ASIC. A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform a particular task. For example, a hardware module may include software executed by a general purpose processor or other programmable processor. When configured by such software, a hardware module becomes a particular machine (or a particular component of a machine) that is uniquely customized to perform a configured function, and is no longer a general purpose processor.It will be appreciated that whether a hardware module is embodied mechanically, with dedicated permanently configured circuitry, or with temporarily configured circuitry (e.g., configured with software) may be determined based on cost and time considerations. Thus, the term "hardware module" (or "hardware embodied module") should be understood to include a tangible entity that is physically configured, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or perform a particular task described herein. When considering an example in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one time. For example, if a hardware module includes a general-purpose processor that is configured by software to be a special-purpose processor, the general-purpose processor may be configured (e.g., configured with different hardware modules) at different times as different special-purpose processors. Accordingly, the software may configure a particular processor or processors to be, for example, a particular hardware module in one instance and other hardware modules in other instances. A hardware module may provide information to and receive information from other hardware modules. Accordingly, the described hardware modules may be considered to be communicatively coupled. Where multiple hardware modules are simultaneously present, communication may occur through signal transmission (e.g., through appropriate circuits and buses) between two or more of the hardware modules. In instances where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules may occur, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled.Thereafter, additional hardware modules may subsequently access the memory device to retrieve and process the stored output. The hardware modules may initiate communication with input or output devices and may operate on resources (e.g., collecting information). Various operations of the exemplary methods and routines described herein may be performed, at least in part, by one or more processors that are temporarily or permanently configured (e.g., by software) to perform the associated operations. Such processors, whether temporarily or permanently configured, may constitute processor implementation modules that operate to perform one or more operations or functions described herein. As used herein, a "processor implementation module" refers to a hardware module that is implemented using one or more processors. Similarly, the methods described herein may be at least partially implemented by a processor, with a particular processor or processor being an example of hardware. For example, at least a portion of the operations of the method may be performed by one or more processors or processor implementation modules. Additionally, one or more processors may operate to facilitate performance of the associated operations in a "cloud computing" environment or as a "software as a service (SaaS)." For example, at least a portion of the operations may be performed by a group of computers (examples of which include machines that include processors), and such operations may be accessed through a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs). Performance of certain operations may be distributed among processors and may reside within a single machine as well as located across multiple machines. In some examples, the processor or processor implementation may be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other examples, the processor or processor implementation may be distributed across multiple geographic locations.

[0164] A "processor" refers to any circuit or virtual circuit (i.e., a physical circuit that is emulated by logic executed in an actual processor) that manipulates data values ​​in response to control signals (e.g., "instructions," "op codes," "machine code," etc.) and generates corresponding output signals that are applied to a machine operation. A processor may be, for example, a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Radio-Frequency Integrated Circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.

[0165] "Signal medium" refers to any intangible medium that can store, encode, or transmit machine-executable instructions, including digital or analog communication signals or other intangible media that facilitate communication of software or data. The term "signal medium" can include any form of modulated data signal, carrier wave, and the like. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a way as to encode information in the signal. The terms "transmission medium" and "signal medium" have the same meaning and may be used interchangeably in this disclosure.

[0166] Various drawings in this application include block diagrams, flow charts, and control flow charts of methods, systems, and program products according to the present invention. It will be understood that each block or step of the block diagrams, flow charts, and control flow charts, and combinations of blocks in the block diagrams, flow charts, and control flow charts, may be embodied by computer program instructions. Such computer program instructions may be loaded into a computer or other programmable device to produce a machine, and the instructions executed by the computer or other programmable device produce means for implementing the functionality specified in the block diagrams, flow charts, or control flow block(s) or step(s). Such computer program instructions may be stored in a computer readable memory that can direct the computer or other programmable device to function in a particular manner, and the instructions stored in the computer readable memory produce an article of manufacture including instruction means for implementing the functionality specified in the block diagrams, flow charts, or control flow block(s) or step(s). The computer program instructions may also be loaded into a computer or other programmable device to cause the computer or other programmable device to perform a series of work steps, thereby creating a computer-implemented process in which the instructions executed on the computer or other programmable device provide steps for implementing the function specified in the block diagram, flowchart, or control flow block(s) or step(s).

[0167] Therefore, the blocks or steps of the block diagrams, flowcharts, or control flow charts support combinations of means for performing the specified functions, combinations of steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block or step of the block diagrams, flowcharts, or control flow charts, and combinations of blocks or steps in the block diagrams, flowcharts, or control flow charts, can be embodied by a special purpose hardware-based computer system that performs the specified functions or steps, or a combination of special purpose hardware and computer instructions.

[0168] The foregoing description has been presented with reference to various embodiments. Those of ordinary skill in the industry and technical fields to which this application pertains will recognize that changes and variations in the structures and methods of operation described may be made without meaningfully departing from the principles, spirit and scope of the present disclosure. Changes and modifications may be made to the disclosed examples without departing from the scope of the present disclosure. Such and other changes and modifications are intended to be within the scope of the present disclosure, as expressed in the following claims.

Claims

1. A method for generating an image-based localization model for navigation of a mobile robot, comprising: performing data collection at a plurality of different service locations where the mobile robot swarm may be deployed to generate collected data; dividing the collected data into a plurality of blocks of contiguous portions of the collected data; generating a first image-based positioning model for a first service location of the plurality of different service locations using the collected data; generating a second image-based positioning model for a second service location of the plurality of different service locations using the collected data; deploying the first image-based positioning model on a first mobile robot of the mobile robot swarm, the first mobile robot being deployed at a first service location of the plurality of different service locations, and the first mobile robot navigating the first service location using the first image-based positioning model; and The method includes a step of deploying the second image-based positioning model to a second mobile robot of the mobile robot swarm, the second mobile robot being deployed to the second service location of the plurality of different service locations, and the second mobile robot navigating the second service location using the second image-based positioning model.

2. 2. The method of claim 1, wherein said collected data comprises image data, and said dividing step comprises dividing said image data into equal sized blocks of contiguous images.

3. 3. The method of claim 2, wherein generating the first image-based positioning model comprises performing training, development and testing of the first image-based positioning model using different blocks of the same size of the consecutive images.

4. 4. The method of claim 3, further comprising shuffling different blocks of the same size in the successive images before allocating the different blocks of the same size in the successive images to training, development and testing of the image-based localization model, respectively.

5. 5. The method of claim 4, further comprising randomly allocating different ones of the equal-sized blocks of the consecutive images to training, development and testing of the first image-based localization model, respectively.

6. 4. The method of claim 3, further comprising automatically determining the size by balancing the size of each of the same-sized blocks of the contiguous images based on a balanced distribution across grid cells of a map grid of the first service locations.

7. the image data includes image timestamps, each timestamp representing a capture period for a corresponding image; image timestamps for images in each equal-sized block of said consecutive images represent the same capture period for all images within the corresponding block; 4. The method of claim 3, further comprising allocating different blocks of the same-sized blocks of the consecutive images to training, development and testing of the first image-based positioning model, respectively, based on a capture period for the images in the same-sized blocks.

8. The method of claim 7, wherein the step of allocating different blocks of the same size of the consecutive images to each of the training, development and testing of the first image-based positioning model includes the step of allocating one or more blocks of consecutive images for a first capture period to only one of the training, development and testing of the first image-based positioning model.

9. the image data includes a mission identifier, each image in the image data being associated with a mission identifier; the image in each equal sized block of consecutive images is associated with a unique mission identifier among a plurality of mission identifiers; 4. The method of claim 3, further comprising allocating different blocks of the same-sized blocks of the consecutive images to training, development, and testing of the first image-based positioning model, respectively, based on a mission identifier for the images in the same-sized blocks.

10. 9. The method of claim 8, wherein the step of allocating different blocks of the same size of the consecutive images to each of the training, development, and testing of the first image-based positioning model includes the step of allocating one or more blocks of the consecutive images for a first mission identifier to only one of the training, development, and testing of the first image-based positioning model.

11. generating the first image-based positioning model for the first service location among the plurality of different service locations, deriving first location data specific to said first service location from said collected data; generating a plurality of online model performance measures based on first location data associated with a current version of the first image-based positioning model; performing an offline evaluation of the current version of the first image-based positioning model using at least a portion of the first location data; and The method of claim 1 , further comprising automatically generating a new version of the first image-based positioning model for the first service location based on the offline evaluation.

12. The method of claim 11 , wherein the plurality of online model performance measures are reported by a navigation stack of the first mobile robot.

13. The method of claim 11 , comprising retraining the image-based positioning model using the plurality of online model performance measures.

14. performing a restart operation by the first mobile robot; in response to the reboot operation, automatically checking, at the first mobile robot, the remote repository to determine whether a new image-based positioning model has been generated and stored in the remote repository; storing the new image-based positioning model in a local memory at the first mobile robot in response to determining whether the new image-based positioning model has been generated and stored at the remote storage location; and The method of claim 1 , further comprising providing, by the first mobile robot, an image-based positioning response to a positioning request.

15. The method of claim 14 , wherein the step of automatically checking the remote repository to determine whether a new image-based positioning model has been generated and stored in the remote repository includes the step of checking whether a retrained version of a current image-based positioning model has been generated.

16. The method of claim 14 , wherein the step of automatically checking the remote repository to determine whether a new image-based positioning model has been created and stored in the remote repository includes the step of checking whether a new image-based positioning model type has been created.

17. maintaining a plurality of image-based positioning model types and a plurality of versions of each of the plurality of image-based positioning model types in a cloud repository; and The method of claim 1 , further comprising implementing, in the first mobile robot, fallback logic that enables the first mobile robot to use the plurality of image-based positioning model types and a plurality of versions of each of the plurality of image-based positioning model types.

18. The method of claim 17 , wherein the maintaining step comprises maintaining a file structure for storing the plurality of image-based positioning model types and a plurality of versions of each of the plurality of image-based positioning model types in the cloud repository.

19. 20. The method of claim 18, wherein the fallback logic is included in a robotics stack of the first mobile robot and accesses the file structure to access at least one of the plurality of image-based positioning model types or a plurality of versions of each of the plurality of image-based positioning model types in the cloud repository.

20. 1. A computing device comprising: at least one processor; and Including memory, The memory, when executed by the at least one processor, causes the apparatus to: performing data collection at a plurality of different service locations where the mobile robot swarm may be deployed to generate collected data; Dividing the collected data into a plurality of blocks of contiguous portions of the collected data; generating a first image-based positioning model for a first service location of the plurality of distinct service locations using the collected data; generating a second image-based positioning model for a second service location of the plurality of distinct service locations using the collected data; deploying the first image-based positioning model on a first mobile robot of the mobile robot swarm, the first mobile robot being deployed at the first service location of the plurality of different service locations, the first mobile robot navigating the first service location using the first image-based positioning model; A computing device storing instructions configured to deploy the second image-based positioning model to a second mobile robot of the mobile robot swarm, the second mobile robot being deployed to a second service location of the plurality of distinct service locations, and the second mobile robot navigating the second service location using the second image-based positioning model.

Images