Geolocation model for perception, prediction or planning

Abstract

The present disclosure relates to geolocation models for perception, prediction, or planning. In one embodiment, a method includes determining, by a computing system associated with a vehicle, a current location of the vehicle in a first region, identifying one or more first set of model parameters associated with the first region and one or more second set of model parameters associated with a second region, generating one or more first inferences based on first sensor data captured by the vehicle using one or more machine learning models having based on the first set of model parameters, switching a configuration of the model from the first set of model parameters to the second set of model parameters, generating one or more second inferences based on second sensor data generated by sensors of the vehicle in the second region using the model having based on the configuration of the second set of model parameters; and causing the vehicle to perform one or more operations based on the second inferences. The method provides for improved performance of the vehicle.

Description

Technical Field

[0001] This disclosure relates to improvements to vehicles, particularly geolocation models for perception, prediction, or planning. Background Technology

[0002] Modern vehicles may include one or more sensors or sensing systems for monitoring the vehicle and its environment, and may use data collected from these sensors to make inferences related to current and future environmental states. These inferences can be used for tasks related to object detection and classification in the environment, as well as route planning for the vehicle. Inferences can be made using statistical models that can be configured with appropriate parameters to generate inferences based on representations of the environment. The parameters of the statistical models can be configured based on training data that specifies the correspondence between particular inputs (e.g., environmental representations) and outputs (e.g., inferences such as object classification or route). These models can then be used to generate inferences based on sensor data collected as the vehicle travels through new, previously unseen environments. These models can identify outputs corresponding to inputs similar to those represented by the sensor data, and the identified outputs can be inferences generated from the sensor data.

[0003] For example, a vehicle can use one or more cameras or LiDAR to detect objects in its surrounding environment. The vehicle can use one or more computing systems (e.g., an onboard computer) to collect and process data from the sensors. The computing system can use statistical models (stored in onboard storage or downloaded from the cloud via a wireless connection) to generate inferences based on this data and use these inferences to operate the vehicle. Summary of the Invention

[0004] At least one exemplary aspect of this disclosure relates to a method comprising performing the following operations via a computing system associated with a vehicle: determining the current position of the vehicle in a first region; identifying one or more first sets of model parameters associated with the first region and one or more second sets of model parameters associated with a second region; generating one or more corresponding first inferences based on first sensor data captured by the vehicle using one or more machine learning models having configurations based on the corresponding first sets of model parameters; switching the configuration of one or more models from the corresponding first sets of model parameters to the corresponding second sets of model parameters associated with the second region; generating one or more corresponding second inferences based on second sensor data generated by the vehicle's sensors when the vehicle is in the second region using one or more models having configurations based on the corresponding second sets of model parameters; and causing the vehicle to perform one or more operations based at least on the second inferences. Attached Figure Description

[0005] Figure 1AThe illustration depicts an example vehicle environment, including a busy intersection with vehicles and numerous pedestrians.

[0006] Figure 1B The illustration shows a sample map divided into regions.

[0007] Figure 1C The illustration shows an example map, which is divided into partially overlapping areas associated with the model.

[0008] Figure 1D The illustration shows an example map associated with a single model and divided into regions with associated performance evaluations.

[0009] Figure 1E The illustration shows an example map, which is divided into areas associated with the model and performance evaluation.

[0010] Figure 1F The illustration shows an example map, which is divided into regions and smaller sub-regions.

[0011] Figure 1G The illustration shows an example map, which is divided into regions and smaller sub-regions associated with the model and performance evaluation.

[0012] Figure 1H The illustration shows a sample map, which is divided into areas of different sizes.

[0013] Figure 2A The illustration shows an example map, which is associated with a single model and is divided into areas corresponding to road segments and intersections.

[0014] Figure 2B The illustration shows an example map associated with a single model, divided into areas corresponding to road segments and intersections with associated performance evaluations.

[0015] Figure 2C The illustration shows an example map, which is divided into areas corresponding to road segments and intersections associated with the model and with associated performance evaluations.

[0016] Figure 2D The illustration shows an example map, which is divided into regions and smaller sub-regions corresponding to road segments associated with the model and having associated performance evaluations.

[0017] Figure 3 The illustration shows example perception, prediction, and planning modules using models associated with map regions.

[0018] Figure 4 The diagram illustrates an example block diagram of a system for loading and activating geolocation models.

[0019] Figure 5The illustration shows an example method for preloading a prediction model configuration as the vehicle approaches a region boundary configuration and switching to the preloaded parameters when the vehicle reaches that boundary.

[0020] Figure 6 The illustration shows an example method for switching between prediction model configurations associated with different map regions at the minimum distance between trajectories predicted using model configurations.

[0021] Figure 7 The illustration shows an example method for associating a region-specific set of model parameters with a map region based on model performance.

[0022] Figure 8 The illustration shows an example of a data acquisition vehicle system collecting vehicle data from nearby vehicles and contextual data of the surrounding environment.

[0023] Figure 9 The diagram illustrates an example block diagram of a transportation management environment used to match ride requesters with autonomous vehicles.

[0024] Figure 10 The diagram illustrates an example block diagram of an algorithmic navigation pipeline.

[0025] Figure 11 The illustration shows an example computer system. Detailed Implementation

[0026] In the following description, various embodiments will be described. Specific configurations and details are set forth for illustrative purposes to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments can be practiced without specific details. Furthermore, well-known features may be omitted or simplified to avoid obscuring the embodiments being described. Moreover, the embodiments disclosed herein are merely examples, and the scope of this disclosure is not limited thereto. Specific embodiments may include all, some, or not include, the components, elements, features, functions, operations, or steps of the embodiments disclosed above. Embodiments of the invention are disclosed particularly in the appended claims for methods, storage media, systems, and computer program products, and any feature mentioned in one claim class (e.g., method) may also be claimed in another claim class (e.g., system). Dependencies or references to backreferences in the appended claims are chosen solely for formal reasons. However, any subject matter obtained by intentionally referencing backreferences to any prior claim (particularly multiple dependencies) may also be claimed such that any combination of claims and their features is disclosed and may be claimed regardless of the dependencies chosen in the appended claims. The subject matter that can be claimed includes not only combinations of features listed in the appended claims, but also combinations of any other features in the claims, wherein each feature mentioned in the claims may be combined with any other feature or combination of features in the claims. Furthermore, any embodiments and features described or depicted herein may be claimed in individual claims and / or in any combination with any embodiments or features described or depicted herein or with any features of the appended claims.

[0027] Figure 1AThe illustration depicts an example vehicle environment 100, comprising a busy intersection with vehicles 107 and numerous pedestrians 108. The computational system can utilize various models (referred to herein as machine learning models) trained to perform tasks related to controlling the vehicle's operation. These tasks can include perception-related tasks, prediction-related tasks, and planning-related tasks, etc. These models are typically trained to infer outcomes in environments sufficiently similar to other environments (with which the model has been trained) so that the model can identify the correct outcome. The similarity between two environments can be determined by comparing the representations of the two environments using appropriate metrics such as the location, shape, size, or number of specific features in each environment. It is difficult to train a general model to identify unusual environments or know how to respond appropriately to unusual situations. An unusual environment can be, for example, an environment with one or more unusual features, road conditions, objects, and / or situations. An unusual environment may share one or more features with typical environments, such as the same type of objects, the same type of roads, the same type of traffic patterns, etc. Therefore, an unusual environment can differ from a typical environment in at least one characteristic, such as having unusual features, road conditions, objects, situations, road types, traffic patterns, etc., while also possessing other characteristics that are the same as or similar to those of a typical environment. For example, Figure 1AThe vehicular environment 100 shown includes several pedestrians 106 located in traffic lanes on a street, which are unusual locations for pedestrians. Furthermore, the vehicular environment 100 includes a diagonal crosstalk across an intersection, which is likely an unusual feature. Pedestrians are likely to be on this diagonal crosstalk, especially when traffic signals allow them to use it. The relative rarity of unusual environments makes it difficult to train a general machine learning model to identify them and respond appropriately. General machine learning models are trained on many different environments where most events / training data are not unusual features or events. Therefore, general models tend to be better at identifying and responding to common, typical environments. This can conflict with the goal of identifying and responding to the specifics of the local environment. It is difficult to train a general model to make correct inferences about both typical and unusual environments. Moreover, machine learning models generalized for different environments are designed to identify patterns that appear in different environments across different geographic locations. Typical environments or features are expected to be more likely to appear in multiple different geographic locations compared to unusual environments or features. Therefore, machine learning models can be trained to work well globally (e.g., in many different locations and / or in geographically distributed locations spanning great distances). Training a global model with regard to location-specific "local" environments or characteristics can lead to overfitting, where the model performs well in one location but poorly in others.

[0028] For example, a predictive model can predict the future location or movement (e.g., trajectory) of an object detected in the environment. The location of pedestrians affects the performance of the predictive model. A predictive model trained not necessarily on unusual environments such as intersections with diagonal crosswalks may make predictions about pedestrian movement or location that differ from those made by a human driver or a predictive model trained on intersections with diagonal crosswalks. Furthermore, a predictive model trained on streets or intersections with few pedestrians may make predictions different from those made by a human driver or a predictive model trained on streets or intersections with pedestrians. For example, a general predictive model may fail to predict the trajectory of pedestrian 106 located in the driving lane (e.g., no prediction), or may predict a partially incorrect (e.g., correct direction but too fast or too slow) or completely incorrect (e.g., wrong direction and speed) trajectory of pedestrian 106. These differing predictions may be partially or completely incorrect and may lead to inefficient or incorrect vehicle operation.

[0029] Percept models can perform perception-related tasks, such as identifying the location and type of objects in an environment (e.g., pedestrians 106 and vehicles 107). Perception tasks can be affected by lighting conditions, which may be relevant to the vehicle environment. For example, in vehicle environment 100, the sun 108 may be very bright. The light from the sun 108 may be blocked by obstacles in the environment (e.g., buildings) at times. Therefore, vehicle environment 100 may have unusual lighting characteristics that are not used as factors by the model when making perception-related inferences. Since image recognition operations may be affected by light from the sun 108, models not trained on such local lighting conditions will identify or classify objects differently than humans or models trained on such lighting conditions. These different object identifications or classifications may be incorrect and may lead to inefficient or incorrect vehicle operations.

[0030] A planning model can perform planning-related tasks, such as determining the path a vehicle should follow. The planning model can identify a set of potential maneuvers or trajectories to be taken at an intersection in vehicle environment 100, and can also identify the corresponding probability of each trajectory being executed. The planning module (or another module) of the vehicle system can select one of the trajectories (e.g., the trajectory with the highest probability) for use. The planning model can identify these potential trajectories and their probabilities based on the topology of the intersection, which may include diagonal pedestrian crossings and / or the locations of pedestrians 106 and vehicles 107. For example, the planning model may have been trained or otherwise configured to identify potential trajectories based on intersections without diagonal pedestrian crossings. In the event of unusual environments (such as diagonal pedestrian crossings), such a planning model may identify potential trajectories that differ from those that a human driver or a planning model trained or otherwise configured to handle unusual environments (e.g., intersections with diagonal pedestrian crossings) would make. These different tracks may be incorrect and could lead to inefficient or incorrect vehicle operation.

[0031] Training general models to make correct inferences for both typical and unusual vehicle environments is challenging because, for example, including unusual vehicle environments in the training data with sufficient weights (e.g., the number of example environments) to produce a model that makes correct inferences for unusual environments can lead to overfitting errors, where the model makes incorrect inferences for typical environments. Therefore, it is difficult to train general perception, prediction, or planning models to make correct inferences for both typical and unusual environments.

[0032] A model's performance (e.g., how well it performs its task) is likely related to how similar the newly encountered environment is to the environment used to train the model. Therefore, models are typically trained on large amounts of data that generally resemble average (e.g., common) vehicle environments expected to be encountered frequently. If the training data is not specific enough (e.g., contains few pedestrians), the model may produce underfitting error. If the training data is too specific (e.g., contains many examples that are very different from average vehicle environments), the model may produce overfitting error. Therefore, to generate a model that performs well in common environments, the training process typically uses training data similar to common environments and attempts to minimize both overfitting and underfitting errors. However, training a general-purpose model that performs well for both rare, unusual environments and common environments is difficult and may not be feasible.

[0033] Figure 1B The illustration depicts an example map divided into regions. In a particular embodiment, a machine learning model performing tasks such as perception, prediction, or planning can be trained and used for map regions that are expected to be unusual or rare environments, while other general-purpose models can be used for other regions. Additionally and / or alternatively, separate localized machine learning models can be trained for each road segment a vehicle might travel on without using a "general" model. Dividing map 104 into regions 111-116 (which can be, for example, geographic regions, road segments, etc.), training region-specific models about one or more of regions 111-116 (rather than about the entire map 104), and using the region-specific models to make inferences in their respective regions, provides a solution to the problem of training general perception, prediction, or planning models to make correct inferences for both typical and unusual environments. In a particular embodiment, map 104 can be divided into any number of regions. Subsequently, one or more of these regions can be associated with corresponding region-specific models (e.g., Figure 1C(As shown). Region-specific models can be trained based on data associated with that region. After training, each region-specific model can be used to make inferences when the vehicle is in a region associated with that model, for example, making inferences related to perception, prediction, and / or planning for a vehicle located in the associated region of that model. The region associated with the general model can be, for example, a region with a relatively large number of typical environments (where the general model 121 performs well (e.g., above the threshold performance)). Regions associated with the general model are generally not associated with region-specific models. Region-specific models can be associated with regions or features that are unusual compared to other features or regions. Training a model specifically for an unusual region may produce a model that performs better than the general model in that unusual region. Region-specific models can be associated with regions with a relatively large number of unusual environments or features (where the general model 121 performs poorly (e.g., below the threshold performance)). When the vehicle is in a first region, the vehicle can use a first model associated with the first region to make inferences. When the vehicle moves into a second region, the vehicle can switch to a second model associated with the second region and use the second model to make inferences when the vehicle is in the second region. In this way, inferences are made using a model specifically trained for the region, and unusual environments or features can be handled by the model without affecting the performance of the general model in other regions.

[0034] While overfitting is generally a problem, specific implementations may benefit from its use in geographic regions. Thus, region-specific models are specifically trained with respect to their associated regions and may be overfitted to features and events specific to those regions compared to other areas of the map. However, because vehicles use each model in its associated regions rather than in others, each model produces accurate results in its associated regions. Furthermore, each model can produce more accurate results compared to a general model trained on a larger area because region-specific models can be trained to make inferences based on unusual vehicle environments in their associated regions (environments rare in other regions). Therefore, when vehicles are located in specific regions, using models trained for those specific regions has an advantage over general models trained for common regions, as region-specific models can generate correct results in unusual regions. Region-specific models advantageously utilize localized training (which would lead to overfitting errors in general models) without the disadvantages of localized training, since each region-specific model is typically not used for other regions.

[0035] In certain embodiments, a model can be trained using data representing a specific input scenario (such as a specific vehicle environment, e.g., a scenario containing roads, intersections, vehicles, pedestrians, and other objects) and a specific corresponding output (such as subsequent movement of an object or an action performed by a vehicle). The trained model can be represented as a set of parameters, such as weights, determined during the training process. The trained model can then perform tasks such as perception, prediction, and planning in a newly encountered environment by determining appropriate outputs for that environment based on the model's training. For example, a predictive model can predict a vehicle's possible current or future driving behavior based on the past behavior of a particular nearby vehicle. For example, the predictive model can be a neural network model that includes multiple weight factors and parameters to be adjusted during training. The predictive model can be trained using past driving behavior data collected, for example, by a team of data collection vehicles. During the training process, the multiple weight factors of the predictive model can be adjusted based on a comparison between the known behavior of observed vehicles (the ground truth) and the predicted behavior of those vehicles generated using the current weight factors of the predictive model.

[0036] In a particular embodiment, a switch to the model described above can be performed when the vehicle enters the second region (e.g., by crossing the boundary between the first and second regions). However, for the same or similar environments, the first and second models may produce significantly different results, so a sudden switch between models can lead to unexpected or undesirable vehicle behavior, such as abrupt changes in speed or direction of travel. Therefore, one or more smoothing techniques can be used to reduce the differences in results when a model switch occurs. For example, the region boundaries can be extended such that each region partially overlaps with each adjacent region, and the training of the model associated with each region can include this overlapping region. The switch between models can be performed when the vehicle is in the overlapping region, thereby reducing or eliminating abrupt changes in vehicle operation. As another example of a smoothing technique, when the vehicle is in the first region, heading towards the second region, and within a threshold distance from the second region, first and second prediction models associated with the first and second regions, respectively, can be used to generate first and second prediction trajectories for the vehicle. These trajectories can be compared to identify the minimum distance between them, and then the switch between models can be made when the vehicle is at that minimum distance.

[0037] In a particular embodiment, a region-specific model may be associated with a region based on the performance of other models associated with a larger enclosed region. When the performance of a model associated with a larger region falls below a performance threshold, the larger region may be divided into two or more sub-regions, and a new region-specific model may be associated with and trained for each new sub-region. Sub-regions may be of equal size, or may have size or shape based on map-based topological features or other features such as traffic flow. Note that the term sub-region is used for explanatory purposes to refer to the result of dividing a larger region, and a sub-region may otherwise be understood as the same as a region; therefore, the terms region and sub-region are used interchangeably.

[0038] In a particular embodiment, the computing system may include modules for different types of tasks such as perception, prediction, and planning. Different tasks may involve different types of models that can be trained differently, so the process of creating region-specific models can produce different region-specific models (which can be mapped to different map regions) for each type of task. As a result, each module in these computing system modules may be associated with a different set of inference models. A perception module may be associated with a set of perception models, a prediction module with a set of prediction models, and a planning module with a set of planning models. Each different set of inference modules may include one or more region-specific models trained to perform the task type associated with that set of models. Perception models may include, for example, three region-specific perception models associated with three corresponding regions of the map. Prediction models may include, for example, four region-specific prediction models associated with four corresponding regions of the map. Planning models may include, for example, three region-specific planning models associated with three corresponding regions of the map. These groups of perception models, prediction models, and planning models, along with their corresponding map regions, can be generated through a process that identifies map regions to be associated with the inference models, trains the inference models based on the associated map regions, and, if appropriate (e.g., to improve model performance), splits the map regions into multiple sub-regions with different sub-region-specific inference models. As a result of using different inference models for each task type, the model for a specific task type uses a region-specific model that differs from those for other tasks. Since map regions can be generated based on region-specific models (e.g., based on the performance of region-specific models), each task type may have map regions that are shaped and arranged differently from those of other task types.

[0039] Map 104 is divided into six equal-sized regions 111-116. These regions are separated by boundaries. Each region 111-116 is shown as being enclosed by a rectangle with dashed lines. For graphic and labelling purposes, the rectangles formed by the dashed lines are shown as being slightly smaller than the area of ​​region 111-116. However, in this example, regions 111-116 cover the entire map 104, and the region boundaries are shown as solid lines. Map 104 can be divided into any suitable number of regions of any suitable size or shape, according to any suitable criteria or map partitioning (e.g., zoning) technique. Example map partitioning techniques that can be used include dividing into a specified number of equal-sized regions, dividing into regions of a specified size, dividing into regions of random size and shape (subject to constraints such as minimum and maximum area limits per region, number of sides, angles, or curves), dividing regions according to map topology, or dividing regions based on other characteristics of the regions. For example, one or more regions may conform to the shape or other features of corresponding roads, road intersections, lanes, etc. on map 104. Therefore, for example, a distinct region can be created for each road segment with a length between a minimum and a maximum length. Each region for a road segment can have the same boundary as the road segment, or it can have a boundary at a predetermined distance from the road segment, such that the shape of the region is similar to but larger than the shape of the road segment. As another example, regions can be created for each intersection that meets criteria such as having at least a minimum size (e.g., square units of area), at least a minimum traffic volume, at least a minimum number of lanes, etc. Each region for an intersection can have the same boundary as the intersection, or it can enclose an area larger or smaller than the intersection, such as a rectangle, bounding box, or circle enclosing the intersection. In other examples, regions can be created based on the density of roads, intersections, buildings, houses, and / or other map features. As an example, a region with a road density less than a threshold and a size at least a threshold (e.g., square units of area) can form a first region, and an adjacent region with a road density greater than the threshold and a size less than the threshold maximum can form a second region. While this article describes a specific example of dividing a map into regions, any suitable technique can be used to divide a map into regions. As another example, a map or a region of a map can be divided into two or more regions based on the characteristics of the map or region; these regions may be referred to herein as sub-regions. These characteristics can include map information such as the type of map features, such as highways, highway entrance or exit ramps, arterial roads, city streets, neighborhood / residential streets, traffic circles, pedestrian crossings, schools, parks, shopping malls, commercial areas, unpaved roads, speed limits (on roads), bridges, ferries, parking lots, etc.Alternatively or additionally, these characteristics may include information that changes more frequently, such as traffic conditions, average speed on the road, the presence of school zones, the presence of road construction or road closures, accident reports, etc.

[0040] A region can be identified based on the location of one or more characteristics (e.g., depiction), such that the boundary of a region or map feature having one or more of these characteristics forms the boundary of the region. Alternatively or additionally, a region can be identified based on the location of one or more characteristics, but the region's area boundary can be larger than the area or map feature having those characteristics. For example, park 111a has no roads within its boundaries, but roads near the park may be affected by the park's presence (e.g., pedestrians can walk to and from the park via roads near the park). Park region 111b can be identified based on park 111a. The boundary of park region 111b can be generated by extending the boundary of park 111a by a threshold distance on each side of park 111a to include roads near park 111a. The boundary of park region 111b is thus shown as a rectangle enclosing roads within a threshold distance of the park.

[0041] In another example, region 113 may include parking area 113a. Region 116 may include road area 116a (which includes roads), another road area 116b (which includes another road leading to the parking lot), and parking area 116c (which includes the parking lot). The boundaries of parking areas 113a and 116c and the boundaries of road areas 116a and 116b may correspond to the boundaries of map features enclosed by these regions. Thus, for example, the boundary of parking area 113a may be the same as the boundary of the parking lot (on which parking area 113a is identified). In other examples, one or more boundaries of the regions may be larger than the features enclosed by the region (e.g., parking lots) by, for example, a threshold distance, similar to the extension of the boundary of park 111a as described above to enclose nearby roads.

[0042] In a particular embodiment, each of regions 111-116 can be associated with a corresponding machine learning model. The general model 121 can be associated with those regions in the map that are not associated with region-specific models 121b, 123a. For example, as... Figure 1BAs shown, the entire map 104 can initially be associated with a general model 121 (before associating any region-specific models). The general model 121 can be used to make inferences about regions of the map associated with that model (as well as regions not associated with other models). If vehicles are typically driven differently in, for example, park area 111b than in other areas of map 104, then region-specific model 121b can be associated with park area 111b. If vehicles are typically driven differently in different park areas, then those different park areas can be associated with different models. As another example, two parking areas 113a and 116c are associated with a single region-specific model 123a because vehicles are typically driven the same way in different parking areas. Road area 116a, corresponding to a road, is separate from enclosed area 116 and associated with a road-specific model (not shown) because vehicles are typically driven differently on the corresponding road than in other areas of enclosed area 116. Another road area 116b leading to parking area 116c is also associated with a road-specific model (not shown) because vehicles are typically driven in the same way in both road areas 116a, 116b. In certain embodiments, a model for an enclosed map or area (such as model 121 associated with map 104) is not used for smaller areas (such as areas 116, 116a, 111c associated with different models) because different models are used for these smaller areas instead of the model associated with the enclosed map or area.

[0043] The general model 121 can be trained on the entire map 104 or on one or more regions 111-116 that are not associated with region-specific models 121b, 123a, 126a. The general model 121 can be trained on map 104, and each region-specific model can be created by replicating the general model 121. The region-specific models can then be trained on training data associated with the corresponding region (e.g., received within that region), which can be, for example, training data captured or detected by the vehicle's sensors when the vehicle is located in that region. As another example, the region-associated training data can include training data with one or more associated locations within that region. The training data can include, for example, sensor data and associated driver actions or vehicle operations.

[0044] Although different regions are described herein as being associated with different models, these regions can alternatively be associated with a single model (which can be configured differently for different regions using a different set of parameters for each region). Therefore, the term "model" can refer to a model configured with a specific set of parameters, and different models can refer to the same model configured with different sets of parameters. Thus, for example, the term "region-specific model" can refer to a region-specific model formed by using a set of region-specific model parameters on a region-independent model. A region-independent model can include, for example, program code or instructions for generating inference based on parameters (which can be changed by loading different parameters into the model). Thus, each model can be configured according to a set of model parameters (e.g., weights determined during training). By loading a set of parameters associated with a specific region into the model, the model can be configured for use in that specific region. Although map 104 is shown as being divided into a specific number of regions 111-116 with specific shapes and sizes, in other examples, map 104 can be divided into a different number of regions, each of which can have a different size and / or shape. Therefore, the boundary of a region is not necessarily a straight line.

[0045] Figure 1C The illustration shows an example map 104, which is divided into partially overlapping regions 161-166 associated with models 121-126. Region 161-166 is similar to... Figure 1BRegions 111-116 are partially overlapping. These partially overlapping regions can be used to smooth the transition between pairs of models 121-126, which can occur when a vehicle moves into a new (e.g., different) region within regions 161-166 and switches to a new (e.g., different) model within those regions. For example, when the vehicle is in the first region 161, it can make inferences using the first model 121 associated with the first region 161. When the vehicle moves into the second region 162, it can switch to the second model 122 associated with the second region 162 and make inferences using the second model 122 while the vehicle is in the second region 162. This switching of models can be performed when the vehicle enters the second region (e.g., by crossing the boundary between the first and second regions). However, the first and second models 121, 122 may produce significantly different results for the same or similar environments, so a sudden switch between models can lead to unexpected or undesirable vehicle behavior, such as sudden changes in speed or direction of travel. Therefore, when model switching occurs, one or more smoothing techniques can be used to reduce the variability in the results. For example, region boundaries can be extended such that each region in regions 161-166 partially overlaps with each of its neighboring regions, and models 121-126 associated with a region (besides the non-overlapping areas of their respective regions) are also trained with respect to the overlapping areas, such that each pair of neighboring models is consistent in the overlapping areas of the pair of regions associated with that pair of neighboring models. When the vehicle is in the overlapping region, switching between models can be performed, thereby reducing or eliminating abrupt changes in vehicle operation.

[0046] exist Figure 1CIn map 104, each region partially overlaps with two or more other regions. For example, region 161 partially overlaps with regions 162, 164, and 165. Therefore, a vehicle traveling between any two regions on map 104 will traverse the overlapping area of ​​those two regions. Each of regions 161-166 is associated with a corresponding model 121-126. Each model 121-126 can be trained with respect to data (e.g., driving events, etc.) associated with a location within the boundary of its corresponding region in regions 161-166 (but in certain embodiments, data not associated with that corresponding region in regions 161-166 is not used for training). Thus, for example, model 121 can be trained based on training data associated with any location in region 161, which may include areas of region 161 that do not overlap with any other region, and areas of region 161 that overlap with regions 162, 164, and 165 (but model 121 is not trained based on training data associated with locations not in these regions). Similarly, each of regions 162-166 can be trained on training data associated with non-overlapping or overlapping surface regions of the corresponding region. When the vehicle moves from one region to another (e.g., from region 161 to region 162), the switching from model 121 (e.g., from model parameters associated with region 161) to model 122 (e.g., to model parameters associated with region 162) can occur when the vehicle is in a surface region where regions 161 and 162 overlap (e.g., Figure 1C The process is performed in the right portion of region 161 and the left portion of region 162 shown in the diagram. In a particular embodiment, the overlapping region may include traffic events used to provide training data for the first and second models. Therefore, the first and second regions may be defined such that at least one intersection exists in their overlapping common area.

[0047] Figure 1D An example map is shown, associated with a single model 121 and divided into regions 111-116 with associated performance assessments. Performance assessments can be generated for the illustrated regions using techniques based on simulation, feedback from human drivers, or a combination thereof. Performance assessments may produce multiple measurements of the quality or correctness of the model output based on different performance metrics, such as accuracy, reversal, etc. Figure 1DThe example performance evaluation shown is expressed as a single number for each model. This number is a percentage and can be based on one or more performance metrics, such as the average of multiple performance metrics converted to a percentage. Therefore, in this example, model performance can range from 0% (worst performance) to 100% (best performance). Although performance is measured as a percentage value in the example described herein, there are many different ways to measure and represent performance, and thresholds of different types and / or values ​​can be used. In other examples, performance may be categorized into specific safety levels (e.g., A, B, C, 1, 2, 3). Furthermore, other performance metric thresholds can be used. For example, instead of specific percentage thresholds, performance metric thresholds can be inherently normalized by the metric itself for each model (or combination of models, if multiple models are evaluated together). As another example, performance can be measured based on passenger comfort level, the suddenness of changes in speed or direction, fuel efficiency, etc.

[0048] Figure 1D The illustration shows example performance measurements of the model when the general model 121 is used for each generated inference in regions 111-116. The example values ​​are shown for illustrative purposes, and the model's performance with respect to the regions may produce different values ​​in other examples. Figure 1D The performance evaluation of the general model 121 shown includes 90% performance for region 111, 60% for region 112, 50% for region 113, 65% for region 114, 65% for region 115, and 70% for region 116. Due to environmental differences in different regions, model 121 may perform differently for different regions. In this example, model 121 performs well for region 111 (region 111 may be similar to the region where model 121 was trained).

[0049] In a particular embodiment, a threshold model performance value can be used to determine whether to create a new model and associate it with a specific region. In the example described herein, the threshold model performance is 80%. Any example model performance evaluation below 80% will result in that model being replaced with one or more region-specific models from the example described herein. Therefore, in regions 112-116, the general model 121 will be replaced with a region-specific model because the performance evaluation for these regions is below 80%. Other threshold values ​​or other model replacement criteria may be available in other examples. For example, instead of using an absolute threshold value, models with performance deviating from the model's average performance by more than a threshold percentage can be replaced with a region-specific model. The region-specific models created for regions 112-116 are... Figure 1E It is shown in the figure and described below.

[0050] Figure 1E Example map 104 illustrates regions 111-116, which are divided into areas associated with models 121-126 and performance evaluation. According to... Figure 1D The performance evaluation shown indicates that the performance of the general model 121 with respect to regions 112-116 is poor enough that the model replacement criterion (performance <80%) is met, and one or more localized models can be trained for each of one or more different regions 122-126 or combinations of regions. Therefore, in this example, the general model 121 is replaced by one or more localized models for regions 122-126. In other examples, the general model 121 can be replaced by a single localized model (not shown) for regions 122-126 (in which case regions 122-126 can be represented as a single region instead of five separate regions). In this example, if a single localized model is used for a single region at this time, then if that single region or multiple sub-regions are identified as meeting the model replacement criterion, that single region can subsequently be divided into two or more sub-regions associated with two or more different models.

[0051] In addition, Figure 1E In the example, the general model 121 remains associated with region 111 (where the model's performance is 90%). New models 122-126 have been created (e.g., by copying the general model 121 or other existing models, or as untrained models). Each of the new models 122-126 is associated with a corresponding region within regions 112-116. Each of the new models 122-126 can be configured according to a corresponding set of model parameters (e.g., weights), which may have been generated during previous training. The new models 122-126 can be trained based on training data associated with their positions in their respective regions 112-116. This training process can be performed before making inferences in the vehicle system using models 122-126. After the training process has been performed, the performance of each model in models 122-126 is assessed using references such as those described above. Figure 1D The performance measurement techniques described were used for evaluation. The obtained performance evaluations were 70% for model 122, 80% for model 123, 85% for model 124, 85% for model 125, and 50% for model 126. Therefore, models 122 (for region 112) and 126 (for region 116) are still below 80% and meet the model replacement criteria. Models 122 and 126 were replaced with new models, such as... Figure 1F As shown and described below.

[0052] Figure 1F Example map 104 is illustrated, divided into regions and smaller sub-regions. According to... Figure 1E The performance evaluations shown indicate that Model 122 performs poorly with respect to Region 112 and Model 126 performs poorly with respect to Region 116, so the model replacement criterion (performance <80%) is met, and these models are replaced by dividing their regions (112 and 116) into smaller sub-regions. In this example, Region 112 is divided into four equal-sized sub-regions 112A-D. In this example, Region 116 is divided into three sub-regions 116A-C. Sub-region 116B is determined based on the location of the road and conforms to the shape of the road in Region 116. The region division process may have determined that the traffic volume or flow on this road is significantly greater than or different from the traffic flow in other areas of Region 116, so this road is assigned to sub-region 116B, which is separate from the rest of Region 116. Areas with different or unique traffic characteristics can be used as regions (or sub-regions) because region-specific (or sub-region) models may perform better with respect to these areas than more general models such as 126. Although a specific number and arrangement of sub-regions are shown and described in this example, any suitable number and / or arrangement of sub-regions may be used. While region 112 is divided into four equal-sized sub-regions in this example, region 112 may be divided into any suitable number and size of sub-regions based on features or other characteristics in the map (e.g., the location of intersections or other partitioning features that could lead to the division of the region into sub-regions). Furthermore, although a particular region is divided into a specific number and size of sub-regions in the example described herein, each region may be divided into any suitable number and size of sub-regions based on features or other characteristics in the map.

[0053] Figure 1G The illustration shows an example map 104 divided into regions and smaller sub-regions associated with the models and performance evaluation. Region-specific models 122A-D have been associated with sub-regions 112A-112D and trained based on data associated with the corresponding sub-regions 112A-D. Similarly, region-specific models 126A-C have been associated with sub-regions 116A-C and trained based on data associated with the corresponding sub-regions 116A-C. The performance of these new models is described in the reference above. Figure 1DThe evaluation was conducted as follows. For sub-regions 122A-D, the obtained performance evaluations were 85% for 122A, 90% for 122B, 95% for model 122C, and 90% for model 122D. For sub-regions 116A-C, the obtained performance evaluations were 85% for model 126A, 90% for model 126B, and 90% for model 126C. Since these performance evaluations are above the 80% threshold, the model replacement criterion is not met for any region of map 104, and no further regions or models are created in this example.

[0054] Figure 1H Example map 104 illustrates areas 131-139 of varying sizes. Map areas can have different sizes and / or shapes. Areas 131-139 were determined based on the topology of map 130 by identifying areas with characteristics (such as road segments or intersections) that may differ from other areas and assigning each potentially different area to a different area. Area 131 includes a residential development area and two road segments that typically have moderate traffic flow (e.g., moderate number of vehicles per unit time and / or moderate average speed). Area 132 includes road segments that typically have low to moderate traffic flow. Area 133 includes intersections that typically have moderate to high-level through and turning traffic flow. Area 134 includes intersections between intersecting road segments that typically have low-level traffic flow. Area 135 includes road segments that typically have low-level traffic flow. Area 136 includes road segments that typically have low-level traffic flow. Zone 137 includes intersections that typically have medium to high levels of through traffic flow and medium levels of turning traffic flow. Zone 138 includes road segments that typically have low to medium levels of traffic flow. Zone 139 includes road segments that typically have medium to high levels of traffic flow. Each of Zones 131-139 is associated with a corresponding model in Zone-specific models 141-149. Zone 140 is not associated with a Zone-specific model, but may be associated with a general model (not shown).

[0055] Figure 2A The illustration shows an example map associated with a single model and divided into areas corresponding to road segments and intersections. The areas representing road segments or intersections are... Figure 2A-2D The example is shown. In a particular embodiment, the area representing a road segment or intersection may be associated with the road segment or intersection itself, rather than with a geographic area that includes those road segments or intersections. Alternatively or additionally, the area representing a road segment or intersection may be associated with a geographic area that includes those road segments or intersections. Map 204 shows the... Figure 1DMap 104 has the same geographic area, but has areas 221-234 that correspond to roads or intersections, instead of areas 111-116 that correspond to geographic areas.

[0056] Figure 2B The illustration shows an example map 204, which is associated with a single model 221M and is divided into regions 221-234 corresponding to road segments and intersections with relevant performance evaluations. Figure 2B The performance evaluation shown is for the general model 221M and can be referenced as above. Figure 1D The model is generated as described, and includes performance of 80% for region 221, 60% for region 222, 50% for region 223, 80% for region 224, 60% for region 225, 55% for region 226, 85% for region 227, 85% for region 228, 90% for region 229, 60% for region 230, 80% for region 231, 90% for region 232, 95% for region 233, and 85% for region 234. For regions 222, 223, 225, 226, and 230, the model replacement criterion (performance <80%) is met; therefore, for those regions, the general model 221M will be replaced by a region-specific model, such as... Figure 2C As shown and described below.

[0057] Figure 2C The illustration shows an example map, divided into regions corresponding to road segments and intersections associated with the model and with associated performance evaluations. According to... Figure 2B As shown in the performance evaluation, the performance of the general model 221M with respect to regions 222, 223, 225, 226, and 230 is poor enough that the model replacement criterion (performance <80%) is met, and the general model 221M is replaced for these regions. The general model 221M remains relevant to the other regions of map 204.

[0058] like Figure 2CAs shown, new models 222M, 223M, 225M, 226M, and 230M have been created (e.g., by replicating the general model 221M or other existing models, or as untrained models). Each of these new models is associated with a corresponding region among regions 222, 223, 225, 226, and 230. Each of the new models can be configured according to a corresponding set of model parameters (e.g., weights), which may have been generated during previous training. The new models can be trained based on training data associated with their positions in their respective regions. This training process can be performed before making inferences in the vehicle system using models 222M, 223M, 225M, 226M, and 230M. After the training process has been performed, the performance of each of models 222M, 223M, 225M, 226M, and 230M is assessed using references such as those described above. Figure 1D The performance measurement techniques described were used for evaluation. The obtained performance evaluations were 90% for model 222M, 60% for model 223M, 85% for model 225M, 65% for model 226M, and 65% for model 230M. Therefore, models 223M, 225M, 226M, and 230M are still below 80% and meet the model replacement criteria. These models were replaced with new models, such as... Figure 2D As shown and described below.

[0059] Figure 2D The example map 204 is illustrated, which is divided into areas and smaller sub-regions corresponding to road segments and intersections associated with the model and having associated performance evaluations. According to... Figure 2C The performance evaluation shown indicates that model 223M performs poorly enough with respect to region 223, model 226M with respect to region 226, and model 230M with respect to region 230 to meet the model replacement criterion (performance <80%). In this example, these models are replaced by dividing their regions (223, 226, 230) into smaller sub-regions. Example region 223 is a road segment intersecting with a bicycle path. Bicycle paths intersecting with road segments are unusual vehicular environments, so region 223 is divided into two sub-regions 223A and 223B, one (223B) including the bicycle path and the other (223A) not including the bicycle path, allowing region-specific models to be trained for each of the two distinct regions 223A and 223B of road segment 223. Therefore, region-specific models 223MA and 223MB are created for regions 223A and 223B, respectively.

[0060] Example region 226 is divided into two sub-regions 226A and 226B. The region division process may have determined that the traffic volume or flow on the roads in sub-region 226A is significantly slower than the traffic flow in region 226B. For example, this is because there is an upcoming intersection 225 for traffic flowing towards intersection 225. Therefore, the road surface area with slower traffic is assigned to sub-region 226A, which is separated from the road surface area with faster traffic (sub-region 226B). Accordingly, region-specific models 226MA and 226MB are created for regions 226A and 226B, respectively.

[0061] Example region 230 is divided into two sub-regions 230A and 230B because topological (e.g., terrain) data indicates that the portion of region (road) 230 located approximately midway between regions (intersections) 225 and 231 is situated on a steep mountaintop, and visibility is almost nonexistent between these two halves of region (road) 230. Therefore, traffic flow on the descending side of the mountaintop is faster than on the ascending side. Since the mountaintop is an unusual topological feature in the ascending side segment, sub-region 230B is created for this segment. The remaining portion of region (road) 230 is associated with the new sub-region 230A. Region-specific models 230MA and 230MB are created for regions 230A and 230B, respectively.

[0062] The performance of these new models is as described above. Figure 1D The evaluation was performed. For sub-regions 223A and 223B, the obtained performance evaluation was 85% for model 223MA and 85% for model 223MB. For sub-regions 226A and 226B, the obtained performance evaluation was 85% for model 226MA and 80% for model 226MB. For sub-regions 230A and 230B, the obtained performance evaluation was 90% for model 230MA and 80% for model 230MB. Since the performance evaluation was above the 80% threshold, the model replacement criterion was not met for any region of map 204, and in this example, no further regions or models were created. Although a specific number and arrangement of sub-regions are shown and described in this example, any suitable number and arrangement of sub-regions may be used.

[0063] Figure 3The illustration shows an example perception, prediction, and planning module 300 using a model 304 associated with a map region. The computing system can include modules 300 for different types of tasks such as perception, prediction, and planning. Different tasks can involve different types of models 304, which can be trained differently, thus the process of creating region-specific models 304 can produce different region-specific models for each type of task. These region-specific models can then correspond to different map regions. As a result, each module in the computing system module 300 can be associated with a different set of inference models 304.

[0064] In a particular embodiment, the perception module 302 may be associated with a set of perception models 306, the prediction module 322 may be associated with a set of prediction models 326, and the planning module 342 may be associated with a set of planning models 346. Each different set 306, 326, 346 of the inference module 304 may include one or more region-specific models trained to perform the task type associated with that set 306, 326, 346 of the inference model 304. As an example, the perception model 306 may include, for example, three region-specific perception models 311, 312, 313 associated with three corresponding regions R1, R2, R3 of a map. The prediction model 326 may include, for example, four region-specific prediction models 331, 332, 333, 334 associated with four corresponding regions R11, R12, R13, R14 of a map. Planning model 346 may include, for example, three region-specific planning models 351, 352, and 353 associated with three corresponding regions R21, R22, and R23 of a map. These groups of perception models 306, prediction models 326, and planning models 346, along with their corresponding map regions, can be generated through a process that identifies the map regions to be associated with inference model 304, trains the inference model based on the associated map regions, and, if appropriate (e.g., to improve model performance), subdivides the map regions into multiple sub-regions with different sub-region-specific inference models. In a particular embodiment, because a different inference model 304 is used for each type of computational task module 300, the model 304 for a particular type of computational task module 300 may use a region-specific model different from those for other types of computational task modules 300. Since map regions can be generated based on region-specific models 304 (e.g., based on the performance of region-specific models), each type of computational task module 300 may have map regions shaped and arranged differently from those for other types of computational task modules.

[0065] Figure 4The diagram illustrates an example block diagram of a system 400 for loading and activating a geolocation model. System 400 can load and activate the model to perform inference in response to a request from module 404. When the model has been loaded and activated, system 400 can send inference requests directly from computation module 404 to the activated model. Methods for improving the smoothness of model changes are described in... Figure 5 and Figure 6 As shown in the diagram, system 400 may include a model interface 408 that can interact with one or more computing modules 404, such as a sensor data module 402 sensing module, a prediction module, a planning module, and a control module. The model interface 408 provides an interface between the computing modules 404 and region-specific models. In response to receiving a request 406 for inference (e.g., a request for a prediction from a prediction module) from one or more computing modules 404, the model interface 408 may send a model request 410 to a model switcher 412 to obtain a model of the type corresponding to the computing module 404 that submitted the request 406 for inference. For example, if the prediction module submitted the request 406 for inference, the model interface may send a model request 410 for a prediction model.

[0066] Model switcher 412 can receive model request 410 and, in response, identify one or more model selection factors 414 from vehicle sensors 402 or other data sources (e.g., map database, weather service, vehicle system clock, etc.), such as current vehicle location, current time of day, current lighting conditions, current weather conditions, etc. Model switcher 412 can generate a data repository query 416 based on the requested model type and model selection factors 414 (e.g., current vehicle location). Model data repository 418 can identify entries in data repository 418 that have model selection factors (e.g., location) 420 that match those model selection factors in data repository query 416. If a match is found between query 416 and the stored selection factors 420, the model 422 associated with the matching selection factor (which may be represented as model parameters (such as weights)) can be retrieved, and the retrieved model parameters 428 can be sent to GPU 430.

[0067] In a particular embodiment, model parameter 428 may be loaded into memory 432 of graphics processing unit (GPU) 430. Loading model parameter 428 may involve retrieving model parameter 428 from model data repository 418, as described above. Model data repository 418 may be stored locally (e.g., on a local storage device of the vehicle system) and / or remotely (e.g., in a cloud storage device accessible via network communication). Therefore, there may be a delay in retrieving model parameter 428 and loading it into memory 432. In a particular embodiment, model parameter 438 may be preloaded (e.g., pre-cached), for example by loading model parameters 438 for a region before entering that region. For example, model parameters 438 for a region may be preloaded when the vehicle is traveling in a direction toward a region and is within a threshold distance of that region or is predicted to arrive at that region within a threshold time, or when the vehicle is at the overlapping boundary of that region before entering it.

[0068] In a particular embodiment, when the new model parameter 438 has been loaded, the model loader 412 may send a model switching instruction 440 to the GPU 430 or other components interacting with the GPU 430 to swap the new model parameter 438 with the current model parameter 436. The model switching instruction 440 may cause the current model pointer 434 to reference the new model parameter 438 instead of the current model parameter 436. By changing pointer 434 or executing a similar instruction to switch to the new model parameter, subsequent inference requests sent to model 410 can use the new model parameter 438 instead of the current model parameter 436. After sending the model switching instruction 440, the model loader 412 may send a model-ready response 442 to the model interface 408. The model interface 408 may forward an inference request 406 to model 410, and model 410 may interact with the GPU 430 via computation request / response 442 to perform the requested inference. The GPU 430 may perform inference by performing computations using the new model parameter 438 and generating computation results. Model 410 can receive computation results from GPU 430 and generate inference results, which may be based on or include these computation results. Model 410 can send the inference results to model interface 408, and model interface 408 can forward the inference as an inference response 444 to computation module 444.

[0069] In a particular embodiment, if a new model parameter 438 for a region has been preloaded, the system 400 can switch to the new model parameter 438 when a vehicle enters the region (e.g., crosses the boundary of the region). To switch to the new model parameter 438, the system 400 (e.g., the model loader 412 or another component of the system 412 invoked at another time after loading the model parameter 438) can change the current model pointer 434 to reference the new model parameter 438 instead of the current model parameter 436. Therefore, switching the configuration of model 410 from the current model parameter 436 to the new model parameter 438 can be performed by updating the reference associated with the configuration of model 410 to reference the new model parameter 438 in memory 432.

[0070] Figure 5 The illustration shows an example method for preloading a prediction model configuration as the vehicle approaches a region boundary and switching to the preloaded parameters when the vehicle reaches the boundary. Method 500 can begin at step 502, where the vehicle system determines the vehicle's current position and direction of travel. At step 504, the vehicle system determines whether the vehicle will cross the region boundary within a threshold distance (or time). If so, at step 508, the vehicle system searches the model data repository for new prediction model parameters associated with the different region the vehicle is expected to enter after crossing the region boundary. If not, at step 506, the vehicle system can wait until the vehicle's position changes by at least a threshold distance before calling step 502 again.

[0071] Continuing from step 508, at step 510, the vehicle system can determine whether the search in step 508 found new predictive model parameters associated with the different region. If not, at step 512, the vehicle system can use alternative model parameters as new predictive model parameters. Alternative model parameters can be, for example, parameters of a general model associated with the different region or a map including the different region. Otherwise, at step 514, the vehicle system can load the new predictive model parameters from the data repository into memory, for example, into the GPU's memory. At step 516, when the vehicle reaches the region boundary or enters the different region, the vehicle system can switch the prediction model configuration to the new predictive model parameters. At step 518, the vehicle system can use the predictive model to generate one or more inferences based on the new predictive model parameters.

[0072] Figure 6The illustration depicts an example method 600 for switching between prediction model configurations associated with different map regions at a minimum distance between predicted trajectories using model configurations. As an example of smoothing transitions between regions, when a vehicle is in a first region, moving towards a second region, and within a threshold distance of the second region, first and second prediction models associated with the first and second regions, respectively, can be used to generate first and second predicted trajectories for the vehicle. These trajectories can be compared to identify the minimum distance between them. This minimum distance can correspond to, for example, a point of maximum consistency between the trajectories. The switching between models can then be made at a location corresponding to the minimum distance. The method can begin at step 602, where the vehicle system uses the current prediction model to determine a first predicted trajectory for the vehicle at its current position and for a predetermined duration. The predetermined duration can be a duration starting at the current time or in the future. At step 604, the vehicle system can determine, based on the first predicted trajectory, whether the vehicle is predicted to cross a region boundary within a duration. If not, at step 606, the vehicle system can wait until the vehicle's position changes by at least a threshold distance and call step 602 again. At step 608, the vehicle system may search the model data repository for new predictive model parameters associated with the different area the vehicle is expected to enter after crossing the boundary. At step 610, the vehicle system may determine whether a new predictive model parameter associated with the different area has been found. If not found, at step 612, the vehicle system may use alternative model parameters as new predictive model parameters. Alternative model parameters may be, for example, parameters of a general model associated with the different area or with a map including the different area.

[0073] At step 614, the vehicle system can begin loading new parameters from the data repository into memory (e.g., the GPU's memory). At step 616, the vehicle system can use the new prediction model parameters to determine a second predicted trajectory for the vehicle over a predetermined duration. At step 618, the vehicle system can identify the location on the first predicted trajectory that is at the minimum distance from the second predicted trajectory. At step 620 (when the new parameters have been loaded into memory), the vehicle system can switch from the current prediction model parameters to the new prediction model parameters when the vehicle reaches the identified location, for example, by instructing the GPU to use the new prediction model parameters.

[0074] Figure 7An example method 700 for associating a region-specific set of model parameters with a map region based on model performance is illustrated. In a particular embodiment, the region-specific model may be associated with a region based on the performance of other models associated with a larger enclosed region. When the performance of the model associated with the larger region is below a performance threshold (and optionally, the size of the region (e.g., area in squares) is above a size threshold), the larger region may be divided into two or more sub-regions, and a new region-specific model may be associated with and trained for each new sub-region. Sub-regions may be of the same size, or may have size or shape based on map-based topological features or other features such as traffic flow. Thus, for example, when the performance of a region is below a performance threshold and the region includes roads and parking lots, the region may be divided into two sub-regions: one for roads (and having a shape consistent with roads), and another for the remainder of the region (which includes parking lots). Each sub-region may be associated with a corresponding model, and each model may be trained based on training data associated with the corresponding sub-region.

[0075] Method 700 may begin at step 702, where the vehicle system can divide the geographic map into initial regions. For example, these regions may be as described above. Figure 1B As described, the region is identified. At step 704, the vehicle system can determine a per-region performance evaluation of a first model(s) using one or more first set of model parameters associated with the region, wherein the performance evaluation of each region is based on sensor data captured in that region. At step 706, the vehicle system can identify one or more regions whose evaluated performance is below a minimum performance threshold. At step 708, the vehicle system can begin a loop of step 710 (and subsequent steps) for each identified region. At step 710, the vehicle system can determine whether the region is already associated with region-specific model parameters. If not, at step 712, the vehicle system can store the association between the region and the region-specific model (which has new parameters for generating inference in that region). At step 714, the vehicle system can generate inference using the region-specific model and specified input parameters. At step 716, the vehicle system can train a new region-specific model based on training data associated with the region of the corresponding new region-specific model (e.g., sensor data received by the vehicle when it is in the corresponding region). Step 716 may call the method portion of step 704 to repeatedly identify regions and associate the model and / or model parameters with these regions.

[0076] Referring again to the result of step 710, if the vehicle system determines that a region is already associated with region-specific model parameters, then at step 718, the vehicle system can divide the region into two or more new regions. At step 720, the vehicle system can remove the association between the region and the region-specific model and parameters. At step 722, for each new region, the vehicle system can store the association between the new region and a new region-specific model with parameters for generating inference in that region. At step 716, the vehicle system can train the new region-specific model as described above and call step 704 to repeat the method portion that began at step 704. In a particular embodiment, the method can stop repeating based on conditions evaluated at one or more steps. For example, step 706 can determine whether each of the identified regions is smaller than a threshold size (in area units). If so, step 706 can end process 700. As another example, step 704 can determine whether the performance evaluation for each region has been re-evaluated for the same region and has not degraded. If no performance evaluation decreases, or if they do not decrease by more than a threshold amount, then step 704 can terminate process 700.

[0077] Figure 8 The illustration shows an example scenario 800 where a data acquisition vehicle system 810 collects vehicle data from nearby vehicles 820 and contextual data of the surrounding environment. In certain embodiments, the vehicle system 810 (e.g., an autonomous vehicle, a manually driven vehicle, a computer-assisted driving vehicle, a human-machine hybrid driving vehicle, etc.) may have multiple sensors or sensing systems 812 for monitoring vehicle status, other vehicles, and the surrounding environment. Sensors or sensing systems 812 may include, for example, but not limited to, cameras (e.g., optical cameras, thermal imaging cameras), LiDAR, radar, speed sensors, steering angle sensors, brake pressure sensors, GPS, inertial measurement units (IMUs), accelerometers, etc. The vehicle system 810 may include one or more computing systems (e.g., data collection devices, mobile phones, tablets, mobile computers, in-vehicle computers, high-performance computers) to collect data about the vehicle, nearby vehicles, the surrounding environment, etc. In a particular embodiment, the vehicle system 810 may collect data about the vehicle itself, which is related to, but is not limited to, vehicle speed, direction of movement, wheel direction, steering angle, steering force on the steering wheel, brake pedal pressure, accelerator pedal pressure, acceleration (e.g., based on IMU output), rotational speed (e.g., based on IMU / gyroscope output), vehicle movement path, vehicle trajectory, position (e.g., GPS coordinates), signal status (e.g., on / off status of turn signal, braking signal, emergency signal), and human driver's eye movements, head movements, etc.

[0078] In a particular embodiment, vehicle system 810 may use one or more sensing signals 822 of sensing system 812 to collect data on nearby vehicles 820. For example, vehicle system 810 may collect vehicle data and driving behavior data, which may be related to, but not limited to, vehicle images, vehicle speed, acceleration, vehicle movement path, vehicle driving trajectory, position, turn signal status (e.g., turn signal on / off state), braking signal status, distance to another vehicle, relative speed to another vehicle, distance to pedestrians, relative speed to pedestrians, distance to traffic signals, distance to intersections, distance to road signs, distance to curbs, relative position to road lines, objects in the vehicle's field of vision, positions of other traffic objects, and aggression measures of other vehicles. Furthermore, sensing system 812 may be used to identify nearby vehicles 820, which may be based on anonymous vehicle identifiers based on license plate numbers, QR codes, or any other suitable identifier that uniquely identifies nearby vehicles.

[0079] In a particular embodiment, vehicle system 810 may collect contextual data of the surrounding environment based on one or more sensors associated with vehicle system 810. In a particular embodiment, vehicle system 810 may collect data related to road conditions or one or more objects in the surrounding environment, such as, but not limited to, road layout, pedestrians, other vehicles (e.g., 820), traffic conditions (e.g., number of nearby vehicles, number of pedestrians, traffic signals), time of day (e.g., morning rush hour, evening rush hour, off-peak hours), traffic type (e.g., high-speed traffic, accident events, slow-moving traffic), location (e.g., GPS coordinates), road conditions (e.g., built-up areas, school zones, wet surfaces, icy surfaces), intersections, road signs (e.g., stop signs 860, road lines 842, pedestrian crossings), nearby objects (e.g., curbs 844, lampposts 850, billboards 870), buildings, weather conditions (e.g., rain, fog, sunny, hot, cold weather), or any object or feature in the surrounding environment. In a particular embodiment, the vehicle's context data may include the vehicle's navigation data, such as navigation maps, navigation destinations, routes, estimated arrival times, detours, etc. In a particular embodiment, the vehicle's context data may include camera-based positioning data, including but not limited to point clouds, depth of field, two-dimensional contours of the environment, three-dimensional contours of the environment, stereoscopic images of the scene, relative positions (e.g., distance, angle) with environmental objects, relative positions (e.g., distance, angle) with road lines, relative positions in the current environment, traffic conditions (e.g., high-speed traffic, low-speed traffic), driving trajectories of other vehicles, movement of other traffic objects, speeds of other traffic objects, directions of movement of other traffic objects, signal status of other vehicles, etc. In a particular embodiment, the vehicle system 810 may have perception of the surrounding environment based on context data collected in real time through one or more sensors and / or based on historical context data stored in a vehicle model database.

[0080] Figure 9The illustration shows an example block diagram of a transportation management environment for matching ride requesters with autonomous vehicles. In a particular embodiment, the environment may include various computing entities, such as a user computing device 930 for user 901 (e.g., a ride provider or requester), a transportation management system 960, an autonomous vehicle 940, and one or more third-party systems 970. These computing entities can be communicatively connected via any suitable network 910. By way of example and not limitation, one or more portions of network 910 may include an ad hoc network, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, or any combination thereof. In a particular embodiment, any suitable network arrangement and protocol that enables the computing entities to communicate with each other may be used. Although Figure 9 The illustration depicts a single user device 930, a single transportation management system 960, a single vehicle 940, multiple third-party systems 970, and a single network 910; however, this disclosure contemplates any suitable number of each of these entities. As an example and not a limitation, a network environment may include multiple users 901, user devices 930, transportation management systems 960, autonomous vehicles 940, third-party systems 970, and networks 910.

[0081] User equipment 930, transportation management system 960, autonomous vehicle 940, and third-party system 970 can be connected or co-located, either wholly or partially, in a communicative manner. These computing entities can communicate via different transmission technologies and network types. For example, user equipment 930 and vehicle 940 can communicate with each other via cable or short-range wireless communication (e.g., Bluetooth, NFC, Wi-Fi, etc.), and they can connect together to the Internet via a cellular network accessible to either device (e.g., user equipment 930 could be a smartphone with LTE connectivity). Conversely, transportation management system 960 and third-party system 970 can connect to the Internet via their respective LAN / WLAN networks and Internet service providers (ISPs). Figure 9The illustration depicts a transmission link 950 connecting user equipment 930, autonomous vehicle 940, transportation management system 960, and third-party system 970 to a communication network 910. This disclosure contemplates any suitable transmission link 950, including, for example, wired connections (e.g., USB, Lightning, Digital Subscriber Line (DSL), or Cable Data Service Interface Specification (DOCSIS), wireless connections (e.g., Wi-Fi, WiMAX, cellular, satellite, NFC, Bluetooth), optical connections (e.g., Synchronous Optical Network (SONET), Synchronous Digital Hierarchy (SDH)), any other wireless communication technology, and any combination thereof. In certain embodiments, one or more links 950 may connect to one or more networks 910, which may include, in part, ad hoc networks, intranets, extranets, VPNs, LANs, WLANs, WANs, WWANs, MANs, PSTNs, cellular networks, satellite networks, or any combination thereof. Computing entities do not necessarily need to use the same type of transmission link 950. For example, user equipment 930 may communicate with the transportation management system via a cellular network and the Internet, but with autonomous vehicle 940 via Bluetooth or a physical wired connection.

[0082] In a particular embodiment, the transportation management system 960 can fulfill the ride requests of one or more users 901 by dispatching suitable vehicles. The transportation management system 960 can receive any number of ride requests from any number of ride requesters 901. In a particular embodiment, a ride request from a ride requester 901 may include an identifier that identifies the ride requester in the system 960. Depending on the privacy settings of the requester 901, the transportation management system 960 can use this identifier to access and store information about the ride requester 901. The information about the ride requester 901 may be stored in one or more data repositories (e.g., relational database systems) associated with and accessible by the transportation management system 960. In a particular embodiment, the ride requester information may include profile information about a specific ride requester 901. In a particular embodiment, a ride requester 901 may be associated with one or more categories or types, through which the ride requester 901 may be associated with aggregated information about certain ride requesters of those categories or types. Ride information may include, for example, preferred pick-up and drop-off locations, driving preferences (e.g., safety and comfort levels, preferred speeds, acceleration / deceleration rates, safe distances from other vehicles at various speeds, routes, etc.), entertainment preferences and settings (e.g., preferred music genres or playlists, volume, display brightness, etc.), temperature settings, whether conversation with the driver is welcome, frequently visited destinations, historical ride patterns (e.g., travel times during the day, start and end locations, etc.), preferred language, age, gender, or any other suitable information. In a particular embodiment, the transportation management system 960 may classify user 901 based on known information about user 901 (e.g., using a machine learning classifier) ​​and use this classification to retrieve relevant aggregated information associated with that category. For example, system 960 may classify user 901 as a young adult and retrieve relevant aggregated information associated with young adults, such as music genres typically preferred by young adults.

[0083] The transportation management system 960 can also store and access ride information. Ride information may include location associated with the ride, traffic data, route options, optimal pick-up or drop-off locations, or any other suitable information associated with the ride. As an example, and not a limitation, when the transportation management system 960 receives a request to travel from San Francisco International Airport (SFO) to Palo Alto, California, the system 960 may access or generate any relevant ride information for that specific ride request. For example, the ride information may include a preferred pick-up location at SFO; alternative pick-up locations if the pick-up location is incompatible with the ride requester (e.g., the ride requester may be disabled and unable to access the pick-up location) or if the pick-up location is otherwise unavailable due to construction, traffic congestion, changes in pick-up / drop-off rules, or any other reason; one or more routes navigating from SFO to Palo Alto; a user's preferred exit ramp; or any other suitable information associated with the ride. In certain embodiments, some portions of the ride information may be based on historical data associated with historical rides facilitated by the system 960. For example, historical data may include aggregated information generated based on past ride information, which may include any ride information described herein as well as telemetry data collected by sensors in autonomous vehicles and / or user devices. Historical data may be associated with a specific user (e.g., the user's preferences, common routes, etc.), a user category / classification (e.g., based on demographics), and / or all users of system 960. For example, historical data specific to a single user may include information about past rides taken by that specific user, including the user's boarding and alighting locations, the music the user prefers to listen to, traffic information associated with the ride, the time of day the user most frequently rides, and any other suitable information specific to that user. As another example, historical data associated with a user category / classification may include, for example, common or popular ride preferences for users within that category / classification, such as teenagers preferring popular music, a ride requester who frequently commutes to the financial district might prefer listening to the news, and so on. As yet another example, historical data associated with all users may include general usage trends, such as traffic and ride patterns. By using historical data, system 960 in a particular embodiment may predict and provide ride suggestions in response to ride requests. In a particular embodiment, system 960 may use machine learning algorithms, such as neural networks, regression algorithms, instance-based algorithms (e.g., k-nearest neighbors), decision tree algorithms, Bayesian algorithms, clustering algorithms, association rule learning algorithms, deep learning algorithms, dimensionality reduction algorithms, ensemble algorithms, and any other suitable machine learning algorithms known to those skilled in the art. The machine learning model may be trained using any suitable training algorithm, including supervised learning based on labeled training data, unsupervised learning based on unlabeled training data, and / or semi-supervised learning based on a mixture of labeled and unlabeled training data.

[0084] In a particular embodiment, the transportation management system 960 may include one or more server computers. Each server may be a single server or a distributed server spanning multiple computers or multiple data centers. Servers may be of various types, such as, but not limited to, web servers, news servers, mail servers, messaging servers, advertising servers, file servers, application servers, exchange servers, database servers, proxy servers, other suitable servers for performing the functions or processes described herein, or any combination thereof. In a particular embodiment, each server may include hardware, software, or embedded logic components, or a combination of two or more such components, for performing appropriate functions implemented or supported by the server. In a particular embodiment, the transportation management system 960 may include one or more data stores. Data stores may be used to store various types of information, such as ride information, ride requester information, ride provider information, historical information, third-party information, or any other suitable type of information. In a particular embodiment, the information stored in the data repository may be organized according to a specific data structure. In a particular embodiment, each data repository may be a relational, columnar, correlated, or any other suitable type of database system. Although this disclosure describes or illustrates specific types of databases, this disclosure contemplates any suitable type of database. Specific embodiments may provide an interface that enables user equipment 930 (which may belong to a ride requester or provider), transportation management system 960, vehicle system 940, or third-party system 970 to process, transform, manage, retrieve, modify, add, or delete information stored in a data repository.

[0085] In a particular embodiment, the transportation management system 960 may include an authorization server (or any other suitable component(s)) that allows user 901 to opt in or out of having their information and actions recorded, logged, sensed by the transportation management system 960, or shared with other systems (e.g., third-party system 970). In a particular embodiment, user 901 can opt in or out by setting appropriate privacy settings. The user's privacy settings may determine what information associated with the user is recorded, how the information associated with the user is recorded, when the information associated with the user is recorded, who can record the information associated with the user, with whom the information associated with the user can be shared, and for what purpose the information associated with the user can be recorded or shared. The authorization server may be used, as appropriate, to enforce one or more privacy settings of user 901 of the transportation management system 960 through blocking, data hashing, anonymization, or other suitable techniques.

[0086] In a particular embodiment, the third-party system 970 may be a network-addressable computing system that can provide high-definition maps or host GPS maps, customer reviews, music or content, weather information, or any other suitable type of information. The third-party system 970 can generate, store, receive, and transmit relevant data, such as, for example, map data, customer review data from a customer review website, weather data, or any other suitable type of data. Other computing entities in the network environment can access the third-party system 970 directly or via network 910. For example, user equipment 930 can access the third-party system 970 via network 910 or via transportation management system 960. In the latter case, if credentials are required to access the third-party system 970, user 901 can provide such information to transportation management system 960 (which can act as a proxy for accessing content from the third-party system 970).

[0087] In a particular embodiment, user equipment 930 may be a mobile computing device, such as a smartphone, tablet, or laptop. User equipment 930 may include one or more processors (e.g., CPU and / or GPU), memory, and storage devices. Operating systems and applications may be installed on user equipment 930, such as transportation applications associated with transportation management system 960, applications associated with third-party system 970, and applications associated with the operating system. User equipment 930 may include functionality for determining its position, orientation, or direction based on integrated sensors such as GPS, compass, gyroscope, or accelerometer. User equipment 930 may also include a wireless transceiver for wireless communication and may support wireless communication protocols such as Bluetooth, near field communication (NFC), infrared (IR) communication, Wi-Fi, and / or 2G / 3G / 4G / LTE mobile communication standards. User equipment 930 may also include one or more cameras, scanners, touchscreens, microphones, speakers, and any other suitable input-output devices.

[0088] In a particular embodiment, vehicle 940 may be an autonomous vehicle and equipped with sensor array 944, navigation system 946, and ride service computing device 948. In a particular embodiment, a fleet of autonomous vehicles 940 may be managed by transportation management system 960. All or part of the fleet of autonomous vehicles 940 may be owned by an entity associated with transportation management system 960, or they may be owned by a third-party entity relative to transportation management system 960. In either case, transportation management system 960 may control the operation of autonomous vehicles 940, including, for example, dispatching selected vehicles 940 to fulfill ride requests, instructing vehicles 940 to perform selected operations (e.g., proceeding to a service center or charging / gas station, pulling over, stopping immediately, self-diagnosing, locking / unlocking the cabin, changing the music station, changing the temperature, and any other suitable operation), and instructing vehicles 940 to enter selected operating modes (e.g., normal operation, driving at a reduced speed, driving under the command of a human operator, and any other suitable operating mode).

[0089] In a particular embodiment, the autonomous vehicle 940 can receive and transmit data from the transportation management system 960 and the third-party system 970. Examples of received data may include, for example, instructions, new software or software updates, maps, 3D models, trained or untrained machine learning models, location information (e.g., the location of the ride requester, the autonomous vehicle 940 itself, other autonomous vehicles 940, and target destinations such as service centers), navigation information, traffic information, weather information, entertainment content (e.g., music, videos, and news), ride requester information, ride information, and any other suitable information. Examples of data transmitted from the autonomous vehicle 940 may include, for example, telemetry and sensor data, determinations / decision-making based on such data, vehicle conditions or status (e.g., battery / fuel levels, tire and braking conditions, sensor conditions, speed, odometer readings, etc.), location, navigation data, passenger input (e.g., passengers can send / receive data to and from the transportation management system 960 and / or the third-party system 970 via a user interface in the vehicle 940), and any other suitable data.

[0090] In certain embodiments, autonomous vehicles 940 can also communicate with each other and with other conventional manually driven vehicles, including those managed by and not managed by the transportation management system 960. For example, one vehicle 940 can transmit data with another vehicle regarding its respective location, condition, status, sensor readings, and any other suitable information. In certain embodiments, vehicle-to-vehicle communication can occur via direct short-range wireless connections (e.g., Wi-Fi, Bluetooth, NFC) and / or via a network (e.g., the Internet or via the transportation management system 960 or a third-party system 970).

[0091] In a particular embodiment, the autonomous vehicle 940 can acquire and process sensor / telemetry data. Such data can be captured by any suitable sensor. For example, the vehicle 940 may have a LiDAR sensor array with multiple light detection and ranging (LiDAR) transceivers configured to rotate 360°, emit pulsed lasers, and measure reflected light from objects around the vehicle 940. In a particular embodiment, the emitting LiDAR can be directed using a gated optical valve, which can be a MEMS device that uses the principle of light diffraction to guide the beam. Such a device may not use a gimbal to direct the beam 360° around the autonomous vehicle. Instead, the gated optical valve can guide the beam into one of several optical fibers, which can be arranged such that the beam can be directed to many discrete locations around the autonomous vehicle. Thus, data can be captured 360° around the autonomous vehicle, but rotating components may not be required. LiDAR is an effective sensor for measuring distances to targets and can therefore be used to generate a three-dimensional (3D) model of the external environment of the autonomous vehicle 940. As an example, and not a limitation, the 3D model can represent the external environment, including objects such as other vehicles, curbs, debris, objects, and pedestrians up to the maximum range of the sensor deployment (e.g., 50, 100, or 200 meters). As another example, the autonomous vehicle 940 can have optical cameras pointing in different directions. The cameras can be used, for example, to identify roads, lane markings, street signs, traffic lights, police, other vehicles, and any other visible objects of interest. To enable the vehicle 940 to "see" at night, an infrared camera can be installed. In a particular embodiment, the vehicle can be equipped with stereo vision for, for example, detecting dangerous situations such as pedestrians or tree branches on the road. As another example, the vehicle 940 can have radar for, for example, detecting other vehicles and / or distant hazards. Furthermore, the vehicle 940 can have ultrasonic equipment for, for example, parking and obstacle detection. In addition to sensors that enable the vehicle 940 to detect, measure, and understand the external world around it, the vehicle 940 can also be equipped with sensors for detecting and self-diagnosing its own state and conditions. For example, vehicle 940 may have wheel sensors for, for example, measuring speed; a Global Positioning System (GPS) for, for example, determining the vehicle's current geographical location; and / or an inertial measurement unit, accelerometer, gyroscope, and / or odometer system for motion or movement detection. While the description of these sensors provides specific examples of their functionality, those skilled in the art will understand that the functionality of the sensors is not limited to these examples. Furthermore, although examples of functionality may be described with respect to specific types of sensors, it should be understood that functionality can be achieved using any combination of sensors. For example, autonomous vehicle 940 may construct a 3D model of its surroundings based on data from its LiDAR, radar, sonar, and cameras, as well as pre-generated maps obtained from transportation management system 960 or third-party system 970. Although sensor 944 appears... Figure 9 The sensor 944 is located at a specific location on or within the autonomous vehicle 940, but it can be located at any suitable location on or within the autonomous vehicle 940. Example locations for the sensor include the front and rear bumpers, doors, windshield, side panels, or any other suitable location.

[0092] In a particular embodiment, the autonomous vehicle 940 may be equipped with a processing unit (e.g., one or more CPUs and GPUs), memory, and storage devices. The vehicle 940 can thus be equipped to perform a variety of computational and processing tasks, including processing sensor data, extracting useful information, and operating accordingly. For example, based on images captured by its cameras and machine vision models, the vehicle 940 can identify specific types of objects captured by the images, such as pedestrians, other vehicles, lanes, curbs, and any other objects of interest.

[0093] In a particular embodiment, the autonomous vehicle 940 may have a navigation system 946 responsible for safely navigating the autonomous vehicle 940. In a particular embodiment, the navigation system 946 may take as input any type of sensor data from, for example, a Global Positioning System (GPS) module, an Inertial Measurement Unit (IMU), a LiDAR sensor, an optical camera, a radio frequency (RF) transceiver, or any other suitable telemetry or sensing mechanism. The navigation system 946 may also utilize, for example, map data, traffic data, accident reports, weather reports, instructions, target destinations, and any other suitable information to determine navigation routes and specific driving operations (e.g., deceleration, acceleration, stopping, turning, etc.). In a particular embodiment, the navigation system 946 may use these determinations to control the vehicle 940 to operate in a prescribed manner and guide the autonomous vehicle 940 to its destination without colliding with other objects. Although the physical embodiment of the navigation system 946 (e.g., a processing unit) appears... Figure 9 The navigation system 946 may be located at a specific location on or within the autonomous vehicle 940, but it can be located at any suitable location within or on the autonomous vehicle 940. Example locations for the navigation system 946 include inside the passenger compartment or cabin of the autonomous vehicle 940, near the engine / battery, near the front seats, the rear seats, or at any other suitable location.

[0094] In a particular embodiment, the autonomous vehicle 940 may be equipped with a ride-service computing device 948, which may be a tablet computer or any other suitable device installed by the transportation management system 960 to allow users to interact with the autonomous vehicle 940, the transportation management system 960, other users 901, or a third-party system 970. In a particular embodiment, the installation of the ride-service computing device 948 can be achieved by placing the ride-service computing device 948 within the autonomous vehicle 940 and configuring it to communicate with the vehicle 940 via a wired or wireless connection (e.g., via Bluetooth). Although Figure 9The illustration shows a single ride-service computing device 948 at a specific location within an autonomous vehicle 940; however, the autonomous vehicle 940 may include several ride-service computing devices 948 at several different locations within the vehicle. By way of example and not limitation, the autonomous vehicle 940 may include four ride-service computing devices 948 located: one in front of the left front passenger seat (e.g., the driver's seat in a conventional U.S. car), one in front of the right front passenger seat, and one in front of each of the left and right rear passenger seats. In certain embodiments, the ride-service computing device 948 may be detachable from any component of the autonomous vehicle 940. This allows a user to dispose of the ride-service computing device 948 in a manner consistent with other tablet computing devices. By way of example and not limitation, a user may move the ride-service computing device 948 to any location in the passenger compartment or cabin of the autonomous vehicle 940, hold the ride-service computing device 948, or dispose of the ride-service computing device 948 in any other suitable manner. Although this disclosure describes providing a particular computing device in a particular manner, this disclosure contemplates providing any suitable computing device in any suitable manner.

[0095] Figure 10 An example block diagram of an algorithmic navigation pipeline is illustrated. In a particular embodiment, the algorithmic navigation pipeline 1000 may include multiple computing modules, such as a sensor data module 1005, a perception module 1010, a prediction module 1015, a planning module 1020, and a control module 1025. The sensor data module 1005 may acquire and preprocess sensor / telemetry data provided to the perception module 1010. Such data may be captured by any suitable sensor of the vehicle. By way of example and not limitation, the vehicle may have a light detection and ranging (LiDAR) sensor configured to emit pulsed laser beams in multiple directions and measure reflected signals from objects around the vehicle. The time of flight of the light signal may be used to measure the distance or depth of the object from the LiDAR. As another example, the vehicle may have optical cameras pointed in different directions to capture images of the area around the vehicle. Radar may also be used by the vehicle to detect other vehicles and / or hazards at a distance. As yet another example, the vehicle may be equipped with ultrasonic sensors for near-field object detection (e.g., parking and obstacle detection) or infrared cameras for object detection in low-light conditions or darkness. In a particular embodiment, the sensor data module 1005 can suppress noise in the sensor data or normalize the sensor data.

[0096] The perception module 1010 is responsible for correlating and fusing data from different types of sensors in the sensor module 1005 to model the vehicle's context environment. The perception module 1010 can use information extracted by multiple independent sensors to provide information that cannot be obtained from any single type of sensor. Combining data from multiple sensor types allows the perception module 1010 to leverage the strengths of different sensors and perceive the environment more accurately and precisely. As an example, and not a limitation, image-based object recognition may not work well in low-light conditions. This can be compensated for by sensor data from LiDAR or radar, which are effective sensors for measuring distances to targets in low-light conditions. As another example, image-based object recognition may incorrectly determine that an object depicted in a poster is an actual three-dimensional object in the environment. However, if depth information from LiDAR is also available, the perception module 1010 can use this additional information to determine that the object in the poster is not actually a three-dimensional object.

[0097] The perception module 1010 can process available data (e.g., sensor data, data from a high-resolution map, etc.) to derive information about the context. For example, the perception module 1010 may include one or more object modelers (e.g., object detectors, object classifiers, or machine learning models trained to derive information from sensor data) to detect and / or classify objects present in the vehicle environment (e.g., other vehicles, pedestrians, moving objects). The perception module 1010 can also determine various characteristics of the objects. For example, the perception module 1010 can track the velocity, direction of movement, acceleration, trajectory, relative distance, or relative position of these objects. In a particular embodiment, the perception module 1010 may also utilize information from a high-resolution map. The high-resolution map may include an accurate 3D model of the environment, including buildings, curbs, road signs, traffic lights, and any stable fixtures in the environment. By using the vehicle's GPS data and / or image-based localization techniques (e.g., simultaneous localization and mapping, or SLAM), the perception module 1010 can determine the vehicle's pose (e.g., position and orientation) or the pose of the vehicle's sensors within the high-resolution map. The posture information can then be used by the perception module 1010 to query the high-definition map and determine what objects are expected to be in the environment.

[0098] The perception module 1010 can use sensor data from one or more types of sensors and / or information derived therefrom to generate a representation of the vehicle's context. As an example, and not a limitation, the representation of the external environment can include objects such as other vehicles, curbs, debris, objects, and pedestrians. The context representation can be limited to the maximum range of the sensor array (e.g., 50, 1000, or 200 meters). The context representation can include information about objects and things around the vehicle, as well as semantic information about traffic lanes, traffic rules, traffic signs, time of day, weather, and / or any other suitable information. The context can be represented in any suitable manner. As an example, and not a limitation, the context representation can be encoded as a vector or matrix of numerical values, where each value in the vector / matrix corresponds to a predetermined category of information. For example, each object in the environment can be represented by a series of values, starting with the object's coordinates, classification (e.g., vehicle, pedestrian, etc.), orientation, speed, trajectory, etc. Alternatively, information about the context can be represented by a raster image (which visually depicts objects, semantic information, etc.). For example, a raster image can be a bird's-eye view of the vehicle and its surroundings up to a predetermined distance. Raster images can include visual information (e.g., bounding boxes, color-coded shapes, etc.) that represent various types of data of interest (e.g., vehicles, pedestrians, lanes, buildings, etc.).

[0099] A representation of the current context from perception module 1010 can be consumed by prediction module 1015 to generate one or more predictions about the future environment. For example, given a representation of the context at time t0, prediction module 1015 can output another context representation for time t1. For example, if the context at t0 is represented by a raster image, the output of prediction module 1015 could be another raster image (e.g., a snapshot of the current environment) depicting where the object will be at time t1 (e.g., a future snapshot). In a particular embodiment, prediction module 1015 may include a machine learning model (e.g., a convolutional neural network, a neural network, a decision tree, a support vector machine, etc.) that can be trained based on previously recorded context and sensor data. For example, a training sample can be generated based on a series of actual sensor data captured by the vehicle at times t0 and t1. The data captured at times t0 and t1 can be used to generate a first context representation (training data) and a second context representation (associated ground reality for training), respectively. During training, the machine learning model can process the first context representation using the model's current configuration parameters and output a predicted context representation. The predicted context representation can then be compared to a known second context representation (i.e., the ground reality at time t1). This comparison can be quantified by a loss value calculated using a loss function. The loss value can be used (e.g., via backpropagation) to update the configuration parameters of the machine learning model so that the loss is smaller if the prediction is to be made again. The machine learning model can be trained iteratively using a large set of training samples until a convergence or termination condition is met. For example, training can terminate when the loss value falls below a predetermined threshold. Once trained, the machine learning model can be used to generate predictions of future context representations based on the current context representation.

[0100] The planning module 1020 can determine the vehicle's navigation route and specific driving actions (e.g., deceleration, acceleration, stopping, turning, etc.) based on the predicted context representation generated by the prediction module 1015. In a particular embodiment, the planning module 1020 can utilize predicted information encoded within the predicted context representation (e.g., predicted location or trajectory of objects, semantic data, etc.) and any other available information (e.g., map data, traffic data, accident reports, weather reports, target destinations, and any other suitable information) to determine one or more targets or navigation instructions for the vehicle. By way of example and not limitation, based on the predicted behavior of objects around the vehicle and traffic data to a specific destination, the planning module 1020 can determine a specific navigation path for the vehicle and associated driving actions to avoid potential collisions with one or more objects.

[0101] In a particular embodiment, the planning module 1020 may generate several different plans (e.g., destination or navigation instructions) for the vehicle based on a given predictive context representation. For each plan, the planning module 1020 may calculate a score representing the desirability of that plan. For example, if a plan would likely result in a collision between the vehicle and an object at the predicted location (as determined based on the predictive context representation), that plan may be penalized accordingly. Another plan that would result in the vehicle violating traffic rules or taking a long detour to avoid a potential collision might also have a penalized score, but the penalty might be less severe than that applied to a previous plan that would result in a collision. A third plan that would simply cause the vehicle to stop or change lanes to avoid a collision with an object in the predicted future might receive the highest score. Based on the scores assigned to the plans, the planning module 1020 may select the optimal plan to implement. Although the above examples use collisions as examples, the disclosure herein contemplates the use of any suitable scoring criteria, such as distance or time traveled, fuel economy, variation in estimated time of arrival at the destination, passenger comfort, proximity to other vehicles, confidence scores associated with the predictive context representation, etc.

[0102] Based on the plan generated by the planning module 1020 (which may include one or more navigation paths or associated driving operations), the control module 1025 can determine the specific commands to be issued to the vehicle's actuators. The vehicle's actuators are components responsible for moving and controlling the vehicle. Actuators control the vehicle's driving functions, such as steering, turning signals, deceleration (braking), acceleration, gear shifting, etc. As an example, and not a limitation, the control module 1025 can send commands to the steering actuator to maintain a specific steering angle for a specific amount of time, thereby moving the vehicle along a specific trajectory to avoid objects predicted to intrude into the vehicle's surface. As another example, the control module 1025 can send commands to the accelerator actuator to enable the vehicle to safely avoid objects predicted to intrude into the vehicle's surface.

[0103] Figure 11 An example computer system 1100 is illustrated. In a particular embodiment, one or more computer systems 1100 perform one or more steps of one or more methods described or illustrated herein. In a particular embodiment, one or more computer systems 1100 provide the functionality described or illustrated herein. In a particular embodiment, software running on one or more computer systems 1100 performs one or more steps of one or more methods described or illustrated herein, or provides the functionality described or illustrated herein. Specific embodiments include one or more portions of one or more computer systems 1100. Throughout this document, references to computer systems may cover computing devices and vice versa, where appropriate. Furthermore, references to computer systems may cover one or more computer systems, where appropriate.

[0104] This disclosure contemplates any suitable number of computer systems 1100. This disclosure contemplates computer systems 1100 in any suitable physical form. By way of example and not limitation, computer system 1100 may be an embedded computer system, a system-on-a-chip (SOC), a single-board computer system (SBC) (such as a computer-level module (COM) or system-level module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a computer system grid, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, computer system 1100 may include one or more computer systems 1100; may be single or distributed; may span multiple locations; may span multiple machines; may span multiple data centers; or may reside in a cloud that may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. By way of example and not limitation, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein in real time or in batch mode. Where appropriate, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein at different times or in different locations.

[0105] In a particular embodiment, computer system 1100 includes a processor 1102, a memory 1104, a storage device 1106, an input / output (I / O) interface 1108, a communication interface 1110, and a bus 1112. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0106] In a particular embodiment, processor 1102 includes hardware for executing instructions, such as those constituting a computer program. By way of example, and not limitation, to execute instructions, processor 1102 may retrieve (or fetch) instructions from internal registers, internal caches, memory 1104, or storage device 1106; decode and execute them; and then write one or more results to internal registers, internal caches, memory 1104, or storage device 1106. In a particular embodiment, processor 1102 may include one or more internal caches for data, instructions, or addresses. Where appropriate, this disclosure contemplates that processor 1102 may include any suitable number of suitable internal caches. By way of example, and not limitation, processor 1102 may include one or more instruction caches, one or more data caches, and one or more translation back buffers (TLBs). Instructions in the instruction cache may be copies of instructions in memory 1104 or storage device 1106, and the instruction cache may accelerate the retrieval of those instructions by processor 1102. The data in the data cache may be a copy of data in memory 1104 or storage device 1106 to be acted upon by computer instructions; the result of previous instructions executed by processor 1102, which may be accessed by subsequent instructions or used to write to memory 1104 or storage device 1106; or any other suitable data. The data cache can accelerate read or write operations of processor 1102. The TLB can accelerate virtual address translation of processor 1102. In a particular embodiment, processor 1102 may include one or more internal registers for data, instructions, or addresses. Where appropriate, this disclosure contemplates that processor 1102 may include any suitable number of suitable internal registers. Where appropriate, processor 1102 may include one or more arithmetic logic units (ALUs), is a multi-core processor, or include one or more processors 1102. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0107] In a particular embodiment, memory 1104 includes main memory for storing instructions to be executed by processor 1102 or data to be acted upon by processor 1102. By way of example and not limitation, computer system 1100 may load instructions from storage device 1106 or another source (such as another computer system 1100) into memory 1104. Processor 1102 may then load instructions from memory 1104 into internal registers or internal caches. To execute instructions, processor 1102 may retrieve instructions from internal registers or internal caches and decode them. During or after instruction execution, processor 1102 may write one or more results (which may be intermediate or final results) to internal registers or internal caches. Processor 1102 may then write one or more of those results to memory 1104. In a particular embodiment, processor 1102 executes only the instructions in one or more internal registers or internal caches or memory 1104 (as opposed to storage device 1106 or elsewhere) and acts only on the data in one or more internal registers or internal caches or memory 1104 (as opposed to storage device 1106 or elsewhere). One or more memory buses (each of which may include an address bus and a data bus) couple processor 1102 to memory 1104. Bus 1112 may include one or more memory buses, as described in further detail below. In a particular embodiment, one or more memory management units (MMUs) reside between processor 1102 and memory 1104 and facilitate access to memory 1104 requested by processor 1102. In a particular embodiment, memory 1104 includes random access memory (RAM). Where appropriate, this RAM may be volatile memory. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Furthermore, where appropriate, the RAM may be single-port or multi-port RAM. This disclosure contemplates any suitable RAM. Where appropriate, memory 1104 may include one or more memories 1104. Although this disclosure describes and illustrates specific memories, this disclosure contemplates any suitable memory.

[0108] In a particular embodiment, memory 1106 includes mass storage for data or instructions. By way of example and not limitation, memory 1106 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk drive, magneto-optical disk drive, magnetic tape drive, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, storage device 1106 may include removable or non-removable (or fixed) media. Where appropriate, memory 1106 may be internal or external to computer system 1100. In a particular embodiment, memory 1106 is non-volatile solid-state memory. In a particular embodiment, memory 1106 includes read-only memory (ROM). Where appropriate, this ROM may be a mask-programmable ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically changeable ROM (EAROM), or flash memory, or a combination of two or more of these. This disclosure contemplates mass storage device 1106 in any suitable physical form. Where appropriate, memory 1106 may include one or more storage control units that facilitate communication between processor 1102 and storage device 1106. Where appropriate, storage device 1106 may include one or more storage devices 1106. Although this disclosure describes and illustrates specific storage, any suitable storage is contemplated.

[0109] In a particular embodiment, I / O interface 1108 includes hardware, software, or both, providing one or more interfaces for communication between computer system 1100 and one or more I / O devices. Where appropriate, computer system 1100 may include one or more of these I / O devices. One or more of these I / O devices can enable communication between a person and computer system 1100. By way of example and not limitation, I / O devices may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet computer, touchscreen, trackball, camera, other suitable I / O devices, or combinations of two or more of these. I / O devices may include one or more sensors. This disclosure contemplates any suitable I / O device and any suitable I / O interface 1108 for them. Where appropriate, I / O interface 1108 may include one or more device or software drivers that enable processor 1102 to drive one or more of these I / O devices. Where appropriate, I / O interface 1108 may include one or more I / O interfaces 1108. Although this disclosure describes and illustrates specific I / O interfaces, this disclosure contemplates any suitable I / O interface.

[0110] In a particular embodiment, the communication interface 1110 includes hardware, software, or both, providing one or more interfaces for communication (such as, for example, packet-based communication) between the computer system 1100 and one or more other computer systems 1100 or one or more networks. By way of example and not limitation, the communication interface 1110 may include a network interface controller (NIC) or network adapter for communicating with Ethernet or any other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network such as a Wi-Fi network. This disclosure contemplates any suitable network and any suitable communication interface 1110 for use therewith. By way of example and not limitation, the computer system 1100 may be connected to one or more portions of an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 1100 may be connected to a wireless PAN (WPAN) (such as, for example, Bluetooth WPAN), a Wi-Fi network, a Wi-Fi Max network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or any other suitable wireless network, or a combination of two or more of these. Where appropriate, computer system 1100 may include a suitable communication interface 1110 for any of these networks. Where appropriate, communication interface 1110 may include one or more communication interfaces 1110. Although specific communication interfaces are described and illustrated in this disclosure, any suitable communication interface is contemplated.

[0111] In a particular embodiment, bus 1112 includes hardware, software, or both, that couple components of computer system 1100 to each other. By way of example and not limitation, bus 1112 may include an Accelerated Graphics Port (AGP) or any other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1112 may include one or more buses 1112. Although this disclosure describes and illustrates specific buses, this disclosure contemplates any suitable bus or interconnect.

[0112] In this document, where appropriate, one or more computer-readable non-transitory storage media may include one or more semiconductor-based or other types of integrated circuits (ICs) such as, for example, field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tape, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, computer-readable non-transitory storage media may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0113] In this document, "or" is inclusive rather than exclusive, unless otherwise expressly stated or the context otherwise requires. Therefore, in this document, "A or B" means "A, B, or both," unless otherwise expressly stated or the context otherwise requires. Furthermore, "and" is both common and separate, unless otherwise expressly stated or the context otherwise requires. Therefore, in this document, "A and B" means "A and B, commonly or separately," unless otherwise expressly stated or the context otherwise requires.

[0114] The scope of this disclosure covers all changes, substitutions, variations, alterations, and modifications to the exemplary embodiments described or illustrated herein that will be understood by those skilled in the art. The scope of this disclosure is not limited to the exemplary embodiments described or illustrated herein. Furthermore, although this disclosure describes and illustrates corresponding embodiments herein as including specific components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or arrangement of any components, elements, features, functions, operations, or steps described or illustrated anywhere herein that will be understood by those skilled in the art. Furthermore, references in the appended claims to an apparatus or system or a component of an apparatus or system that is suitable for, arranged as, capable of, configured as, able to, operable, or operated to perform a particular function cover that device, system, or component, whether or not it or that particular function is activated, turned on, or unlocked, provided that the device, system, or component is so suitable for, arranged as, capable of, configured as, able to, operable, or operated. Furthermore, although this disclosure describes or illustrates specific embodiments to provide particular advantages, specific embodiments may not provide, provide some, or provide all of these advantages.

Claims

1. A method associated with a vehicle, the method comprising performing the following operations via a computing system associated with the vehicle: Determine the current position of the vehicle in the first area; Identify one or more first set of model parameters associated with the first region and one or more second set of model parameters associated with the second region; Using one or more machine learning models with configurations based on corresponding first set of model parameters, one or more corresponding first inferences are generated based on first sensor data captured by the vehicle; Switch the configuration of one or more models from the corresponding first set of model parameters to the corresponding second set of model parameters associated with the second region; Using the one or more models with configurations based on the corresponding second set of model parameters, one or more corresponding second inferences are generated based on second sensor data generated by the vehicle's sensors when the vehicle is in the second region; The vehicle is made to perform one or more operations based at least on a second inference; as well as When the second set of model parameters indicates that the performance of the machine learning model is below a minimum threshold, the second region is divided into at least two sub-regions with new parameters.

2. The method as described in claim 1, wherein, The first set of model parameters and the second set of model parameters were generated by training the corresponding models based on previously captured sensor data collected in the first region and the second region, respectively.

3. The method as described in claim 1, wherein, Identifying the first set of model parameters and the second set of model parameters includes retrieving the first set of model parameters and the second set of model parameters from a data repository, wherein the first set of model parameters and the second set of model parameters are respectively associated with a first region and a second region in the data repository.

4. The method of claim 2, wherein, The model includes one or more perception models trained to identify one or more types of objects in a first region or a second region based on first sensor data or second sensor data, respectively.

5. The method of claim 4, wherein, The perception model includes multiple perception models trained based on multiple lighting conditions, multiple weather conditions, multiple times of day, or combinations thereof.

6. The method of claim 5, wherein, Determining the first set of model parameters and the second set of model parameters also includes: identifying in a data repository the first set of model parameters and the second set of model parameters associated with the current lighting conditions in the vehicle's environment, the weather conditions in the vehicle's environment, the time of day, or a combination thereof.

7. The method of claim 6, wherein, The lighting conditions, weather conditions, time of day, or combinations thereof associated with the first set of model parameters and the second set of model parameters are determined when the first set of model parameters and the second set of model parameters are generated by training the corresponding perception models.

8. The method of claim 2, wherein, The model includes one or more prediction models, which are trained to predict one or more trajectories of one or more objects represented in first sensor data or second sensor data.

9. The method of claim 8, wherein, A prediction model with a configuration based on a corresponding first set of model parameters associated with a first region is trained to predict trajectories based on observed trajectories of objects previously captured in the first region.

10. The method of claim 2, wherein, The model includes one or more planning models, which are trained to identify one or more vehicle trajectories for manipulating the environment through a first area or a second area, based on first sensor data or second sensor data, respectively.

11. The method of claim 10, wherein, The one or more vehicle trajectories include multiple vehicle trajectories that result in the execution of multiple different corresponding maneuvers.

12. The method of claim 1, wherein, The first region is separated from the second region by a region boundary, and the switching of the configuration of the one or more models from the corresponding first set of model parameters to the corresponding second set of model parameters is performed in response to the vehicle crossing the region boundary or being located at a distance less than a first threshold from the region boundary. The method further includes: In response to determining, based on the vehicle's current position and current direction of travel, that the vehicle is less than a second threshold distance from the boundary of the region, wherein the second threshold distance is greater than a first threshold distance: The second set of model parameters is loaded from the data repository into the storage device on the vehicle.

13. The method of claim 1, wherein, The first set of model parameters and the second set of model parameters are loaded into the memory device on the vehicle for a first region and a second region of a predetermined group, wherein the predetermined group is determined based on the route planning of the vehicle.

14. The method of claim 1, wherein, Each of the first and second regions is designated as a geographic area, road segment, intersection, or a combination thereof.

15. The method of claim 1, further comprising: Identify the regions in the map for which the performance of a general machine learning model should be evaluated; Using the general machine learning model configured according to a set of general model parameters associated with the map, the evaluated performance of the general machine learning model for the region of the map is determined; The general machine learning model is determined to have an evaluated performance below a minimum performance threshold for the region of the map. Generate a localized machine learning model with a set of localized model parameters; as well as The localized machine learning model is associated with the region of the map to generate one or more subsequent inferences in the region of the map.

16. The method of claim 1, further comprising: Identify the regions in the map for which the performance of the first localized machine learning model will be evaluated; Using a first localized machine learning model configured according to a first set of localized model parameters associated with the region, the evaluated performance of the first localized machine learning model for the region of the map is determined; The evaluated performance of the first localized machine learning model for the region of the map is determined to be below a minimum performance threshold; The area of ​​the map is divided into two or more sub-areas; Generate a second localized machine learning model with a second set of localized model parameters; as well as A second localized machine learning model is associated with one of the sub-regions to generate one or more subsequent inferences in that sub-region.

17. A system associated with a vehicle, comprising: One or more processors and one or more computer-readable non-transitory storage media coupled to the one or more processors, the one or more computer-readable non-transitory storage media including operable instructions that, when executed by the one or more processors, cause the system to: Determine the vehicle's current location in the first area; Identify one or more first set of model parameters associated with the first region and one or more second set of model parameters associated with the second region; Using one or more machine learning models with configurations based on corresponding first set of model parameters, one or more corresponding first inferences are generated based on first sensor data captured by the vehicle; Switch the configuration of one or more models from the corresponding first set of model parameters to the corresponding second set of model parameters associated with the second region; Using the one or more models with configurations based on the corresponding second set of model parameters, one or more corresponding second inferences are generated based on second sensor data generated by the vehicle's sensors when the vehicle is in the second region; The vehicle is made to perform one or more operations based at least on a second inference; as well as When the second set of model parameters indicates that the performance of the machine learning model is below a minimum threshold, the second region is divided into at least two sub-regions with new parameters.

18. The system of claim 17, wherein, The first set of model parameters and the second set of model parameters were generated by training the corresponding models based on previously captured sensor data collected in the first region and the second region, respectively.

19. One or more computer-readable non-transitory storage media, including operable software that, when executed, causes one or more processors to perform operations including: Determine the vehicle's current location in the first area; Identify one or more first set of model parameters associated with the first region and one or more second set of model parameters associated with the second region; Using one or more machine learning models with configurations based on corresponding first set of model parameters, one or more corresponding first inferences are generated based on first sensor data captured by the vehicle; Switch the configuration of one or more models from the corresponding first set of model parameters to the corresponding second set of model parameters associated with the second region; Using the one or more models with configurations based on the corresponding second set of model parameters, one or more corresponding second inferences are generated based on second sensor data generated by the vehicle's sensors when the vehicle is in the second region; The vehicle is made to perform one or more operations based at least on a second inference; as well as When the second set of model parameters indicates that the performance of the machine learning model is below a minimum threshold, the second region is divided into at least two sub-regions with new parameters.

20. The storage medium of claim 19, wherein, The first set of model parameters and the second set of model parameters were generated by training the corresponding models based on previously captured sensor data collected in the first region and the second region, respectively.

Images