Modifying map annotations using node updates in a sparse pose graph

Abstract

Repositioning and / or resizing map elements using the updated pose graph without introducing anomalies into the map data may include determining a transformation between a source node of the first pose graph and a target node of the second pose graph, and determining modifications to the map elements based at least in part on the transformation. The technique may include determining a stress on the map based at least in part on one or more modifications to the map elements, and determining whether the stress meets or exceeds a threshold. If the stress meets or exceeds a threshold, the modifications may be changed, canceled, and / or indicated in a notification sent to a user interface.

Description

[Technical Field]

[0001] The present invention relates to modifying map annotations using node updates of a sparse pose graph. [Background technology]

[0002] [Related Applications] This PCT international application claims the benefit of priority to U.S. Patent Application No. 17 / 306,499, filed May 3, 2021, the entire contents of which are incorporated herein by reference for all purposes.

[0003] [background] Autonomous vehicles can navigate an environment based at least in part on a map of the environment. Such autonomous vehicles may include sensors, such as lidar sensors, that can be used to acquire data representative of the environment, which is then used in a map-building process to generate a map or update an existing map. However, errors in the map or map-building process can create unsafe situations for such systems that rely on accurate maps, such as in the case of autonomous vehicles. Furthermore, such map update and correction processes can be computationally intensive and / or require extensive manual input. [Brief explanation of the drawings]

[0004] The detailed description is provided with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. The same reference number in different figures indicates similar or identical items.

[0005] [Figure 1]1 illustrates an example scenario in which an autonomous vehicle may receive sensor data associated with an environment that may be used to generate a pose graph that estimates the vehicle's position, orientation, and / or trajectory. The pose graph and sensor data may be used to generate a map of the environment that includes annotations (map elements) that identify key representations of objects / areas within the map. [Figure 2A] FIG. 10 is a schematic flow diagram illustrating an exemplary process for adjusting map elements using at least two sparse pose graphs. [Figure 2B] FIG. 10 is a schematic flow diagram illustrating an exemplary process for adjusting map elements using at least two sparse pose graphs. [Figure 3] 10A-10C are diagrams illustrating various transformations of map elements from simple to more complex examples using two pose graphs. [Figure 4] 10A-10C are diagrams illustrating various transformations of map elements from simple to more complex examples using two pose graphs. [Figure 5] 10A-10C are diagrams illustrating various transformations of map elements from simple to more complex examples using two pose graphs. [Figure 6] FIG. 2C illustrates an exemplary stress / strain map that may be generated as part of FIG. 2B. [Figure 7] FIG. 1 is a block diagram illustrating an example system including map creation components that may generate a pose graph, map elements, and / or map data associated with an environment in accordance with the techniques described herein. DETAILED DESCRIPTION OF THE INVENTION

[0006] As described above, sensor data may be collected to generate a map of an environment, but adjusting map annotations that indicate the location and / or extent (size / scale) of features in the environment can be labor-intensive, error-prone, and computationally expensive. For example, one error introduced by adjusting map elements may include map "tearing" or other anomalies, where automated techniques for correcting map elements may introduce inconsistencies or "blank" spaces in the map. Techniques described herein update map elements throughout map data representing an environment, reducing errors, the need for manual input, and computational load, including reducing map tears. In some examples, map elements may be updated as new sensor data is received from a vehicle. This new sensor data may be used to update and / or modify a previously generated pose graph by using the new sensor data in re-optimizing the pose graph. In additional or alternative examples, updates to the pose graph may be determined based at least in part on the next iteration of an optimization process applied to the pose graph and / or modifications to the optimization process (e.g., changes to parameters of the optimization, such as changes to the error function or constraints). More specifically, an autonomous vehicle may transmit sensor data to a remote computing device, which may use the sensor data to generate a map of the environment. Such a map may include, for example, a two-dimensional representation identifying locations and / or regions associated with semantic labels. According to more basic techniques, a human may manually label the locations of objects of interest in the environment, such as traffic signs, vehicle lanes, bicycle lanes, crosswalks, stop lines, yield lines, and / or the like. According to techniques described herein, a perception component of the vehicle may detect object classifications associated with the sensor data and generate labels associated with the sensor data.The labels may be used to annotate map data as map elements indicating the presence of traffic signs, passenger drop-off or pick-up zones, lane markings, blind spots, intersections or other road forks, drivable surfaces, stop lines, yield lines, and / or the like.

[0007] The remote computing device may transmit such a representation of the environment, including map elements / annotations (and updated map elements), to the autonomous vehicle, which may use it to safely and efficiently navigate the environment. For example, the autonomous vehicle may use the map to determine where the vehicle is in the environment, where drivable surface areas are, where static objects are in the environment, where drop-off locations exist, where landmarks are, etc., and may use such determinations to determine a trajectory for controlling the movement of the autonomous vehicle. Additionally or alternatively, the autonomous vehicle may use the map data to correctly position the environment for dynamic objects detected by the autonomous vehicle relative to key points, landmarks, static objects, etc. therein using sensor data received by the autonomous vehicle. In some examples, the vehicle may use the map data to determine the likelihood that an object is hidden and / or increase or decrease the confidence that an object is present at a particular location (e.g., a dynamic object such as a pedestrian is unlikely to be in the same location as a static object such as a stop sign).

[0008] In some examples, a computing device may have determined a first pose graph (a graphical representation of multiple poses as nodes and associations, e.g., constraints, between such poses as edges in the graph) by aligning a first set of sensor data captured at various vehicle poses. This aligned sensor data is used to generate a three-dimensional model of the environment. Generating the first pose graph may include estimating a first pose (i.e., position and / or orientation) of the vehicle associated with the first sensor data set and the relationship of the first pose to other poses of the vehicle. For example, the vehicle may receive lidar data from one or more lidar sensors associated with the vehicle. This lidar data may include a set of data points, also referred to herein as a point cloud, each set of data points corresponding to a particular pose of the vehicle in the environment. In various examples, such a graph may represent connections between such poses (e.g., between subsequent poses along a trajectory) and / or links between multiple poses in one or more trajectories that have at least some co-visible data points.

[0009] At some point, an additional sensor data set may be received from the same vehicle or a different vehicle. This second sensor data set may be associated with the same portion of the environment and may be used to update and / or modify the first pose graph. In some examples, the second pose graph may be determined based at least in part on the second set of sensor data. The problem is that map elements are not so easily updated and do not automatically update along with the underlying pose graph.

[0010] For example, changes between the first pose graph and the second pose graph may change the three-dimensional representation of the environment and / or the underlying assumptions for determining where the vehicle is within the environment. As a result, certain map elements, such as an indication that a portion of the map data is associated with a drivable surface, may not be displayed correctly in the updated version of the map data. In the worst possible scenario, this may result in the map data indicating that a portion of the environment is a drivable surface when it is not, which may result in damage and / or injury to the vehicle.

[0011] The techniques described herein may modify a map element based at least in part on determining one or more nodes in a first pose graph that are closest in distance to the map element. A node in the first pose graph is referred to herein as a “source node,” and a node in the new pose graph is referred to herein as a “target node.” The map creation component may determine a target node associated with the source node. In a first example, the source node and target node may be known because the pose graph has been updated according to a new optimization (or at least an iteration that is part of the optimization). In such an example, the pose graph is not a new pose graph, but the same pose graph modified. The nodes (and factors) of the pose may have unique identifiers associated with them, allowing the nodes of the updated pose to be easily identified. In an alternative example, the pose graph may be a new pose graph, in which case determining a target node associated with the source node may include determining a node in a second pose graph that is closest (in distance) and / or most similar (in direction of travel) to the source node.

[0012] In some examples, the map generation component may determine one or more source-target node pairs where the source node of each pair is within a threshold distance of the map element. In additional or alternative examples, the nearest n source-target node pairs may be used to determine the source-target node pairs, where n is a positive integer (e.g., regardless of distance). In additional or alternative examples, a Gaussian process, such as Kriging, may use all source nodes in the map to determine modifications to the map element. Note that source-target node pairs are also referred to as node pairs herein.

[0013] In some examples, the map generation component may determine that there are fewer than n node pairs within a threshold distance. If the number of node pairs is less than the threshold, the map generation component may generate and / or send a notification to a user computing device or vehicle (e.g., indicating that an error has occurred and / or there is some risk that the map element may not be accurately transformed), and / or the map generation component may retain the position of the map element from the last version of the map, or the map element may be modified based at least in part on modifications of the nearest m different map elements, where m is a positive integer (e.g., 1, 2, 5, or more, which may depend on the proximity of the nearest map elements, the density of the nearest map elements, and / or the maximum distance from the map element that another map element may be used).

[0014] Upon determining the source-target node pair, the map creation component may determine a transformation between the source node and the target node. For example, the map creation component may determine the difference between the source node and the target node and / or a function that will move the vehicle from the source node to the target node. This transformation may include, for example, a vector indicating how the source node differs from the target node. In some examples, the map creation component may iterate to determine the transformation for the nearest source-target node pair to the map element until a number of transformations equal to the number of pairs have been determined.

[0015] In some examples, the map generation component may determine weights associated with source-target node pairs based at least in part on the distance between the source node of the pair and the location of the map element. In some examples, the weight may decrease as the distance increases. Various loss functions may be used to determine the weights, and in some examples, may determine a weight for each source-target node pair in the nearest source-target node pairs to the map element. In some examples, the weights associated with these nearest source-target node pairs may be normalized to sum to one.

[0016] The map creation component may determine a modification to apply to a map element based at least in part on the transformations and weights associated with the source-target node pairs. For example, the modification itself may be a transformation that alters the position and / or extent of the map element. In some examples, the map creation component may determine a modification based at least in part on a weighted sum of transformations (e.g., of the nearest n node pairs, node pairs with a threshold distance, or a combination thereof). In additional or alternative examples, the modification to a map element may be part of an optimization of multiple map elements that include the map element. For example, the modification to the map element may be made at least in part based on modifying the map element (and one or more other map elements) to reduce the difference between the modification to the map element and the weighted sum of the transformations associated with the map element, as determined above. With respect to other map elements, each modification may be made at least in part based on determining a total difference by summing these differences across different map elements (and their respective weighted transformations), and each modification may be made to reduce that total difference.

[0017] The resulting (changed) map elements may, however, be updated map elements included in second (updated) map data. This updated map data may be transmitted to the vehicle (or another vehicle) for use by the vehicle to control operation of the vehicle. The techniques described herein may require fewer computational resources, may improve the accuracy of the map element locations and the resulting map data, and may improve the safety of vehicles using the map data by correctly positioning map elements, such as drivable surfaces (e.g., roadways vs. sidewalks), stop signs and / or stop lines, etc.

[0018] [Example scenario] FIG. 1 illustrates an example scenario 100 including a vehicle 102. In some examples, the vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which defines a vehicle in which a driver (or passenger) is not expected to control the vehicle at any time and is capable of performing all safety-critical functions throughout the entire journey. However, in other examples, the vehicle 102 may be a fully or partially autonomous vehicle having any other level or classification. It is contemplated that the techniques described herein may be applied to more than just robotic control, such as for autonomous vehicles. For example, the techniques described herein may be applied to mining, manufacturing, augmented reality, etc. Additionally, although the vehicle 102 is depicted as a land vehicle, the vehicle 102 may also be a spacecraft, a surface vessel, and / or the like.

[0019] According to the techniques described herein, the vehicle 102 may receive sensor data from sensors 104 of the vehicle 102. For example, the sensors 104 may include: position sensors (e.g., global positioning system (GPS) sensors, etc.), inertial sensors (e.g., acceleration sensors, gyroscope sensors, etc.), magnetic field sensors (e.g., compasses, etc.), position / speed / acceleration sensors (e.g., speedometers, drive system sensors, etc.), odometry data (which may be determined at least in part based on inertial measurements and / or odometers of the vehicle 102), depth position sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight (ToF) cameras, depth cameras, ultrasonic and / or sonar sensors, and / or other depth-sensing sensors, etc.), image sensors (e.g., cameras, etc.), audio sensors (e.g., microphones, etc.), and / or environmental sensors (e.g., barometers, hygrometers, etc.).

[0020] The sensors 104 may generate sensor data that may be received by a computing device 106 associated with the vehicle 102. However, in other examples, some or all of the sensors 104 and / or computing devices 106 may be separate from and / or located remotely from the vehicle 102, and data capture, processing, command, and / or control may be communicated to or from the vehicle 102 by one or more remote computing devices via wired and / or wireless networks.

[0021] Computing device 106 may include memory 108 that stores recognition component 110, planning component 112, map creation component 114, and / or system controller 116. While depicted in FIG. 1 for purposes of illustration, it is understood that map creation component 114 may reside on a separate computing device (or otherwise) from any one or more of the other components.

[0022] In general, perception component 110 may determine what is in the environment surrounding vehicle 102, and planning component 112 may determine how to maneuver vehicle 102 according to the information received from perception component 110. For example, planning component 112 may determine trajectory 118 based at least in part on perception data and / or other information, such as, for example, one or more maps, such as map data determined according to the techniques described herein, localization information (e.g., where vehicle 102 is in the environment relative to the map and / or features detected by perception component 110), and / or the like. In some examples, perception component 110 may include a hardware and / or software pipeline, which may include one or more GPUs, ML models, Kalman filters, and / or the like.

[0023] For example, the map data may include map elements that indicate various important objects and / or areas in the environment, such as the location (and / or extent, i.e., shape / size) of important objects or areas in the environment. These map elements may include annotations that indicate portions of the map data, such as two-dimensional representations of the environment associated with, for example, drivable surfaces, pedestrian-conditional surfaces such as crosswalks, permanent pedestrian or bicycle surfaces, parking surfaces, traffic signs (e.g., stop lights, stop signs, yield signs, speed signs, commuter lanes, construction signs, etc.), traffic markings (e.g., stop lines, yield lines, crosswalk markings, commuter lane markings, traffic cones, flares, etc.), various passenger drop-off or pick-up zones, etc. In other words, the map indicates the importance of portions of the map data regardless of whether the map data is two-dimensional or three-dimensional. Additional map data may be generated that may include three-dimensional representations of the environment, such as meshes, wireframe models, polygons, surfels (e.g., polygons associated with particular colors and / or intensities, etc.), and / or the like.

[0024] In some examples, trajectory 118 may depend on one or more map elements identified in the two-dimensional map. For example, the trajectory generated for the vehicle may be based at least in part on traffic signs and may be bounded by portions of the map data indicated (by map elements) as being drivable surfaces, except for emergency situations or pick-up / drop-off / parking scenarios, which may generate a trajectory that would result in the vehicle moving into portions of the environment indicated in the map data as being associated with, for example, bicycle lanes, parking areas, passenger pick-up / drop-off areas, and / or the like.

[0025] Trajectory 118 may include instructions to controller 116 to operate drive components of vehicle 102 to cause steering angles, steering rates, accelerations, and / or the like, which may result in vehicle position, vehicle velocity, and / or vehicle acceleration. For example, trajectory 118 may include a target heading, target steering angle, target steering rate, target position, target velocity, and / or target acceleration for controller 116 to track. In some examples, trajectory 118 may be associated with controls sufficient to control vehicle 102 over a time horizon (e.g., 5 milliseconds, 10 milliseconds, 100 milliseconds, 200 milliseconds, 0.55 seconds, 1 second, 2 seconds, etc.) or a distance horizon (e.g., 1 meter, 2 meters, 5 meters, 8 meters, 10 meters, etc.). In some examples, controller 116 may include software and / or hardware for operating drive components of vehicle 102 sufficient to track trajectory 118. For example, the controller 116 may include one or more proportional-integral-derivative (PID) controllers.

[0026] In some examples, the recognition component 110 may receive sensor data from the sensors 104 and may determine data related to objects in the vicinity of the vehicle 102 (e.g., classifications, instance segmentation, semantic segmentation, two-dimensional and / or three-dimensional bounding boxes, tracks, etc. associated with detected objects), path data identifying the vehicle's destination, global map data identifying road features (e.g., features detectable in various sensor modalities useful for localizing an autonomous vehicle), the vehicle's pose (e.g., position and / or orientation within the environment, which may be determined by or in conjunction with the localization component), local map data identifying features detected in proximity to the vehicle (e.g., position and / or size of buildings, trees, fences, fire hydrants, stop signs, and any other features detectable in various sensor modalities), etc.

[0027] In particular, the recognition component 110 may determine, based at least in part on the sensor data, an object detection indicating an association of a portion of the sensor data to an object in the environment. The object detection may indicate an object classification, a region of interest (ROI) that identifies a portion of the sensor data associated with the object, and / or a likelihood (e.g., posterior probability, etc.) that the object classification is correct, and / or a confidence score that the ROI is accurate. For example, the ROI may include a portion of the image or radar data identified by an ML model or ML pipeline of the recognition component 110 as being associated with the object, such as using a bounding box, a mask, an instance segmentation, and / or a semantic segmentation. The object classification determined by the recognition component 110 may distinguish between various object types, such as, for example, a passenger vehicle, a pedestrian, a bicyclist, a delivery truck, a truck trailer, a traffic sign, and / or the like. In some examples, the object detection may be tracked over time. For example, tracking may associate two object detections generated at two different times as being associated with the same object and may include the object's historical, current, and / or predicted position, velocity, acceleration, and / or orientation.

[0028] The data produced by the recognition component 110 may be collectively referred to as recognition data. As the recognition component 110 generates recognition data, the recognition component 110 may provide the recognition data to the planning component 112. The recognition data may additionally or alternatively be stored in association with the sensor data as logged data. This logged data may be transmitted to the map generation component 114, which may be executing on the computing device 106 and / or on a remote computing device (not shown in FIG. 1 for clarity).

[0029] The planning component 112 may use the recognition data received from the recognition component 110 to determine one or more trajectories, control operation of the vehicle 102 to travel along a route or path, and / or otherwise control operation of the vehicle 102, although any such action may be performed by various other components (e.g., localization may be performed by a localization component, which may be based at least in part on the recognition data). For example, the planning component 112 may: determine a path for the vehicle 102 from a first location to a second location; substantially simultaneously, and based at least in part on the recognition data and / or simulated recognition data (which may further include predictions regarding objects detected in such data), generate multiple potential trajectories (e.g., 1 microsecond, 0.5 seconds, etc.) for controlling movement of the vehicle 102 according to a receding horizon technique, and control the vehicle to travel along the path (e.g., to avoid any of the detected objects). and selecting one of the potential trajectories as a trajectory 118 for the vehicle 102 that can be used to generate drive control signals that can be sent to drive components of the vehicle 102. While FIG. 1 depicts one example of such a trajectory 118, represented as arrows indicating heading, velocity, and / or acceleration, the trajectory itself can include instructions for the controller 116, which can then operate the drive system of the vehicle 102.

[0030] In the depicted example, the perception component 110 may detect object detections that may be used to generate map elements 122 based at least in part on sensor data (such as lidar data 120, although additional or alternative types of sensor data may be used). As the vehicle 102 executes trajectory 118 and passes through this portion of the environment, the vehicle 102 may detect object detections and generate map elements 122. In some examples, determining the location and / or range of map elements 122 may be based at least in part on the perception component 110 generating object detections associated with a portion of the environment that result in map elements 122 from various poses of the vehicle 102 as the vehicle 102 travels through the environment. For example, the vehicle 102 may generate a first object detection associated with map element 122 at pose 124 and a second object detection associated with map element 122 at pose 126. A pose may include an orientation and a location. For example, pose 128 includes location 130 and orientation 132. The map creation component 114 may use the vehicle's position and orientation indicated by the pose determined according to the localization technique to determine the relative orientation and position of the object detection. The map creation component 114 may use the detection values ​​associated with the map elements 122 generated by the recognition component 110 while the vehicle 102 was in pose 124 and pose 126 to estimate the position of objects in the environment, such as estimating the position and / or range of the map elements 122.

[0031] In some examples, the map generation component 114 may determine a pose graph including one or more poses estimated by the vehicle as part of a simultaneous localization and mapping (SLAM) technique performed by the localization component of the map generation component. The SLAM technique may include a Bayesian filter (such as a Kalman filter), bundle adjustment, maximum a posteriori estimation (MAP), and / or any simultaneous localization and mapping (SLAM) algorithm.

[0032] Estimating a pose node associated with the vehicle's pose may include modifying the first estimated pose to increase / decrease the fit / distance between a subset of lidar points, called a second lidar point set, associated with the second estimated pose, and a subset of lidar points, called the first lidar point set. Also, a factor (constraint) between the two poses may be modified (e.g., by decreasing the distance between the two sets of lidar points) to fit the sets of sensor data together. The factor between these two estimated poses may be a constraint that identifies how the vehicle moved from one pose to the other. In some examples, the estimated pose and the factor between them may be based at least in part on sensor data (e.g., lidar data, etc.) and / or auxiliary sensor data, such as inertial measurement unit (IMU) sensor data, odometry data (e.g., which may be based at least in part on data received from wheel encoders or other sensors), and / or the like. For example, the vehicle may use IMU and / or odometry data to instantiate a pose estimate and / or factors between two pose estimates, and may use lidar data to refine these initial estimates. The resulting pose graph may be a graph in which nodes correspond to the poses of the vehicle at various points in space and edges represent constraints between the poses. The latter are obtained from observations of the environment or from locomotion actions performed by the robot.

[0033] As used herein, it should be noted that a pose graph may include a factor graph composed of pose nodes and edges (factors / constraints). Here, nodes indicate the pose of the vehicle (i.e., the vehicle's position and orientation (e.g., two-dimensional or three-dimensional orientation, etc.) within the environment / map), and edges (also called factors) indicate a set of constraints that define the relationship between one pose (node) and another pose (node). In some examples, a confidence score, such as covariance, may be associated with an edge. In some examples, the vehicle may determine the pose graph based at least in part on sensor data, including odometry data and / or inertial data. In some examples, pose nodes may be associated with sensor data received by the vehicle over a time frame.

[0034] In some examples, a factor / constraint between two pose nodes (illustrated as an arrow between the nodes in FIG. 1 ) may identify a translation and / or rotation between a first pose node and a second pose node and an associated covariance, which indicates how likely the constraint is to be correct for any number or combination of variables. For example, a constraint may define how a vehicle in a first pose reaches a second pose. In some examples, a constraint may additionally or alternatively include a time component, such as may be indicated by velocity and / or acceleration. In some examples, a first confidence score may be associated with the constraint to indicate the likelihood that the constraint is correct, and a second confidence score may be associated with the association of the two pose nodes to indicate the likelihood that the two pose nodes are associated with sensor data collected at those nodes and that the sensor data represent at least a portion of the environment in which they are the same. In some examples, the association may be a closed-loop factor, while in other examples, this may not be the case. For example, the association factor may simply identify that two poses are close enough to be associated with sensor data including the same object detection.

[0035] In some examples, the map generation component 114 may determine a preposition pose graph that the vehicle 102 may send to the remote computing device. In some examples, the vehicle 102 may send object detection and / or sensor data to the remote computing device in association with the preposition pose graph. In additional or alternative examples, the map data may be generated entirely at the remote computing device based at least in part on sensor data received from the vehicle 102. In some examples, the map generation component 114 at the vehicle 102 may be different from the map generation component at the remote computing device. For example, the map generation component 114 at the vehicle may include a component for localizing the vehicle 102 within the environment based on the map (e.g., the map generation component 114 may include a SLAM component). Additionally or alternatively, the map generation component 114 may include a component for generating a preposition pose graph based at least in part on sensor data collected by the vehicle as part of a SLAM operation. This preposition pose graph may be sent to the remote computing device, which may refine the pose graph into a final pose graph for generating the map data. In some examples, refining the pose graph may be based at least in part on sensor data received from multiple passes at the same location, whether from the same vehicle or multiple vehicles. This new sensor data may be used to re-optimize the pose graph, resulting in updates / modifications to the pose graph. Note that the nodes in the pose graph may be the same nodes, only changed following the pose graph re-optimization.

[0036] For example, refining the pre-position pose graph into the first pose graph may include modifying pose nodes to reduce distances between points in the first type of sensor data associated with the same object (i.e., hard constraints) based at least in part on one or more factors associated with the pose nodes and their associated covariances, based on at least an iteration that is part of the optimization of the pose graph (regardless of whether a truly optimal pose graph is ultimately determined). The optimization iteration may include adjusting pose nodes and / or factors of the pose graph to reduce a loss determined for the entire pose graph or for at least a region of the pose graph. The iteration may additionally or alternatively be performed based on new sensor data. The final pose graph determined by the remote computing device may be used to determine that different sensor data sets associated with different nodes are associated with the same object, and may be used to generate a map. The map may include a representation of the road surface in the environment, such as a two-dimensional and / or three-dimensional model, which may include a polygon mesh for representing the road surface, a wireframe model, and / or the like. For example, and not by way of limitation, while FIG. 1 depicts such a representation 134 as a wireframe model, it is understood that the map may additionally or alternatively include a two-dimensional map, such as the top-down view illustrated in FIG. 1, having semantic labels associated with locations and / or regions within the map that are associated with respective portions of the environment.

[0037] However, while refinements to the pose graph used to generate the map data can be used to generate such a representation 134, map elements representing key objects and regions in the environment are not so easily modified. Once corrections 136 to the pose graph are received that modify one or more pose nodes (illustrated in FIG. 1 as updated poses 138) and / or factors / trajectories therebetween, techniques described herein can be used to determine updated map elements 140, as described further below.

[0038] [Process example] 2A and 2B illustrate a schematic flow diagram of an example process 200 for adjusting map elements using at least two sparse pose graphs. In particular, the example process 200 can be used when an updated pose graph is being generated or received. In such examples, a pre-position pose graph may already be determined, whether it is a pre-position pose graph determined by the vehicle, such as according to a SLAM technique performed by a vehicle computing device, or simply a previous version of the pose graph. In some examples, the pose graph may be a sparse graph determined at least in part based on various sensor data (e.g., by an optimization algorithm that adjusts the pose graph to increase the degree of alignment of spatial and / or other types of sensor data according to constraints determined at least in part based on odometry and / or inertial data). The pose graph may be sparse in the sense that the number of nodes may be fewer than the number of sensor data sets.

[0039] For example, 100 units of odometry data, 500 units of inertial data, and 30 units of lidar data may be captured over the same distance or time period, e.g., 1 meter or 1 second. The portion of the pose graph associated with the data collected over 1 meter or 1 second may, in at least one example, be represented as a single pose node according to the sampling rate associated with the pose graph. In some examples, the updated pose graph may be a new version of a prior pose graph generated as at least part of an optimization or re-optimization of the pose graph, such as a next iteration of optimization, a re-optimization based on new sensor data, and / or a re-optimization after optimization parameters, such as optimization constraints, loss functions, or the like, have been changed.

[0040] At operation 202, the example process 200 may include receiving a first pose graph associated with first map data according to any of the techniques described herein. The first pose graph may be a pre-position graph, whether the pre-position graph is a pose graph determined by the vehicle 102 or a previous version of the pose graph refined by a remote computing device. The first map data may include a representation of the environment generated at least in part based on the first pose graph and / or map elements identifying portions of the environment as associated with key objects or areas. These key objects or areas may include traffic signs (and types such as stop signs, stop signals, exit signs, speed signs, etc.), static objects (e.g., fire hydrants, mailboxes, utility poles, etc.), key areas (e.g., drivable areas, parking areas, lanes, turning lanes, exit lanes, merge lanes, sidewalks, crosswalks, pedestrian zones, passenger pick-up and / or drop-off areas, no-idling zones, etc.), etc. 2A depicts a top-down view of a scenario 204, including a straight road and an example representation of a first pose graph. The first pose graph (or a portion thereof) is depicted as a solid circle (position) and a solid arrow (orientation), and the locations and extents of the map elements are depicted as map elements 206. Note that the first pose graph may include additional data, as described above, and the map elements 206 may have been previously labeled using recognition data generated by a vehicle or a human.

[0041] At operation 208, the example process 200 may include receiving a second (updated) pose graph according to any of the techniques described herein. For example, the remote computing device may refine the prior pose graph or determine a new pose graph based at least in part on a refinement technique, a modification of the optimization algorithm, a next iteration of optimization, and / or new sensor data. While FIG. 2A depicts the second pose graph as a dashed circle (position) and a dashed arrow (orientation), the second pose graph may include additional data, as described above.

[0042] At operation 210, the example process 200 may include determining one or more source nodes of a first pose graph associated with the map element according to any of the techniques described herein. The source nodes may be determined from the first pose graph. For example, operation 210 may include determining n nodes in the first graph that are nearest to the map element 206, where n is a positive integer. In some examples, n may be a static number, while in additional or alternative examples, n may be determined based at least in part on the density of nodes in the pose graph. In some examples, the map generation component (of the vehicle or remote computing device) may limit the search for nearest nodes to those first pose graph nodes that are within a threshold distance 212 of the map element 206, thereby ensuring that corrections to distant nodes do not affect the position and / or range of the map element 206. In some examples, operation 210 may include ranking nodes within threshold distance 212 of the map element according to smallest to largest distance and selecting the top n (n nearest) nodes to the map element. In additional or alternative examples, operation 210 may include determining a nearest first pose graph pose node to the map element and determining the n-1 nearest first pose graph pose nodes to the nearest node. Additionally or alternatively, operation 210 may include determining the n nearest source-target node pairs, source-target node pairs within distance 212, and / or combinations thereof.

[0043] At operation 214, the example process 200 may include determining whether the number of source nodes (or source-target node pairs) identified in operation 210 meets or exceeds a lower threshold number p, according to any of the techniques described herein, where p is a positive integer less than n. If the number of source nodes identified in operation 210 does not meet the lower threshold (e.g., if no source nodes are present within the threshold distance 212 of the map element 206, or if fewer than a threshold number of source nodes are present within the distance 212), the example process 200 may continue to operation 216. If the number of source nodes identified in operation 210 meets or exceeds the lower threshold, the example process 200 may continue to operation 218. In some examples, operation 214 may ensure that the number of source nodes available is sufficiently reliable to determine accurate modifications to the position and / or extent of the map element. In some examples, operation 214 may be skipped.

[0044] At operation 216, the example process 200 may include generating a notification, saving the map element, and / or modifying the map element 206 based at least in part on the second modification in the second map element, according to any of the techniques described herein. In some examples, operation 216 may include generating and / or sending to a user computing device a notification indicating the map element 206 and / or a portion of the pose graph nearest to the map element 206. For example, the notification may include an indication that the map element 206 may be inaccurate and / or may include executable code for presenting a user interface option for providing a manual modification to the map element 206. In some examples, an indication that the map element 206 may be inaccurate may be included in an updated map associated with the map element 206, so that a vehicle planning component may use the indication when determining a trajectory in a region proximate the map element 206. This may result in an increased likelihood that the vehicle will deviate from the map element 206.

[0045] At operation 218, the example process 200 may include determining one or more target nodes (nodes of the second pose graph) associated with one or more source nodes according to any of the techniques described herein. In some examples, operation 218 may include identifying an updated version of the source node based on matching node identifiers between the source node and the target node. For example, each node may be associated with a pose node identifier and a timestamp. Determining the target node may include determining an updated version of a pose node having the same pose node identifier and timestamp. This may be the case in instances where the updated pose graph is a new version of the same pose graph, such as when new sensor data is received and an updated pose graph is received as at least part of a pose graph re-optimization, when optimization parameters are changed (e.g., changing constraints between nodes, changing distance 212, changing the number n of nearest neighbor nodes used, replacing the type of solver, and / or the like), when a next iteration of the optimization process occurs, or the like.

[0046] In an additional or alternative example, an entirely new pose graph may be received in operation 218. In such an example, for a first source node, such as source node 220(a), operation 218 may include determining the node in the second pose graph that is closest to the first source node, i.e., target node 222(a). In an additional or alternative example, the corresponding nodes may already be known, such as following a technique in which the pose graph is determined using the same or similar pose graph generation parameters, such as the same sampling rate, start and / or end positions, and some normal indication that the pose nodes are located in the same place in the environment (e.g., using GPS data or a test course where a vehicle is driven on the exact same path). In the depicted example, three source nodes 220(a) through 220(c) may have been determined in operation 210, and three corresponding target nodes 222(a) through 222(c) may have been determined in operation 218. Additionally, Figure 2A depicts which nodes are associated by using matching subscripts in the numbering on the lines between source-target node pairs (e.g., a first source-target node pair includes source node 220(a) and target node 222(a), and a second source-target node pair includes source node 220(b) and target node 222(b)).

[0047] 2B , in operation 224, the example process 200 may include determining a modification 226 according to any of the techniques described herein based at least in part on one or more source nodes and one or more target nodes (one or more source-target node pairs determined in operations 210 and 218). Operation 224 may include operation 224(a) and, optionally, operation 224(b). In some examples, operations 224(a) and / or 224(b) may be repeated for all source-target node pairs determined in operations 210 and 218, i.e., for those source-target node pairs determined to be associated with map element 206.

[0048] At operation 224(a), the example process 200 may include determining a first transformation 228 between a first source node 222(a) and a first target node 222(a) according to any of the techniques described herein. In some examples, determining the first transformation 228, depicted as a line between the source node 222(a) and the target node 222(a), may include determining differences in position, orientation, and / or constraints between the node pair. In some examples, this transformation may be represented as a transformation function, such as a linear transformation, a geometric transformation, and / or a nonlinear transformation. In some examples, this transformation may be represented in a tensor identifying three dimensions or six degrees of freedom (e.g., x position, y position, z position, yaw, roll, pitch transformation, etc.).

[0049] At operation 224(b), the example process 200 may include determining a weight associated with the first transformation based at least in part on the distance between the map element 206 and the first source node 220(a), according to any of the techniques described herein. The weight decreases the influence of the transformation determined for the source-target node pair on the (final) modification 226 as the distance from the map element 206 to the source-target node pair increases. For example, a weight may be determined associated with the first transformation 228 based at least in part on the distance 230 of the first source node 220(a) from the map element 206. The modification 226 may be determined at least in part based on determining a weighted sum of the transformations determined for the source-target node pair. In some examples, the weight may be determined using a linear, quadratic, or higher order function. For example, the modification 226, i.e., M, may be given by:

[0050]

number

[0051] where f is the weighting function and d i is the distance from the map element 206 to the i-th source node of the source-target node pair set, and T i is the transformation from the i-th source node to the i-th target node. As an example, the linear weighting function may include:

[0052]

number

[0053] where d threshis the threshold distance 212, and "norm" is a normalization of all weights determined for node pairs (e.g., so that the weights associated with various node pairs sum to 1). Note that if a pose node were located exactly at the location of map element 206, the normalization and weighting would result in the pose node located at the location of map element 206 dominatingly or completely influencing modifications to the location of map element 206. As an additional or alternative example, the quadratic weighting function may include:

[0054]

number

[0055] Inverse functions can also include:

[0056]

number

[0057] . Any additional or alternative weighting function may be used. An optimal weighting function will exhibit smoothness and continuity in the determined weights, although this is not necessary, as an inverse function would be appropriate but may have corners / discontinuities. In some examples, the weights may be determined using a robust loss function, such as the Cauchy loss, pseudo-Huber loss, Blake-Zisserman loss, Tukey loss, German-McClure loss, Welsch loss, and / or similar. In such examples, the robust loss function may help ensure that outlier transformations are ignored or minimized in their influence on the final transformation. In some examples, the loss may be determined iteratively as iterations of the transformation are determined as part of the optimization (e.g., determine weights, determine residuals / errors in regions of the map, change corrections to reduce the residuals, redetermine weights, redetermine residuals, and repeat until the loss converges). In some examples, the loss function may reduce the effect of outlier transformations or functionally remove such outlier transformations.

[0058] If weighting is not used, the transformations associated with node pairs may be averaged or otherwise integrated to determine the modification 226. In examples in which weights are determined, the modification 226 may be determined based at least in part on determining a weighted sum of the transformations determined in operation 224(a).

[0059] At operation 232, the example process 200 may include modifying the map element based at least in part on the modification according to any of the techniques described herein as an updated map element 234 of the second map data. In some examples, operation 232 may include moving and / or updating the extent of the map element 206, resulting in the updated map element 234. Updating the extent of the object may include converting the extent to points (e.g., sampling the edges of the area defined to be associated with the map element), modifying those points according to a transformation (described above), and adjusting a polygon or other shape to fit the resulting modified points (the polygon or other shape defining the new extent of the map element).

[0060] At operation 236, the example process 200 may optionally include determining a transformation stress according to any of the techniques described herein. In some examples, the stress may be determined in association with a modification such as modification 226. Additionally or alternatively, the stress may be determined for multiple changes in map elements across a region. In some examples, determining the stress associated with the modification may include determining a residual associated with the modification (e.g., a loss determined by a loss function, etc.). In some examples, the residual may be based at least in part on the same or similar factors on which the weights determined in operation 224(b) are based. In additional or alternative examples, determining the stress may be based at least in part on the transformation of the source node itself. For example, the transformation may be transformed from three or six dimensions to a single numeric value, such as by determining the maximum value of the transformation (in any dimension), the mean value of the transformation, the standard deviation of the transformation (and / or the transformation of neighbors within a threshold distance). In additional or alternative examples, the stress may be the transformation itself. The stress at a location may additionally or alternatively be based at least in part on the number of source nodes on which the correction is based (and whether that number meets or exceeds a threshold), the variance of the transformation in an area surrounding the location (e.g., within the same grid cell, within a threshold distance from the location, or the like), a determination of the difference between the transformation (from source node to target node) and the correction to the map element (e.g., this may result in a vector, or an average value may be determined at least in part based on the vector), or any combination of the described factors.

[0061] In some examples, the stress of a region may be determined by determining the stress of individual transforms within the region and determining the associated distribution and / or variance. Regardless of whether individual stresses or region-associated stresses are determined, a map may be generated associating the determined stresses with locations associated with the respective source nodes from which they were generated. Additionally or alternatively, for locations that do not have a source node associated with them, stress values ​​may be determined by interpolating stress values ​​based at least in part on previously determined stresses of one or more nearby source nodes. In some examples, a visual representation of the stress may be generated at least in part based on coding the stress with a color or grayscale based at least in part on the magnitude of the stress. For stresses that meet or exceed a threshold stress, the example process 200 may continue to operation 240. For stresses that are less than the threshold stress, the example process 200 may continue to operation 242.

[0062] In some examples, operation 236 may additionally or alternatively include determining whether map element 206 has been similarly modified to the source node and / or other map elements in the vicinity of map element 206 according to any of the techniques described herein. Operation 236 may ensure that map element 206 has been updated accurately and not based on outliers or software bugs. Determining similarity may include determining the difference between the modification to map element 206 and a transformation of the source node or a modification to a different map element. This difference may be represented, in some examples, as a vector and / or in terms of orientation and magnitude, tensors, or the like. In some examples, determining whether modification 226 is similar to other modifications or transformations may include determining whether the difference is below a threshold. In some examples, the total difference may be determined by averaging the difference between modification 226 and a number of other transformations and / or modifications.

[0063] At operation 238, the example process 200 may include determining whether the stress meets or exceeds a threshold stress. If the difference exceeds the threshold, the example process 200 may continue to operation 240 ("Yes" branch). If the difference is equal to or less than the threshold, the example process 200 may continue to operation 242 ("No" branch). Determining whether the source node and / or other map elements are "nearby" may include determining whether the source node and / or other map elements are at a distance from the map element 206 that is less than a threshold distance. In some examples, this threshold distance may be greater than the threshold distance 212, but in some examples, the threshold distance may be the same as or less than the threshold distance 212.

[0064] At operation 240, the example process 200 may include generating and / or undoing a modification or modifying the modification based at least in part on the modification of one or more other map elements. For example, for map elements associated with regions of the environment associated with stress values ​​that meet or exceed a threshold stress, a notification may be presented via a user interface for manual change and / or confirmation of the modification. In some examples, stresses in the map having values ​​that exceed the stress threshold, source nodes having stresses associated with source nodes that meet or exceed the threshold stress, or source nodes having stresses that meet or exceed an interpolated threshold stress may be color-coded using a different color to call attention to that portion of the map. Additionally or alternatively, the modification may be undone. This undo may occur automatically (without user input). In some examples, the automatic undo may generate a notification, which may be presented via a user interface. Additionally or alternatively, the undo may occur in response to user input authorizing the undo.

[0065] In an additional or alternative example, the system may modify the modification based at least in part on determining that the stress associated with the modification meets or exceeds a stress threshold. Such modification may be made at least in part based on a modification to the number q of nearest neighbors in other transformed map elements, where q is a positive integer. In generating a new modification based at least in part on the modification to other map elements, a process similar to operation 210, operation 214, and / or operation 224 may be used. For example, the q nearest map elements within a threshold distance of map element 206 may be determined (using a similar determination as to whether there are a sufficient number of map elements or whether there are sufficiently close map elements), and weights may be determined related to how those map elements have been modified based at least in part on the distance between map element 206 and each map element. These weights may be used to determine a new modification for map element 206 by determining a weighted sum of the modifications.

[0066] For example, determining the change may include interpolating the correction in the nearest q other map elements, determining the average change in the nearest q other map elements, or the like.

[0067] At operation 242, the example process 200 may include transmitting second map data including the updated map elements to the vehicle or another vehicle according to any of the techniques described herein. For example, while FIG. 1 depicts the map data as a three-dimensional wireframe model representation 134, other examples are contemplated. In some examples, the map data may be transmitted to a fleet of vehicles. The vehicle may use the map data as part of a SLAM operation to localize the vehicle within the environment, navigate cautiously around areas of the environment that include blind corners or other unusual conditions, etc.

[0068] [Conversion and modification examples] Figures 3 through 5 illustrate examples of source-target node pairs, the transformations between them, and the resulting modifications determined for map elements associated with a pose graph according to the techniques described herein. These examples increase in complexity and the number of modified map elements. In particular, these figures illustrate a two-dimensional transformation and resulting modification of the map elements in Figure 3, and three-dimensional transformations and resulting three-dimensional modifications of the map elements in Figures 4 and 5. Also noteworthy is the regional effect that the local transformation of a source-target node pair has on the local map elements. Distant node pairs do not affect distant map elements. Figures 3 through 5 depict source nodes in the first (previous or previous) pose graph as squares, target nodes in the second (updated) pose graph as circles, map elements as stars, updated map elements as triangles, and node-to-node transformations / modifications between map elements as lines.

[0069] [Stress map example] FIG. 6 illustrates an example of a stress / strain map that may be generated as part of FIG. 2B . The stress map depiction may include deformations depicted as lines superimposed on a satellite image of the environmental region. Before the modification is made, a grid may be overlaid on the region to be updated, and the grid may be distorted according to the transformations and / or modifications, as described above. For example, the grid may be generated in a 5-meter by 5-meter square or according to any other division. FIG. 6 includes multiple transformations, such as transformation 602. Here, no transformation occurs and the grid accordingly exhibits no distortion; the grid still appears uniform / square in such regions. In examples where the modification was greater, the grid may exhibit more distortion, such as in transformation 604 and / or transformation 606. FIG. 6 also includes a grayscale depiction of stress values, with the grayscale coded darker to indicate higher stress and lighter to indicate lower stress. Areas of the environment surrounding transformation 604 and transformation 606 include dark patches indicating high stress values ​​(e.g., stress values ​​that meet or exceed a threshold). In some instances, stress values ​​may be thresholded so that only stress values ​​that meet or exceed a threshold stress are depicted. Figure 6 depicts such an example, which is why the satellite imagery is see-through at multiple points. Note that while Figure 6 is illustrated in grayscale, the stress map may also be color-coded.

[0070] [System example] FIG. 7 illustrates a block diagram of an example system 700 that implements the techniques described herein. In some examples, the example system 700 may include a vehicle 702, which may represent vehicle 102 in FIG. 1. In some examples, the vehicle 702 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which defines an autonomous vehicle as one in which a driver (or passenger) is not expected to control the vehicle at any time and is capable of performing all safety-critical functions throughout the entire journey. However, in other examples, the vehicle 702 may be a fully or partially autonomous vehicle having any other level or classification. Furthermore, in some examples, the techniques described herein may be usable by non-autonomous vehicles as well.

[0071] Vehicle 702 may include vehicle computing device 704, sensors 706, emitters 708, network interfaces 710, and / or drive components 712. Vehicle computing device 704 may represent computing device 106, and sensors 706 may represent sensors 104. System 700 may additionally or alternatively include computing device 714.

[0072] In some examples, sensors 706 may represent sensors 104 and may include: lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., global positioning system (GPS), compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), image sensors (e.g., visible light (RGB), infrared (IR), intensity, depth, time-of-flight cameras, etc.), microphones, wheel encoders, environmental sensors (e.g., thermometers, hygrometers, light sensors, pressure sensors, etc.). Sensors 706 may include multiple examples of each of these or other types of sensors. For example, radar sensors may include individual radar sensors positioned on the corners, front, back, sides, and / or top of vehicle 702. As another example, cameras may include multiple cameras positioned at various locations on the exterior and / or interior of vehicle 702. Sensors 706 may provide input to vehicle computing device 704 and / or computing device 714 .

[0073] Vehicle 702 may also include emitters 708 for emitting light and / or sound, as described above. Emitters 708 in this example may include interior audio and visual emitters for communicating with passengers of vehicle 702. By way of example and not limitation, interior emitters may include: speakers, lights, signs, display screens, touch screens, tactile emitters (e.g., vibration and / or force feedback, etc.), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.), and the like. Emitters 708 in this example may also include exterior emitters. By way of example and not limitation, exterior emitters in this example include: speakers, lights, signs, display screens, touch screens, tactile emitters (e.g., vibration and / or force feedback, etc.), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.), and the like, one or more of which may include acoustic beam steering technology. These include lights or other indicators of vehicle operation (e.g., signal lights, signs, light arrays, etc.) to indicate direction of travel, and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) to audibly communicate with pedestrians or other nearby vehicles.

[0074] Vehicle 702 may also include a network interface 710 that enables communication between vehicle 702 and one or more other local or remote computing devices. For example, network interface 710 may facilitate communication with other local computing devices on vehicle 702 and / or drive components 712. Network interface 710 may also additionally or alternatively enable the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). Network interface 710 may also or alternatively enable vehicle 702 to communicate with computing device 714. In some examples, computing device 714 may include one or more nodes in a distributed computing system (e.g., a cloud-based computing architecture, etc.).

[0075] Network interface 710 may include a physical interface and / or a logical interface for connecting vehicle computing device 704 to another computing device or to a network, such as network 716. For example, network interface 710 may enable Wi-Fi-based communications, such as over frequencies defined by the IEEE 700.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 6G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables each computing device to interface with other computing devices. In some examples, vehicle computing device 704 and / or sensors 706 may transmit sensor data to computing device 714 via network 716 at a particular frequency, after a predetermined period of time, in near real time, etc.

[0076] In some embodiments, vehicle 702 may include one or more drive components 712. In some embodiments, vehicle 702 may have a single drive component 712. In some embodiments, drive component 712 may include one or more sensors for detecting conditions in the drive component 712 and / or the vehicle's 702 surroundings. By way of example and not limitation, the sensors in drive component 712 may include one or more wheel encoders (e.g., rotary encoders, etc.) for sensing the rotation of the wheels in the drive component, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive component, cameras or other imaging sensors, ultrasonic sensors for acoustically detecting objects in the drive component's surroundings, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to drive component 712. In some cases, sensors on drive component 712 may overlap or supplement corresponding systems in vehicle 702 (e.g., sensors 706, etc.).

[0077] The drive components 712 include numerous vehicle systems, including a high-voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as headlights / taillights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., a cooling system, safety systems, an on-board charging system, other electrical components such as a DC / DC converter, a high-voltage junction, high-voltage cables, a charging system, a charge port, etc.). Additionally, the drive components 712 may include a drive component controller that receives and preprocesses data from sensors and controls the operation of various vehicle systems. In some examples, the drive component controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more components for performing various functionality in the drive component 712. Additionally, the drive component 712 may include one or more communication connections that enable each drive component to communicate with one or more other local or remote computing devices.

[0078] Vehicle computing device 704 may include a processor 718 and a memory 720 communicatively coupled to the one or more processors 718. Memory 720 may represent memory 108. Computing device 714 may also include a processor 722 and / or a memory 724. Processor 718 and / or 722 may be any suitable processor capable of processing data and executing instructions to perform operations as described herein. By way of example and not limitation, processor 718 and / or 722 may include one or more central processing units (CPUs), graphics processing units (GPUs), integrated circuits (e.g., application specific integrated circuits (ASICs), etc.), gate arrays (e.g., field programmable gate arrays (FPGAs), etc.), and / or any other device or portion of a device that processes electronic data and converts it into other electronic data that may be stored in registers and / or memory.

[0079] Memory 720 and / or 724 may be examples of non-transitory computer-readable media. Memory 720 and / or 724 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and functions attributed to the various systems. In various implementations, memory may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include numerous other logical, programmatic, and physical components, although those shown in the accompanying figures are merely examples relevant to the description herein.

[0080] In some embodiments, memory 720 and / or memory 724 may store a localization component 726, a recognition component 728, a planning component 730, a map creation component 732, a map 734, and / or a system controller 736. The recognition component 728 may represent the recognition component 110, the planning component 730 may represent the planning component 112, and / or the map creation component 732 may represent the map creation component 114. Memory 724 may store the map creation component 738.

[0081] In at least one example, the localization component 726 may include hardware and / or software for receiving data from the sensors 706 and determining the position, velocity, and / or orientation of the vehicle 702 (e.g., one or more of x position, y position, z position, roll, pitch, or yaw). For example, the localization component 726 may include and / or request / receive a map 734 of the environment and continuously determine the position, velocity, and / or orientation of the autonomous vehicle within the map 734. In some examples, the localization component 726 may receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, and the like to accurately determine the position, pose, and / or velocity of the autonomous vehicle using SLAM (simultaneous localization and mapping), CLAMS (simultaneous calibration, localization, and mapping), relative SLAM, bundle adjustment, nonlinear least squares optimization, and / or the like. In some examples, localization component 726 may provide data to various components in vehicle 702 to determine an initial position of the autonomous vehicle for generating a trajectory and / or generating map data, as described herein. In some examples, localization component 726 may provide map creation component 732 with the relative position and / or orientation of vehicle 702 with respect to the environment and / or sensor data associated therewith. In some examples, localization component 726 may generate a preposition pose graph and associate pose nodes with frames of sensor data in the logged data stored in memory 720. This logged data (including the preposition pose graph) may be transmitted to computing device 714 via network 716.

[0082] In some examples, perception component 728 may include a primary perception and / or prediction system implemented in hardware and / or software. Perception component 728 may detect objects in the environment surrounding vehicle 702 (e.g., identify that an object is present), classify objects (e.g., determine a type of object associated with a detected object), segment sensor data and / or other representations of the environment (e.g., identify portions of the sensor data and / or representations of the environment as associated with a detected object and / or type of object), determine characteristics associated with objects (e.g., tracks identifying a current location, predicted location, and / or previous locations, heading, speed, and / or acceleration associated with an object, etc.), and the like. Data determined by perception component 728 is referred to as perception data.

[0083] The planning component 730 may receive the position and / or orientation of the vehicle 702 from the positioning component 726 and / or the perception data from the perception component 728, and may determine instructions for controlling operation of the vehicle 702 based at least in part on any of this data. In some examples, determining the instructions may include determining the instructions based at least in part on a format associated with a system with which the instructions are associated (e.g., initial instructions for controlling operation of the autonomous vehicle may be initialized in a first format in messages and / or signals (e.g., analog, digital, pneumatic, kinematic) that the system controller 736 and / or the drive component 712 may interpret / execute, and second instructions for the emitter 708 may be initialized according to a second format associated therewith).

[0084] Map generation component 732 may operate in conjunction with localization component 726 to generate a preposition pose graph and / or preposition sensor data alignment for near-term planning and / or trajectory validation. In additional or alternative examples, the preposition pose graph may be determined by map generation component 732. In some examples, map generation component 732 may differ from map generation component 738 in that map generation component 738 may perform the soft constraint optimization described herein. In additional or alternative examples, soft constraint optimization may also occur in map generation component 732. In some examples, a user may provide input at computing device 714 via a user interface, such as to add or delete proposed locations or to fix portions of the pose graph. In some examples, map generation component 738 may additionally or alternatively generate map data based at least in part on the refined pose graph and transmit the map data to vehicle 702, which may be stored in memory 720.

[0085] Although the localization component 726, the recognition component 728, the planning component 730, the map 734, and / or the system controller 736 are illustrated as being stored in memory 720, any of these components may include processor-executable instructions, machine learning models (e.g., neural networks, etc.), and / or hardware, and all or a portion of any of these components may be stored in memory 724 or configured as part of the computing device 714.

[0086] As described herein, localization component 726, recognition component 728, planning component 730, map generation component 732 and / or map generation component 738, and / or other components in system 700 may include one or more ML models. For example, localization component 726, recognition component 728, and / or planning component 730 may each include various ML model pipelines. In some examples, the ML models may include neural networks. An exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to create an output. Each layer in a neural network may include another neural network or may include any number of layers (convolutional or not). As can be understood in the context of the present disclosure, neural networks may utilize machine learning, which may refer to a broad class of such algorithms in which output is generated based on learned parameters.

[0087] Although described in the context of neural networks, any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to: Namely, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression spline (MARS), local estimation scatterplot smoothing (LOESS), etc.), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS), etc.), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomy 6 (ID6), chi-squared automated interaction detection (CHAID), decision stumps, conditional decision trees, etc.), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average-one-dependence estimator (AODE), Bayesian belief network (BNN), Bayesian network, etc.), clustering algorithms (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering, etc.), association rule learning algorithms (e.g., perceptual These include: TRON, backpropagation, Hopfield networks, radial basis function networks (RBFN), etc.), deep learning algorithms (e.g., deep Boltzmann machines (DBM), deep confidence networks (DBN), convolutional neural networks (CNN), stacked autoencoders, etc.), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA), etc.), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), Adaboost, stacked generalization (blending), gradient boosting machines (GBM), gradient boosting regression trees (GBRT), random forests, etc.), support vector machines (SVM), supervised learning, unsupervised learning, and semi-supervised learning.Additional example architectures include neural networks such as ResNet-50, ResNet-101, VGG, DenseNet, EfficientNet, PointNet, Xception, ConvNeXt, and similar, or visual transforms (ViTs) such as Bidirectional Encoder from Image Transformation (BEiT), Visual Bidirectional Encoder from Transformation (VisualBERT), Image Generative Pre-Trained Transform (Image GPT), Data Efficient Image Transform (DeiT), Deeper Vision Transform (DeepViT), Convolutional Vision Transform (CvT), Detector Transform (DETR), Miti-DETR, or similar.

[0088] Memory 720 may additionally or alternatively store one or more system controllers 736, which may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems in vehicle 702. These system controllers 736 may communicate with and / or control systems corresponding to drive component 712 and / or other components of vehicle 702. For example, planning component 730 may generate instructions based at least in part on perception data generated by perception component 728, which trajectory validation component 732 may verify and / or send to system controller 736. System controller 736 may control operation of vehicle 702 based at least in part on instructions received from planning component 730. In some examples, these instructions may be based at least in part on map data generated according to the techniques described herein. In some examples, the trajectory verification component 732 may replace the instructions generated by the planning component 730 with alternative instructions associated with a contingent trajectory, such as, for example, a contingent trajectory that may specify an alternative maneuver, and / or the like.

[0089] 7 is illustrated as a distributed system, it should be noted that in alternative examples, components of vehicle 702 may be associated with computing device 714 and / or components of computing device 714 may be associated with vehicle 702. That is, vehicle 702 may perform one or more of the functions associated with computing device 714, and vice versa.

[0090] [Example clauses] A. A method comprising: receiving a first pose graph associated with first map data, the first pose graph including a first set of nodes indicating one or more of a position or orientation of a vehicle navigating an environment, and the first map data including a map element indicating an initial location of an object within the environment; receiving a second pose graph; determining a source node of the first pose graph associated with the map element; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element based at least in part on the transformation as an updated map element of the second map data; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and, based at least in part on determining that the stress is less than the threshold stress, transmitting the updated map element to the vehicle or another vehicle and navigating the environment based at least in part on the updated map element.

[0091] B. The method of section A, wherein determining stresses associated with locations in the environment is based at least in part on at least one of the following: determining whether a transformation meets or exceeds a transformation threshold; determining variances associated with one or more transformations within a threshold distance of the location; determining weights based at least in part on distances between map elements and source nodes; a loss determined by a loss function based at least in part on at least one of a transformation or a difference between a transformation and another transformation; the output of an optimization that transforms two or more source nodes; determining whether the number of additional source nodes used to determine a modification meets or exceeds a threshold; or determining the difference between a transformation and a modification.

[0092] C. The method of either paragraph A or B, wherein the map element is a first element, the method further comprising: determining a stress map based at least in part on interpolating a second stress value based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds a threshold stress; determining that the second map element or location within the environment is associated with the second stress value; and performing at least one of the following based at least in part on determining that the second stress value meets or exceeds the threshold stress: generating a notification associated with the second map element or location within the environment; undoing a second modification associated with the second map element; or modifying the second modification.

[0093] D. The method of any one of paragraphs A through C, wherein determining a modification includes determining a first weight based at least in part on a first distance between the map element and the source node, and modifying the map element further based at least in part on the first weight.

[0094] E. The method of any one of paragraphs A through D, wherein determining source nodes associated with the map element includes at least one of: determining a set of source nodes within a predefined distance of the initial location of the map element; or determining the n nearest source nodes to the initial location of the map element, where n is a positive integer.

[0095] F. The method of any one of paragraphs A-E, wherein the modification results in a change to at least one of the initial location of the map element or the magnitude size of the map element.

[0096] G. A system comprising: one or more processors; and memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations including receiving a first pose graph associated with first map data; receiving a second pose graph; determining a source node of the first pose graph associated with a map element of the first map data; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element based at least in part on the transformation as an updated map element of the second map data; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and, based at least in part on determining that the stress is less than the threshold stress, sending the updated map element to a vehicle and navigating an environment based at least in part on the updated map element.

[0097] H. The system of paragraph G, wherein determining the stress associated with a location in the environment is based at least in part on at least one of the following: determining whether a transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations that are within a threshold distance of the location; determining a weight based at least in part on a distance between a map element and a source node; a loss determined by a loss function based at least in part on at least one of a transformation or a difference between a transformation and another transformation; the output of an optimization that transforms two or more source nodes; determining whether the number of additional source nodes used to determine a modification meets or exceeds a threshold, or determining the difference between a transformation and a modification.

[0098] I. The system of any of paragraphs G or H, wherein the map element is the first element, the method further comprising: determining a stress map based at least in part on the stress and interpolating a second stress value based at least in part on one or more other stress values; determining that the second stress value meets or exceeds a threshold stress; determining that the second map element or location in the environment is associated with the second stress value; and, based at least in part on determining that the second stress value meets or exceeds the threshold stress, doing at least one of the following: generating a notification associated with the second map element or location in the environment; undoing a second modification associated with the second map element; or modifying the second modification.

[0099] J. The system of any one of paragraphs G through I, wherein determining a modification includes determining a first weight based at least in part on a first distance between the map element and the source node, and modifying the map element further based at least in part on the first weight.

[0100] K. The method of term J, wherein determining the first weight is based at least in part on a robust loss function.

[0101] L. The system of any one of terms G through K, wherein determining source nodes associated with the map element further includes at least one of: determining a set of source nodes within a predefined distance from the initial location of the map element; or determining the n nearest source nodes to the initial location of the map element, where n is a positive integer.

[0102] M. A system of any one of paragraphs G through L, wherein the modification results in a change to at least one of the initial location of the map element or the magnitude size of the map element.

[0103] N. The system of any one of paragraphs G through M, further comprising: determining a second modification to a second map element based at least in part on a second transformation between a second source node and a second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and, based on determining that the second stress meets or exceeds a threshold stress, doing at least one of the following: generating a notification associated with the second map element or a location in the environment; undoing the second modification associated with the second map element; or modifying the second modification.

[0104] O. The method of paragraph N, further comprising: receiving user input associated with the second map element; and determining a change to the second modification based at least in part on the user input.

[0105] P. The system of any one of paragraphs G through O, wherein the first pose graph is determined based at least in part on first sensor data received from one or more sensors of the vehicle or another vehicle, and the second pose graph is determined based at least in part on second sensor data received from one or more sensors.

[0106] Q. A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations including receiving a first pose graph associated with first map data; receiving a second pose graph; determining source nodes in the first pose graph associated with map elements of the first map data; determining associations of target nodes in the second pose graph with source nodes; determining a transformation between the source and target nodes; modifying the map elements based at least in part on the transformation as updated map elements of the second map data; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and, based at least in part on determining that the stress is less than the threshold stress, sending the updated map elements to a vehicle and navigating an environment based at least in part on the updated map elements.

[0107] R. A non-transitory computer-readable medium of a term Q, wherein determining stresses associated with locations in the environment is based at least in part on at least one of the following: determining whether a transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between a map element and a source node; a loss determined by a loss function based at least in part on at least one of a transformation or a difference between a transformation and another transformation; the output of an optimization that transforms two or more source nodes; determining whether the number of additional source nodes used to determine a modification meets or exceeds a threshold; or determining the difference between a transformation and a modification.

[0108] S. A non-transitory computer-readable medium of either term Q or R, wherein the map element is a first element, the method further comprising: determining a stress map based at least in part on the stress and interpolating a second stress value based at least in part on one or more other stress values; determining that the second stress value meets or exceeds a threshold stress; determining that the second map element or a location in the environment is associated with the second stress value; and, based at least in part on determining that the second stress value meets or exceeds the threshold stress, doing at least one of the following: generating a notification associated with the second map element or a location in the environment; undoing a second modification associated with the second map element; or modifying the second modification.

[0109] T. The non-transitory computer-readable medium of any one of paragraphs Q through S, further comprising: determining a second modification to a second map element based at least in part on a second transformation between a second source node and a second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and, based on determining that the second stress meets or exceeds a threshold stress, doing at least one of the following: generating a notification associated with the second map element or a location in the environment; undoing the second modification associated with the second map element; or modifying the second modification.

[0110] Although the example provisions described above are described with respect to one particular implementation, it should be understood that in the context of this document, the content of the example provisions may also be implemented via a method, device, system, computer-readable medium, and / or another implementation. Furthermore, any of Examples A-T may be implemented alone or in combination with any one or more of the other Examples A-T.

[0111] [Conclusion] Although the present subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.

[0112] The components described herein represent instructions that may be stored on any type of computer-readable medium and that may be implemented in software and / or hardware. All of the methods and processes described above may be embodied in and fully automated via components of software code and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof. Parts or all of the methods may alternatively be embodied in specialized computer hardware.

[0113] At least some of the processes described herein are illustrated as logical flow graphs, each operation of which represents a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, these operations represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, cause a computer or autonomous vehicle to perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which these operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to perform a process.

[0114] In particular, conditional language such as "may," "could," "may," or "might" is understood within the context to indicate that a particular example includes a particular feature, element, and / or step, while another example does not include a particular feature, element, and / or step, unless specifically stated otherwise. As such, such conditional language is not generally intended to imply that a particular feature, element, and / or step is in any way required for one or more examples, or that one or more examples necessarily include logic for determining, with or without user input or prompting, whether a particular feature, element, and / or step is included in or is subject to being performed in any particular example.

[0115] Conjunctions such as "at least one of X, Y, or Z" are understood to indicate that the item, term, etc. can be either X, Y, or Z, or any combination thereof, including multiples of each element, unless specifically stated otherwise. Unless expressly stated as singular, "a" refers to both the singular and the plural.

[0116] It should be understood that the descriptions, elements, or blocks of the routines in the flow diagrams described herein and / or depicted in the accompanying figures may represent modules, segments, or portions of code that include one or more computer-executable instructions for implementing particular logical functions or elements within the routines. Alternative embodiments are included within the scope of the examples described herein, in which elements or functions may be omitted or performed in a different order from that shown or described, including substantially synchronously, in reverse order, with additional operations, or omitting operations, depending on the functionality involved, as will be understood by one of ordinary skill in the art.

[0117] Numerous variations and modifications may be made to the examples described above, and those elements should be understood to be included among other acceptable examples. All such variations and modifications are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

1. 1. A system comprising: one or more processors; a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to: receiving a first pose graph associated with the first map data; receiving a second pose graph; determining a source node of the first pose graph associated with a map element of the first map data; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element based at least in part on the transformation as an updated map element of second map data; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; transmitting the updated map element to a vehicle based at least in part on determining that the stress is less than the threshold stress, and navigating an environment based at least in part on the updated map element; and Perform operations involving memory and A system comprising:

2. Determining the stress associated with a location within the environment comprises: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of said location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined by a loss function based at least in part on at least one of the transformation or a difference between the transformation and another transformation; the output of an optimization that transforms two or more source nodes; determining that the number of additional source nodes used to determine the modification meets or exceeds a threshold; or determining a difference between said transformation and said modification; The system of claim 1 , based at least in part on at least one of:

3. The map element is a first element, and the operation is determining a stress map based at least in part on the stress value and interpolating a second stress value based at least in part on the one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or location within the environment is associated with the second stress value; based at least in part on determining that the second stress value meets or exceeds the threshold stress; generating a notification associated with the second map element or a location within the environment; undoing a second modification associated with the second map element; or Modifying the second amendment and The system of claim 1 further comprising:

4. The said modification is determining a first weight based at least in part on a first distance between the map element and the source node; Including, modifying the map elements is based at least in part further on the first weights. The system of claim 1 .

5. The system of claim 4 , wherein determining the first weight is based at least in part on a robust loss function.

6. Determining the source node associated with the map element comprises: determining a set of source nodes that are within a predefined distance from the initial location of said map element; or determining n nearest source nodes from the initial location of the map element, where n is a positive integer; The system of claim 1 , further comprising at least one of:

7. The system of claim 1 , wherein the modification results in a change to at least one of an initial location of the map element or a magnitude size of the map element.

8. determining a second modification to the second map element based at least in part on a second transformation between the second source node and the second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; The second stress is generating a notification associated with the second map element or a location within the environment; undoing the second modification associated with the second map element; or Modifying the second amendment determining whether at least one threshold stress of The system of claim 1 further comprising:

9. receiving a user input associated with the second map element; determining changes to the second modification based at least in part on the user input; and The system of claim 8 further comprising:

10. 2. The system of claim 1, wherein the first pose graph is determined based at least in part on first sensor data received from one or more sensors in the vehicle or another vehicle, and the second pose graph is determined based at least in part on second sensor data received from the one or more sensors.

11. A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to: receiving a first pose graph associated with the first map data; receiving a second pose graph; determining a source node of the first pose graph associated with a map element of the first map data; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element based at least in part on the transformation as an updated map element of second map data; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; transmitting the updated map element to a vehicle based at least in part on determining that the stress is less than the threshold stress, and navigating an environment based at least in part on the updated map element; and A non-transitory computer-readable medium for causing operations to be performed, including:

12. Determining the stress associated with a location within the environment comprises: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of said location; determining a weight based at least in part on a distance between the current map element and the source node; a loss determined by a loss function based at least in part on at least one of the transformation or a difference between the transformation and another transformation; the output of an optimization that transforms two or more source nodes; determining that the number of additional source nodes used to determine the modification meets or exceeds a threshold; or Determining the difference between the transformation and the modification.

12. The non-transitory computer-readable medium of claim 11, wherein the method is performed at least in part based on at least one of:

13. The map element is a first element, and the operation is determining a stress map based at least in part on the stress value and interpolating a second stress value based at least in part on the one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or location within the environment is associated with the second stress value; based at least in part on determining that the second stress value meets or exceeds the threshold stress; generating a notification associated with the second map element or a location within the environment; undoing a second modification associated with the second map element; or Modifying the second amendment and 12. The non-transitory computer-readable medium of claim 11, further comprising:

14. determining a second modification of the second map element based at least in part on a second transformation between the second source node and the second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; The second stress is generating a notification associated with the second map element or a location within the environment; undoing the second modification associated with the second map element; or Modifying the second amendment determining whether at least one threshold stress of 12. The non-transitory computer-readable medium of claim 11, further comprising:

15. The non-transitory computer-readable medium of claim 11 , wherein the modification results in a change to at least one of a position of the map element or a size of the map element.

Images