Map annotation modification using sparse attitude graph node updates

Abstract

Relocating and / or resizing a map element without introducing anomalies to map data using an updated pose map may include determining a transformation between a source node of a first pose map and a target node of a second pose map, and determining a modification to the map element based at least in part on the transformation. The techniques may include determining a stress on a map based at least in part on one or more modifications to a map element, and determining whether the stress meets or exceeds a threshold. Where the stress meets or exceeds a threshold, the modification may be changed, restored, and / or indicated in a notification transmitted to a user interface.

Description

[0001] Related Applications

[0002] This PCT international application claims priority to U.S. Patent Application No. 17 / 306,499, filed on May 3, 2021, the entire contents of which are hereby incorporated by reference for all purposes. Background Art

[0003] An autonomous vehicle is capable of navigating an environment based at least in part on a map of the environment. Such an autonomous vehicle may include sensors, such as lidar sensors, that can be used to acquire data representative of the environment, which is then used in a mapping process to generate a map or update an existing map. However, errors in maps or mapping processes can create unsafe situations for systems that rely on accurate maps, such as in the case of an autonomous vehicle, for example. Furthermore, the process of updating and correcting such maps can be computationally intensive and / or require significant manual input. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The present invention is described in detail with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The same reference numbers in different drawings indicate similar or identical items.

[0005] Figure 1 An example scenario is illustrated in which an autonomous vehicle may receive sensor data associated with an environment, which may be used to generate a pose graph estimating the vehicle's position, orientation, and / or trajectory. The pose graph and the sensor data may be used to generate a map of the environment, including annotations (map elements) identifying key representations of objects / areas in the map.

[0006] Figure 2A and Figure 2B A graphical flow diagram of an example process for adjusting map element(s) using at least two sparse gesture graphs is illustrated.

[0007] Figure 3-Figure 5 Illustrated are different transformations of map elements from a simple example to a more complex example using two gesture graphs.

[0008] Figure 6 The diagram shows the Figure 2B Example stress / distortion plot generated for a portion of a .

[0009] Figure 7 Illustrated is a block diagram of an example system that includes a mapping component that can generate a gesture graph, map elements, and / or map data associated with an environment according to the techniques discussed herein. DETAILED DESCRIPTION

[0010] As discussed above, sensor data can be collected to generate a map of the environment, but adjusting the map annotations indicating the location and / or range (size / dimension) of features in the environment may be labor-intensive, error-prone, and computationally expensive. For example, one of the errors introduced by adjusting map elements may include map "tearing" or other anomalies, wherein inconsistent or "blank" spaces may be introduced into the map by automated techniques for modifying map elements. The techniques discussed herein reduce the computational load of errors, the need for manual input, and updating (one or more) map elements across the map data representing the environment, including reducing map tearing. In some examples, (one or more) map elements may be updated when new sensor data has been received from the vehicle. The new sensor data may be used to update and / or correct a posture graph that may have been previously generated by re-optimizing the posture graph using the new sensor data. In additional or alternative examples, the update to the posture graph may be determined at least in part based on the next iteration of the optimization process applied to the posture graph and / or a modification of the optimization process (e.g., changing the parameters of the optimization, such as changing the error function or constraint). More specifically, the autonomous vehicle can transmit sensor data to a remote computing device, which can use the sensor data to generate a map of the environment. Such a map may include, for example, a two-dimensional representation that identifies locations and / or areas associated with semantic tags. According to more basic techniques, humans can manually mark the locations of objects of interest in the environment, such as traffic signs, lanes, bike lanes, crosswalks, stop lines, yield lines, etc. According to the techniques discussed in this article, the perception component of the vehicle can detect object classifications associated with the sensor data, and can generate labels associated with the sensor data. The label can be used to annotate the map data as a map element, which indicates the presence of traffic signs, passenger drop-off or boarding areas, (one or more) lane indications, blind spots, intersections or other road intersections, drivable surfaces, stop lines, yield lines, etc.

[0011] The remote computing device may transmit a representation of the environment including map elements / annotations (and updated map element(s)) to the autonomous vehicle, which may use the representation to safely and efficiently navigate the environment. For example, the autonomous vehicle may use the map to determine the location of the vehicle in the environment, the location of drivable surface areas, the location of static objects in the environment, the location of drop-off locations, the location of landmarks, etc., and may use such determinations to determine trajectories for controlling the movement of the autonomous vehicle. Additionally or alternatively, the autonomous vehicle may use the map data to correctly locate a dynamic object environment, which the autonomous vehicle detects relative to key points, landmarks, static objects, etc. in the environment using sensor data received by the autonomous vehicle. In some examples, the vehicle may use the map data to determine the likelihood that an object is occluded and / or to increase or decrease the confidence that an object exists at a particular location (e.g., a dynamic object such as a pedestrian is less likely to be in the same location as a static object such as a stop sign).

[0012] In some examples, a computing device may have determined a first pose graph (a graphical representation of multiple poses as nodes and dependencies between such poses (such as constraints between such poses, as edges in the graph, etc.) by aligning a first set of sensor data captured at different vehicle poses. The aligned sensor data can be used to generate a three-dimensional model of the environment. Generating the first pose graph can include estimating a first pose (i.e., position and / or orientation) of the vehicle associated with the first set of sensor data and a relationship of the first pose to another pose of the vehicle. For example, the vehicle can receive lidar data from one or more lidar sensors associated with the vehicle. The lidar data can include a collection of data points, also referred to herein as a point cloud, where each collection of data points corresponds to a particular pose of the vehicle within the environment. In various examples, such a graph can represent connections between such poses (e.g., between subsequent poses along a trajectory) and / or links between multiple poses within one or more trajectories that have at least a portion of common visible data points.

[0013] An additional set of sensor data may be received at some point from the same or different vehicle. This second set of sensor data may be associated with the same portion of the environment and may be used to update and / or correct the first gesture graph. In some examples, a second gesture graph may be determined based at least in part on the second set of sensor data. The problem is that map elements are not so easy to update and are not automatically updated using the underlying gesture graph.

[0014] For example, a change between a first pose graph and a second pose graph may ultimately change the three-dimensional representation of the environment and / or the underlying assumptions used to determine the position of the vehicle in the environment. As a result, certain map elements (such as an indication that a portion of the map data is associated with a drivable surface) may not be correctly indicated in an updated version of the map data. In the worst case, this may result in the map data indicating a portion of the environment that is not a drivable surface as a drivable surface, which may result in damage and / or damage to the vehicle.

[0015] Technology discussed in this article can change map elements based on determining at least in part one or more nodes nearest to map elements in the first gesture graph. The node of the first gesture graph is referred to as "source node" in this article, and the node of the new gesture graph is referred to as "target node" in this article. The mapping component can determine the target node associated with the source node. In the first example, because the gesture graph has been updated according to the new optimization (or at least the iteration of a part of the optimization), the source node and the target node may be known. In such a case, the gesture graph is not a new gesture graph, but the same gesture graph through modification. The gesture node (and factor) can have a unique identification associated with it, so that the updated gesture node is easy to identify. In an alternative example, the gesture graph can be a new gesture graph, in which case, determining the target node associated with the source node can include determining the node nearest (in distance) and / or the most similar (in heading) to the source node in the second gesture graph.

[0016] In some examples, the mapping component can determine one or more source-target node pairs, where the source nodes of the corresponding pairs are within a threshold distance of the map element. In additional or alternative examples, the source-target node pairs can be determined using the closest n source-target node pairs, where n is a positive integer (e.g., independent of distance). In additional or alternative examples, a Gaussian process (such as Kriging) can use all source nodes in the map to determine the modification of the map element. Note that source-target node pairs are also referred to as node pairs in this article.

[0017] In some examples, the mapping component may determine that there are fewer than n node pairs within a threshold distance. If the number of node pairs is less than the threshold, the mapping component may generate and / or transmit a notification to a user computing device or vehicle (e.g., indicating that an error has occurred and / or there is a risk that the map element may not be accurately transformed) and / or the mapping component may retain the position of the map element from the last version of the map, or may modify the map element based at least in part on modifications to the closest m different map elements, where m is a positive integer (e.g., 1, 2, 3, 4, 5, or more - the number may depend on the proximity of the nearest map element(s), the density of the nearest map element(s), and / or the maximum distance from the map element that another map element may use).

[0018] After determining the source node-target node pair, the mapping component can determine a transformation between the source node and the target node. For example, the mapping component can determine the difference between the source node and the target node and / or a function that will cause the vehicle to move from the source node to the target node. The transformation can include, for example, a vector indicating the difference between the source node and the target node. In some examples, the mapping component can repeatedly determine the transformation of the source-target node pair closest to the map element until a number of transformations equal to the number of pairs has been determined.

[0019] In some examples, the mapping component can determine a weight associated with a source-target node pair based at least in part on the distance between the source node of the pair and the map element location. In some examples, the weight can decrease as the distance increases. Various loss functions can be used to determine the weight, and in some examples, a weight can be determined for each source-target node pair in the source-target node pair closest to the map element. In some examples, the weights associated with these nearest source-target node pairs can be standardized to sum to 1.

[0020] The mapping component may determine the change to be applied to the map element based at least in part on the transformation and weight associated with the source-target node pair. For example, the change itself may be a transformation that changes the position and / or range of the map element. In some examples, the mapping component may determine the change (e.g., the nearest n node pairs, the node pairs with a threshold distance, or a combination thereof) based at least in part on the weighted sum of the transformations. In additional or alternative examples, the change to the map element may be part of the optimization of multiple map elements including the map element. For example, the change to the map element may be based at least in part on changing the map element (and one or more other map elements) to reduce the difference between the weighted sum of the change to the map element and the transformation associated with the map element, as determined above. For other map elements, the corresponding change may be based at least in part on determining the total difference by summing these differences on different map elements (and corresponding weighted transformations), and the corresponding change may be changed to reduce the total difference.

[0021] Regardless, the resulting (changed) map element may be an updated map element contained in the second (updated) map data. The updated map data may be transmitted to the vehicle (or another vehicle) for use by the vehicle to control the operation of the vehicle. The techniques discussed herein may require fewer computing resources, improve the accuracy of map element locations and resulting map data, and improve the safety of (one or more) vehicles using the map data by correctly locating map elements such as drivable surfaces (e.g., roads and sidewalks), stop signs and / or stop lines, etc.

[0022] Example scenario

[0023] Figure 1 An example scenario 100 including a vehicle 102 is illustrated. In some cases, the vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification published by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout a journey where the driver (or occupant) is not expected to control the vehicle at any time. However, in other examples, the vehicle 102 may be a fully or partially autonomous vehicle having any other level or classification. It is contemplated that the techniques discussed herein may be applied to areas other than robotic control for autonomous vehicles. For example, the techniques discussed herein may be applied to mining, manufacturing, augmented reality, and the like. Furthermore, even though the vehicle 102 is depicted as a land vehicle, the vehicle 102 may also be a spacecraft, a ship, and / or the like.

[0024] According to the techniques discussed herein, the vehicle 102 may receive sensor data from the sensor(s) 104 of the vehicle 102. For example, the sensor(s) 104 may include a position sensor (e.g., a global positioning system (GPS) sensor), an inertial sensor (e.g., an accelerometer sensor, a gyroscope sensor, etc.), a magnetic field sensor (e.g., a compass), a position / velocity / acceleration sensor (e.g., a speedometer, a drive system sensor), odometer data (which may be determined based at least in part on inertial measurements and / or an odometer of the vehicle 102), a depth position sensor (e.g., a lidar sensor, a radar sensor, a sonar sensor, a time-of-flight (ToF) camera, a depth camera, an ultrasonic and / or sonar sensor, and / or other depth sensing sensor), an image sensor (e.g., a camera), an audio sensor (e.g., a microphone), and / or an environmental sensor (e.g., a barometer, a hygrometer, etc.).

[0025] The sensor(s) 104 may generate sensor data that may be received by computing device(s) 106 associated with the vehicle 102. However, in other examples, some or all of the sensor(s) 104 and / or computing device(s) 106 may be separate and / or located remotely from the vehicle 102, and data capture, processing, commands, and / or control may be transmitted to / from the vehicle 102 by one or more remote computing devices via a wired and / or wireless network.

[0026] The computing device(s) 106 may include a memory 108 for storing a sensing component 110, a planning component 112, a mapping component 114, and / or a system controller 116. Figure 1 1 and 12. The mapping component 114 is described for illustration purposes only, but it is understood that the mapping component 114 can be located in / on a different computing device (or other location) than one or more of the other components.

[0027] In general, perception component 110 can determine what is in the environment around vehicle 102, and planning component 112 can determine how to operate vehicle 102 based on information received from perception component 110. For example, planning component 112 can determine trajectory 118 based at least in part on perception data and / or other information such as, for example, one or more maps, such as map data determined according to the techniques discussed herein, positioning information (e.g., the location of vehicle 102 in the environment relative to the map and / or features detected by perception component 110), etc. In some examples, perception component 110 can include a pipeline of hardware and / or software, which can include one or more GPUs, ML models, Kalman filters, etc.

[0028] For example, the map data may include map elements indicating various key objects and / or areas in the environment, such as the location (and / or extent - i.e., shape / size) of important objects or areas in the environment. These (one or more) map elements may include, for example, annotations indicating a portion of the map data, such as two-dimensional representations of the environment associated with a drivable surface, a pedestrian condition surface (e.g., a crosswalk), a permanent pedestrian or bicycle lane, a parking surface, a traffic sign (e.g., a stop light, a stop sign, a yield sign, a speed limit sign, a commuter lane, a construction sign), a traffic indication (e.g., a stop line, a yield line, a crosswalk indication, a commuter lane indication, a traffic cone, a signal light), various drop-off or boarding areas, etc. In other words, whether the map data is two-dimensional or three-dimensional, the map indicates the importance of a portion of the map data. Additional map data may be generated, which may include a three-dimensional representation of the environment, such as a grid, a wireframe model, a polygon, a facet (e.g., a polygon associated with an individual color and / or intensity), etc.

[0029] In some examples, trajectory 118 may be dependent upon one or more of the map elements identified in the two-dimensional map. For example, a trajectory generated for the vehicle may be based at least in part on traffic signs and may be limited by portions of the map data indicated (by the map elements) as drivable surfaces, except in emergency situations or pickup / parking scenarios, where the vehicle may generate a trajectory that will cause the vehicle to move into portions of the environment indicated in the map data as being associated with bicycle lanes, parking areas, pickup / drop-off areas, etc.

[0030] The trajectory 118 may include instructions for the controller(s) 116 to activate the drive components of the vehicle 102 to achieve a steering angle, steering rate, acceleration, etc., which may result in a vehicle position, a vehicle speed, and / or a vehicle acceleration. For example, the trajectory 118 may include a target heading, a target steering angle, a target steering rate, a target position, a target speed, and / or a target acceleration tracked by the controller(s) 116. In some examples, the trajectory 118 may be associated with controls sufficient to control the vehicle 102 within a time range (e.g., 5 milliseconds, 10 milliseconds, 100 milliseconds, 200 milliseconds, 0.5 seconds, 1 second, 2 seconds, etc.) or a distance range (e.g., 1 meter, 2 meters, 5 meters, 8 meters, 10 meters). In some examples, the controller(s) 116 may include software and / or hardware for activating the drive components of the vehicle 102 sufficient to track the trajectory 118. For example, the controller(s) 116 may include one or more proportional-integral-derivative (PID) controllers.

[0031] In some examples, perception component 110 can receive sensor data from sensor(s) 104 and determine data related to objects near vehicle 102 (e.g., classifications related to detected objects, instance segmentation(s), semantic segmentation(s), two-dimensional and / or three-dimensional bounding boxes, trajectories), route data specifying a vehicle destination, global map data identifying features of a road (e.g., features detectable in different sensor modalities that can be used to locate an autonomous vehicle), a pose of the vehicle (e.g., a position and / or orientation in an environment, which can be determined by or coordinated with a localization component), local map data identifying features detected near the vehicle (e.g., locations and / or sizes of buildings, trees, fences, fire hydrants, stop signs, and any other features detectable in various sensor modalities), and the like.

[0032] Specifically, the perception component 110 may determine an object detection based at least in part on the sensor data, the object detection indicating the association of a portion of the sensor data with an object in the environment. The object detection may indicate an object classification, a region of interest (ROI) identifying a portion of the sensor data associated with the object, and / or a confidence score indicating the likelihood (e.g., a posterior probability) that the object classification is correct and / or a confidence score that the ROI is accurate. For example, the ROI may include a portion of an image or radar data identified as being associated with an object by an ML model or ML pipeline of the perception component 110, such as using a bounding box, mask, instance segmentation, and / or semantic segmentation. The object classification determined by the perception component 110 may distinguish between different object types, such as, for example, a passenger car, a pedestrian, a bicycle, a delivery truck, a semi-trailer truck, a traffic sign, and the like. In some examples, object detections may be tracked over time. For example, a trajectory may associate two object detections generated at two different times as being associated with the same object, and may include the object's historical, current, and / or predicted object position, velocity, acceleration, and / or heading.

[0033] The data generated by the perception component 110 may be collectively referred to as perception data. Once the perception component 110 has generated the perception data, the perception component 110 may provide the perception data to the planning component 112. The perception data may be stored as log data in addition or in lieu of the sensor data. The log data may be transmitted to the mapping component 114, which may be on the computing device(s) 106 and / or a remote computing device (for clarity, in Figure 1 Executed at (not shown).

[0034] The planning component 112 can use the sensory data received from the sensing component 110 to determine one or more trajectories, control the movement of the vehicle 102 to traverse a path or route, and / or otherwise control the operation of the vehicle 102, although any such operations can be performed in various other components (e.g., positioning can be performed by a positioning component, which can be based at least in part on the sensory data). For example, the planning component 112 can: determine a route for the vehicle 102 from a first location to a second location; generate a plurality of potential trajectories based substantially simultaneously and at least in part on the sensory data and / or simulated sensory data (which can further include predictions about objects detected in such data) for controlling the movement of the vehicle 102 according to a receding horizon technique (e.g., 1 microsecond, half a second) to control the vehicle to traverse the route (e.g., to avoid any of the detected objects); and select one of the potential trajectories as a trajectory 118 for the vehicle 102, which can be used to generate a drive control signal that can be transmitted to a drive component of the vehicle 102. Figure 1 An example of such a trajectory 118 is depicted, represented as arrows to indicate heading, speed, and / or acceleration, but the trajectory itself may include instructions for controller(s) 116 , which in turn may activate the drive system of vehicle 102 .

[0035] In the depicted example, the perception component 110 can detect object detections that can be used to generate a map element 122 based at least in part on sensor data (such as lidar data 120, although additional or alternative types of sensor data can be used). The vehicle 102 can detect the object detections to generate the map element 122 as the vehicle executes the trajectory 118 and traverses this portion of the environment. In some examples, determining the location and / or range of the map element 122 can be based at least in part on the perception component 110 generating an object detection associated with a portion of the environment, which will change to the map element 122 from different poses of the vehicle 102 as the vehicle 102 traverses the environment. For example, the vehicle 102 can generate a first object detection associated with the map element 122 at a pose 124 and a second object detection associated with the map element 122 at a pose 126. The pose can include a heading and a position. For example, the pose 128 includes a position 130 and a heading 132. The mapping component 114 can use the position and heading of the vehicle indicated by the pose determined according to the positioning technology to determine the relative heading and position of the object detection. The mapping component 114 can estimate the location of objects in the environment, such as estimating the location and / or range of the map element 122 using detections related to the map element 122 generated by the perception component 110 when the vehicle 102 is at the pose 124 and the pose 126 .

[0036] In some examples, mapping component 114 can determine a pose graph including one or more poses estimated by the vehicle as part of a simultaneous localization and mapping (SLAM) technique performed by a localization component of the mapping component. The SLAM technique can include a Bayesian filter (such as a Kalman filter), a bundle adjustment, a maximum a posteriori estimate (MAP), and / or any simultaneous localization and mapping (SLAM) algorithm.

[0037] Estimating a posture node associated with the posture of the vehicle may include modifying a first estimated posture to increase the fit between a subset of lidar points in a first set of lidar points and a subset of lidar points in a second set of lidar points associated with a second estimated posture / reducing the distance between them. The factors (constraints) between the two postures may also be modified to fit the sets of sensor data together (e.g., by reducing the distance between the lidar points of the two sets). The factors between the two estimated postures may be constraints that identify how the vehicle moves from one posture to another. In some examples, the estimated postures and the factors between them may be based at least in part on sensor data (e.g., lidar data) and / or auxiliary sensor data, such as inertial measurement unit (IMU) sensor data, odometer data (e.g., which may be based at least in part on data received from wheel encoders or other sensors), etc. For example, the vehicle may use IMU data and / or odometer data to instantiate a posture estimate and / or a factor between two posture estimates, and may use lidar data to refine these initial estimates. The resulting posture graph may be a graph whose nodes correspond to the postures of the vehicle at different points in space and whose edges represent constraints between postures. The latter are obtained from observations of the environment or from movement actions performed by the robot.

[0038] Note that as used herein, a posture graph may include a factor graph including posture nodes and edges (factors / constraints), wherein a node indicates a posture of a vehicle (i.e., a position and orientation (e.g., two-dimensional or three-dimensional heading) of the vehicle within an environment / map), and an edge (also referred to as a factor) indicates a set of constraints defining the relationship of one posture (node) to another posture (node). In some examples, confidence scores (such as covariances) may be associated with edges. In some examples, a vehicle may determine a posture graph based at least in part on sensor data (including odometer data and / or inertial data). In some examples, a posture node may be associated with sensor data received by the vehicle within a time window.

[0039] In some examples, the factors / constraints between two pose nodes (in Figure 1In some examples, a first confidence score may be associated with a constraint, indicating the likelihood that the constraint is correct, and a second confidence score may be associated with the association of two pose nodes, indicating the likelihood that the two pose nodes are associated with sensor data collected at those nodes, and the sensor data indicates that at least a portion of the environment is the same. In some examples, this association may be a closed-loop factor, although this may not be the case in other examples, such as where the association factor merely identifies that the two poses are close enough to be associated with sensor data including the same object detection.

[0040] In some examples, mapping component 114 can determine a preliminary pose graph that vehicle 102 can transmit to a remote computing device. In some examples, vehicle 102 can transmit (one or more) object detection and / or sensor data to a remote computing device in association with the preliminary pose graph. In additional or alternative examples, the map data can be generated at a remote computing device completely at least in part based on sensor data received from vehicle 102. In some examples, mapping component 114 at vehicle 102 can be different from mapping component at a remote computing device. For example, mapping component 114 at vehicle can include a component for locating vehicle 102 within an environment based on a map (e.g., mapping component 114 can include a SLAM component). Additionally or alternatively, mapping component 114 can include a component that generates a preliminary pose graph based at least in part on sensor data collected by the vehicle as part of a SLAM operation. The preliminary pose graph can be transmitted to a remote computing device, which can refine the pose graph into a final pose graph for generating map data. In some examples, refining the posture graph can be based at least in part on sensor data received from multiple passes of the same location, whether from the same vehicle or multiple vehicles. This new sensor data can be used to re-determine the optimization of the posture graph, resulting in an update / correction to the posture graph. Note that the posture graph nodes are the same nodes and may only change based on the re-optimization of the posture graph.

[0041] For example, refining the preliminary posture graph into a first posture graph may include changing the posture nodes based at least in part on one or more factors associated with the posture nodes and the covariance associated therewith to reduce the distances (i.e., hard constraints) between points in the first type of sensor data associated with the same object based on at least one iteration as part of the optimization of the posture graph (regardless of whether a truly optimal posture graph is ultimately determined). The optimized iteration may include adjusting the posture nodes and / or factors of the posture graph to reduce the loss determined for the entire posture graph or for at least one region of the posture graph. The iteration may be based on new sensor data in addition or alternatively. The final posture graph determined by the remote computing device may be used to determine that different sets of sensor data associated with different nodes are associated with the same object, and may be used to generate a map. The map may include a representation of a surface in the environment, such as a two-dimensional and / or three-dimensional model, which may include a polygonal mesh, a wireframe model, etc. to represent the surface. For example, but not limited to: Figure 1 The representation 134 is depicted as a wireframe model, but it will be appreciated that the map may additionally or alternatively include a two-dimensional map, such as in Figure 1 A top-down view illustrated in , with semantic labels associated with locations and / or areas in the map associated with various parts of the environment.

[0042] However, although refinements of the gesture graph(s) used to generate map data may be used to generate such representations 134, map elements indicating key objects and areas in the environment are not easily modified. Figure 1 When correction 136 of a pose graph illustrated as (one or more) updated poses 138) and / or factors / trajectories therebetween is performed, the techniques discussed herein may be used to determine updated map elements 140, as discussed further below.

[0043] Example Process

[0044] Figure 2A and Figure 2BA graphical flow chart of an example process 200 for adjusting map elements using at least two sparse pose graphs is illustrated. Specifically, the example process 200 can be used when an updated pose graph has been generated or received. In such examples, a preliminary pose graph may have been determined, whether it is a preliminary pose graph determined by the vehicle (such as based on SLAM techniques performed by a vehicle computing device), or simply a previous version of the pose graph. In some examples, the pose graph can be a sparse graph that is determined at least in part based on various sensor data (e.g., the pose graph is adjusted by an optimization algorithm to increase the degree of alignment of spatial and / or other types of sensor data based on constraints determined at least in part based on odometer data and / or inertial data). The pose graph may be sparse in that the number of nodes may be less than the set of sensor data.

[0045] For example, 100 units of odometer data, 500 units of inertial data, and 30 units of lidar data may be captured within the same distance or time period (e.g., 1 meter or 1 second). In at least one example, and depending on the sampling rate associated with the pose graph, the portion of the pose graph associated with the data collected within 1 meter or 1 second may be represented as a single pose node. In some examples, the updated pose graph may be a new version of a preliminary pose graph generated as at least part of an optimization or re-optimization of a pose graph, such as the next iteration of the optimization, a re-optimization based on new sensor data, and / or a re-optimization after an optimization parameter has been changed, such as an optimization constraint, a loss function, etc.

[0046] At operation 202, the example process 200 may include receiving a first pose graph associated with first map data according to any of the techniques discussed herein. The first pose graph may be a preliminary pose graph, whether the preliminary pose graph is a pose graph determined by the vehicle 102 or a previous version of a pose graph refined by a remote computing device. The first map data may include a representation of an environment generated at least in part based on the first pose graph and / or map elements, the map elements identifying a portion of the environment as associated with a key object or area, such as a traffic sign (and type, such as a stop sign, a stop light, a yield sign, a speed limit sign), a static object (e.g., a fire hydrant, a mailbox, a light pole), a key area (e.g., a drivable area, a parking area, a lane, a turn lane, an exit lane, a merge lane, a sidewalk, a crosswalk, a pedestrian zone, a pickup and / or drop-off zone, a no idling zone), etc. Figure 2AA top-down view of a scene 204 including a straight road and an example representation of a first pose graph are depicted. The first pose graph (or portion thereof) is depicted as a solid circle (position) and a solid arrow (heading), and map element positions and ranges are depicted as map elements 206. Note that the first pose graph may include additional data, as discussed above, and that map elements 206 may have been previously labeled using perception data generated by a vehicle or a human.

[0047] At operation 208, example process 200 may include receiving a second (updated) pose graph according to any of the techniques discussed herein. For example, the remote computing device may refine the preliminary pose graph or determine a new pose graph based at least in part on a refinement technique, a modification to the optimization algorithm, a next iteration of the optimization, and / or new sensor data. Figure 2A The second pose graph is depicted as a dashed circle (position) and a dashed arrow (heading), although the second pose graph may include additional data, as discussed above.

[0048] At operation 210, the example process 200 may include determining one or more source nodes of a first gesture graph associated with a map element according to any of the techniques discussed herein. The source node may be determined from the first gesture graph. For example, operation 210 may include determining the n nodes closest to the map element 206 in the first graph, where n is a positive integer. In some examples, n may be a static number, but in additional or alternative examples, n may be determined based at least in part on the node density of the gesture graph. In some examples, a mapping component (of a vehicle or remote computing device) may limit the search for the (one or more) nearest nodes to those first gesture graph nodes within a threshold distance 212 of the map element 206, thereby ensuring that corrections to distant nodes will not affect the position and / or range of the map element 206. In some examples, operation 210 may include sorting the nodes within a threshold distance 212 of the map element according to the minimum to maximum distance and selecting the first n (n nearest) nodes closest to the map element. In additional or alternative examples, operation 210 may include determining the first gesture graph gesture node closest to the map element and determining the n-1 first gesture graph gesture nodes closest to the nearest node. Additionally or alternatively, operation 210 may include determining the nearest n source node-target node pairs, source node-target node pairs, and / or combinations thereof within distance 212 .

[0049] At operation 214, the example process 200 may include determining whether the number of source nodes (or source-target node pairs) identified at operation 210 meets or exceeds a lower threshold number p, where p is a positive integer less than n, according to any of the techniques discussed herein. If the number of source nodes identified at operation 210 does not meet the lower threshold number (e.g., there are no source nodes within a threshold distance 212 of the map element 206, or the number of source nodes within the distance 212 is less than the threshold number), the example process 200 may proceed to operation 216. If the number of source nodes identified in operation 210 meets or exceeds the lower threshold number, the example process 200 may proceed to operation 218. In some examples, operation 214 may ensure that there is a sufficiently reliable number of source nodes available to determine an accurate modification to the location and / or range of the map element. In some examples, operation 214 may be skipped.

[0050] At operation 216, the example process 200 may include generating a notification, saving the map element, and / or modifying the map element 206 based at least in part on the second modification of the second map element according to any of the techniques discussed herein. In some examples, operation 216 may include generating and / or transmitting a notification to a user computing device indicating the map element 206 and / or a portion of the gesture graph that is closest to the map element 206. For example, the notification may include an indication that the map element 206 may be inaccurate and / or may include executable code to present a user interface option for providing a manual modification of the map element 206. In some examples, the indication that the map element 206 may be inaccurate may be included in an updated map associated with the map element 206, such that a planning component of the vehicle may use the indication to determine a trajectory in an area near the map element 206. In some cases, this may result in an increased likelihood that the vehicle will deviate from the trajectory of the map element 206.

[0051] At operation 218, the example process 200 may include determining one or more target nodes (nodes of the second posture graph) associated with one or more source nodes according to any of the techniques discussed herein. In some examples, operation 218 may include identifying an updated version of a source node based on a node identifier that matches between the source node and the target node. For example, each node may be associated with a posture node identifier and a timestamp. Determining the target node may include determining an updated version of a posture node having the same posture node identifier and timestamp. This may be the case in examples where the updated posture graph is a new version of the same posture graph, such as when new sensor data is received and the updated posture graph is received as at least part of a re-optimization of the posture graph, when optimization parameters have been changed (e.g., changing constraints between nodes, changing distances 212, Changing the number of nearest nodes n used, replacing one type of solution Devices and / or similar ), the next iteration of the optimization process occurs, and so on.

[0052] In additional or alternative examples, operation 218 is performed when a completely new posture graph has been received. In such an example, for a first source node (such as source node 220(a)), operation 218 may include determining the node in the second posture graph that is closest to the first source node, i.e., target node 220(a). In additional or alternative examples, the corresponding nodes may already be known, such as based on a technique for determining a posture graph using the same or similar posture graph generation parameters (such as the same sampling rate, starting position and / or ending position, and some specification indicating that the posture nodes are located at the same location in the environment) (e.g., using GPS data or a test route where the vehicle travels along exactly the same path). In the depicted example, three source nodes 220(a)-(c) may have been determined at operation 210, and three corresponding target nodes 222(a)-(c) may be determined at operation 218. Figure 2A Which nodes are associated with the line between the source-target node pairs is further depicted by using matching sub-letters in the numbering (e.g., the first source-target node pair includes source node 220(a) and target node 222(a), and the second source-target node pair includes source node 220(b) and target node 222(b)).

[0053] Go to Figure 2B At operation 224, the example process 200 may include determining a modification 226 based at least in part on one or more source nodes and one or more target nodes (the one or more source-target node pairs determined at operations 210 and 218) according to any of the techniques discussed herein. Operation 224 may include operation 224(a) and optional operation 224(b). In some examples, operations 224(a) and / or 224(b) may be repeated for all source-target node pairs determined at operations 210 and 218 (i.e., those source-target node pairs determined to be associated with map element 206).

[0054] At operation 224(a), the example process 200 may include determining a first transformation 228 between a first source node 222(a) and a first target node 222(a) according to any of the techniques discussed herein. In some examples, determining the first transformation 228 (depicted as a line between the source node 22(a) and the target node 222(a)) may include determining a difference in position, heading, and / or constraints between the pair of nodes. In some examples, the transformation may be indicated as a transformation function, such as a linear transformation, a geometric transformation, and / or a nonlinear transformation. In some examples, the transformation may be indicated as a tensor identifying three or six degrees of freedom (e.g., x-, y-, z-, yaw, roll, pitch transformation).

[0055] At operation 224(b), the example process 200 may include determining a weight associated with a first transformation based at least in part on a distance between the map element 206 and the first source node 220(a), according to any of the techniques discussed herein. As the distance from the map element 206 to the source-target node pair increases, the weight reduces the effect of the transformation determined for the source-target node pair on the (final) modification 226. For example, a weight associated with the first transformation 228 may be determined based at least in part on a distance 230 between the first source node 220(a) and the map element 206. The modification 226 may be determined based at least in part on determining a weighted sum of the transformations determined for the source-target node pair. In some examples, the weight may be determined using a first-order, second-order, or higher-order function. For example, the modification 226, M may be given by the following formula:

[0056] M=∑f(d i )T i (1)

[0057] Among them, f is the weighting function, d i is the distance from map element 206 to the i-th source node in the set of source-target node pairs, and T i is the transformation of the ith source node to the ith target node. As an example, a first-order weighting function may include:

[0058]

[0059] Among them, d 阈值 is the threshold distance 212, and "norm" is the normalization of all weights determined for a node pair (e.g., such that the sum of weights associated with different node pairs is 1). Note that if a pose node happens to be located at the location of map element 206, the normalization and weighting will result in the pose node located at the location of map element 206 primarily or entirely influencing the modification of the location of map element 206. As an additional or alternative example, a second order weighting function may include:

[0060]

[0061] And inverse functions can include:

[0062]

[0063] Any additional or alternative weighting function can be used. The optimal weighting function will show smoothness and continuity in the determined weights, although this is not necessary because the inverse function is suitable, but may have corners / discontinuities. In some examples, a robust loss function can be used to determine the weights, such as Cauchy loss, pseudo-Huber loss, Blake-Zisserman loss, Tukey loss, German-McClure loss, Welsch loss, etc. In such examples, a robust loss function can help ensure that outlier transformations are ignored or their impact on the final transformation is minimized. In some examples, the loss can be iteratively determined because the iteration of the transformation is determined as part of the optimization (e.g., determining weights, determining residuals / errors in the area of ​​the map, changing modifications to reduce residuals, re-determining weights, re-determining residuals, etc., until the loss converges). In some examples, the loss function can reduce the impact of abnormal transformations or functionally eliminate such abnormal transformations.

[0064] If weighting is not used, the transformations associated with the node pair may be averaged or otherwise combined to determine the modification 226. In examples where weights are determined, the modification 226 may be determined based at least in part on determining a weighted sum of the transformations determined at operation 224(a).

[0065] At operation 232, the example process 200 may include modifying the map element to an updated map element 234 of the second map data based at least in part on the modification according to any of the techniques discussed herein. In some examples, operation 232 may include moving and / or updating the extent of the map element 206, thereby producing an updated map element 234. Updating the extent of the object may include converting the extent to points (e.g., sampling the edge of an area defined as associated with the map element), modifying the points according to a transformation (as discussed above), and fitting a polygon or other shape to the resulting modified points - the polygon or other shape defining a new extent of the map element.

[0066] At operation 236, the example process 200 may optionally include determining a transformation stress according to any of the techniques discussed herein. In some examples, the stress may be determined in association with a modification (such as modification 226). Additionally or alternatively, the stress may be determined for multiple modifications of map elements on an area. In some examples, determining the stress associated with the modification may include determining a residual associated with the modification (e.g., a loss determined by a loss function). In some examples, the residual may be based at least in part on the same or similar elements on which the weight determined at operation 224 (b) is based. In additional or alternative examples, determining the stress may be based at least in part on a transformation of the source node itself. For example, the transformation may be converted from three or six dimensions to a single number, such as by determining the maximum value of the transformation (in any dimension), the mean of the transformation values, the standard deviation of the transformation (and / or nearby transformations within a threshold distance). In additional or alternative examples, the stress may be the transformation itself. The stress at a location can additionally or alternatively be based at least in part on the number of source nodes on which the modification is based (and whether that number meets or exceeds a threshold number), the variance of the transformation in an area surrounding the location (e.g., within the same grid cell, within a threshold distance of the location, etc.), determining the difference between the transformation (source node to target node) and the modification to the map element (e.g., which may result in a vector or an average value may be determined at least in part based on the vector), or a combination of any of the factors discussed.

[0067] In some examples, the stress of a region may be determined by determining the stress of individual transformations in the region and determining the distribution and / or variance associated therewith. Whether determining individual stresses or determining stresses associated with a region, a map may be generated in association with the stresses by associating the determined stresses with locations associated with the corresponding source nodes that generated them. Additionally or alternatively, for locations that do not have a source node associated therewith, the value of the stress may be determined by interpolating the value of the stress based at least in part on the stress of one or more nearby source nodes previously determined. In some examples, a visual representation of the stress may be generated based at least in part on the magnitude of the stress based at least in part on color or gray coding the stress. For stresses that meet or exceed a threshold stress, the example process 200 may continue to operation 240. For stresses that are below a threshold stress, the example process 200 may continue to operation 242.

[0068] In some examples, operation 236 may additionally or alternatively include determining whether map element 206 is modified similarly to (one or more) source nodes and / or (one or more) other map elements near map element 206 according to any of the techniques discussed herein. Operation 236 can ensure that map element 206 is accurately updated and is not updated based on outliers or software errors. Determining similarity may include determining the difference between the modification to map element 206 and the transformation of the source node or the modification to a different map element. In some examples, the difference may be indicated as a vector and / or heading and magnitude, a tensor, etc. In some examples, determining whether modification 226 is similar to other modifications or transformations may include determining whether the difference is less than a threshold. In some examples, the total difference may be determined by averaging the difference between modification 226 and multiple other transformations and / or modifications.

[0069] At operation 238, the example process 200 may include determining whether the stress meets or exceeds a threshold stress. If the difference is greater than the threshold, the example process 200 may proceed to operation 240 ("yes" branch). If the difference is less than or equal to the threshold, the example process 200 may proceed to operation 242 ("no" branch). Determining whether the source node(s) and / or the other map element(s) are "nearby" may include determining whether the source node(s) and / or the other map element(s) are less than a threshold distance from the map element 206. In some examples, the threshold distance may be greater than the threshold distance 212, although in some examples, it may be equal to or less than the threshold distance 212.

[0070] At operation 240, the example process 200 may include generating and / or restoring modifications or modifying the modifications based at least in part on modifications to one or more other map elements. For example, for map elements associated with an area of ​​an environment associated with a stress value that meets or exceeds a threshold stress, a notification may be presented via a user interface for manual changes and / or review modifications. In some examples, stresses (one or more) having values ​​above a stress threshold, source nodes having stresses associated therewith that meet or exceed a threshold stress, or source nodes (from which stress values ​​are inserted) in which stresses meet or exceed a threshold stress may be color-coded using different colors to draw attention to that portion of the map. Additionally or alternatively, modifications may be restored. The restoration may occur automatically (without user input). In some examples, automatic modifications may generate notifications that may be presented via a user interface. Additionally or alternatively, restoration may occur in response to user input authorizing restoration.

[0071] In additional or alternative examples, the system may change the modification based at least in part on determining that the stress associated with the modification meets or exceeds a stress threshold. Such a change may be based at least in part on modifications to the most recent q other map elements that have been converted, where q is a positive integer. A new modification may be generated based at least in part on modifications to other map elements using a process similar to operation 210, operation 214, and / or operation 224. For example, the most recent q map elements within a threshold distance of map element 206 may be determined (similarly to determining whether there are a sufficient number or sufficiently close (one or more) map elements), and weights may be determined in association with how these map elements are modified based at least in part on the distance between map element 206 and (one or more) corresponding map elements. These weights may be used to determine a new modification to map element 206 by determining a weighted sum of the modifications.

[0072] For example, determining the change may include determining an interpolation of modification(s) to the most recent q other map elements, an average modification to the most recent q other map elements, and the like.

[0073] At operation 242, the example process 200 may transmit the second map data including the updated map elements to the vehicle or another vehicle according to any of the techniques discussed herein. For example, Figure 1 The map data is depicted as a three-dimensional wireframe model representation 134, although other examples are contemplated. In some examples, the map data can be transmitted to a fleet of vehicles. The vehicles can use the map data as part of SLAM operations to locate the vehicle within the environment, drive carefully in areas of the environment that include blind corners or other unusual conditions, etc.

[0074] Example Transformations and Modifications

[0075] Figure 3-5 The diagrams illustrate examples of source node-target node pairs, transformations therebetween, and resulting modifications determined for (one or more) map elements associated with a gesture graph according to the techniques discussed herein. The examples increase in complexity and the number of modified map elements. Notably, the diagrams illustrate Figure 3 2D transformation and final modification of map elements in Figure 4 and Figure 5 3D transformation and final 3D modification of map elements in . It is also worth noting that the local transformation of the source-target node pair has a regional effect on the local map elements. Distant node pairs do not affect distant map elements. Figure 3-Figure 5Source nodes of the first (preliminary or previous) pose graph are depicted as squares, target nodes of the second (updated) pose graph are depicted as circles, map elements are depicted as stars, updated map elements are depicted as triangles, and transformations between nodes / modifications between map elements are depicted as lines.

[0076] Example Stress Plot

[0077] Figure 6 The diagram shows the Figure 2B . The depiction of the stress map includes a transformation depicted in the form of lines that overlay a satellite image of an environmental area. Before modification, a grid can cover the area to be updated, and the grid can be distorted according to the transformations and / or modifications discussed above. For example, the grid can be generated in 5 meter by 5 meter squares or according to any other division. Figure 6 Included are multiple transformations, such as transformation 602, where no transformation occurs and the grid therefore does not appear distorted - the grid still appears irregular / square in such areas. In instances where the modification is larger, the grid may appear more distorted, such as at transformation 604 and / or transformation 606. Figure 6 A grayscale depiction of stress values ​​is also included, with darker encoding to indicate higher stress and lighter encoding to indicate lower stress. The ambient region around transitions 604 and 606 includes dark spots indicating high stress values ​​(e.g., stress values ​​that meet or exceed a threshold). In some examples, the stress values ​​can be thresholded so that only stress values ​​that meet or exceed a threshold stress are depicted. Figure 6 depicts such a situation, which is why the satellite imagery shows it at multiple points. Note that although Figure 6 Represented in grayscale, but stress maps can be color coded.

[0078] Example System

[0079] Figure 7 A block diagram of an example system 700 that implements the techniques discussed herein is illustrated. In some cases, the example system 700 may include a vehicle 702, which may represent Figure 1 102 in the vehicle 702. In some cases, the vehicle 702 can be an autonomous vehicle that is configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle that is capable of performing all safety-critical functions throughout a journey where the driver (or occupant) is not expected to control the vehicle at any time. However, in other examples, the vehicle 702 can be a fully or partially autonomous vehicle with any other level or classification. In addition, in some cases, the techniques described herein can also be used by non-autonomous vehicles.

[0080] The vehicle 702 may include vehicle computing device(s) 704, sensor(s) 706, transmitter(s) 708, network interface(s) 710, and / or drive component(s) 712. The vehicle computing device(s) 704 may represent computing device(s) 106, and the sensor(s) 706 may represent sensor(s) 104. The system 700 may additionally or alternatively include computing device(s) 714.

[0081] In some cases, the sensor(s) 706 may represent the sensor(s) 104 and may include a lidar sensor, a radar sensor, an ultrasonic transducer, a sonar sensor, a position sensor (e.g., a global positioning system (GPS), a compass, etc.), an inertial sensor (e.g., an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, etc.), an image sensor (e.g., red green blue (RGB), infrared (IR), intensity, depth, time of flight camera, etc.), a microphone, a wheel encoder, an environmental sensor (e.g., a thermometer, a hygrometer, a light sensor, a pressure sensor, etc.), etc. The sensor(s) 706 may include multiple instances of each of these or other types of sensors. For example, the radar sensor may include individual radar sensors located at the corners, front, rear, sides, and / or top of the vehicle 702. As another example, the camera may include multiple cameras disposed at different locations around the exterior and / or interior of the vehicle 702. The sensor(s) 706 may provide input to the vehicle computing device(s) 704 and / or the computing device(s) 714.

[0082] The vehicle 702 may also include (one or more) emitters 708 for emitting light and / or sound. The (one or more) emitters 708 in this example may include (one or more) internal audio and visual emitters to communicate with passengers of the vehicle 702. By way of example and not limitation, the (one or more) internal emitters may include speakers, lights, signs, display screens, touch screens, (one or more) tactile emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.), etc. The (one or more) emitters 708 in this example may also include (one or more) external emitters. By way of example and not limitation, the (one or more) external emitters in this example include lights for signaling direction of travel or other indicators of vehicle motion (e.g., indicator lights, signs, light arrays, etc.), and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audible communication with pedestrians or other nearby vehicles, one or more of which include acoustic beam steering technology.

[0083] The vehicle 702 may also include a network interface 710 that enables communication between the vehicle 702 and one or more other local or remote computing devices. For example, the network interface(s) 710 may facilitate communication with other local computing devices(s) on the vehicle 702 and / or the drive assembly(s) 712. Likewise, the network interface(s) 710 may additionally or alternatively allow the vehicle to communicate with other nearby computing devices(s) (e.g., other nearby vehicles, traffic lights, etc.). The network interface(s) 710 may additionally or alternatively enable the vehicle 702 to communicate with a computing device 714. In some examples, the computing device(s) 714 may include one or more nodes of a distributed computing system (e.g., a cloud computing architecture).

[0084] The network interface(s) 710 may include a physical and / or logical interface for connecting the vehicle computing device(s) 704 to another computing device or network, such as the network(s) 716. For example, the network interface(s) 710 may enable Wi-Fi-based communications, such as via frequencies defined by the IEEE 700.11 standard, short-range wireless frequencies such as ), cellular communications (e.g., 2G, 6G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol, so that the corresponding computing device can interact with (one or more) other computing devices. In some cases, (one or more) vehicle computing devices 704 and / or (one or more) sensors 706 can send sensor data to (one or more) computing devices 714 via (one or more) networks 716 at a specific frequency, after a predetermined time period, in near real time, etc.

[0085] In some cases, vehicle 702 may include one or more drive assemblies 712. In some cases, vehicle 702 may have a single drive assembly 712. In some cases, drive assembly(ies) 712 may include one or more sensors to detect conditions of drive assembly(ies) 712 and / or the environment surrounding vehicle 702. By way of example and not limitation,

[0086] The sensor(s) of the drive assembly(s) 712 may include one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the wheels of the drive assembly, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive assembly, cameras or other image sensors, ultrasonic sensors to acoustically detect objects around the drive assembly, lidar sensors, radar sensors, etc. Some sensors (such as wheel encoders) may be unique to the drive assembly(s) 712. In some cases, the sensor(s) on the drive assembly(s) 712 may overlap or complement corresponding systems (e.g., sensors 706) of the vehicle 702.

[0087] The drive system(s) 712 may include many vehicle systems, including a high voltage battery, an electric motor that drives the vehicle, an inverter that converts direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a braking system including hydraulic or electric brakes, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate traction loss and maintain control, an HVAC system, lighting (such as headlights / taillights for illuminating the vehicle's external environment), and one or more other systems (e.g., cooling systems, safety systems, onboard charging systems, other electrical components such as DC / DC converters, high voltage connectors, high voltage cables, charging systems, charging ports, etc.). In addition, the drive assembly(s) 712 may include a drive assembly controller that may receive and pre-process data from the sensor(s) and control the operation of various vehicle systems. In some cases, the drive assembly controller may include one or more processors and a memory coupled to the one or more processors in communication. The memory may store one or more components to perform various functions of the drive assembly(s) 712. Additionally, the driver component(s) 712 may also include one or more communication connections that enable the respective driver component to communicate with one or more other local or remote computing devices.

[0088] The vehicle computing device(s) 704 may include a processor(s) 718 and a memory 720 communicatively coupled to the processor(s) 718. The memory 720 may represent the memory 108. The computing device(s) 714 may also include a processor(s) 722 and / or a memory 724. The processor(s) 718 and / or 722 may be any suitable processor capable of executing instructions to process data and perform the operations described herein. By way of example, but not limitation, the processor(s) 718 and / or 722 may include one or more central processing units (CPUs), graphics processing units (GPUs), integrated circuits (e.g., application specific integrated circuits (ASICs)), gate arrays (e.g., field programmable gate arrays (FPGAs)), and / or any other device or portion of a device for processing electronic data to transform the electronic data into other electronic data that may be stored in registers and / or memory.

[0089] The memory 720 and / or 724 may be a non-transitory computer-readable medium. The memory 720 and / or 724 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the functions of the methods and various systems described herein. In various implementations, the memory may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architecture, system, and various elements described herein may include many other logical, programmatic, and physical components, of which the components shown in the figures are merely examples relevant to the discussion herein.

[0090] In some cases, memory 720 and / or storage 724 can store positioning component 726, perception component 728, planning component 730, mapping component 732, map 734, and / or system controller 736. Perception component 728 can represent perception component 110, planning component 730 can represent planning component 112, and / or mapping component 732 can represent mapping component 114. Memory 724 can store mapping component 738.

[0091] In at least one example, the positioning component 726 may include functionality to receive data from the sensor(s) 706 to determine the position, velocity, and / or orientation (e.g., one or more of x, y, z position, roll, pitch, or yaw) of the vehicle 702. For example, the positioning component 726 may include and / or request / receive the map(s) 734 of the environment, and may be capable of continuously determining the position and / or orientation of the autonomous vehicle within the map(s) 734. In some cases, the positioning component 726 may utilize SLAM (simultaneous localization and mapping), CLAMS (simultaneous calibration, localization, and mapping), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc. to accurately determine the position, attitude, and / or velocity of the autonomous vehicle. In some cases, the positioning component 726 may provide data to various components of the vehicle 702 to determine the initial position of the autonomous vehicle for use in generating trajectories and / or for use in generating map data, as discussed herein. In some examples, positioning component 726 can provide the position and / or orientation of vehicle 702 relative to the environment and / or sensor data associated therewith to mapping component 732. In some examples, positioning component 726 can generate a preliminary pose graph and can associate pose nodes with sensor data windows in log data stored at memory 720. The log data (including the preliminary pose graph) can be transmitted to computing device(s) 714 via network(s) 716.

[0092] In some cases, perception component 728 may include a primary perception system and / or prediction system implemented in hardware and / or software. Perception component 728 may detect objects in the environment surrounding vehicle 702 (e.g., identify the presence of an object), classify objects (e.g., determine an object type associated with a detected object), segment sensor data and / or other representations of the environment (e.g., identify that a portion of sensor data and / or a representation of the environment is associated with a detected object and / or object type), determine features associated with an object (e.g., identify a trajectory of current, predicted, and / or previous position, heading, velocity, and / or acceleration associated with an object), etc. The data determined by perception component 728 is referred to as perception data.

[0093] Planning component 730 can receive the location and / or orientation of vehicle 702 from positioning component 726, and / or perception data from perception component 728, and can determine, based at least in part on any of the data, instructions for controlling operation of vehicle 702. In some examples, determining the instructions can include determining the instructions based at least in part on a format associated with a system associated with the instructions (e.g., a first instruction for controlling movement of the autonomous vehicle can be formatted as a message and / or signal in a first format (e.g., analog, digital, pneumatic, kinematic) that system controller 736 and / or drive component 712 can parse / cause execution, and a second instruction for transmitter 708 can be formatted according to a second format associated therewith).

[0094] Mapping component 732 can cooperate with positioning component 726 to generate preliminary posture graph and / or preliminary sensor data alignment, for near-term planning and / or trajectory verification. In additional or alternative examples, preliminary posture graph can be determined by mapping component 732. In some examples, mapping component 732 may be different from mapping component 738, because mapping component 738 can perform soft constraint optimization discussed in this article. In additional or alternative examples, soft constraint optimization can also occur at mapping component 732. In some examples, a user can provide input at computing device 714 via a user interface, such as adding or deleting a part of a suggested position or a fixed posture graph. In some examples, mapping component 738 can generate map data based on a refined posture graph at least in part in addition or alternatively, and map data can be transmitted to vehicle 702, which can be stored in memory 720.

[0095] Although positioning component 726, perception component 728, planning component 730, map 734, and / or system controller 736 are illustrated as being stored in memory 720, any of these components may include processor-executable instructions, machine learning models (e.g., neural networks), and / or hardware, and all or a portion of any of these components may be stored on memory 724 or configured as part of computing device 714.

[0096] As described herein, the positioning component 726, the perception component 728, the planning component 730, the mapping component 732 and / or the mapping component 738 and / or other components of the system 700 may include one or more ML models. For example, the positioning component 726, the perception component 728 and / or the planning component 730 may each include a different ML model pipeline. In some examples, the ML model may include a neural network. An exemplary neural network is an algorithm inspired by biology that passes input data through a series of connected layers to produce an output. Each layer in the neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network can utilize machine learning, which can refer to a broad class of such algorithms in which an output is generated based on learned parameters.

[0097] Although discussed in the context of neural networks, any type of machine learning consistent with the present disclosure may be used. For example, machine learning algorithms may include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimated scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomy 6 (ID6), chi-squared automatic interaction detection (CHAID), decision stumps, conditional decision trees), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, averaged single dependency estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), association rule learning learning algorithms (e.g., perceptron, back-propagation, jump field network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), hybrid discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (Bagging), Ada boosting, stacked generalization (hybrid), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc. Other examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, EfficientNet, PointNet, Xception, ConvNeXt, etc., or visual transformers (ViT) such as Bidirectional Encoder from Image Transformer (BEiT), Visual Bidirectional Encoder from Transformer (VisualBERT), Image Generative Pretrained Transformer (Image GPT), Data Efficient Image Transformer (DeiT), Deep Visual Transformer (DeepViT), Convolutional Visual Transformer (Convolutional V T), detection transformer (DETR), Miti-DETR, etc.

[0098] The memory 720 may additionally or alternatively store one or more system controllers 736, which may be configured to control the steering, propulsion, braking, safety, transmitter, communication and other systems of the vehicle 702. These system controllers 736 may communicate with and / or control the corresponding systems of the drive component 712 and / or other components of the vehicle 702. For example, the planning component 730 may generate instructions based at least in part on the perception data generated by the perception component 728, and the trajectory verification component 732 may verify the instructions and / or transmit the instructions to the system controller 736. The system controller 736 may control the operation of the vehicle 702 based at least in part on the instructions received from the planning component 730. In some examples, these instructions may be based at least in part on map data generated according to the techniques discussed herein. In some examples, the trajectory verification component 732 may replace the instructions generated by the planning component 730 with alternative instructions associated with the accidental trajectory, for example, an accidental trajectory of an alternative maneuver may be specified, etc.

[0099] It should be noted that although Figure 7 714, and / or components of computing device 714 may be associated with vehicle 702. That is, vehicle 702 may perform one or more functions associated with computing device 714, and vice versa.

[0100] Sample Clauses

[0101] A. A method comprising: receiving a first pose graph associated with first map data, the first pose graph comprising a first set of nodes indicating one or more of a position or orientation of a vehicle traversing an environment, and the first map data comprising a map element, the map element indicating a first position of an object in the environment; receiving a second pose graph: determining a source node of the first pose graph associated with the map element; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element to an updated map element of second map data based at least in part on the transformation; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and transmitting the updated map element to the vehicle or another vehicle based at least in part on determining that the stress is less than the threshold stress to traverse the environment based at least in part on the updated map element.

[0102] B. A method according to paragraph A, wherein determining the stress associated with the location in the environment is based at least in part on at least one of: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined by a loss function based at least in part on at least one of the transformation or the difference between the transformation and another transformation; an output of an optimization of a transformation of two or more source nodes; determining whether a number of additional source nodes used to determine the modification meets or exceeds a threshold; or determining a difference between the transformation and the modification.

[0103] C. A method according to any of paragraphs A or B, wherein the map element is a first element and the method further comprises: determining a stress map based at least in part on interpolating a second stress value, wherein the interpolation of the second stress value is based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or a location in the environment is associated with the second stress value; and based at least in part on determining that the second stress value meets or exceeds the threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; restoring a second modification associated with the second map element; or changing the second modification.

[0104] D. A method according to any one of paragraphs A to C, wherein determining the modification includes: determining a first weight based at least in part on a first distance between the map element and the source node, wherein modifying the map element is also based at least in part on the first weight.

[0105] E. A method according to any one of paragraphs A to D, wherein determining the source node associated with the map element further comprises at least one of: determining a set of source nodes within a predefined distance of the first position of the map element; or determining the n closest source nodes to the first position of the map element, wherein n is a positive integer.

[0106] F. A method according to any of paragraphs A to E, wherein the modification results in a change in at least one of the first position of the map element or a size of the map element.

[0107] G. A system comprising: one or more processors; and a memory storing processor-executable instructions, wherein the processor-executable instructions, when executed by the one or more processors, cause the system to perform the following operations, including: receiving a first pose graph associated with first map data; receiving a second pose graph; determining a source node of the first pose graph associated with a map element of the first map data; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element to an updated map element of the second map data based at least in part on the transformation; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and transmitting the updated map element to a vehicle based at least in part on determining that the stress is less than the threshold stress, so that the vehicle traverses an environment based at least in part on the updated map element.

[0108] H. A system according to paragraph G, wherein determining the stress associated with the location in the environment is based at least in part on at least one of: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined by a loss function based at least in part on at least one of the transformation or the difference between the transformation and another transformation; an output of an optimization of a transformation of two or more source nodes; determining that a number of additional source nodes used to determine the modification meets or exceeds a threshold; or determining a difference between the transformation and the modification.

[0109] I. A system according to any of paragraphs G or H, wherein the map element is a first element and the method further includes: determining a stress map based at least in part on interpolating a second stress value, wherein the interpolation of the second stress value is based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or a location in the environment is associated with the second stress value; and based at least in part on determining that the second stress value meets or exceeds the threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; reverting a second modification associated with the second map element; or changing the second modification.

[0110] J. A system according to any of paragraphs G to I, wherein determining the modification includes: determining a first weight based at least in part on a first distance between the map element and the source node, wherein modifying the map element is also based at least in part on the first weight.

[0111] K. A method according to paragraph J, wherein determining the first weight is based at least in part on a robust loss function.

[0112] L. A system according to any one of paragraphs G to K, wherein determining the source node associated with the map element further comprises at least one of: determining a set of source nodes within a predefined distance of the first position of the map element; or determining the n closest source nodes to the first position of the map element, wherein n is a positive integer.

[0113] M. A system according to any of paragraphs G to L, wherein the modification results in a change in at least one of the first position of the map element or a size of the map element.

[0114] N. A system according to any of paragraphs G to M, further comprising: determining a second modification of a second map element based at least in part on a second transformation between a second source node and a second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and determining that the second stress meets or exceeds a threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; restoring the second modification associated with the second map element; or changing the second modification.

[0115] O. The method of paragraph N further includes: receiving user input associated with the second map element; and determining a change to the second modification based at least in part on the user input.

[0116] P. A system according to any of paragraphs G to O, wherein the first posture graph is determined at least in part based on first sensor data received from one or more sensors of the vehicle or another vehicle, and the second posture graph is determined at least in part based on second sensor data received from the one or more sensors.

[0117] Q. A non-transitory computer-readable medium storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving a first pose graph associated with first map data; receiving a second pose graph; determining a source node of the first pose graph associated with a map element of the first map data; determining an association of a target node of the second pose graph with the source node; determining a transformation between the source node and the target node; modifying the map element to an updated map element of the second map data based at least in part on the transformation; determining a stress based at least in part on at least one of the modification or the transformation; determining that the stress is less than a threshold stress; and transmitting the updated map element to a vehicle based at least in part on determining that the stress is less than the threshold stress so that the vehicle traverses an environment based at least in part on the updated map element.

[0118] R. A non-transitory computer-readable medium according to paragraph Q, wherein determining the stress associated with the location in the environment is based at least in part on at least one of: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined by a loss function based at least in part on at least one of the transformation or a difference between the transformation and another transformation; an output of an optimization of a transformation of two or more source nodes; determining that a number of additional source nodes used to determine the modification meets or exceeds a threshold; or determining a difference between the transformation and the modification.

[0119] S. A non-transitory computer-readable medium according to any of paragraphs Q or R, wherein the map element is a first element and the method further comprises: determining a stress map based at least in part on interpolating a second stress value, interpolating the second stress value based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or a location in the environment is associated with the second stress value; and based at least in part on determining that the second stress value meets or exceeds the threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; reverting a second modification associated with the second map element; or changing the second modification.

[0120] T. A non-transitory computer-readable medium according to any one of paragraphs Q to S, further comprising: determining a second modification to a second map element based at least in part on a second transformation between a second source node and a second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and determining that the second stress meets or exceeds a threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; restoring the second modification associated with the second map element; or changing the second modification.

[0121] Although the above example clauses are described for a specific implementation, it should be understood that in the context of this article, the content of the example clauses can also be implemented by methods, devices, systems, computer-readable media and / or other implementation methods. In addition, any example AT can be implemented alone or in combination with any other one or more example ATs.

[0122] in conclusion

[0123] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

[0124] The components described herein represent instructions that can be stored in any type of computer-readable medium and implemented in software and / or hardware. All of the above methods and processes can be embodied in software code components and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof, and are fully automated. Some or all of the methods can alternatively be embodied in dedicated computer hardware.

[0125] At least some of the processes discussed herein are illustrated as logical flow diagrams, in which each operation represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, an operation represents a computer-executable instruction stored on one or more non-temporary computer-readable storage media that, when executed by one or more processors, causes a computer or autonomous vehicle to perform the operation. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be limiting, and any number of the described operations may be combined in any order and / or in parallel to implement the processes.

[0126] Unless expressly stated otherwise, conditional language (such as "may," "could," "might," or "might") should be understood in context to mean that certain examples include certain features, elements, and / or steps, while other examples do not. Thus, such conditional language is generally not intended to imply that certain features, elements, and / or steps are in any way required for one or more examples, or that one or more examples necessarily include logic (whether or not with user input or prompting) for determining whether certain features, elements, and / or steps are included or whether to be performed in any particular example.

[0127] Unless expressly stated otherwise, linking language such as the phrase "at least one of X, Y, or Z" should be understood to mean that the item, term, etc. can be X, Y, or Z, or any combination thereof, including multiples of each element. Unless expressly described as singular, "a" and "an" refer to both the singular and the plural.

[0128] Any routine description, element or block in the flowcharts described herein and / or shown in the drawings should be understood to represent a module, segment or code portion, which includes one or more computer executable instructions for implementing specific logical functions or elements in the routine. Alternative implementations are included within the scope of the examples described herein, in which elements or functions can be deleted, or performed in a different order than shown or described, including substantially simultaneous, in reverse order, additional operations or omitted operations, depending on the functions involved as understood by those skilled in the art.

[0129] Many changes and modifications may be made to the above examples, the elements of which should be understood as one of other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the following claims.

Claims

1. A system comprising: one or more processors; as well as a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations including: receiving a first gesture graph associated with first map data; receiving a second posture graph; determining a source node of the first gesture graph associated with a map element of the first map data; Determining the relevance of a target node of the second posture graph to the source node; Determining a transformation between the source node and the target node; modifying the map element to an updated map element of second map data based at least in part on the transformation; determining a stress based at least in part on at least one of the modifying or the transforming; determining that the stress is less than a threshold stress; as well as Based at least in part on determining that the stress is less than the threshold stress, the updated map elements are transmitted to a vehicle to cause the vehicle to traverse an environment based at least in part on the updated map elements.

2. The system according to claim 1, wherein: Determining the stress associated with a location in the environment is based at least in part on at least one of: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined based at least in part on at least one of the transform or a difference between the transform and another transform via a loss function; An optimized output of a transformation of two or more source nodes; determining that a number of additional source nodes used to determine the modification meets or exceeds a threshold; or A difference between the transformation and the modification is determined.

3. The system according to claim 1, wherein: The map element is a first element, and the operations further include: determining a stress map based at least in part on interpolating a second stress value based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or location in the environment is associated with the second stress value; and Based at least in part on determining that the second stress value meets or exceeds the threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; reverting the second modification associated with the second map element; or Change the second modification.

4. The system according to claim 1, wherein: The modifications include: determining a first weight based at least in part on a first distance between the map element and the source node, Wherein modifying the map element is also based at least in part on the first weight.

5. The system according to claim 4, wherein: Determining the first weight is based at least in part on a robust loss function.

6. The system according to claim 1, wherein: Determining the source node associated with the map element further includes at least one of the following: determining a set of source nodes within a predefined distance of a first position of the map element; or Determine n source nodes closest to the first position of the map element, where n is a positive integer.

7. The system according to claim 1, wherein: The modification results in a change to at least one of a first position of the map element or a size of the map element.

8. The system of claim 1, further comprising: determining a second modification to the second map element based at least in part on a second transformation between the second source node and the second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and Determine that the second stress meets or exceeds a threshold stress, and perform at least one of the following: generating a notification associated with the second map element or a location in the environment; reverting the second modification associated with the second map element; or Change the second modification.

9. The system according to claim 8, further comprising: receiving user input associated with the second map element; as well as A change to the second modification is determined based at least in part on the user input.

10. The system according to claim 1, wherein: The first pose graph is determined based at least in part on first sensor data received from one or more sensors of the vehicle or another vehicle, and the second pose graph is determined based at least in part on second sensor data received from the one or more sensors.

11. A non-transitory computer readable medium storing processor executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving a first gesture graph associated with first map data; receiving a second posture graph; determining a source node of the first gesture graph associated with a map element of the first map data; Determining the relevance of a target node of the second posture graph to the source node; Determining a transformation between the source node and the target node; modifying the map element to an updated map element of second map data based at least in part on the transformation; determining a stress based at least in part on at least one of the modifying or the transforming; determining that the stress is less than a threshold stress; as well as Based at least in part on determining that the stress is less than the threshold stress, the updated map elements are transmitted to a vehicle to cause the vehicle to traverse an environment based at least in part on the updated map elements.

12. The non-transitory computer readable medium of claim 11, wherein: Determining the stress associated with a location in the environment is based at least in part on at least one of: determining whether the transformation meets or exceeds a transformation threshold; determining a variance associated with one or more transformations within a threshold distance of the location; determining a weight based at least in part on a distance between the map element and the source node; a loss determined based at least in part on at least one of the transform or a difference between the transform and another transform via a loss function; An optimized output of a transformation of two or more source nodes; determining that a number of additional source nodes used to determine the modification meets or exceeds a threshold; or A difference between the transformation and the modification is determined.

13. The non-transitory computer readable medium of claim 11, wherein: The map element is a first element, and the operations further include: determining a stress map based at least in part on interpolating a second stress value based at least in part on the stress and one or more other stress values; determining that the second stress value meets or exceeds the threshold stress; determining that a second map element or location in the environment is associated with the second stress value; and Based at least in part on determining that the second stress value meets or exceeds the threshold stress, performing at least one of the following: generating a notification associated with the second map element or the location in the environment; reverting the second modification associated with the second map element; or Change the second modification.

14. The non-transitory computer readable medium of claim 11, wherein: Also includes: determining a second modification to the second map element based at least in part on a second transformation between the second source node and the second target node; determining a second stress associated with the second modification based at least in part on at least one of the second transformation or the second modification; and Determine that the second stress meets or exceeds a threshold stress, and perform at least one of the following: generating a notification associated with the second map element or a location in the environment; reverting the second modification associated with the second map element; or Change the second modification.

15. The non-transitory computer readable medium of claim 11, wherein: The modification results in a change in at least one of a position of the map element or a size of the map element.

Images