System and method for generating multi-resolution voxel space

Abstract

Techniques are discussed herein for representing a scene or map based on statistics of captured environmental data. In some cases, data (e.g., covariance data, mean data, etc.) may be stored as a multi-resolution voxel space that includes multiple semantic layers. In some cases, individual semantic layers may include multiple voxel grids having different resolutions. Based on the detected voxel covariance at one or more resolutions, multiple multi-resolution voxel spaces may be merged or aligned to generate a combined scene.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 17 / 804,744, filed on May 31, 2022, entitled “SYSTEM AND METHOD FORGENERATING MULTI-RESOLUTION VOXELSPACES,” the entire contents of which are incorporated herein by reference. Background Art

[0003] Data may be captured in an environment and represented as a map of the environment. Typically, such a map may be used by a vehicle navigating within the environment, although maps may be used for a variety of purposes. In some cases, the environment may be represented as a two-dimensional map, while in other cases, the environment may be represented as a three-dimensional map. Furthermore, surfaces within the environment are often represented using multiple polygons or triangles. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical components or features.

[0005] Figure 1 is an example process flow diagram illustrating an example data flow for a system configured to align data representing a physical environment with a scene, as described herein.

[0006] Figure 2 is another flow chart illustrating an example associated with generating an alignment between multi-resolution voxel spaces, as described herein.

[0007] Figure 3 is an example flow chart illustrating an example process associated with generating an alignment between multi-resolution voxel spaces, as described herein.

[0008] Figure 4 is an example flow chart illustrating an example process associated with generating an alignment between multi-resolution voxel spaces, as described herein.

[0009] Figure 5 is a block diagram of an example system for implementing a multi-resolution voxel spatial alignment system, as described herein.

[0010] Figure 6 is a schematic diagram of an example of a multi-resolution voxel space, as described herein. DETAILED DESCRIPTION

[0011] The techniques described herein relate to generating alignments between map data that include a multi-resolution voxel space. In some examples, such a multi-resolution voxel space may include voxels that store statistical information related to the associated measurements, including but not limited to the spatial mean, covariance, and weights of the point distribution of data representing the physical environment. The map data may include multiple voxel grids (e.g., including "volumetric pixels," or discrete volume representations of voxels) or layers of the physical environment representing different resolutions or physical distances. For example, each voxel layer may represent the physical environment at a resolution several times (e.g., twice) that of the previous layer. That is, the voxels of the first layer may represent a first volume (e.g., 10cm×10cm×10cm), while the voxels of the second layer may represent a second volume (e.g., 20cm×20cm×20cm).

[0012] Data associated with voxels of a multi-resolution voxel space can be represented as multiple covariance ellipsoids. The covariance ellipsoid representation can be generated based on the calculated mean and covariance values ​​of the data points associated with the individual voxels. For example, each ellipsoid can have a shape determined by one or more eigenvectors associated with the covariance matrix of the voxel (e.g., three eigenvectors associated with the X, Y, and Z measurements of the associated data points). In some cases, voxel data can be associated with semantic information such as classification and / or segmentation information, and data associated with a specified classification can be associated with a specific multi-resolution voxel space associated with the specified classification. In this example, each voxel covariance semantic layer can include data points associated with a specific semantic classification (e.g., tree, vehicle, building, etc.) as a covariance ellipsoid.

[0013] In some cases, map data represented by a multi-resolution voxel space can be generated based on data points representing a physical environment, such as the output of a light detection and ranging (lidar) system. For example, the system can receive multiple lidar points or lidar data represented as a point cloud. The system can assign or otherwise associate the lidar points to voxels of a voxel grid with multiple resolutions (e.g., finer and coarser resolutions) based at least in part on a local reference frame of the vehicle (e.g., a system that captures the lidar points). The system can then merge or otherwise combine voxels (or data associated with voxels) of one or more resolutions substantially simultaneously to generate a final multi-resolution voxel space. As a non-limiting example, measurements can first be associated with the finest resolution voxel grid, and other layers (with coarser resolutions) can be calculated, such as merged, based on the finer resolution. In some examples, such associations and calculations can be performed substantially simultaneously (e.g., within technical tolerances). In a specific example, voxels in a neighborhood are merged by taking a weighted sum of a separate Gaussian distribution for each voxel of a finer resolution grid.

[0014] In some embodiments, the system can utilize multi-resolution voxel spaces to generate alignments between voxel spaces to assist in generating maps and scenes of physical environments, and to assist in locating vehicles within maps or scenes. For example, once a multi-resolution voxel space (e.g., a target multi-resolution voxel space) is generated for a particular scan or data set representing a physical environment (e.g., determined during driving), the system can determine the alignment between the generated multi-resolution voxel space and a reference multi-resolution voxel space representing a scene. In some cases, the alignment can be generated by finding correspondences between voxels at each resolution of the reference and target multi-resolution voxel spaces. For example, for each voxel of a particular resolution in the target multi-resolution voxel space, the system can search for occupied voxels within a threshold distance, or within a threshold number of voxels containing an average target point in the corresponding particular resolution of the reference multi-resolution voxel space. In an example including semantic layers, for each voxel of a particular resolution of each semantic layer in the target multi-resolution voxel space, the system can search for a neighborhood of voxels containing an average target point in the particular resolution of the corresponding semantic layer of the reference multi-resolution voxel space.

[0015] Among the voxels identified in the reference multiresolution voxel space, the system can select voxels with centroids that are close to voxels in the target multiresolution voxel space. The system can then determine the residual (or error, etc.) for each matching voxel, which in at least some examples can be based at least in part on the normal vector of such a match, and then perform an optimization on all such residuals. The optimization can minimize the distance between pairs of such voxel centroids and / or means. In this way, the alignment or error between two voxels can be determined.

[0016] During alignment, even though each layer may be merged substantially simultaneously, a coarser resolution (e.g., a resolution corresponding to larger voxels) may result in a match before a finer resolution. In this way, matches in the coarser resolution may help bring the two multi-resolution voxel spaces into closer alignment, enabling the finer resolution to begin matching and complete the alignment process. In some cases, by merging captured sensor data into a multi-resolution voxel space representing the environment, a vehicle may be able to initialize position or localization in the environment more accurately and / or faster than a system utilizing traditional map data including polygons and / or meshes. In addition, by storing voxels in a multi-resolution voxel space, the data may be stored in a manner that is more easily indexed / retrieved, thereby increasing processing speed and throughput. For example, if a coarser resolution is acceptable for an actual task, the coarse layer may be loaded into memory, thereby reducing the amount of data accessed and processed for the required operation.

[0017] In some examples, when generating data for generating a target multi-resolution voxel space, the system can use positioning data, position data, and / or the like associated with the autonomous vehicle to initiate alignment or pre-alignment of the target and reference multi-resolution voxel spaces. For example, the positioning data and / or position data can include global positioning system (GPS) data (or other satellite-based position data), odometer data, inertial measurement unit (IMU) data, previously known positions and / or degrees of freedom determined based on alignment of a previous multi-resolution voxel space with a reference voxel space, and / or the like.

[0018] In some cases, depending on the seed data (e.g., LiDAR or point cloud data) used to generate the two multi-resolution voxel spaces, it may be difficult for the system to generate an alignment, and / or generating an alignment between the voxel spaces may require a significant amount of processing resources and / or time to converge. For example, when the voxels are sparse or far apart from each other, two voxels (e.g., one voxel in each voxel space) may rarely overlap, and aligning the voxels based on covariance may be time and resource intensive. In other examples, when the initial scanning error is large (e.g., such as greater than 10 meters), in addition to consuming a significant amount of resources and / or time, the voxel spaces may be difficult to converge at all. In these cases, the systems and methods discussed herein can assist and / or improve the convergence rate and speed, as well as reduce the processing or computational resource consumption associated with generating an alignment between multi-resolution voxel spaces.

[0019] In one example, the system may align or converge a coarser voxel resolution before initiating alignment of a finer resolution. For example, the system may allow voxels greater than a predetermined threshold to be aligned in the initial stages of the process. For example, in one specific example, the system may allow voxels with a resolution of 25 meters or greater to begin aligning in the initial stages. For example, when the system determines an error less than an error threshold, the system may then iteratively add the next finer resolution to the convergence process. In some cases, the error threshold for the next finer resolution may be the average error of voxels less than or equal to half the size of the current finest resolution (for example, if the current finest resolution is 25 meters, the system may add the next finer resolution when the error is less than or equal to 12.5 meters). In other examples, the error threshold for the next finer resolution may be the average error of voxels less than or equal to one-quarter the size of the current finest resolution, etc.

[0020] In this example, the system can continue to iterate the alignment phase until each resolution has been added. In some cases, the system can continue to iterate after all resolutions have been added for a predetermined number of iterations (e.g., one iteration, two iterations, three iterations, five iterations, etc.) for all resolutions, or until the average error is less than or equal to a final error threshold, or until the change in the residual sum of the voxels is less than or equal to a change threshold.

[0021] In another example, the system may also evaluate, weight or otherwise score the eigenvalues ​​of the voxels during alignment, and use the weights to select eigenvectors and / or voxels for alignment. For example, as described above, a voxel may have a set of three or more eigenvalues ​​(providing an episode shape of the voxel). The system may then determine the score or weight of the eigenvector or voxel based at least in part on the size of the eigenvalues. In some embodiments, the system may evaluate individual eigenvalues ​​according to one or more predetermined heuristics to determine the weights. For example, when generating a score, the system may then determine whether one or more eigenvalues ​​of the voxel are less than or equal to one or more thresholds. In some cases, one or more thresholds may be related to the size of the voxel or the size of the associated resolution. In other embodiments, the system may evaluate and / or score individual eigenvalues ​​using one or more machine learning models. The system may then select voxels associated with higher scores during the alignment process (e.g., based on their eigenvalues ​​or other characteristics as discussed herein, e.g., the number of points, quality metrics, etc.).

[0022] In some examples, the multi-resolution voxel space may include multiple voxel layers for a given resolution, and individual layers may be associated with different semantic classifications. For example, the first voxel layer of the resolution may be associated with a building, the second voxel layer of the resolution may be associated with a ground plane, and / or the third voxel layer of the resolution may be associated with vegetation. In this example, the system may also fit or regress the quality or trust metric of the individual voxels within the resolution using the semantic classification assigned to the voxels. For example, the system may generate and / or train a noise model to evaluate the quality of the voxels based at least in part on the eigenvalues ​​of the voxels, the voxel resolution, the number of points associated with the merged voxels, and the semantic classification of the merged voxels. The system may then select the voxels used in the alignment process using the quality or trust metric. For example, the system may determine the scale factor correction for the multi-resolution voxel space by randomly sampling the voxels (such as Monte Carlo techniques) based at least in part on the quality or trust metric. In some cases, the scale factor correction may be used as a metric to determine the overall quality of the resulting alignment.

[0023] In some cases, by aligning a coarser resolution before a finer resolution, utilizing weights of the eigenvalues ​​of the voxels to determine eigenvectors for alignment, and utilizing a quality or confidence metric for the voxels used for alignment, the system can converge a multi-resolution voxel space in a more time-efficient manner with fewer processing resources, and generate a higher quality multi-resolution voxel space that better represents the corresponding physical environment.

[0024] As discussed herein, systems and methods allow processes for alignment between multi-resolution voxel spaces to converge in a more efficient manner. For example, systems and methods allow multi-resolution voxel spaces to converge in a shorter time period while consuming fewer resources than conventional systems. In some cases, by reducing the time period associated with convergence, autonomous vehicles can make operational decisions, including safety-related decisions, in a more timely manner. In addition, the systems and methods discussed herein may result in a more precise alignment between a target multi-resolution voxel space and a reference multi-resolution voxel space, thereby allowing the autonomous vehicle to perform operations with a more accurate perception of the vehicle's surroundings, generally resulting in safer operation of such systems.

[0025] Figure 1 100 is an example process flow diagram illustrating an example data flow of a system configured to align data representing a physical environment with a scene, as described herein. In the example shown, the system can be configured to store the scene and the data representing the environment as a multi-resolution voxel space. As described above, the multi-resolution voxel space can have multiple semantic layers, where each semantic layer includes multiple voxel grids that represent voxels as covariance ellipsoids of different resolutions.

[0026] In a specific example, a sensor system 102, such as a lidar, radar, sonar, infrared, camera, or other image capture device, can capture data representing the physical environment surrounding the system. In some cases, the captured data can be a plurality of data points 104, such as a point cloud generated from the output of a lidar scan. In this example, the data points 104 can be received by a multi-resolution voxel space component 106.

[0027] The multi-resolution voxel space component 106 can be configured to generate a target multi-resolution voxel space based on the data points 104. In some cases, the multi-resolution voxel space component 106 can process the data points via classification and / or segmentation techniques. For example, the multi-resolution voxel space component 106 can assign types or classifications to data points using one or more neural networks (e.g., deep neural networks, convolutional neural networks, etc.), regression techniques, etc., to identify and classify the data points 104 using semantic labels. In some cases, the semantic labels may include classifications or entity types, such as vehicles, pedestrians, cyclists, animals, buildings, trees, pavement, curbs, sidewalks, unknown, etc. In additional and / or alternative examples, the semantic labels may include one or more characteristics associated with the data points 104. For example, the characteristics may include, but are not limited to, x position (global and / or local position), y position (global and / or local position), z position (global and / or local position), orientation (e.g., roll, pitch, yaw), entity type (e.g., classification), etc.

[0028] In some examples, generating the target multi-resolution voxel space 108 may include filtering data associated with dynamic objects (e.g., representing pedestrians, vehicles, etc.) while associating data associated with static objects (e.g., buildings, trees, leaves, etc.) with the target multi-resolution voxel space 108. In alternative embodiments, the data points 104 may be output by a perception pipeline or component with semantic labels attached. For example, the data points 104 may be received as part of a sparse object state representation output by a perception component, details of which are discussed in U.S. Application Serial No. 16 / 549,694, which is incorporated herein by reference in its entirety.

[0029] In the current example, the multi-resolution voxel space component 106 can assign the semantically labeled data points 104 to a semantic layer of a target multi-resolution voxel space 108 having a corresponding semantic label (e.g., tree, building, pedestrian, etc.). For example, the multi-resolution voxel space component 106 can project the data points 104 into a common coordinate system and then assign them to the appropriate point cloud associated with the corresponding semantic classification. For each point cloud, the multi-resolution voxel space component 106 can then assign each data point 104 to a voxel of the finest resolution voxel grid (e.g., a base voxel grid) of each semantic layer. In some specific instances, the multi-resolution voxel space may be a single layer that stores multiple statistical values, including the semantic classification of each voxel.

[0030] Once each data point 104 of the corresponding cloud is assigned to a voxel, the multi-resolution voxel space generation component 106 can calculate spatial statistics (e.g., spatial mean, covariance, weight, and / or number of data points 104 assigned to the voxel) for each voxel of the finest resolution grid of the semantic layer. Once the base or finest resolution voxel grid of the semantic layer is completed, the multi-resolution voxel space generation component 106 can iteratively or recursively generate the next coarser resolution voxel grid for each semantic layer. For example, example processes associated with generating a voxel space are discussed in U.S. Patent No. 11,288,861 and U.S. Application No. 16 / 722,771, which are incorporated herein by reference in their entirety for all purposes.

[0031] Once a target multi-resolution voxel space 108 is generated based on the data points 104, the target multi-resolution voxel space 108 can be aligned with a reference multi-resolution voxel space 110 (e.g., a previously generated multi-resolution voxel space representing a shared scene or physical environment). For example, in the example shown, the multi-resolution voxel space alignment component 112 can generate an alignment 114 between the newly generated target multi-resolution voxel space 108 and the reference multi-resolution voxel space 110, for example, to assist in positioning, object tracking, and / or navigation of an autonomous vehicle relative to the physical environment. In some cases, to generate the alignment 110 between the target multi-resolution voxel space 108 and the reference multi-resolution voxel space 110, the multi-resolution voxel space alignment component 112 can initially select one or more coarse resolutions (e.g., resolutions above a size threshold) for separate semantic layers and begin determining an alignment or offset between voxels of the target multi-resolution voxel space 108 and voxels of the reference multi-resolution voxel space 110. In some examples, the multiresolution voxel space alignment component 112 may utilize odometers, position data, orientation data, trajectory data, etc. to determine an initial alignment from which to begin alignment between voxels of the target multiresolution voxel space 108 and voxels of the reference multiresolution voxel space 110 .

[0032] When the system determines an error less than the error threshold, the multi-resolution voxel spatial alignment component 112 can then, for example, iteratively add the next finer resolution to the convergence process. In some cases, the error threshold for the next finer resolution can be an average error of voxels (e.g., distance between centroids or other means described herein) less than or equal to half the voxel size of the current resolution (e.g., the width of individual voxels used for alignment). In other examples, the error threshold for the next finer resolution can be an average error of voxels less than or equal to one-quarter of the current finest resolution size, etc.

[0033] In some examples, the multi-resolution voxel space alignment component 112 may continue to iterate the alignment phase until each resolution has been added / used. In some cases, the multi-resolution voxel space alignment component 112 may continue to iterate after all resolutions have been added for a predetermined number of iterations (e.g., one iteration, two iterations, three iterations, five iterations, etc.) of all resolutions, or until the average error is less than or equal to a final error threshold, or until the change in the residual sum of the voxels is less than or equal to a change threshold.

[0034] In some cases, the multi-resolution voxel space alignment component 112 may also evaluate, weight, or otherwise score the feature values ​​of individual voxels during alignment and use the weights to select feature vectors and / or voxels for alignment 114. For example, as described above, a voxel may have a set of three or more feature values, and the multi-resolution voxel space component 106 may determine the score or weight of the feature vector or voxel based at least in part on any one of the one or more feature values. In the current example, a smaller feature value may result in a higher score. In some embodiments, the system may evaluate individual feature values ​​according to one or more predetermined heuristics or thresholds to determine the weights. For example, when generating a score, the multi-resolution voxel space alignment component 112 may determine whether one or more feature values ​​of a voxel are less than or equal to one or more thresholds (such as a size threshold, an error threshold, etc.). In some cases, one or more thresholds may be relative to the size of the voxel or the size of the associated resolution. In other embodiments, the multi-resolution voxel space alignment component 112 may use one or more machine learning models to evaluate and / or score individual feature values. The multi-resolution voxel space alignment component 112 can then select the voxel associated with the higher score (eg, based on the number of points, eigenvalues, eigenvectors, etc.) for use in the alignment process. In some examples, the system can select the eigenvalue with the smallest value.

[0035] In some embodiments, the multi-resolution voxel space alignment component 112 can also fit or regress the quality or trust metrics of individual voxels using the semantic classification assigned to the voxels. For example, the multi-resolution voxel space alignment component 112 can generate and / or train a noise model to evaluate the quality of the aligned voxels based at least in part on the eigenvalues ​​of the voxels, the voxel resolution, the number of points associated with the voxels, and the semantic classification of the voxels. The multi-resolution voxel space alignment component 112 can then select or weight the voxels used in the alignment process using the quality or trust metrics. In various examples, multiple multi-resolution voxel spaces can be generated for each such classification, and the processes described herein can be run on any number of different classifications, where the classifications have generated transformations between the combined voxel spaces. In at least some such examples, such combinations can be based on, for example, covariances associated with the respective classifications.

[0036] Figure 2-Figure 4 is a flow chart illustrating an example process associated with generating a multi-resolution voxel space as discussed herein. The process is illustrated as a series of boxes in a logical flow chart, which represents a sequence of operations, some or all of which may be implemented in hardware, software, or a combination thereof. In the context of software, a box represents computer-executable instructions stored on one or more computer-readable media, which, when executed by one or more processors, perform the recited operations. In general, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types.

[0037] The order in which the operations are described should not be construed as limiting. Any number of the described blocks may be combined in any order and / or in parallel to implement the process or an alternative process, and all blocks need not be executed. For purposes of discussion, although the processes of this article are described with reference to the frameworks, architectures, and environments described in the examples of this article, these processes may be implemented in a variety of other frameworks, architectures, or environments.

[0038] Figure 2 is another flow chart illustrating an example process 200 associated with generating a multi-resolution voxel space, as described herein. As described above, the system can generate an alignment between a target multi-resolution voxel space representing a physical environment and a reference multi-resolution voxel space representing the same physical environment. In some cases, convergence between voxels of the target multi-resolution voxel space and voxels of a reference multi-resolution voxel space (e.g., generated based on one or more previous scans of the physical environment) can be time and resource intensive. As described below, process 200 reduces the time and resources required to achieve convergence or a desired alignment between two multi-resolution voxel spaces.

[0039] At 202, the system may receive a first multi-resolution voxel space and a second multi-resolution voxel space. For example, the system may receive two or more unaligned multi-resolution voxel spaces, such as a target and a reference. For example, the two or more multi-resolution voxel spaces may be generated from data captured as part of multiple spins of a lidar system, such as, for example, two or more vehicles capturing data representing the same physical environment.

[0040] At 204, the system may determine a resulting voxel and a residual representing the resulting voxel based at least in part on a first voxel associated with the first multi-resolution voxel space and a second voxel associated with the second multi-resolution voxel space. For example, for a pair of associated voxels (e.g., having closest centroids), the system may determine various metrics corresponding to a combination of all associated points (e.g., a mean, a number of points, a covariance, an eigenvalue, an eigenvector, etc.), a vector or distance between the means of the first voxel and the second voxel, and the like. In at least some examples, a residual associated with the alignment of the first voxel and the second voxel may be determined as a dot product between a vector of means of the first voxel and the second voxel and a separate eigenvalue (e.g., a minimum eigenvalue). In other words, the residual may be equal to the dot product of a unit vector of the eigenvectors of the combined voxels and the result of subtracting the second mean from the first mean.

[0041] At 206, the system may determine the quality metric of the resulting voxels based at least in part on voxel resolution, feature values, the number of points in the voxel, semantic classification, and one or more models. For example, a first voxel layer of resolution may be associated with a building, a second voxel layer of the same resolution may be associated with a ground plane, and / or a third voxel layer of the same resolution may be associated with vegetation. In this example, the system may fit or regress the quality or trust metric of the individual voxels matched within the voxel layer using the semantic classification assigned to the resulting voxels. For example, for each voxel, the system may input the voxel resolution of the resulting voxel, the feature value of the resulting voxel, the number of points in the resulting voxel, and the semantic classification of the resulting voxel into a machine learning model or other pre-loads of the output quality metric. For example, the system may generate and / or train a noise model to evaluate the quality of the voxel match based at least in part on the feature value of the voxel, the resolution, the number of points associated with the voxel, and the semantic classification. In other cases, the system may utilize a predetermined function whose result or output is a quality metric. In these cases, the function may also receive as input the voxel resolution of the resulting voxel, the feature value of the resulting voxel, the number of points in the resulting voxel, and the semantic classification of the resulting voxel.

[0042] At 208, the system may generate an alignment between the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on the quality metric and the residuals. For example, the system may scale or weight pairs of voxels (e.g., residuals) based at least in part on the quality metric. In some examples, the system may compare the quality metric to an expected error and generate the alignment using the residuals of pairs of voxels having a quality metric that is less than or equal to the expected error. In some examples, the residuals may be scaled based at least in part on the quality metric. In such examples, those pairs associated with lower quality metrics will be less dependent than other pairs with higher quality metrics. In various examples, such a quality metric may be a number between 0 and 1.

[0043] At 210, the system can determine whether the voxel has converged. For example, the system can iterate until the convergence is less than or equal to a distance threshold, a number of iterations, steps, or levels are performed, and / or the residual difference between the previous iteration and the current iteration changes by less than or equal to a residual threshold. As an illustrative example, convergence can be achieved when the error represented by the vector between the average of the voxel pairs in the target multi-resolution voxel space and the reference multi-resolution voxel space is less than an error threshold. If the voxel has not converged, the method 200 can return to 204. Otherwise, once the voxel has converged, the method 200 can proceed to 212.

[0044] At 212, the system can apply a scale factor correction to the alignment. For example, the system can determine the scale factor correction for the alignment by randomly sampling the residuals of the voxels (such as Monte Carlo techniques). The system can then apply the scale factor correction to one or more alignments and output the alignment at 214.

[0045] Figure 3 is an example flow chart illustrating an example method 200 associated with aligning multi-resolution voxel spaces, as described herein. As described above, a system can generate an alignment between a target multi-resolution voxel space representing a physical environment and a reference multi-resolution voxel space. In some cases, convergence between voxels of a target multi-resolution voxel space and voxels of a reference multi-resolution voxel space (e.g., generated based on one or more previous scans of the physical environment) can be time and resource intensive. As described below, method 200 reduces the time and resources required to achieve convergence or a desired alignment between two multi-resolution voxel spaces.

[0046] At 302, the system can receive a resolution of a multi-resolution voxel space. For example, the system can evaluate or select feature values ​​of voxels at each semantic level and each resolution to use in the alignment process. As described above, an individual voxel can include three feature vectors that represent or provide a physical shape of corresponding data associated with the particular voxel. These feature vectors can be used to determine whether the voxel is suitable or preferred for the alignment process.

[0047] At 304, the system can determine at least one voxel having a resolution of at least one eigenvalue less than or equal to a threshold, which threshold can be related to the size of the voxel (e.g., in any dimension). For example, for each eigenvalue greater than a size threshold, the system can determine that data in the direction represented by the eigenvalue should not be relied upon. However, rather than discarding the entire voxel, the system can determine availability based on each individual direction (e.g., eigenvalue). Accordingly, if a voxel has three eigenvalues ​​greater than the size threshold, the entire voxel may be ignored for the alignment process and / or for the current iteration of the alignment process. In some cases, the size threshold can be based on a resolution associated with the voxel, a predetermined value, a classification of an associated point, etc.

[0048] At 306, the system can associate weights with individual voxels to assist in selecting voxels and / or feature values ​​that will provide usable data for generating an alignment. For example, the system can assign weights between zero and one, where values ​​greater than or equal to a first size threshold can be zero, and values ​​less than or equal to a second size threshold will be assigned a weight of one. Values ​​between the first size threshold and the second size threshold will be assigned weights between zero and one, such as 0.3, 0.5, 0.7, etc.

[0049] At 308, the system may align at least one voxel with a second voxel based at least in part on the eigenvalues ​​and / or weights. For example, the system may select at least one voxel based on the weights. However, the system may only utilize two of the three eigenvalues ​​of the voxel as input to the alignment process based on separate weights for each separate eigenvalue. For example, if a voxel has two eigenvalues ​​equal to one and a single eigenvalue with a weight of one, the system may ignore the single eigenvalue when determining alignment. In other examples, the system may utilize one or three eigenvalues ​​as input to the alignment process. In this manner, higher quality eigenvectors and / or eigenvalues ​​may be used to merge or align voxels.

[0050] Figure 4is an example flow chart illustrating an example process 400 associated with generating a multi-resolution voxel space, as described herein. As described above, a system can generate an alignment between two multi-resolution voxel spaces representing a physical environment. In some cases, convergence of an alignment between voxels of a target multi-resolution voxel space (e.g., generated based on a current scan of the physical environment) and voxels of a reference multi-resolution voxel space (e.g., a previously generated map of the physical environment) can be time and resource intensive. As described below, method 400 reduces the time and resources required to achieve convergence of an alignment between two multi-resolution voxel spaces (e.g., a target multi-resolution voxel space and a reference multi-resolution voxel space).

[0051] At 402, the system may receive a target multi-resolution voxel space and a reference multi-resolution voxel space. As discussed herein, the target multi-resolution voxel space may be generated based on data captured by an autonomous vehicle operating within a physical environment. For example, an autonomous vehicle may use various sensors associated with the vehicle to capture and / or generate data representing the physical environment. In some cases, the sensor data may include image data, lidar data, point cloud data, environmental data, radar data, sonar data, infrared data, etc. The system may generate semantic point cloud data based on data representing the physical environment, for example, where semantic classifications are associated with individual points and the classifications are isolated. For example, the system may segment and classify data representing the physical environment. For example, the system may segment and classify data representing the physical environment using one or more machine learning models. In some cases, the segmented and classified data may be stored or organized into semantic layers (e.g., each layer includes data corresponding to the assigned classification). For example, in a specific example, the assignment of semantic classifications to data points is discussed in U.S. Application Serial No. 15 / 820,245, which is incorporated herein by reference in its entirety. In some examples, the system may generate a voxel covariance grid for each semantic classification at multiple resolutions for at least a portion (including all) of the semantic classifications. For example, for each semantic layer, the system may generate voxels of one or more resolutions (e.g., each resolution may have voxels of different physical sizes). In some cases, the size of the resolution may be based on a power of 2, such as 25 cm, 1 m, 16 m, 25 m, etc. A reference multi-resolution voxel space may be previously generated based on previously scanned or captured data of a physical environment and provided as a map of the physical environment available to the autonomous vehicle in operational decisions and processes. For example, a specific example of generating a multi-resolution voxel space is discussed in U.S. application serial number 17 / 446,344, which is incorporated herein by reference in its entirety.

[0052] At 404, the system can determine a set of resolutions greater than a resolution threshold. For example, the system can start the alignment process by limiting the resolution at which voxels are to be processed to one or more coarse resolutions. For example, the system limits the resolution to resolutions greater than or equal to 25 meters, or 16 meters, etc. In some examples, the set of resolutions may be a single coarsest resolution.

[0053] At 406, the system can align the voxels of the resolution set to update the alignment between the target multi-resolution voxel space and the reference multi-resolution voxel space. For example, the system can allow the voxels of the resolution set to be aligned based at least in part on voxel covariance, as described above with respect to Figure 2 and Figure 3 In this manner, since only a coarse resolution is used to generate an alignment within a predetermined error threshold, the system and method 400 allows for increased efficiency associated with generating an alignment and improved convergence to finer resolutions (e.g., faster and with less resource consumption).

[0054] As a non-limiting example, the system can generate an alignment based on voxel pairs (e.g., corresponding voxels of target and reference multi-resolution voxel spaces). In some cases, the alignment can be generated by determining a matching residual between two voxels of the same semantic classification (e.g., one voxel in each space). For example, the system can determine the average of a first voxel (a voxel in the target space) and a second voxel (a voxel in the reference space), and determine a vector between the averages of the first voxel and the second voxel.

[0055] The system can also generate the resulting voxels (such as via statistical analysis or summing two voxels), and based at least in part on the average of the resulting voxels and vectors, the system can determine eigenvalues ​​for the pair of voxels. The system can then select the smallest eigenvalue or each eigenvalue that is less than or equal to a threshold (because the longer the eigenvalue, the greater the error and the less accurate the characterization information), and use the selected eigenvalue and vector (such as a dot product) to generate a scalar value or residual. In this example, the residual can represent the error between the first voxel and the second voxel in the eigenvalue / eigenvector direction. In this example, it should be understood that the residual can be calculated for each of the three eigenvalues, thereby producing three residuals representing errors in three directions, which can be used to align the pair of voxels and thereby assist in generating an alignment between the target multi-resolution voxel space and the reference multi-resolution voxel space.

[0056] The system can then compare the residual to an expected error determined as the output of a predetermined function that utilizes the number of points in the voxel, the semantic classification of the voxel, the resolution of the voxel, and the eigenvalues, as described above with respect to Figure 3In this example, if the residual is less than or equal to the expected error, the system can determine a rotation and translation between the first voxel and the second voxel (eg, via a least squares operation) for updating the alignment.

[0057] At 408, the system can determine whether the average residual of the voxels associated with the resolution set is less than or equal to an error threshold. In some cases, the error threshold can be half the size of the finest resolution in the resolution set. For example, if the finest resolution of the resolution set is 25 meters, the system can iteratively perform the alignment step until the average residual is less than 12.5 meters. In other examples, the error threshold for the next finer resolution can be an average residual less than or equal to one-quarter the size of the current finest resolution, etc.

[0058] If the mean residual of the voxels is not less than or equal to the error threshold, the method 400 returns to 406 and performs another iteration to improve the alignment by continuing to align voxels of the resolution set. However, if the mean residual of the voxels is less than or equal to the error threshold, the method 400 proceeds to 410.

[0059] At 410, the system can determine whether additional finer resolutions are available. If no more resolutions are available, the system can output an alignment between the target multi-resolution voxel space and the reference multi-resolution voxel space at 412. In some examples, it should be appreciated that once the finest resolution is added to the set of resolutions, method 400 can perform one or more iterations of step 406 (e.g., until a final error threshold is met or exceeded and / or a predetermined number of iteration positions have been resolved).

[0060] Otherwise, method 400 proceeds to 414. At 414, the system can add the next finer resolution to the set of resolutions and update the error threshold. For example, the error threshold can be reduced to a value proportional to the size of the next finer resolution (e.g., half the size of the next finer resolution). Once the next finer resolution is added to the set of resolutions and the error threshold is updated, method 400 can return to 410 and the system can align the voxels of the resolution set again, as described above.

[0061] Figure 5 5 is a block diagram of an example system 500 for implementing a multi-resolution voxel spatial alignment system, as described herein. In this embodiment, system 502 is an autonomous vehicle 502, which may include a vehicle computing device 504, one or more sensor systems 506, one or more communication connections 508, and one or more drive systems 510.

[0062] The vehicle computing device 504 may include one or more processors 512 (or processing resources) and a computer-readable medium 514 communicatively coupled to the one or more processors 512. In the example shown, the vehicle 502 is an autonomous vehicle; however, the vehicle 502 may be any other type of vehicle, or any other system (e.g., a robotic system, a camera-enabled smartphone, etc.). In the example, the computer-readable medium 514 of the vehicle computing device 504 stores a multi-resolution voxel space component 516, a planning component 518, a prediction component 520, and other components 522 related to the autonomous vehicle. The computer-readable medium 514 may also store sensor data 524 and a multi-resolution voxel space 526. In some embodiments, it should be understood that the system and the data stored on the computer-readable medium may additionally or alternatively be accessible to the vehicle 502 (e.g., stored on other computer-readable media remote from the vehicle 502, or accessible by other means).

[0063] The multiresolution voxel space generation component 516 may generate a multiresolution voxel space as described above, and the multiresolution voxel space component 516 may output an alignment between two or more multiresolution voxel spaces as described above.

[0064] In some embodiments, the prediction component 520 can be configured to estimate a current characteristic or state of an object (e.g., a vehicle, a pedestrian, an animal, etc.) and / or predict a future characteristic or state of the object, such as posture, speed, trajectory, velocity, yaw, yaw rate, roll, roll rate, pitch, pitch rate, position, acceleration, or other characteristic based at least in part on the multi-resolution voxel space 526 output by the multi-resolution voxel space component 516.

[0065] The vehicle 502 may also include one or more communication connections 508 that enable communication between the vehicle 502 and one or more other local or remote computing devices. For example, the communication connection 508 may facilitate communication with other local computing devices on the vehicle 502 and / or the drive system 510. In addition, the communication connection 508 may allow the vehicle 502 to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). The communication connection 508 may also enable the vehicle 502 to communicate with a remote operation computing device or other remote service.

[0066] Communication connection 508 may include a physical and / or logical interface for connecting vehicle computing device 504 to another computing device (e.g., computing device 530) and / or a network such as network 528. For example, communication connection 508 may enable communication based on Wi-Fi, such as frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables the respective computing devices to interact with other computing devices. In some examples, the communication connection 508 of the vehicle 502 can transmit or send the multi-resolution voxel space 526 to the computing device 530.

[0067] In at least one example, the sensor system 506 may include: a lidar sensor, a radar sensor, an ultrasonic sensor, a sonar sensor, a position sensor (e.g., GPS, compass, etc.), an inertial sensor (e.g., an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, etc.), a camera (e.g., RGB, IR, intensity, depth, time of flight, etc.), a microphone, a wheel encoder, an environmental sensor (e.g., a temperature sensor, a humidity sensor, a light sensor, a pressure sensor, etc.), and one or more time of flight (ToF) sensors, etc. The sensor system 506 may include multiple instances of each of these or other types of sensors. For example, the lidar sensor may include separate lidar sensors located at the corners, front, rear, sides, and / or top of the vehicle 502. As another example, the camera sensor may include multiple cameras arranged at various locations around the exterior and / or interior of the vehicle 502. The sensor system 506 may provide input to the vehicle computing device 504. Additionally or alternatively, the sensor system 506 may transmit the sensor data to the one or more computing devices 530 via the one or more networks 528 after a predetermined period of time has passed, in near real time, or the like.

[0068] In at least one example, the vehicle 502 may include one or more drive systems 510. In some examples, the vehicle 502 may have a single drive system 510. In at least one example, if the vehicle 502 has multiple drive systems 510, the individual drive systems 510 may be positioned at opposite ends (e.g., front and rear, etc.) of the vehicle 502. In at least one example, the drive system 510 may include one or more sensor systems 506 to detect conditions of the drive system 510 and / or the surrounding environment of the vehicle 502, as discussed in the text. By way of example and not limitation, the sensor system 506 may include: one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the wheels of the drive system; inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive system; cameras or other image sensors, ultrasonic sensors to acoustically detect objects in the surrounding environment of the drive system; lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to the drive system 510. In some cases, sensor system 506 on drive system 510 may overlap or supplement a corresponding system on vehicle 502 .

[0069] In at least one example, the components discussed herein can process the sensor data 524 as described above, and can send their respective outputs to the one or more computing devices 530 via the one or more networks 528. In at least one example, the components discussed herein can send their respective outputs to the one or more computing devices 530 at a particular frequency, after a predetermined period of time, in near real time, and the like.

[0070] In some examples, the vehicle 502 may send sensor data to one or more computing devices 530 via the network 528. In some examples, the vehicle 502 may send raw sensor data 524 or processed multi-resolution voxel space 526 to the computing device 530. In other examples, the vehicle 502 may send processed sensor data 524 and / or representations of sensor data (e.g., object-aware trajectories) to the computing device 530. In some examples, the vehicle 502 may send sensor data to the computing device 530 at a particular frequency, after a predetermined period of time has passed, in near real time, etc. In some cases, the vehicle 502 may send sensor data 524 (raw or processed) to the computing device 530.

[0071] The computing system 530 may include a processor 532 and a computer-readable medium 534 storing a multi-resolution voxel space component 536, as well as other components 538, sensor data 540 received from the vehicle 502, and a multi-resolution voxel space 542. In some examples, the multi-resolution voxel space component 536 may be configured to generate a multi-resolution voxel space 542 or align the multi-resolution voxel space 542 generated from data captured by multiple vehicles 502 to form a more complete scene of various physical environments and / or connect various scenes together as a signal-extended physical environment. In some cases, the multi-resolution voxel space component 536 may be configured to generate one or more models based on the sensor data 524, which can be used for machine learning and / or future code testing.

[0072] The processor 512 of the vehicle 502 and the processor 532 of the computing device 530 can be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, the processors 512 and 532 can include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data to transform the electronic data into other electronic data that can be stored in registers and / or computer-readable media. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices can also be considered processors configured to implement encoded instructions.

[0073] Computer readable media 514 and 534 are examples of non-temporary computer readable media. Computer readable media 514 and / or 534 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions belonging to various systems. In various embodiments, the computer readable medium can be implemented using any suitable computer readable medium technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of computer readable medium capable of storing information. The architecture, system, and separate elements described herein may include many other logical components, programmed components, and physical components, of which those components shown in the accompanying drawings are merely examples relevant to the discussion herein.

[0074] As can be appreciated, the components discussed herein are described as separate for illustrative purposes. However, the operations performed by various components may be combined or performed in any other component.

[0075] It should be noted that although Figure 5502 may be associated with computing device 530, and / or components of computing device 530 may be associated with vehicle 502. That is, vehicle 502 may perform one or more functions associated with computing device 530, and vice versa.

[0076] Figure 6 600 is a schematic diagram of an example resolution of a multi-resolution voxel space 602 compared to a representation 604 of captured data, as described herein. In this example, the multi-resolution voxel space 602 includes multiple layers or resolutions, generally represented by 602(A)-(C), semantic layers, generally represented by 606(A)-(C). For example, in this example, voxels of layer 606(A) correspond to leaves and are represented as shadow voxels with dark outlines, voxels of layer 606(B) correspond to ground planes and are represented as non-shadow voxels with light outlines, and voxels of layer 606(C) correspond to buildings and stationary objects and are represented as non-shadow voxels with dark outlines. As shown, the multi-resolution voxel space 602 and the representation 604 both correspond to physical locations or spaces in the real world.

[0077] Sample Clauses

[0078] A. A system, comprising: one or more processors, and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein when the instructions are executed, the system performs the following operations, including: receiving a target multi-resolution voxel space, the target multi-resolution voxel space comprising a first plurality of voxels representing discrete volumetric portions of a physical environment and defined by a first resolution and a second resolution, the first resolution being coarser than the second resolution; receiving a reference multi-resolution voxel space, the reference multi-resolution voxel space comprising a second plurality of voxels representing discrete volumetric portions of a physical environment and defined by the first resolution and the second resolution; and generating a target multi-resolution voxel space based at least in part on a target multi-resolution voxel space. The invention relates to a method for determining an autonomous vehicle that is operable to identify a first voxel of a first resolution in a target multiresolution voxel space and a second voxel of a first resolution in a reference multiresolution voxel space; determining a first quality metric for the first resulting voxel based at least in part on a number of points associated with the first resulting voxel, a semantic classification associated with the first resulting voxel, a feature value associated with the first resulting voxel, or one or more of the first resolution; determining a first residual based at least in part on the first resulting voxel; determining an alignment between the target multiresolution voxel space and the reference multiresolution voxel space based at least in part on the first quality metric and the first residual; and performing operation of an autonomous vehicle based at least in part on the alignment.

[0079] B. A system as described in claim A, wherein the operation further includes, in response to determining that the average residual of the alignment is less than or equal to an error threshold: determining a second resulting voxel based at least in part on a third voxel of the second resolution of the target multi-resolution voxel space and a fourth voxel of the second resolution of the reference multi-resolution voxel space; determining a second quality metric of the second resulting voxel based at least in part on a number of points associated with the second resulting voxel, a semantic classification associated with the second resulting voxel, a feature value associated with the second resulting voxel, or one or more of the second resolution; determining a second residual based at least in part on the second resulting voxel; determining a final alignment based at least in part on the second quality metric and the second residual; and performing operation of the autonomous vehicle based at least in part on the final alignment.

[0080] C. The system of claim A, wherein the first voxel and the second voxel are determined based on having a minimum distance between their centers of mass.

[0081] D. The system of claim A, wherein determining the alignment further comprises scaling the first residual by a first quality metric.

[0082] E. A system as described in claim A, wherein determining the first residual includes: determining a data set associated with the first voxel and the second voxel; determining a eigenvector of the data set; determining a vector between an average value of the data associated with the first voxel and an average value of the data associated with the second voxel; and determining a dot product between the eigenvector and the vector as the residual.

[0083] F. A system as described in claim A, wherein the first voxel includes a first eigenvalue, a second eigenvalue, and a third eigenvalue, and the operation also includes determining that at least one of the first eigenvalue, the second eigenvalue, or the third eigenvalue is less than or equal to a size threshold: determining a first weight based at least in part on the size of the first eigenvalue; determining a second weight based at least in part on the size of the second eigenvalue; determining a third weight based at least in part on the size of the third eigenvalue; and before determining the first resulting voxel, applying the first weight to the first eigenvalue, applying the second weight to the second eigenvalue, and applying the third weight to the third eigenvalue.

[0084] G. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including: determining a quality metric for voxels associated with a first multi-resolution voxel space and a second multi-resolution voxel space based at least in part on a number of points associated with the voxels, a semantic classification associated with the voxels, a feature value associated with the voxels, or a resolution associated with the voxels; determining a residual associated with the voxels; and determining an alignment between the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on the quality metric and the residual.

[0085] H. A non-transitory computer-readable medium as described in clause G, wherein a first multi-resolution voxel space includes a first plurality of voxels representing discrete volumetric portions of a physical environment and defined by a first resolution and a second resolution, the first resolution being coarser than the second resolution; and a second multi-resolution voxel space includes a second plurality of voxels representing discrete volumetric portions of a physical environment and defined by the first resolution and the second resolution; and the resolution is the first resolution.

[0086] I. A non-transitory computer-readable medium as described in clause H, wherein the voxel is a first voxel and is associated with a first resolution; and the operation further includes: in response to determining that the average residual of the alignment is less than or equal to a threshold: determining a second quality metric for a second voxel associated with the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on a number of points associated with the second voxel, a semantic classification associated with the second voxel, a feature value associated with the second voxel, or a resolution associated with the voxel, a second voxel of a second resolution; determining a second residual associated with the second voxel; and generating an updated alignment based at least in part on the second quality metric, the second residual, and the alignment.

[0087] J. A non-transitory computer-readable medium as described in clause I, comprising one or more of: controlling a vehicle based at least in part on the updated alignment, or creating a map based at least in part on the updated alignment.

[0088] K. The non-transitory computer-readable medium of clause G, wherein the voxel is a statistical combination of first data associated with a voxel in a first multi-resolution voxel space and second data associated with a voxel in a second multi-resolution voxel space.

[0089] L. A non-transitory computer-readable medium as described in clause K, wherein determining the residual further comprises: determining a dot product of a feature vector of the voxel and a vector indicating a separation between a voxel in the first multi-resolution voxel space and a voxel in the second multi-resolution voxel space.

[0090] M. A non-transitory computer-readable medium as described in clause G, wherein the voxel is a first voxel, and determining a first residual includes: determining a data set associated with the first voxel and the second voxel; determining a eigenvector of the data set; determining a vector between an average value of the data associated with the first voxel and an average value of the data associated with the second voxel; and determining a dot product between the eigenvector and the vector as a residual.

[0091] N. A method comprising: determining a quality metric for voxels associated with a first multiresolution voxel space and a second multiresolution voxel space based at least in part on one or more of a number of points associated with the voxel, a semantic classification associated with the voxel, a feature value associated with the voxel, or a resolution associated with the voxel; determining a residual associated with the voxel; and determining an alignment between the first multiresolution voxel space and the second multiresolution voxel space based at least in part on the quality metric and the residual.

[0092] O. A method as described in clause N, wherein: the first multi-resolution voxel space includes a first plurality of voxels representing discrete volumetric portions of a physical environment and defined by a first resolution and a second resolution, the first resolution being coarser than the second resolution; and the second multi-resolution voxel space includes a second plurality of voxels representing discrete volumetric portions of a physical environment and defined by the first resolution and the second resolution; and the resolution is the first resolution.

[0093] P. A method as described in clause N, wherein: the voxel is a first voxel and is associated with a first resolution: and the method further includes: in response to determining that the average residual of the alignment is less than or equal to a threshold: determining a second quality metric of a second voxel associated with the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on a number of points associated with the second voxel, a semantic classification associated with the second voxel, a feature value associated with the second voxel, or a resolution associated with the voxel, a second voxel of a second resolution; determining a second residual associated with the second voxel; and generating an updated alignment based at least in part on the second quality metric, the second residual, and the alignment.

[0094] Q. The method of clause P further includes: in response to determining that the updated alignment achieves or exceeds a convergence threshold, outputting the updated alignment.

[0095] R. A method as described in clause P, wherein the voxel is a statistical combination of first data associated with a voxel in a first multi-resolution voxel space and second data associated with a voxel in a second multi-resolution voxel space.

[0096] S. A method as described in clause R, wherein determining the residual further comprises: determining the dot product of the unit vector of the voxel and the average value of the voxels in the first multi-resolution voxel space minus the average value of the voxels in the second multi-resolution voxel space.

[0097] T. A method as described in clause N, wherein the operation also includes: performing operation of the autonomous vehicle based at least in part on the alignment.

[0098] Although the example clauses described above are described with respect to a specific implementation, it should be understood that in the context of this article, the content of the example clauses may also be implemented via a method, an apparatus, a system, a computer-readable medium, and / or another implementation. In addition, any one of the example ATs may be implemented alone or in combination with any other one or more of the example ATs.

[0099] in conclusion

[0100] As can be appreciated, the components discussed herein are described as separate for illustrative purposes. However, the operations performed by each component can be combined or performed in any other component. It should also be understood that the components or steps discussed with respect to one example or implementation can be used in combination with the components or steps of other examples. For example, Figure 5 Components and directives can be used Figure 1-Figure 4 methods and processes.

[0101]

[0066] While one or more examples of the techniques described herein have been described, various alternatives, additions, permutations, and equivalents thereof are included within the scope of the techniques described herein.

[0102] In the description of the examples, reference is made to the accompanying drawings forming a part thereof, which illustrate specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used, and changes or substitutions such as structural changes may be made. Such examples, changes or substitutions do not necessarily deviate from the scope of the subject matter that is desired to be claimed. Although the steps herein may be presented in a certain order, in some cases, the ordering may be changed so that certain inputs are provided at different times or in different orders without changing the functionality of the described systems and methods. The disclosed processes may also be performed in different orders. In addition, the various calculations described herein do not need to be performed in the order disclosed, and other examples of alternative ordering using calculations may be easily implemented. In addition to being reordered, in some cases, the calculations may also be decomposed into sub-calculations with the same results.

Claims

1. A method comprising: determining a quality metric for a voxel associated with a first multi-resolution voxel space and a second multi-resolution voxel space based at least in part on one or more of a number of points associated with the voxel, a semantic classification associated with the voxel, a feature value associated with the voxel, or a resolution associated with the voxel; determining a residual associated with the voxel; as well as Based at least in part on the quality metric and the residual, an alignment between the first multiresolution voxel space and the second multiresolution voxel space is determined.

2. The method of claim 1, wherein: the first multi-resolution voxel space comprising a first plurality of voxels representing discrete volumetric portions of a physical environment and defined by a first resolution and a second resolution, the first resolution being coarser than the second resolution; the second multi-resolution voxel space comprising a second plurality of voxels representing discrete volumetric portions of the physical environment and defined by the first resolution and the second resolution; as well as The resolution is the first resolution.

3. The method according to any one of claims 1 or 2, wherein: The voxel is a first voxel and is associated with the first resolution, and the method further comprises: In response to determining that the average residual of the alignment is less than or equal to a threshold: determining a second quality metric for the second voxel associated with the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on one or more of a number of points associated with the second voxel, a semantic classification associated with the second voxel, a feature value associated with the second voxel, or a resolution associated with the voxel, the second voxel at the second resolution; determining a second residual associated with the second voxel; and An updated alignment is generated based at least in part on the second quality metric, the second residual, and the alignment.

4. The method according to any one of claims 1 to 3, further comprising one or more of the following: controlling the vehicle based at least in part on the updated alignment, or A map is created based at least in part on the updated alignment.

5. The method according to any one of claims 1 to 4, wherein: The voxel is a statistical combination of first data associated with a voxel in the first multi-resolution voxel space and second data associated with a voxel in the second multi-resolution voxel space.

6. The method of claim 5, wherein: Determining the residual also includes determining a dot product of an eigenvector of the voxel and a vector indicative of a separation between a voxel of the first multiresolution voxel space and a voxel of the second multiresolution voxel space.

7. The method according to any one of claims 1 to 6, wherein: The voxel is a first voxel, and determining the first residual comprises: determining a data set associated with the first voxel and the second voxel; determining a feature vector of the data set; determining a vector between the mean of the data associated with the first voxel and the mean of the data associated with the second voxel; and The dot product between the eigenvector and the vector is determined as a residual.

8. A computer program product comprising coded instructions for implementing the method according to any one of claims 1 to 7 when the instructions are executed on a computer.

9. A system comprising: one or more processors; as well as One or more non-transitory computer-readable media storing instructions that, when executed, cause the one or more processors to perform operations including: determining a quality metric for a voxel associated with a first multi-resolution voxel space and a second multi-resolution voxel space based at least in part on one or more of a number of points associated with the voxel, a semantic classification associated with the voxel, a feature value associated with the voxel, or a resolution associated with the voxel; determining a residual associated with the voxel; and Based at least in part on the quality metric and the residual, an alignment between the first multiresolution voxel space and the second multiresolution voxel space is determined.

10. The system of claim 9, wherein: the first multi-resolution voxel space comprising a first plurality of voxels representing discrete volumetric portions of a physical environment and defined by a first resolution and a second resolution, the first resolution being coarser than the second resolution; and the second multi-resolution voxel space comprising a second plurality of voxels representing discrete volumetric portions of the physical environment and defined by the first resolution and the second resolution; as well as The resolution is the first resolution.

11. A system as claimed in any one of claims 9 or 10, wherein: The voxel is a first voxel and is associated with the first resolution: and The operations also include: In response to determining that the average residual of the alignment is less than or equal to a threshold: determining a second quality metric for the second voxel associated with the first multi-resolution voxel space and the second multi-resolution voxel space based at least in part on one or more of a number of points associated with the second voxel, a semantic classification associated with the second voxel, a feature value associated with the second voxel, or a resolution associated with the voxel, the second voxel at the second resolution; determining a second residual associated with the second voxel; and An updated alignment is generated based at least in part on the second quality metric, the second residual, and the alignment.

12. The system of any one of claims 9 to 11, wherein the operations further comprise: In response to determining that the updated alignment achieves or exceeds a convergence threshold, the updated alignment is output.

13. The system of any one of claims 9 to 12, wherein: The voxel is a statistical combination of first data associated with a voxel in the first multi-resolution voxel space and second data associated with a voxel in the second multi-resolution voxel space.

14. The system of any one of claims 9 to 13, wherein: Determining the residual also includes determining a dot product of a unit vector of the voxel and a mean value of the voxels in the first multiresolution voxel space minus a mean value of the voxels in the second multiresolution voxel space.

15. The system of any one of claims 9 to 14, wherein: The operations also include performing an operation of the autonomous vehicle based at least in part on the alignment.

Images