Methods and Apparatus

Abstract

A computer-implemented method for locating a radar sensor includes obtaining a first radar scan of a first environment of the radar sensor, the first environment including a power range spectrum set including a first power range spectrum; extracting from the first radar scan a first landmark set including a first landmark defined by a range and an azimuth angle; calculating respective first descriptor sets including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets of each environment and calculating a descriptor reference set for each of the landmark reference sets; matching the first descriptor set with a corresponding first descriptor reference set; and identifying a first location of the radar sensor using a first result of the matching.

Description

[Technical Field]

[0001] The present invention relates to radar sensor location. [Background technology]

[0002] To navigate reliably within an environment, autonomous vehicles must achieve robust localization and navigation despite changing conditions (e.g., lighting and weather) and moving objects (e.g., pedestrians and other vehicles). Currently, most platforms employ lidar, vision, GPS, internal sensors, or a combination of these systems to acquire information about their surroundings and perform motion estimation. Lidar is very fast and high-resolution, but is sensitive to weather conditions, especially rain and fog, cannot penetrate first-encounter surfaces, and has a much lower practical range (e.g., 50–100 m). Vision systems are versatile and inexpensive, but are easily compromised by scene changes, such as insufficient lighting or the sudden appearance of adverse weather conditions, such as snow or rain. Both optical sensors provide reliable results only for short-range measurements. Typical GPS systems offer meter-range accuracy but rely on external infrastructure, frequently experiencing reception loss near obstacles. Furthermore, proprioceptive sensors such as wheel encoders and IMUs are subject to significant systematic errors (such as drift), among other deleterious effects.

[0003] In contrast, radar is a long-range (e.g., up to 600 m) vehicle-mounted system that is independent of lighting conditions, performs well in a variety of weather conditions, and is more affordable and efficient than lidar. Because radar has a relatively long wavelength, it can see through certain materials, returning multiple readings from the same transmission and generating a grid representation of the environment. As a result, radar sensors detect stable, long-range features in the environment.

[0004] However, conventional radar localization and ranging methods do not provide sufficiently accurate and / or precise localization, e.g., attitude estimation, for purposes such as route planning. Typically, navigation is at a higher level, such as "go to location X," while route planning is at a lower level, defining the exact path a vehicle will traverse. More generally, conventional approaches to radar localization and ranging are not accurate or robust enough for precise attitude estimation for the purposes of accurate localization or large-scale localization of autonomous vehicles, more generally vehicles such as landcraft or watercraft.

[0005] Many applications, such as mining or off-road environments, require radar-only localization and ranging systems. These applications require radar-based localization and ranging systems to be deployed and operated on low-power hardware with limited computational resources. An example of such a radar-based localization and ranging system includes the Navtech CTS350-X sensor (available from Navtech Radar Limited, UK), which has available computations from a 1.6 GHz quad-core ARM A53 processor that consumes close to 1 W of power. Conventional radar-based localization and ranging systems are too slow on these types of low-specification computational platforms to be practical for real-time control of vehicles, such as autonomous vehicles, or more generally, land or water vehicles.

[0006] Therefore, there is a need to improve radar sensor localization. Summary of the Invention [Problem to be solved by the invention]

[0007] One object of the present invention is to provide a method for localizing a radar sensor that at least partially obviates or mitigates, among other things, at least some of the drawbacks of the prior art mentioned herein or elsewhere. For example, it is an object of embodiments of the present invention to provide a method for localizing a radar sensor that provides sufficiently accurate and / or precise localization, e.g., attitude estimation, for navigation, etc. For example, it is an object of embodiments of the present invention to provide a reliable and accurate radar-only system for precise driving range measurements and localization attitude estimation. For example, it is an object of embodiments of the present invention to provide a method for localizing a radar sensor that is implemented by a relatively low-spec computing platform during real-time control of a vehicle, such as an autonomous vehicle, or more generally, a land or water vehicle. For example, it is an object of embodiments of the present invention to provide a fast and efficient implementation on a low-power embedded platform. [Means for solving the problem]

[0008] A computer-implemented method for locating a radar sensor according to a first aspect includes: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Includes.

[0009] The term "descriptor" may be understood to mean a vector whose elements define the location of a point in the radar point cloud of a detected feature relative to an origin, which may be the radar sensor that captured the radar scan that detected the descriptor.

[0010] The first landmark from the first set of landmarks may correspond to at least one landmark from the first set of landmarks. The at least one landmark may be an arbitrarily selected landmark from the first set of landmarks.

[0011] A computer-implemented method for locating a radar sensor according to a second aspect includes: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, Matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptors to first projected descriptors.

[0012] The step of projecting the first descriptor into the first projected descriptor may include reducing the dimension of the first descriptor to the size of the first projected descriptor. Despite the dimension reduction, the first descriptor and the first projected descriptor may be one-dimensional vectors.

[0013] A computer-implemented method for locating a radar sensor according to a third aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, The method comprises: further comprising projecting the first descriptor into a first projected descriptor, wherein the first descriptor has dimension N and the first projected descriptor has dimension M, where M≠N; The step of matching the first descriptor set with a corresponding first descriptor reference set includes the step of matching a first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.

[0014] A computer-implemented method for locating a radar sensor according to a fourth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, The method comprises: The method further includes calculating one or more values ​​in response to a request, storing the calculated one or more values, and returning the stored one or more values ​​or one or more values ​​derived therefrom in response to a subsequent request.

[0015] A computer-implemented method for locating a radar sensor according to a fifth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, The method comprises: Further steps include calculating two or more values ​​simultaneously and / or calculating in a correlated manner using two or more values.

[0016] A computer-implemented method for locating a radar sensor according to a sixth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first set of descriptors including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, The method comprises: representing the first set of landmarks as a first signature; representing the landmark reference set as respective reference signatures; correlating the first signature with a reference signature to approximate the first position of the radar sensor; Further includes:

[0017] The term "signature" may be understood to mean a vector containing a number of values, each corresponding to a count of the number of features within a circular segment of a radar scan.

[0018] A computer-implemented method for controlling a land or water vehicle including a radar sensor according to a seventh aspect comprises: locating a radar sensor according to the first, second, third, fourth, fifth and / or sixth aspects; controlling the land or water vehicle using the first position; Includes.

[0019] A computer according to an eighth aspect includes a processor and memory configured to perform the methods according to the first, second, third, fourth, fifth, sixth and / or seventh aspects.

[0020] A computer program according to a ninth aspect comprises instructions which, when executed by a computer including a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect, the third aspect, the fourth aspect, the fifth aspect, the sixth aspect and / or the seventh aspect.

[0021] A non-transitory computer-readable storage medium according to a tenth aspect includes instructions that, when executed by a computer including a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect, and / or the third aspect.

[0022] A land or water vehicle according to an eleventh aspect includes a radar sensor and a computer according to the eighth aspect.

[0023] (Detailed Description of the Invention) According to the present invention there is provided a method for locating a radar sensor, and a method for controlling a land or water vehicle, as set out in the accompanying claims. Further features of the invention will become apparent from the dependent claims and the following description.

[0024] (Radar sensor location) A computer-implemented method for locating a radar sensor according to a first aspect includes: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Includes:

[0025] In this manner, because the radar sensor is located by matching the first set of descriptors with a corresponding first reference set of descriptors, e.g., provided by mapping the environment, it is located sufficiently accurately and / or precisely, e.g., for navigation of a land or water vehicle, thereby providing a reliable and accurate radar-only system for precise driving range measurements and localization attitude estimation. Furthermore, because the radar sensor is located by matching the first set of descriptors with a corresponding first reference set of descriptors, the method can be implemented by a relatively low specification computing platform while controlling the land or water vehicle in real time, thereby providing a fast and efficient implementation on, e.g., a low power embedded platform.

[0026] The method is a computer-implemented method. Suitable computing platforms (i.e., computers including a processor and memory) are known, for example based on a 1.6 GHz quad-core ARM A53 processor or similar, with power consumption approaching 1 W. In one example, the method is implemented on a computer with power consumption of at most 5 W, preferably at most 3 W, and more preferably at most 1 W.

[0027] This method is known as localization (also known as position determination), e.g., estimating (to a certain accuracy) a geographical location, e.g., represented by GPS coordinates and / or orientation, as is the term in the art.

[0028] Suitable radar sensors, such as millimeter-wave radar sensors, are known. For example, the Navtech CTS350-X is a frequency-modulated continuous wave (FMCW) scanning radar with no Doppler information, returning 399 azimuth and 2000 range readings at 0.25 m range resolution, with a beam spread of 2 degrees in azimuth and 25 degrees in elevation starting directly below horizontal. Typically, radar sensors are placed on the roof of a ground vehicle (also known as a land vehicle) or water vehicle with the axis of rotation perpendicular to the surface of the vehicle.

[0029] The method includes obtaining a first radar scan of a first environment of the radar sensor, the first environment including a set of power range spectra including a first power range spectrum (i.e., at least one power range spectrum). The first environment of the radar sensor is understood to be the surrounding environment in which the radar sensor operates, e.g., on-road, off-road, an industrial facility such as a mining operation, or on water, as a term used in the art. The radar scan is typically represented as a set of power range spectra (i.e., a 1D signal), e.g., one power range spectrum per azimuth angle, where each power spectrum is represented as an array or vector of values ​​s(t)∈R. N×1

[0033] In one example, the power range spectrum set includes P power range spectra, where P is a natural number greater than or equal to 1, e.g., 1, 30, 60, 90, 120, 180, 360, 399, 720, 1080, 1440, or more. Increasing P improves azimuth resolution and increases processing. In one example, obtaining a first radar scan of a first environment of a radar sensor includes obtaining (also known as capturing) the first radar scan of the first environment with the radar sensor, e.g., in real time (i.e., while a land or water vehicle including the radar sensor is moving through the first environment).

[0030] The method includes extracting a first landmark set from a first radar scan (also known as radar data or raw radar scene data), the first landmark set including a first landmark (i.e., at least one landmark) defined by a range and an azimuth angle. This step is also known as landmark extraction. The process of landmark extraction is known, and one exemplary process of landmark extraction is described with reference to FIGS. 3 and 4. Typically, a landmark is a static and / or unchanging feature (also known as an object) in an environment, a term used in the art. Typically, a landmark is defined by a range and an azimuth angle in the art. In one example, the first landmark set includes L landmarks, where L is a natural number greater than or equal to 1, such as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 100, 200, 500, or more. In one example, extracting a first landmark set including the first landmark from the first radar scan includes, for example, using a sliding or moving window average filter instead of a sliding or moving window median filter. This reduces operations and therefore improves computational speed without appreciably changing the quality of the extracted landmarks. In one example, extracting a first landmark set including the first landmark from the first radar scan includes using non-maximum suppression. For example, in areas with relatively high radar reflectivity, multiple reflections may result in multiple detections at different ranges. Non-maximum suppression can be used to remove or eliminate such repetitive patterns.

[0031] The method includes calculating a first set of descriptors (also known as scene point descriptors) for each of the first landmark sets, the first descriptor defining the first landmark by its respective relative range and azimuth with respect to one or more landmarks included in the first landmark set. This step is also known as a first substep of pose estimation. It should be understood that the descriptors are unary descriptors for each landmark and define the interrelationships between the landmarks. In one example, the first descriptor is represented as a vector of values ​​that uniquely describes the first landmark. In this manner, the landmark can be identified and matched in other radar scans. In one example, the first descriptor identifies the first landmark by radial statistics of neighboring landmarks, e.g., by both range and azimuth. Processes for calculating descriptors are known, and one exemplary process for calculating descriptors is described with reference to FIG. 5.

[0032] The method includes accessing one or more landmark reference sets for each environment and calculating a descriptor reference set for each of the landmark reference sets. It will be appreciated that the landmark reference sets may include a landmark map as stored in a landmark database and / or be such a landmark map extracted from radar scans previously acquired by the radar sensor and / or another radar sensor to provide known references for locating the radar sensor.

[0033] The method includes matching the first descriptor set with a corresponding (i.e., best-matching) first descriptor reference set. This step is also known as a second substep of pose estimation and is known as data association. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes aligning the first descriptor set with the first descriptor reference set and estimating a difference therebetween. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes aligning the first descriptor set with each descriptor reference set, e.g., each descriptor reference set, estimating each difference therebetween, and selecting the corresponding first descriptor reference set with the smallest difference. The matching process is known, and one exemplary matching process is described with reference to FIGS. 6, 7, and 8.

[0034] The method includes identifying a first location of the radar sensor using a first result of the matching, such that the first location of the radar sensor is identified relative to a corresponding (i.e., best matching) first descriptor reference set.

[0035] (Driving range measurement) In one example, the method comprises: obtaining a second radar scan of the first environment of the radar sensor; extracting a second set of landmarks from the second radar scan, the second set including the first landmark; calculating a second set of descriptors for each of the second set of landmarks, the second set of descriptors including the first descriptor; matching the second descriptor set with a corresponding second descriptor reference set; determining a second location of the radar sensor using a second result of the matching; determining a motion of the radar sensor using the second position and the first position; Includes.

[0036] In this manner, the motion (eg, velocity) of the radar sensor is calculated using the second position and the first position and, implicitly, the times of the second radar scan and the first radar scan, respectively.

[0037] In one example, the method includes repeatedly obtaining, eg, periodically or intermittently, radar scans of a first environment of the radar sensor and repeatedly calculating, mutatis mutandis, the motion of the radar sensor.

[0038] (effect) The method according to the first aspect is suitable for implementation by a low-power computing platform, in particular the method optionally comprises an algorithm implemented in a highly optimized manner that allows it to run on a low-power computing platform, as explained below.

[0039] (Dimensional Search) In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptors into first projected descriptors. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor set into each first projected descriptor set. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor reference set into each first projected descriptor reference set. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes comparing the first projected descriptor set including the first projected descriptor with the corresponding first projected descriptor reference set.

[0040] The method includes matching a first set of descriptors with a corresponding (i.e., best-matching) first reference set of descriptors. Preferably, the method provides a fast and efficient search for matching descriptors from one scan with those of another scan. For example, given point descriptors for landmarks in scan A, a search is performed for the best-matching point descriptor (and therefore landmark) in scan B. A conventional approach is to perform a full comparison for each landmark in scan A with all landmarks in scan B. However, such a conventional approach produces quadratic algorithms that are slow and do not scale well to the number of descriptors in each set. The inventors have developed two specific techniques for matching a first set of descriptors with a corresponding first reference set of descriptors: eigenprojection and distance projection.

[0041] (Proper Projection) In one example, matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptors to a first unique projected descriptor, the first descriptor having a dimension n>1, and the first projected unique descriptor having a dimension n p = 1. Thus, projecting the first descriptors into a lower dimensional space allows for faster lookups, resulting in fewer comparisons and less computation. More specifically, projecting the first set of descriptors into a lower dimensional space allows for easier (fewer comparisons and operations) and faster comparisons, thereby improving the speed of unary candidate matching, which matches points in one scan with their best matches in other scans.

[0042] for example, Take input = descriptor (n-dimensional, say 700-d) and project it into a much smaller (1D space) specified by the projection direction. Output = For a given point descriptor in scan A, the output is the M closest point descriptors in scan B.

[0043] In one example, matching the first descriptor set with the corresponding first descriptor reference set includes comparing the first unique projection descriptor set including the first unique projection descriptor with the corresponding first unique projection descriptor reference set. Thus, the first descriptor reference set is a set of descriptors that are in a dimension n of both the first unique projection descriptor set and the first unique projection descriptor reference set. p = 1.

[0044] In one example, comparing the first unique projection descriptor set including the first unique projection descriptor with the corresponding first unique projection descriptor reference set includes identifying M closest unique projection descriptors in the corresponding first unique projection descriptor reference set. It should be understood that M is a natural number greater than or equal to 1 and less than the number of unique projection descriptors included in the first unique projection descriptor reference set, i.e., a subset of the first unique projection descriptor reference set is identified. In this way, the number of closest unique projection descriptors identified for subsequent matching is reduced, thereby improving efficiency.

[0045] More specifically, the descriptor projection can also be described as follows.

[0046] The files used as input to the script are point_descriptorrs_urban.txt, and point_descriptors_quarry.txt is.

[0047] Both files contain point descriptors of the radar landmark scene (landmarks extracted from the raw scans) in a text-readable format.

[0048] The format of the file is scan with each line containing a point descriptor for each point in the scan separated by a semicolon. So a file for 20 scans would be 20 lines long. If there are 600 points in each scene, there would be 599 (600 - 1) semicolons per line to separate the 600 point descriptors. If 700-dimensional point descriptors were used, there would be 700 comma-separated numbers between the two semicolons on a line. So there would be 420,000 (600 points * 700 descriptors) values ​​per line.

[0049] S1PD1;S1PD2;S1PD3;…;S1PD600; S2PD1;S2PD2;S2PD3;…;S2PD600; … S19PD1;S19PD2;S19PD3;…;S19PD600; S20PD1;S20PD2;S20PD3;…;S20PD600;

[0050] In the above, S corresponds to the scan number and PD corresponds to the point descriptor of that scan.

[0051] Figure 11 shows the code used to read the text file.

[0052] The Matlab strsplit script is used to read the scan-delimited text files of point descriptors for the points in each scene.

[0053] K is a matrix of size row by column (with elements zero), where rows = number of points in each scan and columns = 700 (size of point descriptor). K is set by converting a string to a numeric representation.

[0054] D is a collection of scans, and D{i} is the matrix of point descriptors for scan i (corresponding to row i in the text file). DUrban can be a collection of scans from point_descriptors_urban.txt. DQuarry can be a collection of scans from point_descriptors_quarry.txt.

[0055] DUrban and DQuarry are concatenated onto D, followed by a random permutation of integers. This can be achieved by running randperm to mix up all the scans.

[0056] The full scan set is split into training scenes (approximately 80% of scans) and testing (approximately 20% of scans). The training scenes may be concatenated along axis 1 using the Matlab cat function. This means that matrices are added vertically on top of each other, so the number of columns is also equal to the size of the point descriptor, and the number of rows is point_per_scan * number of scans in the training set.

[0057] Figure 12 shows the code used to calculate trained_mean and trained_basis.

[0058] Afterwards, we call decompose, which does all the heavy lifting to compute trained_mean and trained_basis. [projection_mean,projection_basis]=decompose(training_descriptors,num_projection_dimensions); (where num_projection_dimensions is the number of dimensions to be projected, which is 1 here)

[0059] FIG. 13 shows the code used to reduce the dimension of the matrix.

[0060] The Matlab mean function is used to obtain a vector of mean values ​​for each element of the point descriptor. Such a vector of mean values ​​is later saved as trained_mean. For example, if A is a matrix, mean(A) returns a row vector containing the mean value of each column. This mean value is then used to shift all point descriptors (shifted_observations).

[0061] The Matlab cov function is used, e.g., C = cov(A) returns the covariance. If A is a matrix with columns representing random variables and rows representing observations, then C is the covariance matrix containing the corresponding column variances along the diagonal. In other words, for a matrix A whose columns are each random variable composed of observations, the covariance matrix is ​​the pairwise covariance estimate between each combination of columns. This estimate can be expressed as C(l,j) = cov(A(:,i),A(:,j)).

[0062] It is important to note that the descriptor vectors themselves are used to construct the covariance matrix. In other words, the descriptor elements themselves are used as signals. This differs from prior art methods, where differences are used instead. Using the descriptor elements themselves makes training much easier.

[0063] Given all the observations, the mutual covariance of all point descriptor variables is calculated. For example, a covariance matrix P is generated using P=cov(shifted_observations).

[0064] We then use eig to compute the eigenvalues ​​and eigenvectors of the covariance matrix, which can be explained as A*V=V*D, where [V,D]=eig(A) returns a diagonal matrix D of eigenvalues ​​and a matrix V whose columns are the corresponding right eigenvectors.

[0065] The eigenvalues ​​are then sorted in descending order. This sorting can be described in two steps. In the first step, use B=sort(A,direction) to sort the elements of A in the order specified by the direction using one of the previous syntaxes. 'ascend' indicates ascending order (the default), and 'descend' indicates descending order. In the second step, use [B,I]=sort(_) to return a collection of index vectors for one of the previous syntaxes. I is the same size as A and describes the arrangement of the elements of A in B along the sorted dimension. For example, if A is a vector, then B=A(I).

[0066] The principal eigenvector is then obtained for the future reduced dimension N, i.e., the eigenvector corresponding to the top N eigenvalues ​​after sorting. In this case, since the reduction is to a single dimension, the eigenvector corresponding to the largest eigenvalue is obtained. This single largest eigenvalue is the basis vector.

[0067] It is important to note that dimensionality reduction in this method is a means of "summarizing" or "compressing" descriptors as a first step in the matching process, thereby reducing the search space and speeding up the overall data association process by making them easier to compare and match with other descriptors. Such methods differ from prior art methods that use dimensionality reduction to filter dimensions that do not provide information about what will or will not match.

[0068] Dimensionality reduction can be used in this way to speed up the process of matching one point descriptor with the point descriptor of another scan by reducing the dimensionality of the point descriptor itself.

[0069] Importantly, with regard to training the basis vectors, the present subject matter provides a method for training and generating the basis vectors or principal eigenvectors from the sampled point descriptors themselves.

[0070] (Distance projection) In one example, matching the first descriptor set with the corresponding first descriptor reference set includes identifying M closest descriptors in the corresponding first descriptor reference set and finding a single closest descriptor from the M closest descriptors.

[0071] In one example, the method comprises: summing first absolute differences between the first set of descriptors and the first reference set of descriptors and setting a threshold absolute difference as the summed (i.e., total) first absolute differences; summing second absolute differences between the first set of descriptors and the second reference set of descriptors, with the summing (i.e., running total) of second absolute differences being at most the threshold absolute difference; if the second absolute difference in the sum exceeds a threshold absolute difference, stopping the sum of the second absolute differences and starting a sum of a third absolute difference between the first set of descriptors and a third reference set of descriptors; If the summed second absolute difference does not exceed the threshold absolute difference, resetting the threshold absolute difference as the summed second absolute difference.

[0072] In this way, the process improves early based on the distance metric (i.e., the absolute difference in the sum) to date, which is the sum of absolute differences (also known as the L1 norm), thereby enabling fast searches by reducing the search space based on distance. In other words, the method abandons or moves on from descriptors that already exhibit a larger sum of absolute differences (SAD) than the current best candidate (i.e., the descriptor reference set for which the termination threshold absolute difference was set or reset). This process is sometimes known as the distance trick.

[0073] for example, Input = For a given point descriptor in scan A, the M closest point descriptors in scan B. Output = single closest (by L1-norm) point descriptor from a set of M descriptors. Distance tricks make this calculation (finding the closest descriptor) much faster and more efficient.

[0074] In one example, the method includes ordering (also known as sorting) the descriptor reference set by likelihood of match, e.g., by decreasing likelihood of match. By ordering candidates by likelihood of match in this manner, significant savings are achieved by examining the most likely, and therefore lowest, SAD dF substescriptors first.

[0075] (Descriptor resize) In one example, the method comprises: projecting the first descriptor into a first projected descriptor, the first descriptor having dimension N and the first projected descriptor having dimension M, where M≠N; Matching the first descriptor set with a corresponding first descriptor reference set includes matching the first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.

[0076] Thus, rather than recalculating the first descriptor to a larger or smaller dimension M, the first descriptor is resized (i.e., projected) to a larger or smaller dimension M. Smooth descriptor resizing for velocity control = the ability to resize (i.e., scale) radar point descriptors so that existing calculated descriptors can be converted to larger or smaller sizes rather than recalculating them. Any implementation or method that does this is superior.

[0077] for example, Input = size N, e.g. 700 point descriptors (i.e. vectors of size N). Output = size M, e.g. 400 equivalence point descriptors (i.e. vectors of size M).

[0078] In one example, projecting the first descriptor into the first projected descriptor comprises interpolating its elements, thus reducing the dimension M.

[0079] In one example, projecting the first descriptor into the first projected descriptor comprises averaging or dropping its elements, thus increasing the dimension M.

[0080] (Memory of calculated values) In one example, the method includes calculating one or more values ​​in response to a request, storing the calculated one or more values, and returning the stored one or more values ​​or one or more values ​​derived therefrom in response to a subsequent request.

[0081] The inventors have developed techniques to further optimize the computation, especially for low-power hardware with limited computational resources. The inventors have developed two specific techniques to reduce or minimize repetitive computation: caching and the use of lookup tables.

[0082] (caching) In one example, the one or more values ​​include and / or are a descriptor set, and calculating the one or more values ​​includes calculating the descriptor set. In one example, the one or more values ​​include and / or are a descriptor reference set, and calculating the one or more values ​​includes calculating the descriptor reference set.

[0083] In one example, the method includes storing a descriptor reference set; Matching the first descriptor set with a corresponding first descriptor reference set includes matching the first descriptor set with a corresponding first descriptor set of stored descriptor reference sets.

[0084] In this way, the descriptors are cached (ie, stored) for later use, thereby avoiding recomputation of the descriptors, further improving efficiency.

[0085] For example, for a particular radar scan, e.g., the most recent live scan from a radar sensor, the output and / or intermediate values ​​(i.e., calculated values) of the calculations (typically estimates) for that particular radar scan are stored as follows: 1. This method computes a value only once and then reuses that computed value, which means that this method does not repeat the same computation. 2. This method caches all the calculations for a particular scan together in an instance of the software object.

[0086] for example, Input = Point cloud of landmarks extracted from radar scan. Output = A RadarSceneData object containing the scan and all calculations performed by it (including point descriptors which are matrices of values, each column is the point descriptor of a point, thus the matrix is ​​NxM, where N is the dimension of the point descriptor and M is the number of points in the radar landmark scan).

[0087] More specifically, the following data is required: Landmark point cloud, which is point cloud data of the 3D positions of landmarks. (The landmark point cloud can be provided in the form of a live landmark point cloud or a map point cloud.) A sampling mask vector that defines the mask used by the sampling policy to select landmarks in the returned point cloud. (The sampling mask vector can be constructed using a user-defined sampling policy.) A descriptor template containing trained vectors and basis vectors used to project the complete point descriptor in the scene into a low-dimensional space for fast lookup. (This descriptor template is provided pre-trained.)

[0088] From the required data the following can be calculated: The calculated data can be cached / stored for efficiency and reused later. An intrinsic 3D matrix of the x, y, z positions of the landmarks in the scene. (This matrix is ​​more convenient as the intrinsic type for other calculations.) A matrix of inter-point distances, i.e. the distances from each point to every other point. (This matrix is ​​needed for the later calculation of point descriptors. This matrix is ​​calculated from the landmark points.) A matrix of angles between points. (The angles may be calculated using a graphics processing unit (GPU). The angles may be from each point to every other point, and these angles may be used to calculate the point descriptors. These angles may be calculated from the landmark point cloud.) A point descriptor matrix, which is a matrix of point descriptors of the scene. (This matrix can be computed from landmarks in the point cloud.) A scene descriptor vector, which is a vector of scene descriptors projected into a low-dimensional space (e.g., 1D) by the trained basis vectors. (This makes it easier to reduce the search space when matching point descriptors between scenes, since the search may initially be done in only 1D to obtain a reduced set and search more thoroughly. The scene descriptor vector may be computed using the descriptor template and the computed point descriptor matrix.) A range vector, which is the range of a point within the landmark point cloud. (The range vector can be calculated from the landmark point cloud.)

[0089] (Lookup table) In one example, the method includes calculating one or more values ​​in response to a request, storing the calculated one or more values, and returning the stored one or more values ​​or one or more values ​​derived therefrom in response to a subsequent request.

[0090] In one example, the one or more values ​​include and / or are a set of descriptors and / or an intermediate value thereof, and storing the calculated one or more values ​​includes storing the calculated one or more values ​​in a set of lookup tables including a first lookup table.

[0091] In one example, the step of calculating the first set of descriptors including the first descriptor for the first set of landmarks uses a set of lookup tables including a first lookup table.

[0092] In this way, a lookup table (i.e., a pre-computed lookup table) is used for the complex function, which eliminates the need to repeatedly calculate the complex function, which is more efficient and significantly faster.

[0093] In one example, the method includes generating a first lookup table at compile time (i.e., at the time of compilation, during compilation) and using the first lookup table at run time (i.e., at run time, during execution).

[0094] For example, the "ArcCosineApproximator" class may be used when computing descriptors, providing a fast lookup solution for "acos". A lookup table is generated at compile time using the quotient of two polynomials that approximate "acos", and used at run time.

[0095] In one example, the method includes generating the first lookup table at run time during a first calculation of a given value of the first lookup table.

[0096] (process) In one example, the method includes calculating two or more values ​​simultaneously and / or calculating correlated values ​​using two or more values.

[0097] The inventors have developed techniques to further optimize the computation, especially for low-power hardware with limited computational resources. The inventors have developed two specific techniques to accelerate and / or simplify the computation: the use of parallel processing and triangulation.

[0098] (Parallel processing) In one example, computing a first descriptor set including a first descriptor for a first landmark set includes processing the first landmark set in parallel, e.g., using single instruction multiple data processing (SIMD).

[0099] In this way, the method optimizes the use of available hardware, improving speed and efficiency: for example, the first descriptor set for the first landmark set may be computed in parallel since the same instructions are applied.

[0100] for example, Input = Point cloud of landmarks extracted from radar scan. M is the number of points in the radar landmark scan. Output = Radar point descriptor, a matrix of values ​​(each column is the point descriptor of a point, thus this matrix is ​​NxM, where N is the dimension of the point descriptor and M is the number of points in the radar landmark scan).

[0101] (triangulation) In one example, computing the first descriptor set including the first descriptor for the first landmark set includes triangulating the first landmark with respect to each node and landmark in the first landmark set, thus improving efficiency.

[0102] In one example, triangulating the first landmark with respect to each node and landmark in the first landmark set includes using the cosine law (also known as the cosine theorem, cosine formula, or Al-Qasi's theorem), e.g., as described with reference to FIG. 9.

[0103] (Signature) In one example, the method comprises: representing the first set of landmarks as a first signature; representing a set of landmark references as respective reference signatures; and correlating the first signature with a reference signature to approximate a first position of the radar sensor.

[0104] In this manner, a first location of the radar sensor is approximated with a relatively low degree of accuracy by correlating the first signature with the reference signature, and then the first location of the radar sensor is identified with a relatively high degree of accuracy using each landmark reference set represented by the correlated reference signature. In other words, this provides a first-path approximation of the first location for subsequent refinement, generally identifying the location of the first sensor on the map.

[0105] More specifically, the radar signature feature provides a method for finding loop closures using radar sensor data (i.e., radar scans) to identify and locate map nodes, for example, during initialization or when localization is lost. While external sources can be used for radar signatures, the advantage of radar signatures is that only radar data is required for radar signatures, eliminating dependency on other sensor modalities.

[0106] It should be understood that a radar signature (i.e., signature) is a very compact representation of a radar landmark point cloud. The plane around the radar is divided into a series of regions in a polar representation derived from the radar itself. The points are assigned to corresponding n two-dimensional histograms with bins for combinations of distance (i.e., range) and angle (i.e., azimuth). The histograms are normalized to sum to one.

[0107] Two signatures are compared using the complement of the histogram intersection metric as the similarity measure, resulting in a value of 0 representing the best possible match. The best candidate node to initialize in the map can be selected as the node with the lowest score for the current radar sensor data (considering both nodes converted to signature representations). Because all signatures for each map node can be computed at startup, the algorithm is fast enough to find the best matching node in a fraction of a second.

[0108] The threshold signature_similarity_threshold is the maximum similarity measure allowed for the best candidate node; otherwise, the algorithm reports that no suitable map nodes have been found. The remaining parameters control the structure of the set of regions used to derive the histogram. Such a structure is a disk with the radar at its center and a radius equal to signature_max_range. This disk is divided into regions radiating from the center, defined by signature_num_angle_bins and signature_num_range_bins. The resulting histogram therefore has a total number of bins equal to the product of signature_num_angle_bins and signature_num_range_bins.

[0109] In one example, accessing the landmark reference set includes selectively accessing the landmark reference set represented by the reference signature.

[0110] (Land or water vehicle) In one example, a land or water vehicle includes a radar sensor.

[0111] In one example, the land or water vehicle is an unmanned, semi-autonomous, and / or autonomous land or water vehicle. Generally, an unmanned vehicle (also known as a pilotless vehicle) is a vehicle without a human on board. An unmanned vehicle can be either a remote-controlled vehicle, a semi-autonomous vehicle, or an autonomous vehicle that can sense its environment and navigate autonomously. Unmanned vehicles include unmanned ground vehicles (UGVs), such as autonomous cars, and unmanned surface vehicles (USVs), which are intended to operate on water's surfaces.

[0112] Typically, land vehicles (also known as vehicles) include military vehicles, commercial vehicles, and / or personal land vehicles. Military vehicles include combat and transport vehicles, such as military ambulances, amphibious military vehicles, armored fighting vehicles, electronic warfare vehicles, military engineer vehicles, improvised combat vehicles, joint light tactical vehicles, military utility light vehicles, off-road military vehicles, reconnaissance vehicles, recovery vehicles, self-propelled weapons, self-propelled anti-aircraft weapons, self-propelled artillery, tanks, tracked military vehicles, half-tracks, military trucks, and wheeled military vehicles. Commercial vehicles include trucks (such as box trucks, articulated trucks, and pickup trucks), buses and coaches, heavy equipment (such as those used in mining, construction, and agriculture), and passenger cars such as taxis. Personal land vehicles include passenger cars and trucks. Other land vehicles are also known.

[0113] Watercraft typically include military, commercial, and / or recreational watercraft, including vehicles that travel on the water's surface. Military watercraft classes include aircraft carriers, cruisers, destroyers, frigates, corvettes, large patrol vessels, small surface combatants such as missile boats, torpedo boats, and patrol boats, including rigid inflatable boats (RIBs), mine warfare vessels such as mine countermeasures vessels, minesweepers, minesweepers, and minelayers, amphibious warfare vessels such as amphibious assault ships, dock landing ships, landing craft and landing ships, and air-cushion landing craft. Commercial watercraft classes include passenger ships such as container ships, bulk carriers, tankers, ferries, and cruise ships, and specialized vessels such as coastal trade vessels, anchor carriers, supply ships, tugboats, salvage ships, research vessels, trawlers, and whalers. The class of recreational (also known as recreational) watercraft includes boats and yachts, such as pontoons, bowriders, cabin cruisers, houseboats, trawlers, motor yachts, catamarans, etc. Other watercraft are also known.

[0114] (Dimensional Search) A computer-implemented method for locating a radar sensor according to a second aspect includes: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, Matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptors to first projected descriptors.

[0115] The method according to the second aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.

[0116] (Descriptor resize) A computer-implemented method for locating a radar sensor according to a third aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is further comprising projecting the first descriptor into a first projected descriptor, the first descriptor having a dimension N and the first projected descriptor having a dimension M, where M≠N; Matching the first descriptor set with a corresponding first descriptor reference set includes matching the first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.

[0117] The method according to the third aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.

[0118] (Memory of calculated values) A computer-implemented method for locating a radar sensor according to a fourth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is The method further includes calculating one or more values ​​in response to the request, storing the calculated one or more values, and returning the stored one or more values ​​or one or more values ​​derived therefrom in response to a subsequent request.

[0119] The method according to the fourth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.

[0120] (process) A computer-implemented method for locating a radar sensor according to a fifth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is Further steps include calculating two or more values ​​simultaneously and / or calculating in a correlated manner using two or more values.

[0121] The method according to the fifth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.

[0122] (Signature) A computer-implemented method for locating a radar sensor according to a sixth aspect includes the steps of: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first landmark sets, the first descriptors defining the first landmark by a respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is representing the first set of landmarks as a first signature; representing a set of landmark references as respective reference signatures; correlating the first signature with a reference signature to approximate a first position of the radar sensor; Includes.

[0123] The method according to the sixth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.

[0124] (Control of land or water vehicles) A computer-implemented method for controlling a land or water vehicle including a radar sensor according to a seventh aspect comprises: locating a radar sensor according to the first, second, third, fourth, fifth and / or sixth aspects; controlling a land or water vehicle using the first position; Includes.

[0125] In this way, the land or water vehicle is controlled using the first position, for example for navigation purposes. The land or water vehicle may be as described in relation to the first aspect.

[0126] In one example, controlling the land or water vehicle using the first position includes controlling the land or water vehicle in response to the first position.

[0127] In one example, controlling the land or water vehicle using the first position includes steering the land or water vehicle.

[0128] In one example, controlling the land or water vehicle using the first position includes semi-autonomously or autonomously controlling the land or water vehicle using the first position.

[0129] (Computer, computer program, non-transitory computer-readable storage medium) A computer according to an eighth aspect includes a processor and memory configured to perform the methods according to the first, second, third, fourth, fifth, sixth and / or seventh aspects.

[0130] A computer program according to a ninth aspect comprises instructions which, when executed by a computer including a processor and a memory, cause the computer to perform the methods according to the first, second, third, fourth, fifth, sixth and / or seventh aspects.

[0131] A non-transitory computer-readable storage medium according to a tenth aspect includes instructions that, when executed by a computer including a processor and a memory, cause the computer to perform the methods according to the first aspect, the second aspect, and / or the third aspect.

[0132] (Land or water vehicle) A land or water vehicle according to an eleventh aspect includes a radar sensor and a computer according to the eighth aspect.

[0133] The land or water vehicle may be as described in relation to the first aspect.

[0134] (definition) Throughout this specification, the terms "comprising" or "comprises" mean including the specified component, but not excluding the presence of other components. The terms "consisting essentially of" or "consists essentially of" mean including the specified component, but excluding other components except for materials present as impurities and unavoidable materials present as a result of the process used to provide the component, and components added for purposes other than achieving the technical effect of the present invention, such as colorants.

[0135] The terms "consisting of" or "consists of" mean the inclusion of certain components but the exclusion of other components.

[0136] Wherever appropriate and depending on the context, use of the words "comprises" or "comprising" may be interpreted to include the meaning of "consisting essentially of" or "consisting essentially of" or "consisting of" or "consisting of."

[0137] The optional features described herein can be used individually or in combination with one another as appropriate, particularly in the combinations set forth in the appended claims. Optional features of each aspect or exemplary embodiment of the present invention described herein are also applicable to all other aspects or exemplary embodiments of the present invention as appropriate. In other words, those skilled in the art who read this specification will recognize that the optional features of each aspect or exemplary embodiment of the present invention are interchangeable and combinable between different aspects and exemplary embodiments. [Brief explanation of the drawings]

[0138] For a better understanding of the present invention and to show how exemplary embodiments thereof may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0139] [Figure 1] 1A and 1B schematically illustrate a plan view of an exemplary embodiment of a radar sensor; [Figure 2] 1 illustrates a schematic diagram of a method according to an exemplary embodiment; [Figure 3] 3 shows the method of FIG. 2 in more detail. [Figure 4] 3 is an example of an algorithm for landmark extraction in the method of FIG. 2. [Figure 5] 3 shows the method of FIG. 2 in more detail. [Figure 6] 3 shows the method of FIG. 2 in more detail. [Figure 7] 3 shows the method of FIG. 2 in more detail. [Figure 8] 3 is an example of a data association algorithm for the method of FIG. 2. [Figure 9] 3 shows the method of FIG. 2 in more detail. [Figure 10A] 1 shows a schematic diagram of how a signature is constructed from a radar scan. [Figure 10B] 1 shows a schematic diagram of how a signature is constructed from a radar scan. [Figure 10C] 1 shows a schematic diagram of how a signature is constructed from a radar scan. [Figure 11] 1 shows code used in certain embodiments of the present disclosure. [Figure 12] 1 shows code used in certain embodiments of the present disclosure. [Figure 13] 1 shows code used in certain embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0140] Figure 1 shows a schematic diagram of a radar sensor, specifically an FMCW scanning radar, in an exemplary embodiment. In this example, a radar sensor (green circle) at the center of a vehicle (black box) sequentially collects power-range spectra (radial green dotted lines) at each azimuth angle. The variables a and r represent azimuth angle and range, respectively. The sample signal at a particular azimuth angle is plotted as power (dB) as a function of range bins.

[0141] 2 illustrates a computer-implemented method for locating a radar sensor according to an exemplary embodiment.

[0142] In general, the method involves two main steps: landmark extraction and pose estimation.

[0143] The method includes obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum (ie, a 1D signal).

[0144] The method includes extracting a first landmark set from a first radar scan, the first landmark set including a first landmark defined by range and azimuth angle. This step is also known as landmark extraction or feature extraction. CFAR is a common filtering algorithm, but is not specific enough. In contrast, the method described herein can detect more reliable and distinctive landmarks in radar scene data. For example, a complete radar scan is received (i.e., acquired) from a radar sensor, and the method performs landmark extraction to accurately detect objects in the environment up to the maximum range of the radar sensor, as described, for example, with reference to FIGS. 3 and 4. The radar scan is a set of power-range spectra (i.e., 1D signals), one power-range spectrum for each azimuth angle, as described, for example, with reference to FIG. 1. In this example, the power-range spectrum is represented as an array of values ​​for each azimuth angle across the entire circumference of the radar sensor. In this example, the output from landmark extraction is a point cloud, which is a collection of points (corresponding to landmarks), each identified by range and angle (i.e., azimuth angle) from the centerline.

[0145] The pose estimation step then uses the new landmark point cloud to determine a pose relative to the previous landmark point cloud and a position relative to a map database containing landmark points captured along the route during the mapping phase.

[0146] The pose estimation step involves two main substeps: first, computing scene point descriptors; then, using these scene point descriptors and the landmark point cloud to align the two point clouds and estimate the positional difference between the two point clouds. Each scene point descriptor is a collection of unique "descriptors," one "descriptor" for each point in the point cloud. Thus, a scene point descriptor is represented as a matrix of values, with each column representing the descriptor for each point. A descriptor must be computed, represented as a vector of values ​​that uniquely describes the point, thus allowing it to be identified and matched in other scans. The descriptor identifies landmark points by the radial statistics of neighboring points in both range and angular slices.

[0147] The method includes calculating, for a first landmark set, each first descriptor set including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first landmark set, e.g., as described with reference to FIG. 5 .

[0148] The method includes the steps of accessing one or more landmark reference sets for each environment and calculating a descriptor reference set for each of the landmark reference sets, for example as described with reference to Figures 6-8.

[0149] The method includes matching a first set of descriptors with a corresponding first reference set of descriptors, as described, for example, with reference to FIGS. 6-8. This step, known as data association, matches landmarks across a radar scene. While other approaches use popular feature descriptors for vision systems, these approaches do not work as well for radar data. The inventors use novel feature descriptors that are more suited to radar landmarks, improving data association between live scans and other previously seen scans, such as previous scans in the case of radar driving range measurements and map scans in the case of localization.

[0150] The method includes identifying a first location of the radar sensor using a first result of the matching, for example, as described with reference to Figures 6-8. This step, also known as localization, determines the spatial distance between two sets of landmarks.

[0151] Figure 3 illustrates the method of Figure 2 in more detail. In particular, Figure 3 outlines the steps for extracting landmarks from a power range spectrum. The input (raw signal) is processed from the upper left to produce the output at the bottom right, with landmarks indicated by red stars along the way. Box 6 in this example highlights the approach's ability to remove multipath reflections and noisy detections. Boxes 3 and 5 demonstrate the importance of incorporating high-frequency signals, as using smooth signals alone in Boxes 2 and 4 would ignore the high range resolution offered by FMCW radar.

[0152] The first objective is to accurately detect objects in the radar sensor's environment while minimizing false positives. Specifically, the method must find all landmarks sensed by the radar sensor while minimizing the number of redundant returns per landmark and avoiding the detection of absent landmarks due to noise, multipath reflections, harmonics, sidelobes, etc. The method takes as input a power-range spectrum (i.e., a 1D signal) and returns a set of landmarks, each identified by range and azimuth angle. The core concept is to estimate the noise statistics of the signal and then scale the power value at each range by the probability that it corresponds to an actual detection. Consecutive peaks in this reshaped signal are treated as objects, and for each peak, only the range at the center of the peak is added to the landmark set.

[0153] Vector s(t)∈R N×1 Let be the power range spectrum at time t, and thus, the element s iis the power return in the i-th range bin and a(t) is the associated azimuth angle. Let r(i) = β(i-0:5) give the bin range i ∈ {1, 2, …, N}, where β is the range resolution. If the environment is perfectly recorded, then y(t) ∈ R N×1 Assume that is an ideal signal. Then, s(t) = y(t) + v(y(t)), where v represents unwanted effects such as noise. Therefore, to infer y(t) from s(t) for accurate landmark separation, an approximation of v(y(t)) is required, thus

number

number

number

[0154] FIG. 4 shows Algorithm 1, which is an example of an algorithm for landmark extraction in the method of FIG.

[0155] First, we obtain an unbiased signal q (Box 2) that preserves high-frequency information by subtracting the noise floor of v(s) from s (line 1). Next, we smooth the result to obtain the underlying low-frequency signal p (Box 3), which better exposes obvious landmark peaks (line 2). At this point, q is not discarded because smoothing weakens the presence of radar landmarks, since radar landmarks often appear as high-frequency peaks, and because smoothing blurs the peaks of nearby landmarks, making them difficult to distinguish. Therefore, we integrate information from both q and p. To estimate noise characteristics, we consider values ​​of q below zero to be the mean μ q = 0 and standard deviation σ q (Line 4) f(μ,σ 2 ) with normal distribution N(μ,σ 2 ) at x. Then, for every range bin, the power value is scaled by the probability that it does not correspond to noise in two steps. First, the smoothed signal p i Each value of

number

number

number

number

number

[0156] In this method,

number

number

number

number

number

[0157] FIG. 5 illustrates the method of FIG. 2 in more detail.

[0158] In this example, the pose estimation step uses the output landmark point cloud to determine a position (i.e., a first position) of the radar sensor relative to a map database that includes a reference landmark point cloud (i.e., reference landmarks) captured or acquired along the route during, for example, the mapping phase of the radar sensor. Also in this example, the pose estimation step uses the output landmark point cloud to determine a position relative to a previous landmark point cloud (i.e., for driving range measurements).

[0159] The pose estimation step involves two main substeps: first, computing scene point descriptors; and then aligning the two point clouds using the scene point descriptors and the landmark point clouds to estimate the positional differences between the two point clouds. Each scene point descriptor is a collection of unique "descriptors," one "descriptor" for each point in the point cloud. Thus, in this example, the scene point descriptors are represented as a matrix of values, with each column representing a descriptor for each point. The descriptors are computed and represented as vectors of values ​​that uniquely describe the point, thus allowing it to be identified and matched in other scans. The descriptors identify landmark points by radial statistics of neighboring points in both range and angular slices.

[0160] FIG. 6 illustrates the method of FIG. 2 in more detail.

[0161] Once the scene point descriptors are calculated, a data association step can be performed using the scene point descriptors to match each landmark point from one radar scan to a landmark point from another radar scan. Finding the best matching descriptor from one scan to another can be computationally expensive, so the inventors have developed a number of refinements to improve efficiency, as described herein. Once a set of correspondences from one point cloud to another has been determined, the goal is to select the best match to ensure that the alignment between the point clouds is robust to outliers and false positives. Given good overlap and stable association between the scans, the sensor motion that must have occurred from one scan to another can be calculated. In this example, the motion estimate is output by the computer.

[0162] Figure 7 shows the method of Figure 2 in more detail. The core concept behind the data association algorithm aims to find similar shapes within two landmark point clouds (red) extracted from radar scans. Unary candidate matches (green dotted lines) are generated by comparing the angular characteristics of the points. Selected matches (A,A') and (B,B'2) are determined by the pairwise distance difference (|dAB -d' AB2 |<|d AB -d' AB1 |) is minimized. In this way, shape matching is approximated by sequentially comparing angles and side lengths.

[0163] More specifically, the scan matching algorithm achieves robust point correspondence using high-level information from radar scans. Intuitively, it aims to find the largest subset of two point clouds that share a similar shape. Unlike ICP, this method works without prior information about the scan orientation or relative displacement. Therefore, the algorithm is not constrained by obtaining a good initial estimate of the relative pose and can compare point clouds captured at any time without maps. The only requirement is that the observed regions are coplanar and contain sufficient overlap. One important attribute of the approach described herein is that it performs data association using not only individual landmark (i.e., unary) descriptors but also the relationships between landmarks. For example, imagine three landmarks forming the vertices of a scalene triangle. Then, the set of distances from each point to its neighbors is unique to that point, regardless of the arrangement of the entire point cloud. This allows landmarks to be directly matched to corresponding landmarks in any other point cloud obtained by applying a rigid transformation to the original triangle. The larger the number of points, the less likely it is that the set of pairwise distances from an individual point to its neighbors will be the same as another point. Furthermore, the exact location and orientation of the points do not affect the pairwise relationships within the point cloud, so large differences between the placement and orientation of the points are not important. We exploit these observations to obtain reliable matching of large landmark sets. For real-world data, a major challenge is that landmark location and detection are noisy, meaning that points do not always survive rigid transformations and that the locations of points that survive rigid transformations are affected by noise. A simple example illustrating the concept behind the data association algorithm is shown in Figure 7.

[0164] Figure 8 shows Algorithm 2, an example of a data association algorithm for the method of Figure 2. As input, two point clouds L O and L I Accept the first point group L O is the original landmark set in Cartesian coordinates. Since landmarks are detected in polar space, the resulting point cloud will be dense at low ranges and sparse at high ranges. For this situation, we define a second point cloud L I compensates by generating a binary Cartesian grid of interpolated resolution from a binary polar grid of landmarks. The latter point cloud is less accurate and is only used to avoid range density bias when processing the layout of the environment, while data association is performed on the former point cloud (i.e., the algorithm does not use the landmarks).

number

number

number

number

[0165] This maximization is computationally intractable due to the discretization of m. Therefore, we relax the above constraint and maximize the continuous value of u as follows: * Ask for. u * =uCu

[0166] Under these conditions, u * is the normalized eigenvector of the largest eigenvalue of the semidefinite positive matrix C. Then, the optimal solution m * is calculated using the greedy approach shown in lines 3-11. *In essence, this greedy method iteratively adds satisfactory matches to a set M. At each iteration, the remaining valid matches are evaluated (line 7), the match that returns the maximum reward is accepted (line 9), and matches that conflict with this match are removed from further consideration (lines 10 and 11). The algorithm terminates if the reward of the finally selected match is less than the reward that would be obtained if all matches were equally evaluated (i.e., it is a weak match), or if the reward is greater than the reward that would be obtained if more than α percent of the landmarks in either set match (lines 6 and 8). Note that this match is the only free parameter of the method, and there is no need to remove outliers.

[0167] FIG. 9 illustrates the method of FIG. 2 in more detail.

[0168] In this example, computing a first descriptor set including the first descriptor for the first landmark set includes triangulating the first landmark with respect to each node and landmark in the first landmark set. In this example, triangulating the first landmark with respect to each node and landmark in the first landmark set includes using the cosine law.

[0169] The route (also known as reference) landmarks or points are fixed to landmark or point i. Therefore, for each route landmark, the angles and distances with respect to all landmarks or points can be efficiently calculated.

[0170] In summary, parallel computation using SIMD and cosine law are calculated simultaneously. As an example, point cloud data is obtained in the form of a descriptor. The point cloud data is divided into multiple chunks. The number of points in a chunk is determined based on the processor's capabilities. A single instruction may be executed to apply a single instruction to each point in the group simultaneously. The single instruction may be to calculate the distance between points in the chunk. Once the distance is known, the angle of the points in the chunk is calculated using cosine law, also using a single instruction such as SIMD.

[0171] Once the first chunk has been processed in this manner, a second chunk is selected, and so on. In this manner, parallel processing of each chunk of the multiple chunks occurs. Processing of the multiple chunks occurs serially. In this manner, all chunks may be processed. The order in which chunks are selected for processing may be selected randomly.

[0172] (Signature) In this example, the radar signature header defines three data types: Signature, ExperienceSignature, and MapSignature. Signature is a two-dimensional vector of doubles that represents the radar signature itself. ExperienceSignature is a std::map of Signatures indexed by node_id, and MapSignature is a std::map of ExperienceSignatures indexed by experience_name.

[0173] The radon::loopclosure::RadarSignatureBuilder class has a three-argument constructor that defines and takes parameters corresponding to the signature challenge, azimuth bin, and range bin parameters described above. RadarSignatureBuilder generates a signature corresponding to these parameters. RadarSignatureBuilder objects have a ComputeSignature method that generates a signature given a point cloud of radar landmarks extracted from a raw radar scan.

[0174] There is also a ComputeSignaturesForMap method that returns the MapSignatures data type for the entire map, given a map_client and a string representing the attribute name used to store the radar data.

[0175] A function CompareSignatures is provided that computes the similarity score for a given pair of Signatures. Two further functions make use of this comparison function. FindCandidateLoopClosureNodes takes a Signature, ExperienceSignatures and a threshold and returns a std::vector of all node ids in the experience that have a similarity below the threshold. This function is intended for use with signatures during map construction. For location, the function FindBestCandidateMapNode is provided. Given a Signature, MapSignatures and a threshold, it returns the single best matching node_id in the map, as long as it is below the similarity threshold.

[0176] 10A-10C, as an autonomous vehicle traverses a route on a map, it captures radar scans. Each scan may be referred to as a node 100, with the location in the map where the scan was captured. In this manner, a "node" may refer to the point where the radar scan was captured.

[0177] A scan at a node 100 captures features along multiple azimuth angles, as shown in Figure 1. Multiple range bins are provided for each azimuth angle. These range bins can be visualized as multiple concentric rings, each with an equal number of segments 102 separated according to the distribution of azimuth angles 104 (only four azimuth angles are shown in the figure to avoid obscuring the drawing). Each segment 102 is assigned a number corresponding to the number of features detected in that segment by the radar scan.

[0178] A vector having multiple values ​​may be generated. The number of elements in the vector corresponds to the number of segments. The value of each element is equal to the number of the corresponding segment. This vector is the signature 106. More specifically, this vector is described as a node signature.

[0179] The autonomous vehicle may capture multiple scans of different nodes along the route 108, resulting in a node signature for each node, which is then combined to form the route signature.

[0180] If the same or multiple autonomous vehicles traverse multiple routes, multiple route signatures may be created, and a route signature may be generated that includes multiple corresponding route signatures.

[0181] These previously generated signatures correspond to reference signatures.

[0182] During operation, a signature is generated for the current position. The signature of the current position may be referred to as the first signature. The first signature is compared to a reference signature to determine the closest match. The best matching reference signature correlates to the first signature. Correlated means that the first signature is equivalent to the best matching reference signature. In this way, the best matching reference signature can be used to approximate the position and orientation of the first signature.

[0183] Descriptor matching as described above can then be used to more accurately determine position and attitude. It is computationally much more efficient to obtain an approximation of the radar sensor position and attitude before determining a more precise position and attitude, as the estimate can indicate which points in the radar point cloud are likely to be the starting points for the calculation.

[0184] (References) SHCen and P. Newman, "Precise Ego-Motion Estimation with Millimeter-Wave Radar Under Diverse and Challenging Conditions," 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 6045-6052, doi:10.1109 / ICRA.2018.8460687.

[0185] The subject matter of the references is incorporated herein by reference in its entirety.

[0186] (Note) While preferred embodiments have been shown and described, it should be understood by those skilled in the art that various changes and modifications can be made therein without departing from the scope of the invention as defined in the appended claims and as set forth above.

[0187] At least some of the exemplary embodiments described herein may be configured, partially or entirely, using dedicated hardware. As used herein, terms such as "component," "module," or "unit" may include, but are not limited to, hardware devices, such as circuits, field programmable gate arrays (FPGAs), or application-specific integrated circuits (ASICs), in the form of discrete or integrated components, that perform particular tasks or provide related functionality. In some embodiments, the described elements may be configured to reside on tangible, persistent, addressable storage media and to execute on one or more processors. In some embodiments, these functional elements may include, by way of example, components such as software components, object-oriented software components, class components, task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. While the exemplary embodiments are described with reference to components, modules, and units discussed herein, such functional elements may be combined into fewer elements or separated into additional elements. Various combinations of optional features are described herein, and it will be understood that the described features may be combined in any suitable combination. In particular, features of any one exemplary embodiment may be combined as appropriate with features of any other embodiment, as long as they are not mutually inconsistent. Throughout this specification, the terms "comprising" or "comprises" mean the inclusion of certain components but do not exclude the presence of other components.

[0188] Attention is directed to all articles and documents related to this application, filed contemporaneously or prior to this application, and open to public inspection herewith, and the contents of all such articles and documents are incorporated herein by reference.

[0189] All features disclosed in this specification (including the accompanying claims, abstract and drawings), and / or all steps of any method or process so disclosed, may be combined in any combination, provided that at least some of such features and / or steps are not mutually inconsistent.

[0190] Each feature disclosed in this specification (including the accompanying claims, abstract, and drawings), unless otherwise stated, may be replaced by alternative features serving the same, equivalent, or similar purpose. Thus, unless otherwise stated, each disclosed feature is only an example of a generic series of equivalent or similar features.

[0191] The invention is not limited to the details of the foregoing embodiments, but extends to any novel feature, or any novel combination of such features, disclosed in this specification (including the accompanying claims, abstract and drawings), or to any novel or novel combination of any method or process steps so disclosed.

Claims

1. 1. A computer-implemented method for locating a radar sensor, comprising: acquiring a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by a respective relative range and azimuth angle with respect to at least one landmark included in the first set of landmarks; accessing at least one landmark reference set for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first descriptor set with a corresponding first descriptor reference set; identifying a first location of the radar sensor using a first result of the matching; Including, The method comprises: further comprising projecting the first descriptor into a first projected descriptor, the first descriptor having dimension N and the first projected descriptor having dimension M, where M≠N; the step of matching the first descriptor set with the corresponding first descriptor reference set includes the step of matching a first projection descriptor set including the first projection descriptor with the corresponding first descriptor reference set of the plurality of descriptor sets; the dimension N is a matrix dimension with rows corresponding to the number of points in the scan and columns corresponding to the size of the point descriptor; the point descriptor comprises a vector of values ​​that uniquely describes the point; The step of projecting the first descriptor into a first projected descriptor includes: obtaining a vector of mean values ​​of each element of the first descriptor; obtaining a shifted first descriptor by subtracting the average value from each element of the first descriptor; calculating a covariance matrix of the shifted first descriptor; generating eigenvectors of the covariance matrix; ranking the eigenvalues ​​from the eigenvector in descending order; selecting a subset of the largest eigenvalues ​​as the first projection descriptor; Including, The method, wherein matching the first descriptor set with the corresponding first descriptor reference set includes identifying closest descriptors in the corresponding first descriptor reference set and finding a single closest descriptor from the closest descriptors.

2. The method of claim 1 , wherein the subset includes M eigenvalues.

3. The method of claim 2 , wherein M is 1 and the selected eigenvalue is the largest eigenvalue.

4. obtaining a second radar scan of the first environment of the radar sensor; extracting a second set of landmarks from the second radar scan, the second set including the first landmark; calculating a second set of descriptors for each of the second set of landmarks, the second set of descriptors including the first descriptor; matching said second descriptor set with a corresponding second descriptor reference set; determining a second location of the radar sensor using a second result of the matching; calculating a motion of the radar sensor using the second position and the first position; The method of claim 1 , comprising:

5. The method of claim 1 , wherein matching the first descriptor set with the corresponding first descriptor reference set comprises projecting the first descriptors to first projected descriptors.

6. The method comprises: summing first absolute differences between the first set of descriptors and a first reference set of descriptors and setting a threshold absolute difference as the summed first absolute differences; summing second absolute differences between the first set of descriptors and a second reference set of descriptors, the second absolute differences in the sum being at most the threshold absolute difference; if the second absolute difference being summed exceeds the threshold absolute difference, stopping the summation of the second absolute differences and starting a summation of a third absolute difference between the first set of descriptors and a third reference set of descriptors; if the summed second absolute difference does not exceed the threshold absolute difference, resetting the threshold absolute difference to be the summed second absolute difference; The method of claim 1 , comprising:

7. The method of claim 1 , wherein M<N or M>N.

8. The method of claim 1 , wherein projecting the first descriptor onto the first projected descriptor comprises interpolating its elements.

9. The method of claim 1 , wherein projecting the first descriptor onto the first projected descriptor comprises averaging or dropping elements thereof.

10. storing the descriptor reference set; 2. The method of claim 1, wherein matching the first descriptor set with the corresponding first descriptor reference set comprises matching the first descriptor set with the corresponding first descriptor set of stored descriptor reference sets.

11. 2. The method of claim 1, wherein calculating the first set of descriptors including the first descriptor for the first set of landmarks uses a set of lookup tables including a first lookup table.

12. The method of claim 11 , comprising generating the first lookup table at compile time and using the first lookup table at run time.

13. 12. The method of claim 11, comprising generating the first lookup table at run time during a first calculation of a given value of the first lookup table.

14. 2. The method of claim 1 , wherein computing the first descriptor set including the first descriptor for the first landmark set comprises processing the first landmark set in parallel.

15. 2. The method of claim 1 , wherein computing the first descriptor set including the first descriptor for the first set of landmarks comprises triangulating the first landmark with respect to each node and landmark in the first set of landmarks.

16. representing the first set of landmarks as a first signature; representing the landmark reference set as respective reference signatures; correlating the first signature with a reference signature to approximate the first position of the radar sensor; The method of claim 1 , comprising:

Images