Methods and Apparatus
The method enhances radar sensor localization for autonomous vehicles by extracting landmarks and using efficient data association techniques, addressing accuracy and precision issues in conventional systems for real-time navigation and control.
Patent Information
- Application Number
- JP2024522568
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-10-19
- Filing Date
- 2022-10-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-10-18
AI Technical Summary
Conventional radar localization and range measurement systems lack sufficient accuracy and precision for navigation and pose estimation in autonomous vehicles, especially in challenging weather conditions and resource-constrained environments, making them unsuitable for real-time control.
A computer-implemented method for radar sensor localization that includes acquiring radar scans, extracting landmarks, computing descriptors, matching with reference sets, and determining the sensor's location, utilizing techniques like dimensionality reduction and efficient data association to enhance accuracy and speed on low-power platforms.
Provides accurate and precise radar-only localization for navigation and range measurement, enabling real-time control of autonomous vehicles using low-specification computing platforms, even in adverse conditions.
Smart Images

Figure 2024543787000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to radar sensor location. [Background technology]
[0002] To navigate confidently in an environment, an autonomous vehicle needs to achieve robust localization and navigation despite changing conditions (e.g., lighting and weather) and moving objects (e.g., pedestrians and other vehicles). Currently, most platforms apply lidar, vision, GPS, internal sensors or a combination of these systems to acquire information about the surroundings and perform motion estimation. Lidar is very fast and high resolution, but is sensitive to weather conditions, especially rain and fog, cannot penetrate the first surface it encounters, and has a much lower practical range, e.g., 50-100 m. Vision systems are versatile and cheap, but are easily compromised by scene changes, such as in poor lighting or when bad weather conditions, e.g., snow, rain, etc., suddenly appear. Both optical sensors provide reliable results only for short-range measurements. A typical GPS provides accuracy in the meter range, but relies on external infrastructure and frequently experiences reception failure near obstacles. Furthermore, proprioceptive sensors such as wheel encoders and IMUs are subject to significant systematic errors (e.g., drift), among other detrimental effects.
[0003] In contrast, radar is a long-range (e.g., up to 600 m) vehicle-mounted system that is independent of lighting conditions, works well in a variety of weather conditions, and is more affordable and efficient than lidar. Because radar has a relatively long wavelength and can see through certain materials, it can return multiple readings from the same transmission, producing a grid representation of the environment. As a result, radar sensors detect stable, long-range features in the environment.
[0004] However, conventional radar localization and driving range measurement methods do not provide sufficiently accurate and / or precise localization, e.g., attitude estimation, for route planning and the like. Typically, navigation is at a higher level, such as "go to location X," while route planning refers to a lower level, defining the exact path a vehicle will traverse. More generally, conventional approaches for radar localization and driving range measurement are not accurate or robust enough for precise attitude estimation for the purposes of precise driving range measurement or large-scale localization of vehicles such as autonomous vehicles, and more generally, vehicles such as landcraft or watercraft.
[0005] Many applications, such as mining or off-road environments, require radar-only localization and range measurement systems. These applications require radar-only localization and range measurement systems to be deployed and operated on low-power hardware with limited computational resources. An example of such a radar-only localization and range measurement system includes the Navtech CTS350-X sensor (available from Navtech Radar Limited, UK), which has available computations of a 1.6GHz quad-core ARM A53 processor with a power consumption approaching 1W. Conventional radar-only localization and range measurement systems are too slow on this type of low specification computation platform to be practical for real-time control of vehicles, such as autonomous vehicles, more generally land or water vehicles.
[0006] Therefore, there is a need to improve radar sensor localization. Summary of the Invention [Problem to be solved by the invention]
[0007] One object of the present invention is, inter alia, to provide a method for localizing a radar sensor that at least partially obviates or mitigates at least some of the drawbacks of the prior art mentioned herein or elsewhere. For example, an object of an embodiment of the present invention is to provide a method for localizing a radar sensor that provides sufficiently accurate and / or precise localization, e.g., attitude estimation, for navigation, etc. For example, an object of an embodiment of the present invention is to provide a reliable and accurate radar-only system for precise driving range measurements and attitude estimation of localization. For example, an object of an embodiment of the present invention is to provide a method for localizing a radar sensor that is implemented by a relatively low specification computing platform during real-time control of a vehicle, such as an autonomous vehicle, or more generally a land or water vehicle. For example, an object of an embodiment of the present invention is to provide a fast and efficient implementation on a low power embedded platform. [Means for solving the problem]
[0008] A computer-implemented method for locating a radar sensor according to a first aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Includes.
[0009] The term "descriptor" may be understood to mean a vector whose elements define the location of a point in the radar point cloud of a detected feature relative to an origin, which may be the radar sensor that captured the radar scan that detected the descriptor.
[0010] The first landmark from the first set of landmarks may correspond to at least one landmark from the first set of landmarks. The at least one landmark may be an arbitrarily selected landmark from the first set of landmarks.
[0011] A computer-implemented method for locating a radar sensor according to a second aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, Matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptors to a first projected descriptor.
[0012] The step of projecting the first descriptor into the first projected descriptor may comprise reducing a dimension of the first descriptor to a size of the first projected descriptor. Despite the dimensionality reduction, the first descriptor and the first projected descriptor may be one-dimensional vectors.
[0013] A computer-implemented method for locating a radar sensor according to a third aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, The method comprises: The method further comprises projecting the first descriptor into a first projected descriptor, the first descriptor having a dimension N and the first projected descriptor having a dimension M, where M ≠ N; Matching the first descriptor set with a corresponding first descriptor reference set includes matching a first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.
[0014] A computer-implemented method for locating a radar sensor according to a fourth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, The method comprises: The method further includes the steps of calculating one or more values in response to a request, storing the calculated one or more values, and returning the stored one or more values or one or more values derived therefrom in response to a subsequent request.
[0015] A computer-implemented method for locating a radar sensor according to a fifth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, The method comprises: Further steps include calculating two or more values simultaneously and / or performing correlated calculations using two or more values.
[0016] A computer-implemented method for locating a radar sensor according to a sixth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, The method comprises: representing the first set of landmarks as a first signature; expressing the landmark reference set as respective reference signatures; correlating the first signature with a reference signature to approximate the first position of the radar sensor; Further includes:
[0017] The term "signature" may be understood to mean a vector containing a number of values, each corresponding to a count of the number of features within a circular segment of a radar scan.
[0018] According to a seventh aspect, there is provided a computer-implemented method for controlling a land or water vehicle including a radar sensor, the method comprising the steps of: - locating a radar sensor according to the first, second, third, fourth, fifth and / or sixth aspects; controlling the land or water vehicle using the first position; Includes.
[0019] A computer according to an eighth aspect includes a processor and memory configured to execute the methods according to the first, second, third, fourth, fifth, sixth and / or seventh aspects.
[0020] A computer program according to a ninth aspect comprises instructions which, when executed by a computer comprising a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect, the third aspect, the fourth aspect, the fifth aspect, the sixth aspect and / or the seventh aspect.
[0021] A non-transitory computer-readable storage medium according to a tenth aspect includes instructions that, when executed by a computer including a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect and / or the third aspect.
[0022] A land or water vehicle according to an eleventh aspect includes a radar sensor and a computer according to the eighth aspect.
[0023] Detailed Description of the Invention According to the present invention there is provided a method for locating a radar sensor as set out in the accompanying claims. Also provided is a method for controlling a land or water vehicle. Further features of the invention will become apparent from the dependent claims and the following description.
[0024] (Radar sensor location) A computer-implemented method for locating a radar sensor according to a first aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Includes.
[0025] In this manner, the radar sensor is located sufficiently accurately and / or precisely for, e.g., navigation of a land or water vehicle, since the radar sensor is located by matching the first set of descriptors with a corresponding first reference set of descriptors provided, e.g., by mapping the environment, thereby providing a reliable and accurate radar-only system for precise driving range measurements and positioning attitude estimation. Furthermore, since the radar sensor is located by matching the first set of descriptors with a corresponding first reference set of descriptors, the method can be implemented by a relatively low specification computing platform while controlling the land or water vehicle in real time, thereby providing a fast and efficient implementation, e.g., on a low power embedded platform.
[0026] The method is a computer-implemented method. Suitable computing platforms (i.e., computers including a processor and memory) are known, for example based on a 1.6 GHz quad-core ARM A53 processor or similar, with a power consumption close to 1 W. In one example, the method is implemented on a computer with a power consumption of at most 5 W, preferably at most 3 W, more preferably at most 1 W.
[0027] This method is known as localization (also known as position determination), e.g. estimating (to a certain accuracy) a geographical position, e.g. represented by GPS coordinates and / or orientation, which is a term used in the art.
[0028] Suitable radar sensors are known, for example millimeter wave radar sensors. For example, the Navtech CTS350-X is a frequency modulated continuous wave (FMCW) scanning radar with no Doppler information, returning 399 azimuth and 2000 range readings at 0.25m range resolution, with a beam spread of 2 degrees in azimuth and 25 degrees elevation starting just below the horizontal. Typically, radar sensors are placed on the roof of a ground vehicle (also known as a land vehicle) or water vehicle with the axis of rotation perpendicular to the driving surface.
[0029] The method includes obtaining a first radar scan of a first environment of the radar sensor, the first environment of the radar sensor including a set of power range spectra including a first power range spectrum (i.e., at least one power range spectrum). The first environment of the radar sensor is the surrounding environment in which the radar sensor operates, e.g., on road, off road, industrial facility such as mining operation, on water, as will be understood in the art. The radar scan is typically represented as a set of power range spectra (i.e., a 1D signal), e.g., one power range spectrum per azimuth angle, where each power spectrum is represented as an array or vector of values s(t)∈R. N×1 Other representations are known. In one example, the power range spectrum set includes P power range spectra, where P is a natural number equal to or greater than 1, e.g., 1, 30, 60, 90, 120, 180, 360, 399, 720, 1080, 1440, or more. Increasing P provides better azimuth resolution and increased processing. In one example, obtaining a first radar scan of a first environment of a radar sensor includes obtaining (also known as capturing) a first radar scan of the first environment with the radar sensor, e.g., in real time (i.e., while a land or water vehicle including the radar sensor is moving through the first environment).
[0030] The method includes extracting a first landmark set from a first radar scan (also known as radar data or raw radar scene data), the first landmark (i.e., at least one landmark) defined by range and azimuth. This step is also known as landmark extraction. The process of landmark extraction is known, and one exemplary process of landmark extraction is described with reference to Figs. 3 and 4. Typically, a landmark is a static and / or unchanging feature (also known as an object) in the environment, a term used in the art. Typically, a landmark is defined by range and azimuth in the art. In one example, the first landmark set includes L landmarks, where L is a natural number equal to or greater than 1, e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 100, 200, 500, or more. In one example, extracting the first landmark set including the first landmark from the first radar scan includes, for example, using a sliding or moving window average filter instead of a sliding or moving window median filter. This reduces operations and therefore increases computation speed without appreciably changing the quality of the extracted landmark. In one example, extracting the first landmark set including the first landmark from the first radar scan includes using non-maximum suppression. For example, in areas with relatively high radar reflectivity, multiple reflections may result in multiple detections at different ranges. Non-maximum suppression may be used to remove or eliminate such repeating patterns.
[0031] The method includes a step of computing a first set of descriptors (also known as scene point descriptors) for each of the first set of landmarks, the first set including a first descriptor defining the first landmark by its respective relative range and azimuth with respect to one or more landmarks included in the first set of landmarks. This step is also known as a first sub-step of pose estimation. It should be understood that the descriptors are unary descriptors for each landmark and define the interrelationships between the landmarks. In one example, the first descriptor is represented as a vector of values that uniquely describes the first landmark. In this way, the landmark can be identified and matched in other radar scans. In one example, the first descriptor identifies the first landmark by radial statistics of neighboring landmarks, e.g., by both range and azimuth. Processes for computing the descriptors are known, and one exemplary process for computing the descriptors is described with reference to FIG. 5.
[0032] The method includes accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets, where it will be appreciated that the landmark reference sets may include a landmark map as stored in a landmark database and / or be such a landmark map extracted from a radar scan previously acquired by the radar sensor and / or another radar sensor to provide a known reference for locating the radar sensor.
[0033] The method includes matching the first descriptor set with a corresponding (i.e., best matching) first descriptor reference set. This step is also known as a second sub-step of pose estimation and is known as data association. In one example, matching the first descriptor set with a corresponding first descriptor reference set includes aligning the first descriptor set with the first descriptor reference set and estimating a difference between them. In one example, matching the first descriptor set with a corresponding first descriptor reference set includes aligning the first descriptor set with each of the descriptor reference sets, e.g., each of the descriptor reference sets, estimating each difference between them, and selecting the corresponding first descriptor reference set with the smallest difference. The matching process is known, and one exemplary process of matching is described with reference to FIGS. 6, 7 and 8.
[0034] The method includes identifying a first location of the radar sensor using a first result of the matching, such that the first location of the radar sensor is identified relative to a corresponding (i.e., best matching) first descriptor reference set.
[0035] (Driving range measurement) In one example, the method includes: obtaining a second radar scan of the first environment of the radar sensor; extracting a second set of landmarks from the second radar scan, the second set including the first landmark; calculating a second set of descriptors for each of the second set of landmarks, the second set including the first descriptor; matching the second set of descriptors with a corresponding second reference set of descriptors; determining a second location of the radar sensor using a second result of the matching; and determining a motion of the radar sensor using the second position and the first position; Includes.
[0036] In this manner, the motion (eg, velocity) of the radar sensor is calculated using the second position and the first position, and implicitly the times of each of the second radar scan and the first radar scan.
[0037] In one example, the method includes repeatedly obtaining, eg, periodically or intermittently, radar scans of a first environment of the radar sensor and repeatedly calculating, mutatis mutandis, the motion of the radar sensor.
[0038] (effect) The method according to the first aspect is suitable for implementation by a low-power computing platform, in particular the method optionally comprises an algorithm implemented in a highly optimized manner allowing execution on a low-power computing platform, as described below.
[0039] (Dimensional Search) In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor to a first projected descriptor. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor set to each first projected descriptor set. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor reference set to each first projected descriptor reference set. In one example, matching the first descriptor set with the corresponding first descriptor reference set includes comparing the first projected descriptor set including the first projected descriptor to the corresponding first projected descriptor reference set.
[0040] The method includes matching a first set of descriptors with a corresponding (i.e., best matching) first reference set of descriptors. Preferably, the method provides a fast and efficient search for matching a descriptor from one scan with that of another scan. For example, given point descriptors of landmarks in scan A, a search is performed for best matching point descriptors (and therefore landmarks) in scan B. A conventional approach is to perform a full comparison for each landmark in scan A against all landmarks in scan B. However, such a conventional approach produces quadratic algorithms that are slow and do not scale well with the number of descriptors in each set. The inventors have developed two specific techniques for matching a first set of descriptors with a corresponding first reference set of descriptors: eigenprojection and distance projection.
[0041] (Proper Projection) In one example, matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptor into a first unique projected descriptor, the first descriptor having a dimension n>1, and the first projected unique descriptor having a dimension n p =1. Thus, by projecting the first descriptors into a lower dimensional space, faster lookups are achieved, resulting in fewer comparisons and less computation. More specifically, by projecting the first set of descriptors into a lower dimensional space, making them easier (fewer comparisons and operations) and faster to compare, the speed of unary candidate matching, which matches points in one scan with their best matches in another scan, is improved.
[0042] for example, Take input = descriptor (n-dimensional, say 700-d) and project it into a much smaller (1D space) specified by the projection direction. Output = For a given point descriptor in scan A, the output is the M closest point descriptors in scan B.
[0043] In one example, the step of matching the first descriptor set with the corresponding first descriptor reference set includes comparing the first unique projection descriptor set including the first unique projection descriptor with the corresponding first unique projection descriptor reference set. Thus, the first descriptor reference set is a set of descriptors that are in the dimension n of both the first unique projection descriptor set and the first unique projection descriptor reference set. p = 1.
[0044] In one example, the step of comparing the first unique projection descriptor set including the first unique projection descriptor with the corresponding first unique projection descriptor reference set includes identifying M closest unique projection descriptors in the corresponding first unique projection descriptor reference set. It should be understood that M is a natural number equal to or greater than 1 and less than the number of unique projection descriptors included in the first unique projection descriptor reference set, i.e., a subset of the first unique projection descriptor reference set is identified. In this way, the number of closest unique projection descriptors identified for subsequent matching is reduced, thereby improving efficiency.
[0045] More specifically, the descriptor projection can also be described as follows.
[0046] The files used as input to the script are point_descriptorrs_urban.txt, and point_descriptors_quarry.txt It is.
[0047] Both files contain point descriptors of the radar landmark scene (landmarks extracted from the raw scans) in a text-readable format.
[0048] The format of the file is a scan with each line containing a point descriptor for each point in the scan separated by a semicolon. So a file for 20 scans would be 20 lines long. If there are 600 points in each scene, then there are 599 (600-1) semicolons per line to separate the 600 point descriptors. If 700-dimensional point descriptors were used, then there are 700 comma-separated numbers between two semicolons on a line. So there are 420,000 (600 points * 700 descriptors) values per line.
[0049] S1PD1;S1PD2;S1PD3;…;S1PD600; S2PD1;S2PD2;S2PD3;…;S2PD600; … S19PD1;S19PD2;S19PD3;…;S19PD600; S20PD1;S20PD2;S20PD3;…;S20PD600;
[0050] In the above, S corresponds to the scan number and PD corresponds to the point descriptor of that scan.
[0051] Figure 11 shows the code used to read the text file.
[0052] The Matlab strsplit script is used to read the scan-delimited text files of point descriptors for the points in each scene.
[0053] K is a matrix of size row by column (with zeros at the beginning), where rows = number of points in each scan, and columns = 700 (the size of the point descriptor). K is set by converting a string to a numeric representation.
[0054] D is a collection of scans and D{i} is the matrix of point descriptors for scan i (corresponding to row i in the text file). DUrban can be a collection of scans from point_descriptors_urban.txt. DQuarry can be a collection of scans from point_descriptors_quarry.txt.
[0055] D Urban and D Quarry are concatenated onto D, followed by a random permutation of integers. This can be achieved by running randperm to mix up all the scans.
[0056] The full scan set is split into training scenes (~80% of scans) and testing (~20% of scans). Using the Matlab cat function, the training scenes may be concatenated along axis 1. This means that matrices are added vertically on top of each other, so the number of columns also equals the size of the point descriptor, and the number of rows is the number of points_per_scan * number of scans in the training set.
[0057] Figure 12 shows the code used to calculate trained_mean and trained_basis.
[0058] Afterwards, we call decompose, which does all the heavy lifting to compute trained_mean and trained_basis. [projection_mean,projection_basis]=decompose(training_descriptors,num_projection_dimensions); (where num_projection_dimensions is the number of dimensions to be projected, which is 1.)
[0059] FIG. 13 shows the code used to reduce the dimensions of the matrix.
[0060] The Matlab mean function is used to obtain a vector of mean values for each element of the point descriptor. Such a vector of mean values is later saved as trained_mean. For example, if A is a matrix, mean(A) returns a row vector containing the mean value of each column. This mean value is then used to shift all point descriptors (shifted_observations).
[0061] The Matlab cov function is used, e.g., C=cov(A) returns the covariance. If A is a matrix with columns representing random variables and rows representing observations, then C is a covariance matrix containing the corresponding column variances along the diagonal. In other words, for a matrix A whose columns are each random variable composed of observations, the covariance matrix is the pairwise covariance estimate between each combination of columns. This estimate can be expressed as C(l,j)=cov(A(:,i),A(:,j)).
[0062] It is important to note that the descriptor vectors themselves are used to construct the covariance matrix. In other words, the descriptor elements themselves are used as signals. This differs from prior art methods where differences are used instead. Using the descriptor elements themselves makes training much easier.
[0063] Given all the observations, the mutual covariance of all the point descriptor variables is calculated. For example, a covariance matrix P is generated using P=cov(shifted_observatoins).
[0064] Afterwards, we use eig to compute the eigenvalues and eigenvectors of the covariance matrix, which can be explained as A*V=V*D, where [V,D]=eig(A) returns a diagonal matrix D of eigenvalues and a matrix V whose columns are the corresponding right eigenvectors.
[0065] The eigenvalues are then sorted in descending order. This sorting can be described in two steps. In the first step, use B=sort(A,direction) to sort the elements of A in the order specified by the direction using one of the previous syntaxes, where 'ascend' indicates ascending order (default) and 'descend' indicates descending order. In the second step, use [B,I]=sort(_) to return a collection of index vectors for one of the previous syntaxes, where I is the same size as A and describes the arrangement of the elements of A in B along the sorted dimension. For example, if A is a vector, then B=A(I).
[0066] Then, the principal eigenvector is taken for the future reduced dimension N, i.e., the eigenvector corresponding to the top N eigenvalues after sorting. In this case, since the reduction is to a single dimension, the eigenvector corresponding to the largest eigenvalue is taken. This single largest eigenvalue is the basis vector.
[0067] It is important to note that dimensionality reduction in this method is a means of "summarizing" or "compressing" descriptors to facilitate comparison and matching with other descriptors as a first step in the matching process, thereby reducing the search space and speeding up the overall data association process. Such methods differ from prior art methods that use dimensionality reduction to filter dimensions that give no information about whether or not to match.
[0068] Dimensionality reduction can be used in this way to speed up the process of matching one point descriptor with those of another scan by reducing the dimensionality of the point descriptors themselves.
[0069] Importantly, with regard to training the basis vectors, the present subject matter provides a method for training and generating the basis vectors or principal eigenvectors from the sampled point descriptors themselves.
[0070] (Distance projection) In one example, matching the first descriptor set with the corresponding first descriptor reference set includes identifying M closest descriptors in the corresponding first descriptor reference set and finding a single closest descriptor from the M closest descriptors.
[0071] In one example, the method includes: summing first absolute differences between the first set of descriptors and the first reference set of descriptors and setting a threshold absolute difference as the summed (i.e., total) first absolute difference; summing second absolute differences between the first set of descriptors and the second reference set of descriptors, where the summing (i.e. running sum) of second absolute differences is at most the threshold absolute difference; if the second absolute difference in the summation exceeds a threshold absolute difference, stopping the summation of the second absolute differences and starting a summation of a third absolute difference between the first set of descriptors and the third reference set of descriptors; and if the summed second absolute difference does not exceed the threshold absolute difference, resetting the threshold absolute difference as the summed second absolute difference.
[0072] In this way, the process improves early based on the distance metric (i.e., absolute difference in the sum) to date, which is the sum of absolute differences (also known as the L1 norm), allowing for faster searches by reducing the search space based on distance. In other words, the method gives up or moves on from descriptors that already exhibit a larger sum of absolute differences (SAD) than the current best candidate (i.e., the descriptor reference set for which the termination threshold absolute difference was set or reset). This process is sometimes known as the distance trick.
[0073] for example, Input = For a given point descriptor in scan A, the M closest point descriptors in scan B. Output = single closest (by L1-norm) point descriptor from a set of M descriptors. Distance tricks make this calculation (finding the closest descriptor) much faster and more efficient.
[0074] In one example, the method includes ordering (also known as sorting) the set of descriptor references by likelihood of match, e.g., by decreasing likelihood of match. By ordering the candidates by likelihood of match in this manner, great savings are achieved by examining the most likely, and therefore lowest, SAD dFsubstescriptors first.
[0075] (Descriptor size change) In one example, the method includes: projecting the first descriptor into a first projected descriptor, the first descriptor having a dimension N and the first projected descriptor having a dimension M, where M ≠ N; Matching the first descriptor set with a corresponding first descriptor reference set includes matching a first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.
[0076] Thus, rather than recalculating the first descriptor to a larger or smaller dimension M, the first descriptor is resized (i.e., projected) to a larger or smaller dimension M. Smooth Descriptor Resizing for Velocity Control = Ability to resize (i.e., scale) radar point descriptors so that existing calculated descriptors can be converted to larger or smaller sizes rather than recalculating existing calculated descriptors. Any implementation or method for doing this is fine.
[0077] for example, Input = size N, e.g. 700 point descriptors (i.e. vectors of size N). Output = size M, e.g. 400 equivalence point descriptors (i.e. vectors of size M).
[0078] In one example, projecting the first descriptor into the first projected descriptor comprises interpolating its elements, thus reducing the dimension M.
[0079] In one example, projecting the first descriptor into the first projected descriptor comprises averaging or dropping its elements, thus increasing the dimension M.
[0080] (Memory of calculated value) In one example, the method includes calculating one or more values in response to a request, storing the calculated one or more values, and returning the stored one or more values or one or more values derived therefrom in response to a subsequent request.
[0081] The inventors have developed techniques to further optimize the computation, especially for low-power hardware with limited computational resources. The inventors have developed two specific techniques to reduce or minimize repetitive computation: caching and the use of lookup tables.
[0082] (caching) In one example, the one or more values include and / or are a descriptor set and calculating the one or more values includes calculating the descriptor set. In one example, the one or more values include and / or are a descriptor reference set and calculating the one or more values includes calculating the descriptor reference set.
[0083] In one example, the method includes storing a descriptor reference set; Matching the first descriptor set with a corresponding first descriptor reference set includes matching the first descriptor set with the corresponding first descriptor set of the stored descriptor reference sets.
[0084] In this way, the descriptors can be cached (ie, stored) for later use, thereby avoiding recomputation of the descriptors, further improving efficiency.
[0085] For example, for a particular radar scan, eg, the most recent live scan from a radar sensor, the computational (usually estimated) outputs and / or intermediate values (ie, estimated values) for that particular radar scan may be stored as follows: 1. The method computes a value once and then reuses that computed value, which means that the method does not repeat the same computation. 2. In this method, all the calculations for a particular scan are cached together in one instance of the software object.
[0086] for example, Input = point cloud of landmarks extracted from radar scan. Output = A RadarSceneData object containing the scan and all the computations performed by it (including the point descriptor, which is a matrix of values, where each column is the point descriptor of a point, thus the matrix is NxM, where N is the dimension of the point descriptor and M is the number of points in the radar landmark scan).
[0087] More specifically, the following data is required: A landmark point cloud, which is point cloud data of the 3D positions of landmarks. (The landmark point cloud can be provided in the form of a live point cloud of landmarks or a map point cloud.) A sampling mask vector that defines the mask used by the sampling policy to select landmarks in the returned point cloud. (The sampling mask vector can be constructed using a user-defined sampling policy.) A descriptor template containing trained vectors and basis vectors that are used to project the complete point descriptor in the scene into a low-dimensional space for fast lookup. (This descriptor template is provided to you as a trained template.)
[0088] From the required data the following can be calculated: The calculated data can be cached / stored for efficiency and reused later: An intrinsic 3D matrix of the x, y, z positions of the landmarks in the scene. (This matrix is more convenient as the intrinsic type for other calculations.) A matrix of inter-point distances, i.e. distances from each point to every other point. (This matrix is needed for the later calculation of point descriptors. This matrix is calculated from the landmark points.) A matrix of angles between points. (The angles may be calculated using a Graphics Processing Unit (GPU). The angles may be the angles from each point to every other point, and these angles may be used to calculate the point descriptors. These angles may be calculated from the landmark point cloud.) A point descriptor matrix, which is a matrix of point descriptors of the scene. (This matrix can be computed from landmarks of the point cloud.) A scene descriptor vector, which is a vector of scene descriptors projected into a low-dimensional space (e.g., 1D) by the trained basis vectors. (This makes it easier to reduce the search space when matching point descriptors between scenes, since the search may be done in only 1D initially to get a reduced set and search more completely. The scene descriptor vector may be computed using the descriptor template and the computed point descriptor matrix.) A range vector, which is the range of a point in the landmark point cloud. (The range vector can be calculated from the landmark point cloud.)
[0089] (Lookup table) In one example, the method includes calculating one or more values in response to a request, storing the calculated one or more values, and returning the stored one or more values or one or more values derived therefrom in response to a subsequent request.
[0090] In one example, the one or more values include and / or are a set of descriptors and / or an intermediate value thereof, and storing the calculated one or more values includes storing the calculated one or more values in a set of lookup tables including a first lookup table.
[0091] In one example, the step of computing a first set of descriptors including a first descriptor for a first set of landmarks uses a set of lookup tables including a first lookup table.
[0092] In this way, a lookup table (i.e., a pre-computed lookup table) is used for the complex function, which eliminates the need to repeatedly calculate the complex function, which is more efficient and significantly increases speed.
[0093] In one example, the method includes generating a first lookup table at compile time (ie, at the time of compilation, during compilation) and using the first lookup table at run time (ie, at the time of execution, during execution).
[0094] For example, the "ArcCosineApproximator" class may be used when computing the descriptors, providing a fast lookup solution for "acos". A lookup table is generated at compile time using the quotient of two polynomials that approximate "acos", and used at run time.
[0095] In one example, the method includes generating the first lookup table at run-time during a first computation of a given value of the first lookup table.
[0096] (process) In one example, the method includes calculating two or more values simultaneously and / or performing a correlated calculation using two or more values.
[0097] The inventors have developed techniques to further optimize the computation, especially for low-power hardware with limited computational resources. The inventors have developed two specific techniques to accelerate and / or simplify the computation: the use of parallel processing and triangulation.
[0098] (Parallel processing) In one example, computing a first descriptor set including a first descriptor for a first landmark set includes processing the first landmark set in parallel, e.g., using single instruction multiple data processing (SIMD).
[0099] In this way, the method optimizes the use of available hardware, improving speed and efficiency, for example, a first set of descriptors for a first set of landmarks may be computed in parallel since the same instructions are being applied.
[0100] for example, Input = Point cloud of landmarks extracted from radar scan. M is the number of points in the radar landmark scan. Output = Radar point descriptor, which is a matrix of values (each column is the point descriptor of a point, thus this matrix is NxM, where N is the dimension of the point descriptor and M is the number of points in the radar landmark scan).
[0101] (triangulation) In one example, computing the first descriptor set including the first descriptor for the first landmark set includes triangulating the first landmark with respect to each node and landmark in the first landmark set, thus improving efficiency.
[0102] In one example, triangulating the first landmark with respect to each node and landmark in the first landmark set includes using the cosine law (also known as the cosine theorem, cosine formula or Al-Khasi's theorem), e.g., as described with reference to FIG. 9.
[0103] (Signature) In one example, the method includes: representing the first set of landmarks as a first signature; expressing the landmark reference set as respective reference signatures; and correlating the first signature with a reference signature to approximate a first position of the radar sensor.
[0104] In this manner, a first location of the radar sensor is approximated with a relatively low degree of accuracy by correlating the first signature with the reference signature, and then the first location of the radar sensor is identified with a relatively high degree of accuracy using the respective landmark reference sets represented by the correlated reference signatures. In other words, this provides a first pass approximation of the first location for subsequent refinement, generally identifying the location of the first sensor on the map.
[0105] More specifically, the radar signature feature provides a method to find loop closure using radar sensor data (i.e., radar scans) to identify and locate map nodes, for example, at initialization or when localization is lost. While an external source can be used for the radar signature, the advantage of the radar signature is that only radar data is required for the radar signature, thus eliminating dependency on other sensor modalities.
[0106] It should be appreciated that a radar signature (i.e., signature) is a very compact representation of a radar landmark point cloud. The plane around the radar is divided into a series of regions in a polar representation derived from the radar itself. The points are assigned to a corresponding n-dimensional histogram with bins for combinations of distance (i.e., range) and angle (i.e., azimuth). The histograms are normalized to sum to one.
[0107] The two signatures are compared using the complement of the histogram intersection metric as the similarity measure, resulting in a value of 0 representing the best possible match. The best candidate node to initialize in the map can be selected as the node with the lowest score for the current radar sensor data (considering both nodes converted to signature representations). Since all signatures for each map node can be computed at startup, the algorithm is fast enough to find the best matching node in a fraction of a second.
[0108] The threshold signature_similarity_threshold is the maximum similarity measure allowed for the best candidate node, otherwise the algorithm reports that no suitable map node has been found. The remaining parameters control the structure of the set of regions used to derive the histogram. Such a structure is a disk with the radar at its center and a radius equal to signature_max_range. This disk is divided into regions radiating from the center, defined by signature_num_angle_bins and signature_num_range_bins. The resulting histogram therefore has a total number of bins equal to the product of signature_num_angle_bins and signature_num_range_bins.
[0109] In one example, accessing the landmark reference set includes selectively accessing a landmark reference set represented by a reference signature.
[0110] (Land or water vehicles) In one example, a land or water vehicle includes a radar sensor.
[0111] In one example, the land or water vehicle is an unmanned, semi-autonomous and / or autonomous land or water vehicle. In general, an unmanned vehicle (also known as a pilotless vehicle) is a vehicle that does not have a human on board. An unmanned vehicle can be either a remote-controlled vehicle, a semi-autonomous vehicle or an autonomous vehicle that can sense the environment and navigate autonomously. Unmanned vehicles include unmanned ground vehicles (UGVs), such as autonomous cars, and unmanned surface vehicles (USVs), which are intended to operate on water surfaces.
[0112] Typically, land vehicles (also known as vehicles) include military vehicles, commercial vehicles, and / or personal land vehicles. Military vehicles include combat and transport vehicles, such as military ambulances, amphibious military vehicles, armored fighting vehicles, electronic warfare vehicles, military engineer vehicles, improvised combat vehicles, joint light tactical vehicles, military utility light vehicles, off-road military vehicles, reconnaissance vehicles, recovery vehicles, self-propelled weapons, self-propelled anti-aircraft weapons, self-propelled artillery, tanks, tracked military vehicles, half-tracks, military trucks, and wheeled military vehicles. Commercial vehicles include trucks (such as box trucks, articulated trucks, and pickup trucks), buses and coaches, heavy equipment (such as those used in mining, construction, and agriculture), and passenger cars such as taxis. Personal land vehicles include passenger cars and trucks. Other land vehicles are known.
[0113] Typically, watercraft include military watercraft, commercial watercraft, and / or recreational watercraft, including vehicles on the surface of the water. Classes of military watercraft include aircraft carriers, cruisers, destroyers, frigates, corvettes, large patrol vessels, small surface combatants such as missile boats, torpedo boats, and patrol boats including rigid inflatable boats (RIBs), mine warfare vessels such as mine countermeasures vessels, minesweepers, minesweepers, and minelayers, amphibious warfare vessels such as amphibious assault ships, dock landing ships, landing craft and landing ships, and air cushion landing craft. Classes of commercial watercraft include passenger ships such as container ships, bulk carriers, tankers, ferries, and cruise ships, and specialized vessels such as coastal traders, anchor carriers, supply ships, tugboats, salvage ships, research vessels, trawlers, and whalers. The class of recreational (also known as recreational) watercraft includes boats and yachts, such as pontoons, bowriders, cabin cruisers, houseboats, trawlers, motor yachts, catamarans, etc. Other watercraft are also known.
[0114] (Dimensional Search) A computer-implemented method for locating a radar sensor according to a second aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, Matching the first descriptor set with a corresponding first descriptor reference set includes projecting the first descriptors to a first projected descriptor.
[0115] The method according to the second aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.
[0116] (Descriptor size change) A computer-implemented method for locating a radar sensor according to a third aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is The method further includes projecting the first descriptor into a first projected descriptor, the first descriptor having a dimension N and the first projected descriptor having a dimension M, where M ≠ N; Matching the first descriptor set with a corresponding first descriptor reference set includes matching a first projection descriptor set including the first projection descriptor with a corresponding first descriptor reference set of the plurality of descriptor sets.
[0117] The method according to the third aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.
[0118] (Memory of calculated value) A computer-implemented method for locating a radar sensor according to a fourth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is The method further includes the steps of calculating one or more values in response to the request, storing the calculated one or more values, and returning the stored one or more values or one or more values derived therefrom in response to a subsequent request.
[0119] The method according to the fourth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.
[0120] (process) A computer-implemented method for locating a radar sensor according to a fifth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is Further steps include calculating two or more values simultaneously and / or performing correlated calculations using two or more values.
[0121] The method according to the fifth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.
[0122] (Signature) A computer-implemented method for locating a radar sensor according to a sixth aspect includes the steps of: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from a first radar scan, the first landmark being defined by a range and an azimuth angle; calculating a first set of first descriptors for each of the first set of landmarks, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets; matching a first set of descriptors with a corresponding first reference set of descriptors; identifying a first location of the radar sensor using a first result of the matching; Including, This method is representing the first set of landmarks as a first signature; expressing the landmark reference set as respective reference signatures; correlating the first signature with a reference signature to approximate a first position of the radar sensor; Includes.
[0123] The method according to the sixth aspect is as described in relation to the first aspect mutatis mutandis and may include any step described in relation to the first aspect.
[0124] (Control of land or water vehicles) According to a seventh aspect, there is provided a computer-implemented method for controlling a land or water vehicle including a radar sensor, the method comprising the steps of: - locating a radar sensor according to the first, second, third, fourth, fifth and / or sixth aspects; controlling a land or water vehicle using the first position; Includes.
[0125] In this manner, the land or water vehicle is controlled using the first position, for example for navigation. The land or water vehicle may be as described in relation to the first aspect.
[0126] In one example, controlling the land or water vehicle using the first position includes controlling the land or water vehicle in response to the first position.
[0127] In one example, controlling the land or water vehicle using the first position includes steering the land or water vehicle.
[0128] In one example, controlling the land or water vehicle using the first position includes semi-autonomously or autonomously controlling the land or water vehicle using the first position.
[0129] (Computer, computer program, non-transitory computer-readable storage medium) A computer according to an eighth aspect includes a processor and memory configured to execute the methods according to the first, second, third, fourth, fifth, sixth and / or seventh aspects.
[0130] A computer program according to a ninth aspect comprises instructions which, when executed by a computer comprising a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect, the third aspect, the fourth aspect, the fifth aspect, the sixth aspect and / or the seventh aspect.
[0131] A non-transitory computer-readable storage medium according to a tenth aspect includes instructions that, when executed by a computer including a processor and a memory, cause the computer to perform a method according to the first aspect, the second aspect and / or the third aspect.
[0132] (Land or water vehicles) A land or water vehicle according to an eleventh aspect includes a radar sensor and a computer according to the eighth aspect.
[0133] The land or water vehicle may be as described in relation to the first aspect.
[0134] (definition) Throughout this specification, the terms "comprising" or "comprises" mean including the specified component but not excluding the presence of other components. The terms "consisting essentially of" or "consists essentially of" mean including the specified component but excluding other components except for materials present as impurities and unavoidable materials present as a result of the process used to provide the component, and components added for purposes other than achieving the technical effect of the present invention, such as colorants.
[0135] The terms "consisting of" or "consists of" mean the inclusion of certain components but the exclusion of other components.
[0136] Wherever appropriate and depending on the context, use of the words "comprise" or "comprising" may be interpreted to include the meaning "consisting essentially of" or "consisting essentially of" and may be interpreted to include the meaning "consisting of" or "consisting of."
[0137] The optional features described herein can be used individually or in combination with each other as appropriate, in particular in the combinations described in the accompanying claims. The optional features of each aspect or exemplary embodiment of the present invention described herein can also be applied to all other aspects or exemplary embodiments of the present invention as appropriate. In other words, a person skilled in the art reading this specification considers that the optional features of each aspect or exemplary embodiment of the present invention are interchangeable and combinable between different aspects and exemplary embodiments. [Brief description of the drawings]
[0138] For a better understanding of the present invention and to show how exemplary embodiments thereof may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which:
[0139] [Figure 1] 1 illustrates a schematic plan view of an exemplary embodiment of a radar sensor; [Diagram 2] 1 illustrates a schematic diagram of a method according to an exemplary embodiment; [Diagram 3] 3 shows the method of FIG. 2 in more detail. [Figure 4] 3 is an example of an algorithm for landmark extraction in the method of FIG. 2. [Diagram 5] 3 shows the method of FIG. 2 in more detail. [Figure 6] 3 shows the method of FIG. 2 in more detail. [Figure 7] 3 shows the method of FIG. 2 in more detail. [Figure 8] 3 is an example of a data association algorithm for the method of FIG. 2. [Figure 9] 3 shows the method of FIG. 2 in more detail. [Figure 10A] 1 illustrates generally how a signature is constructed from a radar scan. [Figure 10B] 1 illustrates generally how a signature is constructed from a radar scan. [Figure 10C] 1 illustrates generally how a signature is constructed from a radar scan. [Figure 11] 1 shows code used in certain embodiments of the present disclosure. [Figure 12] 1 shows code used in certain embodiments of the present disclosure. [Figure 13] 1 shows code used in certain embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0140] Figure 1 shows a schematic of a plan view of an exemplary embodiment of a radar sensor, in particular an FMCW scanning radar. In this example, a radar sensor (green circle) at the center of a vehicle (black box) sequentially collects power range spectra (radial green dotted lines) at each azimuth angle. The variables a and r denote azimuth angle and range, respectively. Sample signals at particular azimuth angles are plotted as power (dB) as a function of range bins.
[0141] 2 illustrates a schematic of a method according to an exemplary embodiment, which is a computer-implemented method for localizing a radar sensor.
[0142] In general, the method involves two main steps: landmark extraction and pose estimation.
[0143] The method includes obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum (ie, a 1D signal).
[0144] The method includes extracting a first landmark set from a first radar scan, the first landmark being defined by range and azimuth angle. This step is also known as landmark extraction or feature extraction. CFAR is a common filtering algorithm, but is not distinctive enough. In contrast, the method described herein can detect more reliable and distinctive landmarks in radar scene data. For example, a complete radar scan is received (i.e., acquired) from a radar sensor, and the method performs landmark extraction to accurately detect objects in the environment up to the maximum range of the radar sensor, for example, as described with reference to Figs. 3 and 4. The radar scan is a set of power range spectra (i.e., 1D signals), one power range spectrum for each azimuth angle, for example, as described with reference to Fig. 1. In this example, the power range spectrum is represented as an array of values for each azimuth angle, over the entire circumference of the radar sensor. In this example, the output from the landmark extraction is a point cloud, which is a set of points (corresponding to landmarks), each identified by range and angle (i.e., azimuth angle) from the center line.
[0145] The pose estimation step then uses the new landmark points to determine a pose relative to the previous landmark points and a position relative to a map database containing the landmark points captured along the route during the mapping phase.
[0146] The pose estimation step involves two main sub-steps: first, computing scene point descriptors, then using these scene point descriptors and the landmark point cloud to align the two point clouds and estimate the position difference between the two point clouds. Each scene point descriptor is a collection of unique "descriptors", one "descriptor" for each point in the point cloud. Thus, the scene point descriptor is represented as a matrix of values where each column represents the descriptor for each point. A descriptor needs to be computed, which is represented as a vector of values that uniquely describes the point, thus allowing it to be identified and matched in other scans. The descriptor identifies the landmark point by the radial statistics of its neighbors in both range and angle slices.
[0147] The method includes calculating a first set of first descriptors for each of the first landmark sets, the first descriptors defining the first landmarks by their respective relative ranges and azimuth angles with respect to one or more landmarks included in the first landmark set, for example as described with reference to FIG.
[0148] The method includes the steps of accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of the landmark reference sets, for example as described with reference to Figures 6-8.
[0149] The method includes matching a first set of descriptors with a corresponding first reference set of descriptors, e.g., as described with reference to Figs. 6-8. This step is known as data association and matches landmarks across a radar scene. Other approaches use popular feature descriptors for vision systems, but other approaches do not work as well for radar data. We use novel feature descriptors that are more suitable for radar landmarks, improving data association between a live scan and other scans seen in the past, e.g., previous scans in the case of radar driving range measurements, and map scans in the case of localization.
[0150] The method includes identifying a first location of the radar sensor using a first result of the matching, e.g., as described with reference to Figures 6 to 8. This step, also known as position estimation, determines the spatial distance between the two landmark sets.
[0151] Figure 3 illustrates the method of Figure 2 in more detail. In particular, Figure 3 shows in schematic form the procedure for extracting landmarks from a power range spectrum. The input (raw signal) is processed from the top left to produce the output at the bottom right, where landmarks are indicated by red stars during the process. Box 6 in this example highlights the ability of the approach to remove multipath reflections and detection due to noise. Boxes 3 and 5 show the importance of incorporating high frequency signals, as using a smooth signal alone in boxes 2 and 4 would ignore the high range resolution provided by FMCW radar.
[0152] The first objective is to accurately detect objects in the radar sensor's environment with minimal false positives. Specifically, the method needs to find all landmarks sensed by the radar sensor while minimizing the number of redundant returns per landmark and avoiding detection of absent landmarks due to noise, multipath reflections, harmonics, side lobes, etc. The method takes as input a power range spectrum (i.e., a 1D signal) and returns a set of landmarks, each identified by range and azimuth angle. The core concept is to estimate the noise statistics of the signal and then scale the power value at each range by the probability that it corresponds to an actual detection. Consecutive peaks in this reshaped signal are treated as objects, and for each peak, only the range at the center of the peak is added to the landmark set.
[0153] Vector s(t) ∈ R N×1 Let be the power range spectrum at time t, and thus, the element s iis the power return in the i-th range bin and a(t) is the associated azimuth angle. Let r(i) = β(i-0:5) give the bin range i ∈ {1, 2, …, N}, where β is the range resolution. If the environment is perfectly recorded, then y(t) ∈ R N×1 Assume that s(t) is an ideal signal. Then, s(t) = y(t) + v(y(t)), where v represents unwanted effects such as noise. Therefore, to infer y(t) from s(t) in order to accurately separate the landmarks, an approximation of v(y(t)) is needed, thus
number
number
number
[0154] FIG. 4 shows Algorithm 1, which is an example of an algorithm for landmark extraction in the method of FIG.
[0155] First, we obtain an unbiased signal q (box 2) that preserves high-frequency information by subtracting the noise floor of v(s) from s (line 1). We then smooth the result to obtain the underlying low-frequency signal p (box 3), which better exposes obvious landmark peaks (line 2). At this point, q is not discarded because smoothing weakens the presence of radar landmarks, since they often appear as high-frequency peaks, and because smoothing blurs nearby landmark peaks, making them difficult to distinguish. Therefore, we integrate information from both q and p. To estimate noise characteristics, we consider values of q below zero to be the mean μ q = 0 and standard deviation σ q We treat the noise as Gaussian noise with f(μ,σ 2 ) with normal distribution N(μ,σ 2 ) at x. Then, for every range bin, the power value is scaled by the probability that it does not correspond to noise in two steps. First, the smoothed signal p i Each value of is
number
number
number
number
number
[0156] In this method,
number
number
number
number
number
[0157] FIG. 5 illustrates generally the method of FIG. 2 in more detail.
[0158] In this example, the pose estimation step uses the output landmark point cloud to determine a position (i.e., a first position) of the radar sensor relative to a map database that includes a cloud of reference landmark points (i.e., reference landmarks) captured or acquired along the route during, for example, the mapping phase of the radar sensor, and in this example, the pose estimation step also uses the output landmark point cloud to determine a position relative to a previous landmark point cloud (i.e., for driving range measurements).
[0159] The pose estimation step includes two main sub-steps: first, computing the scene point descriptors, then using the scene point descriptors and the landmark point clouds to align the two point clouds and estimate the position difference between the two point clouds. The scene point descriptors are each a collection of unique "descriptors", one "descriptor" for each point in the point cloud. Thus, in this example, the scene point descriptors are represented as a matrix of values where each column represents a descriptor for each point. The descriptors are computed and represented as vectors of values that uniquely describe the point, thus allowing it to be identified and matched in other scans. The descriptors identify landmark points by radial statistics of neighboring points in both range and angular slices.
[0160] FIG. 6 illustrates generally the method of FIG. 2 in more detail.
[0161] Once the scene point descriptors are calculated, a data association step can be performed using the scene point descriptors to match each landmark point of one radar scan to the landmark points of other radar scans. Finding the best matching descriptor from one scan to the other can be computationally expensive, so the inventors have developed a number of improvements to improve efficiency, as described herein. Once a set of correspondences from one point cloud to another has been determined, the objective is to select the best match to ensure that the registration between the point clouds is robust to outliers and false positives. Given good overlap between the scans and stable associations, the sensor motion that must have occurred from one scan to another can be calculated. In this example, the motion estimate is output by the computer.
[0162] Figure 7 shows the method of Figure 2 in more detail. The core concept behind the data association algorithm aims to find similar shapes within two landmark point clouds (red) extracted from radar scans. Unary candidate matches (green dotted lines) are generated by comparing the angular characteristics of the points. The selected matches (A,A') and (B,B'2) are then compared using the pairwise distance difference (|dAB -d' AB2 |<|d AB -d' AB1 In this way, the matching of shapes is approximated by sequentially comparing angles and side lengths.
[0163] More specifically, the scan matching algorithm achieves robust point correspondence using the high level information of the radar scan. Intuitively, it aims to find the largest subset of two point clouds that share a similar shape. Unlike ICP, this method works without prior information on the orientation of the scans or their mutual displacement. Thus, the algorithm herein is not constrained to obtain a good initial estimate of the relative pose and can compare point clouds captured at any time without maps. The only requirement is that the observed regions are on the same plane and contain sufficient overlap. One important attribute of the approach herein is that it performs data association using not only the individual landmark (i.e., unary) descriptors but also the relationships between the landmarks. For example, imagine three landmarks that form the vertices of a scalene triangle. Then, the set of distances from each point to its neighbors is unique to that point, regardless of the arrangement of the entire point cloud. This allows landmarks to be directly matched with corresponding landmarks in any other point cloud obtained by applying a rigid transformation to the original triangle. The larger the number of points, the less likely it is that the set of pairwise distances from an individual point to its neighbors will be the same as another point. Furthermore, the exact location and orientation of the points does not affect the pairwise relationships within the point cloud, so large differences between the placement and orientation of the points are not important. We exploit these observations to obtain reliable matching of large landmark collections. For real data, the main challenge is that landmark location and detection are noisy, which means that points do not always survive rigid transformations, and that the locations of points that survive rigid transformations are affected by noise. A simple example illustrating the concept behind the data association algorithm is shown in Figure 7.
[0164] FIG. 8 shows an example of a data association algorithm for the method of FIG. 2, Algorithm 2. As input, two point clouds L for each of two radar scans are taken. O and L I The first group of points L O is the original landmark set in Cartesian coordinates. Since landmarks are detected in polar space, the resulting point cloud will be dense at low ranges and sparse at high ranges. For this situation, we define a second point cloud L I compensates by generating a binary Cartesian grid of the interpolated resolution from a binary polar grid of landmarks. The latter point cloud is less accurate and is only used to avoid range density bias when processing the layout of the environment, while data association is performed on the former point cloud (i.e., the algorithm does not use the landmarks).
number
number
number
number
[0165] This maximization is computationally intractable due to the discretization of m. Therefore, we relax the above constraint and maximize it for continuous values of u as follows: * Request. u * =uCu
[0166] Under these conditions, u * is the normalized eigenvector of the maximum eigenvalue of the semidefinite positive matrix C. Then, the optimal solution m * is calculated using the greedy approach shown in lines 3 to 11. *In essence, the greedy method iteratively adds satisfactory matches to a set M. After each iteration, the remaining valid matches are evaluated (line 7), the match that returns the maximum reward is accepted (line 9), and matches that conflict with this match are removed from further consideration (lines 10 and 11). The algorithm terminates if the reward of the last selected match is less than the reward that would be obtained if all matches were equally evaluated (i.e., it is a weak match), or if the reward is greater than the reward that would be obtained if more than α percent of the landmarks in either set match (lines 6 and 8). Note that the match is the only free parameter of the method, and there is no need to remove outliers.
[0167] FIG. 9 illustrates generally the method of FIG. 2 in more detail.
[0168] In this example, computing a first descriptor set including the first descriptor for the first landmark set includes triangulating the first landmark with respect to each node and landmark in the first landmark set. In this example, triangulating the first landmark with respect to each node and landmark in the first landmark set includes using the cosine law.
[0169] The route (also known as reference) landmarks or points are fixed to landmark or point i. Therefore, for each route landmark, the angles and distances with respect to all landmarks or points can be efficiently calculated.
[0170] In summary, parallel computation using SIMD and cosine law are computed simultaneously. As an example, point cloud data in the form of a descriptor is obtained. The point cloud data is divided into multiple chunks. The number of points in a chunk is determined based on the capabilities of the processor. A single instruction may be executed to apply a single instruction to each point in the group simultaneously. The single instruction may be to calculate the distance between the points in the chunk. Once the distance is known, calculate the angle of the points in the chunk using cosine law, also using a single instruction such as SIMD.
[0171] Once a first chunk has been processed in this manner, a second chunk is selected, and so on, resulting in parallel processing of each chunk of the multiple chunks. Processing of the multiple chunks occurs serially. In this manner, all of the chunks may be processed. The order in which chunks are selected for processing may be selected randomly.
[0172] (Signature) In this example, the radar signature header defines three data types: Signature, ExperienceSignature, and MapSignature. Signature is a two-dimensional vector of doubles that represents the radar signature itself. ExperienceSignature is a std::map of Signatures indexed by node_id, and MapSignature is a std::map of ExperienceSignature indexed by experience_name.
[0173] The radon::loopclosure::RadarSignatureBuilder class has a constructor with three arguments that correspond to the signature challenge, azimuth bin, and range bin parameters defined and described above. RadarSignatureBuilder generates a signature that corresponds to these parameters. RadarSignatureBuilder objects have a ComputeSignature method that generates a Signature given a point cloud of radar landmarks extracted from a raw radar scan.
[0174] There is also a ComputeSignaturesForMap method that returns the MapSignatures data type for the entire map, given a map_client and a string representing the attribute name used to store the radar data.
[0175] A function CompareSignatures is provided that computes a similarity score for a given pair of Signatures. Two further functions make use of this comparison function. FindCandidateLoopClosureNodes takes a Signature, ExperienceSignatures and a threshold and returns a std::vector of all node ids in the experience with a similarity below the threshold. This function is intended for use with signatures during map construction. For location, the function FindBestCandidateMapNode is provided. Given a Signature, MapSignatures and a threshold, the function returns the single best matching node_id in the map as long as it is below the similarity threshold.
[0176] 10A-10C, as an autonomous vehicle traverses a route on a map, it captures radar scans. Each scan may be referred to as a node 100 having a location in the map where the scan was captured. In this manner, a "node" may refer to a point where a radar scan was captured.
[0177] A scan at a node 100 captures features along multiple azimuth angles, as shown in Figure 1. Multiple range bins are provided for each azimuth angle. These range bins can be visualized as multiple concentric rings, each with an equal number of segments 102 separated according to the distribution of azimuth angles 104 (only four azimuth angles are shown in the figure to avoid obscuring the drawing). Each segment 102 is assigned a number that corresponds to the number of features detected in that segment by the radar scan.
[0178] A vector having multiple values may be generated. The number of elements in the vector corresponds to the number of segments. The value of each element is equal to the number of the corresponding segment. This vector is the signature 106. More specifically, this vector is described as a node signature.
[0179] The autonomous vehicle may capture multiple scans of different nodes along the route 108, resulting in a node signature for each node, which are then combined to form a route signature.
[0180] Multiple route signatures may be created if the same or multiple autonomous vehicles traverse multiple routes, and a route signature may be generated that includes multiple corresponding route signatures.
[0181] These previously generated signatures correspond to reference signatures.
[0182] During operation, a signature is generated for a current position. The signature of the current position may be referred to as a first signature. The first signature is compared to a reference signature to determine the closest match. The best matching reference signature is correlated to the first signature. Correlated means that the first signature is equivalent to the best matching reference signature. In this manner, the best matching reference signature can be used to approximate the position and orientation of the first signature.
[0183] Descriptor matching as described above can then be used to more accurately determine position and attitude. It is computationally much more efficient to obtain an approximation of the radar sensor position and attitude before determining a more precise position and attitude, since an estimate can indicate which points in the radar point cloud are likely to be the starting points for the calculation.
[0184] (References) SHCen and P. Newman, "Precise Ego-Motion Estimation with Millimeter-Wave Radar Under Diverse and Challenging Conditions," 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 6045-6052, doi:10.1109 / ICRA.2018.8460687.
[0185] The subject matter of the references is incorporated herein by reference in its entirety.
[0186] (Notes) While preferred embodiments have been shown and described, it should be understood by those skilled in the art that various changes and modifications can be made therein without departing from the scope of the invention as defined in the appended claims and as set forth above.
[0187] At least some of the exemplary embodiments described herein may be partially or entirely configured using dedicated hardware. Terms such as "component," "module," or "unit" as used herein may include, but are not limited to, hardware devices, such as circuits, field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs), in the form of discrete or integrated components, that perform specific tasks or provide related functionality. In some embodiments, the described elements may be configured to reside on a tangible, persistent, addressable storage medium and configured to execute on one or more processors. In some embodiments, these functional elements may include, by way of example, components such as software components, object-oriented software components, class components, task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. Although the exemplary embodiments are described with reference to components, modules, and units discussed herein, such functional elements may be combined into fewer elements or separated into additional elements. It will be understood that various combinations of optional features have been described herein, and that the described features may be combined in any suitable combination. In particular, features of any one exemplary embodiment may be combined as appropriate with features of any other embodiment, as long as they are not mutually inconsistent. Throughout this specification, the terms "comprising" or "comprises" mean the inclusion of the particular component, but not the exclusion of the presence of other components.
[0188] Attention is directed to all articles and documents related to this application which have been filed contemporaneously or prior to this application and which are open to public inspection together with this application, the contents of all such articles and documents being incorporated herein by reference.
[0189] All features disclosed in this specification (including the accompanying claims, abstract and drawings), and / or all steps of any method or process so disclosed, may be combined in any combination, unless at least some of such features and / or steps are mutually inconsistent.
[0190] Each feature disclosed in this specification (including the accompanying claims, abstract and drawings), unless otherwise stated, may be replaced by alternative features serving the same, equivalent or similar purpose. Thus, unless otherwise stated, each disclosed feature is only an example of a generic series of equivalent or similar features.
[0191] The invention is not limited to the details of the foregoing embodiments, but extends to any novel feature, or any novel combination of such features, disclosed in this specification (including the accompanying claims, abstract and drawings), or to any novel or novel combination of any method or process steps so disclosed.
Claims
1. 1. A computer-implemented method for locating a radar sensor, comprising: obtaining a first radar scan of a first environment of the radar sensor, the first radar scan including a set of power range spectra including a first power range spectrum; extracting a first landmark set from the first radar scan, the first landmark set including a first landmark defined by range and azimuth angle; calculating a first set of descriptors for each of the first set of landmarks, the first set of landmarks including a first descriptor defining the first landmark by its respective relative range and azimuth angle with respect to one or more landmarks included in the first set of landmarks; accessing one or more landmark reference sets for each environment and computing a descriptor reference set for each of said landmark reference sets; matching said first set of descriptors with a corresponding first reference set of descriptors; determining a first location of the radar sensor using a first result of the matching; Including, The method comprises: The method further comprises projecting the first descriptor into a first projected descriptor, the first descriptor having a dimension N and the first projected descriptor having a dimension M, where M ≠ N; The method, wherein matching the first descriptor set with the corresponding first descriptor reference set comprises matching a first projection descriptor set including the first projection descriptor with the corresponding first descriptor reference set of the plurality of descriptor sets.
2. The method of claim 1 , wherein the dimension N is a matrix dimension having a number of rows corresponding to the number of points in a scan and a number of columns corresponding to the size of a point descriptor.
3. The step of projecting the first descriptor into a first projected descriptor includes: obtaining a vector of the mean values of each element of said first descriptor; - obtaining a shifted first descriptor by subtracting the mean value from each element of said first descriptor; - calculating a covariance matrix of the shifted first descriptor; generating eigenvectors of the covariance matrix; ranking the eigenvalues from the eigenvector in descending order; selecting a subset of maximum eigenvalues as the first projection descriptor; The method of claim 2 , comprising:
4. The method of claim 3 , wherein the subset comprises M eigenvalues.
5. The method of claim 4 , wherein M is 1 and the selected eigenvalue is the largest eigenvalue.
6. obtaining a second radar scan of the first environment of the radar sensor; extracting a second set of landmarks from the second radar scan, the second set including the first landmark; calculating a second set of descriptors for each of the second set of landmarks, the second set of descriptors including the first descriptor; matching said second set of descriptors with a corresponding second reference set of descriptors; determining a second location of the radar sensor using a second result of the matching; and determining a motion of the radar sensor using the second position and the first position; The method of claim 1 , comprising:
7. The method according to any one of claims 1 to 6, wherein the step of matching the first descriptor set with the corresponding first descriptor reference set comprises the step of projecting the first descriptors into a first projected descriptor.
8. The step of matching the first descriptor set with the corresponding first descriptor reference set includes projecting the first descriptor into a first unique projected descriptor, the first descriptor having a dimension n>1, and the first projected unique descriptor having a dimension n p The method according to any one of claims 1 to 7, wherein: = 1.
9. 9. The method of claim 8, wherein matching the first descriptor set with the corresponding first descriptor reference set comprises comparing a first unique projection descriptor set including the first unique projection descriptor with the corresponding first unique projection descriptor reference set.
10. 10. The method of claim 9, wherein comparing the first-eigenprojection descriptor set including the first-eigenprojection descriptor with a corresponding first-eigenprojection descriptor reference set comprises identifying M closest eigenprojection descriptors in the corresponding first-eigenprojection descriptor reference set.
11. 11. The method according to claim 1, wherein the step of matching the first descriptor set with the corresponding first descriptor reference set comprises the steps of identifying M closest descriptors in the corresponding first descriptor reference set and finding a single closest descriptor from the M closest descriptors.
12. The method comprises: summing first absolute differences between the first set of descriptors and a first reference set of descriptors and setting a threshold absolute difference as the summed first absolute differences; summing second absolute differences between the first set of descriptors and a second reference set of descriptors, the second absolute differences in the sum being at most the threshold absolute difference; if the second absolute difference in the sum exceeds the threshold absolute difference, stopping the sum of the second absolute differences and starting a sum of a third absolute difference between the first set of descriptors and a third reference set of descriptors; if the summed second absolute difference does not exceed the threshold absolute difference, resetting the threshold absolute difference to be the summed second absolute difference; The method of claim 11 , comprising:
13. The method according to any one of claims 1 to 12, wherein M<N or M>N.
14. A method according to any preceding claim, wherein the step of projecting the first descriptor onto the first projected descriptor comprises the step of interpolating its elements.
15. The method according to any one of claims 1 to 14, wherein the step of projecting the first descriptor onto the first projected descriptor comprises the step of averaging or dropping its elements.
16. 16. A method according to any preceding claim, comprising the steps of calculating one or more values in response to a request, storing the calculated one or more values, and returning the stored one or more values or one or more values derived therefrom in response to a subsequent request.
17. storing said descriptor reference set; 17. The method according to claim 1, wherein the step of matching the first descriptor set with the corresponding first descriptor reference set comprises the step of matching the first descriptor set with the corresponding first descriptor set of stored descriptor reference sets.
18. 18. The method of claim 1, wherein the step of computing the first set of descriptors comprising the first descriptor for the first set of landmarks uses a set of lookup tables comprising a first lookup table.
19. 20. The method of claim 18, comprising generating the first lookup table at compile time and using the first lookup table at run time.
20. 20. The method of claim 18, comprising generating the first lookup table at run-time during a first computation of a given value of the first lookup table.
21. A method according to any one of claims 1 to 20, comprising the step of calculating two or more values simultaneously and / or performing correlated calculations using two or more values.
22. 22. The method of any one of claims 1 to 21, wherein computing the first descriptor set including the first descriptor for the first landmark set comprises processing the first landmark set in parallel.
23. 23. The method of any one of claims 1 to 22, wherein computing the first descriptor set including the first descriptor for the first set of landmarks comprises triangulating the first landmark with respect to each node and landmark in the first set of landmarks.
24. representing the first set of landmarks as a first signature; expressing the landmark reference set as respective reference signatures; correlating the first signature with a reference signature to approximate the first position of the radar sensor; The method according to any one of claims 1 to 23, comprising:
25. 25. The method of claim 24, wherein accessing the landmark reference set comprises selectively accessing the landmark reference set represented by the reference signature.
26. A method according to any preceding claim, wherein a land or water vehicle comprises the radar sensor.
27. 1. A computer-implemented method for controlling a land or water vehicle including a radar sensor, comprising: Locating a radar sensor according to any one of claims 1 to 26; controlling the land or water vehicle using the first position; A method comprising:
28. A computer comprising a processor and a memory configured to perform the method of any one of claims 1 to 27, a computer program comprising instructions which, when executed by a computer comprising a processor and a memory, cause the computer to perform the method of any one of claims 1 to 27, or a non-transitory computer readable storage medium comprising instructions which, when executed by a computer comprising a processor and a memory, cause the computer to perform the method of any one of claims 1 to 27.
29. 30. A land or water vehicle comprising a radar sensor and a computer according to claim 28.
Citation Information
Patent Citations
Multidimensional data visualization apparatus, method and program
JP2013161226A
Position estimation method, position estimation device, and position estimation program
JP2017003363A
