Processing LIDAR sensor data

Abstract

A method of processing data from a LIDAR sensor 10 comprises: acquiring LIDAR sensor data, the data comprising a range for each of a plurality of points along a scan line 16a,16b,16c; converting each

Description

The present invention relates to a method of processing LIDAR sensor data, in particular for use in processing data from LIDAR sensors mounted to detect vehicles travelling along a road. 5 BACKGROUND TO THE INVENTION LIDAR (“light detection and ranging”) sensors can be used to accurately determine the range of a target, i.e. the distance between the sensor and the target. In particular, these sensors are often used mounted on for example a gantry above a road, to detect vehicles as they travel along the road. 10 This arrangement can be used for example on a toll road. The LIDAR scanners are mounted to detect vehicles, and in particular distinguish for example, cars, vans, lorries, buses, which may be charged different tolls. The scanners can also be used to accurately direct a camera which can then photograph a licence plate of each vehicle. A LIDAR sensor typically scans along a line, and outputs data as to the range of an 15 object from the sensor at each point along the line. The sensor may scan along multiple lines concurrently, for example three lines separated by a few degrees (typically around 12 degrees). It can be seen that by repeatedly scanning as a vehicle travels past, a 3D “point cloud” of the vehicle can be generated. This can be analysed to determine vehicle type, for example for toll charging purposes, and to direct a camera 20 to photograph the licence plate. One particular challenge in this vehicle detection I identification application is distinguishing vehicles with trailers from vehicles closely “tailgating”. Especially in slow-moving traffic, drivers can sometimes drive very close to the vehicle in front, with about the same spacing between vehicles as would be seen between a towing vehicle and 25 its trailer. One solution is to look for the tow hitch, but this is a fairly small object compared with a vehicle and therefore high resolution / fast scan rate LIDAR data is required to reliably detect it. Another problem with LIDAR sensors in this application is that they can potentially pick up rain, dust and other interference. This “noise” in the output can be removed with 30 known processes. However, again a high resolution signal is required to ensure that the desired information is not obscured by noise from heavy weather. LIDAR sensors with sufficiently high resolution I fast scan rates to detect these objects are readily available. However, with high resolution comes a high output data rate. Typically a single sensor has an output data rate of approximately 25 Mbps. There may be for example 15-20 sensors at a single toll site, resulting in a data rate of around 500 Mbps or even higher. It is costly to provide networking to transfer this amount of data continuously to be processed. Processing such a high data rate also requires significant computing resources, which cannot easily be provided on a gantry or at the roadside. There are known techniques of reducing I compressing data from LIDAR sensors. However, these are focused on processing 3D point cloud data. In other words, the data from the LIDAR sensors first has to be aggregated to create a 3D point cloud, and then the point cloud can be compressed. This does not therefore solve the problem of the high data rate from the sensor array. The data already has to be transmitted and processed to create the point cloud in the first place. Accordingly it is an object of the invention to provide a method of processing LIDAR sensor data, which can preferably take place on hardware mounted close to a sensor, to reduce its data rate for onward transmission and processing, whilst retaining information to allow for reliable vehicle discrimination. STATEMENT OF INVENTION According to the present invention, there is therefore provided a method of processing data from a LIDAR sensor, the method comprising: acquiring LIDAR sensor data from at least one LIDAR sensor positioned above, or above and to the side of, a road, the data comprising a range for each of a plurality of points along a scan line; converting each range to a height above the road; subdividing the scan line into segments, each segment having a plurality of data points and each data point being a height associated with a point along the scan line within the segment; for each segment: sorting the data points by height; comparing the height of the data point at a first predetermined quantile with a first predetermined threshold value, and if the height is lower than the first predetermined threshold value then determining the height associated with the segment to be the height of the data point at the first predetermined 5 quantile; otherwise, comparing the height of the data point at a second predetermined quantile with a second predetermined threshold value, and if the height is higher than the second predetermined threshold value then determining the height associated with the segment to be the height of the data 10 point at the second predetermined quantile; otherwise, determining the height associated with the segment to be a value between the height of the data point at a first predetermined quantile and the height of the data point at the second predetermined quantile; 15 outputting processed data, the processed data comprising the determined height associated with each segment along the scan line. The method of the invention results in significant reduction in the data rate of the processed data, compared to the data rate of the raw data from the sensor. The data is reduced by a factor according to the number of data points in each segment. For example, if there are 100 data points in each segment, then the data rate of the processed signal will be 100 times less than the data rate from the sensor. The method effectively reduces the resolution of the signal, so there will be (for example) 100 times fewer points at which a height is recorded. However, the data is reduced in a way that preserves the important information required to discriminate vehicle types, and in particular to detect important small objects such as tow hitches between vehicles and 25 trailers. Each segment is a series of adjacent points along the scan line. A key advantage of the inventive method is that the representative height chosen for each segment is likely to be a real height of an object above the ground (the road), as opposed to for example a “false” reading caused by heavy rain. 30 Compared with the prior art, the method operates on an individual scan line. It does not require a 3D point cloud to be generated first. The method can therefore be operated continuously on raw data streamed directly from sensors. The algorithm is computationally simple and can be implemented on low-cost hardware, positioned close to the sensors, even mounted in the same housing as a sensor on a gantry over a road. In an example, the first predetermined quantile is the 95th percentile, and the first predetermined threshold value is 150cm. Therefore, after sorting the data points in a segment, the data point at the 95th percentile (i.e. the data point which is higher than 95% of data points in the segment but lower than 5% of the data points in the segment) is selected. If the 95th percentile data point is below 150cm, then that 95th percentile height is taken as “the height” of the segment, i.e. that is the height which is associated with the segment in the output data. However, if the 95th percentile height is greater than 150cm then the height of the data point at the second predetermined quantile is considered. In this example, the second predetermined quantile is the 50th percentile, i.e. the median height in the segment. The second predetermined threshold value in this example is 300cm. If the median height is 300cm or more then it is this height, the median height, which is taken as “the height” of the segment in the compressed I reduced output data. If neither of those conditions are met (i.e., in this example, if the 95th percentile height is greater than 150cm but the median height is less than 300cm) then a value needs to be chosen which is between the 95th percentile height and the median height (more generally, between the height at the first predetermined quantile and the height at the second predetermined quantile). In embodiments, this may be done by selecting a data point from within the data points of the segment which is at a quantile between the first predetermined quantile and the second predetermined quantile. This may be done by iterating through the sorted measurements, from the measurement at the first predetermined quantile and working downwards in height order through the sorted heights, until a measurement is found which meets a threshold condition. The threshold condition may be that the height of the data point is below a threshold interpolated between the first predetermined threshold value and the second predetermined threshold value. Returning to the example where the first predetermined threshold value is 150cm and the second predetermined threshold value is 300cm, the 95th percentile height may be for example 250cm, and so the first condition is not met. The 50th percentile height may be for example 200cm, and so the second condition is not met either. A measurement then has to be selected by iterating through the sorted measurements downwards, i.e. starting with the next lowest height after the 95th percentile height. Each measurement is tested against a threshold associated with the quantile at which that measurement appears. The threshold is interpolated between the first and second predetermined thresholds. Once a threshold is met (the height is lower than the threshold), that height is selected as the representative height for that segment and the iterative process terminates. The height threshold at each step may be determined as: _ (t2 -q) — Z Z t f 1 Qi Q2 Where tq is the threshold associated with a particular quantile q, and t2 are the first and second predetermined thresholds, and q} and q2 are the first and second predetermined quantiles. For example, at the 80th percentile, q = 0.8, t0 8 = 200, so if the iterative process has continued down to the 80th percentile then the 80th percentile height will be selected if and only if it is lower than 200cm. Equally at each iteration, the quantile q at which the height h appears in the sorted list could be tested against a threshold: h — Q <Qi — _ Q2) h — h Continuing the example, a height of 225cm would need to appear at a quantile before 0.725 to be “accepted” as the representative height for the segment. In embodiments, a lookup table may be created for various values of tq. This will allow the iterative process to complete very quickly. In some embodiments the lookup table may to some extent be an approximation of the interpolation explained above. However, since there are a fixed number of data points per segment, the values of q for which tq needs to be calculated are known in advance and so all required values of tq could usually be calculated easily. The method of the invention may be applied to data from a LIDAR sensor mounted on a gantry above a roadway. The LIDAR sensor may be positioned to measure ranges on a scan line substantially across a traffic lane. The processed data output by the method may be used in order to distinguish vehicle types for the purposes of toll charging, for example. Preferably, the method of the invention is applied to data from multiple sensors concurrently. The processed data may then be transmitted for onward processing, for example in a data centre remote from the sensors. The method of the invention may additionally include filtering steps which may be 5 carried out on the raw data before it is reduced. The filtering steps aim to remove clearly anomalous data and reduce to some extent noise in the data produced by weather, for example. Such filtering to remove clearly anomalous data points is known in the art, but will never remove all noise completely. A key advantage of the inventive method is that when the data is reduced, values are chosen which are unlikely to be 10 affected by any remaining anomalous data points caused by rain, for example. BRIEF DESCRIPTION OF THE DRAWINGS For a better understanding of the present invention, and to show more clearly how it may be carried into effect, reference will now be made by way of example only to the accompanying drawings, in which: xt 15 Figure 1 is a perspective view of a LIDAR sensor mounted on a gantry over a roadway; CM Figure 2 is a schematic view from overhead of a roadway, showing scan lines associated with a LIDAR sensor mounted on a gantry over the roadway; and co Figure 3 shows how a threshold is interpolated between first and second thresholds in the method according to the invention. 20 DESCRIPTION OF PREFERRED EMBODIMENTS Referring to Figure 1, a LIDAR sensor 10 is mounted on a gantry 12 over a roadway 14. For clarity, only one LIDAR sensor 10 is shown. However, in many embodiments of the invention, for example for charging on a major toll road, there could be around fifteen to twenty, or even more, sensors, on a single toll site. There could be multiple 25 gantries, and gantries could also mount other equipment, for example cameras, which may work together with the LIDAR sensors to ensure that every vehicle is reliably detected, classified, and charged the correct toll. Each LIDAR sensor will typically have multiple scan lines. For example, the scan lines may be separated by around 12 degrees of altitude. These correspond to multiple lines 30 spaced apart along the roadway, with each scan line being substantially across the roadway, for example extending across at least one traffic lane. Example scan lines are indicated at 16a, 16b, 16c in Figure 1. The position of the LIDAR sensor 10 in Figure 1 is just one example. LIDAR sensors could be positioned in other lateral positions along the gantry, or at the sides of the road or in other locations. The sensors may be positioned so that the scan lines are disposed in other places along the road, and do not necessarily have to extend laterally across a lane along the road from the sensor as shown in Figure 1. For example, a sensor can be placed with at least one of the scan lines being substantially directly below the sensor. It will be appreciated that data will be aggregated from multiple scans to detect and identify vehicles as they drive along the lane. In the overhead view of Figure 2, a vehicle 18 is also shown. As the vehicle drives past scan lines 16a, 16b, 16c, sufficient data is collected to generate a 3D point cloud of the vehicle, from which the vehicle can be classified. The method of the invention can be applied continuously on a per-scan-line basis. It can be applied to multiple sensors. Although most LIDAR sensors in this application output multiple scan lines, it does not matter for these purposes whether the scan lines 16a, 16b, 16c are from one sensor or three sensors. Considering the data from a single scan line, typically a LIDAR sensor could scan across the line around 100 times per second. There could be for example several thousand points along the scan line, with the range measured at each point. It can be seen that a large amount of data will be generated. The LIDAR sensor outputs a measured range at each point along the scan line. Firstly, this needs to be converted to a height (above the ground). This may be done in the following way: height = (measured_range * [x, y, z coefficients] * [rotation matrix]).Z + sensor_height where: - [x,y,z coefficients] are coefficients of each beam’s angle from a sensor to translate measured range from x,y,z coordinates with respect to the sensor’s reference frame; - [rotation matrix] rotates the 3D coordinates from the sensors reference frame to our reference frame where the Z axis becomes the vertical axis pointing upwards, i.e. perpendicular to the road surface; - the (...).Z is added to show that we taking the Z coordinate; - sensor_height is the height of the sensor above ground. This is added for the translation of the reference frame such that the Z axis datum is at road level. Accordingly, each scan along the scan line becomes a series of measured heights. There could be, for example, 10,000 measured heights along the scan line for each scan. The scan line is then divided into segments. Each segment has a fixed number of data points. For example, in this embodiment a segment may have 100 data points. This means that there will be 100 segments along the scan line (based on 10,000 data points on the scan line in the raw date). For each segment, the 100 measured heights are sorted into height order. Then, the 95th percentile (being the first predetermined quantile in this example) height is considered. If the 95th percentile height is below 150cm (the first predetermined threshold in this example) then the 95th percentile height is selected as the representative height for the whole of that segment. If the 95th percentile height is 150cm or more then the 50th percentile (the second predetermined quantile) height is considered. If the 50th percentile (median) height is above 300cm (the second predetermined threshold in this example) then the 50th percentile height is selected as the representative height for the whole of that segment. If neither of these conditions is met, i.e. the 95th percentile height is more than 150cm and the 50th percentile height is less than 300cm, then an iterative process is carried out, starting with the next lowest height from the 95th percentile. Each height is tested against a threshold according to the percentile at which it appears (or, alternatively, the percentile at which each height appears is tested against a threshold according to the measured height at that percentile). Figure 3 illustrates the interpolation process, according to the equations set out above. The solid line on the graph indicates the height thresholds associated with each percentile. The horizontal sections of the solid line indicate where the predetermined thresholds act as “caps” - in this example a value greater than the 95th percentile or less than the median will never be chosen. However, it can be seen that by setting the predetermined quantiles and thresholds, including to zero or maximum values in some embodiments, this can be varied. In an example, the 95th percentile height is 250cm (marked Xi on the graph). The 50th 5 percentile height is 200cm (marked X2). Therefore neither of the tests against predetermined thresholds is met and the interpolation process takes place. The broken line on the graph indicates the measured height data at each percentile. The iterative process starts at the next lowest height after the 95th percentile height, and continues until the measured height is lower than the threshold associated with the particular 10 percentile at which it appears. The arrows on the broken line in Figure 3 show how the iterative process repeatedly tests measured values against thresholds. Point I, where the broken line intersects the solid line, indicates where the threshold is first met and this is the height which will be selected as the single representative height for the segment. 15 The shape of the broken line on the graph depends on the distribution of the data within the segment, between the 95th percentile and the 50th percentile. It could be straight or curved or take an irregular path. However it will always run generally down and to the left on the graph, from an Xi point above and to the right, to an X2 point below and to the left of the solid line. In some embodiments, a faster interpolation algorithm may be produced by effectively plotting the Xi and X2 points, making the assumption that the broken line between those points is a straight line (or indeed another shape which approximates typical distributions), and determining the intersection with the solid line. This avoids an iterative process of testing multiple data points against thresholds, and will therefore 25 run faster. It may provide sufficiently good results in some embodiments. The data output from the method of the invention has been greatly reduced in volume. In this example, the data rate after applying the method of the invention is 100 times lower since in each segment 100 data points have been replaced by one representative data point. The output data is therefore of lower resolution than the input 30 data. However, the way this is done ensures that the information needed to reliably detect vehicle types is retained and common interference sources (e.g. rain) are rejected. The embodiments described above are provided by way of example only, and various changes and modifications will be apparent to persons skilled in the art without departing from the scope of the present invention as defined by the appended claims. 18 1224

Claims

1. A method of processing data from a LIDAR sensor, the method comprising:acquiring LIDAR sensor data from at least one LIDAR sensor positioned above, or above and to the side of, a road, the data comprising a range5 for each of a plurality of points along a scan line;converting each range to a height above the road;subdividing the scan line into segments, each segment having a plurality of data points and each data point being a height associated with a point along the scan line within the segment;10 for each segment:sorting the data points by height;, comparing the height of the data point at a first predetermined\Jquantile with a first predetermined threshold value, and if the height is lower than the first predetermined threshold value then15 determining the height associated with the segment to be theheight of the data point at the first predetermined quantile;00“ otherwise, comparing the height of the data point at a secondpredetermined quantile with a second predetermined threshold value, and if the height is higher than the second predetermined20 threshold value then determining the height associated with thesegment to be the height of the data point at the second predetermined quantile;otherwise, determining the height associated with the segment to be a value between the height of the data point at the first25 predetermined quantile and the height of the data point at thesecond predetermined quantile;outputting processed data, the processed data comprising a height associated with each segment along the scan line.

2. A method as claimed in claim 1, in which determining the height associated30 with the segment to be a value between the height of the data point at the firstpredetermined quantile and the height of the data point at the second predetermined quantile includes selecting from the data points of the segment a data point which is at a quantile between the first predetermined quantile and the second predetermined quantile.

3. A method as claimed in claim 2, in which selecting a data point from the data points of the segment includes iterating through the sorted data points, from the data point immediately below the first predetermined quantile and iterating downwards in height order until a data point is found which meets a threshold condition.

4. A method as claimed in claim 3, in which the threshold condition is that the height of the data point is below a threshold interpolated between the first predetermined threshold value and the second predetermined threshold value.

5. A method as claimed in claim 4, in which the threshold at each step tq is determined as:t„ =--1- Ciq qi~q2where tq is the threshold associated with a particular quantile q, tr and t2 are the first and second predetermined thresholds, and and q2 are the first and second predetermined quantiles.

6. A method as claimed in claim 4 or claim 5, in which a precalculated lookup table is provided, the precalculated lookup table containing thresholds tq associated with multiple values of q between the first predetermined quantile and the second predetermined quantile.

7. A non-transitory computer readable medium containing instructions which when executed on a processor cause the processor to carry out the method of any of the preceding claims.

8. A vehicle detection system comprising a LIDAR sensor directed to scan along a scan line disposed substantially across a traffic lane of a roadway, the LIDAR sensor outputting data to processing means configured to carry out the method according to any of claims 1 to 6, and the processing means outputting data to further processing means configured to detect and classify vehicles.

9. A vehicle detection system as claimed in claim 8, in which the LIDAR sensor and the processing means are mounted to a gantry over the roadway.

10. A vehicle detection system as claimed in claim 9, in which the further processing means are provided remote from the roadway.18 1224

Images