Vehicle detection and classification based on vehicle-infrastructure interaction via existing communication cables

Abstract

We disclose a vehicle-infrastructure interaction system and method using a distributed optical fiber sensing (DFOS) system that operates using pre-deployed optical fiber communication cables buried alongside or near highways / roads. It provides a continuous stream of information about vehicle traffic 24 / 7 at multiple locations, requiring only a single optical sensor cable to detect / monitor multiple target locations and multiple lanes. This single optical sensor cable measures multiple relevant vehicle information (multi-parameters), such as vehicle speed, wheelbase, number of axles, and tire pressure. This information can be advantageously used to obtain secondary information, such as vehicle weight, as well as overall information about the vehicle fleet, such as traffic congestion and freight volume. Unlike simple traffic counts, our approach can provide counts grouped by vehicle type and load weight. The high temporal sampling rate of distributed acoustic sensing and a dedicated peak detection algorithm for reliably extracting timing information provide accurate measurements.

Description

[Technical Field]

[0001] This application relates generally to distributed fiber optic sensing (DFOS) systems, methods, structures, and related technology. More particularly, this application relates to DFOS systems, methods, and structures that detect and measure vehicle-infrastructure interactions over existing communication cables. [Background technology]

[0002] The movement of people and goods is facilitated by vehicles traveling on highways and roads. (1) Information about individual vehicles, such as vehicle type (wheelbase, number of axles), speed, tire condition, and road weight, and (2) information about vehicle aggregations during periods of traffic congestion and traffic volume, reveals the flow of goods and services and provides objective indicators of economic activity in local geographic areas of interest.

[0003] Automated collection of such information in real time is useful for road planners and managers. Such information is often measured individually using specialized sensors, such as cameras, pneumatic road tubes, piezoelectric sensors, magnetic sensors, and fiber Bragg grating (FBG) fiber sensors. However, these sensor solutions have many limitations, such as point sensors scattered along highways and roads. These limitations include limited availability of data transmission and power supply only; the inability to distinguish vehicles traveling close to each other under dense or heavy traffic conditions; the multi-lane nature of highways; sensitivity to lighting conditions and weather conditions such as rain and fog; additional installation costs for privacy protection; the risk of damaging the road; and the use of dedicated fiber cables. Traditional distance fiber sensing and waterfall-based approaches can only provide coarse-grained information, such as traffic volume and average speed over a certain period of time, without knowing precise parameters such as vehicle type and instantaneous speed. They also require sophisticated computer vision and machine learning algorithms, which require expensive hardware and high latency. Summary of the Invention

[0004] Aspects of the present disclosure provide an advancement in the art with respect to vehicle-infrastructure interaction systems and methods employing a distributed fiber optic sensing (DFOS) system that operates on pre-deployed fiber optic communication cables buried alongside / close to highways / roadways.

[0005] In contrast to the prior art, the systems and methods according to aspects of the present disclosure a) provide a continuous 24 / 7 information stream about vehicle traffic at multiple locations along a roadway, requiring only a single optical sensor cable to detect / monitor multiple target locations and multiple lanes. This single optical sensor cable measures multiple relevant vehicle information (multi-parameters), such as vehicle speed, wheelbase, number of axles, and tire pressure. This information can be advantageously used to obtain secondary information, such as vehicle weight, as well as overall fleet information, such as traffic congestion and freight volume. Unlike simple traffic counts, the inventive approach can provide counts grouped by vehicle type and load weight. Finally, the systems and methods according to aspects of the present disclosure provide accurate measurements, enabled by the high temporal sampling rate of distributed acoustic sensing and specialized peak detection algorithms for reliably extracting timing information. [Brief explanation of the drawings]

[0006] [Figure 1(A)] FIG. 1 is a schematic diagram illustrating an exemplary prior art uncoded DFOS system. [Figure 1(B)] FIG. 1 is a schematic diagram illustrating an exemplary prior art coded DFOS system.

[0007] [Figure 2] FIG. 1 is a schematic block diagram illustrating an exemplary sensing setup in which a vehicle having multiple axles passes a pair of target locations along a roadway and multiple parameters related to the vehicle are sensed via DFOS, according to aspects of the present disclosure.

[0008] [Figure 3] 1 is a schematic block diagram illustrating exemplary vehicle measurements based on DAS waveform signals of a two-axle vehicle passing a pair of target locations along a roadway. According to aspects of the present disclosure, the location and number of self-repeating patterns, the gaps between patterns, and the magnitude and frequency of the acoustic signal convey information about the vehicle type, tire condition, and running weight.

[0009] [Figure 4] 1A-1C are schematic diagrams illustrating example designs / layouts for special scenarios such as high density and / or multi-lane traffic, according to aspects of the present disclosure.

[0010] [Figure 5] FIG. 1 is a schematic diagram illustrating an example operational pipeline for peak detection-based vehicle information estimation according to aspects of the present disclosure.

[0011] [Figure 6(A)] 10A-10C illustrate a comparison of raw waveform data before and after a bandpass filter, in accordance with aspects of the present disclosure. [Figure 6(B)] FIG. 10 illustrates estimating the number of axles by calculating the cumulative magnitude of peaks and counting the number of consecutively increasing intervals, according to aspects of the present disclosure.

[0012] [Figure 7] FIG. 1 is a schematic block / flow diagram illustrating prediction of multi-parameters of a vehicle using a supervised model trained on multivariate labels, according to aspects of the present disclosure.

[0013] [Figure 8] 10 illustrates the results of using a peak-based method for wheelbase and velocity estimation according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following is merely illustrative of the principles of the present disclosure, and it will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the present disclosure and are included within its spirit and scope.

[0015] Furthermore, all examples and conditional language set forth herein are intended to be for educational purposes only to aid the reader in understanding the concepts contributed by the inventors to further the principles and techniques of the present disclosure, and should not be construed as being limited to such specifically recited examples and conditions.

[0016] Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, such equivalents are intended to include both currently known equivalents as well as equivalents developed in the future, i.e., elements developed that perform the same function, regardless of structure.

[0017] Thus, for example, it will be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure.

[0018] Unless otherwise specified herein, the figures comprising the drawings are not drawn to scale.

[0019] As some additional background, note that a distributed fiber optic sensing system interconnects an optoelectronic integrator to an optical fiber (or cable), transforming the fiber into an array of sensors distributed along the fiber. In effect, the fiber becomes the sensor, and the interrogator generates / injects laser light energy into the fiber to sense / detect events along the fiber.

[0020] As those skilled in the art will understand and appreciate, DFOS technology can be deployed to continuously monitor vehicle movement, human traffic, drilling activity, seismic activity, temperature, structural integrity, liquid and gas leaks, and many other conditions and activities. It is used worldwide to monitor power plants, communication networks, railroads, roads, bridges, borders, critical infrastructure, onshore and offshore power lines and pipelines, and downhole applications in oil, gas, and enhanced geothermal power generation. Advantageously, distributed fiber optic sensing is not constrained by line of sight or remote power access and, depending on the system configuration, can be deployed over continuous lengths of more than 30 miles, with sensing / detection possible at every point along that length. Therefore, the cost per sensing point over long distances is typically incomparable to competing technologies.

[0021] Distributed fiber optic sensing measures changes in the "backscatter" of light that occurs within an optical sensing fiber when the fiber encounters environmental changes, including vibration, strain, or temperature change events. As previously mentioned, the optical sensing fiber acts as a sensor along its entire length, providing real-time information about the physical and environmental surroundings and the integrity and security of the fiber. Furthermore, distributed fiber optic sensing data pinpoints the precise location of events and conditions occurring on or near the sensing fiber.

[0022] A schematic diagram illustrating the generalized arrangement and operation of a distributed optical fiber sensing system that may advantageously include artificial intelligence / machine learning (AI / ML) analysis is illustratively shown in Figure 1(A). Referring to Figure 1(A), it can be seen that the optical sensing fiber is connected to an interrogator. Although not shown in detail, the interrogator can include a coded DFOS system that can employ a coherent receiver arrangement known in the art, such as that shown in Figure 1(B).

[0023] As is well known, a modern interrogator is a system that generates an input signal into an optical sensing fiber and detects and analyzes the reflected / backscattered signal that is then received. The received signal is analyzed and an output is generated that is indicative of the environmental conditions encountered along the fiber. The received backscattered signal may be due to reflections within the fiber, such as Raman backscattering, Rayleigh backscattering, or Brillouin backscattering.

[0024] As will be appreciated, modern DFOS systems include an interrogator that periodically generates optical pulses (or any coded signal) and launches them into an optical sensing fiber, which transmits the optical pulse signal along the optical fiber.

[0025] At certain locations along the fiber, a small portion of the signal is backscattered / reflected back to the interrogator, where it is received. The backscattered / reflected signal carries information that the interrogator uses to detect, such as changes in power level that indicate mechanical vibrations.

[0026] The received backscattered signal is converted to the electrical domain and processed within the interrogator. Based on the time of pulse incidence and the time the received signal is detected, the interrogator can determine from which location along the optical sensing fiber the received signal returned, thereby sensing activity at each location along the optical sensing fiber. Classification methods may also be used to detect and locate events or other environmental conditions, including acoustic and / or vibration and / or heat, along the optical sensing fiber.

[0027] As will be further shown and described, systems, methods, and structures according to aspects of the present invention incorporate the linear architecture of distributed optical fiber sensing (DFOS), thus inheriting the advantages of conventional distributed optical fiber sensing. At the same time, they utilize vehicle vibration / acoustic patterns that occur in close proximity to the optical sensor fiber that is part of the DFOS. As a result, not only can they enjoy the high-precision measurement advantages of conventional point sensors, but they can also be used in combination with wireless communication technologies such as non-acoustic sensors and cameras for sensor fusion, enabling other advanced applications (e.g., vehicle license plate recognition and real-time driver notification).

[0028] As will be understood and appreciated by those skilled in the art, potential applications of the systems, methods, and structures of the present invention include vehicle fault diagnosis and early warning, gasoline cost and road damage estimation, accurate toll determination and estimation, economic data generation, speeding detection and charging systems, and the like.

[0029] The systems, methods, and structures of the present invention detect, collect, and analyze vehicle-to-infrastructure (V2I) interactions. Advantageously, the systems, methods, and structures of the present invention improve the safety and productivity of highways and roads while providing a DOFS-based system for smart transportation and smart roads.

[0030] As a further advantage, the systems, methods, and structures of the present invention utilize existing cable architectures, eliminating the additional cost of deploying dedicated sensors, and instead utilize acoustic signals generated by vehicles passing auxiliary target locations on or along roadways, including, but not limited to, speed bumps, bridge decks, manholes, potholes, or acoustic warning patterns (SNAPs). Because detection is based on peak detection or self-similarity between patterns / signatures generated by the same vehicle, the systems, methods, and structures of the present disclosure are more robust to heterogeneous factors caused by different geographic locations and soil types, as well as various types of auxiliary target locations. Dedicated digital signal processing (DAS) and machine learning (ML) algorithms for data processing are low-cost, lightweight, and run in real time.

[0031] FIG. 2 is a schematic block diagram illustrating an exemplary sensing setup in which a vehicle having multiple axles passes a pair of target locations along a roadway and multiple parameters related to the vehicle are sensed via DFOS, according to an embodiment of the present disclosure.

[0032] As shown in this diagram, a sensing layer is installed over the existing deployed optical fiber network. The Distributed Optical Fiber Sensing System (DFOS) is a distributed acoustic sensing (DAS) integrator installed in the control station / central station, which collects data from the entire optical fiber sensor cable route. The DFOS system is connected to the field optical sensor fiber and provides real-time sensing capabilities at multiple locations using location division multiplexing (LDM). The fiber can be dark fiber or service provider-operated fiber.

[0033] To measure multiple vehicle parameters with high accuracy, a pair of targeted locations is required. Such locations are readily available on roadways, such as speed bumps, bridge decks, or acoustic warning patterns (SNAP). They can also be easily deployed at specific locations or times of interest, for example, in some facilities (parking lots, gates, stadiums, etc.) during special music or sporting events.

[0034] As is known, DFOS measures displacement along a fiber cable and provides a phase waveform change as an output. When a vehicle passes a targeted location, it generates a self-similar acoustic pattern that can be detected. The mechanism is shown in Figure 3 for a two-axle vehicle.

[0035] 3 is a schematic block diagram illustrating an exemplary vehicle measurement based on DAS waveform signals in which a two-axle vehicle passes a pair of target locations along a roadway. According to aspects of the present disclosure, the location and number of self-repeating patterns, the gaps between patterns, and the magnitude and frequency of the acoustic signal convey information about the vehicle type, tire condition, and running weight.

[0036] Algorithm operations for measuring basic vehicle information include peak detection, delayed correlation calculation, moving window, spatial clustering, filtering, local averaging, and power spectral density calculation. Vehicle classification can be accurately determined based on wheelbase and number of axles (e.g., sedan, truck) regardless of load weight. Vehicle count and speed indicate short-term or seasonal traffic volume and congestion. Vehicles with abnormal wheelbases and individual vehicles exceeding speed limits can also be detected.

[0037] It should be noted that secondary information such as tire pressure information and road weight requires comprehensive consideration of multiple relevant factors (instantaneous speed, axle weight, and integrated inference of vibration patterns), which is not possible with traditional point sensor-based solutions that can only provide (or focus on) some of the factors.

[0038] The inventive systems, methods, and structures according to aspects of the present disclosure advantageously handle high density and multi-lane scenarios by utilizing a set of multiple target locations to provide more accurate estimations. These special scenarios are illustrated in Figure 4, which is a schematic diagram illustrating an example design / layout for special scenarios such as high density and / or multi-lane traffic according to aspects of the present disclosure.

[0039] More specifically, (1) when there are multiple pairs of targeted locations within the same lane, averaging over multiple pairs can improve the accuracy of the estimation results (as shown in B2-FIG. 8, which shows the results using a peak-based method for wheelbase and speed estimation according to an embodiment of the present disclosure) and can also provide a distribution of parameter estimates; (2) the order of peaks appearing at these targeted locations can be used to infer the vehicle's direction of travel; and (3) vehicles passing through multiple lanes can be naturally identified by pairs of targeted locations in different lanes. Note that the targets must be embedded in the road, not on the side of the road (e.g., utility poles are not acceptable).

[0040] The aforementioned fiber sensing setup allows for the acquisition of waveform signals from multiple fiber sensors for vehicle measurement. Here, the adopted algorithms are described in detail.

[0041] As can be seen, one of the key challenges is extracting accurate timing information of events from raw waveform data. One estimation method is based on calculating the distance between the peaks of events in the original waveform space or in the transformed space. The pipeline is shown in Figure 5. Figure 5 is a schematic diagram illustrating an example operational pipeline for vehicle information estimation based on peak detection according to an aspect of the present disclosure.

[0042] Each raw waveform data is first pre-processed by a band-pass filter with a pass frequency of 50 Hz to 200 Hz.

[0043] 6(A) and 6(B) illustrate the following: Fig. 6(A) is a comparison of raw waveform data before and after a bandpass filter, according to an embodiment of the present disclosure; Fig. 6(B) is an estimation of the number of axles by calculating the cumulative magnitude of peaks and counting the number of consecutively increasing intervals, according to an embodiment of the present disclosure;

[0044] As shown in Figure 6(A), the amplitude is weaker, but the waveform noise is reduced and the peaks are denser. This is advantageous for peak detection because the detector is less likely to miss a peak. The filtered signal is then normalized by the mean and standard deviation of all collected data, ensuring that different signals have a similar range of values, making the threshold universally applicable.

[0045] Estimate the number of axles.

[0046] As shown in Figure 6(B), the number of axles is estimated by calculating the cumulative amplitude of the peaks and counting the number of consecutively increasing intervals. For each waveform signal, peaks are first detected by simply comparing neighboring values ​​that exceed an absolute amplitude threshold. Orange crosses indicate all detected peaks and their absolute amplitudes. Next, the cumulative amplitude of all points is calculated from the original amplitude values. The amplitudes of all non-peak points are zero. The cumulative amplitude is used as a feature for DBSCAN clustering. Because the neighborhood radius threshold is very small and the minimum number of points is very large, this clustering algorithm can accurately detect consecutive zero intervals and consecutively increasing intervals. The total number of consecutively increasing intervals is the number of axles obtained from this waveform. "Group 1" and "Group 2" in Figure 6(B) show the consecutively increasing intervals detected in this waveform signal. The cumulative amplitude feature can be downsampled before clustering to speed up the calculation. The final estimated number of axles is voted on by multiple sensors.

[0047] Find the representative peak.

[0048] After obtaining the number of axles, i.e., the number of groups of peaks in the waveform data, the next step is to determine the timestamp (x-coordinate) of each group. One intuitive approach would be to average the x-coordinates of all peaks belonging to a group, but this requires all peaks to fall within the expected range, which is difficult when the waveform is noisy. Therefore, the peak with the largest amplitude is considered the representative peak of this group. This peak is not necessarily located in the center, but is likely to be within the "hill" range. To achieve this, we first calculate the z-score of each peak's x-coordinate and filter out outlier peaks by removing peaks that exceed a threshold (i.e., subtract the mean and then divide by the standard deviation). Next, we apply the K-means algorithm, with the estimated number of axles as a parameter, to cluster the peaks using their x-coordinates. This allows us to find the representative peak, the highest peak, for each cluster.

[0049] Calculate vehicle measurements.

[0050] Before using a representative peak to calculate vehicle measurements, we first check whether the distance between adjacent peaks is within a reasonable range; if not, this waveform data is discarded.

[0051] Wheelbase. The wheelbase can then be calculated by calculating the ratio of the wheelbase to the distance between the target positions. Specifically, as shown in Figure 2, assuming the distance between adjacent positions is d meters, the wheelbase is

number

number

[0052] Here, T1 is the time difference between the left and right peaks, corresponding to the front and rear wheels passing through position 1-a or position 1-b. And T2 is the time difference between the two left peaks or two right peaks, corresponding to the front or rear wheel passing through two locations. Because there are two estimates for T1 and T2, the ratio can be estimated in four combinations. Take the average of the four ratios and multiply by d to get the wheelbase in meters.

[0053] Speed. The instantaneous running speed can be estimated using the following formula:

number

[0054] In addition,

number

[0055] To demonstrate the effectiveness of this method, we conducted 16 test runs using a two-axle vehicle. The results are shown in Table 1. "Gt speed" is the speed read by the driver from the vehicle's speedometer, and "estimated speed" and "estimated wheelbase" are the speed and wheelbase estimated using the above method. The vehicle's ground truth wheelbase is 2.64 m. The estimation errors for speed and wheelbase were 0.15 ± 0.98 mph and 0.031 ± 0.19 m, respectively, demonstrating that the estimation is accurate and stable. Furthermore, setting multiple target locations can yield more accurate results.

[0056] Peak-based methods allow for accurate estimation of wheelbase and speed, and the location of peaks is also useful for real-time vehicle detection as a first step. However, they lack robustness when the density of peaks is low or when the magnitude of the peaks changes due to factors such as weather conditions and the type of auxiliary target location.

[0057] An alternative solution is to look for repeating patterns in the signal, assuming that the peak groups are similar. Specifically, T1 is estimated by calculating the autocorrelation and finding the corresponding lag of the second largest peak. For vehicles with more than two axles, such as a three-axle vehicle, the lags of the second and third largest peaks in the correlation plot correspond to the two wheelbases. To estimate T2, we simply calculate the cross-correlation between the two signals. Then, we can estimate the wheelbase and speed using equations (1) and (2).

[0058] Another solution is to use a sliding window to search for the closest pattern from all subsequences. The characteristic vibration patterns of the same vehicle passing through the secondary target location show high reproducibility. When this vibration occurs, the minimum distance, which measures the similarity between patterns, becomes small. On the other hand, background noise patterns are not reproducible, and therefore the minimum distance becomes large. Therefore, the proposed peak detection algorithm can also work in this transformed minimum distance space. Even if the original pattern contains multiple peaks, there will be only one peak in this transformed minimum distance space. Therefore, the peak detection procedure is more robust in this space.

[0059] The type of vehicle (e.g., sedan, SUV, truck, etc.) can be directly inferred from the wheelbase and number of axles.

[0060] Additional vehicle parameters predicted from vibration patterns.

[0061] Given the location of the peak, the signal segment containing the vibration pattern can be extracted. The same event can be detected at multiple locations along the cable, making multi-channel measurements. By calculating the power spectral density of the vibration pattern, magnitude and frequency information can be obtained.

[0062] Additionally, supervised machine learning models such as regression and classification can be trained to predict vehicle parameters. Given ground truth labels for cargo running weight and tire pressure, models can be trained to predict from raw waveforms or power spectra. This has at least two important aspects:

[0063] First, instead of training a model for each label separately, we need to use a multivariate label model to consider the interaction between multiple labels and predict them jointly (e.g., axle weight and tire pressure).

[0064] Second, the interaction also depends on the vehicle type and driving speed, which are included as covariates in the prediction model.

[0065]

[0013] Figure 1 is a schematic block / flow diagram illustrating multi-parameter prediction of a vehicle using a supervised model trained with multivariate labels and the overall architecture of the prediction model, according to an embodiment of the present disclosure. The outputs are axle weight and tire pressure, which are jointly predicted based on two sets of inputs: the first set is the raw waveform or power spectrum of the acoustic signal, which contains information about magnitude and frequency; and the second set of inputs includes vehicle type and speed, which have already been measured and serve as conditioning factors.

[0066] Those skilled in the art will appreciate the present fiber optic sensing and machine learning based solution for accurately measuring vehicle parameters based on vehicle-infrastructure interactions. As previously discussed, the present sensing system includes 1) a DAS interrogator, 2) communication cables laid or buried along the road, and 3) a pair of secondary targeted locations (such as speed bumps, bridge decks, or acoustic warning patterns (SNAPs)) that generate vehicle vibrations.

[0067] The software of the present invention includes algorithms for data pre-processing, estimation, and prediction of multiple vehicle parameters. Sensing functions operate on both individual vehicles and fleets of vehicles. The inventive systems, methods, and structures enable a variety of smart road and smart transportation applications that improve the safety and efficiency of highways and roads. [Table 1]

[0068] While the present disclosure has been presented above using some specific examples, those skilled in the art will recognize that the present teachings are not so limited. Accordingly, the present disclosure should be limited only by the scope of the claims appended hereto.

Claims

1. 1. A method for detecting and classifying vehicles based on vehicle-infrastructure interactions, comprising: operating a distributed fiber optic sensing (DFOS) system having a pair of targeted sensing locations along a roadway; determining from the DFOS data received from the operation the location and number of at least self-repeating patterns in the DFOS data; and determining information about a vehicle traveling on the roadway from the DFOS data received from the operation and associated with the location and self-repeating pattern, the information including one or more of vehicle type, tire condition, and running weight of the vehicle.

2. The method of claim 1 , further comprising identifying gaps between the location and number of the self-repeating patterns in the DFOS data.

3. The method of claim 2 , further comprising determining the magnitude and frequency of acoustic signals from the DFOS data from the locations and number of self-repeating patterns in the DFOS data.

4. The method of claim 3 further comprising determining an instantaneous velocity of the vehicle from the DFOS data from the location and number of self-repeating patterns in the DFOS data.

5. The method of claim 4 further comprising a plurality of pairs of targeted sensing locations along the roadway.

6. The method of claim 5 , wherein the pairs of targeted sensing locations along the roadway are located in at least two travel lanes of the roadway.

7. 7. The method of claim 6, wherein the at least two lanes of travel of the roadway carry vehicles in different directions.

8. 3. The method of claim 2, comprising determining an estimate of the number of axles from the DFOS data from the cumulative magnitude of peaks in the DFOS data and the number of successively increasing intervals in the DFOS data.

9. 9. The method of claim 8, further comprising applying the DFOS data to a trained neural network predictive model to determine axle weights and tire pressures of vehicles traveling on the roadway.

10. 10. The method of claim 9, wherein the DFOS data includes raw waveform data and power spectrum data, and a second input data set includes vehicle type and vehicle speed, and the raw waveform data and power spectrum data are used along with the vehicle type and vehicle speed as inputs to the predictive model.

Images