Vehicle detection and classification based on vehicle-infrastructure interaction via existing communication cables.

Abstract

We disclose a vehicle-infrastructure interaction system and method using a distributed optical fiber sensing (DFOS) system that operates using pre-deployed optical fiber communication cables buried alongside or near highways / roads. It provides a continuous stream of information about vehicle traffic 24 / 7 at multiple locations, requiring only a single optical sensor cable to detect / monitor multiple target locations and multiple lanes. This single optical sensor cable measures multiple relevant vehicle information (multi-parameters), such as vehicle speed, wheelbase, number of axles, and tire pressure. This information can be advantageously used to obtain secondary information, such as vehicle weight, as well as overall information about the vehicle fleet, such as traffic congestion and freight volume. Unlike simple traffic counts, our approach can provide counts grouped by vehicle type and load weight. The high temporal sampling rate of distributed acoustic sensing and a dedicated peak detection algorithm for reliably extracting timing information provide accurate measurements.

Description

Technical Field

[0001] This application generally relates to distributed fiber optic sensing (DFOS) systems, methods, structures, and related technologies. More particularly, this application relates to DFOS systems, methods, and structures for detecting and measuring vehicle-infrastructure interactions via existing communication cables.

Background Art

[0002] The movement of people and goods is facilitated by vehicles traveling on highways and general roads. Information regarding individual vehicles, such as (1) vehicle type (wheelbase, number of axles), traveling speed, tire condition, traveling weight, and (2) information regarding the collection of vehicles during periods such as traffic congestion and traffic volume, reveals the flow of goods and services and provides an objective indicator regarding economic activity in a local geographical area of interest.

[0003] The real-time, automated collection of such information is useful for road planners and road administrators. This information is often measured individually by specialized sensors such as cameras, pneumatic road tubes, piezoelectric sensors, magnetic sensors, and fiber optic Bragg grating (FBG) fiber sensors. However, solutions using these sensors have many limitations, including the need for point sensors to be installed scattered along highways and roads. These limitations include being limited to areas where only data transmission and power supply are available, lacking the ability to identify vehicles traveling in close proximity to each other in congested or heavy traffic conditions, being limited to multi-lane highways, being affected by lighting conditions and weather such as rain and fog, incurring additional installation costs for privacy protection, damaging roads, using dedicated fiber optic cables, and the fact that conventional distance fiber sensing and waterfall-based approaches can only provide coarse-grained information such as traffic volume and average speed over a certain period without knowing precise parameters such as vehicle type and instantaneous speed, and require expensive hardware and advanced computer vision or machine learning algorithms that involve high latency. [Overview of the project]

[0004] The present disclosure describes a vehicle-infrastructure interaction system and method employing a distributed optical fiber sensing (DFOS) system that operates on pre-installed optical fiber communication cables buried alongside or in close proximity to highways / roadways, thereby advancing the technology.

[0005] In contrast to prior art, the systems and methods of the present invention according to the embodiments of this disclosure require only a single optical sensor cable to detect / monitor multiple target locations and multiple lanes, providing a continuous 24 / 7 information stream of vehicle traffic at multiple locations along a roadway. This single optical sensor cable measures multiple relevant vehicle parameters, such as speed, wheelbase, number of axles, and tire pressure. This information can be suitably used to obtain secondary information, such as vehicle weight, as well as overall information about vehicle groups, such as traffic congestion and cargo volume. Unlike simple traffic volume counting, the approach of the present invention can provide counts grouped by vehicle type and load weight. Finally, the systems and methods according to the embodiments of this disclosure provide accurate measurements achieved by a high temporal sampling rate of distributed acoustic sensing and a dedicated peak detection algorithm for reliably extracting timing information. [Brief explanation of the drawing]

[0006] [Figure 1(A)] This is a schematic diagram illustrating an exemplary prior art unencoded DFOS system. [Figure 1(B)] This is a schematic diagram illustrating an exemplary prior art encoded DFOS system.

[0007] [Figure 2] This schematic block diagram illustrates an exemplary sensing setup according to an aspect of the present disclosure, in which a multi-axle vehicle passes a pair of target locations along a roadway, and multiple parameters relating to the vehicle are sensed via DFOS.

[0008] [Figure 3] This schematic block diagram illustrates an exemplary vehicle measurement based on DAS waveform signals as a vehicle having two axles passes a pair of target positions along a roadway. According to aspects of this disclosure, the position and number of self-repeating patterns, the gaps between patterns, and the magnitude and frequency of the acoustic signals convey information about the type of vehicle, tire condition, and running weight.

[0009] [Figure 4] This is a schematic diagram illustrating an exemplary design / layout for specific scenarios, such as high-density and / or multi-lane traffic, according to aspects of this disclosure.

[0010] [Figure 5] This is a schematic diagram illustrating an exemplary operational pipeline for peak detection-based vehicle information estimation according to an aspect of the present disclosure.

[0011] [Figure 6(A)] This figure shows a comparison of raw waveform data before and after applying a bandpass filter according to an aspect of this disclosure. [Figure 6(B)] This figure shows the estimation of the number of axles by calculating the cumulative magnitude of the peak and counting the number of continuously increasing intervals, according to an aspect of this disclosure.

[0012] [Figure 7] This is a schematic block / flow diagram illustrating the multi-parameter prediction of a vehicle using a supervised model trained with multivariate labels, according to the aspects of this disclosure.

[0013] [Figure 8] The results of using a peak-based method to estimate wheelbase and speed according to an aspect of this disclosure are shown. [Modes for carrying out the invention]

[0014] The following are merely illustrative examples of the principles of this disclosure. Those skilled in the art will therefore understand that various configurations embodying the principles of this disclosure, and that fall within its spirit and scope, can be devised, although not expressly described or illustrated herein.

[0015] Furthermore, all examples and conditional terms described herein are intended solely for educational purposes to help readers understand the concepts to which the inventors have contributed to advance the principles and art of this disclosure, and should be construed as not being limited to such specifically listed examples and conditions.

[0016] Furthermore, all descriptions herein describing the principles, aspects, and embodiments of this disclosure, as well as specific examples thereof, are intended to encompass both their structural and functional equivalents. Moreover, such equivalents are intended to include both currently known equivalents and future-developed equivalents, i.e., developed elements that perform the same function regardless of structure.

[0017] Therefore, it will be understood by those skilled in the art that, for example, any block diagram in this specification represents a conceptual diagram of an exemplary circuit for implementing the principles of this disclosure.

[0018] Unless otherwise specified in this specification, the figures comprising the drawings are not drawn to scale.

[0019] As some additional background, it should be noted that distributed optical fiber sensing systems interconnect optoelectronic integrators into optical fibers (or cables), transforming the fiber into an array of sensors dispersed along the fiber. In practice, the fiber becomes the sensor, and the interrogator generates / injects laser light energy into the fiber to sense / detect events along the fiber.

[0020] As will be understood and recognized by those skilled in the art, DFOS technology can be deployed to continuously monitor vehicle movement, human traffic, excavation activities, seismic activity, temperature, structural integrity, leaks of liquids and gases, and many other conditions and activities. This is used worldwide to monitor power plants, communication networks, railways, roads, bridges, borders, critical infrastructure, onshore and offshore power lines and pipelines, and downhole applications in oil, gas, and enhanced geothermal power. Advantageously, distributed fiber optic sensing is not constrained by line of sight or remote power access and, depending on the system configuration, can be deployed in continuous lengths exceeding 30 miles with sensing / detection possible at all points along that length. Thus, the cost per sensing point over long distances is typically not comparable to competing technologies.

[0021] Distributed fiber optic sensing measures changes in the "backscattering" of light that occur within an optical sensing fiber when the optical sensing fiber encounters an environmental change that includes events of vibration, strain, or temperature change. As described above, the optical sensing fiber functions as a sensor over its entire length, providing real-time information regarding the physical / environmental surroundings and the integrity / security of the fiber. Further, distributed fiber optic sensing data identifies the exact location of events and conditions that occur along or near the sensing fiber.

[0022] A schematic diagram illustrating a generalized arrangement and operation of a distributed fiber optic sensing system that can advantageously include artificial intelligence / machine learning (AI / ML) analysis is exemplarily shown in FIG. 1(A). Referring to FIG. 1(A), it can be seen that an optical sensing fiber is connected to an interrogator. Although not shown in detail, the interrogator can include an encoded DFOS system that can employ a coherent receiver arrangement known in the art as shown in FIG. 1(B).

[0023] As is well known, modern interrogators are systems that generate an input signal to an optical sensing fiber, detect / analyze the reflected / backscattered signal, and then receive it. The received signal is analyzed to produce an output that indicates the environmental conditions encountered along the fiber. The received backscattered signal may be due to reflections within the fiber, such as Raman backscattering, Rayleigh backscattering, and Brillouin backscattering.

[0024] As is recognized, modern DFOS systems include an interrogator that periodically generates optical pulses (or any encoded signals) and injects them into an optical sensing fiber. The incident optical pulse signals are transmitted along the optical fiber.

[0025] Along the fiber, a small portion of the signal is backscattered / reflected and returned to the interrogator, where it is received. The backscattered / reflected signal carries information that the interrogator uses to detect, such as changes in power levels indicating mechanical vibrations.

[0026] The received backscattered signal is converted into the electrical domain and processed within the interrogator. Based on the pulse incidence time and the time the received signal was detected, the interrogator determines the location from which the received signal returned along the optical sensing fiber, and as a result, can sense activity at each location along the optical sensing fiber. A classification method may be used to further detect and locate events or other environmental conditions, including acoustic and / or vibration and / or heat, along the optical sensing fiber.

[0027] As will be further demonstrated and explained, the systems, methods, and structures according to aspects of the present invention combine a linear structure of distributed optical fiber sensing (DFOS) and thus inherit the advantages of conventional distributed optical fiber sensing. At the same time, it utilizes the vibration / acoustic patterns of a vehicle that occur in close proximity to a point along the optical sensor fiber, which is part of the DFOS. As a result, it not only enjoys the advantages of high-precision measurement of conventional point sensors, but also enables other advanced applications (such as vehicle license plate recognition and real-time driver notification) when used in combination with sensor fusion, i.e., wireless communication technologies such as non-acoustic sensors and cameras.

[0028] As those skilled in the art will understand and recognize, potential applications of the systems, methods, and structures of the present invention include vehicle fault diagnosis and early warning, fuel cost and road damage estimation, accurate toll determination and estimation, economic data generation, speeding detection and billing systems, and the like.

[0029] The system, method, and structure of the present invention detect, collect, and analyze vehicle-to-infrastructure (V2I) interactions. The system, method, and structure of the present invention have the advantage of improving the safety and productivity of highways and roads, while also providing a DOFS-based system for smart traffic and smart roads.

[0030] As a further advantage, the systems, methods, and structures of the present invention utilize existing cable architectures, thus eliminating the additional cost of deploying dedicated sensors, and leverage acoustic signals generated by vehicles passing over auxiliary target locations on or along roads, including but not limited to speed bumps, bridge decks, manholes, road potholes, or acoustic alarm patterns (SNAPs). Since detection is based on peak detection or self-similarity between patterns / signatures generated by the same vehicle, the systems, methods, and structures of the present disclosure are more robust to heterogeneous factors caused by different geographical locations and soil types, as well as various types of auxiliary target locations. Dedicated digital signal processing (DAS) and machine learning (ML) algorithms for data processing are low-cost, lightweight, and run in real time.

[0031] Figure 2 is a schematic block diagram showing an exemplary sensing setup according to an aspect of the present disclosure, in which a multi-axis vehicle passes a pair of target locations along a roadway, and multiple parameters relating to the vehicle are sensed via DFOS.

[0032] As shown in this diagram, the sensing layer is superimposed on the existing deployed fiber optic network. The Distributed Fiber Optic Sensing System (DFOS) is a Distributed Acoustic Sensing (DAS) integrator installed at the control station / central station, collecting data from the entire fiber optic sensor cable path. The DFOS system is connected to field optical sensor fibers and provides real-time sensing capabilities at multiple locations using a location division multiplexing (LDM) scheme. Dark fiber or service provider operational fiber can be used for the fibers.

[0033] To measure multiple vehicle parameters with high precision, a pair of targeted locations are required. Such locations can be easily utilized on the road, such as speed bumps, bridge decks, or acoustic warning patterns (SNAP). They can also be easily deployed at specific locations or during periods of interest, for example, at certain facilities (parking lots, gates, stadiums, etc.) during special music or sporting events.

[0034] As is known, DFOS measures displacement on a fiber optic cable and provides changes in phase waveform as output. When a vehicle passes a targeted location, a self-similar acoustic pattern is generated, which can be detected. The mechanism is shown in Figure 3, using a two-axle vehicle as an example.

[0035] Figure 3 is a schematic block diagram illustrating an exemplary vehicle measurement based on DAS waveform signals as a vehicle having two axles passes a pair of target positions along a roadway. According to aspects of this disclosure, the position and number of self-repeating patterns, the gaps between patterns, and the magnitude and frequency of the acoustic signals convey information about the type of vehicle, tire condition, and running weight.

[0036] The algorithmic operations used to measure basic vehicle information include peak detection, delayed correlation calculation, movement window, spatial clustering, filtering, local averaging, and power spectral density calculation. Vehicle classification can be accurately determined based on wheelbase and number of axles, such as sedans and trucks, regardless of load weight. The number of vehicles and their speeds represent short-term or seasonal traffic volume and congestion. Vehicles with unusual wheelbases or individual vehicles exceeding the speed limit can also be detected.

[0037] It should be noted that secondary information such as tire pressure and vehicle weight requires a comprehensive consideration of multiple related factors (integrated inference of instantaneous velocity, axle weight, and vibration pattern). This is impossible with conventional point sensor-based solutions, which can only provide (or focus on) some of the factors.

[0038] The systems, methods, and structures of the present invention, according to aspects of this disclosure, advantageously handle high-density and multi-lane scenarios and provide more accurate estimations by utilizing a set of multiple target locations. These specific scenarios are shown in Figure 4. Figure 4 is a schematic diagram showing an exemplary design / layout for specific scenarios such as high-density and / or multi-lane traffic according to aspects of this disclosure.

[0039] More specifically, (1) if there are multiple pairs of target locations within the same lane, the estimation results can be made more accurate by averaging across multiple pairs (as shown in Figure B2-8, which illustrates the results using the peak-based method for wheelbase and speed estimation according to an embodiment of this disclosure), and a distribution of parameter estimations can also be provided. (2) The direction of travel of a vehicle can be inferred from the order of the peaks appearing at these target locations. (3) Vehicles traveling in multiple lanes can be naturally identified by pairs of target locations in different lanes. Note that the targets must be embedded in the road, not just on the side of the road (for example, utility poles cannot be used).

[0040] The aforementioned fiber sensing setup allows for the acquisition of waveform signals from multiple fiber sensors for vehicle measurement. The algorithm employed is described in detail below.

[0041] As is understood, one of the key challenges is to extract accurate timing information of events from raw waveform data. One estimation method is based on calculating the distance between event peaks in the original waveform space or the transformed space. The pipeline is shown in Figure 5. Figure 5 is a schematic diagram showing an exemplary operational pipeline for vehicle information estimation based on peak detection according to an aspect of this disclosure.

[0042] Each raw waveform data is first pre-processed using a bandpass filter with a pass frequency of 50Hz to 200Hz.

[0043] Figures 6(A) and 6(B) illustrate the following: Figure 6(A) shows a comparison of raw waveform data before and after applying a bandpass filter according to an aspect of the disclosure. Figure 6(B) shows the estimation of the number of axles by calculating the cumulative magnitude of the peaks and counting the number of continuously increasing intervals, according to an aspect of the disclosure.

[0044] As shown in Figure 6(A), the amplitude is reduced, but the waveform noise decreases and the peaks become denser. This is advantageous for peak detection because the detector is less likely to miss peaks. The filtered signal is then normalized by the mean and standard deviation of all the collected data, so that different signals take on similar ranges of values, making the threshold universally applicable.

[0045] Estimate the number of axles.

[0046] As shown in Figure 6(B), the number of axles is estimated by calculating the cumulative amplitude of peaks and counting the number of continuously increasing intervals. For each waveform signal, peaks are first detected by simply comparing neighboring values ​​that exceed the absolute amplitude threshold. The orange cross symbols indicate all detected peaks and their absolute amplitudes. Next, the cumulative amplitude of all points is calculated from the original amplitude values. The amplitude of all points that are not peaks is zero. The cumulative amplitude is used as a feature for DBSCAN clustering. Because the neighboring radius threshold is very small and the minimum number of points is very large, this clustering algorithm can accurately detect continuous zero intervals and continuously increasing intervals. The total number of continuously increasing intervals is the number of axles obtained from this waveform. "Group 1" and "Group 2" in Figure 6(B) show the continuously increasing intervals detected in this waveform signal. The cumulative amplitude feature can be computed faster by downsampling before clustering. The final estimated number of axles is voted on by multiple sensors.

[0047] Find the representative peak.

[0048] After obtaining the number of axles, i.e., the number of peak groups in the waveform data, the next step is to determine the timestamp (x-coordinate) of each group. One intuitive method is to average the x-coordinates of all peaks belonging to a group, but this requires all peaks to be within the expected range, which is difficult if the waveform is noisy. Therefore, the peak with the largest amplitude is considered the representative peak of this group. This peak is not necessarily located in the center, but it is likely to be within the range of the "hill". To achieve this, first, the z-score of the x-coordinate of each peak is calculated, and outlier peaks are filtered out by excluding peaks that exceed a threshold (i.e., by subtracting the mean value and then dividing by the standard deviation). Next, the K-means method is applied with the estimated number of axles as a parameter, and the peaks are clustered using the x-coordinates. In this way, the representative peak, which is the highest peak, can be found for each cluster.

[0049] Calculate the vehicle's measurements.

[0050] Before calculating vehicle measurements using representative peaks, first check whether the distance between adjacent peaks is within a reasonable range. If not, this waveform data is discarded.

[0051] Wheelbase. The wheelbase can then be calculated by calculating the ratio of the wheelbase to the distance between the target positions. Specifically, as shown in Figure 2, assuming the distance between adjacent positions is d meters, the wheelbase is...

number

number

[0052] Here, T1 is the time difference between the left and right peaks, corresponding to the front and rear wheels passing through position 1-a or position 1-b. Then, T2 is the time difference between the two left peaks or the two right peaks, corresponding to the front wheel or rear wheel passing through two locations. Since there are two estimates for T1 and T2, the ratio can be estimated in four combinations. The wheelbase in meters is obtained by taking the average of the four ratios and multiplying by d.

[0053] Speed. Instantaneous speed can be estimated using the following formula.

number

[0054] In addition,

number

[0055] To demonstrate the effectiveness of this method, 16 test runs were conducted with a two-axle vehicle. The results are shown in Table 1. "Gt speed" is the speed read by the driver from the vehicle's speedometer, while "estimated speed" and "estimated wheelbase" are the speed and wheelbase estimated using the method described above. The vehicle's ground truth wheelbase is 2.64 m. The estimation errors for speed and wheelbase were 0.15 ± 0.98 mph and 0.031 ± 0.19 m, respectively, indicating that the estimation is accurate and stable. Furthermore, more accurate results can be obtained by setting multiple target positions.

[0056] Peak-based methods allow for accurate estimation of wheelbase and speed, and the location of peaks is also useful for real-time vehicle detection as a first step. However, they lack robustness when the peak density is low, or when the peak size varies due to factors such as weather conditions or the type of auxiliary target location.

[0057] One alternative solution is to assume that the peak groups are similar and look for repeating patterns in the signals. Specifically, T1 is estimated by calculating the autocorrelation and finding the corresponding lag of the second largest peak within it. For vehicles with two or more axles, such as a three-axle vehicle, the lags of the second and third largest peaks in the correlation plot correspond to their two wheelbases. To estimate T2, the cross-correlation between the two signals can be calculated. Then, the wheelbase and speed can be estimated using equations (1) and (2).

[0058] Another solution involves using a sliding window to search for the closest pattern from all subsequences. Characteristic vibration patterns of the same vehicle passing through a secondary target location exhibit high reproducibility. When this vibration occurs, the shortest distance for measuring similarity between patterns becomes small. On the other hand, background noise patterns are not reproducible, and therefore the shortest distance becomes large. Thus, the proposed peak detection algorithm can also function in this transformed shortest distance space. Even if the original pattern contains multiple peaks, only one peak exists in this transformed shortest distance space. Therefore, the peak detection procedure is more robust in this space.

[0059] The type of vehicle (e.g., sedan, SUV, truck) can be directly estimated from the wheelbase and the number of axles.

[0060] Additional vehicle parameters predicted from the vibration pattern.

[0061] Given the peak location, a signal segment containing the vibration pattern can be extracted. The same event can be detected at multiple locations on the cable, allowing for multi-channel measurements. By calculating the power spectral density of the vibration pattern, magnitude and frequency information can be obtained.

[0062] Furthermore, supervised machine learning models, such as regression and classification models, can be trained to predict vehicle parameters. Given ground truth labels for cargo weight and tire pressure, models can be trained to predict from raw waveforms or power spectra. This has at least two important aspects.

[0063] First, instead of training a model individually for each label, we need to use a multivariate label model to consider the interactions between multiple labels and predict them together (e.g., axle weight and tire pressure).

[0064] Next, the interaction also depends on the type of vehicle and its speed. These are included as covariates in the predictive model.

[0065] This is a schematic block / flow diagram illustrating the overall configuration of a multi-parameter prediction and prediction model for a vehicle using a supervised model trained with multivariate labels, according to aspects of this disclosure. The outputs are axle weight and tire pressure, which are predicted together based on two sets of inputs. The first set of inputs is the raw waveform or power spectrum of an acoustic signal, including information on magnitude and frequency. The second set of inputs includes the vehicle type and speed, which have already been measured and serve as conditioning factors.

[0066] Those skilled in the art will understand the optical fiber sensing and machine learning-based solution of the present invention for accurately measuring vehicle parameters based on vehicle-infrastructure interaction. As described above, the sensing system of the present invention includes 1) a DAS interrogator, 2) a communication cable laid or buried along the road, and 3) a pair of auxiliary targeted locations that generate vehicle vibrations (such as speed bumps, bridge decks, or acoustic alarm patterns (SNAPs)).

[0067] The software of the present invention includes algorithms for data preprocessing, estimation, and prediction of multiple vehicle parameters. The sensing function operates on both individual vehicles and groups of vehicles. The ingenious system, method, and structure of the present invention enable a variety of smart road and smart traffic applications that improve the safety and efficiency of highways and public roads. [Table 1]

[0068] While the disclosure has been presented using several specific examples, those skilled in the art will recognize that the teachings are not limited in this way. Therefore, the disclosure should be limited only by the claims appended to this specification.

Claims

1. A method for detecting vehicles and classifying them based on the interaction of vehicle infrastructure, To operate a distributed fiber optic sensing (DFOS) system having a pair of targeted detection locations along the roadway, From the DFOS data received from that operation, the position and number of at least the self-repeating patterns in the DFOS data are determined. From the DFOS data received from the aforementioned operation and associated with the position and self-repeating pattern, information about the vehicle traveling on the roadway is determined, including one or more of the vehicle type, tire condition, and the vehicle's weight. Identifying the position and number of the self-repeating patterns in the DFOS data, and the gaps between the patterns, A method comprising determining an estimate of the number of axles from the DFOS data, based on the cumulative magnitude of the peaks in the DFOS data and the number of continuously increasing intervals in the DFOS data.

2. The method according to claim 1, further comprising determining the magnitude and frequency of an acoustic signal from the DFOS data based on the position and number of self-repeating patterns in the DFOS data.

3. The method of claim 2, further comprising determining the instantaneous speed of the vehicle from the DFOS data from the position and number of self-repeating patterns in the DFOS data.

4. The method according to claim 3, further comprising a plurality of pairs of targeted detection positions along the roadway.

5. The method according to claim 4, wherein the plurality of pairs of targeted detection positions along the roadway are located in at least two driving lanes of the roadway.

6. The method according to 5, characterized in that the at least two driving lanes of the roadway carry vehicles in different directions.

7. The method according to claim 1, further comprising applying the DFOS data to a trained neural network predictive model to determine the axle weight and tire pressure of a vehicle traveling on the roadway.

8. The method according to claim 7, wherein the DFOS data includes raw waveform data and power spectrum data, the second input dataset includes vehicle type and vehicle speed, and the raw waveform data and power spectrum data are used together with the vehicle type and vehicle speed as input to the prediction model.

Images