Outdoor Application of Distributed Fiber Optic Sensing / Acoustic Sensing

Abstract

Aspects of the present disclosure describe distributed fiber optic sensing (DFOS) systems, methods, and structures that advantageously sense / monitor outdoor facilities and structures, including outdoor cabinets housing fiber optic facilities, with the cabinets / fiber optic cables contained therein configured to provide superior acoustic sensing. Further monitored outdoor facilities and structures include manhole structures. Machine learning based analytical techniques using temporal relationship networks (TRNs) are employed to provide superior DFOS / DAS monitoring results.

Description

Technical Field

[0001] Cross-reference This application claims the benefit of U.S. Provisional Patent Application No. 63 / 270,651, filed Oct. 2, 2021; U.S. Provisional Patent Application No. 63 / 311,523, filed Feb. 18, 2022; U.S. Provisional Patent Application No. 63 / 313,028, filed Feb. 23, 2022; and U.S. Patent Application No. 17 / 958,415, filed Oct. 2, 2022, and each is incorporated herein by reference in its entirety as if fully set forth herein.

[0002] The present disclosure generally relates to distributed fiber optic sensing (DFOS) systems, methods, and structures, and outdoor applications thereof, which may advantageously include a time-relationship network methodology for providing excellent sensing.

Background Art

[0003] Recently, DFOS systems and methods have been used to provide excellent acoustic and / or vibration monitoring of roads, bridges, and buildings. The reliability, robustness, and sensitivity of such systems are generally known to be incomparable to existing conventional systems and methods. Considering such characteristics, various applications of DFOS to outdoor facilities including communication facilities, in combination with new analysis systems and methods, are welcome in the art.

Summary of the Invention

[0004] Advances in the art are made in accordance with aspects of the present disclosure directed to DFOS systems, methods, and structures that are particularly advantageous for outdoor applications including monitoring of telecommunication facilities.

[0005] Viewed from a first aspect, the present disclosure describes a DFOS system, method, and structure for monitoring an outdoor cabinet housing fiber optic equipment, wherein the cabinet / fiber optic cable contained therein is configured to provide excellent acoustic sensing.

[0006] From a second aspect, the present disclosure describes a DFOS system, method, and structure for monitoring a manhole structure.

[0007] Finally, from yet another aspect, the present disclosure describes a DFOS system, method, and structure that uses a machine learning-based analysis method using a temporal relational network.

Brief Description of the Drawings

[0008] A more complete understanding of the present disclosure can be realized by referring to the accompanying drawings.

[0009]

Figure 1(A)

[0010]

Figure 1(B)

[0011]

Figure 2

[0012]

Figure 3

[0013]

Figure 4

[0014]

Figure 5

[0015]

Figure 6

[0016]

Figure 7

[0017]

Figure 8

[0018]

Figure 9

[0019]

Figure 10

[0020]

Figure 11

[0021]

Figure 12

[0022]

Figure 13

[0023]

Figure 14

[0024]

Figure 15

[0025]

Figure 16

[0026]

Figure 17

[0027]

Figure 18

[0028]

Figure 19

DETAILED DESCRIPTION OF THE INVENTION

[0029] The following are merely examples to illustrate the principles of the present disclosure. Accordingly, it will be understood that those skilled in the art can devise various configurations that embody the principles of the present disclosure and are within its spirit and scope, although not explicitly described or illustrated herein.

[0030] Furthermore, all examples and conditional terms described herein are intended solely for the educational purpose of assisting the reader in understanding the concepts contributed by the inventors to facilitate the principles and techniques of the present disclosure, and should not be construed as being limited to such specifically recited examples and conditions.

[0031] Furthermore, all descriptions in this specification that describe the principles, aspects, and embodiments of the present disclosure, as well as its specific examples, are intended to encompass both its structural and functional equivalents. Furthermore, such equivalents are intended to include both currently known equivalents and equivalents developed in the future, i.e., developed elements that perform the same function regardless of structure.

[0032] Accordingly, for example, it will be understood by those skilled in the art that any block diagram in this specification represents a conceptual diagram of an exemplary circuit that implements the principles of the present disclosure.

[0033] Unless otherwise specified herein, the figures constituting the drawings are not drawn to scale.

[0034] As additional background, begin by noting that distributed fiber optic sensing (DFOS) is an important and widely used technology for detecting environmental conditions (such as temperature, vibration, acoustically excited vibration, stretch levels, etc.) at any location along an optical fiber cable that is serially connected to an interrogator. As is known, modern interrogators are systems that generate an input signal to the fiber, which is reflected / scattered and then the received signal is detected / analyzed. The signal is analyzed and an output is generated that indicates the environmental conditions encountered along the fiber. The signal received in this way can be due to reflections within the fiber such as Raman backscattering, Rayleigh backscattering, Brillouin backscattering, etc. DFOS can also use forward signals that utilize the velocity differences of multiple modes. Without loss of generality, the following description assumes a reflected signal, but the same approach can be equally applied to transmitted signals.

[0035] Figure 1(A) is a schematic diagram of a generalized prior art DFOS system. As will be understood, modern DFOS systems include an interrogator that periodically generates optical pulses (or any encoded signal) and injects them into an optical fiber. The injected optical pulse signal is transmitted along the optical fiber.

[0036] At positions along the fiber, a small portion of the signal is reflected and returned to the interrogator. The reflected signal transmits information that the interrogator uses to detect, such as a change in power level indicating mechanical vibration. Although not shown in detail, the interrogator can include an encoded DFOS system that can employ a coherent receiver configuration known in the art as shown in Figure 1(B).

[0037] The reflected signal is converted to the electrical domain and processed within the interrogator. Based on the pulse injection time and the time the signal is detected, the interrogator can determine from which position on the fiber the signal is coming and can sense the behavior of each position on the fiber.

[0038] One skilled in the art will understand and recognize that by implementing signal encoding on the interrogation signal, more optical power can be transmitted into the fiber, thereby advantageously improving the signal-to-noise ratio (SNR) of Rayleigh scattering-based systems (e.g., distributed acoustic sensing, i.e., DAS) and Brillouin scattering-based systems (e.g., Brillouin optical time domain reflectometry, i.e., BOTDR).

[0039] As implemented in many modern implementations today, dedicated fibers are assigned to the DFOS system of an optical fiber cable and are physically separated from existing optical communication signals transmitted on different fibers. However, considering the explosively increasing bandwidth demand, it has become very difficult to economically operate and maintain optical fibers for DFOS operation only. As a result, there is a growing interest in integrating communication systems and sensing systems on a common fiber that is part of a larger multi-fiber cable.

[0040] Operationally, assume that the DFOS system is a Rayleigh scattering-based system (e.g., distributed acoustic sensing, i.e., DAS) and a Brillouin scattering-based system (e.g., Brillouin optical time domain reflectometry, i.e., BOTDR) with a coding implementation. In such a coding design, these systems are likely to be integrated with the fiber communication system due to their low operating power, and the influence of the response time of the optical amplifier also becomes significant.

[0041] In the configuration exemplarily shown in the block diagram, assume that an encoded interrogation sequence is digitally generated and modulated onto a sensing laser via digital-to-analog conversion (DAC) and an optical modulator. The modulated interrogation sequence may be amplified to an optimal operating power before being sent into the fiber for interrogation.

[0042] Advantageously, the DFOS operation can also be integrated with communication channels via WDM within the same fiber. In the sensing fiber, the interrogation sequence and the returned sensing signal are optically amplified either via a discrete (EDFA / SOA) or distributed (Raman) scheme. The returned sensing signal is sent to a coherent receiver after amplification and optical bandpass filtering. The coherent receiver detects the optical fields of both polarizations of the signal and down-converts them to four baseband lanes for analog-to-digital conversion (ADC) sampling and digital signal processor (DSP) processing. As would be readily understood and recognized by those skilled in the art, the decoding operation is performed by the DSP to generate the Rayleigh or Brillouin response interrogated of the fiber, after which the changes in the response are identified and interpreted for sensor readout.

[0043] Continuing to refer to the figure, since the encoded interrogation sequence is generated digitally, the out-of-band signal is also generated digitally and then integrated with the code sequence before the waveform is generated by the DAC. When generated digitally together, the out-of-band signal is generated only outside the period of the code sequence, so when added together, the amplitude of the integrated waveform is constant.

[0044] As would be understood and recognized by those skilled in the art, the DFOS / DAS system has been shown to detect, record, and listen to acoustic vibrations in the audible frequency range. However, one of the factors limiting the sensitivity is the physical layout of the optical fiber cable used as the sensor.

[0045] In outdoor applications, communication-grade thick fiber cables do not physically respond much to low-amplitude vibrations in the audible range. Thus, the quality of the acoustic signal depends greatly on the type of fiber, the layout, and how the acoustic pressure wave is coupled to the fiber cable.

[0046] To improve the quality of acoustic signals, in the prior art, wrapped fiber cables without a jacket or with a very thin jacket have been used. However, such a configuration is very fragile, relatively bulky, and has insufficient resistance to outdoor environmental conditions that may be encountered.

[0047] To overcome this drawback in the art, a system, method, and structure are disclosed for converting an outdoor fiber cabinet (such as an optical cross-connect (OCC) cabinet) into an optical fiber microphone that advantageously enhances a DAS system.

[0048] As will be understood and recognized by those skilled in the art, such an outdoor fiber cabinet with an integrated fiber microphone can be used to monitor external (outside the cabinet) or internal (inside the cabinet) acoustic events such as traffic noise, construction noise, rain, wind, technician conversations, intrusion into the cabinet, and the operation of electronic or mechanical devices within the cabinet. As a result, the structure and method of the present invention can monitor the state of the outdoor fiber cabinet, such as permitted or unpermitted opening and closing events, timing, and repairs.

[0049] An outdoor fiber cabinet (a sample image is shown below) typically has a large metal door. The surface of this large door has a large interaction area with sound waves and absorbs these vibrations. In the present invention, a fiber patch about 10 meters in length is attached to this large-area cabinet door, and additional fibers can also be attached to the side panels if possible.

[0050] FIG. is a schematic diagram showing an outdoor optical fiber cabinet including an optical sensing fiber that can advantageously detect the open / closed state of the door and other internal and external environmental conditions according to an aspect of the present disclosure.

[0051] As shown in this figure, what is shown there is a DAS sensing system, and the outdoor fiber cabinet includes a dedicated fiber attached to the cabinet door in a spool shape. As further shown in the figure, the field configuration can include multiple cabinets on a single fiber path, and each of the individual ones of the multiple cabinets is connected in series along the same optical sensing fiber.

[0052] Although not specifically shown in this figure, the DFOS / DAS system processes the interrogator signal, and according to the aspects of the present disclosure, since the fiber cabinet functions as an acoustic filter due to its non-uniform frequency response, it will return a response signal showing an improvement in signal quality. Due to the non-uniform frequency response, it affects the acoustic filter. When configured in this way with an optical fiber cabinet, the attachment of the optical fiber cable needs to be done so as not to prevent the cabinet door from opening.

[0053] As those skilled in the art will understand and recognize, the cabinet / optical fiber cable structure of the present invention utilizes the mechanical characteristics of the large surface area of the cabinet door and uses it to improve the acoustic coupling of acoustic vibrations to the fiber cable and improve the signal quality. In other words, the large surface of the cabinet is used as a mechanical amplifier / coupler to increase the received signal with the fiber cable. Advantageously, such amplification is purely mechanical and does not require an external power source. Finally, as will be shown and described in detail in the latter part of the present disclosure, additional signal processing and analysis structures and techniques can be used to correct the frequency response of the cabinet structure by acoustic calibration and other methods to further improve the received signal quality.

[0054] Figure 3 is a schematic flow diagram showing the operation of DFOS / DAS sensing of an outdoor optical fiber cabinet including an optical sensing fiber according to an aspect of the present disclosure.

[0055] As can be seen from this figure, a fiber cabinet (a structure including an optical fiber cable that can be used for telecommunications in addition to the sensor function) includes an optical fiber sensor cable attached in a circular coil configuration inside the fiber cabinet. Such a sensor fiber is optically and / or mechanically connected to the fiber path to be investigated / questioned / analyzed / monitored.

[0056] When an acoustic event occurs sufficiently close to the cabinet, mechanical vibrations are generated within the fiber cabinet, amplified by a relatively large area of the cabinet surface (door), and coupled to the sensor fiber contained therein.

[0057] Such mechanical disturbances are detected by the operation of the DFOS / DAS system and analyzed / corrected according to a cabinet response function that can be advantageously determined in advance.

[0058] The analyzed signal is used to generate a report and / or notification of the event of interest.

[0059] At present, there is no recognized "good" solution for manhole intrusion detection (and continuous condition monitoring). Therefore, the operator of the facility needs to regularly dispatch technicians to the manhole site. However, such efforts are inefficient and time-consuming.

[0060] Accordingly, the disclosure continues of a DFOS system, method, and structure that can be advantageously used to monitor the condition of other outdoor communication facilities including manholes. Thus, a method for detecting manhole / handhole intrusion activities is described by examining the patterns of spatio-temporal data collected by a distributed fiber optic sensing (DFOS) system including DFOS / DAS distributed acoustic sensing, and / or DFOS / distributed vibration sensor (DVS). Particularly advantageous is that the disclosed systems and methods of the present invention can provide automated real-time manhole / handhole intrusion detection. By this method, a DFOS-based manhole / handhole intrusion detection solution can be implemented.

[0061] As will be illustrated and described hereinafter, the DFOS systems and methods of the present invention employ artificial intelligence (AI) techniques such that they are an integrated solution for automatically and in real-time monitoring manholes / handholes along an entire optical fiber cable path that can simultaneously transmit communication traffic along with the fiber optic sensing signal.

[0062] As further shown, the DFOS systems and methods of the present invention advantageously use the DFOS system to collect field vibration signals around all manholes along the fiber path year-round, and an AI engine is employed to automatically identify traces of manhole / handhole intrusion activities (opening the cover, touching / shaking the cable, knocking on the wall, etc.) and respond in real-time, providing risk assessment and activity classification from the AI engine, and when a high-risk event is detected, optionally waiting for user confirmation while providing an alarm popup on a graphical user interface (GUE), accurately identifying the high-risk event on a geographic / geographic information system (GIS), providing a warning message to the operator, and logging an archival record of the event including time and manhole / handhole ID information.

[0063] A key technical problem related to the systems and methods according to aspects of the present disclosure is to provide a mechanism for distinguishing vibrations caused by intrusion events from vibrations caused by normal environmental conditions, i.e., vibrations caused by traffic.

[0064] As is known, optical cables are often laid along roads / highways, pipelines, railway lines, etc. In the case of roadways / highways, such manholes / handholes are typically installed in the center of the road or adjacent sidewalks. As a result, while vehicles are using the highway, the covers of the manholes / handholes vibrate, generating DFOS pattern data similar to other DFOS data generated by unnatural events. Extracting traffic vibration data traces further incurs computational overhead. Therefore, the systems and methods of the present invention employ a multi-stage approach. That is, detection is performed by examining the intrusion pattern within each manhole area, and confirmation is based on the signal-to-noise ratio (SNR) metric calculated from adjacent (i.e., left and right) adjacent areas. Therefore, the systems and methods of the present invention guarantee a high intrusion detection rate and a low false alarm rate, while avoiding the need to explicitly extract traffic traces from the DFOS data.

[0065] FIG. 5 is a schematic flowchart showing an exemplary procedure for detecting manhole intrusion using DFOS according to aspects of the present disclosure. As shown in this flowchart, the DFOS system is connected to an optical fiber cable deployed for monitoring. The DFOS system operates and vibration signals from the site are received, including ambient noise, road traffic, vibrations at the site, etc. The received signals are automatically analyzed using a manhole / handhole / intrusion detection AI engine configured according to aspects of the present disclosure. When the AI engine detects traces of intrusion behavior and detects an event determined to be of high risk, information about the location, time, and threat level of the event is displayed on the GUI for investigation / corrective action.

[0066] Figure 6 is a schematic diagram showing an exemplary arrangement for manhole intrusion detection / manhole monitoring using DFOS according to an aspect of the present disclosure.

[0067] As shown in this figure, this arrangement includes a sensing layer overlaid on an existing deployed fiber network. That is, in addition to transmitting live telecommunications traffic, the active telecommunications network can also simultaneously provide DFOS sensor signals.

[0068] In an exemplary arrangement, a DFOS system that can include an interrogator and a detection / AI analysis / reporting system is conveniently located at a control station / central office for remotely monitoring the entire optical fiber cable path. The DFOS system is connected to the optical fiber, and the sensing function is provided in real time over a long period. As described above, the optical fiber used for sensing can be either a "dark" fiber that does not transmit telecommunications or optical signals other than DFOS signals, or an operational ("light") optical fiber that can transmit live telecommunications traffic including the traffic of the service provider.

[0069] To train and test the AI model, as shown in Figure 7, specific intrusion behaviors were simulated on site. Figure 7 shows behaviors related to manholes, including those related to the lid / cover, namely, opening the simulated lid, closing the lid, knocking on the lid, knocking on the case, and pulling the cable.

[0070] From the behaviors shown in Figure 7, it can be seen that different intrusion behaviors generate different vibration patterns in the sensing data. Among the detected intrusion behaviors, actions such as touching or shaking the cable are considered high-risk intrusion events because they interact directly with the cable. These actions require immediate measures to be taken as soon as they are detected, especially if they are unauthorized or unknown to the operator.

[0071] As will be understood and recognized by those skilled in the art, the high dynamic range of the DFOS system can be utilized to detect manhole intrusion events and classify them into critical intrusion events and intrusion events. For example, as schematically shown in FIG. 8, exemplary intrusion events (e.g., lid tapping) and critical intrusions (e.g., cable sway) according to aspects of the present disclosure are shown in schematic waterfall plots and binary masks. Critical intrusion events require a higher level of attention. The temporal and spatial positions of the detected events are indicated by the binary mask. A flowchart of the intrusion detection algorithm is shown in FIG. 4, with two main algorithm modules (detection and false alarm control) and three possible results (critical intrusion, intrusion, normal). As will be understood and recognized, detected critical intrusion events generally have a higher intensity than intrusion events.

[0072] FIG. 9 is a schematic flow diagram showing exemplary intrusion detection for each manhole being monitored according to aspects of the present disclosure. Operationally, the sensing data is analyzed every few seconds. FIG. 10 is a schematic diagram showing an analysis window including a central manhole area and two adjacent areas on the left and right according to aspects of the present disclosure. The position of the manhole along the fiber is known in advance by on-site surveys, for example, using a cable mapping solution from latitude to longitude. FIG. 10 further shows the corresponding slack fiber within the manhole. The left and right areas function as contexts for understanding the cause of the vibration. When the vibration of the manhole is caused by traffic, the vibration pattern is included in both the manhole area and the context area. However, when an actual manhole intrusion occurs, the vibration pattern is mainly limited to the manhole area. Based on this observation, an SNR-based metric is used: SNR = total intensity of the manhole area / total intensity of the context area.

[0073] In practice, this metric effectively removes false alarms caused by traffic.

[0074] FIG. 11 is a schematic block diagram showing an exemplary data processing pipeline in which the input / output dependency relationships of each module are indicated by arrows, according to an aspect of the present disclosure.

[0075] Note that when evaluating the system and method of the present invention, real-time monitoring of 39 handholes from one path was carried out. Intrusion events at individual handholes were detected normally. Note that traffic occurring on manhole / handhole covers also causes vibrations on the manhole covers, and in functions with a low SNR ratio, these events are still classified as normal. Despite the fact that there were active field structures near individual handholes, the handholes were within a safe range from the manholes, so the state of the handholes was also classified as normal. The DFOS system and processing procedure of the present invention provide a sufficiently high spatial resolution to distinguish such events that could not be distinguished by prior art methods.

[0076] Advantageously, the system and method of the present invention employ a user-friendly display interface that can advantageously include cable path information and detected abnormal signals along with risk assessment, thereby providing visualization to operators and telecommunications carriers. Operationally, such a GUI can advantageously include a real-time waterfall trace received from the DFOS system to visualize field vibration signals, a path map including the locations of manholes / handholes, and the status of manholes / handholes. Based on the risk assessment, different levels of risk are displayed in colors such as green, yellow, and orange indicating normal, intrusion, and critical intrusion states, respectively.

[0077] The GUI presentation is even more advantageous when there are multiple intrusion events including events in different states. Since the state of the manhole to be monitored is usually displayed using graduated colors of green, yellow, or orange corresponding to intrusion and significant intrusion respectively, if multiple levels of intrusion are detected for a manhole, the displayed color may be the highest intrusion level. For example, if a detected series of intrusion events includes first tapping and opening the lid, then shaking the cable, and finally closing the lid. The orange light overwrites the yellow light to indicate the highest risk so far. Then, for the event of closing the lid or a normal event, the color does not return to green (normal) unless the user clicks the check button. On the other hand, at the time of detection, the design of the checkpoint button also allows checking whether the detected event is approved or known to the user.

[0078] FIG. 12 is a schematic block diagram showing an exemplary overall manhole intrusion detection process by a DFOS system (DFOS / DAS / DVS) according to an aspect of the present disclosure.

[0079] Here, a distributed acoustic sensing (DAS) and machine learning-based solution for detecting opening and closing events of a manhole cover using an optical fiber pre-deployed inside the manhole will be described in more detail. Furthermore, by adopting the level of automation enabled by the deep learning algorithm of the present invention, the opening and closing events of the manhole can be detected more accurately compared to the prior art monitoring methods, and the state of the manhole can be continuously monitored 24 / 7. As a further advantage, the method of the present invention can monitor hundreds of manholes simultaneously using a single fiber. As known in the DFOS method, the method of the present invention does not require power along the fiber sensor path since the sensor data is directly sent back to the interrogator via the optical backscattering phenomenon.

[0080] Advantageously, without installing additional tracking devices / sensors, the systems and methods of the present invention can detect vibration signals along existing optical fibers via DAS. The DAS signals are preprocessed to generate one or more waterfall images that are analyzed in real time by machine learning algorithms. The deep learning module classifies the events displayed in the waterfall images by investigating the unique data structures characterized by the conversion and temporal relationships. This is achieved by a Temporal Relationship Network (TRN) module customized for distributed fiber sensing data, enabling temporal relationship inference at multiple time scales of the waterfall samples of the neural network. The systems and methods of the present invention provide significant improvements compared to baselines such as conventional CNN-based methods.

[0081] In addition to manhole-related event detection, the TRN configuration of the present invention can also be used in other fiber sensing applications where there is a temporal relationship to the operation / event and the appearance characteristics of the data, such as predicting the moving direction of a machine, are lacking.

[0082] As will be illustrated and described hereinafter, once the model of the present invention is trained with a limited amount of labeled data, it can achieve high accuracy in classifying open / closed events compared to conventional systems and methods. Advantageously, the systems and methods of the present invention according to aspects of the present disclosure predict open / closed events based on temporal relationship inference instead of pattern matching / recognition. As will be appreciated by those skilled in the art, the patterns of two events can be very similar, and thus it is difficult to distinguish them with conventional convolutional neural networks (CNNs). Therefore, the waterfall image is regarded as time-series data on the y-axis, and the relationships within the pattern are examined based on the order of occurrence of the events.

[0083] To achieve this, a Temporal Relationship Network (TRN) module is employed that enables temporal relationship inference between different time frames of the waterfall sensing data.

[0084] In operation, instead of a normal two-dimensional image, each waterfall image is captured as time-series data, and the time information in the data is explicitly used. That is, the waterfall image is regarded as an ID "video". Since the y-axis of the image is time, it naturally becomes the form of (multivariate) time-series data. As a result, several techniques such as video analysis technology can be adopted. This includes using data sampled from combinations of multiple time points as model inputs. Therefore, many combinations of rows can be generated without duplication, reducing the problem of data shortage and having advantages for model training processing. At the same time, it can cope with the variations in the sensing rates of various DAS sensors and various periods of the passage of events over time.

[0085] Another technique that can be used includes exploring the temporal relationship between the features of different frames. In this example, the relationship between the features (time stamps) of multiple rows is analyzed. A TRN module that captures the temporal relationship between multiple ordered rows is adopted. This module can also be applied to other event detection problems based on waterfall images and can improve the accuracy.

[0086] Instead of using the raw pixels of the waterfall image as features, a convolutional neural network is applied as an encoder to extract the features within each row of the waterfall data. Regarding the sensing data at each time point as an ID image, an encoder is constructed using the ID instead of a two-dimensional convolutional layer. If the manhole is different, different-width vibration patterns may be generated depending on the amount of internal slack fibers. Therefore, zeros are padded into each row to obtain the same length of input. Without performing image conversion, the underlying event structure is maintained.

[0087] FIG. 13 is a schematic flow chart showing an exemplary and overall manhole intrusion detection process by a DFOS system (DFOS / DAS / DVS) according to an aspect of the present disclosure.

[0088] FIG. 14 is a schematic diagram showing an exemplary generated waterfall image according to an aspect of the present disclosure.

[0089] Note that since the opening and closing events are continuous operations, it is natural to investigate the basic correlation between different timestamps encoded in the rows of the waterfall image as shown in FIG. 14. Assume that the time window is N, that is, there are N rows in the image. As the event progresses from row j to row i, the corresponding vibration pattern also changes in the same order. For example, when the event is "open", there are several vibration patterns corresponding to lifting the manhole, and then there may be a pattern representing the cover contacting the ground. The inherent relationship within the action also exists in the "closed" event (aligning the position before placing the manhole cover). To model the temporal relationship between observations in the waterfall image, the TRN module is adopted. The temporal relationship for each pair is

Number

[0090] Here, the input is a waterfall image I having n ordered rows selected as I = {f1, f2, …, f n}, and f i represents the i-th row of the image. The functions h φ and g θ fuse the features of rows in different orders. A multi-layer perceptron (MLP) with parameters φ and θ respectively is used. This definition can be further extended to higher-order row relationships such as the 3-row relation function

Number

[0091] When the event is completed and cannot be captured in a single-scale relationship, the following function can be used to accumulate relationships at different scales.

Number

[0092] Here, Td captures the temporal relationships between d ordered rows. All the relational functions are end-to-end trainable using a base CNN.

[0093] FIG. 15 is a schematic diagram showing the configuration of a manhole opening / closing event detection system and sample waterfall data of the opening / closing of a manhole cover or a normal background. A distributed acoustic sensor (DAS) is connected to an existing deployed fiber network and monitors vibrations along it. This is installed in a control room / central office and is equipped with an AI engine based on a temporal relationship event detection algorithm. This system can continuously sense and analyze related events in real time.

[0094] In the waterfall data, it can be seen that due to differences such as the shape, weight, and type of the ground surface of the cover, the shapes and intensities of the vibration patterns from different manholes are significantly different. On the other hand, although the opening / closing events seem similar, due to the actions of different operators, the time course of the events is different. Intra-class heterogeneity and intra-class similarity are characteristics of the main technical challenges of machine learning. Conventional CNN-based models can effectively detect manhole intrusion events from the background but cannot distinguish between opening and closing. In practice, accurately detecting manhole opening events is an important function for preventing theft of manhole covers.

[0095] FIG. 16 shows a flowchart of an overall processing procedure including training and inference stages according to an aspect of the present disclosure. In the training stage, the first step is to obtain training patches with specific spatial and temporal resolutions. To do this, filters are applied to smooth the signal and candidate high-vibration patches are selected based on intensity. Next, the selected patches are transformed with random perturbations in position and pixel space. According to the conventional supervised training procedure, the TRN model is trained using pairs of labels and patches.

[0096] In the inference stage, the input waterfall image is converted into overlapping local patches by a sliding window. Next, the patches are classified by the trained TRN model to determine whether there are opening and closing events within the patches. The AI engine obtains a continuous stream of input data, performs inference in real time on the GPU, and provides the timestamp, event type, and confidence score as outputs.

[0097] Figure 17 shows the overall architecture of the TRN-equipped fiber sensing event classification model according to an aspect of the present disclosure. For each input of a waterfall patch, rows are randomly sampled when the event signal occurs on the y-axis. The number of rows is a hyperparameter of the model. The classification model consists of a 1D convolutional layer and a TRN module. The convolutional layer is used to extract features of each row along the x-axis. These features can encode features corresponding to the intensity along the x-axis. The next TRN module can summarize and sequentially transform and consider the features of the rows. The features generated from the TRN module can describe the progress of the event in the time dimension. The output of the TRN module is the probability that the input is each category, and the final prediction is the one with the highest value. The entire model is trained with regular cross-entropy loss by a stochastic gradient descent (SGD) optimizer.

[0098] Figure 18 is a schematic diagram showing details of the two-frame time relationship module. f represents the features from the previous convolutional layer block according to an aspect of the present disclosure. g θAn MLP block consisting of two fully connected layers is adopted. The first layer has 512 units following the ReLU activation function for non-linearity. The number of units in the second layer matches the class number. h φ For h, simply the identity function is used. In the case of more complex events existing within the image, in combination with the multi-scale implemented by adding the next block using Equation (3), h φ can be replaced with an MLP block.

[0099] When training the multi-scale temporal network, a cascade sampling method is adopted to improve efficiency. (1) Sample N rows uniformly from N segments along the y-axis. (2) For each d < N, select k random samples of d frames to calculate d-frame relationships. This enables kN temporal relationships using only N frames. Since N varies for each epoch, the model tends to recognize events by combinations of rows from N uniform segments. This significantly speeds up the training procedure compared to using the entire image.

[0100] As will be understood and recognized by those skilled in the art, the disclosure of the present invention provides a solution for detecting manhole opening and closing events in real time using pre-deployed optical cables. By using fiber optic sensing technology leveraging the AI deep learning-based algorithm of the present invention, a complete solution is provided without installing additional sensors. However, since the waterfall images of the two generated events are similar in appearance, it is difficult to perform this task using a conventional convolutional neural network. Instead, utilize the unique characteristics along the time dimension as the event progresses, independent of external factors. This is achieved by using the temporal relationship after a one-dimensional convolutional layer. Since the input is one-dimensional (a row instead of a two-dimensional image), the design of the convolutional layer block is one-dimensional. The end-to-end trainable model takes a selected representative row as input and outputs a prediction by selecting the category with the highest probability. Enabling temporal relationship inference with fiber optic sensing data for the first time aids in fine-grained event recognition such as the opening and closing of manhole covers based on DAS fiber optic sensing.

[0101] FIG. 19 is a schematic block diagram showing the features of manhole detection according to the present invention according to an aspect of the present disclosure.

[0102] At this point, several specific examples have been used to present the present disclosure, but those skilled in the art will recognize that the teachings of the present invention are not limited thereto. Therefore, the present disclosure should be limited only by the appended claims.

Claims

1. An optical fiber sensor cable, A DFO S interrogator system that optically communicates with the optical fiber sensor cable, An intelligent analyzer configured to analyze DFO S sensing data received by the DFO S interrogator system, a distributed optical fiber sensing (DFO S) system comprising: An outdoor fiber cabinet disposed in the optical path of the optical fiber sensor cable, the outdoor fiber cabinet being configured to amplify an acoustic vibration wave and mechanically transmit the acoustic vibration wave to the optical fiber sensor cable such that the acoustic vibration signal is detected by the intelligent analyzer, The outdoor fiber cabinet includes a door, a fiber coil formed from the optical fiber sensor cable is attached to the door, and the door is configured as a fiber microphone / amplifier, a distributed optical fiber sensing (DFO S) system.

2. The system according to claim 1, wherein the door configured as a fiber microphone / amplifier includes an interaction region that interacts with the acoustic vibration wave.

3. The system according to claim 1, wherein the intelligent analyzer is configured to detect whether the door is open or closed.

4. The system according to claim 3, wherein the intelligent analyzer is configured to detect whether the open / closed state of the door is permitted.

5. The system according to claim 4, wherein the door is made of metal.

6. The system according to claim 5, wherein the optical fiber sensor cable transmits electrical communication traffic simultaneously with any DFO S signal.

Images