Optical fiber anomaly detection method and device based on deep learning, and storage medium

Abstract

The invention discloses an optical fiber anomaly detection method and device based on deep learning and a storage medium, and relates to the technical field of machine learning, and the method comprises the steps: obtaining the temperature change data of a test optical fiber; preprocessing the test optical fiber temperature change data to obtain preprocessed test optical fiber temperature change data; the preprocessed test optical fiber temperature change data serve as input, data reconstruction output is carried out based on an optical fiber anomaly detection model, abnormal temperature change data in the test optical fiber temperature change data are obtained, and the optical fiber anomaly detection model is an auto-encoder. Abnormal temperature change data are screened out according to reconstruction errors by using a deep learning model, signal features reflecting abnormal phenomena in the optical fiber can be rapidly and accurately detected, the abnormal occurrence position of the optical fiber can be accurately positioned under dangerous conditions such as tunnel fire and slope slippage, an alarm is given in time, and the safety of the optical fiber is improved. And a decision basis is provided for quick response and seasonal processing.

Description

Technical Field

[0001] The present invention relates to a method, device and storage medium for detecting anomalies in an optical fiber based on deep learning, and belongs to the technical field of machine learning. Background Art

[0002] Brillouin scattering spectroscopy is an optical analysis method based on the interaction between light and acoustic waves. It can reveal the physical properties of a medium by measuring the frequency shift of scattered light. In optical fibers, Brillouin scattering is affected by temperature and strain, so the frequency shift directly reflects changes in these parameters. This technology is widely used in distributed fiber optic sensing systems, enabling high-precision monitoring of temperature, pressure, and structural strain. It is particularly valuable in fields such as civil engineering, energy pipelines, and aerospace. As a non-destructive testing tool, Brillouin scattering spectroscopy provides strong support for materials science and optical communications research.

[0003] Brillouin scattering technology has significant practical value in high-speed rail and subway tunnels. Due to the narrow and long tunnels and the complex geological environment along the railway, tunnels often face potential risks such as slope slippage, rockfall, and even fires. These issues pose a significant threat to traffic safety and operational efficiency. Brillouin fiber optic sensing technology enables real-time distributed monitoring of temperature and strain inside tunnels, particularly providing early warning of heat sources. If the temperature inside the tunnel rises abnormally, the system can quickly locate the location and issue a warning signal, buying valuable time for evacuation and firefighting.

[0004] Currently, the main methods for fiber optic anomaly detection include optical time-domain reflectometer (OTDR), optical power monitoring, and distributed temperature sensing / distributed acoustic sensing (DTS / DAS). Although these methods have achieved certain results in practical applications, they still have problems such as limited detection accuracy, insufficient real-time performance, high false alarm rate, poor adaptability to complex environments, and lack of intelligent analysis. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a fiber optic anomaly detection method, device and storage medium based on deep learning. The deep learning model is used to filter out abnormal temperature change data according to the reconstruction error, which can quickly and accurately detect the signal characteristics reflecting abnormal phenomena in the optical fiber. In dangerous situations such as tunnel fires and slope slippage, the location of the optical fiber anomaly can be accurately located, and an alarm can be issued in time, providing a decision-making basis for rapid response and seasonal processing.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions: In a first aspect, the present invention provides a method for detecting optical fiber anomalies based on deep learning, comprising: Obtain test optical fiber temperature change data; Preprocessing the test optical fiber temperature variation data to obtain preprocessed test optical fiber temperature variation data; The pre-processed test optical fiber temperature variation data is used as input, and the data is reconstructed and output based on the optical fiber anomaly detection model to obtain the reconstructed test optical fiber temperature variation data; The optimal anomaly detection threshold is obtained by setting the threshold of the reconstructed test optical fiber temperature change data; Comparing the reconstructed test optical fiber temperature change data with the optimal anomaly detection threshold to obtain abnormal temperature change data; The test optical fiber position corresponding to the abnormal temperature change data is used as the test optical fiber abnormal position; The optical fiber anomaly detection model is an autoencoder, which includes an input layer, an encoder, a decoder, and an output layer connected in sequence.

[0007] Furthermore, obtaining the test optical fiber temperature variation data includes: dividing the test optical fiber into a plurality of test optical fiber segments of equal length; Sending a laser sending instruction to the BOTDR system so that the BOTDR system sends a narrow pulse laser into the test optical fiber segment to generate a Brillouin scattering signal; Sending a signal capture instruction to the receiver so that the receiver captures the reflected light of the Brillouin scattering signal and converts it into an electrical signal and transmits it to the BOTDR system; receiving a Brillouin scattering spectrum of the test optical fiber segment obtained by converting an electrical signal by the BOTDR system using a short-time Fourier transform, wherein the Brillouin scattering spectrum is composed of a plurality of Brillouin frequency shift values; The Brillouin scattering spectra of all tested fiber segments constitute the fiber temperature variation data.

[0008] Furthermore, the test optical fiber temperature variation data is stored via an Excel file, wherein each column in the Excel file represents a Brillouin scattering spectrum of a test optical fiber segment.

[0009] Furthermore, the preprocessing of the test optical fiber temperature variation data to obtain the preprocessed test optical fiber temperature variation data includes: Traverse the test fiber temperature variation data. If there are missing values, delete the test fiber temperature variation data where the missing values are located or use interpolation to fill them in, and obtain preliminary test fiber temperature variation data. Normalize the preliminary test fiber temperature variation data. The normalization is to map the Brillouin scattering spectrum data of each test fiber segment to [0, 1]. The expression is as follows: ; in, represents the normalized Brillouin scattering spectrum data of the i-th test fiber segment, represents the original Brillouin scattering spectrum data of the i-th test fiber segment, express The minimum value in express The maximum value in ; The normalized Brillouin scattering spectrum data of all test optical fiber segments constitute the preprocessed test optical fiber temperature variation data.

[0010] Furthermore, the Brillouin scattering spectrum is composed of a plurality of Brillouin frequency shift values, and the sizes of the input layer and the output layer correspond to the number of Brillouin scattering frequency shift values contained in the pre-processed test optical fiber temperature variation data; The encoder includes multiple linear layers connected in sequence, and the linear layers are used to gradually reduce the dimension of the input data. The decoder includes multiple fully connected layers connected in sequence, and the fully connected layers are used to gradually restore the data dimension after the dimension reduction to the original data size. The linear layer and the fully connected layer both use the ReLU function as the activation function.

[0011] Furthermore, the threshold setting of the reconstructed test optical fiber temperature variation data to obtain the optimal anomaly detection threshold includes: Comparing the reconstructed test fiber temperature variation data with the test fiber temperature variation data to obtain the reconstruction error of each test fiber temperature variation data; The Gaussian mixture model is used to model the probability density of the reconstruction error of the temperature variation data of each test fiber. The expression is: ; in, represents the reconstruction error of the temperature variation data of the i-th test fiber segment The probability density function of represents the kth Gaussian distribution, represents the mean of the k-th Gaussian distribution, represents the standard deviation of the k-th Gaussian distribution, represents the kth weight; The optimal anomaly detection threshold is determined using maximum likelihood estimation, and its expression is: ; in, represents the optimal anomaly detection threshold, Represents the 5% quantile of the reconstruction error probability density.

[0012] Furthermore, the method further includes pre-training the optical fiber anomaly detection model, wherein the pre-training method includes: S1. Obtain a data set of normal optical fiber temperature variation and a data set of abnormal optical fiber temperature variation; S2. Divide the optical fiber normal temperature variation dataset into a training set, a validation set, and a partial test set, and combine the optical fiber abnormal temperature variation dataset and the partial test set into a test set; S3, initializing the optical fiber anomaly detection model; S4. Using the training set data as input, training the optical fiber anomaly detection model, adjusting the model parameters using a loss function during the training process, and adjusting the learning rate using an Adam optimizer; S5. After each round of training, the fiber anomaly detection model is validated using the validation set data as input to obtain validation results, and the number of training rounds and batch size are adjusted according to the validation results; S6. Repeat steps S4 to S6 until the training set loss no longer decreases or the validation set loss does not improve within 5 rounds. The training is terminated, and the model parameters corresponding to the minimum validation set loss are taken as the optimal model parameters to obtain a preliminary optical fiber anomaly detection model. S7. Use the test set data as input to test the preliminary optical fiber anomaly detection model to obtain test results. Calculate an evaluation index based on the test results. If the evaluation index is higher than a preset value, the training ends and the pre-trained optical fiber anomaly detection model is obtained. If the evaluation index is lower than the preset value, repeat S4 to S7 until the evaluation index is higher than the preset value and the pre-trained optical fiber anomaly detection model is obtained.

[0013] Furthermore, the initializing the optical fiber anomaly detection model includes setting a batch size and a number of training rounds; The loss function adopts mean square error, which is expressed as: ; in, represents the mean square error loss function, Indicates the total number of data, represents the i-th input data, Represents the i-th output data; The evaluation indicators include AUC value, accuracy and false alarm rate.

[0014] In a second aspect, the present invention further provides an optical fiber anomaly detection device based on deep learning, comprising: An optical fiber temperature variation data acquisition module is configured to acquire the temperature variation data of the test optical fiber; A data preprocessing module is configured to preprocess the test optical fiber temperature variation data to obtain preprocessed test optical fiber temperature variation data; The optical fiber abnormal temperature variation data detection module is configured to take the pre-processed test optical fiber temperature variation data as input, reconstruct the data based on the optical fiber abnormality detection model, and output the reconstructed test optical fiber temperature variation data; An optimal anomaly detection threshold acquisition module is configured to set a threshold for the reconstructed test optical fiber temperature change data to obtain an optimal anomaly detection threshold; The abnormal temperature variation data acquisition module is configured to compare the reconstructed test fiber temperature variation data with the optimal abnormality detection threshold to obtain abnormal temperature variation data, and use the test fiber position corresponding to the abnormal temperature variation data as the test fiber abnormal position.

[0015] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the optical fiber anomaly detection method based on deep learning as described in any one of the above items is implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention utilizes the fact that the reconstruction error of normal optical fiber temperature variation data is small, while the reconstruction error of abnormal optical fiber temperature variation data is large. It detects the test optical fiber temperature variation data based on an autoencoder, and screens out abnormal temperature variation data based on the reconstruction error. This can quickly and accurately detect the signal characteristics reflecting abnormal phenomena in the optical fiber. In dangerous situations such as tunnel fires and slope slippage, it can accurately locate the location of optical fiber anomalies and issue alarms in time, providing a decision-making basis for rapid response and seasonal processing. This not only improves the intelligent level of tunnel safety management, but also significantly reduces the accident rate and losses, providing more reliable safety protection for critical infrastructure. The fiber anomaly detection method of the present invention leverages the efficient computing power of deep learning algorithms to achieve real-time processing and analysis of fiber optic signals. The system can issue early warnings, buying valuable time for emergency responses (such as fire fighting and landslide disposal), reducing accident losses, and supports long-distance, multi-point distributed fiber optic sensing, making it suitable for large-scale safety monitoring in scenarios such as subways and high-speed railways. Through a distributed architecture, it overcomes the efficiency bottleneck of traditional centralized processing, making full-line monitoring more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flow chart of a method for detecting optical fiber anomalies based on deep learning in an embodiment of the present invention; Figure 2 Schematic diagram of the structure of an optical fiber anomaly detection model of an optical fiber anomaly detection method based on deep learning in one embodiment of the present invention; Figure 3 Schematic diagram showing the comparison of AUC values using an adaptive threshold and a fixed threshold in an optical fiber anomaly detection method based on deep learning in one embodiment of the present invention; Figure 4 Schematic diagram comparing the accuracy of an optical fiber anomaly detection method based on deep learning using an adaptive threshold and a fixed threshold in one embodiment of the present invention; Figure 5 The figure is a schematic diagram comparing the false alarm rates of an optical fiber anomaly detection method based on deep learning using an adaptive threshold and a fixed threshold in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Example 1

[0019] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting optical fiber anomalies based on deep learning, which specifically includes the following steps: First, obtain the test fiber temperature change data: The test fiber is divided into multiple test fiber segments. It should be noted that the length of each test fiber segment is equal.

[0020] A BOTDR system using STFT (Short-Time Fourier Transform) technology measures the Brillouin scattering spectrum at different locations on an optical fiber. Specifically, the BOTDR system first transmits a narrow laser pulse into a test fiber segment. As the light pulse propagates along the test fiber, it interacts with phonons in the segment, generating a Brillouin scattering signal. A receiver then captures the reflected light from the Brillouin scattering signal and converts it into an electrical signal. To extract useful information, the BOTDR system uses STFT to analyze the electrical signal, obtaining the spectral distribution of the signal at different times and locations, known as the Brillouin scattering spectrum.

[0021] In this embodiment, the test optical fiber is divided into 7991 segments of equal length. Therefore, 7991 Brillouin scattering spectra are obtained by using the STFT BOTDR system. By adjusting the sampling resolution, each Brillouin scattering spectrum corresponds to 614 Brillouin frequency shift values. Therefore, the test optical fiber temperature variation data of this embodiment are the Brillouin scattering spectra of the 7991 test optical fiber segments, and each Brillouin scattering spectrum includes 614 Brillouin frequency shift values.

[0022] The acquired test optical fiber temperature variation data is stored in an Excel file, and each column in the Excel file corresponds to the Brillouin scattering spectrum of a test optical fiber segment.

[0023] Next, the test optical fiber temperature variation data is preprocessed to obtain preprocessed test optical fiber temperature variation data: Read the test fiber temperature change data from the Excel file.

[0024] Perform data cleaning on the test fiber temperature variation data. If there are missing values in the data, you can choose to delete the missing values or use interpolation to fill them. In this example, the temperature variation data was reduced from 7991 columns to 6000 columns, and invalid fiber segments were deleted to improve detection efficiency.

[0025] Normalize the temperature variation data of the test fiber after data cleaning (data standardization). This is because the subsequent steps are sensitive to the scale of the input data. Therefore, it is necessary to map the Brillouin scattering spectrum data of each test fiber segment to [0,1] through normalization to reduce the interference of irrelevant information. The expression is as follows: ; in, represents the normalized Brillouin scattering spectrum data of the i-th test fiber segment, represents the original Brillouin scattering spectrum data of the i-th test fiber segment, express The minimum value in express The maximum value in .

[0026] The normalized Brillouin scattering spectrum data of all test optical fiber segments constitute the preprocessed test optical fiber temperature variation data.

[0027] Build a fiber anomaly detection model.

[0028] The fiber anomaly detection model in this embodiment uses an autoencoder, a neural network model that learns a low-dimensional representation of input data. It maps the input data into a latent space and then reconstructs the input data from the latent space to learn a compressed representation of the data.

[0029] For optical fiber temperature variation data, under normal circumstances, the reconstruction error of normal optical fiber temperature variation data is small, while the reconstruction error of abnormal temperature variation data is large due to the difference in distribution with the training set.

[0030] like Figure 2 As shown, the optical fiber anomaly detection model consists of an input layer, an encoder, a decoder, and an output layer connected in sequence. The size of the input layer and the output layer is the same as the number of Brillouin scattering RF shift values contained in the Brillouin scattering spectrum of the test optical fiber segment. Therefore, since each test optical fiber segment in this embodiment contains 614 Brillouin scattering RF shift values, the size of the input layer and the output layer is 614.

[0031] The encoder compresses the input data into a lower-dimensional representation, extracting key features of the data. The encoder consists of several linear layers (four in this example), which gradually reduce the dimensionality of the input data. The first layer reduces the dimensionality from 614 to 512, the second layer from 512 to 256, and the third layer from 256 to 128, resulting in a latent space of 64 dimensions.

[0032] The decoder will restore the dimension of the latent space to the dimension of the original data. The structure of the decoder is the inverse process of the encoder. The decoder gradually restores the low-dimensional data in the latent space to the dimension of the input data through 4 fully connected layers. Therefore, in this embodiment, the first layer of the encoder restores the dimension from 64 to 128, the second layer restores the dimension from 128 to 256, the third layer restores the dimension from 256 to 512, and the last layer restores it to 614, that is, the dimension of the original data.

[0033] After building the fiber anomaly detection model, it needs to be pre-trained. The pre-training methods include: S1. Obtain a normal optical fiber temperature variation dataset and an abnormal optical fiber temperature variation dataset.

[0034] S2. Divide the normal optical fiber temperature variation dataset into a training set, a validation set, and a partial test set, and combine the abnormal optical fiber temperature variation dataset and the partial test set into a test set. In this embodiment, the training set is set with 4000 optical fiber segments, the validation set is set with 1000 optical fiber segments, and the test set is set with 1000 optical fiber segments.

[0035] S3. Initialize the fiber anomaly detection model and select the Adam optimizer, an adaptive gradient descent optimization method that can automatically adjust the learning rate according to the sparsity of the parameters. Set the loss function. Since fiber anomaly detection is a regression problem and the goal of the autoencoder is to minimize the error between the input data and the reconstructed data, the mean square error (MSE) is selected as the loss function. Its expression is: ; in, represents the mean square error loss function, Indicates the total number of data, represents the i-th input data, Represents the i-th output data.

[0036] S4. Use the training set data as input to train the fiber anomaly detection model. During the training process, use the loss function to adjust the model parameters and adjust the learning rate through the Adam optimizer.

[0037] S5. After each round of training, the validation set data is used as input to validate the optical fiber anomaly detection model to obtain a validation result. The number of training rounds and the batch size are adjusted according to the validation result. In this embodiment, the initial number of training rounds is set to 50 and the batch size is set to 64.

[0038] S6. Repeat steps S4 to S6 to prevent overfitting during the training process until the training set loss no longer decreases or the validation set loss does not improve within 5 rounds. The model parameters corresponding to the minimum validation set loss are used as the optimal model parameters to obtain a preliminary fiber anomaly detection model and determine the reconstruction error threshold.

[0039] S7: Using the test set data as input, the preliminary fiber anomaly detection model is tested to obtain test results, and evaluation metrics are calculated based on the test results. In this embodiment, the evaluation metrics include ROC curves, precision, recall, and F1-score. If the evaluation metrics are higher than preset values, training is completed, resulting in a pre-trained fiber anomaly detection model. If the evaluation metrics are lower than the preset values, S4-S7 are repeated until the evaluation metrics are higher than the preset values, resulting in a pre-trained fiber anomaly detection model.

[0040] The preprocessed test fiber temperature variation data is used as input, and data reconstruction is performed based on the pre-trained fiber anomaly detection model to obtain the reconstructed test fiber temperature variation data.

[0041] The optimal anomaly detection threshold is obtained by thresholding the reconstructed test fiber temperature variation data. In this embodiment, to improve the robustness of the fiber anomaly detection method, the GMM-MLE adaptive threshold setting method is used to replace the traditional fixed quantile threshold setting method. Specifically, the following method is used: The reconstructed test optical fiber temperature variation data is compared with the test optical fiber temperature variation data to obtain the reconstruction error of the temperature variation data of each test optical fiber segment.

[0042] The Gaussian mixture model (GMM) is used to perform probability density modeling on the reconstruction error of the temperature variation data of each test fiber segment, and its expression is: ; in, represents the reconstruction error of the temperature variation data of the i-th test fiber segment The probability density function of represents the kth Gaussian distribution, represents the mean of the k-th Gaussian distribution, represents the standard deviation of the k-th Gaussian distribution, represents the kth weight.

[0043] The maximum likelihood estimation (MLE) is used to automatically determine the optimal anomaly detection threshold, which is expressed as: ; in, represents the optimal anomaly detection threshold, Represents the 5% quantile of the reconstruction error probability density.

[0044] Finally, the reconstructed test fiber temperature change data is compared with the optimal anomaly detection threshold to obtain the abnormal temperature change data. The test fiber segment corresponding to the abnormal temperature change data is found, and the abnormal test fiber segment in the test fiber can be determined, thereby determining the location where the anomaly occurs.

[0045] This embodiment uses two methods, GMM-MLE adaptive threshold setting and artificial fixed threshold setting (fixed 95% threshold) to detect optical fiber anomalies. The comparison results of the two methods are shown in Figure 2. Figure 3 、 Figure 4 and Figure 5 As shown, Figure 3 、 Figure 4 and Figure 5 The figures are the comparison results of the AUC value graph, the accuracy graph, and the false alarm rate. It can be seen very intuitively from the figures that the GMM-MLE adaptive threshold setting method proposed in this embodiment has higher AUC value and accuracy and lower false alarm rate than the fixed 95% threshold setting. This effectively illustrates that the use of adaptive threshold setting can improve detection accuracy and adaptability.

[0046] This paper proposes a new method for optical fiber anomaly detection based on deep learning using an autoencoder. Compared to traditional detection methods that rely on the limitations of linear feature extraction and empirical threshold setting, this method creatively proposes a complete anomaly detection system with unsupervised feature learning as its core and optimized for the characteristics of Brillouin scattering spectrum data. This system can more accurately capture nonlinear features in high-dimensional complex data and automatically identify abnormal states. It is particularly suitable for actual optical fiber monitoring scenarios where abnormal samples are scarce, significantly improving anomaly detection accuracy and real-time response capabilities, and reducing false alarm rates. This method comprehensively utilizes the self-supervisory characteristics of deep learning to effectively improve the accuracy and real-time performance of optical fiber anomaly detection, and has the following outstanding features and innovations: First, to address the challenges of Brillouin scattering spectrum data, which is characterized by high dimensionality, complex features, and unclear anomaly signatures, the autoencoder effectively maps the raw data into a latent low-dimensional feature space through an encoder, significantly compressing the data dimension while preserving its key features. Unlike traditional linear dimensionality reduction methods, the autoencoder automatically learns the inherent nonlinear characteristic structure of the data, more precisely capturing subtle changes caused by anomalies and addressing the limited detection accuracy of traditional methods in complex environments.

[0047] Secondly, the decoder detects anomalies by reconstructing the original data from the latent feature space. This paper innovatively proposes using reconstruction error as an anomaly discrimination metric. This strategy designs an anomaly threshold based on the different distribution characteristics of normal and anomaly data in the latent space. Unlike the subjectivity of traditional threshold setting, this method adaptively determines the optimal anomaly threshold by analyzing the distribution of reconstruction error in normal data, significantly improving the model's accuracy and reliability.

[0048] Furthermore, the present invention incorporates a refined hyperparameter tuning mechanism during the training and validation of the autoencoder model, including parameters such as the number of training epochs and batch size. This, combined with real-time monitoring of the loss function, ensures a balance between the model's training accuracy and generalization capabilities. This refined tuning mechanism effectively addresses the overfitting issue that can easily occur in actual fiber optic monitoring, ensuring the model's stability and robustness in practical applications.

[0049] Furthermore, considering the urgent need for real-time detection capabilities in actual production environments, the autoencoder model designed in this invention features a simple structure and efficient computation, enabling rapid deployment on-site, enabling real-time monitoring and rapid response to fiber anomalies. This model is particularly valuable in situations where abnormal data samples are scarce, as it requires only normal data for effective training.

[0050] In summary, this paper systematically applies the autoencoder method to the field of optical fiber anomaly detection for the first time. It innovatively develops a comprehensive and innovative theoretical and application system based on multiple dimensions, including data feature compression extraction, reconstruction error anomaly identification mechanism, adaptive threshold optimization, and low data requirements. This research not only significantly improves the accuracy and sensitivity of anomaly detection, but also provides an important theoretical basis and practical reference for the further development of intelligent optical fiber monitoring technology. Example 2

[0051] Based on Example 1, this embodiment provides an optical fiber anomaly detection device based on deep learning, including: An optical fiber temperature variation data acquisition module is configured to acquire the temperature variation data of the test optical fiber; A data preprocessing module is configured to preprocess the test optical fiber temperature variation data to obtain preprocessed test optical fiber temperature variation data; The optical fiber abnormal temperature variation data detection module is configured to take the pre-processed test optical fiber temperature variation data as input, reconstruct the data based on the optical fiber abnormality detection model, and output the reconstructed test optical fiber temperature variation data; An optimal anomaly detection threshold acquisition module is configured to set a threshold for the reconstructed test optical fiber temperature change data to obtain an optimal anomaly detection threshold; The abnormal temperature variation data acquisition module is configured to compare the reconstructed test fiber temperature variation data with the optimal abnormality detection threshold to obtain abnormal temperature variation data, and use the test fiber position corresponding to the abnormal temperature variation data as the test fiber abnormal position. Example 3

[0052] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the optical fiber anomaly detection method based on deep learning of embodiment 1 is implemented.

[0053] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A fiber anomaly detection method based on deep learning, characterized in that: include: Obtain test optical fiber temperature change data; Preprocessing the test optical fiber temperature variation data to obtain preprocessed test optical fiber temperature variation data; The pre-processed test optical fiber temperature variation data is used as input, and the data is reconstructed and output based on the optical fiber anomaly detection model to obtain the reconstructed test optical fiber temperature variation data; The optimal anomaly detection threshold is obtained by setting the threshold of the reconstructed test optical fiber temperature change data; Comparing the reconstructed test optical fiber temperature change data with the optimal anomaly detection threshold to obtain abnormal temperature change data; The test optical fiber position corresponding to the abnormal temperature change data is used as the test optical fiber abnormal position; The optical fiber anomaly detection model is an autoencoder, which includes an input layer, an encoder, a decoder, and an output layer connected in sequence.

2. The optical fiber anomaly detection method based on deep learning according to claim 1, characterized in that: The obtaining of the test optical fiber temperature variation data includes: dividing the test optical fiber into a plurality of test optical fiber segments of equal length; Sending a laser sending instruction to the BOTDR system so that the BOTDR system sends a narrow pulse laser to the test optical fiber segment to generate a Brillouin scattering signal; Sending a signal capture instruction to a receiver, so that the receiver captures the reflected light of the Brillouin scattering signal and converts it into an electrical signal and transmits it to the BOTDR system; receiving a Brillouin scattering spectrum of the test optical fiber segment obtained by converting the electrical signal by the BOTDR system using short-time Fourier transform; The Brillouin scattering spectra of all test fiber segments constitute the test fiber temperature variation data.

3. The optical fiber anomaly detection method based on deep learning according to claim 2, characterized in that: The test optical fiber temperature variation data is stored via an Excel file, wherein each column in the Excel file represents a Brillouin scattering spectrum of a test optical fiber segment.

4. The optical fiber anomaly detection method based on deep learning according to claim 1, characterized in that: The preprocessing of the test optical fiber temperature variation data to obtain the preprocessed test optical fiber temperature variation data includes: Traverse the test fiber temperature variation data. If there are missing values, delete the test fiber temperature variation data where the missing values are located or use interpolation to fill them in, and obtain preliminary test fiber temperature variation data. Normalize the preliminary test fiber temperature variation data. The normalization is to map the Brillouin scattering spectrum data of each test fiber segment to [0, 1]. The expression is as follows: ； in, represents the normalized Brillouin scattering spectrum data of the i-th test fiber segment, represents the original Brillouin scattering spectrum data of the i-th test fiber segment, express The minimum value in express The maximum value in ; The normalized Brillouin scattering spectrum data of all test optical fiber segments constitute the preprocessed test optical fiber temperature variation data.

5. The optical fiber anomaly detection method based on deep learning according to claim 2, characterized in that: The Brillouin scattering spectrum of the test optical fiber segment is composed of a plurality of Brillouin frequency shift values, and the sizes of the input layer and the output layer correspond to the number of Brillouin scattering frequency shift values included in the preprocessed test optical fiber temperature variation data; The encoder includes multiple linear layers connected in sequence, and the linear layers are used to gradually reduce the dimension of the input data. The decoder includes multiple fully connected layers connected in sequence, and the fully connected layers are used to gradually restore the data dimension after the dimension reduction to the original data size. The linear layer and the fully connected layer both use the ReLU function as the activation function.

6. The optical fiber anomaly detection method based on deep learning according to claim 1, characterized in that: The threshold setting of the reconstructed test optical fiber temperature change data to obtain the optimal anomaly detection threshold includes: Comparing the reconstructed test fiber temperature variation data with the test fiber temperature variation data to obtain a reconstruction error of the temperature variation data of each test fiber segment; The Gaussian mixture model is used to model the probability density of the reconstruction error of the temperature variation data of each test fiber segment, and its expression is: ； in, represents the reconstruction error of the temperature variation data of the i-th test fiber segment The probability density function of represents the kth Gaussian distribution, represents the mean of the k-th Gaussian distribution, represents the standard deviation of the k-th Gaussian distribution, represents the kth weight; The optimal anomaly detection threshold is determined using maximum likelihood estimation, and its expression is: ; in, represents the optimal anomaly detection threshold, Represents the 5% quantile of the reconstruction error probability density.

7. The optical fiber anomaly detection method based on deep learning according to claim 1, characterized in that: The method further includes pre-training the optical fiber anomaly detection model, wherein the pre-training method includes: S1. Obtain a data set of normal optical fiber temperature variation and a data set of abnormal optical fiber temperature variation; S2. Divide the optical fiber normal temperature variation dataset into a training set, a validation set, and a partial test set, and combine the optical fiber abnormal temperature variation dataset and the partial test set into a test set; S3, initializing the optical fiber anomaly detection model; S4. Using the training set data as input, training the optical fiber anomaly detection model, adjusting the model parameters using a loss function during the training process, and adjusting the learning rate using an Adam optimizer; S5. After each round of training, the fiber anomaly detection model is validated using the validation set data as input to obtain validation results, and the number of training rounds and batch size are adjusted according to the validation results; S6. Repeat steps S4 to S6 until the loss function value of the training set no longer decreases or the loss function value of the validation set does not improve within 5 rounds, and then terminate the training. The model parameters corresponding to the minimum loss function value of the validation set are taken as the optimal model parameters to obtain a preliminary optical fiber anomaly detection model. S7. Use the test set data as input to test the preliminary optical fiber anomaly detection model to obtain test results. Calculate an evaluation index based on the test results. If the evaluation index is higher than a preset value, the training ends and the pre-trained optical fiber anomaly detection model is obtained. If the evaluation index is lower than the preset value, repeat S4 to S7 until the evaluation index is higher than the preset value and the pre-trained optical fiber anomaly detection model is obtained.

8. The optical fiber anomaly detection method based on deep learning according to claim 7, characterized in that: Initializing the optical fiber anomaly detection model includes setting a batch size and a number of training rounds; The loss function adopts mean square error, which is expressed as: ； in, represents the mean square error loss function, Indicates the total number of data, represents the i-th input data, Represents the i-th output data; The evaluation indicators include AUC value, accuracy and false alarm rate.

9. A fiber anomaly detection device based on deep learning, characterized in that: include: An optical fiber temperature variation data acquisition module is configured to acquire the temperature variation data of the test optical fiber; A data preprocessing module is configured to preprocess the test optical fiber temperature variation data to obtain preprocessed test optical fiber temperature variation data; The optical fiber abnormal temperature variation data detection module is configured to take the pre-processed test optical fiber temperature variation data as input, reconstruct the data based on the optical fiber abnormality detection model, and output the reconstructed test optical fiber temperature variation data; An optimal anomaly detection threshold acquisition module is configured to set a threshold for the reconstructed test optical fiber temperature change data to obtain an optimal anomaly detection threshold; The abnormal temperature variation data acquisition module is configured to compare the reconstructed test fiber temperature variation data with the optimal abnormality detection threshold to obtain abnormal temperature variation data, and use the test fiber position corresponding to the abnormal temperature variation data as the test fiber abnormal position.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the optical fiber anomaly detection method based on deep learning as described in any one of claims 1 to 8 is implemented.

Images