Method and device for rapidly processing distributed optical fiber monitoring data based on AI large model

Abstract

The present invention provides a method and device, terminal, and medium for quickly processing distributed optical fiber monitoring data based on an AI large model, and belongs to the field of monitoring data processing technology. The method includes: obtaining monitoring data of each monitoring point in a target area; wherein the target area is divided into multiple sub-areas, each sub-area includes multiple monitoring points, and each monitoring point is a distributed optical fiber sensor; for each sub-area, based on the first AI large model, the monitoring data of each monitoring point in the sub-area are fused in the spatial dimension to obtain the time series vector of the sub-area, and a preliminary abnormality judgment is performed on the time variation characteristics of the time series vector; if the data matrices of all sub-areas pass the preliminary abnormality judgment, the time series vectors of each sub-area are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target area, and a secondary abnormality judgment is performed on the spatial variation characteristics of the spatial vector. The present invention can process distributed optical fiber monitoring data efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the field of monitoring data processing technology, and in particular to a method and device for quickly processing distributed optical fiber monitoring data based on an AI large model. Background Art

[0002] In civil engineering, distributed fiber optic sensing technology has been widely used, particularly for health monitoring of critical infrastructure such as bridges, tunnels, and slopes, due to its high accuracy, long-distance monitoring capabilities, and high sensitivity to environmental changes. This technology deploys a large network of fiber optic sensors to monitor physical parameters of a structure (such as strain, temperature, and vibration) in real time, providing a crucial basis for assessing its safety and durability.

[0003] However, as monitoring systems continue to expand, data volumes are growing exponentially, posing significant challenges for data processing and anomaly detection. Current methods for distributed fiber optic monitoring data are inefficient when processing large datasets, particularly in situations requiring rapid response, such as during natural disasters, and struggle to meet the demands of real-time monitoring. Furthermore, due to the lack of effective data fusion strategies, these methods often exhibit low adaptability and accuracy in complex and diverse real-world scenarios. Summary of the Invention

[0004] Embodiments of the present invention provide a method and device for rapidly processing distributed optical fiber monitoring data based on an AI large model, so as to solve the problem that current methods for distributed optical fiber monitoring data are inefficient when processing large-scale data sets.

[0005] In a first aspect, an embodiment of the present invention provides a method for rapidly processing distributed optical fiber monitoring data based on an AI large model, comprising:

[0006] Acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into multiple sub-areas, each sub-area includes multiple monitoring points, and each monitoring point is a distributed optical fiber sensor;

[0007] For each sub-region, the monitoring data of each monitoring point in the sub-region are spatially fused based on the first AI model to obtain the time series vector of the sub-region, and a preliminary abnormality judgment is made on the time variation characteristics of the time series vector;

[0008] If the data matrices of all sub-regions pass the preliminary anomaly judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region, and a secondary anomaly judgment is performed on the spatial change characteristics of the spatial vector.

[0009] In one possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is spatially fused based on the first AI large model to obtain a time series vector for the sub-region. A preliminary abnormality judgment is then made on the time variation characteristics of the time series vector, including:

[0010] Stacking the monitoring data of each monitoring point in the first sub-region according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region;

[0011] Inputting the monitoring data matrix into the first AI large model to obtain a time series vector for the first sub-region; wherein the first AI large model is trained using a first training data set, the first training data set using the monitoring data matrix as input variables and the time series vector as labels;

[0012] The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0013] In a possible implementation, before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, the method further includes:

[0014] Acquire multiple monitoring data matrices;

[0015] Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix and form a first training data set;

[0016] The monitoring data matrix is:

[0017]

[0018] in, For the monitoring data matrix, For the The monitoring point Monitoring data;

[0019] The time series vector is:

[0020]

[0021]

[0022] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The median of the monitoring data, is the indicator function;

[0023] The initial model is trained based on the first training data set to obtain a first AI large model.

[0024] In one possible implementation, the time series vectors of each sub-region are fused in the time dimension based on the second AI model to obtain the spatial vector of the target region. Secondary abnormality judgment is then performed on the spatial variation characteristics of the spatial vector, including:

[0025] The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region;

[0026] Inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label;

[0027] The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0028] In one possible implementation, before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, the following steps are further included:

[0029] Obtain multiple spatiotemporal data matrices;

[0030] Calculate the spatial vector of each spatiotemporal data matrix respectively as the label of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0031]

[0032] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements;

[0033] The space vector is:

[0034]

[0035]

[0036] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the first order of the time series vectors of all sub-regions The median of the elements, is the indicator function;

[0037] The initial model is trained based on the second training data set to obtain a second AI large model.

[0038] In a possible implementation, before obtaining monitoring data of each monitoring point in the target area, the method further includes:

[0039] Acquire multiple historical monitoring data of each monitoring point in the target area;

[0040] Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters;

[0041] For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0042] In one possible implementation, before integrating the monitoring data of each monitoring point in each sub-region based on the first AI large model to obtain a data matrix for the sub-region and performing a preliminary abnormality judgment on the time series variation characteristics of the data matrix, the following steps may be further included:

[0043] For each monitoring point, the monitoring data of the monitoring point is judged as abnormal based on the threshold value corresponding to the monitoring point;

[0044] If all monitoring points in the first sub-area pass the pre-anomaly judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary anomaly judgment is made on the time change characteristics of the time series vector.

[0045] In a second aspect, an embodiment of the present invention provides a device for rapidly processing distributed optical fiber monitoring data based on an AI large model, comprising:

[0046] An acquisition module is used to acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into multiple sub-areas, each sub-area includes multiple monitoring points, and each monitoring point is a distributed optical fiber sensor;

[0047] A preliminary judgment module is used to fuse the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region and make a preliminary abnormality judgment on the time variation characteristics of the time series vector;

[0048] The secondary judgment module is used to fuse the time series vectors of each sub-region in the time dimension based on the second AI large model when the data matrices of all sub-regions have passed the preliminary abnormality judgment, obtain the spatial vector of the target region, and perform secondary abnormality judgment on the spatial change characteristics of the spatial vector.

[0049] In a third aspect, an embodiment of the present invention provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any possible implementation of the first aspect.

[0051] The embodiments of the present invention provide a method and apparatus for rapidly processing distributed optical fiber monitoring data based on an AI large model. The hierarchical processing strategy is adopted to reduce the burden on the data processing system and speed up the response. At the same time, for situations that fail the initial abnormality judgment, an emergency response can be directly triggered to avoid unnecessary consumption of computing resources. Furthermore, by combining the spatial and temporal dimensions of the distributed optical fiber sensor monitoring data with the AI large model for multi-level data fusion and anomaly detection, the dual spatial and temporal attributes of the distributed optical fiber monitoring data can be fully utilized to improve its application effect in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flowchart of a method for rapidly processing distributed optical fiber monitoring data based on an AI large model provided by one embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of a device for rapidly processing distributed optical fiber monitoring data based on an AI large model provided by one embodiment of the present invention;

[0055] Figure 3 FIG. 4 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below with reference to the accompanying drawings.

[0058] See also Figure 1 , which shows a flowchart of the implementation of the method for quickly processing distributed optical fiber monitoring data based on an AI large model provided by an embodiment of the present invention, as detailed below:

[0059] Step 101 : Acquire monitoring data of each monitoring point in a target area; wherein the target area is divided into multiple sub-areas, each sub-area includes multiple monitoring points, and each monitoring point is a distributed optical fiber sensor.

[0060] In this embodiment, to rapidly process monitoring data, the entire monitoring area can be divided into multiple smaller areas based on the spatial layout of the monitored object (such as a bridge, tunnel, or slope). The data collected by the fiber optic sensors within each smaller area exhibits similar time series characteristics. An appropriate time window is set for each monitoring point to capture dynamic changes in the data. For example, a shorter time window can be used for conditions that may change rapidly (such as earthquakes), while a longer time window can be used for conditions that change slowly (such as long-term settlement).

[0061] A hierarchical anomaly detection algorithm is then used to judge and process the data. Different reporting strategies are adopted based on the level of anomaly detection results. Minor anomalies can be reported to the remote monitoring center in the form of regular summaries; while serious anomalies require immediate transmission via high-speed network channels and the activation of emergency plans. Specific examples include:

[0062] Level 1: Simple threshold judgment

[0063] At the most basic level, simple thresholding methods are applied to identify significant anomalies. For example, if the strain value at a given moment exceeds a preset safety threshold, an alert is immediately triggered. This approach is computationally inexpensive and suitable for real-time execution on resource-constrained edge devices.

[0064] Level 2: Statistical-based methods

[0065] For the data after the first level of screening, statistical anomaly detection methods such as Z-score and IQR are further applied. These methods can identify data points that deviate from the normal range but do not immediately trigger an emergency response.

[0066] Level 3: Machine Learning Models

[0067] For more complex pattern recognition tasks, lightweight machine learning models (such as decision trees and random forests) can be deployed on edge devices. These models can provide higher detection accuracy than the first two levels without significantly increasing the computational burden. Alternatively, the filtered and further simplified data can be submitted to the cloud for more sophisticated analysis.

[0068] In step 102, for each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormality judgment is made on the time variation characteristics of the time series vector.

[0069] In this embodiment, based on the concept of data dimensionality reduction, the monitoring data from multiple monitoring points in a subregion is spatially integrated and reduced to form time series vectors. Anomalies are then detected based on the characteristics of time series variation. For example, principal component analysis (PCA) can be applied to reduce data dimensionality. Through linear transformation, the original variables are converted into a set of new, uncorrelated variables—principal components. The top k principal components are selected so that they can explain the majority of data variation, thus optimizing the data volume.

[0070] In distributed fiber optic monitoring systems, performing principal component analysis directly at the edge can present challenges. Although the computing power of modern edge devices has significantly improved, PCA, as a matrix-based method, requires high computing resources and memory, especially when processing high-dimensional data. The computational complexity can lead to increased latency.

[0071] Based on this, in this embodiment, the monitoring point data in each sub-area is integrated in the spatial dimension through the pre-trained first AI large model. Even if the computing power of the edge device is limited, complex dimensionality reduction operations can be performed to enhance the fluctuation of the monitoring data in the same sub-area in the time dimension, reduce the complexity of data processing, and improve data utilization.

[0072] Furthermore, during the initial anomaly assessment phase, by analyzing the time series vectors of each sub-region, local anomalies can be detected promptly. This approach facilitates early warning and reduces overall system risk caused by local failures.

[0073] Step 103: If the data matrices of all sub-regions pass the preliminary abnormality judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region, and a secondary abnormality judgment is performed on the spatial change characteristics of the spatial vector.

[0074] In this embodiment, a second AI model further integrates the time series vectors of each sub-region to form a spatial vector of the target region. This model then performs a more comprehensive analysis of its spatial variation characteristics, thereby enhancing the fluctuations in the three-dimensional deformation information of the surface scene. This effectively captures the correlations and underlying patterns between different locations within the region, enhancing understanding of the health status of complex structures. This multi-level detection mechanism significantly improves the overall system's anomaly detection accuracy.

[0075] The embodiments of the present invention adopt a hierarchical processing strategy to reduce the burden on the data processing system and speed up the response. At the same time, for situations that fail the initial anomaly judgment, an emergency response can be directly triggered, avoiding unnecessary consumption of computing resources. Furthermore, by combining the spatial and temporal dimensions of distributed fiber optic sensor monitoring data with a large AI model for multi-level data fusion and anomaly detection, it can fully utilize the dual spatial and temporal attributes of distributed fiber optic monitoring data, improving its application effectiveness in complex environments.

[0076] In one possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is spatially fused based on the first AI large model to obtain a time series vector for the sub-region. A preliminary abnormality judgment is then made on the time variation characteristics of the time series vector, including:

[0077] Stacking the monitoring data of each monitoring point in the first sub-region according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region;

[0078] Inputting the monitoring data matrix into the first AI large model to obtain a time series vector for the first sub-region; wherein the first AI large model is trained using a first training data set, the first training data set using the monitoring data matrix as input variables and the time series vector as labels;

[0079] The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0080] In this embodiment, the data matrix monitoring process involves dividing the data from each monitoring point into fixed-length time series segments based on a set time window (e.g., every 5 minutes, every hour, etc.). A monitoring data matrix is then constructed, with the data within each time window as a row. If there are n monitoring points and each time window contains m data points, the resulting matrix size is n × m.

[0081] You can choose an autoencoder (AE), variational autoencoder (VAE), or other deep learning architectures suitable for dimensionality reduction as your first AI model. These models can learn an effective representation of the input data, thereby achieving data compression and feature enhancement.

[0082] The monitoring data matrix is input into the first AI large model, and at each sampling moment, a single numerical value is used to represent the level of each monitoring data in the monitoring data of different monitoring points that is greater than the median or less than the median, thereby integrating the monitoring data of each monitoring point in the sub-area in the spatial dimension, retaining only the description of the fluctuation in the time dimension, simplifying the data structure and enhancing the feature expression capability.

[0083] By comparing the resulting time series vector with historical time series vectors (based on monitoring data from the previous time window), a preliminary assessment of the sub-region's health status can be achieved. This approach can promptly identify abnormal trends, provide early warnings, and reduce overall system risks caused by localized failures.

[0084] In a possible implementation, before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, the method further includes:

[0085] Acquire multiple monitoring data matrices;

[0086] Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix and form a first training data set;

[0087] The monitoring data matrix is:

[0088]

[0089] in, For the monitoring data matrix, For the The monitoring point Monitoring data;

[0090] The time series vector is:

[0091]

[0092]

[0093] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The median of the monitoring data, is the indicator function;

[0094] The initial model is trained based on the first training data set to obtain a first AI large model.

[0095] In this embodiment, This function selects the larger absolute value of the positive or negative deviation as the final deviation measure. In the indicator function, it returns 1 if the condition is met and 0 otherwise.

[0096] In order to express the level of each monitoring data in the monitoring data of different monitoring points greater than the median or less than the median by a single value, a cumulative calculation formula based on directionality is adopted. First, for each data point in a set of data , calculate its median with the median of the entire data set The difference , for all positive deviations ( ), which is accumulated to get the total positive deviation, for all negative deviations ( ), and their absolute values are accumulated to obtain the total negative deviation. Then, based on the magnitude relationship between the two, the dominant direction is determined and the final deviation value is calculated. The dominant direction is used as the sign of the deviation value, so that the time series vector can have positive or negative values.

[0097] In one possible implementation, the time series vectors of each sub-region are fused in the time dimension based on the second AI model to obtain the spatial vector of the target region. Secondary abnormality judgment is then performed on the spatial variation characteristics of the spatial vector, including:

[0098] The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region;

[0099] Inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label;

[0100] The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0101] In this example, similar to the initial anomaly assessment, the time series vectors of each sub-region are stacked according to their positional relationships to form a spatiotemporal data matrix. This allows for data integration from local (sub-region) to global (target region). This captures the changing trends within each sub-region and reveals the interactions and influences between different sub-regions.

[0102] The second AI model generates a comprehensive spatial vector by learning the spatiotemporal data matrix. This vector condenses the key feature information of the entire target area.

[0103] By comparing the current spatial vector with the historical spatial vector from the previous time window, the changing trend of the health status of the target area can be accurately tracked. This helps to detect early warning signals in a timely manner and take preventive measures to avoid major accidents.

[0104] Initial anomaly detection can be performed on edge devices, quickly responding to local anomalies. Secondary anomaly detection is centralized in the cloud or on a central server, fully leveraging high-performance computing resources to complete complex data analysis tasks. This architectural design ensures real-time performance while improving processing efficiency.

[0105] In one possible implementation, before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, the following steps are further included:

[0106] Obtain multiple spatiotemporal data matrices;

[0107] Calculate the spatial vector of each spatiotemporal data matrix respectively as the label of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0108]

[0109] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements;

[0110] The space vector is:

[0111]

[0112]

[0113] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the first order of the time series vectors of all sub-regions The median of the elements, is the indicator function;

[0114] The initial model is trained based on the second training data set to obtain a second AI large model.

[0115] In this embodiment, the spatiotemporal data matrix can be randomly generated with content of a specific size. Spatial vectors are calculated similarly to time series vectors, ensuring that they accurately reflect the key features of the original spatiotemporal data matrix. Compared to training directly with the raw data, this approach can guide the model to focus on the features most important for anomaly detection.

[0116] In a possible implementation, before obtaining monitoring data of each monitoring point in the target area, the method further includes:

[0117] Acquire multiple historical monitoring data of each monitoring point in the target area;

[0118] Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters;

[0119] For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0120] In this embodiment, historical monitoring data is analyzed through a clustering algorithm to dynamically determine the sub-region division method. Compared with the traditional static division method, this method can better adapt to the distribution characteristics of actual monitoring data and ensure the consistency and representativeness of the data in each sub-region. Clustering based on sampling time and monitoring point coordinates helps to classify data that are close in time and space and have similar change patterns into one category, ensuring that the length of the time window matches the change cycle of the actual data, thereby enhancing the correlation of data in the sub-region and improving the rationality of the time window setting, which is beneficial to subsequent data fusion and anomaly detection.

[0121] In one possible implementation, before integrating the monitoring data of each monitoring point in each sub-region based on the first AI large model to obtain a data matrix for the sub-region and performing a preliminary abnormality judgment on the time series variation characteristics of the data matrix, the following steps may be further included:

[0122] For each monitoring point, the monitoring data of the monitoring point is judged as abnormal based on the threshold value corresponding to the monitoring point;

[0123] If all monitoring points in the first sub-area pass the pre-anomaly judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary anomaly judgment is made on the time change characteristics of the time series vector.

[0124] In this embodiment, after monitoring data is collected at the edge and before the AI model performs anomaly detection, a simple threshold determination method can be applied to identify obvious anomalies. For example, if the strain value at a certain moment exceeds the corresponding preset safety threshold, an alarm is immediately triggered. This method has low computational complexity and is suitable for real-time execution on resource-constrained edge devices.

[0125] Thresholds for each monitoring point can be set based on the spatiotemporal distribution characteristics of the monitoring data. For example, time series analysis can be performed on historical data from each monitoring point to identify trends, periodicity, and sudden events. Geographic Information System (GIS) tools or spatial statistical methods can be used to assess the spatial correlation between monitoring points at different locations. For example, the Moran's I index can be calculated to measure the degree of spatial autocorrelation. Specific thresholds can then be set for specific monitoring points and time periods.

[0126] Furthermore, machine learning algorithms can be used to automatically adjust thresholds. For example, using adaptive control theory or deep learning models, thresholds can be continuously updated based on real-time data analysis to adapt to environmental changes and the effects of structural aging.

[0127] It should be understood that the order of execution of the steps in the above embodiments does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0128] The following are device embodiments of the present invention. For details not fully described therein, reference may be made to the corresponding method embodiments described above.

[0129] Figure 2 The following is a schematic diagram showing the structure of an apparatus for rapidly processing distributed optical fiber monitoring data based on an AI large model according to an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0130] like Figure 2 As shown, the device 2 for rapidly processing distributed optical fiber monitoring data based on the AI large model includes:

[0131] An acquisition module 21 is configured to acquire monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor;

[0132] A preliminary judgment module 22 is configured to, for each sub-region, fuse the monitoring data of each monitoring point in the sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region and perform a preliminary abnormality judgment on the time variation characteristics of the time series vector;

[0133] The secondary judgment module 23 is used to fuse the time series vectors of each sub-region in the time dimension based on the second AI large model when the data matrices of all sub-regions pass the preliminary abnormality judgment, obtain the spatial vector of the target region, and perform secondary abnormality judgment on the spatial change characteristics of the spatial vector.

[0134] In a possible implementation, the preliminary determination module 22 is specifically configured to:

[0135] Stacking the monitoring data of each monitoring point in the first sub-region according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region;

[0136] Inputting the monitoring data matrix into the first AI large model to obtain a time series vector for the first sub-region; wherein the first AI large model is trained using a first training data set, the first training data set using the monitoring data matrix as input variables and the time series vector as labels;

[0137] The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0138] In a possible implementation, the preliminary determination module 22 is further configured to:

[0139] Before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, obtaining multiple monitoring data matrices;

[0140] Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix and form a first training data set;

[0141] The monitoring data matrix is:

[0142]

[0143] in, For the monitoring data matrix, For the The monitoring point Monitoring data;

[0144] The time series vector is:

[0145]

[0146]

[0147] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The median of the monitoring data, is the indicator function;

[0148] The initial model is trained based on the first training data set to obtain a first AI large model.

[0149] In a possible implementation, the base secondary judgment module 23 is specifically configured to:

[0150] The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region;

[0151] Inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label;

[0152] The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0153] In a possible implementation, the secondary judgment module 23 is further configured to:

[0154] Before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, multiple spatiotemporal data matrices are obtained;

[0155] Calculate the spatial vector of each spatiotemporal data matrix respectively as the label of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0156]

[0157] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements;

[0158] The space vector is:

[0159]

[0160]

[0161] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the first order of the time series vectors of all sub-regions The median of the elements, is the indicator function;

[0162] The initial model is trained based on the second training data set to obtain a second AI large model.

[0163] In a possible implementation, the obtaining module 21 is further configured to:

[0164] Before obtaining monitoring data of each monitoring point in the target area, obtaining a plurality of historical monitoring data of each monitoring point in the target area;

[0165] Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters;

[0166] For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0167] In one possible implementation, the acquisition module 21 is further configured to: for each sub-region, integrate the monitoring data of each monitoring point in the sub-region based on the first AI large model to obtain a data matrix for the sub-region, and before performing a preliminary abnormality judgment on the time series variation characteristics of the data matrix, perform a pre-abnormality judgment on the monitoring data of each monitoring point based on a threshold value corresponding to the monitoring point;

[0168] If all monitoring points in the first sub-area pass the pre-anomaly judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary anomaly judgment is made on the time change characteristics of the time series vector.

[0169] The embodiments of the present invention adopt a hierarchical processing strategy to reduce the burden on the data processing system and speed up the response. At the same time, for situations that fail the initial anomaly judgment, an emergency response can be directly triggered, avoiding unnecessary consumption of computing resources. Furthermore, by combining the spatial and temporal dimensions of distributed fiber optic sensor monitoring data with a large AI model for multi-level data fusion and anomaly detection, it can fully utilize the dual spatial and temporal attributes of distributed fiber optic monitoring data, improving its application effectiveness in complex environments.

[0170] Figure 3 Schematic diagram of a terminal provided by an embodiment of the present invention. Figure 3 As shown, the terminal 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, the steps in the above-mentioned method for rapidly processing distributed optical fiber monitoring data based on the AI large model are implemented, for example Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 2 Functions of the modules / units 21 to 23 are shown.

[0171] Exemplarily, the computer program 32 may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 32 in the terminal 3. For example, the computer program 32 may be divided into Figure 2 Modules / units 21 to 23 are shown.

[0172] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0173] The processor 30 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0174] The memory 31 can be an internal storage unit of the terminal 3, such as a hard drive or memory of the terminal 3. The memory 31 can also be an external storage device of the terminal 3, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped with the terminal 3. Furthermore, the memory 31 can include both the internal storage unit of the terminal 3 and an external storage device. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 can also be used to temporarily store data that has been output or is about to be output.

[0175] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0176] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0177] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0178] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms.

[0179] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0181] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-mentioned method embodiments for rapidly processing distributed optical fiber monitoring data based on an AI large model. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0182] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for rapidly processing distributed optical fiber monitoring data based on an AI large model, characterized in that: include: Acquiring monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, each monitoring point is a distributed optical fiber sensor, and the target area includes a bridge, a tunnel, or a slope; For each sub-region, the monitoring data of each monitoring point in the sub-region are spatially fused based on the first AI large model to obtain the time series vector of the sub-region. A statistical anomaly detection method is then used to perform a preliminary anomaly judgment on the time variation characteristics of the time series vector. If the data matrices of all sub-regions pass the preliminary anomaly judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region. The spatial variation characteristics of the spatial vector are then subjected to secondary anomaly judgment using an anomaly detection method based on a machine learning model. Before fusing the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain a time series vector for the sub-region and performing a preliminary abnormality judgment on the time variation characteristics of the time series vector using a statistically based abnormality detection method, the method further includes: For each monitoring point, the monitoring data of the monitoring point is judged as abnormal based on the threshold value corresponding to the monitoring point; If all monitoring points in the first sub-area pass the pre-anomaly judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area. A statistical anomaly detection method is then used to perform a preliminary anomaly judgment on the time variation characteristics of the time series vector. If there are any situations in the data matrix of all sub-areas that fail the preliminary abnormality judgment, an emergency response will be triggered directly.

2. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 1 is characterized in that: For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormality judgment is made on the time variation characteristics of the time series vector, including: Stacking the monitoring data of each monitoring point in the first sub-region according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region; Inputting the monitoring data matrix into a first AI large model to obtain a time series vector of the first sub-region; wherein the first AI large model is trained using a first training data set, the first training data set using the monitoring data matrix as an input variable and the time series vector as a label; The time series vector is compared with a historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on a monitoring data matrix of the first sub-region in a previous time window.

3. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 2 is characterized in that: Before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, the method further includes: Acquire multiple monitoring data matrices; Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix and form a first training data set; The monitoring data matrix is: in, For the monitoring data matrix, For the The monitoring point Monitoring data; The time series vector is: in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The median of the monitoring data, is the indicator function; The initial model is trained based on the first training data set to obtain a first AI large model.

4. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 1 is characterized in that: The second AI model is used to fuse the time series vectors of each sub-region in the time dimension to obtain the spatial vector of the target region, and perform secondary abnormality judgment on the spatial variation characteristics of the spatial vector, including: Stacking the time series vectors of each sub-region according to the position relationship to obtain the spatiotemporal data matrix of the target region; Inputting the spatiotemporal data matrix into a second AI large model to obtain a spatial vector of the target area; wherein the second AI large model is trained using a second training data set, the second training data set using the spatiotemporal data matrix as an input variable and the spatial vector as a label; The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain a secondary abnormality judgment result.

5. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 4 is characterized in that: Before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, the method further includes: Obtain multiple spatiotemporal data matrices; Calculate the spatial vector of each spatiotemporal data matrix respectively as the label of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is: in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements; The space vector is: in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the first order of the time series vectors of all sub-regions The median of the elements, is the indicator function; The initial model is trained based on the second training data set to obtain a second AI large model.

6. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 2, characterized in that: Before acquiring the monitoring data of each monitoring point in the target area, the method further includes: Obtain multiple historical monitoring data of each monitoring point in the target area; Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters; For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

7. A device for rapidly processing distributed optical fiber monitoring data based on an AI large model, characterized in that: include: an acquisition module, configured to acquire monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, each monitoring point is a distributed optical fiber sensor, and the target area includes a bridge, a tunnel, or a slope; A preliminary judgment module is used to fuse the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and use a statistical anomaly detection method to perform preliminary anomaly judgment on the time variation characteristics of the time series vector; A secondary judgment module is used to fuse the time series vectors of each sub-region in the time dimension based on the second AI large model when the data matrices of all sub-regions pass the preliminary anomaly judgment, obtain the spatial vector of the target region, and use the anomaly detection method based on the machine learning model to perform secondary anomaly judgment on the spatial variation characteristics of the spatial vector; Before the preliminary judgment module, it also includes: For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region. Before performing a preliminary abnormality judgment on the time variation characteristics of the time series vector using a statistically based abnormality detection method, a pre-anomaly judgment is performed on the monitoring data of each monitoring point based on the threshold corresponding to the monitoring point. If all monitoring points in the first sub-area pass the pre-anomaly judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area. A statistical anomaly detection method is then used to perform a preliminary anomaly judgment on the time variation characteristics of the time series vector. The preliminary judgment module is also used for: If there are any situations in the data matrix of all sub-areas that fail the preliminary abnormality judgment, an emergency response will be triggered directly.

8. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images