Method and device for rapidly processing distributed optical fiber monitoring data based on AI large model, terminal and medium

Abstract

The invention provides a method and device for rapidly processing distributed optical fiber monitoring data based on an AI large model, a terminal and a medium, and belongs to the technical field of monitoring data processing. The method comprises the following steps: acquiring monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area comprises a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; for each sub-region, fusing the monitoring data of each monitoring point in the sub-region in a spatial dimension based on a first AI large model to obtain a time sequence vector of the sub-region, and performing preliminary anomaly judgment on time change characteristics of the time sequence vector; and if the data matrixes of all the sub-regions pass the preliminary anomaly judgment, fusing the time sequence vectors of all the sub-regions in the time dimension based on a second AI large model to obtain a spatial vector of the target region, and performing secondary anomaly judgment on the spatial change characteristics of the spatial vector. According to the invention, distributed optical fiber monitoring data can be processed efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the field of monitoring data processing technology, and in particular to a method, device, terminal, and medium for quickly processing distributed optical fiber monitoring data based on an AI large model. Background Art

[0002] In the field of civil engineering, especially for the health monitoring of key infrastructure such as bridges, tunnels and slopes, distributed fiber optic sensing technology has been widely used due to its high accuracy, long-distance monitoring capability and high sensitivity to environmental changes. This technology deploys a large number of fiber optic sensor networks to monitor the physical parameters of the structure (such as strain, temperature, vibration, etc.) in real time, thus providing an important basis for evaluating the safety and durability of the structure.

[0003] However, as the scale of monitoring systems continues to expand, the amount of data is growing exponentially, which poses significant challenges to data processing and anomaly detection. Current methods for distributed fiber monitoring data are inefficient when processing large-scale data sets, especially in situations where rapid response is required (such as during natural disasters), and it is difficult to meet the needs of real-time monitoring. At the same time, due to the lack of effective data fusion strategies, these methods often show low adaptability and accuracy when faced with complex and changing practical application scenarios. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, terminal, and medium for quickly processing distributed optical fiber monitoring data based on an AI large model to solve the problem that current methods for distributed optical fiber monitoring data are inefficient when processing large-scale data sets.

[0005] In a first aspect, an embodiment of the present invention provides a method for quickly processing distributed optical fiber monitoring data based on an AI big model, comprising: Acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector; If the data matrices of all sub-regions pass the preliminary anomaly judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region, and a secondary anomaly judgment is performed on the spatial change characteristics of the spatial vector.

[0006] In a possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is fused in the spatial dimension based on the first AI big model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector, including: The monitoring data of each monitoring point in the first sub-region are stacked according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region; Input the monitoring data matrix into the first AI big model to obtain the time series vector of the first sub-region; wherein the first AI big model is trained using the first training data set, and the first training data set uses the monitoring data matrix as an input variable and the time series vector as a label; The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0007] In a possible implementation, before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, the method further includes: Obtain multiple monitoring data matrices; Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix, and form a first training data set; The monitoring data matrix is:

[0008] in, For monitoring data matrix, For the The monitoring point Monitoring data; The time series vector is:

[0009]

[0010] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The average value of monitoring data, is the indicator function; The initial model is trained based on the first training data set to obtain a first AI large model.

[0011] In a possible implementation, the time series vectors of each sub-region are fused in the time dimension based on the second AI big model to obtain the spatial vector of the target region, and a secondary abnormality judgment is performed on the spatial variation characteristics of the spatial vector, including: The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region; Input the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, and the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label; The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0012] In a possible implementation, before the spatiotemporal data matrix is ​​input into the second AI large model to obtain the spatial vector of the target area, the method further includes: Get multiple spatiotemporal data matrices; The spatial vectors of each spatiotemporal data matrix are calculated respectively as the labels of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0013] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements; The space vector is:

[0014]

[0015] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the time series vector of all sub-regions The average value of the elements, is the indicator function; The initial model is trained based on the second training data set to obtain a second AI large model.

[0016] In a possible implementation, before acquiring the monitoring data of each monitoring point in the target area, the method further includes: Acquire multiple historical monitoring data of each monitoring point in the target area; Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters; For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0017] In a possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is integrated based on the first AI large model to obtain a data matrix of the sub-region, and before a preliminary abnormal judgment is made on the time series change characteristics of the data matrix, the method further includes: For each monitoring point, a pre-anomaly judgment is performed on the monitoring data of the monitoring point based on the threshold value corresponding to the monitoring point; If all monitoring points in the first sub-area pass the preliminary abnormality judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary abnormality judgment is made on the time change characteristics of the time series vector.

[0018] In a second aspect, an embodiment of the present invention provides a device for quickly processing distributed optical fiber monitoring data based on an AI big model, including: An acquisition module is used to acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; A preliminary judgment module is used to fuse the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and make a preliminary abnormal judgment on the time variation characteristics of the time series vector; The secondary judgment module is used to fuse the time series vectors of each sub-area in the time dimension based on the second AI large model when the data matrices of all sub-areas have passed the preliminary abnormality judgment, obtain the spatial vector of the target area, and perform secondary abnormality judgment on the spatial change characteristics of the spatial vector.

[0019] In a third aspect, an embodiment of the present invention provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation manner of the first aspect are implemented.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation method of the first aspect are implemented.

[0021] The embodiments of the present invention provide a method and device, terminal, and medium for quickly processing distributed optical fiber monitoring data based on an AI big model. The hierarchical processing strategy is adopted to reduce the burden on the data processing system and speed up the response speed. At the same time, for situations that fail the preliminary abnormality judgment, an emergency response can be directly triggered to avoid unnecessary consumption of computing resources. In addition, the use of an AI big model to perform multi-level data fusion and anomaly detection in combination with the spatial and temporal dimensions of the distributed optical fiber sensor monitoring data can fully utilize the dual attributes of space and time of distributed optical fiber monitoring data and improve its application effect in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 It is a flowchart of a method for rapidly processing distributed optical fiber monitoring data based on an AI big model provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of a device for rapidly processing distributed optical fiber monitoring data based on an AI large model provided by an embodiment of the present invention; Figure 3 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0026] See also Figure 1, which shows a flowchart of the implementation of the method for quickly processing distributed optical fiber monitoring data based on an AI big model provided by an embodiment of the present invention, which is described in detail as follows: Step 101, acquiring monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor.

[0027] In this embodiment, in order to realize the rapid processing of monitoring data, the entire monitoring area can be divided into multiple small areas according to the spatial layout of the monitoring object (such as a bridge, tunnel or slope). The data collected by the optical fiber sensor in each small area has similar time series characteristics. And set a suitable time window for each monitoring point to capture the dynamic changes of the data. For example, for situations that may change rapidly (such as earthquake effects), a shorter time window can be used; while for situations that change slowly (such as long-term settlement), a longer time window can be used.

[0028] Then, a hierarchical anomaly detection algorithm is used to judge and process the data, and different reporting strategies are adopted according to the level of the anomaly detection results. Minor anomalies can be reported to the remote monitoring center in the form of regular summaries; while serious anomalies need to be immediately transmitted through high-speed network channels and emergency plans are activated. Specifically, it may include: Level 1: Simple Threshold Judgment At the most basic level, a simple threshold judgment method is applied to identify obvious anomalies. For example, if the strain value at a certain moment exceeds the preset safety threshold, an alarm is triggered immediately. This method has low computational complexity and is suitable for real-time operation on resource-constrained edge devices.

[0029] Level 2: Statistical-based methods For the data after the first level of screening, statistical anomaly detection methods are further applied, such as Z-score, IQR, etc. These methods can identify data points that deviate from the normal range but do not immediately trigger an emergency response.

[0030] Level 3: Machine Learning Models For more complex pattern recognition tasks, lightweight machine learning models (such as decision trees and random forests) can be deployed on edge devices. These models can provide higher detection accuracy than the first two levels without significantly increasing the computational burden. Alternatively, the filtered and further simplified data can be submitted to the cloud for more sophisticated analysis.

[0031] Step 102: For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector.

[0032] In this embodiment, based on the idea of ​​data dimensionality reduction, the monitoring data of multiple monitoring points in the sub-region are integrated and reduced in the spatial dimension to form a time series vector, and abnormal judgment is performed based on the time series change characteristics. For example, principal component analysis (PCA) can be applied to reduce the data dimension, and the original variables are converted into a set of new, unrelated variables - principal components through linear transformation. The first k principal components are selected so that they can explain most of the data variation and optimize the data volume.

[0033] In distributed fiber monitoring systems, performing principal component analysis directly on the edge may face some challenges. Although the computing power of modern edge devices has been significantly improved, PCA, as a matrix-based method, has high requirements for computing resources and memory, especially when processing high-dimensional data, and the computational complexity may lead to increased latency.

[0034] Based on this, in this embodiment, the monitoring point data in each sub-area is integrated in the spatial dimension through the pre-trained first AI large model. Even if the computing power of the edge device is limited, it can perform complex dimensionality reduction operations, strengthen the fluctuation of the monitoring data in the same sub-area in the time dimension, reduce the complexity of data processing, and improve data utilization.

[0035] In addition, by analyzing the time series vector of each sub-area in the initial abnormality judgment stage, local abnormalities can be discovered in time. This method helps to provide early warning and reduce the overall system risk caused by local failures.

[0036] Step 103: If the data matrices of all sub-regions pass the preliminary abnormality judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region, and a secondary abnormality judgment is performed on the spatial change characteristics of the spatial vector.

[0037] In this embodiment, the time series vectors of each sub-region are further integrated through the second AI model to form the spatial vector of the target region, and a more comprehensive analysis of its spatial variation characteristics is performed, thereby strengthening the fluctuation of the three-dimensional deformation information of the surface scene, and being able to effectively capture the correlation and potential patterns between different positions in the region, and enhancing the understanding of the health status of complex structures. Such a multi-level detection mechanism greatly improves the anomaly detection accuracy of the overall system.

[0038] The embodiment of the present invention adopts a hierarchical processing strategy, which can reduce the burden on the data processing system and speed up the response speed. At the same time, for situations that fail the preliminary abnormality judgment, an emergency response can be directly triggered to avoid unnecessary consumption of computing resources. In addition, the use of AI big models to perform multi-level data fusion and anomaly detection in combination with the spatial and temporal dimensions of the distributed optical fiber sensor monitoring data can fully utilize the dual attributes of space and time of distributed optical fiber monitoring data to improve its application effect in complex environments.

[0039] In a possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is fused in the spatial dimension based on the first AI big model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector, including: The monitoring data of each monitoring point in the first sub-region are stacked according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region; Input the monitoring data matrix into the first AI big model to obtain the time series vector of the first sub-region; wherein the first AI big model is trained using the first training data set, and the first training data set uses the monitoring data matrix as an input variable and the time series vector as a label; The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0040] In this embodiment, the process of monitoring the data matrix includes: according to the set time window (for example, every 5 minutes, every hour, etc.), the data of each monitoring point is divided into time series segments of fixed length. Then, the data in each time window is used as a row to construct a monitoring data matrix. If there are n monitoring points in total and each time window contains m data points, the size of the formed matrix is ​​n×m.

[0041] You can choose autoencoder (AE), variational autoencoder (VAE) or other deep learning architectures suitable for dimensionality reduction tasks as the first AI model. These models can learn effective representations of input data, thereby achieving data compression and feature enhancement.

[0042] The monitoring data matrix is ​​input into the first AI large model, and at each sampling moment, a single numerical value is used to represent the level at which each monitoring data in the monitoring data of different monitoring points is greater than or less than the average value, thereby integrating the monitoring data of each monitoring point in the sub-area in the spatial dimension, retaining only the description of the fluctuations in the time dimension, simplifying the data structure and enhancing the feature expression capability.

[0043] By comparing the obtained time series vector with the historical time series vector (based on the monitoring data of the previous time window), a preliminary assessment of the health status of the sub-region can be achieved. This method can detect abnormal trends in a timely manner, provide early warnings, and reduce the overall system risk caused by local failures.

[0044] In a possible implementation, before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, the method further includes: Obtain multiple monitoring data matrices; Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix, and form a first training data set; The monitoring data matrix is:

[0045] in, For monitoring data matrix, For the The monitoring point Monitoring data; The time series vector is:

[0046]

[0047] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The average value of monitoring data, is the indicator function; The initial model is trained based on the first training data set to obtain a first AI large model.

[0048] In this embodiment, The function is used to select the one with the larger absolute value of the positive and negative deviations as the final deviation measure. In the indicator function, 1 is returned when the condition is met, otherwise 0 is returned.

[0049] In order to use a single value to represent the level of each monitoring data in the monitoring data of different monitoring points that is greater than or less than the average value, a direction-based cumulative calculation formula is used. First, for each data point in a set of data , calculate its average value with the entire data set The difference , for all positive deviations ( ), which is accumulated to get the total positive deviation, for all negative deviations ( ), and the absolute values ​​are accumulated to obtain the total negative deviation. Then, the dominant direction is determined according to the magnitude relationship between the two and the final deviation value is calculated. The dominant direction is used as the sign of the deviation value, so that positive or negative values ​​can exist in the time series vector.

[0050] In a possible implementation, the time series vectors of each sub-region are fused in the time dimension based on the second AI big model to obtain the spatial vector of the target region, and a secondary abnormality judgment is performed on the spatial variation characteristics of the spatial vector, including: The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region; Input the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, and the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label; The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0051] In this embodiment, a similar method to the preliminary abnormality judgment is adopted. On the basis of the preliminary abnormality judgment, the time series vectors of each sub-region are stacked according to the position relationship to form a spatiotemporal data matrix, thereby realizing data integration from local (sub-region) to global (target region). In this way, the change trend within each sub-region is captured, and the interaction and influence between different sub-regions are revealed.

[0052] The second AI model generates a comprehensive spatial vector by learning the spatiotemporal data matrix. This vector condenses the key feature information of the entire target area.

[0053] By comparing the current spatial vector with the historical spatial vector of the previous time window, the changing trend of the health status of the target area can be accurately tracked. This helps to detect early warning signals in time and take preventive measures to avoid major accidents.

[0054] Initial anomaly judgment can be performed on edge devices to quickly respond to local anomalies, while secondary anomaly judgment is centralized on the cloud or central server, making full use of high-performance computing resources to complete complex data analysis tasks. This architecture design ensures real-time performance and improves processing efficiency.

[0055] In a possible implementation, before the spatiotemporal data matrix is ​​input into the second AI large model to obtain the spatial vector of the target area, the method further includes: Get multiple spatiotemporal data matrices; The spatial vectors of each spatiotemporal data matrix are calculated respectively as the labels of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0056] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements; The space vector is:

[0057]

[0058] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the time series vector of all sub-regions The average value of the elements, is the indicator function; The initial model is trained based on the second training data set to obtain a second AI large model.

[0059] In this embodiment, the spatiotemporal data matrix can randomly generate content according to a specific size. The spatial vectors are calculated in a similar way to the time series vectors, ensuring that these vectors can accurately reflect the key features of the original spatiotemporal data matrix. Compared to directly using the original data for training, this method can guide the model to focus on the features that are most important for anomaly detection.

[0060] In a possible implementation, before acquiring the monitoring data of each monitoring point in the target area, the method further includes: Acquire multiple historical monitoring data of each monitoring point in the target area; Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters; For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0061] In this embodiment, the historical monitoring data is analyzed by a clustering algorithm, and the division method of the sub-areas can be dynamically determined. Compared with the traditional static division method, this method can better adapt to the distribution characteristics of the actual monitoring data and ensure the consistency and representativeness of the data in each sub-area. Clustering according to the sampling time and the coordinates of the monitoring points helps to classify data that are close in time and space and have similar change patterns into one category, ensuring that the length of the time window matches the change cycle of the actual data, thereby enhancing the correlation of the data in the sub-area and improving the rationality of the time window setting, which is conducive to subsequent data fusion and anomaly detection.

[0062] In a possible implementation, for each sub-region, the monitoring data of each monitoring point in the sub-region is integrated based on the first AI large model to obtain a data matrix of the sub-region, and before a preliminary abnormal judgment is made on the time series change characteristics of the data matrix, the method further includes: For each monitoring point, a pre-anomaly judgment is performed on the monitoring data of the monitoring point based on the threshold value corresponding to the monitoring point; If all monitoring points in the first sub-area pass the preliminary abnormality judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary abnormality judgment is made on the time change characteristics of the time series vector.

[0063] In this embodiment, after the monitoring data is collected on the edge side and before the abnormality judgment is made by the AI ​​big model, a simple threshold judgment method can be applied to identify obvious abnormalities. For example, if the strain value at a certain moment exceeds the corresponding preset safety threshold, an alarm is triggered immediately. This method has a small amount of calculation and is suitable for real-time operation on resource-constrained edge devices.

[0064] The threshold of each monitoring point can be set according to the spatiotemporal distribution characteristics of the monitoring data. For example, the historical data of each monitoring point can be analyzed in time series to identify the trend, periodicity and sudden events of data changes, and the spatial correlation between monitoring points in different locations can be evaluated using geographic information system (GIS) tools or spatial statistical methods. For example, the degree of spatial autocorrelation can be measured by calculating the Moran's I index. Then set specific thresholds for specific monitoring points and specific time periods.

[0065] In addition, machine learning algorithms can be used to automatically adjust the thresholds. For example, using adaptive control theory or deep learning models, the thresholds can be continuously updated based on real-time data analysis to adapt to the effects of environmental changes and structural aging.

[0066] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0067] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0068] Figure 2 The schematic diagram of the structure of the device for quickly processing distributed optical fiber monitoring data based on the AI ​​big model provided by an embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows: like Figure 2 As shown, the device 2 for rapidly processing distributed optical fiber monitoring data based on the AI ​​big model includes: An acquisition module 21 is used to acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; A preliminary judgment module 22 is used to fuse the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and perform preliminary abnormal judgment on the time variation characteristics of the time series vector; The secondary judgment module 23 is used to fuse the time series vectors of each sub-region in the time dimension based on the second AI large model when the data matrices of all sub-regions have passed the preliminary abnormality judgment, obtain the spatial vector of the target region, and perform secondary abnormality judgment on the spatial change characteristics of the spatial vector.

[0069] In a possible implementation, the preliminary determination module 22 is specifically configured to: The monitoring data of each monitoring point in the first sub-region are stacked according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region; Input the monitoring data matrix into the first AI big model to obtain the time series vector of the first sub-region; wherein the first AI big model is trained using the first training data set, and the first training data set uses the monitoring data matrix as an input variable and the time series vector as a label; The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

[0070] In a possible implementation, the preliminary determination module 22 is further configured to: Before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-region, a plurality of monitoring data matrices are obtained; Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix, and form a first training data set; The monitoring data matrix is:

[0071] in, For monitoring data matrix, For the The monitoring point Monitoring data; The time series vector is:

[0072]

[0073] in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The average value of monitoring data, is the indicator function; The initial model is trained based on the first training data set to obtain a first AI large model.

[0074] In a possible implementation, the base secondary judgment module 23 is specifically used to: The time series vectors of each sub-region are stacked according to the position relationship to obtain the spatiotemporal data matrix of the target region; Input the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area; wherein the second AI large model is trained using the second training data set, and the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label; The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain the secondary anomaly judgment result.

[0075] In a possible implementation, the secondary determination module 23 is further configured to: Before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, a plurality of spatiotemporal data matrices are obtained; The spatial vectors of each spatiotemporal data matrix are calculated respectively as the labels of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is:

[0076] in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements; The space vector is:

[0077]

[0078] in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the time series vector of all sub-regions The average value of the elements, is the indicator function; The initial model is trained based on the second training data set to obtain a second AI large model.

[0079] In a possible implementation, the acquisition module 21 is further configured to: Before acquiring the monitoring data of each monitoring point in the target area, acquiring a plurality of historical monitoring data of each monitoring point in the target area; Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters; For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

[0080] In a possible implementation, the acquisition module 21 is further used to: for each sub-region, integrate the monitoring data of each monitoring point in the sub-region based on the first AI large model to obtain the data matrix of the sub-region, and before making a preliminary abnormality judgment on the time series change characteristics of the data matrix, for each monitoring point, make a pre-abnormal judgment on the monitoring data of the monitoring point based on the threshold value corresponding to the monitoring point; If all monitoring points in the first sub-area pass the preliminary abnormality judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary abnormality judgment is made on the time change characteristics of the time series vector.

[0081] The embodiment of the present invention adopts a hierarchical processing strategy, which can reduce the burden on the data processing system and speed up the response speed. At the same time, for situations that fail the preliminary abnormality judgment, an emergency response can be directly triggered to avoid unnecessary consumption of computing resources. In addition, the use of AI big models to perform multi-level data fusion and anomaly detection in combination with the spatial and temporal dimensions of the distributed optical fiber sensor monitoring data can fully utilize the dual attributes of space and time of distributed optical fiber monitoring data to improve its application effect in complex environments.

[0082] Figure 3 is a schematic diagram of a terminal provided by an embodiment of the present invention. Figure 3 As shown, the terminal 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, the steps in the above-mentioned method embodiments for rapidly processing distributed optical fiber monitoring data based on an AI large model are implemented, for example Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 2 Functions of modules / units 21 to 23 are shown.

[0083] Exemplarily, the computer program 32 may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 32 in the terminal 3. For example, the computer program 32 may be divided into Figure 2 Modules / units 21 to 23 are shown.

[0084] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0085] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0086] The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal 3. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0087] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0088] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0090] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments for rapidly processing distributed optical fiber monitoring data based on the AI ​​large model can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0094] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for rapidly processing distributed optical fiber monitoring data based on an AI large model, characterized in that: include: Acquire monitoring data of each monitoring point in the target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector; If the data matrices of all sub-regions pass the preliminary abnormality judgment, the time series vectors of each sub-region are fused in the time dimension based on the second AI large model to obtain the spatial vector of the target region, and a secondary abnormality judgment is performed on the spatial change characteristics of the spatial vector.

2. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 1 is characterized in that: For each sub-region, the monitoring data of each monitoring point in the sub-region are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and a preliminary abnormal judgment is made on the time variation characteristics of the time series vector, including: Stacking the monitoring data of each monitoring point in the first sub-region according to the sampling time to obtain a monitoring data matrix of the first sub-region; wherein the first sub-region is any sub-region; Input the monitoring data matrix into a first AI big model to obtain a time series vector of the first sub-region; wherein the first AI big model is trained using a first training data set, and the first training data set uses the monitoring data matrix as an input variable and the time series vector as a label; The time series vector is compared with the historical time series vector of the first sub-region to obtain a preliminary abnormality judgment result; wherein the historical time series vector is obtained based on the monitoring data matrix of the first sub-region in the previous time window.

3. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 2 is characterized in that: Before inputting the monitoring data matrix into the first AI large model to obtain the time series vector of the first sub-area, the method further includes: Obtain multiple monitoring data matrices; Calculate the time series vector of each monitoring data matrix respectively as the label of the monitoring data matrix, and form a first training data set; The monitoring data matrix is: in, For monitoring data matrix, For the The monitoring point Monitoring data; The time series vector is: in, is the time series vector, is the first elements, For the The monitoring point Monitoring data, For all monitoring points The average value of the monitoring data, is the indicator function; The initial model is trained based on the first training data set to obtain a first AI large model.

4. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 1 is characterized in that: The second AI big model is used to fuse the time series vectors of each sub-area in the time dimension to obtain the space vector of the target area, and a secondary abnormality judgment is performed on the spatial variation characteristics of the space vector, including: Stacking the time series vectors of each sub-region according to the position relationship to obtain the spatiotemporal data matrix of the target region; Input the spatiotemporal data matrix into a second AI large model to obtain a spatial vector of the target area; wherein the second AI large model is trained using a second training data set, and the second training data set uses the spatiotemporal data matrix as an input variable and the spatial vector as a label; The spatial vector is compared with the historical spatial vector of the target area in the previous time window to obtain a secondary abnormality judgment result.

5. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 4 is characterized in that: Before inputting the spatiotemporal data matrix into the second AI large model to obtain the spatial vector of the target area, the method further includes: Get multiple spatiotemporal data matrices; The spatial vectors of each spatiotemporal data matrix are calculated respectively as the labels of the spatiotemporal data matrix and form the second training data set; the spatiotemporal data matrix is: in, is the spatiotemporal data matrix, For the The time series vector of the sub-region elements; The space vector is: in, is a space vector, is the first vector in the space element, For the The time series vector of the sub-region elements, is the time series vector of all sub-regions The average value of the elements, is the indicator function; The initial model is trained based on the second training data set to obtain a second AI large model.

6. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 2 is characterized in that: Before acquiring the monitoring data of each monitoring point in the target area, the method further includes: Acquire multiple historical monitoring data of each monitoring point in the target area; Clustering each historical monitoring data based on sampling time and monitoring point coordinates to obtain multiple clusters; For each cluster, the monitoring points corresponding to each historical monitoring number in the cluster are divided into a sub-region, and the difference between the upper and lower limits of the sampling time of each historical monitoring data in the cluster is used as the time window of the sub-region.

7. The method for rapidly processing distributed optical fiber monitoring data based on an AI large model according to claim 1 is characterized in that: Before integrating the monitoring data of each monitoring point in each sub-region based on the first AI large model to obtain a data matrix of the sub-region and making a preliminary abnormal judgment on the time series change characteristics of the data matrix, the method further includes: For each monitoring point, a pre-anomaly judgment is performed on the monitoring data of the monitoring point based on the threshold value corresponding to the monitoring point; If all monitoring points in the first sub-area pass the preliminary abnormality judgment, the monitoring data of each monitoring point in the first sub-area are fused in the spatial dimension based on the first AI large model to obtain the time series vector of the first sub-area, and a preliminary abnormality judgment is made on the time change characteristics of the time series vector.

8. A device for rapidly processing distributed optical fiber monitoring data based on an AI large model, characterized in that: include: An acquisition module, used to acquire monitoring data of each monitoring point in a target area; wherein the target area is divided into a plurality of sub-areas, each sub-area includes a plurality of monitoring points, and each monitoring point is a distributed optical fiber sensor; A preliminary judgment module is used to fuse the monitoring data of each monitoring point in each sub-region in the spatial dimension based on the first AI large model to obtain the time series vector of the sub-region, and make a preliminary abnormal judgment on the time variation characteristics of the time series vector; The secondary judgment module is used to fuse the time series vectors of each sub-region in the time dimension based on the second AI large model when the data matrices of all sub-regions have passed the preliminary abnormality judgment, obtain the spatial vector of the target region, and perform secondary abnormality judgment on the spatial change characteristics of the spatial vector.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images