Coal mine sensor time series data anomaly detection and adaptive repair reconstruction method

Abstract

The present application relates to a kind of coal mine sensor time series data anomaly detection and adaptive repair reconstruction method, belong to coal mine intelligent technology field.It includes: real-time acquisition of multiple original time series data streams in coal mine underground;Each original time series data stream is identified by time series data stream type identification algorithm Data type;According to the data type of each original time series data stream, each original time series data stream is carried out abnormal value detection, adaptive repair;According to the data type of each original time series data stream, all types of original time series data stream are carried out time alignment and equal time interval reconstruction.Through original time series data stream type identification, different abnormal value detection based on data type and abnormal value adaptive repair, and final time alignment, equal time interval reconstruction and warehousing storage, through a series of automation, targeted processing, finally form neat, continuous high-quality time series data stream, provide reliable data basis for subsequent intelligent analysis and decision-making.

Description

Technical Field

[0001] This invention relates to the field of intelligent coal mining technology, and in particular to a method for detecting and adaptively repairing and reconstructing anomalies in time-series data from coal mine sensors. Background Technology

[0002] With the rapid development of emerging technologies such as the Internet of Things, big data, and artificial intelligence, the coal mining industry is at a critical stage of digital and intelligent transformation. Under the strategic constraints of "carbon peaking and carbon neutrality," coal mining urgently needs to achieve a balance between safety and efficiency through digital means. Modern mines commonly deploy various types of sensors, including pressure, vibration, and displacement sensors, to collect data in real time on changes in support system loads, equipment operating status, and environmental parameters. These coal mine sensor time-series data are generated at frequencies ranging from seconds to milliseconds, exhibiting high-frequency continuity, multi-source heterogeneity, and strong spatiotemporal correlation. Their quality directly affects the reliability of mining process optimization, equipment fault early warning, and safety decisions.

[0003] However, the time-series data from different types of coal mine sensors exhibit varying temporal characteristics, and errors or anomalies can directly affect the training and operational analysis of intelligent coal mine models. To obtain high-fidelity and highly consistent coal mine sensor time-series data, this invention proposes a coal mine sensor time-series data anomaly detection and adaptive repair and reconstruction technology. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors. The technical solution of this invention is as follows: A method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors, comprising: S1, real-time acquisition of various raw time-series data streams collected by multiple types of sensors in underground coal mines; S2, identify the data type of each type of original time-series data stream using a time-series data stream type identification algorithm. The data types include long-period smoothing, short-period smoothing, sawtooth, and aperiodic smoothing. Wherein, any type of original time-series data stream is defined as the target original time-series data stream. S2 includes: S21, extracting the time-domain features, frequency-domain features, and autocorrelation features of each sliding window of the target original time-series data stream using a sliding window, forming a window feature set for each sliding window; S22, identifying the data type of the target original time-series data stream within the sliding window based on the window features in the window feature set. S3, perform outlier detection on each raw time series data stream according to the data type of each raw time series data stream; S4, adaptively repair outliers in each type of original time series data according to the data type of each type of original time series data stream; S5 performs time alignment and equal time interval reconstruction on all types of original time-series data streams according to the data type of each original time-series data stream.

[0005] Preferably, any sliding window is defined as the target sliding window, and step S21, when extracting the temporal features of the target sliding window, includes: S211, DC removal processing is performed on the original target time series data stream within the target sliding window to obtain the DC-removed time series data sequence; S212, calculate the window standard deviation of the target sliding window based on the time series data sequence after DC removal; S213, construct a first-order difference sequence based on the time-series data after DC removal, and calculate the variance of the first-order difference sequence; S214, calculate the relative difference variance based on the variance of the first difference sequence and the window standard deviation; S215, calculates kurtosis based on the time-series data sequence after DC removal; S216, calculate the zero-crossing rate based on the timing data sequence after DC removal; S21, in extracting the frequency domain features of the target sliding window, includes: S21-1, the spectral amplitude of the target sliding window is calculated using fast Fourier transform; S21-2, Calculate the main frequency, main frequency energy ratio and spectral entropy of the target sliding window based on the spectral amplitude; S21, when extracting the autocorrelation features of the target sliding window, includes: calculating the standardized autocorrelation function of the target sliding window, and determining the main period delay, local peaks, and autocorrelation peaks based on the standardized autocorrelation function.

[0006] Preferably, S22 includes: S221, based on the autocorrelation peak value in the window feature set and the ratio of main frequency energy Determine whether the original time-series data stream of the target within the target sliding window is periodic; when it satisfies or At that time, it is determined that the original time-series data stream of the target within the target sliding window is periodic, where, and These are the autocorrelation peak threshold and the dominant frequency energy ratio threshold, respectively; when the following conditions are met... and At that time, it is determined that the original time-series data stream of the target within the target sliding window is aperiodic; S222, when the target's original time-series data stream within the target sliding window is periodic, calculate the period length T of the target's original time-series data stream within the target sliding window. At times, or, in the case of an unknown sampling rate, the main cycle delay. If the target original time series data stream in the target sliding window is determined to be of long-period smooth type, then the target original time series data stream in the target sliding window is determined to be of short-period smooth type. For the long-period time domain length threshold, The threshold for the number of sampling points in a long period; S223, based on relative difference variance Spectral entropy and kurtosis To determine whether the original time-series data stream of the target within the target sliding window is of aperiodic smooth type, the rule is defined as follows: ; in, , , These are the smoothness threshold, the spectral entropy threshold, and the kurtosis threshold, respectively. When none of the above conditions are met, the original time-series data stream of the target within the target sliding window is determined to be sawtooth-shaped.

[0007] Preferably, after S22, the method further includes: S23, calculate the confidence level for each type of data based on the data type of the target raw time-series data stream within the sliding window, where: Periodic confidence level The calculation formula is: ; Non-periodic confidence level The calculation formula is: ; in, For relative difference variance, This represents the median of the relative difference variance index within the historical sliding window. Confidence level of the sawtooth pattern The calculation formula is: ; in, For logical mapping functions, , These are the weighting coefficients. , Historical baseline parameters; S24. For the target original time-series data stream located in the overlapping area of ​​adjacent sliding windows, calculate its score for each data type, and take the data type with the highest score as its data type.

[0008] Preferably, any type of raw time-series data stream is defined as the target raw time-series data stream, and step S3 includes: S31, When the target original time series data stream is a long-period smooth type or a short-period smooth type, the coal mine sensor data anomaly detection algorithm based on proximity time series feature learning performs anomaly detection on the target original time series data stream; S32, When the target's original time-series data stream is sawtooth-shaped, the isolated forest coal mine sensor data anomaly detection algorithm based on feature-based diversion performs anomaly detection on the target's original time-series data stream; S33, When the target original time series data stream is non-periodic and smooth, the coal mine sensor data anomaly detection algorithm based on physical constraints and robust statistical characteristics performs outlier detection on the target original time series data stream.

[0009] Preferably, when the S31 algorithm for detecting outliers in the target raw time-series data stream is based on proximity-based temporal feature learning, it includes: S311, standardizes the target raw time-series data stream; S312 performs outlier screening on the standardized target original time series data stream using the distance-based KNN outlier detection algorithm and the density-based LOF outlier detection algorithm, respectively, and takes the intersection of the two as the outlier screening result, and generates outlier labels for the outliers included in the outlier screening result. S313, extracts lag features, statistical features, difference features, FFT features and time coding features from the standardized target original time series data stream through a dual-scale sliding window, and constructs a first multi-dimensional feature matrix based on the lag features, statistical features, difference features, FFT features and time coding features; S314, input the outliers and their outlier labels, and the first multidimensional feature matrix, including the initial outlier screening results, into the pre-trained anomaly detection XGBoost model, and the anomaly detection XGBoost model outputs the outliers in the target original time series data stream.

[0010] Preferably, S32 includes: S321, Determine whether the target's original time-series data stream is angle data; S322, If the original time series data stream of the target is angle data, then convert the original time series data stream of the target into radian data stream, and extract the statistical features, rate of change features, trigonometric function features and circumferential difference features of the radian data stream; S323, If the target's original time series data stream is non-angular data, then the target's original time series data stream is standardized, and the statistical characteristics, rate of change characteristics, and linear difference characteristics of the standardized original time series data stream are extracted. S324. Based on the statistical characteristics, rate of change characteristics, trigonometric function characteristics, and circumferential difference characteristics of the radian data stream and the statistical characteristics, rate of change characteristics, and linear difference characteristics of the target original time series data stream after standardization, a second multidimensional feature matrix is ​​constructed, and the features in the second multidimensional feature matrix are standardized by a robust scaler. S325, the standardized second multi-dimensional feature matrix is ​​input into the pre-trained anomaly detection isolation forest model, and the anomaly detection isolation forest model outputs the outlier values ​​in the target's original time series data stream; S326, Based on the circumferential difference features and linear difference features, adjacent point jump detection is performed on the target original time series data stream to obtain the jump point detection results; S327. Based on the outlier and jump point detection results in the target's original time series data stream output by the anomaly detection isolated forest model, determine the outliers in the target's original time series data stream.

[0011] Preferably, S33 includes: S331, Perform physical boundary detection on the target's original time-series data stream, and determine whether any data in the target's original time-series data stream exceeds the upper and lower threshold ranges of the corresponding sensor. If it exceeds, mark it as an outlier. S332, if not exceeded, calculate the MAD of the target original time series data stream; when the MAD is valid, detect outliers in the target original time series data stream based on the Modified Z-score algorithm; when the MAD is invalid, use the IQR detection algorithm to detect outliers in the target original time series data stream. S333 performs differential mutation detection on the target's original time-series data stream; S334, determine the outliers in the target's original time-series data stream based on the physical boundary detection results, MAD-based detection results, and differential mutation detection results.

[0012] Preferably, any type of raw time-series data stream is defined as the target raw time-series data stream, and step S4 includes: S41, When the target original time series data stream is a long-period smooth type, the seasonal autoregressive integral moving average model is used to adaptively repair the outliers in the target original time series data. S42, When the target original time series data stream is a short-period smooth type, a bidirectional long short-term memory network combined with a self-attention model is used to adaptively repair the outliers in the target original time series data. S43, When the target original time series data stream is jagged, a Transformer model based on self-attention mechanism is introduced to adaptively repair outliers in the target original time series data; S44. When the target original time series data stream is non-periodic and smooth, a unidirectional LSTM model with an attention mechanism is used to adaptively repair outliers in the target original time series data.

[0013] Preferably, S5 includes: S51: Collect timestamps from all types of raw time-series data streams, take the union of all timestamps to generate a global timeline, and use the global timeline as a reference to perform time-aligned interpolation on each type of raw time-series data stream using a hierarchical interpolation strategy based on the data type of each raw time-series data stream. Specifically: for long-period smooth raw time-series data streams, a cubic spline algorithm is used for interpolation; for short-period smooth and sawtooth types, a piecewise cubic Hermite algorithm is used for interpolation; and for non-periodic smooth types, a time-weighted algorithm is used for linear interpolation. S52 performs equal-interval interpolation on all types of original time-series data streams after time-aligned interpolation and stores all types of data after equal-interval interpolation in the time-series database.

[0014] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.

[0015] By means of the above solution, the beneficial effects of the present invention are as follows: By identifying the type of the original time-series data stream, detecting outliers based on data type differences, adaptively repairing outliers, aligning time, and finally reconstructing and storing the data at equal time intervals, a series of automated and targeted processes are used to form a regular, continuous, high-quality time-series data stream, providing a reliable data foundation for subsequent intelligent analysis and decision-making. Attached Figure Description

[0016] Figure 1 This is a flowchart of the coal mine sensor time-series data anomaly detection and adaptive repair and reconstruction method provided in the embodiments of the present invention.

[0017] Figure 2 This is a schematic diagram illustrating the process of extracting sliding window features in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram showing the distribution of long-period smooth anomalies in an embodiment of the present invention.

[0019] Figure 4 This is a flowchart of the coal mine sensor data anomaly detection algorithm based on proximity temporal feature learning in an embodiment of the present invention.

[0020] Figure 5 This is a schematic diagram of the features extracted by temporal feature engineering through multi-scale fusion in an embodiment of the present invention.

[0021] Figure 6 This is a flowchart of the anomaly detection algorithm for sensor data in isolated forest coal mines based on feature-based diversion in an embodiment of the present invention.

[0022] Figure 7 This is a flowchart of the coal mine sensor data anomaly detection algorithm based on physical constraints and robust statistical characteristics in an embodiment of the present invention. Detailed Implementation

[0023] The coal mine sensor time-series data anomaly detection and adaptive repair reconstruction method provided in this invention can be executed by any electronic device with computing capabilities, such as a server, mobile terminal, or PC. Specifically, based on the classification method of industrial data in the China Industrial Big Data Technology and Application White Paper, underground coal mine data can be divided into structured data, unstructured data, and semi-structured data. Among them, structured data refers to relational data, stored in a database, and can use a two-dimensional table structure to express entities and their relationships. Structured data can be divided into long-period smooth, short-period smooth, sawtooth, and non-periodic smooth types according to the spatial distribution of the data, as shown in Table 1, which is a schematic table of the classification and characteristics of structured data from coal mine sensors.

[0024]

[0025] To address the four types of structured sensor time-series data in coal mines mentioned above, and to achieve high-fidelity and high-consistency data reconstruction, this invention proposes a method for anomaly detection and adaptive repair reconstruction of coal mine time-series data. For example... Figure 1 As shown, the process includes the following steps S1 to S5: S1 acquires multiple raw time-series data streams collected in real time from various types of sensors in underground coal mines.

[0026] The sensors include various types such as pressure sensors, temperature sensors, displacement sensors, speed sensors, and current sensors. These sensors can be distributed across different devices, such as hydraulic supports or coal mining machines. Each type of sensor on each device acquires one type of raw time-series data stream.

[0027] S2, the data type of each raw time-series data stream is identified by a time-series data stream type identification algorithm. The data types include long-period smoothing, short-period smoothing, sawtooth smoothing, and non-periodic smoothing.

[0028] When acquiring downhole data, the type of data may not be clear, making it impossible to accurately process the data. In order to process the data accurately, this embodiment of the invention uses a time-series data stream type identification algorithm to automatically determine the data type of the original time-series data stream during real-time acquisition.

[0029] Specifically, the embodiments of the present invention adopt a streaming processing architecture, using a sliding window as the basic analysis unit, to achieve real-time data type identification without changing the original sampling time sequence. The technical architecture is as follows: Figure 2 As shown.

[0030] For ease of description, this embodiment of the invention defines any type of raw time-series data stream as the target raw time-series data stream. Step S2 includes: S21, using a sliding window to extract the time-domain features, frequency-domain features, and autocorrelation features of each sliding window of the target raw time-series data stream, forming a window feature set for each sliding window; S22, identifying the data type of the target raw time-series data stream within the sliding window based on the window features in the window feature set. Furthermore, for ease of description, this embodiment of the invention defines any sliding window as the target sliding window.

[0031] In specific implementation, step S21, when extracting the temporal features of the target sliding window, includes steps S211 to S216: S211, DC removal processing is performed on the original target time-series data stream within the target sliding window to obtain the DC-removed time-series data sequence.

[0032] Specifically, assume the window length of the sliding window is... (The window length is determined based on the data period, and must cover at least one period length T). The sampling frequency is... The original time-series data stream of the target within the target sliding window is To eliminate the influence of reference level differences under different sensor or operating conditions, the original target timing data stream is first subjected to DC removal processing to obtain the DC-removed timing data sequence: ; in, The target is the time series data at time t after DC removal from the original time series data stream. For the target original time-series data at time t, This represents the window mean.

[0033] By removing DC, it can be ensured that the features extracted subsequently only reflect the fluctuation pattern of the signal and are not affected by the overall offset.

[0034] S212, calculate the window standard deviation of the target sliding window based on the time series data sequence after DC removal.

[0035] Specifically, the window standard deviation of the target sliding window. .

[0036] S213, construct a first-order difference sequence based on the time-series data sequence after DC removal, and calculate the variance of the first-order difference sequence.

[0037] Specifically, first-order difference sequences ,in, The target is the time series data after DC removal from the time series data at time t+1 in the original time series data stream.

[0038] Variance of first-order difference sequences ,in, It is the mean of the first-order difference sequence.

[0039] S214, calculate the relative difference variance based on the variance of the first difference sequence and the window standard deviation.

[0040] Specifically, relative difference variance (smoothness). ,in, It is a small constant used to prevent the denominator from being zero. The smaller the value, the smoother the signal change; the larger the value, the more drastic the signal fluctuation.

[0041] S215 calculates kurtosis based on the time-series data sequence after DC removal.

[0042] Specifically, kurtosis Kurtosis is used to characterize the sharpness of a waveform. The larger the kurtosis, the more likely there are spikes or pulses in the signal.

[0043] S216 calculates the zero-crossing rate based on the timing data sequence after DC removal.

[0044] Specifically, the zero-crossing rate .in, For indicator functions, when When the value is less than 0, the value is 1; otherwise, it is 0. ZCR reflects the frequency of the signal alternation between positive and negative values. The larger the value, the shorter the period or the more frequent the fluctuations.

[0045] In summary, the time-domain characteristics of the target sliding window include window standard deviation, variance of the first-order difference sequence, relative difference variance, kurtosis, and zero-crossing rate.

[0046] S21, in extracting the frequency domain features of the target sliding window, includes: S21-1 uses Fast Fourier Transform to calculate the spectral amplitude of the target sliding window.

[0047] Specifically, spectral amplitude The calculation formula is: ; in, .

[0048] S21-2, calculate the main frequency, main frequency energy ratio and spectral entropy of the target sliding window based on the spectral amplitude.

[0049] Specifically, based on the spectral distribution, the dominant frequency index Defined as main frequency for The main frequency energy ratio is .when A high value indicates that the signal energy is concentrated in a single frequency band and exhibits obvious periodicity. To quantify the uniformity of the spectral energy distribution, spectral entropy is introduced. , The calculation formula is: ; .

[0050] Among them, spectral entropy The range of values ​​is ,when When energy is concentrated at a single frequency, the signal exhibits strong periodicity; when When the energy distribution is uniform, the signal tends to be aperiodic and smooth.

[0051] In summary, the frequency domain characteristics of the target sliding window include spectral amplitude, dominant frequency, dominant frequency energy ratio, and spectral entropy.

[0052] S21, when extracting the autocorrelation features of the target sliding window, includes: calculating the standardized autocorrelation function of the target sliding window, and determining the main period delay, local peaks, and autocorrelation peaks based on the standardized autocorrelation function.

[0053] Specifically, to more stably identify periodic structures, the standardized autocorrelation function of the target sliding window is calculated: ; Through testing The local peak value can be used to obtain the main period delay of the target sliding window. It is a local peak and ,in, The significance threshold is an empirical value. The autocorrelation peak value of the main period. The higher the value, the more significant the periodicity of the signal.

[0054] In summary, the autocorrelation characteristics of the target sliding window include main period delay, local peaks, and autocorrelation peak values.

[0055] It should be noted that by performing feature extraction on each sliding window of the target original time series data stream according to the feature extraction method of the target sliding window described above, the time domain features, frequency domain features, and autocorrelation features of each sliding window can be obtained.

[0056] In one specific embodiment, step S22 mainly utilizes the combination of periodicity, autocorrelation characteristics, spectral characteristics, and smoothness indices to automatically classify the original time-series data stream into four categories: long-period smoothing, short-period smoothing, sawtooth smoothing, and aperiodic smoothing, specifically including steps S221 to S223: S221, based on the autocorrelation peak value in the window feature set and the ratio of main frequency energy Determine whether the original time-series data stream of the target within the target sliding window is periodic; when it satisfies or At that time, it is determined that the original time-series data stream of the target within the target sliding window is periodic, where, and These are the autocorrelation peak threshold and the dominant frequency energy ratio threshold, respectively; when the following conditions are met... and At that time, it is determined that the original time-series data stream of the target within the target sliding window is aperiodic.

[0057] S222, when the target's original time-series data stream within the target sliding window is periodic, calculate the period length T of the target's original time-series data stream within the target sliding window. At times, or, in the case of an unknown sampling rate, the main cycle delay. If the target original time series data stream in the target sliding window is determined to be of long-period smooth type, then the target original time series data stream in the target sliding window is determined to be of short-period smooth type. For the long-period time domain length threshold, This is the threshold for the number of sampling points in a long period. The formula for calculating the period length T is: .

[0058] If the periodicity condition is not met, then further consideration is given based on smoothness. Determine whether the spectrum distribution characteristics belong to the non-periodic smooth type. See step S223 for details.

[0059] S223, based on relative difference variance Spectral entropy and kurtosis To determine whether the original time-series data stream of the target within the target sliding window is of aperiodic smooth type, the rule is defined as follows: ; in, , , These are the smoothness threshold, the spectral entropy threshold, and the kurtosis threshold, respectively. When none of the above conditions are met, the original time-series data stream of the target within the target sliding window is determined to be sawtooth-shaped.

[0060] Furthermore, to enhance the robustness of sawtooth pattern recognition, an average slope ratio between the rising and falling phases is introduced. : ; in, and These are the average differences between the rising and falling phases, respectively. or When the confidence level increases, it indicates that the signal (the original time-series data stream of the target within the target sliding window) exhibits a significant sawtooth pattern. The set of difference values ​​for the rising phase is represented as follows: The set of differences during the descent phase is represented as: .

[0061] ; ; in It is the number of positive difference scores. It represents the number of negative difference values.

[0062] Preferably, in order to characterize the reliability of the classification results, embodiments of the present invention also calculate the confidence level for each data type. In other words, after S22, it also includes: S23, calculate the confidence level for each type of data based on the data type of the target raw time-series data stream within the sliding window, where: Periodic confidence level The calculation formula is: ; Non-periodic confidence level The calculation formula is: ; in, For relative difference variance, The median of the relative difference variance index in the historical sliding window is used to normalize the current smoothness. Confidence level of the sawtooth pattern The calculation formula is: ; in, For logical mapping functions, , These are the weighting coefficients. , These are historical baseline parameters, and these are empirical values.

[0063] S24. For the target original time-series data stream located in the overlapping area of ​​adjacent sliding windows, calculate its score for each data type, and take the data type with the highest score as its data type.

[0064] Specifically, due to the overlap between adjacent sliding windows, in order to maintain temporal continuity and noise resistance stability, this embodiment of the invention smooths the data classification results of consecutive sliding windows through a weighted voting method. Let the first... The type label for each sliding window is (The data type of the target's original time-series data stream within the target sliding window, determined by S22), with a confidence level of The sliding window covers a time range of 1000 days. , If the values ​​are long-period smoothing, short-period smoothing, sawtooth smoothing, and aperiodic smoothing, then for any time... Define the scoring function : ;

[0065] in, For indicator functions, when When the value is 1, it is 1; otherwise, it is 0.

[0066] The one with the highest final category score: .

[0067] For example, sliding window 1 covers 0-10 seconds, sliding window 2 covers 5-15 seconds, and sliding window 3 covers 10-20 seconds. The target raw time-series data stream is segmented and analyzed using S22. However, a single time point (e.g., 10 seconds) may be covered by multiple sliding windows simultaneously, and different sliding windows may classify that time point as having different data types (e.g., sliding window 1 classifies it as long-period smoothing, while sliding window 3 classifies it as non-periodic smoothing). During the classification process, the confidence score C is calculated, and then Li iterates through all... The system calculates scores and then compares the three data types. The data type with the highest score is chosen as the final data type.

[0068] It should be noted that if the highest confidence level is below the threshold... If the data type within the sliding window is not specified, it is marked as "uncertain". The final label sequence is further smoothed by sliding median filtering to remove isolated jumps.

[0069] S3 performs outlier detection on each raw time-series data stream based on its data type.

[0070] Based on the above, S3 includes the following situations: S31, When the target original time series data stream is a long-period smooth type, the coal mine sensor data anomaly detection algorithm based on proximity time series feature learning performs outlier detection on the target original time series data stream.

[0071] Specifically, analysis of long-term smoothing patterns in underground coal mines reveals three main types of anomalies: persistent dense noise anomalies, instantaneous pulse anomalies, and data gap anomalies. These problems severely restrict the reliability and usability of the data. Persistent dense noise anomalies, such as... Figure 3 As shown in Figure a, the data exhibits a clear dense clustering pattern over time; the outlier data points are not isolated but appear in clusters over consecutive time periods. Instantaneous pulse anomalies are shown in... Figure 3 As shown in Figure b, it exhibits strong local bias, where a single data point or a very small number of points momentarily differs significantly from the normal data stream composed of their immediate and adjacent data points. The handling of missing data varies across mines; generally, it involves marking it as Null, 0, or -1. In this embodiment, the selected coal mine marks missing data as -1. The data missing anomaly is as follows: Figure 3 As shown in Figure c. Figure 3 In the diagram, blue dots represent normal data from the original dataset, red dots represent abnormal data, and the green line represents the original data curve.

[0072] Given that anomalies exhibit distinct high-density clustering and significant local deviations in the spatiotemporal dimension, in order to detect these anomalies, this embodiment of the invention selects the distance-based KNN algorithm and the density-based LOF algorithm from the proximity anomaly detection algorithms for long-period smooth anomalies.

[0073] However, relying solely on distance or density information between adjacent points is often insufficient to fully capture the complex temporal dependencies (such as periodic fluctuations, trend changes, and long-term / short-term dependencies) inherent in sensor data. To more comprehensively and accurately identify complex anomaly patterns in the original time-series data stream and further improve the robustness and intelligence of detection, this invention proposes an anomaly detection algorithm for coal mine sensor data based on proximity-based temporal feature learning. This algorithm introduces deep temporal feature engineering and combines it with the high-performance gradient boosting decision tree model XGBoost to construct the final anomaly discrimination model.

[0074] The technical architecture of the coal mine sensor data anomaly detection algorithm based on proximity temporal feature learning is as follows: Figure 4As shown, the raw time-series data from downhole sensors is first acquired in real time via OPC or MQTT protocol and transmitted through a Kafka message queue. Z-score standardization is then performed to eliminate dimensional differences. Next, based on the spatial distribution characteristics of the abnormal data, a combination of proximity anomaly detection algorithms (KNN+LOF) is used to perform preliminary anomaly screening, and anomaly points are automatically labeled as the tag vectors y required for subsequent supervised learning. Then, time-series features are extracted using a dual-scale sliding window, and a multi-scale fusion time-series feature engineering model is constructed to fully mine the time-series information, forming the first multi-dimensional feature matrix X. Finally, the first multi-dimensional feature matrix X is input into a pre-trained anomaly detection XGBoost model for anomaly detection.

[0075] In one specific embodiment, when the coal mine sensor data anomaly detection algorithm based on proximity temporal feature learning performs anomaly detection on the target original temporal data stream, S31 includes S311 to S314: S311 standardizes the target raw time-series data stream.

[0076] Specifically, the standardization process employs the Z-score standardization algorithm. Standardization eliminates dimensional differences between data points, thereby improving the performance and stability of subsequent detection algorithms.

[0077] S312 performs outlier screening on the standardized target original time-series data stream using the distance-based KNN outlier detection algorithm and the density-based LOF outlier detection algorithm, respectively, and takes the intersection of the two as the outlier screening result, and generates outlier labels for the outliers included in the outlier screening result.

[0078] Specifically, after Z-score standardization, an initial outlier screening is performed. Its main purpose is to quickly identify and label data points that significantly deviate from the normal distribution without affecting the overall data structure. This provides a preliminary basis and data selection foundation for subsequent, more complex anomaly detection models. The initial outlier screening uses an unsupervised learning method, which does not rely on labeled data and is suitable for large-scale, high-dimensional time-series datasets. The outliers obtained through this initial screening can serve as training samples for subsequent supervised models, thus achieving a transition from unsupervised to supervised methods, avoiding the process of manual labeling, and improving the overall accuracy and efficiency of detection.

[0079] Proximity-based outlier detection (POD) is an anomaly identification technique based on spatial distribution characteristics, including distance-based, density-based, and clustering-based outlier detection. Based on the spatial distribution characteristics of raw time-series data from underground coal mine sensors, this embodiment of the invention selects the distance-based KNN outlier detection algorithm and the density-based LOF outlier detection algorithm for initial outlier screening. The KNN method performs well when the data distribution is relatively uniform, especially suitable for datasets with small density variations. However, when the data density varies significantly, KNN is prone to misjudgments. For example, in sparse regions, normal points may have large KNN distances, thus being incorrectly identified as anomalies; while in high-density regions, true anomalies may go undetected due to their numerous neighbors. The LOF algorithm, on the other hand, is a local density-based outlier detection algorithm that determines whether a data point is an anomaly by calculating the local reachability density and local anomaly factor for each data point. Compared to KNN, LOF can more accurately identify local anomalies, especially performing better when the data distribution is uneven. Therefore, a joint detection of KNN and LOF algorithms is used to construct an initial outlier screening mechanism.

[0080] S313 extracts lag features, statistical features, difference features, FFT features, and time coding features from the standardized target original time series data stream through a dual-scale sliding window, and constructs a first multi-dimensional feature matrix based on the lag features, statistical features, difference features, FFT features, and time coding features.

[0081] Specifically, the raw time-series data stream from downhole sensors exhibits significant temporal characteristics. To fully exploit temporal information and extract features from the raw time-series data stream that effectively reflect data patterns and trends, this invention employs a dual-scale sliding window to extract temporal features from the raw time-series data stream, including hysteresis features, statistical features, differential features, FFT features, and time coding. This constructs a multi-scale fused temporal feature engineering model, such as... Figure 5 As shown.

[0082] Lag features construct the correlation between historical and current states by shifting time series data backward a specific number of steps along the time axis. Specifically, they include short-window lag and long-window lag. Short-window lag typically selects historical data closer to the current moment, reflecting the continuity of the system's state within a small timescale, and is used to capture short-term fluctuation patterns. Long-window lag, on the other hand, focuses on historical information over a longer time span to reflect periodic or long-term dependencies.

[0083] Statistical features are calculated by measuring statistics of time-series data within a specific time window, and are also divided into short-window statistics and long-window statistics. Short-window statistics may include the mean, variance, maximum, or minimum values ​​over the past few hours, used to characterize local data distribution features; long-window statistics perform similar calculations over a longer period, helping anomaly detection models identify statistical patterns in long-term trends or seasonal fluctuations. By calculating statistics within a window, local data distribution characteristics can be captured.

[0084] Difference features are used to eliminate trends or periodicity in time series and enhance the stationarity of data. They mainly include first-order differencing, short-span differencing, and long-span differencing. First-order differencing reflects the instantaneous rate of change by the difference between the current value and the previous value; short-span differencing and long-span differencing capture the changing trends at different time scales, respectively.

[0085] The Fast Fourier Transform (FFT) feature can convert a time-domain signal into a frequency-domain signal, thereby extracting the frequency characteristics of the signal. By performing an FFT on short-window data, the average amplitude of low-frequency and mid-frequency components can be extracted as features to reflect slow-changing trends and capture periodic fluctuations.

[0086] Time-based encoding features can transform periodic time information into continuous features to better capture the periodicity and regularity of time. For periodic time units such as hours and minutes, trigonometric function encoding is used to avoid problems such as numerical breaks, and the day of the week is added as a feature.

[0087] Furthermore, for each time point ,from The five types of features mentioned above are extracted starting from when the window begins to slide, forming a feature vector. ,in For feature dimension, The length of the long window. The first multidimensional feature matrix. The number of rows represents the number of time points, and the number of columns represents the feature dimension d. ;

[0088] Corresponding label vector Generated from the initial screening anomaly detection results: ; in, .

[0089] S314, input the outliers and their outlier labels, and the first multidimensional feature matrix, including the initial outlier screening results, into the pre-trained anomaly detection XGBoost model, and the anomaly detection XGBoost model outputs the outliers in the target original time series data stream.

[0090] It should be noted that before step S314, the anomaly detection XGBoost model needs to be trained first. Specifically, after obtaining the sample data, since the proportion of abnormal data in the total data is only about 5%, the ratio of positive to negative samples is severely imbalanced. Therefore, negative samples (normal values) are undersampled to make the ratio of positive to negative samples closer to 1:4, and the first multidimensional feature matrix is ​​constructed using temporal feature engineering. and label vector As input to supervised learning algorithms.

[0091] To ensure the reliability and generalization ability of the temporal anomaly detection model, after constructing the supervised sample set, a strategy of dividing the data into training, validation, and test sets according to time sequence was adopted. The first 60% of the samples were used to train the model, the next 20% were used as the validation set for threshold selection and hyperparameter tuning, and the last 20% were used as the test set for final performance evaluation. The temporal order of the data was strictly maintained during the partitioning process to ensure that the validation and test sets included future data not seen in the training set, avoiding information leakage and simulating real-world online monitoring scenarios. Finally, the XGBoost classification model was used to calculate the anomaly probability, and precision and recall were calculated. The PR curve was analyzed to optimize the discrimination threshold, and the F1 score was used as the core evaluation metric to confirm the optimal threshold.

[0092] Common classification models include logistic regression, decision trees, support vector machines, random forests, LSTM, Transformer, and XGBoost. Logistic regression, as a classic linear model, is simple in structure, fast in training, and highly interpretable, with some robustness to noise, but it struggles to effectively capture nonlinear feature relationships in high-dimensional time-series data. Decision trees offer good interpretability and intuitiveness, automatically selecting features and handling missing values, performing reasonably well on small datasets, but their stability is poor, prone to overfitting, and they struggle to characterize the continuity and dynamic evolution of time-series data. Support vector machines have high computational complexity, long training times, and are sensitive to missing values, making them unsuitable for real-time monitoring. Random forests can handle high-dimensional nonlinear data, but their fixed-tree-structure ensemble approach has limitations in capturing complex feature interactions. LSTM and Transformer, as deep learning methods, have advantages in modeling long sequences and capturing temporal dependencies, but they may still be affected by sparse or noisy data environments. In contrast, XGBoost overcomes the limitations of the aforementioned models. By constructing multi-dimensional temporal feature engineering, it can dynamically model spatiotemporal feature interactions; its built-in sample weighting mechanism can automatically identify and compensate for sample distribution imbalances, thereby improving the recognition ability of minority classes; and its gradient boosting mechanism and regularization strategy further enhance the model's nonlinear discrimination ability and robustness. Therefore, this embodiment of the invention selects XGBoost as the core model for anomaly detection in coal mine sensor data, and compares its performance with other models in subsequent experiments. The model performance comparison results are shown in Table 2.

[0093]

[0094] The results show that XGBoost performs best across all evaluation metrics, demonstrating its superior performance in anomaly detection. Regarding the meaning of each metric: ROC and AUC reflect the model's overall classification ability, unaffected by the classification threshold. XGBoost achieves 0.9953 on this metric, significantly outperforming other models, indicating its strong ability to distinguish between positive and negative samples, mainly due to its ensemble learning characteristics and modeling ability for complex nonlinear relationships. Accuracy reflects the proportion of correct predictions by the model. While it may have biases in imbalanced datasets, it still has reference value. XGBoost, Transformer, and Random Forest perform excellently on this metric, indicating their advantages in overall prediction accuracy. Precision and recall measure the model's ability to "precisely" and "completely" detect anomalies, respectively. Both metrics are crucial in anomaly detection tasks: low precision means normal samples are misclassified as anomalies, and low recall means anomaly samples are missed. XGBoost achieved a good balance between precision and recall; Transformer also demonstrated high overall performance, while Decision Tree, although having high recall, had relatively low precision and a high number of false positives; LSTM lagged significantly behind in both precision and recall, indicating severe missed detections; Support Vector Machine's recall was only 0.4270, showing similarly limited performance. The F1 score, as the harmonic mean of precision and recall, comprehensively reflects the model's balanced performance. XGBoost significantly outperformed the other models, followed by Transformer, Random Forest performed robustly, while LSTM and Support Vector Machine were relatively poor.

[0095] In summary, XGBoost demonstrates superior performance in hydraulic support pressure anomaly detection tasks due to its powerful nonlinear modeling capabilities, adaptive handling of feature importance, and good adaptability to imbalanced data. Transformer performs near-optimal in long sequence modeling, while traditional single-model methods and LSTM still have certain limitations under complex nonlinear and high-noise data. XGBoost provides an effective reference and solution for anomaly detection in industrial equipment.

[0096] It should be noted that short-period smooth anomaly detection can be achieved simply by "real-time and refining" a few parameters and operating strategies of the long-period smooth anomaly detection method. The main adjustments include: reducing the lag order and shortening the sliding window length to improve temporal resolution; lowering the number of neighbors K and the initial screening threshold in KNN / LOF to avoid false alarms during high-frequency oscillations; using short-window FFT and differential features to enhance sensitivity to high-frequency pulses; fine-tuning the undersampling ratio based on the anomaly proportion under short periods; adjusting the tree depth, number of estimators, and number of iterations of XGBoost to meet real-time requirements while retaining early stopping and regularization; and verifying that time-based segmentation and PR / F1 threshold selection still allow for efficient and robust short-period anomaly detection within the original framework.

[0097] S32, When the target's original time-series data stream is sawtooth-shaped, the isolated forest coal mine sensor data anomaly detection algorithm based on feature-based diversion performs outlier detection on the target's original time-series data stream.

[0098] In multi-source time-series data acquired by downhole sensors, data such as angle, temperature, and current typically exhibit periodic sawtooth waveforms. Compared to smooth periodic data, sawtooth data shows more dramatic changes during the rising and falling phases, with frequent local fluctuations, exhibiting obvious non-stationarity and nonlinear characteristics. This type of data often displays complex local density structures and periodic jump characteristics in its spatial distribution. On the other hand, angle signals possess circular periodicity, while non-angle signals such as temperature and current exhibit linear distributions, showing significant differences in numerical characteristics and variation patterns. Therefore, it is necessary to extract features from angle and non-angle sawtooth data separately to accurately characterize their inherent patterns. To address these characteristics, this invention proposes an anomaly detection algorithm for isolated forest coal mine sensor data based on feature-based diversion, with the technical architecture as follows: Figure 6 As shown, the raw time-series data streams acquired by downhole sensors are first collected in real time via OPC or MQTT protocols and transmitted through a Kafka message queue. Differential feature processing is then performed based on data type: for non-angle numerical signals, a sliding window is used to extract statistical features such as linear difference, rate of change, rolling mean / median / standard deviation to characterize short-term / long-term behavior; for angle signals, the sliding window features and rate of change features are retained, and trigonometric function codes (sin / cos) and circular difference are calculated to characterize circular jump characteristics. Robust Scaler is then used for robust normalization to eliminate scale differences and suppress outlier effects. Subsequently, an Isolation Forest model is used for unsupervised anomaly detection, with the proportion of outliers controlled by setting the contamination parameter; finally, adjacent point jump detection (using a circular difference threshold for angle data) is combined for final anomaly determination.

[0099] In one specific embodiment, S32 includes S321 to S327: S321, determine whether the target's original time-series data stream is angle data.

[0100] S322, if the original time series data stream of the target is angle data, then convert the original time series data stream of the target into radian data stream, and extract the statistical features, rate of change features, trigonometric function features and circumferential difference features of the radian data stream.

[0101] Specifically, for angle data, a specialized feature processing method is employed to retain the same sliding window statistical features and rate of change features as non-angle signals. Linear difference features are replaced with trigonometric function features and circular difference features. Trigonometric function features convert angle values ​​into sine and cosine values ​​(sin / cos), effectively preserving the periodicity of the angle data and avoiding numerical jumps at the boundary between 0° and 360°. Circular difference features use a circular distance calculation method to accurately calculate the shortest circular distance between adjacent angle values, effectively characterizing the circular jump characteristics of the angle data. This accurately reflects the periodicity and rotational behavior of the angle data.

[0102] S323, If the target original time series data stream is non-angular data, then the target original time series data stream is standardized, and the statistical characteristics, rate of change characteristics and linear difference characteristics of the standardized original time series data stream are extracted.

[0103] Specifically, for conventional non-angular sawtooth data (such as temperature, pressure, and current), the sliding window technique is used to extract multi-dimensional time-series features, including linear difference features, rate of change features, and statistical features. Linear difference features: By calculating the absolute value of the difference between values ​​at adjacent time points, the instantaneous changes in the data are captured, reflecting the dynamic response characteristics of the system. Rate of change features: The linear difference value is divided by the standard deviation within the sliding window to obtain a standardized rate of change index, used to measure the relative drasticness of data changes. Statistical features: Within the sliding window, statistics such as the mean, median, and standard deviation are calculated to characterize the distribution characteristics and stability of the data on short and long time scales. These features can effectively characterize the short-term fluctuation patterns and long-term trend changes of non-angular signals.

[0104] S324. Based on the statistical characteristics, rate of change characteristics, trigonometric function characteristics, and circumferential difference characteristics of the radian data stream and the statistical characteristics, rate of change characteristics, and linear difference characteristics of the target original time-series data stream after standardization, a second multidimensional feature matrix is ​​constructed, and the features in the second multidimensional feature matrix are standardized using a robust scaler.

[0105] Specifically, after completing the differential feature extraction, a robust scaler is used to perform robust normalization on all features, effectively suppressing the impact of outliers on the standardization process, improving the robustness of feature processing, and eliminating dimensional differences between different features.

[0106] S325 inputs the standardized second multidimensional feature matrix into the pre-trained anomaly detection isolation forest model, and the anomaly detection isolation forest model outputs the outliers in the target original time series data stream.

[0107] Specifically, the Isolation Forest model is used for unsupervised anomaly detection. This model isolates outliers by constructing random decision trees, where outliers typically have short path lengths. By setting the contamination parameter, the expected proportion of outliers is controlled, enabling the Isolation Forest model to adapt to the distribution characteristics of outlier data in different scenarios.

[0108] S326, based on the circumferential difference features and linear difference features, adjacent point jump detection is performed on the target original time series data stream to obtain the jump point detection results.

[0109] Specifically, to improve detection accuracy, an adjacent point jump detection mechanism is introduced as a supplement: for angular data, a circular difference threshold is used for jump detection; when the circular difference between adjacent points exceeds a preset threshold, it is marked as an anomaly. For non-angular data, a linear difference threshold is used for jump detection.

[0110] S327. Based on the outlier and jump point detection results in the target's original time series data stream output by the anomaly detection isolated forest model, determine the outliers in the target's original time series data stream.

[0111] Specifically, the final anomaly determination result is a logical OR combination of the detection results from the isolated forest model and the jump point detection results; that is, if either method detects an anomaly, it is determined to be an anomaly. This fusion strategy can effectively improve the recall rate of anomaly detection and avoid the missed detection problem of a single method.

[0112] S33, When the target original time series data stream is non-periodic and smooth, the coal mine sensor data anomaly detection algorithm based on physical constraints and robust statistical characteristics performs outlier detection on the target original time series data stream.

[0113] Specifically, in the multi-source time-series data streams acquired by underground sensors, some data exhibit a non-periodic and generally smooth trend. This type of data typically has small fluctuations, a gentle rate of change, and lacks obvious periodic patterns, exhibiting strong stability and continuity. Compared to periodic data, anomalies in non-periodic smooth data often manifest as sudden jumps, abnormal peaks, or drift shifts, and are usually distributed in the extreme ranges of the data. Therefore, periodic feature extraction methods are difficult to effectively capture anomalies in this type of data, requiring more robust statistical detection strategies. Based on the above analysis, this invention proposes an anomaly detection algorithm for coal mine sensor data based on physical constraints and robust statistical features for non-periodic smooth data. The technical architecture is as follows: Figure 7 As shown.

[0114] In one specific embodiment, S33 includes S331 to S334: S331, Perform physical boundary detection on the target's original time-series data stream, and determine whether any data in the target's original time-series data stream exceeds the upper and lower threshold ranges of the corresponding sensor. If it exceeds, mark it as an outlier.

[0115] Specifically, physical upper and lower threshold values ​​are set based on the sensor's engineering measurement range or the equipment's rated operating range. When data exceeds these limits, it is determined to be abnormal data.

[0116] S332, if not exceeded, calculate the MAD of the target original time-series data stream; if MAD is valid (MAD is greater than the preset threshold of 10), then... -9 When MAD is invalid (MAD is less than or equal to a preset threshold of 10), outliers in the target's original time-series data stream are detected based on the Modified Z-score algorithm; when MAD is invalid (MAD is less than or equal to a preset threshold of 10), the algorithm detects outliers in the target's original time-series data stream. -9 When the target is in the original time-series data stream, the IQR detection algorithm is used to detect outliers.

[0117] Specifically, after filtering out extreme values ​​through physical constraints, a Modified Z-score algorithm based on Median Absolute Deviation (MAD) is used for statistical anomaly identification. The Modified Z-score algorithm, which centers detection on the median rather than the mean, effectively suppresses the influence of extreme values ​​on the statistics and is suitable for smooth signals with low noise.

[0118] Let the observation sequence be If the median is m, then the Modified Z-score is defined as: ;

[0119] ; when If the MAD exceeds a set threshold, the point is considered an outlier. If the MAD is too small, causing a decrease in stability, the algorithm automatically switches to the IQR detection algorithm for auxiliary judgment, thereby ensuring the robustness and generalization of the algorithm.

[0120] S333 performs differential mutation detection on the target's original time-series data stream.

[0121] Specifically, to address the characteristic of slow changes in non-periodic smoothed data over time, a differential threshold detection is further introduced. When the difference between values ​​at adjacent time points exceeds a set threshold, it indicates a sudden change or abnormal transition, which can be identified as abrupt anomalies.

[0122] S334, determine the outliers in the target's original time-series data stream based on the physical boundary detection results, MAD-based detection results, and differential mutation detection results.

[0123] The detection results based on MAD include those based on the Modified Z-score algorithm and the IQR detection algorithm. The final anomaly determination result is obtained by logically ORing and fusing the results of physical boundary detection, robust statistical detection, and differential detection to ensure that extreme anomalies are captured.

[0124] Through the above detection, high-precision identification of extreme outliers, mutation points, and drift anomalies can be achieved in non-periodic smoothing scenarios.

[0125] S4 adaptively repairs outliers in each type of original time series data based on the data type of each original time series data stream.

[0126] Specifically, after outlier detection of the original time-series data stream, in order to recover outliers and missing segments caused by sensor failure, signal drift, or data interruption, differential interpolation is used based on data type to complete the continuity of the original time-series data stream and equal-time resampling, thereby obtaining high-fidelity, time-consistent continuous data. This embodiment of the invention employs differential repair for different types of time-series signals to preserve the statistical characteristics and dynamic trends of the original data to the greatest extent possible.

[0127] In one specific embodiment, S4 includes: S41, When the target original time series data stream is of the long-period smooth type, the seasonal autoregressive integral moving average model is used to adaptively repair the outliers in the target original time series data.

[0128] Specifically, for long-period smoothing models, which typically exhibit obvious seasonality and trends, a Seasonal Auto-Regressive Integrated Moving Average (SARIMA) model is used for correction. The SARIMA model can capture periodic components by introducing a seasonal difference operator, and its mathematical form can be expressed as: ; in, For lag operators, Indicates the length of the season. and These are autoregressive and moving average polynomials, respectively. and This is a seasonal term. The model order is selected by minimizing the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC). After training the model using normal samples within a window to obtain the parameters, first-order predictions are performed on outliers to obtain the corrected values. SARIMA can effectively suppress random disturbances and restore the global trend in stress signals with stable trends and significant cycles.

[0129] S42, when the target original time series data stream is a short-period smooth type, a bidirectional long short-term memory network combined with a self-attention model is used to adaptively repair outliers in the target original time series data.

[0130] Specifically, for short-period smooth-type data, which have short periods, frequent changes, but smooth overall fluctuations, traditional linear prediction often struggles to capture their phase relationships. Therefore, this invention employs a bidirectional long short-term memory network combined with a self-attention mechanism (SL-BiLSTM-SA) for outlier repair. The SL-BiLSTM-SA model considers both historical and future time dependencies within a local window. Its basic structure includes a forward LSTM, a backward LSTM, and a self-attention module. The hidden state vectors obtained from the bidirectional structure are concatenated and input into the attention layer. A weighted summation method is used to model local importance, improving the sensitivity of the SL-BiLSTM-SA model to phase and amplitude changes. The training objective of the SL-BiLSTM-SA model uses masked mean squared error (MSE) loss, calculating errors only at outlier locations to prevent overfitting.

[0131] S43, When the target original time series data stream is jagged, a Transformer model based on self-attention mechanism is introduced to adaptively repair outliers in the target original time series data.

[0132] Specifically, for sawtooth-shaped data streams, the original time-series data exhibits periodic abrupt changes and non-smooth jumps, with significant local variations. Data restoration for this type of data requires maintaining both the abruptness of the boundaries and the smoothness of the internal structure. This is addressed by introducing a Transformer model based on a self-attention mechanism for interpolation reconstruction. The Transformer model based on self-attention learns the dependencies at different time scales through a multi-head self-attention structure; its attention calculation process can be represented as follows: ; in, These are query, key, and value matrices, respectively. The key vector dimension is used. The multi-head mechanism of Transformer enables it to capture both long-term dependencies and short-term mutations simultaneously, exhibiting good shape preservation in boundary recovery of jagged data.

[0133] S44. When the target original time series data stream is non-periodic and smooth, a unidirectional LSTM model with an attention mechanism is used to adaptively repair outliers in the target original time series data.

[0134] Specifically, for aperiodic smoothing models, the changes are irregular but the overall trend is stable. To enhance the model's robustness to random perturbations, a one-way LSTM model (LSTM-SA) with an attention mechanism is used for repair. First, LSTM units are used to extract the latent dynamic features of the time series, and then the importance weights of each historical time point are calculated through an attention layer. ; ; ; Ultimately, the context vector will be With the current hidden state The predicted output is obtained after splicing. The one-way LSTM model with an attention mechanism can effectively preserve smooth trends and suppress random noise in aperiodic fluctuating data.

[0135] It should be noted that by processing each type of raw time-series data stream in the same way as the target raw time-series data stream described above, the data type, outlier detection results, and outlier adaptive repair results of each type of raw time-series data stream can be obtained.

[0136] S5 performs time alignment and equal time interval reconstruction on all types of original time-series data streams according to the data type of each original time-series data stream.

[0137] In one specific embodiment, S5 includes: S51: Collect timestamps of all types of raw time-series data streams, take the union of all timestamps to generate a global timeline, and use the global timeline as a reference to perform time-aligned interpolation on each type of raw time-series data stream using a hierarchical interpolation strategy. Specifically: for long-period smooth raw time-series data streams, cubic spline interpolation is used; for short-period smooth and sawtooth types, piecewise cubic Hermite interpolation is used; and for non-periodic smooth types, a time-weighted algorithm is used for linear interpolation.

[0138] Specifically, for long-period smooth types, a cubic spline algorithm is used for interpolation. Let the interval be... The cubic polynomial on is: ; By ensuring the continuity of function values, first derivatives, and second derivatives, and by setting natural boundary conditions... The coefficient matrix can be obtained and solved. This algorithm can maintain the continuity and differentiability of the overall curve in periodic and smooth data.

[0139] For short-period smooth and sawtooth splines, piecewise cubic Hermite interpolation (PCHIP) is employed. This algorithm avoids the overshoot problem that may occur with traditional splines by introducing endpoint derivative constraints in each interval. Its interval interpolation function can be expressed as: ; in, , These are standard Hermite basis functions. This represents the endpoint derivative under monotonicity constraints. PCHIP interpolation has significant advantages in preserving data monotonicity and abrupt boundary conditions, and is suitable for signals with rapid local changes.

[0140] For aperiodic smoothing, since its sampling time intervals may be uneven and lack obvious periodic characteristics, the time-weighted linear interpolation method is used. Let the time between adjacent valid points be... Its value is Then at any time The interpolation result can be expressed as: ; When the sampling interval is large, a local weighted regression term can be introduced in the neighborhood to correct the time drift trend.

[0141] S52 performs equal-interval interpolation on all types of original time-series data streams after time-aligned interpolation and stores all types of data after equal-interval interpolation in the time-series database.

[0142] Specifically, based on time-aligned interpolation, equal-interval interpolation is performed with a set sampling step size. (For example, 1 second, or 0.5 seconds, 10 seconds, etc.) Resample the entire sequence to obtain a unified timeline. If a certain resampling time point If no precise observations were found in the original observation timestamps, a hierarchical interpolation strategy is used for data backfilling. The interpolation method is as described above.

[0143] After completing adaptive repair and time alignment for all data types, the data streams are integrated according to their timestamps to construct a unified and complete time-series dataset. Finally, this dataset is persistently stored in a time-series database for subsequent analysis and applications.

[0144] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors, characterized in that, include: S1, real-time acquisition of various raw time-series data streams collected by multiple types of sensors in underground coal mines; S2, identify the data type of each type of original time-series data stream using a time-series data stream type identification algorithm. The data types include long-period smoothing, short-period smoothing, sawtooth, and aperiodic smoothing. Define any type of original time-series data stream as the target original time-series data stream. S2 includes: S21, extracting the time-domain features, frequency-domain features, and autocorrelation features of each sliding window of the target original time-series data stream using a sliding window, forming a window feature set for each sliding window; define any sliding window as the target sliding window. The time-domain features of the target sliding window include relative difference variance and kurtosis; the frequency-domain features of the target sliding window include the main frequency energy ratio and spectral entropy; and the autocorrelation features of the target sliding window include the main period delay and autocorrelation peak value. S22, identify the data type of the target original time-series data stream within the sliding window based on the window features in the window feature set. Wherein, S22 includes: S221, based on the autocorrelation peak value in the window feature set and the ratio of main frequency energy Determine whether the original time-series data stream of the target within the target sliding window is periodic; when it satisfies or At that time, it is determined that the original time-series data stream of the target within the target sliding window is periodic, where, and These are the autocorrelation peak threshold and the dominant frequency energy ratio threshold, respectively; when the following conditions are met... and At that time, it is determined that the original time-series data stream of the target within the target sliding window is aperiodic; S222, when the target's original time-series data stream within the target sliding window is periodic, calculate the period length T of the target's original time-series data stream within the target sliding window. At times, or, in the case of an unknown sampling rate, the main cycle delay. If the target original time series data stream in the target sliding window is determined to be of long-period smooth type, then the target original time series data stream in the target sliding window is determined to be of short-period smooth type. For the long-period time domain length threshold, The threshold for the number of sampling points in a long period; S223, based on relative difference variance Spectral entropy and kurtosis To determine whether the original time-series data stream of the target within the target sliding window is of aperiodic smooth type, the rule is defined as follows: ； in, These are the smoothness threshold, the spectral entropy threshold, and the kurtosis threshold, respectively. When none of the above conditions are met, the original time-series data stream of the target within the target sliding window is determined to be sawtooth-shaped. S3, perform outlier detection on each raw time series data stream according to the data type of each raw time series data stream; S4, adaptively repair outliers in each type of original time series data according to the data type of each type of original time series data stream; S5 performs time alignment and equal time interval reconstruction on all types of original time-series data streams according to the data type of each original time-series data stream.

2. The method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors according to claim 1, characterized in that, S21, in extracting the temporal features of the target sliding window, includes: S211, DC removal processing is performed on the original target time series data stream within the target sliding window to obtain the DC-removed time series data sequence; S212, calculate the window standard deviation of the target sliding window based on the time series data sequence after DC removal; S213, construct a first-order difference sequence based on the time-series data after DC removal, and calculate the variance of the first-order difference sequence; S214, calculate the relative difference variance based on the variance of the first difference sequence and the window standard deviation; S215, calculates kurtosis based on the time-series data sequence after DC removal; S216, calculate the zero-crossing rate based on the timing data sequence after DC removal; S21, in extracting the frequency domain features of the target sliding window, includes: S21-1, the spectral amplitude of the target sliding window is calculated using fast Fourier transform; S21-2, Calculate the main frequency, main frequency energy ratio and spectral entropy of the target sliding window based on the spectral amplitude; S21, when extracting the autocorrelation features of the target sliding window, includes: calculating the standardized autocorrelation function of the target sliding window, and determining the main period delay, local peaks, and autocorrelation peaks based on the standardized autocorrelation function.

3. The coal mine sensor time-series data anomaly detection and adaptive repair and reconstruction method according to claim 1, characterized in that, Following S22, the following is also included: S23, calculate the confidence level for each type of data based on the data type of the target raw time-series data stream within the sliding window, where: Periodic confidence level The calculation formula is: ； Non-periodic confidence level The calculation formula is: ； in, For relative difference variance, This represents the median of the relative difference variance index within the historical sliding window. Confidence level of the sawtooth pattern The calculation formula is: ； in, For logical mapping functions, , These are the weighting coefficients. , Historical baseline parameters; S24. For the target original time-series data stream located in the overlapping area of ​​adjacent sliding windows, calculate its score for each data type, and take the data type with the highest score as its data type.

4. The method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors according to claim 1, characterized in that, Define any type of raw time-series data stream as the target raw time-series data stream, wherein S3 includes: S31, When the target original time series data stream is a long-period smooth type or a short-period smooth type, the coal mine sensor data anomaly detection algorithm based on proximity time series feature learning performs anomaly detection on the target original time series data stream; S32, When the target's original time-series data stream is sawtooth-shaped, the isolated forest coal mine sensor data anomaly detection algorithm based on feature-based diversion performs anomaly detection on the target's original time-series data stream; S33, When the target original time series data stream is non-periodic and smooth, the coal mine sensor data anomaly detection algorithm based on physical constraints and robust statistical characteristics performs outlier detection on the target original time series data stream.

5. The coal mine sensor time-series data anomaly detection and adaptive repair and reconstruction method according to claim 4, characterized in that, S31, when performing outlier detection on the target original time-series data stream using the coal mine sensor data anomaly detection algorithm based on proximity time-series feature learning, includes: S311, standardizes the target raw time-series data stream; S312 performs outlier screening on the standardized target original time series data stream using the distance-based KNN outlier detection algorithm and the density-based LOF outlier detection algorithm, respectively, and takes the intersection of the two as the outlier screening result, and generates outlier labels for the outliers included in the outlier screening result. S313, extracts lag features, statistical features, difference features, FFT features and time coding features from the standardized target original time series data stream through a dual-scale sliding window, and constructs a first multi-dimensional feature matrix based on the lag features, statistical features, difference features, FFT features and time coding features; S314, input the outliers and their outlier labels, and the first multidimensional feature matrix, including the initial outlier screening results, into the pre-trained anomaly detection XGBoost model, and the anomaly detection XGBoost model outputs the outliers in the target original time series data stream.

6. The coal mine sensor time-series data anomaly detection and adaptive repair and reconstruction method according to claim 4, characterized in that, S32 includes: S321, Determine whether the target's original time-series data stream is angle data; S322, If the original time series data stream of the target is angle data, then convert the original time series data stream of the target into radian data stream, and extract the statistical features, rate of change features, trigonometric function features and circumferential difference features of the radian data stream; S323, If the target's original time series data stream is non-angular data, then the target's original time series data stream is standardized, and the statistical characteristics, rate of change characteristics, and linear difference characteristics of the standardized original time series data stream are extracted. S324. Based on the statistical characteristics, rate of change characteristics, trigonometric function characteristics, and circumferential difference characteristics of the radian data stream and the statistical characteristics, rate of change characteristics, and linear difference characteristics of the target original time series data stream after standardization, a second multidimensional feature matrix is ​​constructed, and the features in the second multidimensional feature matrix are standardized by a robust scaler. S325, the standardized second multi-dimensional feature matrix is ​​input into the pre-trained anomaly detection isolation forest model, and the anomaly detection isolation forest model outputs the outlier values ​​in the target's original time series data stream; S326, Based on the circumferential difference features and linear difference features, adjacent point jump detection is performed on the target original time series data stream to obtain the jump point detection results; S327. Based on the outlier and jump point detection results in the target's original time series data stream output by the anomaly detection isolated forest model, determine the outliers in the target's original time series data stream.

7. The coal mine sensor time-series data anomaly detection and adaptive repair reconstruction method according to claim 4, characterized in that, S33 includes: S331, Perform physical boundary detection on the target's original time-series data stream, and determine whether any data in the target's original time-series data stream exceeds the upper and lower threshold ranges of the corresponding sensor. If it exceeds, mark it as an outlier. S332, if not exceeded, calculate the MAD of the target original time series data stream; when the MAD is valid, detect outliers in the target original time series data stream based on the Modified Z-score algorithm; when the MAD is invalid, use the IQR detection algorithm to detect outliers in the target original time series data stream. S333 performs differential mutation detection on the target's original time-series data stream; S334, determine the outliers in the target's original time-series data stream based on the physical boundary detection results, MAD-based detection results, and differential mutation detection results.

8. The method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors according to claim 1, characterized in that, Define any type of raw time-series data stream as the target raw time-series data stream, wherein S4 includes: S41, When the target original time series data stream is a long-period smooth type, the seasonal autoregressive integral moving average model is used to adaptively repair the outliers in the target original time series data. S42, When the target original time series data stream is a short-period smooth type, a bidirectional long short-term memory network combined with a self-attention model is used to adaptively repair the outliers in the target original time series data. S43, When the target original time series data stream is jagged, a Transformer model based on self-attention mechanism is introduced to adaptively repair outliers in the target original time series data; S44. When the target original time series data stream is non-periodic and smooth, a unidirectional LSTM model with an attention mechanism is used to adaptively repair outliers in the target original time series data.

9. The method for anomaly detection and adaptive repair and reconstruction of time-series data from coal mine sensors according to claim 1, characterized in that, S5 includes: S51: Collect timestamps from all types of raw time-series data streams, take the union of all timestamps to generate a global timeline, and use the global timeline as a reference to perform time-aligned interpolation on each type of raw time-series data stream using a hierarchical interpolation strategy based on the data type of each raw time-series data stream. Specifically: for long-period smooth raw time-series data streams, a cubic spline algorithm is used for interpolation; for short-period smooth and sawtooth types, a piecewise cubic Hermite algorithm is used for interpolation; and for non-periodic smooth types, a time-weighted algorithm is used for linear interpolation. S52 performs equal-interval interpolation on all types of original time-series data streams after time-aligned interpolation and stores all types of data after equal-interval interpolation in the time-series database.

Images