Big data anomaly detection method based on time-series multi-dimensional autoencoder

Abstract

The present application relates to the technical field of data processing, and more particularly to a big data anomaly detection method based on a time series multi-dimensional self-encoder.The method comprises the following steps: step 1: decomposing multi-dimensional time series data, first extracting the internal oscillation mode of the multi-dimensional time series data by using Hilbert-Huang transform, adaptively extracting multi-level space-time characteristics from the multi-dimensional time series data, and reconstructing the multi-level space-time characteristics as a decomposition result; step 2: mapping the decomposition result to a low-dimensional and continuous manifold space; on the obtained manifold space, the structural information of the multi-dimensional time series data is extracted by calculating topological invariants such as persistent homology; step 3: based on the extracted structural information, the local entropy of the multi-dimensional time series data on a set scale is calculated; and according to the local entropy, the anomaly probability is calculated.The present application can efficiently capture short-term sudden anomalies, long-term trend changes and multi-dimensional coupling anomalies.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to a big data anomaly detection method based on a time sequence multi-dimensional self-encoder. BACKGROUND

[0002] With the continuous development of industrial internet, internet of things and smart city scenes, massive multi-source heterogeneous data is explosively growing in various fields, especially in device state monitoring, production process control, financial risk control and smart traffic, and large-scale time sequence data is increasingly accessible. However, how to effectively identify potential anomalies in the massive data containing multi-dimensional, nonlinear and non-stationary characteristics and conduct risk early warning has become a common concern of the academic and industrial circles. Traditional anomaly detection methods are mostly based on statistical analysis or machine learning technology, typical representatives of which include threshold detection methods based on single-dimensional statistics (such as mean, variance, skewness, etc.), dimension reduction detection based on principal component analysis (PCA), and classification models relying on supervised or semi-supervised algorithms such as support vector machines and random forests. These existing technologies can indeed achieve certain effects in small-scale, low-dimensional data scenes, but their shortcomings are increasingly apparent when facing current multi-dimensional coupling, strong noise and non-stationary big data sequences.

[0003] The anomaly detection method based on simple statistics or a single threshold is relatively powerless for sudden and transient anomalies. For example, for sensor data of industrial equipment, some mechanical component failures may only show high-frequency jitter for a very short time and then quickly return to the normal range. If only global statistics such as mean or variance are relied on, it is often difficult to capture such short-time anomalies in time. With the explosive growth of the number of sensors and sampling frequency, the scale and dimension of data have significantly increased, and traditional detection methods based on Euclidean distance or single-variable threshold will be subjected to the dual impact of "dimension disaster" and noise interference, resulting in rapid decline in detection accuracy and serious false positives and false negatives. SUMMARY

[0004] The main purpose of the application is to provide a big data anomaly detection method based on a time sequence multi-dimensional self-encoder, which fuses adaptive time-frequency decomposition, wavelet packet transformation, topological data analysis and local entropy measurement technology to accurately extract multi-scale features and topological structure information of multi-dimensional time sequence data. Through nonlinear dimension reduction and persistent homology analysis, the connectivity and void distribution of multi-dimensional data in manifold space are described, and local entropy and anomaly probability models are used to dynamically evaluate the system state. The application can efficiently capture short-time sudden anomalies, long-term trend changes and multi-dimensional coupling anomalies, and significantly improves the sensitivity, accuracy and robustness of anomaly detection in complex environments.

[0005] To solve the above problems, the technical scheme of the application is as follows:

[0006] The method comprises the following steps:

[0007] Step 1: decompose the multi-dimensional time series data, first extract the intrinsic oscillation mode of the multi-dimensional time series data by using Hilbert-Huang transform, then capture the local features in different scales and time periods by wavelet packet transform, adaptively extract multi-level spatio-temporal characteristics from the multi-dimensional time series data, and reconstruct the multi-level spatio-temporal characteristics as the decomposition result;

[0008] Step 2: map the decomposition result to a low-dimensional and continuous manifold space; on the obtained manifold space, the structural information of the multi-dimensional time series data is extracted by calculating topological invariants such as persistent homology;

[0009] Step 3: based on the extracted structural information, calculate the local entropy of the multi-dimensional time series data at a set scale; according to the local entropy, calculate the anomaly probability.

[0010] Further, in step 1, the multi-dimensional time series data is:

[0011]

[0012] Wherein, M represents the dimension of the multi-dimensional time series data; x m (t) is the mth dimension of the multi-dimensional time series data, m=1, 2,..., M; T is the observation time length or the number of discrete sampling points; t is the time; when the intrinsic oscillation mode of the multi-dimensional time series data is extracted by using Hilbert-Huang transform, the following processing is performed on each dimension x m (t): decompose x m (t) into a group of intrinsic mode functions and a residual term:

[0013]

[0014] Wherein, c m，k (t) is the kth intrinsic mode function of the mth dimension of the multi-dimensional time series data; K m is the total number of intrinsic mode functions of the mth dimension of the multi-dimensional time series data; r m (t) is the residual term of the mth dimension of the multi-dimensional time series data; for the kth intrinsic mode function, the initial candidate intrinsic mode function is defined as:

[0015]

[0016] l is an integer index; in the current candidate intrinsic mode function , find all local maxima and local minima, and obtain the upper envelope and the lower envelope by spline interpolation, respectively; the local average envelope is defined as:

[0017]

[0018] The candidate intrinsic mode function is updated as:

[0019]

[0020] The normalized error is calculated as:

[0021]

[0022] When SD < ∈, ∈ is a preset convergence threshold, it is considered that The intrinsic mode function condition is considered to be met, denoted as

[0023] Further, in step 1, for each intrinsic mode function c m，k (t), the Hilbert-Huang transform is calculated by the following formula:

[0024]

[0025] Where p.v. represents the Cauchy principal value integral; τ is the time integral variable; the corresponding analytical multi-dimensional time series data z m，k (t) is constructed by the following formula:

[0026]

[0027] Where j is the imaginary unit, satisfying j 2 =-1; is the instantaneous amplitude; is the instantaneous phase; is the instantaneous frequency.

[0028] Further, in step 1, for each intrinsic mode function c m，k (t) obtained, a wavelet packet transform is used to capture local features of different scales and time periods; let the mother wavelet be ψ(t), and construct the wavelet packet basis function corresponding to the scale v and the translation l:

[0029] ψ v，l (t) = 2 -v / 2 ψ(2 -v t-l);

[0030] Where v ∈ {v min , v min +1,..., v max} is the scale; v min and v max represent the minimum scale and the maximum scale respectively; the intrinsic mode function c m，k (t) is expanded under the basis ψ v，l (t), and the coefficients as local features.

[0031] Furthermore, in step 1, for the m-th dimension of multidimensional time series data, the reconstruction expression is:

[0032]

[0033] Among them, Λ v represents the set of all valid translation indices under scale v; α m，k (v, l) is the reconstruction coefficient, which is calculated taking into account the wavelet packet coefficient W m，k (v, l) and the instantaneous amplitude A extracted by Hilbert-Huang transform m，k (t) and frequency ω m，k (t), defined as:

[0034] α m，k (v, l) = 0.3·W m，k (v, l)+0.4·A m，k (t)+0.3·ω m，k (t);

[0035] is the decomposition result of the m-th dimension multidimensional time series data. The decomposition result of the multidimensional time series data is

[0036] Furthermore, in step 2, the decomposition result is mapped to a low-dimensional and continuous manifold space using the following formula:

[0037]

[0038] in, The obtained manifold space is a smooth manifold; is the Kronecker product operation; ∥·∥ HS is the Hilbert-Schmidt norm; τ is the time integral variable; R(τ) is the Riemann curvature tensor.

[0039] Furthermore, in step 2, on the obtained manifold space, the structural information of the multidimensional time series data is extracted by calculating topological invariants such as continuous homology using the following formula:

[0040]

[0041] Where VR(M(X),∈) represents the Vietoris–Rips complex constructed based on M(X) at scale ∈; H m (VR(M(X),∈)) is the m-th dimensional homology group under scale ∈; γ m，i is the i-th homology class in the m-th dimension; b m，i with d m，iγ m，i The birth scale and death scale of each barcode pair (b m，i d m，i ) is converted to numerical features; inf{·} is the lower bound operation; inf{∈:γ m,i vanish} means that for a given homology class γ m，i , in all scales ∈ during the filtering process, such that γ m，i The infimum of those ∈ that disappear; For a given homology class γ m，i , find the minimum value of the scale parameter ∈ in the filtering process so that γ m，i First appeared in the m-th dimensional homology group H m (VR(M(X),∈)); ψ m (M(X)) is the structural information of the m-th dimension of multidimensional time series data under scale ∈.

[0042] Furthermore, in step 3, the structural information of the multi-dimensional time series data is assumed to be:

[0043] Ψ(M(X),∈)=[Ψ1(M(X),∈),Ψ2(M(X),∈),...,Ψ M (M(X),∈)];

[0044] Exponential normalization is applied to each component to define the normalized probability distribution:

[0045]

[0046] m=1,2,…,M;c=1,2,…,M;σ(∈) is the normalization factor under scale ∈, which is the set value; using the normalized probability distribution p m (∈) Calculate the local entropy H(∈) at each scale ∈:

[0047]

[0048] Furthermore, in step 3, the anomaly probability is calculated based on the local entropy using the following formula:

[0049]

[0050] Among them, P anom is the abnormal probability.

[0051] The big data anomaly detection method based on time series multidimensional autoencoder of the present invention has the following beneficial effects:

[0052] The application provides a more fine and adaptive overall scheme for multi-dimensional big data anomaly detection by fusing adaptive time-frequency decomposition, wavelet packet subdivision and topological structure extraction and the like based on a multi-dimensional self-encoder, and achieves several obvious beneficial effects.

[0053] Firstly, the application combines adaptive decomposition and local feature capturing in the preprocessing stage of multi-dimensional time series data, so that signals from different sensors or data channels are decomposed into multiple modes capable of reflecting different frequency bands and different time scales, and wavelet packet analysis and instantaneous amplitude and instantaneous frequency extraction are performed on each mode. When facing non-stationary, nonlinear and strong noise data, this method can accurately capture short-term intense fluctuations or weak long-period interferences, laying a high-resolution feature foundation for subsequent anomaly detection. Compared with the traditional analysis scheme which only uses fixed basis functions or is limited to a single dimension, the application can more adaptively reveal potential fault points or abnormal areas, while greatly reducing the dependence on specific prior models.

[0054] Secondly, the application realizes nonlinear dimension reduction of high-dimensional feature vectors through a multi-dimensional self-encoder, projects complex multi-dimensional time series signals into a relatively low-dimensional and continuous space, and then uses topological structure extraction means to determine the overall shape and local connectivity in the space. Compared with simple linear dimension reduction methods or anomaly detection strategies based solely on distance measurement, the nonlinear dimension reduction used in the application can more fully retain the coupling relationship and nonlinear characteristics of the data itself, and by introducing the core idea of topological data analysis, it can dynamically track the connected components, cavities and ring structures of the data in the manifold space at different scales. When some dimensions or subgroups are significantly deformed or topologically broken after dimension reduction due to anomalies, the algorithm can identify them in the multi-scale filtering process, thereby greatly improving the sensitivity to complex anomalies and effectively reducing false positives and false negatives.

[0055] In addition, the application uses local entropy, an information theory index, to combine the uncertainty measure and probability mapping of multi-dimensional data in the topological space, which can to a certain extent take into account both short-term rapid impact and long-term slow drift of two different types of anomalies. Because local entropy can describe the disorder and dispersion of data in a multi-scale environment, when some channels or features appear out of the ordinary or the concentration abnormality increases in a certain time period, the local entropy will change significantly. By additionally designing a function to map the entropy to an easily interpretable abnormal probability interval, the application can not only provide intuitive quantitative results at the business end, but also avoid frequent false positives due to local noise or slight fluctuations. This linkage strategy of entropy value and probability has flexibility and stability, and can adjust the steepness or offset of the mapping function according to different scene requirements. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The method flowchart of the big data anomaly detection method based on the time sequence multi-dimensional self-encoder provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0057] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should belong to the protection scope of the present application.

[0058] Embodiment 1, reference Figure 1 The big data anomaly detection method based on the time sequence multi-dimensional self-encoder, the method comprises:

[0059] Step 1: decompose the multi-dimensional time sequence data, first extract the internal oscillation mode of the multi-dimensional time sequence data by using the Hilbert-Huang transform, then capture the local features in different scales and time periods by using the wavelet packet transform, adaptively extract the multi-level space-time characteristics from the multi-dimensional time sequence data, and reconstruct the multi-level space-time characteristics as the decomposition result;

[0060] In specific implementation, firstly, the inherent mode functions are gradually extracted from the original multi-dimensional time series data through a screening process, which can reflect the main change patterns of the signal in different energy collection intervals. At the same time, since this method does not depend on specific basis functions or preset signal assumptions, it has stronger adaptability to transient characteristics and nonlinear patterns. After the separation of the inherent mode functions, the system uses Hilbert transform to conduct overall review of each mode from the perspective of time axis and instantaneous frequency distribution. Through the linkage of instantaneous energy and instantaneous frequency, the frequency drift and amplitude change characteristics of each mode in different time segments can be more intuitively revealed. Since multi-dimensional time series data often come from multiple sensors or multiple source data channels, which contain both trend information generated over time and coupling and correlation between different dimensions, it is difficult to accurately locate potential subtle abnormalities or abnormal activities in a short period of time by only using the original signal or a single mode. Based on this, after obtaining each inherent mode function, the system also performs multi-level wavelet packet decomposition on the mode functions in different frequency bands and time periods. As a more refined frequency band division method, wavelet packet decomposition can flexibly perform binary decomposition on the signal within the full frequency band range, so that more detailed components can be obtained in both high and low frequency bands. In this way, each inherent mode after wavelet packet decomposition can further decompose more targeted sub-band information and form more rich local feature description on the time-frequency plane.

[0061] Since any high-dimensional time series usually contains different scale dynamic processes, for example, some long-period gentle changes may be related to the steady state or slow trend of the system, while short-period sharp fluctuations may indicate local bursts or abnormal situations, wavelet packet decomposition can help capture such feature differences. When the intrinsic oscillation mode extracted by Hilbert-Huang transform is combined with the advantage of wavelet packet in frequency domain subdivision, an adaptive and multi-scale analysis framework can be formed, which provides more comprehensive and deep feature representation for subsequent anomaly detection. At the same time, in the context of multi-dimensional data, the modal decomposition results of each channel or dimension can be respectively wavelet packet subdivided, and then combined with the correlation and synchronization analysis between multiple dimensions to obtain cross-dimensional spatiotemporal features, and finally these decomposition and reconstruction results are combined as the overall representation of multi-dimensional time series data at different time scales and frequency scales. This process helps to build more accurate low-dimensional mapping in subsequent autoencoder modeling and topology extraction, because the autoencoder must rely on effective feature engineering to alleviate the curse of dimensionality and highlight potential abnormal pattern signals when facing high-dimensional time series information, and the multi-level spatiotemporal characteristics obtained through the above decomposition steps can provide high-resolution and high-discrimination input for subsequent dimensionality reduction and topology analysis. In other words, this decomposition and reconstruction idea is not just a simple signal preprocessing method, but by combining Hilbert-Huang transform and wavelet packet transform, it condenses multiple modal features of multi-dimensional time series data in time and frequency distribution, and realizes the dual description of micro-disturbance and overall trend on the cross-scale level.

[0062] Step 2: Map the decomposition results to a low-dimensional and continuous manifold space; on the obtained manifold space, extract the structural information of multi-dimensional time series data by calculating topological invariants such as persistent homology;

[0063] In specific implementations, since step 1 has distilled a reconstructed signal with rich spatio-temporal features for the multi-dimensional time-series data, in the face of high-dimensional and large-scale data, direct clustering or anomaly analysis using original features often encounters dimension disaster or significant visualization difficulties. Through the dimension reduction capability of the autoencoder, high-dimensional features can be first mapped to a lower-dimensional latent space that still retains the main structure and local geometric relationship. The latent space can be regarded as a manifold as continuous as possible, so that the originally complex data distribution can exhibit a more analyzable form after dimension reduction. At this time, in order to further reveal the inherent topological structure of the data in this manifold space, topological data analysis tools such as persistent homology are needed to hierarchically depict the connectivity, hollow features and higher-dimensional topological hole distribution of data samples at different scales. The calculation process of persistent homology can be understood as dynamically tracking the connection relationship between data points when gradually increasing or decreasing the neighborhood scale, observing how connected components are born and die, and recording the appearance and disappearance of ring structures or cavities at different stages, thereby forming a barcode or persistence diagram reflecting the topological evolution process. For time-series multi-dimensional autoencoder-based large data anomaly detection, these topological invariants can largely depict the global shape and local sparsity of the data, helping to identify abnormal distributions that are very different from normal patterns. Compared with traditional anomaly detection methods that simply rely on distance measurement or density estimation, topological data analysis can break away from strict dependence on Euclidean metric and more from global structure to capture potential nonlinear distribution characteristics, thereby making up for the defects of ordinary distance measurement that is easily disturbed by noise or scale changes in high-dimensional complex scenarios. When some channels in the multi-dimensional time-series data exhibit out-of-range behavior or dramatic fluctuations in frequency distribution at a special time period, these outliers or abnormal clusters are likely to differ from the original normal pattern in topological structure in the manifold space, such as forming new independent connected components, or causing prominent cavities or branch deformations in the originally smooth low-dimensional manifold. Through the calculation and comparison of these persistent homology invariants, the system can accurately capture the deviation of anomalies in the topological level without relying on explicit probability distribution assumptions. At the same time, with the nonlinear dimension reduction strategy realized by the autoencoder, the construction of the manifold can also be more consistent with the complex coupling relationship of the multi-dimensional time-series data itself, avoiding feature distortion or information loss caused by simple linear dimension reduction. Since the data is more easily visualized after dimension reduction, researchers or system managers can also intuitively evaluate the position of the abnormal point, the topological properties of the surrounding neighborhood, and the evolution trajectory of the anomaly at different scales based on the performance of the persistence diagram or barcode diagram.

[0064] Especially in the context of big data, this combination of autoencoder and topological data analysis can mine global geometric features that are difficult to detect by distance-based methods while maintaining high parallelism and larger scale. In summary, the work done in step 2 is to construct a suitable topological perspective using the reduced manifold space, so that the reconstructed spatiotemporal features are no longer just a series of abstract vectors or matrices, but retain interpretable shapes, holes, and connectivity distribution on the manifold. At the same time, the persistent homology multi-scale analysis attribute can cover a series of changes from micro-local to macro-global, and the persistent features extracted from it are often robust to noise and can give significant difference to the spatial distribution of anomalies, providing a solid structural foundation for the subsequent probability evaluation combined with local entropy in step 3. In actual deployment, in order to obtain the most accurate dimensionality reduction and topological description effect, reasonable parameter selection is needed according to the data size, the characteristics of modal decomposition and the network structure of the autoencoder, and combined with iterative training and cross-validation to ensure that the reconstruction error after manifold embedding is within an acceptable range. When the system monitors new time series in the data stream, it can quickly project it into the low-dimensional manifold that has been learned, and observe its topological relationship with the main group in the persistent homology graph, which makes online detection or semi-real-time alarm possible, effectively improving the identification efficiency of potential risks for industrial control systems or Internet of Things platforms.

[0065] Step 3: Based on the extracted structural information, calculate the local entropy of multi-dimensional time series data at a given scale; according to the local entropy, calculate the anomaly probability.

[0066] Since the high-dimensional, multi-modal time series data has been mapped to a low-dimensional manifold space in step 2, and a number of topological invariants representing the global and local structural differences of the data have been obtained by means of persistent homology analysis and other methods, in this step, the system needs to further measure the complexity or disorder degree of the data in the local neighborhood in combination with the entropy measurement idea, according to the distribution of these topological features in different regions. When the data distribution in a sub-region presents a relatively uniform and stable structure, it usually means that it maintains good coherence with the main group or normal mode on the manifold, and the corresponding local entropy value is also usually smaller. On the contrary, if some data points or sub-regions present signs of outlying, dispersion, extreme sparsity or even complete disconnection with the surrounding regions, the neighborhood of these points is likely to deviate significantly from the normal interval in the topological structure, and will present a higher disorder degree or complexity, and the corresponding local entropy value may also surge. By comparing these local entropies with the pre-set discrimination threshold or statistical model, the abnormal degree of a certain time series at the current scale can be quantitatively evaluated. Further, since multi-dimensional time series data usually has multi-time scale characteristics, the calculation of local entropy can also be performed for different time windows or frequency bands, so that both long-term stable trends and short-term sudden changes can be taken into account.

[0067] When the time window is large, the system focuses on the overall structural stability, and if some channels or modes continuously deviate from the normal state in a long time scale, their corresponding local entropy values will be repeatedly high in multiple iterations, indicating possible signs of slowly evolving abnormalities. When the time window is small, more attention is paid to the response to transient or subtle disturbances. In industrial control or Internet of Things monitoring scenarios, such short-time high-frequency abnormal changes often indicate potential faults or emergency situations, which need to be detected and warned in time. By fusing the calculation results of local entropy with topological structure analysis and autoencoder dimensionality reduction information, the system can more accurately identify the local regions that are isolated or abnormally aggregated in the manifold space. In order to improve robustness, sliding window, exponential weighting and other methods will be combined in actual deployment to dynamically update the local entropy index over time, which can both track the evolution of the overall distribution in the process of slow data drift and capture sudden jumps. When some channels or modes frequently appear in the form of entropy value fluctuation in a short period or continuously maintain a high entropy state in a long period, the system will mark these situations as high risk, and then judge whether to perform further diagnostic analysis in combination with other conditions.

[0068] It is worth noting that, unlike traditional global entropy or simple probability distribution-based anomaly detection methods, the local entropy calculation adopted by the present application is based on the neighborhood structure on the manifold, and also echoes the previous persistent homology information, which means that the discrimination criterion does not only rely on whether the density or global distribution conforms to a certain hypothesis, but starts from the topological and spatiotemporal characteristics of the data itself, giving the algorithm higher sensitivity to nonlinear and non-stationary characteristics. In many application scenarios involving multi-sensor fusion or multi-source heterogeneous data, this topological and local entropy-based anomaly detection method is more likely to discover subtle distribution changes that are difficult to capture by conventional mathematical statistics methods, thereby effectively reducing the risk of false negatives and improving overall detection accuracy. It is especially worth emphasizing that the autoencoder provides a dimensionality reduction channel with nonlinear representation capability in step 2, enabling the manifold space to better fit the multi-dimensional coupling relationship of the actual data, laying a solid representation foundation for local entropy calculation. Without the nonlinear mapping of the autoencoder, simple linear dimensionality reduction often distorts or superimposes potential abnormal patterns in the mapping process, making it difficult to clearly express the differences in subsequent entropy measurement. At the same time, in a big data environment, this method can also take advantage of parallel computing and distributed storage means to complete anomaly detection for all nodes or time segments in the global range through batch evaluation of local entropy values, even if the data is large in size and numerous in dimensions, it will not become infeasible due to high computational overhead.

[0069] In step 1 of embodiment 2, the multi-dimensional time series data is:

[0070]

[0071] wherein M represents the number of dimensions of the multi-dimensional time series data; x m (t) is the mth dimension of the multi-dimensional time series data, m = 1, 2,... M; T is the observation time length or the number of discrete sampling points; t is time; when the intrinsic oscillation mode of the multi-dimensional time series data is extracted by using the Hilbert-Huang transform, the following processing is performed on each dimension x m (t): x m (t) is decomposed into a group of intrinsic mode functions and a residual term:

[0072]

[0073] wherein c m,k (t) is the kth intrinsic mode function of the mth dimension of the multi-dimensional time series data; K m is the total number of intrinsic mode functions of the mth dimension of the multi-dimensional time series data; r m (t) is the residual term of the mth dimension of the multi-dimensional time series data; for the kth intrinsic mode function, the initial candidate intrinsic mode function is defined as:

[0074]

[0075] l is an integer index; in the current candidate intrinsic modal function , find all local maxima and minima, respectively, by spline interpolation to get the upper envelope and the lower envelope Define the local average envelope as:

[0076]

[0077] Update the candidate intrinsic modal function as:

[0078]

[0079] Calculate the normalized error as:

[0080]

[0081] When SD < ∈, ∈ is a preset convergence threshold, it is considered that the intrinsic modal function condition has been met, denoted as

[0082] Specifically, the multi-dimensional time series data is denoted as X(t), where each dimension corresponds to an x m (t), each dimension can be iteratively screened, and x m (t) is decomposed into the sum of several intrinsic modal functions, and a smooth or low-frequency residual is left at the end. The theoretical support behind this approach is that in many real scenarios, signals often mix multiple levels of fluctuations and trends, and if only fixed basis functions or linear assumptions are used to separate high and low frequencies, some key burst patterns or non-stationary components will be missed. Therefore, the present application uses the decomposition formula to better capture subtle but potentially abnormal short-term changes in complex systems through the adaptive characteristics of Hilbert-Huang transform. When writing , the left side of the equation represents the original time series signal of the mth dimension, and the right side is the sum of the extracted intrinsic modal functions, and the residual term that has not been explained by any modal function. The reason why such a decomposition form is suitable for multi-dimensional time series scenarios is that it can follow the same idea to screen different dimensions, and then compare and combine all modal functions in the same multi-dimensional framework in subsequent analysis.

[0083] In the specific screening steps, first, the sum of the previously extracted modal functions is removed from the original signal to obtain an initial candidate function Then construct the upper and lower envelopes by local extremum search and spline interpolation, and calculate their average to obtain the local average trend of the candidate function in the current iteration, and then use The envelope subtraction is performed in this way. The reason for performing such envelope subtraction is to ensure that the final intrinsic modal function satisfies the characteristic of oscillation center symmetry, avoiding obvious residual trend components or asymmetric distribution. At the same time, in order to measure the amplitude of function change before and after each iteration, the algorithm uses the normalized error to judge whether it converges, as long as its value is lower than a certain threshold ∈, it means that the current candidate function meets the definition requirements of the intrinsic mode, and there is no need to continue to screen. This is repeated until no new oscillation mode can be separated, and a complete set of modal functions and a relatively smooth residual are obtained. Because in the real big data scene, the time series signal may show rapid oscillation, slow rise, sudden pulse and other complex patterns, if they are not separated first, it will often cause a lot of interference in the subsequent frequency domain analysis and anomaly detection process. Through this decomposition method, the invention can extract multiple subdivided fluctuation components for each dimension, which not only provides a high-resolution basis for subsequent wavelet packet-based multi-scale analysis, but also lays a reference for the coupling relationship between multi-dimensions in different time segments. For example, some modal functions may have sharp peak and valley fluctuations in the high-frequency interval, corresponding to a short-term abnormal event, while other modal functions show more stable long-period characteristics, implying a slow evolution process that may contain systemic risk; When comparing across dimensions, if some dimensions appear synchronous pulse or coupled fluctuation in the same modal sequence, it may mean that there is a linkage effect between multiple subsystems, which has great diagnostic value for large-scale industrial monitoring or Internet of Things platforms. Further, the invention uses such modal decomposition results as input to the autoencoder, so as to focus on the differences between modalities when constructing a low-dimensional manifold space and capture the global structure through persistent homology, and finally make the anomaly detection of local entropy calculation more targeted and robust. In short, by splitting the original time series data into several independent oscillation components and a smooth residual in each dimension, these formulas not only demonstrate the adaptability of Hilbert-Huang transform to non-stationary signals, but also provide sufficient detailed underlying feature support for industrial control, fault warning and other practical applications. Without prior basis function constraints, these modal functions adaptively map the multiple change patterns of signals in time and frequency domains, and perfectly connect with subsequent multi-dimensional feature aggregation and autoencoder dimension reduction. At the same time, the existence of the residual term makes it possible to capture the long-term trend or fixed offset that may be ignored, thereby achieving a more comprehensive description of the overall system operation health.

[0084] In step 1 of embodiment 3, for each intrinsic modal function c m，k (t) is calculated by the following formula:

[0085]

[0086] where p.v. denotes the Cauchy principal value integral; τ is the time integral variable; the corresponding analytic multidimensional time series data z m，k (t) is constructed by the following equation

[0087]

[0088] where j is the imaginary unit satisfying j 2 = -1; instantaneous amplitude; is the instantaneous phase; is the instantaneous frequency.

[0089] Specifically, for any one intrinsic mode function c m，k (t) obtained from empirical mode decomposition, its corresponding Hilbert transform is first constructed by some integral operation, denoted as H{c m，k}(t), where the concept of Cauchy principal value (i.e., p.v.) is used to deal with the singularity of the integral kernel at t-τ = 0, so as to avoid the mathematical non-convergence or not meeting the condition of absolute integrability in the usual sense. Because the multidimensional time series data often presents complex characteristics such as non-stationary, strong noise, nonlinearity, etc. in real industrial or Internet of Things scenarios, the Hilbert transform can make the approximation of the analytic signal for each mode function, so that there is a relatively independent but correlated corresponding relationship between the real part and the imaginary part of the mode at the same time. Taking the original mode function as the real part and the result of the Hilbert transform as the imaginary part, and then using addition to obtain a complex number of the form c m，k (t) + jH{c m，k}(t), it can be rewritten into the exponential form of amplitude and phase by means of Euler's formula, so as to obtain the instantaneous amplitude A m，k (t) and the instantaneous phase θ m，k (t). Among them, the amplitude is obtained by squaring the real part and the imaginary part and then taking the square root, and the phase comes from the arctangent function, which represents the local oscillation position of the mode function at each time point. When the instantaneous phase is differentiated with respect to time, the instantaneous frequency ω m，k (t) is obtained, which means that at each time, the oscillation speed of the intrinsic mode function can be measured by a more continuous frequency index. Since each dimension of the multidimensional time series data needs to perform similar transformation, the instantaneous amplitude and the instantaneous frequency of different dimensions can be compared in parallel at the same time segment, so as to judge whether there is a synchronous peak fluctuation or frequency drift phenomenon.

[0090] If several dimensions show significant coupling in instantaneous amplitude or instantaneous frequency, it often implies that they have some correlation in the underlying physical or system mechanism, which is an extremely important clue in big data anomaly detection, because some fault signs often come from the abnormal state jointly presented by multiple subsystems or sensors. At the same time, by analyzing the signal form, it can also be more intuitive to track whether certain modes have a dramatic phase jump or amplitude mutation at a special time period, which often corresponds to the possible abnormal burst point. If combined with subsequent wavelet packet multi-scale analysis and autoencoder dimension reduction, these instantaneous amplitude and frequency information can be mapped to the manifold space together, and the topological structure and local entropy can be used for further anomaly judgment. Compared with global Fourier transform or fixed window length short-time Fourier analysis, the instantaneous parameters of Hilbert transform can more fully reflect the dynamic evolution of the signal on the time axis, especially suitable for non-stationary or multi-component overlapping data. In industrial field, some faults may only have a sharp frequency drift in a very short time window, and traditional algorithms based on global average energy or single frequency band statistics often ignore these details. In this invention, by analyzing the instantaneous amplitude and instantaneous frequency of each intrinsic mode function, the internal law of these short-time mutations can be presented to the greatest extent. Moreover, the slight fluctuation of instantaneous frequency of a single dimension may not be enough to explain the anomaly in a multi-dimensional scenario, but if some modes show simultaneous rise or fall in instantaneous frequency in multiple dimensions, they can be identified as high-risk signals by the subsequent autoencoder. In practical application, this also helps the automatic monitoring system to realize the alarm integration of multi-source data, which not only reduces the probability of single-point false alarm, but also takes into account the multi-dimensional dependence structure in complex environment.

[0091] In step 1, the obtained each intrinsic mode function c m，k (t) is analyzed by wavelet packet transform to capture local features at different scales and time periods; let the mother wavelet be ψ(t), and construct the wavelet packet basis function corresponding to scale v and translation l:

[0092] ψ v，l (t) = 2 -v / 2 ψ(2 -v t-l);

[0093] where v∈{v min , v min +1,..., v max} is the scale; v min and v max represent the minimum scale and the maximum scale respectively; the intrinsic mode function c m，k (t) is expanded under the basis ψ v，l (t) to calculate the coefficient as the local feature.

[0094] Specifically, first, a mother wavelet function ψ(t) is selected, and a set of basis functions ψ v，l (t) is constructed by scaling and translating the mother wavelet function, where v and l represent the stretching and moving of the wavelet packet in the time and frequency dimensions, respectively. Because the actual data scale is often extremely large and contains many non-stationary and nonlinear characteristics, it is often difficult to obtain the same degree of decomposition in all sub-bands by relying on a single layer or simple discrete wavelet transform. Wavelet packets can continue to be split into two in each frequency band, not limited to refining only the low-frequency part, and thus obtain uniform subdivision capability for the full frequency range on the multi-scale level, which is very beneficial for capturing sudden events and weak periodic oscillations. Specifically, when a certain intrinsic mode function c m，k (t) is projected onto the function family formed by ψ v，l (t), a number of coefficients W m，k (v, l) can be obtained, where the size of the coefficient reflects the energy distribution of the mode function in a given scale and time period, and also reveals the transient characteristics that may be hidden in a specific local window. For example, if there is a coefficient peak value much higher than the adjacent interval in a higher frequency band (corresponding to a small scale parameter v), it indicates that the mode has significant high-frequency disturbance in this time period, and thus may be identified as a potential risk point in the subsequent anomaly detection process. If there are multiple energy concentrated segments in a lower frequency band (corresponding to a large v), it may mean a long-term oscillation or slow decay phenomenon. Similarly, the time shift effect of the translation parameter l enables the algorithm to more flexibly scan the time axis of the entire signal, which is particularly important in non-stationary signals, because many abnormalities do not occur at fixed times or fixed frequency bands, but appear in different intervals as the system state or external environment changes.

[0095] By observing the wavelet packet coefficients, the c m，kThe dynamic structure of (t) provides highly distinguishable feature vectors for subsequent dimension reduction combined with the autoencoder. Compared with traditional wavelet decomposition, the biggest feature of wavelet packet is the high resolution and balanced coverage brought by full-band binary decomposition, which is crucial for the multi-dimensional big data scenario faced by the present application. Because when the modal function coefficients of multiple dimensions are combined into the same analysis framework, the same or similar scale parameter v can be used to compare the local energy distribution of different dimensions, so as to judge whether there is coupling fluctuation or synchronous anomaly in similar frequency bandwidth of multi-source signals; if the abnormal surge of coefficients occurs in certain dimensions in a certain sub-band, it is most likely caused by systematic failure or sudden events, and subsequent focus monitoring or high-risk marking in the manifold space can be performed. At the same time, this multi-scale feature can also improve the training performance of the subsequent autoencoder: since the network can gradually capture the nonlinear combination relationship between the coefficients of each sub-band during the learning process, it is easier to show the unique topological structure of the abnormal pattern in the low-dimensional latent representation after encoding. It is worth emphasizing that the wavelet packet coefficients can not only distinguish different signal intervals in scalar size, but also complement the features such as instantaneous amplitude and instantaneous frequency of the previously analyzed signals, forming a multi-angle and multi-scale feature fusion. In this way, even in an extreme noise environment, the algorithm can rely on the high-dimensional vector aggregated by different features to maintain strong distinguishability, thereby minimizing the sensitivity to single noise or local sampling error.

[0096] In step 1, for the mth multi-dimensional time series data, the reconstruction expression is:

[0097]

[0098] wherein Λ v represents the set of all effective translation indexes at scale v; α m,k (v, l) is the reconstruction coefficient, which takes into account the wavelet packet coefficient W m,k (v, l) and the instantaneous amplitude A m,k (t) and frequency ω m,k (t) extracted by Hilbert-Huang transformation, which are defined as:

[0099] α m，k (v, l) = 0.3·W m,k (v, l) + 0.4·A m，k (t) + 0.3·ω m，k (t);

[0100] is the decomposition result of the mth multi-dimensional time series data, and the decomposition result of the multi-dimensional time series data is:

[0101]

[0102] Specifically, although the traditional wavelet packet transform can effectively capture the local energy distribution of each scale and time period, if only relying on the simple wavelet packet coefficient, it may not be able to fully characterize the transient mode or short-time mutation that occurs in a non-stationary and nonlinear environment; and the instantaneous amplitude and instantaneous frequency obtained in the Hilbert-Huang transform stage can more flexibly describe the specific rate and amplitude of the dynamic change of different modal functions with time. Based on this, the present application proposes to let the reconstruction coefficient simultaneously refer to the three elements of wavelet packet coefficient, instantaneous amplitude and instantaneous frequency, and assign different weight coefficients to them. In this way, each projection quantity under the wavelet packet basis function not only reflects the local energy information, but also has a response to the instantaneous behavior characteristics, and can more fully represent the multi-layer characteristics of the original multi-dimensional sequence in the short-time high-frequency or long-time low-frequency interval during signal reconstruction, thereby providing more accurate input for subsequent dimension reduction and anomaly discrimination. Specifically, the coefficient obtained by each intrinsic modal function under the wavelet packet basis function ψ v，l (t) can be considered as a coarse-grained local energy measure first, and these coefficients are combined with the instantaneous amplitude and frequency obtained in the Hilbert-Huang transform to finally gather into the reconstruction coefficient α m，k (v, l), and this reconstruction coefficient is the key to determining the contribution degree of the modal function to the overall signal under a certain scale v and translation l. When the amplitude of a certain modal function becomes very large at the moment, or its frequency suddenly rises, it means that there may be a more violent fluctuation or abnormal precursor in the signal at this time period, and then the reconstruction coefficient at the corresponding position will be amplified through the weighting method, highlighting the importance of these potential abnormal fragments in the entire reconstruction process; on the contrary, if some modes maintain a small fluctuation amplitude at a certain scale, then their reconstruction coefficients will be relatively small, thereby giving a lower weight in reconstruction.

[0103] After all the coefficients under the scale and translation index are assigned, the sub-band components are superimposed relying on the wavelet packet basis function, which can balance between different levels of time-frequency analysis results, not only retaining weak but possibly important transient information, but also not over-amplifying accidental peaks caused by simple noise. At the same time, the residual term r m(t) plays the role of preserving the long-term or global trends that can no longer be accurately represented by intrinsic mode functions or wavelet packet basis functions. This is especially critical in scenarios where certain systemic faults evolve slowly. Many times, faults are not only manifested in local instantaneous fluctuations, but may also accumulate slowly over a long period of time. If this trend component is not retained, the system may ignore some low-frequency but continuous danger signals. The present invention retains the residual at the time of final reconstruction to ensure that the subsequent autoencoder input contains not only diverse oscillations, but also possible overall translation or slow fluctuations, meeting the need to give equal weight to global and local features of non-stationary data. In multidimensional scenarios, each dimension can be obtained through this reconstruction formula. Then put all Merge together to form reconstruction data for the entire system Because these data include wavelet packet energy distribution, Hilbert instantaneous parameters, and residual trends, subsequent autoencoders, when faced with such rich input, can more accurately capture complex patterns such as coupling effects, phase synchronization, and frequency resonance between multidimensional data, thereby retaining key information that is most sensitive to anomalies during dimensionality reduction.

[0104] Example 6: In step 2, the decomposition result is mapped to a low-dimensional and continuous manifold space using the following formula:

[0105]

[0106] in, The obtained manifold space is a smooth manifold; is the Kronecker product operation; ||·|| HS is the Hilbert-Schmidt norm; τ is the time integral variable; R(τ) is the Riemann curvature tensor.

[0107] Specifically, in order to compactly embed these high-dimensional features into a low-dimensional manifold, it is necessary to use the idea of ​​differential geometry to To modify the space in which the feature is located, it is necessary to define a mapping that is linked to a certain curvature structure so that the features collected at different time points can reflect local similarity while maintaining a certain geometric coherence overall. In this way, the exponential function in the formula plays an important role in stretching or compressing. By Kronecker product (i.e. The result of multiplying the result of the operation) with its own transpose is converted into a matrix that can reflect the coupling relationship between the components, and then the Hilbert-Schmidt norm is used The purpose of standardizing it is to maintain a relatively balanced scale between time segments of different magnitudes. Doing so can prevent certain component values ​​from being too large and monopolizing the overall geometric structure in a big data environment, and can also ensure that those weak but important abnormal signals will not be completely submerged during mapping. Next, through exponential mapping, the matrix is ​​mapped to a structure related to the Lie group, and then it can be multiplied with the Riemann curvature tensor R(τ) to integrate the result that was originally just the characteristic inner product or outer product into a differentiable manifold measurement framework. In other words, τ is both a variable of the time series integral and a link between the entire topological structure and the manifold geometry, because at each moment Each can be considered a tiny fragment occupying this manifold, and R(τ) constructs these fragments by applying differentiated curvatures to different time segments, piecing them together into a continuous, internally coherent geometric entity. When integrating time from 0 to T, it is equivalent to scanning all features within the entire observation window, accumulating them through exponential mapping and curvature modulation, and shaping a low-dimensional space that can reflect the relationship between time evolution and multidimensional coupling. This space is referred to as M(X) in the formula. Once M(X) forms a smooth manifold, it means that geodesic distances between different points can be calculated on it, the connectivity of local neighborhoods can be tracked, and topological data analysis methods such as persistent homology can be used to identify the presence of anomalous clusters, void structures, or other topological features that differ significantly from conventional distributions.

[0108] More importantly, this construction method allows the ordered geometric integration of high-dimensional features in a big data environment. Compared with traditional dimension reduction methods based on Euclidean metrics, it can more sensitively capture key factors such as nonlinear distribution, instantaneous coupling, and spatiotemporal multiscale interaction, thereby bringing higher precision and reliability to the anomaly detection process of the application. Especially in complex industrial control or Internet of Things monitoring scenarios, different sensor data often exhibit complex coupling relationships at the same time. If only simple linear mapping is used for dimension reduction, many subtle but potentially important patterns will be ignored. The present method, by virtue of the Kronecker product, expands the interaction description of each component, and then uses the Hilbert-Schmidt norm for regularization to prevent numerical explosion or principal component effects from causing deformities. Finally, the Riemann curvature tensor is coupled into the mapping process, so that these feature components form a geometric structure that is sufficiently smooth, slightly bendable, and can accommodate abnormal mutations in the time dimension. Because this mapping step provides a coherent and sufficiently differentiated manifold for subsequent persistent homology, it can quickly identify possible anomalies in local neighborhoods or changes in multiscale holes. If the reconstructed data in a certain period or certain dimensions appear discontinuous branches, isolated connected components, or significant deformations on the manifold, it often means that the system is experiencing or about to experience an anomaly. At the same time, in the context of big data, the integral form in the formula also facilitates batch parallel computing, because local summation can be performed on multiple time periods, and then combined into a whole manifold. As long as the uniform definition of and HS is maintained, and R(τ) is effectively stored, the manifold mapping of massive features can be completed in a distributed manner.

[0109] Example 7: In step 2, structural information of multi-dimensional time series data is extracted by calculating topological invariants such as persistent homology on the obtained manifold space through the following formula:

[0110]

[0111] where VR(M(X), ∈) represents the Vietoris-Rips complex constructed based on at scale ∈; H m m(VR(M(X), ∈)) is the mth homology group at scale ∈; γ m，i is the ith homology class in the mth dimension; b m，u and d m，i are the birth scale and death scale of γ m，i , respectively; each barcode pair (b m，i , d m，i ) is converted into a numerical feature; inf{·} is the lower limit operation; inf{∈: γ m，i disappears} indicates that for a given homology class γm，i , in the filtering process, all scales ∈, such that γ m，i , the lower bound of those ∈ that disappear; denotes for a given homology class γ m，i , find the minimum value of the scale parameter ∈ in the filtering process, such that γ m，i first appears in the m-th dimensional homology group H m (VR(M(X), ∈)); is the structural information of the m-th dimensional multi-dimensional time series data under the scale ∈.

[0112] Specifically, after obtaining a data set M(X) embedded in a manifold through previous steps, the corresponding Vietoris-Rips complex VR(M(X), ∈) can be constructed under different scale parameters ∈. The so-called Vietoris-Rips complex refers to connecting an edge between any two points in the complex as long as the distance between them does not exceed ∈ under a given distance metric; for groups of three points, four points, or even more points, simplexes (such as triangles, tetrahedrons, etc.) are added layer by layer according to whether the distance between any two points within the group is within the range of ∈. By gradually increasing ∈ from a minimum value, the complex will undergo a filtering process from discrete to continuously increasing connected components, ring structures, and even higher-dimensional voids; in topological data analysis (TDA), each increase in scale corresponds to a "filtration", and the system will track the homology groups H m how to generate new homology classes, and how to disappear at a larger scale. Thus, the topological features appearing at a certain dimension m can be recorded as the corresponding homology class γ m，i , and their first appearance scale and final disappearance scale in the filtering process are called birth scale b m，i and death scale d m，i , respectively.

[0113] In the formula used in the present application, inf{∈: γ m，i ∈ H m (VR(M(X), ∈))} represents the minimum ∈ at which the homology class first appears, and inf{∈: γ m，i disappear} corresponds to the minimum ∈ at which it disappears in the filtering process. By analyzing these scale pairs (b m，i , d m，iThe statistics or visualization of the bar code expression, which marks the birth and death interval on the horizontal axis, and each line segment corresponds to the life cycle of a topological feature, can be obtained. Those features with short life are usually regarded as noise or local disturbance, while the long-life coherent class often means that there are significant connected components, loops or hollow structures in a certain scale range of the data. It is in this distinction that the system can identify the global and local geometric distribution differences of the multi-dimensional time series data, and then be linked to the manifold structure after dimension reduction based on the time series multi-dimensional self-encoder, from which the abnormal patterns are found.

[0114] For example, for multi-sensor time series in industrial processes, when ∈ is small, if the distribution of data points on the manifold is relatively sparse, a large number of zero-dimensional homology classes, that is, many independent connected components, will often appear; as ∈ increases, these components will be connected to each other, and the zero-dimensional homology classes will gradually merge. If some components cannot be connected to the main component for a long time, or even remain isolated until a relatively large ∈, it is likely to correspond to some outlier behavior or abnormal cluster. In one-dimensional homology, if a stable ring structure is formed as the filtering process evolves, it means that the data has a more significant cyclic or periodic distribution in this dimension; if in a certain period or sub-group, the one-dimensional homology class appears and maintains a long life cycle, it generally indicates some significant internal feedback mechanism or abnormal cyclic pattern of the system. For higher-dimensional homology (such as two-dimensional, three-dimensional), it may be related to the holes, cavities or more complex structures of the data in the manifold space. In the big data scenario, this method of constructing Vietoris-Rips complex and filtering at multiple scales can well adapt to the distribution of high-dimensional and large-scale samples: even if the data volume is extremely large, as long as the point-to-point distance can be calculated efficiently in a distributed architecture, the homology information can be continuously updated in an incremental or parallel manner, without being disturbed by too much noise or scale parameter selection as in traditional outlier analysis based on Euclidean distance or probability model. At the same time, persistent homology provides a systematic description of the multi-resolution structure, so that fine patterns that may be ignored at a small ∈ can gradually appear as ∈ increases; if these fine patterns are detected to exist at a relatively large scale, they can be judged to have sufficient importance, and thus marked as potential outliers.

[0115] In order to numerically express such topological results, the present application also converts the bar code interval (b m，i , d m，i ) into several feature indicators that can be used for machine learning or statistical analysis, for example, the length d m，i -b m，i, the collection of barcodes can be mapped into topological signatures, so that the topological structure difference can be integrated into the next step of local entropy calculation or classification discrimination. If the barcodes of some homology classes are abnormally long, or a batch of features appear in the same time window, it can prompt the system that there is an unusual behavior in this period or this component combination, and it is necessary to further check. In addition, these topological invariants can also be combined with the instantaneous amplitude, instantaneous frequency or wavelet packet energy distribution generated by decomposition and reconstruction to form multi-angle abnormal measurement, so as to ensure that the abnormal identification method is neither limited to single-dimensional local change nor only focuses on global average distribution, but can perceive the geometric deformation of the whole signal in different scales and dimensions. It is worth mentioning that this topological analysis driven by persistent homology complements the deep dimension reduction of the autoencoder: the autoencoder provides a way to map high-dimensional data to a manifold space that is compressible but retains as much structural information as possible, while the multi-scale filtering process of persistent homology further reveals how the manifold is connected or decomposed at different radii, so that the abnormal signals hidden behind the multi-dimensional coupling or nonlinear relationship are amplified and highlighted. For industrial control systems, Internet of Things monitoring platforms or various complex big data applications, this method can be deployed in practice by first segmenting the time series data based on block or sliding window to obtain the corresponding low-dimensional representation through the autoencoder, and then constructing the Vietoris-Rips complex and calculating the homology group to form the barcode on these low-dimensional representations. After a batch of barcodes are output for each time period, the pattern difference of these barcodes in different time periods is compared, or the homology information of adjacent intervals is combined to identify whether there is a persistent anomaly. If some abnormal topological features persist significantly in the time-frequency domain, it can be determined by the system as a high-risk event and automatically trigger an alarm or start a fault-tolerant strategy, so as to improve the coverage of short-term faults or slow-evolving problems. As can be seen, the formula given in embodiment 7 describes a complete multi-dimensional topological extraction process: first, through the filtering process of Vietoris-Rips complex at different ∈, the birth and death of homology classes γ m in the H m，i group are tracked, and the birth inf{∈: γ m，i ∈H m (·)} and death inf{∈: γ m，i disappear} are recorded, and then they are mapped into barcode pairs or other numerical signatures of the form (b m，i , d m，i ), and these information is regarded as Ψ m m，i , d m，i ), and these information is regarded as Ψ mThe multi-dimensional time series structure features encapsulated by (M(X), ∈) can be said to extend the point-to-point or local neighborhood measurement in traditional signal processing to the global geometric level, allowing the algorithm to perceive potential abnormalities from the "shape" rather than being limited to linear statistics such as mean and variance. When used in conjunction with the dimensionality reduction results based on the time series multi-dimensional self-encoder, it can accurately distinguish whether abnormal topological deformations such as independent components, loops or voids have occurred on the manifold, thereby improving the abnormal detection idea of the present application in the nonlinear scenario of big data and ensuring higher sensitivity.

[0116] In step 3 of embodiment 8, the structure information of the multi-dimensional time series data is:

[0117] Ψ(M(X), ∈) = [Ψ1(M(X), ∈), Ψ2(M(X), ∈),..., Ψ M (M(X), ∈)];

[0118] Exponential normalization is used for each component to define the normalized probability distribution:

[0119]

[0120] m = 1, 2,..., M; c = 1, 2,..., M; σ(∈) is the normalization factor at scale ∈, and is a set value; the normalized probability distribution p m (∈) is used to calculate the local entropy H(∈) at each scale ∈:

[0121]

[0122] Specifically, in embodiment 8 of the big data anomaly detection method based on the time series multi-dimensional self-encoder, by performing exponential normalization on the topological structure information on the manifold space and calculating the local entropy, the complex distribution and abnormal risk of the multi-dimensional time series data at different scales can be described more subtly. When analyzing the structure information of the multi-dimensional time series data, the connectivity, void characteristics and barcode performance obtained in persistent homology analysis of different dimensions on the manifold need to be considered comprehensively, so it is summarized as a vector form Ψ(M(X), ∈), where each component Ψ m (M(X), ∈) represents the numerical description of a dimension or a topological feature at scale ∈. In order to facilitate the subsequent measurement of the difference between different components and the combination with information theory methods, the numerical values need to be normalized. The present application defines p m(∈), such that for a given ∈, the normalized results of all components can be regarded as a set of non-negative weights of similar probability distributions, while the normalization factor σ(∈) is used to regulate the relative difference between components. When σ(∈) takes a small value, the exponential function is more sensitive to larger or smaller Ψ m (M(X),∈) will produce more significant amplification or weakening, making the distribution tend to extreme forms; if σ(∈) is relatively large, it can be smoothly compressed to reduce the gap between components, and all Ψ m (M(X),∈) will occupy a certain proportion in the probability distribution. The advantage of this is that when the numerical difference of the topological feature values of different dimensions is huge, a more reasonable weight allocation can also be obtained through this mechanism, and other potential abnormalities will not be hidden by the excessive prominence of individual components. At the same time, the local entropy H(∈) defined on the basis of this set of probability distributions can reflect the overall uncertainty or chaos degree of Ψ(M(X),∈) at scale ∈. Specifically, if only a few components dominate at a certain ∈, and their normalized weights are significantly higher than those of other components, it will make H(∈) lower; on the contrary, if all components are close in size and there is no dominant component, the entropy value will increase. For multi-dimensional time series anomaly detection in big data scenarios, this local entropy can to some extent reflect the degree of heterogeneity of the internal structure of the system: if some topological features suddenly stand out, causing the proportion of the corresponding components to increase rapidly after normalization, it will cause the local entropy to drop significantly, thus becoming a sign of anomaly; if all components show consistent rising or falling trends, it means that there may be a systematic disturbance or global change. Further, by gradually changing the value of ∈ and dynamically tracking p m (∈) and H(∈), the system can observe the fluctuation curve of the local entropy at multiple scales, and then determine whether the anomaly is limited to a certain specific scale range or persists in a wide spectrum of topological filtering.

[0123] For industrial monitoring or massive data of the Internet of Things, if a few dimensions of Ψ m (M(X),∈) are extremely prominent at certain small scales, the corresponding p m (∈) will be very biased towards these components, and the local entropy will also jump rapidly near this scale; if this situation is still obvious at larger ∈, it means that the structural deformation caused by potential anomalies has passed through multiple filtering processes, and it is unlikely to be isolated noise or short-term random disturbance. In the automated operation scenario, the algorithm can calculate multiple H(∈) groups according to the discrete values of ∈, and then correspond them with the time axis to capture the entropy fluctuation features produced in the sliding window at different times. The dimension reduction and manifold construction done by the autoencoder in the previous step provide these topological components Ψ mThe acquisition of (M(X),∈) lays a solid foundation for data to still be able to calculate a reasonable topological signature through a unified spatial metric in a high-dimensional heterogeneous environment. Since the application uses exponential normalization in the calculation process of local entropy, it can still maintain balanced consideration of different components in the face of extreme cases, thereby avoiding some bias of traditional entropy methods on multi-dimensional data. Such entropy values also have certain interpretability: when H(∈) is low, it often means that the data distribution on the manifold presents a highly unbalanced pattern, and the topological features of some channels or sub-populations are extremely prominent, which may correspond to anomalies; when H(∈) is high, the system is in a relatively smooth or dispersed structure state, and does not necessarily mean anomalies, but more likely reflects the consistency or uniform distribution of the signal. If you want to accurately locate which dimensions or features significantly contributed to the change of this entropy, you can also directly observe the distribution of p m The distribution of (∈) to determine whether there is a major deviation in the instantaneous amplitude level, persistent homology level, or wavelet packet energy level. On the other hand, the original intention of designing the normalization factor σ(∈) is to flexibly adjust the sensitivity of entropy calculation to numerical differences at different scales. By moderately relaxing or tightening σ(∈), the system can adjust the response strength when some topological components deviate significantly in specific application scenarios, so that the application is suitable for complex environments that require both wide-range monitoring and often face small-probability severe anomalies. In industrial sites or large distributed networks, such multi-dimensional data often accompanied by noise and interference is particularly common, and by dynamically adjusting σ(∈) and combining the aforementioned barcode information, it is often possible to quickly capture sudden failures while taking into account the identification of gradual hazards.

[0124] In step 3, the abnormal probability is calculated according to the local entropy by the following formula:

[0125]

[0126] Where P anom is the abnormal probability.

[0127] Specifically, the application maps the entropy value to the interval [0, 1] by introducing a function like on the basis of local entropy, thereby obtaining an abnormal probability that is easy to interpret and can be flexibly thresholded. When the local entropy value H(∈) is small and less than 0.5, it means that the data in the manifold space presents a strong imbalance or concentrated distribution at this scale, and it may also mean that the topological features of some dimensions or channels are abnormally prominent, resulting in a decrease in the disorder degree of the overall system structure; for this period or component, the abnormal probability P anomH(∈) will increase and gradually approach 1, thus triggering the system's attention. When H(∈) is at or above 0.5, it means that the entropy value of the multi-dimensional data feature is relatively balanced, and no dimension or a few dimensions have extreme fluctuations in the topology, so the abnormal probability in this area tends to remain at a relatively low level. In the big data scenario, such a function form provides a relatively smooth and segmented adjustment mapping, which does not produce dramatic on-off changes due to slight fluctuations in entropy, but gradually adjusts the abnormal probability according to the up and down deviation of the entropy value, thereby reducing false alarms caused by noise interference. At the same time, this also allows decision makers to directly observe at the reference point of 0.5 whether the deviation of local entropy is sufficient to trigger the weighting mechanism, so that the system can make corresponding interventions or alarm prompts. For example, if the local entropy in a plurality of scale intervals at a certain time is less than 0.5, and the corresponding abnormal probability exceeds a certain safety threshold, further confirmation can be made whether there is a systematic failure by combining the manifold structure, topological barcode and the like mentioned above. Because the present application emphasizes the combination of wavelet packet energy, instantaneous amplitude and instantaneous frequency, and persistent coherence into the autoencoder and manifold space, the local entropy calculated each time has already integrated the contribution of each component; if some potential risks only occur in a short period or in a small dimension, the local entropy may still be high in most scales, thereby keeping the abnormal probability in a medium or low range, avoiding excessive sensitivity of the alarm. When some large-scale, multi-dimensional out-of-range conditions occur, a significant decrease in local entropy will be quickly converted into a high abnormal probability, helping the system accurately screen possible sudden failures or long-term hidden dangers in a large amount of data stream. It is worth noting that in actual application, the coefficient of-0.5 can be adjusted moderately, because different industries or scenarios have different definitions and tolerance intervals for abnormalities. If any entropy value is extremely sensitive in some high-risk process, the coefficient can be appropriately increased, so that a slight decrease in entropy value can significantly increase the abnormal probability; for systems that require smooth operation but tolerate a small amount of dramatic fluctuations, the coefficient can be reduced to make the increase in abnormal probability more smooth, thereby reducing false positives. The present application also allows adaptive learning on the threshold of entropy: if it is found in the training stage or historical data that the system is actually in a safe mode when the local entropy is generally in a certain interval, the offset or steepness of the mapping function can be automatically fine-tuned during online detection, thereby further improving the accuracy and flexibility of detection. In addition, this probability mapping based on the Sigmoid function transformation also has good numerical stability. For extreme cases of entropy value, such as extremely low entropy or extremely high entropy, it can converge between 0 and 1, and there is no problem of numerical overflow. Moreover, the function presents a slow gain in the middle region, so that normal fluctuations in local entropy will not easily trigger false alarms, and only when there are significant out-of-range signs will the abnormal probability be increased.This is in harmony with the persistent homology and wavelet packet, Hilbert instantaneous amplitude of the capture of abnormal characteristics, let the real systemic or significant anomalies quickly emerge.

[0128] The above-described and above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A big data anomaly detection method based on a time-series multi-dimensional autoencoder, characterized in that, The method comprises: Step 1: decomposing the multi-dimensional time series data, first extracting the internal oscillation mode of the multi-dimensional time series data by using Hilbert-Huang transform, then capturing the local characteristics in different scales and time periods by wavelet packet transform, adaptively extracting multi-level spatio-temporal characteristics from the multi-dimensional time series data, and reconstructing the same as the decomposition result; Step 2: mapping the decomposition result to a low-dimensional and continuous manifold space; on the obtained manifold space, the structural information of the multi-dimensional time series data is extracted by calculating persistent homology topological invariants; Step 3: based on the extracted structural information, the local entropy of the multi-dimensional time series data at a set scale is calculated; and according to the local entropy, the abnormal probability is calculated; In step 2, the decomposition result is mapped to a low-dimensional and continuous manifold space by the following formula: ； wherein, is a smooth manifold; is a Kronecker product operation; is a Hilbert-Schmidt norm; is a time integration variable; is a Ricci curvature tensor; In step 2, on the obtained manifold space, the structural information of the multi-dimensional time series data is extracted by calculating persistent homology topological invariants by the following formula: ； wherein, denotes the scale Vietoris–Rips complex constructed from the data set is the number of simplices of dimension in the scale is the number of simplices of dimension is the number of simplices of dimension is the number of simplices of dimension is the number of simplices of dimension and are the birth and death scales, respectively; each barcode pair is converted into a numerical feature; is the infimum operation; denotes the infimum of those for a given homology class that disappear during the filtering process in all scales ; denotes the infimum of those for a given homology class that disappear during the filtering process in all scales ; denotes the infimum of those for a given homology class that first appear in the scale parameter ; is the structural information of the multi-dimensional time series data of dimension in the scale .​ 2.The big data anomaly detection method based on time-series multi-dimensional auto-encoder of claim 1, wherein, In step 1, the multi-dimensional time series data is: ； wherein, represents the dimensionality of the multi-dimensional time series data; is the dimensional time series data, ; is the observation length or the number of discrete sampling points; is the time; when the intrinsic oscillation mode of the multi-dimensional time series data is extracted by using the Hilbert-Huang transform, the following processing is performed for each dimension : the is decomposed into a set of intrinsic mode functions and a residual term: ； wherein, is the first intrinsic mode function of the dimensional time series data; is the first total number of intrinsic mode functions of the dimensional time series data; is the residual term of the dimensional time series data; and for the first intrinsic mode function, the initial candidate intrinsic mode function is defined as: ； is an integer index; in the current candidate intrinsic modal function , find all local maxima and minima, respectively, get the upper envelope and lower envelope by spline interpolation; define the local average envelope as: ； The candidate intrinsic mode function is updated as: ； The normalized error is calculated as: ； When , is a preset convergence threshold, it is considered that the intrinsic mode function condition has been met, denoted as . 3.The big data anomaly detection method based on time-series multi-dimensional auto-encoder of claim 2, wherein, In step 1, for each eigenmode function The Hilbert-Huang transform is calculated by the following equation: ； wherein, denotes the Cauchy principal value integral; is the time integral variable; the corresponding analytical multi-dimensional time series data is constructed by the following formula : ； wherein is the imaginary unit, satisfying ; is the instantaneous amplitude; is the instantaneous phase; is the instantaneous frequency. 4.The big data anomaly detection method based on time-series multi-dimensional auto-encoder of claim 3, wherein, In step 1, each eigenmode function obtained is The wavelet packet transform is used to capture local features at different scales and time periods. Let the mother wavelet be The wavelet packet basis functions corresponding to the scales and shifts are constructed. ； wherein, is a scale; and denote the minimum and maximum scales, respectively; the intrinsic mode functions are expanded in a basis and the coefficients are computed as local features.

5. The big data anomaly detection method based on time series multidimensional autoencoder according to claim 4 is characterized in that: In step 1, the first The reconstructed expression of the multidimensional time series data is: ； wherein denotes the scale the set of all valid translation indices below; are the reconstruction coefficients, whose computation takes into account the wavelet packet coefficients and the instantaneous amplitude extracted by the Hilbert-Huang transform and the frequency defined as: ； For the first The decomposition result of the multi-dimensional time series data is: 。 6.The big data anomaly detection method based on time-series multi-dimensional auto-encoder of claim 1, wherein, In step 3, the structural information of the multi-dimensional time series data is: ； Exponential normalization is used for each component, and a normalized probability distribution is defined: ； ; ; for scale is a normalization factor, is a set value; the normalized probability distribution is computed at each scale ; : 。 7. The big data anomaly detection method based on time-series multi-dimensional autoencoder of claim 6, wherein, In step 3, according to the local entropy, the abnormal probability is calculated by the following formula: ； wherein, is the probability of abnormality.

Images