Risk signal identification method based on big data mining
By adopting big data mining technology in risk signal analysis, including multimodal feature construction, topological spatial analysis, nonlinear dynamics recognition and multi-time scale analysis, the shortcomings of multimodal data processing, topological feature extraction, dynamic anomaly point detection and time scale analysis in the existing technology are solved, and high-precision risk signal identification and dynamic visualization are achieved.
Patent Information
- Application Number
- CN202510160824.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-13
- Publication Date
- 2025-06-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the risk signal analysis, the existing technology has problems such as lack of global uniformity in multimodal data processing, insufficient topological feature extraction, insufficient detection capability of dynamic anomaly point, difficulty in capturing complex risk signals on a single time scale analysis, and difficulty in static visualization to show signal propagation paths and evolution processes.
Using a method based on big data mining, we can realize the global unified processing of multimodal data, accurate extraction of topological features, efficient identification of dynamic outliers, risk signal analysis and dynamic visualization through technical solutions based on nonlinear dynamics, multi-time scale analysis, and risk signal identification and visualization.
It improves the accuracy and scope of application of risk signal recognition, can accurately capture abnormal patterns, identify dynamic abnormal points, characterize the diverse performance of complex risk signals, and display the distribution and propagation path of risk signals through dynamic visualization.
Smart Images

Figure CN120086766A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big data analysis and risk management, and specifically to a risk signal recognition method based on big data mining. Background Art
[0002] With the rapid development of digitization and informatization, the amount of data generated by various industries has increased exponentially, and the data types have become increasingly diverse. Especially in the fields of financial risk control, network security, and intelligent manufacturing, a large amount of structured data, unstructured data, and time series data are used to identify potential risk signals.
[0003] Certain progress has been made in the prior art in risk signal analysis. For example, simple fusion methods for multi-modal data can process some heterogeneous data, improving the limitations of single data analysis. At the same time, anomaly detection methods based on statistics have been widely applied, and can identify some abnormal patterns through clustering analysis or density evaluation. In addition, the fixed window method introduced in time series analysis has also improved the ability to capture short-term risk signals. The use of visualization tools has also enhanced the intuitive display of data to a certain extent, facilitating analysts to make a preliminary judgment on risk points.
[0004] However, there are still several key deficiencies in the prior art, which seriously limit the accuracy and scope of application of risk signal recognition; firstly, traditional methods lack global unity in the processing of multi-modal data and cannot fully integrate the correlation information between different data types; secondly, the technology for extracting topological features of high-dimensional data is relatively lacking, and existing methods mostly rely on simple statistical features and cannot reveal complex global structures and local abnormal patterns; in addition, the neglect of non-linear dynamic characteristics in time series analysis leads to insufficient detection ability for dynamic anomaly points; more importantly, single-time-scale analysis methods are difficult to capture the diverse manifestations of risk signals in short-term fluctuations and long-term trends; finally, existing static visualization methods are difficult to display the signal propagation path and evolution process in dynamic scenarios. Summary of the Invention
[0005] In view of the deficiencies of the prior art, the present invention provides a risk signal recognition method based on big data mining, which solves the problems of the prior art in multi-modal data fusion, topological feature extraction, dynamic anomaly point detection, and risk signal visualization.
[0006] To achieve the above object, the present invention is realized through the following technical solutions: A risk signal recognition method based on big data mining, comprising the following steps: S1. Data preprocessing and multi-modal feature construction: By collecting structured data, unstructured data, and time series data, a multi-modal high-dimensional feature vector is uniformly constructed; S2. Topological Space Construction and Persistent Homology Analysis: Construct an adjacency graph using the Euclidean distance between data, extract topological features of the high-dimensional space, and calculate homology groups and Betti numbers to extract abnormal patterns; S3. Dynamic Outlier Identification Based on Nonlinear Dynamics: Construct a phase space trajectory through time-delay embedding, and calculate Lyapunov exponents and fractal dimensions to identify high-risk outliers; S4. Multi-Time Scale Analysis: Decompose the signal into multiple time-scale features using wavelet transform and fuse the topological features on different time scales; S5. Risk Signal Identification and Visualization: Combine dynamic adjacency matrix optimization and threshold strategy to output a set of high-risk points and generate a dynamic topological situation map to display the distribution and propagation path of risk signals.
[0007] Preferably, the data preprocessing and multi-modal feature construction include: Collect structured data, where the structured data includes transaction records and financial information; Collect unstructured data, where the unstructured data includes text descriptions and social network relationship data; Collect time series data, where the time series data includes behavior logs and the time dynamics of risk signals; Perform standardization and denoising processing on the collected data, and use the principal component analysis method to reduce the dimension of the data and retain the main feature information.
[0008] Preferably, the standardization and denoising processing include: For structured data, use the normalization method to map all feature values to the interval [0, 1]; For unstructured data, use stop word filtering, word segmentation, and duplicate removal to clean the text content; For time series data, use the sliding window mean method to smooth data fluctuations and eliminate random noise interference.
[0009] Preferably, the topological space construction and persistent homology analysis include: Calculate the Euclidean distance between data points and construct an adjacency graph according to the set distance threshold; Generate a simplicial complex based on the adjacency graph, where the simplicial complex is composed of combinations of point sets; Calculate the homology group and its Betti number for the generated simplicial complex to extract topological features of different dimensions.
[0010] Preferably, the dynamic outlier identification based on nonlinear dynamics includes: The time-delay embedding method is adopted to construct the phase space trajectory, which is composed of time series points and their delayed points; calculate the Lyapunov exponent of the trajectory, and judge the sensitivity of the system to the initial conditions through the Lyapunov exponent; Calculate the fractal dimension of the trajectory, characterize the complexity of the trajectory through the fractal dimension, and determine the abnormal points by combining the Lyapunov exponent and the fractal dimension.
[0011] Preferably, the multi-time scale analysis includes: Decompose the signal by wavelet transform, and the decomposition result is characterized by multiple scale parameters and time translation parameters; For the signal of each time scale, calculate its topological features; Perform weighted fusion on the topological features of different time scales to generate a risk feature vector under multiple time scales.
[0012] Preferably, the risk signal identification and visualization include: According to the optimization result of the dynamic adjacency matrix, combined with the set risk threshold, output the set of high-risk points; Generate a dynamic topological situation map based on the set of high-risk points to display the distribution and propagation path of the risk signal; Through the real-time update of the risk situation map, dynamically adjust the risk identification threshold and the adjacency matrix optimization strategy.
[0013] Preferably, the optimization of the dynamic adjacency matrix adopts objective function optimization, and the objective function is defined as: Maximize the separation degree of abnormal points. The objective function includes the distance optimization term between the abnormal point and the center of its neighborhood and the adjacency matrix regularization term, and the adjacency matrix regularization is used to constrain the optimization range of the adjacency matrix.
[0014] Preferably, the dynamic topological situation map includes: The risk points are used as the nodes of the graph, and the weight of the node represents the risk intensity; The risk propagation path is used as the edge of the graph, and the weight of the edge represents the correlation intensity of signal propagation; Through the real-time update of the graph structure, display the dynamic changes of the risk signal.
[0015] Preferably, the dynamic adjustment of the threshold strategy is based on the following rules: Combine the historical risk data and the current business data to update the risk threshold; The threshold adjustment is determined according to the distribution characteristics of the abnormal points and the clustering results of the abnormal points.
[0016] The present invention provides a risk signal identification method based on big data mining. It has the following beneficial effects: 1. Through the technical solutions of topological space construction and persistent homology analysis, the present invention can extract global topological features in high-dimensional spaces, achieving the effect of accurately capturing abnormal patterns. Most of the existing technologies rely on simple statistical features and cannot reveal the global structure problems of complex data. The present invention effectively solves this limitation.
[0017] 2. By using nonlinear dynamics analysis, combined with time-delay embedding, Lyapunov exponents, and fractal dimensions, the present invention accurately locates dynamic abnormal points. Traditional methods are difficult to capture the chaotic characteristics in dynamic signals, while the present invention can fully explore the nonlinear features of data, improving the accuracy of risk identification.
[0018] 3. By introducing multi-time scale analysis, decomposing signals using wavelet transform, and fusing topological features at different scales, the present invention can simultaneously characterize short-term fluctuations and long-term trends, solving the problem of insufficient analysis of complex risk signals by single-time scale methods.
[0019] 4. Through dynamic adjacency matrix optimization and risk threshold adjustment, the present invention realizes the accurate identification of high-risk points and reduces false alarms. Compared with traditional methods with fixed thresholds, the present invention flexibly adapts to the dynamic changes of risk signals, improving the stability and reliability of identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 is the flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the specification of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0022] Please refer to the attached Figure 1 , the embodiments of the present invention provide a risk signal identification method based on big data mining, including the following steps: S1. Data preprocessing and multi-modal feature construction: By collecting structured data, unstructured data, and time series data, a multi-modal high-dimensional feature vector is uniformly constructed; Through the collection, standardization, noise reduction, and dimensionality reduction of multi-modal data, the unified representation of multi-source data is realized, providing accurate and reliable input for subsequent topological analysis and dynamic abnormal point identification. This step fully considers the complexity and heterogeneity problems in multi-modal data and provides a standardized technical processing flow.
[0023] Generally, multi-modal data comes from structured data, unstructured data, and time series data. The data is diverse and widely distributed. To ensure the quality of the input data, the present invention adopts an optimized acquisition and processing scheme.
[0024] The collected data includes: Structured data: Transaction records (such as amount, frequency, transaction time interval, etc.) and financial information (such as loan amount, repayment period, overdue status, etc.).
[0025] Unstructured data: Text descriptions (such as loan application instructions, customer evaluations, etc.) and social network relationship data (such as friend relationships, interaction frequencies, etc.).
[0026] Time series data: Behavioral logs (such as login time, page switching times, etc.) and dynamic monitoring data.
[0027] In a possible implementation, data acquisition directly interfaces with the enterprise business management system (such as the ERP system) and the customer relationship management platform (CRM) through API interfaces. For public data, it can be obtained through web crawler technology and uniformly stored and formatted.
[0028] In a possible implementation, feature extraction methods are designed for different modal data respectively: Structured data: After direct normalization processing, it is mapped to a unified feature vector space.
[0029] Unstructured data: Adopt natural language processing (NLP) technology, specifically including: Word segmentation: Segment continuous text into independent lexical units.
[0030] TF-IDF calculation: Assign weights to each vocabulary, and the weight calculation formula is: TF-IDF(t,d) = TF(t,d) · IDF(t) Where: t is the term; d is the document; represents the frequency of the term t in the document d; Where N is the total number of documents, and n(t) is the number of documents containing the term t.
[0031] Time series data: For time series such as behavioral logs, use the sliding window method and feature extraction technology to generate statistical features, specifically including: Mean: The average value of the data points within the window; Standard deviation: The volatility of the data within the window; Kurtosis and skewness: Used to describe the distribution shape.
[0032] In some embodiments, advanced statistical features such as extreme points and fluctuation frequencies within the window can be further extracted.
[0033] To ensure the consistency and quality of data features, the present invention performs standardization and noise reduction on the collected raw data: Standardization: Map the structured data feature values to the interval [0,1], and the formula is: where: x i : The raw data value; min(x) and max(x): The minimum and maximum values in the dataset; x i ′ : The normalized data value.
[0034] Time series data noise reduction: Use the exponential smoothing method to process time series noise, and the smoothing value formula is: S t =αX t +(1 - α)S t-1 where: S t : The smoothing value at time t; X t : The raw data value at time t; α: The smoothing coefficient, with a range of [0,1].
[0035] In one implementation, the value of α can be dynamically adjusted according to the sequence fluctuation range to balance short-term fluctuations and long-term trends.
[0036] Dimensionality reduction processing aims to compress high-dimensional feature vectors and retain the main information. The present invention uses the principal component analysis (PCA) technique to perform dimensionality reduction on the feature matrix, and the specific implementation is as follows: Input feature matrix where n is the number of samples and d is the original feature dimension.
[0037] Calculate the covariance matrix of the feature matrix
[0038] Perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues λ 1 ,λ 2 ,…,λ d and the corresponding eigenvectors.
[0039] Sort by the eigenvalue size, select the top k eigenvectors, and form the dimensionality-reduced feature matrix X k =XW k .
[0040] In some embodiments, the retained feature contribution rate after dimensionality reduction is usually set above 95% to ensure that the main information of the data is not lost.
[0041] The feature vectors after dimensionality reduction will be integrated into a unified input format for subsequent analysis steps. Generally, the integrated feature vector matrix is in the form of where each row represents the comprehensive features of a sample.
[0042] Through the full process of data collection, feature construction, standardization, noise reduction, dimensionality reduction, and integration, step S1 provides high-quality feature input for subsequent steps.
[0043] S2. Topological space construction and persistent homology analysis: Construct an adjacency graph using the Euclidean distance between data, extract topological features of the high-dimensional space, and calculate homology groups and Betti numbers to extract abnormal patterns; After completing data preprocessing and multi-modal feature construction, use topological space construction and persistent homology analysis techniques to extract high-dimensional topological features of the data. Generate a simplicial complex by constructing an adjacency graph and calculate persistent homology to obtain the global geometry and local anomaly characteristics of the data. The extraction and analysis of topological features can effectively identify important patterns in the data while eliminating noise interference. As a continuation of the foregoing module, this step further deepens the mining of data relationships and provides accurate input for dynamic outlier identification.
[0044] Generally, the key to topological space construction lies in generating a graph structure through the relationship between points and further constructing a high-dimensional topological representation. In some embodiments, in order to adapt to the dynamic characteristics of the data, the threshold can also be dynamically adjusted to generate topological representations at different scales.
[0045] In this embodiment, first construct an adjacency graph by calculating the similarity relationship between data points. Generally, the Euclidean distance can be used as a metric, and its calculation formula is: where: x i , x j : data points; x ik , x jk : the value of the data point in the k-th feature dimension; m: the feature dimension of the data.
[0046] As a possible improvement, if there are differences in the importance of features, weights can be assigned to each dimension. The modified distance formula is: where: w k : the weight factor of the k-th feature, usually determined by feature importance analysis or cross-validation.
[0047] According to the calculated distance, a threshold ∈ is set, and the points with a distance less than ∈ are connected to generate an adjacency graph G = (V, E). Where: V: the set of vertices corresponding to the data points; E: the set of edges connecting the vertices.
[0048] Based on the adjacency graph, in this embodiment, a high-dimensional topological structure - a simplicial complex is generated. Specifically, a simplicial complex is a combinatorial structure formed by a set of vertices through geometric relationships.
[0049] In a possible implementation, the specific rules for generating the simplicial complex are as follows: If there is an edge connecting two points, then the pair of points generates a 1-simplicial complex; If three points are pairwise connected, then a 2-simplicial complex (i.e., a triangle) is generated; If k + 1 points are pairwise connected, then a k-simplicial complex is generated.
[0050] The generated complex reflects the geometric characteristics and local connectivity of the data to a certain extent. Generally, the generation rule of the simplicial complex depends on the Vietoris-Rips method, and its core is to judge the connection relationship of points through the ∈-neighborhood.
[0051] In this embodiment, the topological characteristics of the data are extracted through persistent homology analysis. Specifically: By calculating the homology group H k of the complex, the topological structures of different dimensions are extracted.
[0052] Using the Betti number β k to characterize the number of k-dimensional topological characteristics.
[0053] When calculating the homology group, a boundary operator is introduced, and its definition is: where σ: the simplicial complex; boundary operator: k-dimensional complex; the set of points after removing the i-th vertex.
[0054] Through the calculation of the boundary operator, a chain complex structure of the complex is constructed, and the generators of the homology group are further decomposed. The calculation formula of the Betti number is: β k = Rank(Z k ) - Rank(B k ) where: Z k : k-cycle group; B k : k-boundary cycle group; Rank(Z k ) : Z kThe rank of, representing the k-dimensional connected component; Rank(B k ): The rank of B k represents the number of k-dimensional boundary chains.
[0055] In this embodiment, in order to eliminate noise interference, a persistence analysis technique is further introduced. Persistence analysis quantifies the importance of topological features by calculating the appearance and disappearance times of topological structures.
[0056] Persistence can be represented as an interval [b, d].
[0057] The persistence of a feature is defined as: Pers = d - b where: Pers: The persistence of the topological feature; b: The birth time of the topological feature; d: The death time of the topological feature.
[0058] As an optimization strategy, this embodiment sets a persistence threshold τ, and only retains topological features with Pers > τ. Topological features with low persistence are regarded as noise and filtered out.
[0059] In some embodiments, in order to more comprehensively describe the multi-scale features of data, this embodiment dynamically adjusts the threshold ∈ to generate topological complexes at different scales. Visualize the persistence results at multiple scales as persistence barcodes or persistence diagrams to facilitate the intuitive display of the changing trends of topological features.
[0060] Through topological space construction and persistent homology analysis, the present invention effectively extracts high-dimensional topological features in multi-modal data, and at the same time reduces noise through persistence analysis, enhancing the expressive ability of the topological structure.
[0061] S3. Dynamic outlier identification based on nonlinear dynamics: Construct a phase space trajectory through time delay embedding, and calculate the Lyapunov exponent and fractal dimension to identify high-risk outliers; After completing the extraction of topological features, by using the theory of nonlinear dynamics, potential high-risk outliers are identified from the dynamic evolution characteristics of the data. By constructing a phase space trajectory through time delay embedding technology and combining nonlinear features such as the Lyapunov exponent and fractal dimension, the dynamic changes of the data can be accurately described. Generally, these features reveal the chaotic characteristics and complexity of time series data, providing a sufficient basis for the location of dynamic outliers.
[0062] Specifically, the present invention maps time series data to a high-dimensional phase space, enabling the visual expression of hidden dynamic patterns. Combining with the objective function optimization strategy enhances the ability to separate outliers. The following are the specific technical details.
[0063] Time Delay Embedding In this embodiment, the time-delay embedding technique is used to transform time series data into high-dimensional phase space trajectories. Let the original time series be X = {x 1 , x 2 , …, x n}, and the formula for time-delay embedding is: Φ(X) = {(x(t), x(t + τ), …, x(t + (d - 1)τ)) | t = 1, 2, …, n - (d - 1)τ} where: Φ(X): the embedded high-dimensional trajectory; x(t): the original sequence value at time t; τ: the time delay, representing the sampling interval; d: the embedding dimension, representing the reconstruction dimension of the phase space; n: the total length of the time series.
[0064] As an option, the selection of the time-delay parameter τ can be determined by the first zero point of the autocorrelation function. For example, for a time series with periodic characteristics, τ usually corresponds to a part of the period, so as to ensure the minimum correlation between trajectory points.
[0065] In a possible implementation, the choice of the embedding dimension d is determined according to the False Nearest Neighbors (FNN) method. The FNN method finds the appropriate d by measuring the change in the proportion of false nearest neighbor points when the embedding dimension increases. When the proportion of false nearest neighbor points tends to be stable, the corresponding d is selected.
[0066] Lyapunov Exponent Calculation In this embodiment, the Lyapunov exponent is used to characterize the sensitivity of the time series trajectory to the initial value. The definition of the Lyapunov exponent is: where: λ: the Lyapunov exponent; ∥ΔX(0)∥: the distance between adjacent trajectories at the initial moment; ∥ΔX(t)∥: the distance between adjacent trajectories at time t; t: time; ln: the natural logarithm; lim t→∞ : the limit operation.
[0067] To improve the calculation efficiency, this embodiment adopts an improved trajectory separation method. Specifically, multiple groups of initial point pairs are selected, the divergence rate on the trajectory is calculated, and the results are averaged: where: λ: the Lyapunov exponent; m: the number of initial point pairs; ΔX i (t): the distance between the i-th pair of initial points at time t; ΔX i (0): the distance between adjacent trajectory points at the initial moment; t: time; ln: the natural logarithm; lim t→∞ : the limit operation.
[0068] Generally, if λ > 0, the trajectory shows a chaotic state, indicating a high degree of dynamic instability of the system. The abnormal points usually show significantly higher local Lyapunov exponents.
[0069] Calculation of fractal dimension In this embodiment, the geometric complexity of the trajectory is characterized by the fractal dimension. The fractal dimension is used to describe the self-similarity of the trajectory, and its calculation formula is: where: D f : fractal dimension; ∈: covering radius; N(∈): the minimum number of balls required to cover the trajectory.
[0070] In a possible implementation, the box-counting method is used to approximately calculate the fractal dimension. The specific process is as follows: Select multiple different ∈ values and calculate the minimum number of boxes N(∈) required to cover the trajectory respectively; Perform a linear fit on ln(N(∈)) and ln(1 / ∈), and the slope is the fractal dimension.
[0071] For abnormal points, their fractal dimensions usually show local outliers, reflecting complex geometric features.
[0072] Dynamic weight optimization To enhance the ability to identify abnormal points, in this embodiment, a dynamic weight optimization strategy is introduced by combining the Lyapunov exponent and the fractal dimension. The weight definition formula is: where: γ i : the dynamic weight of the i-th data point; λ i : the Lyapunov exponent of the i-th point; D f,i : the fractal dimension of the i-th point; w 1 , w 2 : weight factor, used to adjust the relative importance of the Lyapunov exponent and the fractal dimension.
[0073] The dynamic weight is introduced into the objective function to optimize the separation effect of abnormal points. The objective function expression is: where: J: objective function; X i : the i-th data point; μ(X neighborhood ): the neighborhood center of point X i .
[0074] Through time-delay embedding technology, combined with Lyapunov exponents and fractal dimensions, the dynamic characteristics of time-series data are comprehensively revealed, and the introduced dynamic weight optimization strategy further enhances the ability to identify outliers.
[0075] S4. Multi-time scale analysis: The wavelet transform is used to decompose the signal into multiple time-scale features, and the topological features on different time scales are fused; After completing the identification of dynamic outliers, deeper mining of the complex features in the time series is carried out through multi-time scale analysis technology. The performance of risk signals is often affected by time scales. In the short term, it may show violent fluctuations, while in the long term, it may exhibit a stable trend or periodic behavior. Therefore, in the present invention, the time series data is decomposed by wavelet transform, combined with the topological feature extraction and weighted fusion strategy on different scales, to form a comprehensive and accurate characterization of risk signals.
[0076] Generally, the advantage of wavelet transform is that it has the ability to analyze both time domain and frequency domain, and is suitable for processing non-stationary signals. In a possible implementation, the fusion effect of multi-scale features is further optimized by dynamically adjusting the weight distribution of time scales. The following are the specific implementation details.
[0077] Wavelet transform decomposition In this embodiment, the continuous wavelet transform (CWT) is used to decompose the time series. Let the original time series be X = {x 1 , x 2 , …, x n}, and its wavelet transform formula is: where: W X (a, b): The wavelet transform result of the time series X at scale a and position b; a: Scale parameter, controlling the compression or expansion of the wavelet; b: Translation parameter, controlling the position of the wavelet; ψ: Wavelet mother function, satisfying finite support and orthogonality; ψ * *(t): The complex conjugate of the wavelet mother function.
[0078] As an option, the wavelet mother function can be selected as Morlet wavelet or Daubechies wavelet, etc. In this embodiment, the Morlet wavelet is preferably selected, and its mathematical expression is: where: ψ(t): Morlet wavelet mother function; ω 0 : Central frequency, controlling the concentration of the wavelet in the frequency domain; t: Time variable; π: Pi; π -1 / 4 : Amplitude normalization factor; Gaussian envelope; e jω0t : Complex exponential term.
[0079] By adjusting the scale parameter a, wavelet transform can decompose a time series into multiple components, each corresponding to different time-scale characteristics. A smaller a corresponds to high-frequency components, capturing short-term fluctuations; a larger a corresponds to low-frequency components, reflecting long-term trends.
[0080] Multi-scale topological feature extraction In one possible implementation, this embodiment performs topological analysis on each time-scale component. Specifically, for the component X a (t) at each scale a, its adjacency graph is calculated and the corresponding reduced complex is generated.
[0081] In persistent homology analysis, the persistence of a topological feature is defined as: Pers(H k (a,t))=d k -b k where: Pers(H k (a,t)): the persistence of the k-dimensional topological feature at scale a and time t; b k : the birth time of the topological feature; d k : the death time of the topological feature; a: the time-scale parameter.
[0082] The persistence of a topological feature reflects the stability of the feature at different time scales. This embodiment only retains features with higher persistence to remove noise interference.
[0083] Multi-time-scale feature fusion To integrate feature information from different time scales, this embodiment adopts a weighted fusion strategy to integrate features at each scale into a unified representation. The fusion formula is: where: F fused : the feature representation after weighted fusion; F a : the feature representation at scale a; w a : the scale weight, satisfying ∑w a =1; The set of all time scales.
[0084] As an option, the weight w a can be dynamically adjusted according to the persistence distribution of features at each scale. Generally, the weight of the short time scale is higher to facilitate capturing rapidly changing risk signals; the weight of the long time scale is lower to describe trend features.
[0085] Dynamic optimization of time scale In this embodiment, to improve the accuracy of feature fusion, a dynamic optimization mechanism for time scale is further designed. By analyzing the contribution degree of features at each time scale, the scale parameter a of wavelet transform and the fusion weight w are adjusteda The specific optimization goals are: Among them: a opt : Optimal scale parameter; Pers(H k (a,t)): persistence of topological features at scale a; Time range, indicating the time length of the sequence; argmax: maximum value index operation; The length of the time frame; Σ t : The sum of the time dimension.
[0086] Through the above dynamic optimization method, the distribution of features at different time scales can be balanced, and the comprehensive capture ability of risk signals can be improved. a The calculation of can combine signal energy and topological persistence. For example, the normalized persistence value can be used as the basis for weight allocation; If there is significant noise in the time series, baseline filtering can be introduced before wavelet transform to enhance the accuracy of signal decomposition; For long-term trend characteristics, periodic indicators (such as Fourier transform) can be further calculated as auxiliary features.
[0087] Through wavelet transform decomposition, multi-scale topological feature extraction and dynamic optimization strategy, the complex patterns of time series at different time scales are fully captured.
[0088] S5. Risk signal identification and visualization: Combine dynamic adjacency matrix optimization and threshold strategy to output a set of high-risk points and generate a dynamic topological situation map to display the distribution and propagation path of risk signals; After completing the dynamic outlier point identification and multi-time scale analysis in the previous steps, the present invention realizes the accurate identification and real-time visualization of high-risk points through dynamic adjacency matrix optimization and topological situation map generation. In general, the distribution of risk signals is dynamic and complex, and a single analysis method is difficult to fully reveal the risk pattern. Therefore, the present invention comprehensively utilizes adjacency matrix optimization, dynamic threshold setting, and topological visualization technology to effectively identify high-risk points and reveal the full picture of risk signal propagation.
[0089] Specifically, by dynamically adjusting the adjacency matrix weight and threshold strategy, a set of high-risk points is screened out, and an intuitive topological situation map is generated based on this. The following is a further supplement and improvement of the details of the technical implementation.
[0090] Dynamic adjacency matrix optimization In this embodiment, the optimization process of the dynamic adjacency matrix is based on the iterative solution of the objective function. dynIt represents the dynamic association relationship between data points, and its optimization goal is to maximize the separation between high-risk points and other points while ensuring the sparsity and robustness of the adjacency matrix.
[0091] The objective function is defined as: where: J: objective function; γ i : the dynamic weight of the i-th point; X i : the position of data point i; μ(X neighborhood ): the neighborhood center of point X i ; L dyn : dynamic adjacency matrix; L init : initial adjacency matrix; β: regularization parameter, used to control the sparsity of the adjacency matrix.
[0092] As an optimization strategy, the calculation of the neighborhood center μ(X neighborhood ) is based on the weighted average of all neighborhood points, and the formula is: where: the neighborhood set of point X i ; w ij : the weight between point X i and point X j , usually taking the reciprocal of the similarity between two points; μ(X neighborhood ): the neighborhood center of point X i ; X j : the data point in the neighborhood; the neighborhood set of point X i ; the sum of neighborhood points; normalization factor.
[0093] The optimization of the dynamic adjacency matrix is achieved through the gradient descent algorithm, and the adjacency matrix is iteratively updated until the objective function j converges.
[0094] Setting the dynamic risk threshold In this embodiment, the dynamic risk threshold τ is set to screen out the high-risk point set R. The identification rule for risk points is: R = {X i | ∥X i - μ(X neighborhood ) ∥ > τ} where: R: high-risk point set; X i : data point to be detected; τ: dynamic risk threshold, determined by the statistical characteristics of historical data and the current business scenario; μ(X neighborhood ): the neighborhood center of point X i ; ∥X i - μ(X neighborhood ): point Xi Distance to the neighborhood center.
[0095] Generally, the threshold τ is set according to the dispersion degree and density characteristics of the risk point distribution. For scenarios with a high concentration of risk points, the threshold can be lowered to improve the recognition sensitivity; conversely, the threshold can be appropriately increased to reduce false alarms.
[0096] As a possible implementation method, the dynamic adjustment of the threshold τ can be achieved through the following formula: τ = κ·σ(X) + μ(X) Where: κ: Sensitivity adjustment factor; σ(X): Standard deviation of all data points; μ(X): Mean of all data points.
[0097] By introducing the statistical characteristics σ(X) and μ(X), it is possible to dynamically adapt to scenarios with different data distributions.
[0098] In this embodiment of generating the topological situation map, based on the optimized dynamic adjacency matrix L dyn , a topological situation map G risk =(V risk , E risk ) is generated. The nodes and edges of the topological situation map respectively represent high-risk points and their associated relationships. The specific generation rules are as follows: Node definition The node set V risk includes all identified high-risk points, and its node weight w v is defined as: w v = γ i Where: γ i is the dynamic weight of the i-th risk point.
[0099] Edge set E risk represents the associated relationships between high-risk points, and its edge weight w e is defined as: Where: w e : Edge weight, representing the associated strength of the risk signals between points; X i , X j : Coordinates of the two endpoints of the edge.
[0100] Graph structure update Changes in the dynamic adjacency matrix L dyn directly affect the update of the topological situation map. After each update, the weights of the nodes and edges are recalculated to ensure that the situation map reflects the propagation of risk signals in real time.
[0101] Real-time Verification and Dynamic Update To improve the practicality of the model, a real-time verification and dynamic update mechanism is designed in this embodiment. Generally, real-time verification analyzes the latest business data and dynamically adjusts risk identification parameters to adapt to environmental changes.
[0102] The specific implementation includes: Real-time data input: Receive the latest data and update the dynamic adjacency matrix L dyn and the set of risk points R; Threshold reset: Recalculate the threshold τ according to the real-time data; Situation map update: Adjust the weights of nodes and edges according to the latest adjacency matrix.
[0103] Through the real-time update mechanism, this embodiment can dynamically track the changes of risk signals and provide reliable support for risk management.
[0104] Through the optimization of the dynamic adjacency matrix, the dynamic setting of risk thresholds, and the generation of topological situation maps, the accurate identification and visual expression of high-risk signals are realized. Combining with the real-time verification mechanism, the adaptability of the model to dynamic changes is further enhanced.
[0105] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A risk signal identification method based on big data mining, characterized in that: The following steps are involved: S1. Data preprocessing and multimodal feature construction: By collecting structured data, unstructured data and time series data, a multimodal high-dimensional feature vector is uniformly constructed; S2. Topological space construction and persistent homology analysis: Use the Euclidean distance between data to construct an adjacency graph, extract the topological features of the high-dimensional space, and extract abnormal patterns through homology group and Betti number calculation; S3, dynamic outlier identification based on nonlinear dynamics: construct phase space trajectories through time delay embedding, calculate Lyapunov exponents and fractal dimensions to identify high-risk outliers; S4, multi-time scale analysis: use wavelet transform to decompose the signal into multiple time scale features, and fuse the topological features at different time scales; S5. Risk signal identification and visualization: Combine dynamic adjacency matrix optimization and threshold strategy to output a set of high-risk points and generate a dynamic topological situation map to show the distribution and propagation path of risk signals.
2. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The data preprocessing and multimodal feature construction include: Collecting structured data, including transaction records and financial information; Collecting unstructured data, wherein the unstructured data includes text descriptions and social network relationship data; Collecting time series data, the time series data including behavior logs and time dynamics of risk signals; The collected data are standardized and denoised, and the principal component analysis method is used to reduce the data dimension and retain the main feature information.
3. The risk signal identification method based on big data mining according to claim 2 is characterized in that: The standardization and denoising process includes: For structured data, the normalization method is used to map all eigenvalues to the [0,1] interval; For unstructured data, use stop word filtering, word segmentation, and deduplication to clean text content; For time series data, the sliding window mean method is used to smooth data fluctuations and eliminate random noise interference.
4. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The topological space construction and persistent homology analysis include: Calculate the Euclidean distance between data points and construct an adjacency graph based on the set distance threshold; generating a reduced complex based on the adjacency graph, wherein the reduced complex is formed by a combination of point sets; The homology group and its Betti number are calculated for the generated simplified complex to extract topological features of different dimensions.
5. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The dynamic outlier point identification based on nonlinear dynamics includes: A time-delay embedding method is used to construct a phase space trajectory, wherein the phase space trajectory is composed of time series points and their delayed points; Calculate the Lyapunov exponent of the trajectory and use the Lyapunov exponent to determine the sensitivity of the system to the initial conditions; The fractal dimension of the trajectory is calculated, the complexity of the trajectory is characterized by the fractal dimension, and the abnormal points are determined by combining the Lyapunov index and fractal dimension.
6. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The multi-time scale analysis includes: The signal is decomposed by wavelet transform, and the decomposition result is characterized by multiple scale parameters and time shift parameters; For each time scale signal, calculate its topological characteristics; The topological features at different time scales are weighted and fused to generate risk feature vectors at multiple time scales.
7. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The risk signal identification and visualization includes: According to the optimization results of the dynamic adjacency matrix and the set risk threshold, a set of high-risk points is output; Generate a dynamic topological situation map based on a set of high-risk points to show the distribution and propagation path of risk signals; Through real-time updates of the risk situation map, the risk identification threshold and adjacency matrix optimization strategy are dynamically adjusted.
8. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The dynamic adjacency matrix optimization adopts objective function optimization, and the objective function is defined as: The separation degree of the outliers is maximized, and the objective function includes a distance optimization term between the outliers and the centers of their neighborhoods and an adjacency matrix regularization term, wherein the adjacency matrix regularization is used to constrain the optimization range of the adjacency matrix.
9. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The dynamic topology situation diagram includes: The risk points are nodes of the graph, and the weight of the nodes indicates the risk intensity; The risk propagation paths are taken as the edges of the graph, and the edge weights represent the strength of the association of signal propagation; The dynamic changes of risk signals are displayed through real-time updates of the graph structure.
10. The risk signal identification method based on big data mining according to claim 1 is characterized in that: The threshold policy dynamic adjustment is based on the following rules: Update risk thresholds based on historical risk data and current business data; The threshold adjustment is determined according to the distribution characteristics of the outliers and the clustering results of the outliers.
Citation Information
Patent Citations
Helium leak detection method and system for switch cabinet
CN118882947A
Method and apparatus for machine learning
US20170147946A1
Cited By
Project cost full-period digital cost prediction system
CN121280058A
