A data leakage detection system and method based on time series data anomaly detection

Abstract

The application discloses a data leakage detection system and method based on time series data anomaly detection, relates to the technical field of data leakage detection, and solves the problems of a traditional data leakage detection method based on rule matching or static threshold detection, high false positive rate, high false negative rate, poor adaptability and early warning lag when facing zero-day vulnerabilities, malicious theft by internal personnel and advanced persistent threats. The application comprises a data acquisition and analysis module, a user entity behavior portrait construction module, a core detection engine, a model management and optimization module and an alarm aggregation and response arrangement module; the data acquisition and analysis module is used for collecting original security logs and network metadata from heterogeneous data sources in real time and performing standardized analysis. The application can more accurately identify various leakage scenarios from single-point anomalies to coordinated attacks by capturing complex time patterns through "multimodal time series coding" and revealing associated threats through "graph context analysis".

Description

Technical Field

[0001] This invention relates to the field of data breach detection technology, specifically to a data breach detection system and method based on time-series data anomaly detection. Background Technology

[0002] Data breaches refer to the unauthorized access or disclosure of sensitive data. They are a significant issue in computer security and can cause serious harm at the individual, business, and societal levels. At the individual level, individuals may experience privacy breaches, harassment, fraud, financial losses, and even bankruptcy. At the business level, they may face customer loss, hefty fines, damage to brand image, and business shutdowns. At the societal level, data breaches can undermine public trust, impact financial stability, and weaken security defenses. Therefore, it is necessary to prevent data breaches. Data breach detection methods include: detection based on rule matching or static thresholds; data encryption, access control, data loss prevention protocols and firewalls; data classification and grading, security policy formulation and third-party risk management, etc.

[0003] Traditional data breach detection methods based on rule matching or static thresholds suffer from high false positive and false negative rates, poor adaptability, and delayed warnings when facing zero-day vulnerabilities, malicious insider theft, and advanced persistent threats. Therefore, they do not meet current needs. To address this, we propose a data breach detection system and method based on time-series data anomaly detection. Summary of the Invention

[0004] The purpose of this invention is to provide a data leakage detection system and method based on time-series data anomaly detection, in order to solve the problems of high false alarm rate, high false negative rate, poor adaptability, and delayed early warning in traditional data leakage detection methods based on rule matching or static thresholds mentioned in the background art when facing zero-day vulnerabilities, malicious theft by insiders, and advanced persistent threats.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a data leakage detection system based on time-series data anomaly detection, including a data acquisition and parsing module, which is used to collect raw security logs and network metadata from heterogeneous data sources in real time and perform standardized parsing; The user entity behavior profile building module is connected to the data acquisition and parsing module. It is used to generate multi-dimensional behavior time sequence feature vectors with users and / or core data entities as the aggregation subjects and according to a preset time granularity. The core detection engine is connected to the behavior profile construction module, and includes a multimodal temporal encoder and an anomaly scoring unit that integrates adaptive threshold mechanism and contextual correlation analysis. The model management and optimization module is used to store, version, and periodically retrain the models in the core detection engine. The alarm aggregation and response orchestration module is connected to the core detection engine. It is used to enrich and merge native alarms for risk assessment and to trigger a predefined response workflow.

[0006] Preferably, the feature vector includes at least: access frequency, operation type entropy, data throughput, access time regularity index and access target dispersion. The user entity behavior profile construction module also includes a dynamic feature selection submodule. This submodule dynamically selects and combines the most distinctive behavioral feature dimensions for different business departments or user roles based on mutual information or feature importance scoring to construct personalized monitoring profiles.

[0007] Preferably, the multimodal timing encoder adopts a cascaded network structure, which includes two layers: The first layer consists of multiple parallel feature-specific temporal neural networks, each used to deeply extract temporal dependency patterns of single-dimensional features. The second layer is the feature fusion layer, which uses a cross-attention mechanism to interact with and weight the feature representations of each dimension output by the first layer to generate a unified context-aware representation vector that contains cross-dimensional correlation information.

[0008] Preferably, the anomaly scoring unit includes: The error generation subunit uses the output of the multimodal temporal encoder to calculate the residual error of each feature dimension at each time point through sequence prediction or sequence reconstruction. The personalized adaptive threshold subunit maintains a set of threshold parameters for each monitored subject. Its update mechanism not only relies on the quantile statistics of its own historical errors, but also introduces a correction factor based on the global error distribution of similar subjects, in order to achieve a balance between individual behavioral fluctuations and the baseline of group behavior. The graph context analysis subunit constructs a temporary behavior graph by combining the subject at the current time point, the data entities it operates on, and other related subjects that are active at the same time. It then uses a graph neural network to analyze the isolation or propagation of the current abnormal behavior in the graph structure and outputs a graph context anomaly score.

[0009] Preferably, the personalized adaptive threshold subunit employs a dual threshold mechanism, including: A short-term sensitive threshold, calculated based on the error within a recent sliding window, is used to quickly capture sudden anomalies; A long-term stable threshold, calculated based on robust statistics of long-term historical errors, is used to detect persistent threats that drift slowly. When any threshold is triggered, an initial abnormal signal is generated, and the graph context analysis subunit performs subsequent processing.

[0010] Preferably, the model management and optimization module includes: The feedback loop learning interface is used to receive the diagnostic results of alarms from the security operations center. The incremental learning and online update unit can use new feedback samples to fine-tune some layer parameters of the multimodal temporal encoder or the parameters of the adaptive threshold subunit without performing full retraining. The model drift detection unit continuously monitors the model's performance metrics on the validation set. When the performance degradation exceeds a preset threshold, it automatically triggers the model's version archiving and a new round of training.

[0011] Preferably, the alarm aggregation and response orchestration module includes: The alarm enrichment engine is used to correlate native alarms with CMDB, identity directory, and data classification and grading system to supplement asset, personnel identity, and data sensitivity level information. The context-aware aggregation engine aggregates multiple related low-level alarms into a single high-level security event based on rules such as temporal proximity, subject identity, and entity association. The dynamic risk assessor assigns a dynamic risk value to each security event based on the alarm enrichment and aggregation results, combined with a predefined risk calculation strategy, and prioritizes response actions accordingly.

[0012] A detection method for a data leakage detection system based on time-series data anomaly detection includes the following steps: S1: Real-time streaming acquisition and parsing of multi-source logs to generate standard event streams; S2: Based on a preset time window and aggregation subject, calculate and update multi-dimensional behavioral temporal feature vectors in real time; S3: Input the temporal feature vector into a pre-trained multimodal temporal encoder to obtain its latent space representation and calculate the prediction / reconstruction error for each feature dimension; S4: Compare the error with the personalized adaptive dynamic threshold for the subject to identify preliminary anomalies; S5: Construct a temporary behavioral graph centered on the initial outlier, and use a graph neural network model to calculate the graph context outlier score of the outlier. S6: Combine the error, graph context anomaly score, and rule score based on external intelligence to generate a comprehensive risk score. If the score exceeds the threshold, trigger the original alarm. S7: Enrich the original alarms, aggregate the scenarios, and conduct dynamic risk assessments to generate actionable security events and drive the execution of the response workflow; S8: Collect feedback from security operations personnel on the handling of security incidents generated by S7. Cases confirmed as real threats and clear false alarms, along with their corresponding original feature sequences, intermediate model outputs, and contextual information, are stored as labeled samples in the training sample library.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention captures complex temporal patterns through multimodal temporal coding and reveals associated threats by combining graph context analysis. It can more accurately identify various leakage scenarios, from single-point anomalies to coordinated attacks, while effectively reducing false alarms caused by legitimate business mutations, thereby improving detection accuracy and coverage. 2. This invention enables the system to automatically adapt to changes in organizational business, personnel role adjustments, and new attack methods through dynamic feature selection, personalized dual thresholds, and a feedback-based incremental learning mechanism, maintaining high detection efficiency over the long term, reducing operation and maintenance costs, and achieving powerful self-adaptation and evolution capabilities. 3. This invention, by capturing the spread of abnormal information in the relationship network through long-term threshold capture and using image and text analysis, can detect the early stages of APT attacks, such as reconnaissance and lateral movement, as well as internal data probe behavior, earlier than traditional methods, thus achieving early and deep threat detection. 4. This invention transforms low-level technical alerts into security events that include context such as assets, identity, data sensitivity, and risk level through alert enrichment, aggregation, and risk assessment. This greatly improves the judgment and response efficiency of the security operations team and outputs highly actionable security intelligence. 5. This invention constructs a self-driven, continuously optimized, and complete intelligent security operation closed loop by collecting data, performing intelligent analysis, assessing risks, and learning responses and feedback. Attached Figure Description

[0014] Figure 1 This is a system framework diagram of the data leakage detection system of the present invention; Figure 2 This is a flowchart of the data leakage detection system of the present invention. Detailed Implementation

[0015] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0016] like Figure 1As shown, a data leakage detection system based on time-series data anomaly detection includes a data acquisition and parsing module. The data acquisition and parsing module is used to collect raw security logs and network metadata from heterogeneous data sources in real time and perform standardized parsing. The feature vector should include at least: access frequency, operation type entropy, data throughput, access time regularity index, and access target dispersion.

[0017] The user entity behavior profile building module also includes a dynamic feature selection submodule. This submodule dynamically selects and combines the most distinctive behavioral feature dimensions for different business departments or user roles based on mutual information or feature importance scores, in order to build personalized monitoring profiles.

[0018] The user entity behavior profile building module is connected to the data acquisition and parsing module. It is used to generate multi-dimensional behavior time-series feature vectors with users and / or core data entities as the aggregation subjects and according to preset time granularity.

[0019] The core detection engine is connected to the behavior profile building module, which includes a multimodal temporal encoder and an anomaly scoring unit that integrates adaptive thresholding and contextual correlation analysis.

[0020] The multimodal timing encoder adopts a cascaded network structure, which includes two layers: The first layer consists of multiple parallel feature-specific temporal neural networks, each used to deeply extract temporal dependency patterns of single-dimensional features. The second layer is the feature fusion layer, which uses a cross-attention mechanism to interact with and weight the feature representations of each dimension output by the first layer to generate a unified context-aware representation vector that contains cross-dimensional correlation information.

[0021] The anomaly scoring unit includes: The error generation subunit uses the output of a multimodal temporal encoder to calculate the residual error of each feature dimension at each time point through sequence prediction or sequence reconstruction. The personalized adaptive threshold subunit maintains a set of threshold parameters for each monitored subject. Its update mechanism not only relies on the quantile statistics of its own historical errors, but also introduces a correction factor based on the global error distribution of similar subjects, in order to achieve a balance between individual behavioral fluctuations and the baseline of group behavior. The graph context analysis subunit constructs a temporary behavior graph by combining the subject at the current time point, the data entities it operates on, and other related subjects that are active at the same time. It then uses a graph neural network to analyze the isolation or propagation of the current abnormal behavior in the graph structure and outputs a graph context anomaly score.

[0022] As a preferred embodiment, the personalized adaptive threshold subunit employs a dual threshold mechanism, including: A short-term sensitive threshold, calculated based on the error within a recent sliding window, is used to quickly capture sudden anomalies; A long-term stable threshold, calculated based on robust statistics of long-term historical errors, is used to detect persistent threats that drift slowly. When any threshold is triggered, an initial abnormal signal is generated, and subsequent processing is performed by the graph context analysis subunit.

[0023] The model management and optimization module is used to store, version, and periodically retrain models in the core detection engine.

[0024] The model management and optimization module includes: The feedback loop learning interface is used to receive the diagnostic results of alarms from the security operations center. The incremental learning and online update unit can use new feedback samples to fine-tune some layer parameters or parameters of the adaptive threshold subunit of the multimodal temporal encoder without performing full retraining. The model drift detection unit continuously monitors the model's performance metrics on the validation set. When the performance degradation exceeds a preset threshold, it automatically triggers the model's version archiving and a new round of training.

[0025] The alarm aggregation and response orchestration module is connected to the core detection engine. It is used to enrich and merge native alarms for risk assessment and to trigger predefined response workflows.

[0026] The alarm aggregation and response orchestration module includes: The alarm enrichment engine is used to correlate native alarms with CMDB, identity directory, and data classification and grading system to supplement asset, personnel identity, and data sensitivity level information. The context-aware aggregation engine aggregates multiple related low-level alarms into a single high-level security event based on rules such as temporal proximity, subject identity, and entity association. The dynamic risk assessor assigns a dynamic risk value to each security event based on alarm enrichment and aggregation results, combined with a predefined risk calculation strategy, and prioritizes response actions accordingly.

[0027] like Figure 2 As shown, a detection method for a data leakage detection system based on time-series data anomaly detection includes the following steps: S1: Real-time streaming acquisition and parsing of multi-source logs to generate standard event streams.

[0028] S2: Based on a preset time window and aggregated subject, calculate and update multi-dimensional behavioral temporal feature vectors in real time.

[0029] Among them, the generation of multi-dimensional behavioral time-series feature vectors adopts online sliding window calculation and introduces exponential weighted moving average to smooth the feature values, so as to suppress short-term noise while preserving long-term trends.

[0030] S3: Input the temporal feature vector into the pre-trained multimodal temporal encoder to obtain its latent space representation and calculate the prediction / reconstruction error for each feature dimension.

[0031] S4: Compare the error with the personalized adaptive dynamic threshold for the subject to identify preliminary anomalies.

[0032] S5: Construct a temporary behavioral graph centered on the initial outlier and use a graph neural network model to calculate the graph context outlier score of the outlier.

[0033] The nodes of the temporary behavior graph include: the subject node that triggers the anomaly, the data entity node that is accessed abnormally, and other related user nodes that have accessed the same entity as the subject within the same time period; the edge weights of the graph are determined by the similarity of operations, the proximity of time, and the sensitivity of data.

[0034] S6: Combine fusion error, graph context anomaly score, and rule score based on external intelligence to generate a comprehensive risk score. If the score exceeds the threshold, an original alarm will be triggered.

[0035] S7: Enriches raw alarms, aggregates scenarios, and performs dynamic risk assessments to generate actionable security events and drive the execution of response workflows.

[0036] The dynamic risk assessment adopts a configurable weighted scoring card model, and the scoring factors include at least: the level of the subject's authority, the sensitivity level of the data involved, the degree of deviation of the abnormal behavior, the persistence of the behavior over time, and the diffusion of the behavior on the behavior graph.

[0037] S8: Collect feedback from security operations personnel on the handling of security incidents generated by S7. Cases confirmed as real threats and clear false alarms, along with their corresponding original feature sequences, intermediate model outputs, and contextual information, are stored as labeled samples in the training sample library.

[0038] Incremental learning processes are initiated periodically or triggered to fine-tune the multimodal temporal encoder and graph neural network model using new samples from the sample library, and personalized adaptive dynamic threshold parameters are updated simultaneously to achieve adaptive evolution of detection capabilities.

[0039] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A data leakage detection system based on time-series data anomaly detection, characterized in that: It includes a data acquisition and parsing module, which is used to collect raw security logs and network metadata from heterogeneous data sources in real time and perform standardized parsing. The user entity behavior profile building module is connected to the data acquisition and parsing module. It is used to generate multi-dimensional behavior time sequence feature vectors with users and / or core data entities as the aggregation subjects and according to a preset time granularity. The core detection engine is connected to the behavior profile construction module, and includes a multimodal temporal encoder and an anomaly scoring unit that integrates adaptive threshold mechanism and contextual correlation analysis. The model management and optimization module is used to store, version, and periodically retrain the models in the core detection engine. The alarm aggregation and response orchestration module is connected to the core detection engine. It is used to enrich and merge native alarms for risk assessment and to trigger a predefined response workflow.

2. The data leakage detection system based on time-series data anomaly detection according to claim 1, characterized in that: The feature vector includes at least: access frequency, operation type entropy, data throughput, access time regularity index and access target dispersion. The user entity behavior profile construction module also includes a dynamic feature selection submodule. This submodule dynamically selects and combines the most distinctive behavioral feature dimensions for different business departments or user roles based on mutual information or feature importance scores to construct personalized monitoring profiles.

3. A data leakage detection system based on time-series data anomaly detection according to claim 2, characterized in that: The multimodal timing encoder adopts a cascaded network structure, which includes two layers: The first layer consists of multiple parallel feature-specific temporal neural networks, each used to deeply extract temporal dependency patterns of single-dimensional features. The second layer is the feature fusion layer, which uses a cross-attention mechanism to interact with and weight the feature representations of each dimension output by the first layer to generate a unified context-aware representation vector that contains cross-dimensional correlation information.

4. A data leakage detection system based on time-series data anomaly detection according to claim 3, characterized in that: The anomaly scoring unit includes: The error generation subunit uses the output of the multimodal temporal encoder to calculate the residual error of each feature dimension at each time point through sequence prediction or sequence reconstruction. The personalized adaptive threshold subunit maintains a set of threshold parameters for each monitored subject. Its update mechanism not only relies on the quantile statistics of its own historical errors, but also introduces a correction factor based on the global error distribution of similar subjects, in order to achieve a balance between individual behavioral fluctuations and the baseline of group behavior. The graph context analysis subunit constructs a temporary behavior graph by combining the subject at the current time point, the data entities it operates on, and other related subjects that are active at the same time. It then uses a graph neural network to analyze the isolation or propagation of the current abnormal behavior in the graph structure and outputs a graph context anomaly score.

5. A data leakage detection system based on time-series data anomaly detection according to claim 4, characterized in that: The personalized adaptive threshold subunit employs a dual threshold mechanism, including: A short-term sensitive threshold, calculated based on the error within a recent sliding window, is used to quickly capture sudden anomalies; A long-term stable threshold, calculated based on robust statistics of long-term historical errors, is used to detect persistent threats that drift slowly. When any threshold is triggered, an initial abnormal signal is generated, and the graph context analysis subunit performs subsequent processing.

6. A data leakage detection system based on time-series data anomaly detection according to claim 5, characterized in that: The model management and optimization module includes: The feedback loop learning interface is used to receive the diagnostic results of alarms from the security operations center. The incremental learning and online update unit can use new feedback samples to fine-tune some layer parameters of the multimodal temporal encoder or the parameters of the adaptive threshold subunit without performing full retraining. The model drift detection unit continuously monitors the model's performance metrics on the validation set. When the performance degradation exceeds a preset threshold, it automatically triggers the model's version archiving and a new round of training.

7. A data leakage detection system based on time-series data anomaly detection according to claim 6, characterized in that: The alarm aggregation and response orchestration module includes: The alarm enrichment engine is used to correlate native alarms with CMDB, identity directory, and data classification and grading system to supplement asset, personnel identity, and data sensitivity level information. The context-aware aggregation engine aggregates multiple related low-level alarms into a single high-level security event based on rules such as temporal proximity, subject identity, and entity association. The dynamic risk assessor assigns a dynamic risk value to each security event based on the alarm enrichment and aggregation results, combined with a predefined risk calculation strategy, and prioritizes response actions accordingly.

8. The detection method of a data leakage detection system based on time-series data anomaly detection according to claim 7, characterized in that, Includes the following steps: S1: Real-time streaming acquisition and parsing of multi-source logs to generate standard event streams; S2: Based on a preset time window and aggregation subject, calculate and update multi-dimensional behavioral temporal feature vectors in real time; S3: Input the temporal feature vector into a pre-trained multimodal temporal encoder to obtain its latent space representation and calculate the prediction / reconstruction error for each feature dimension; S4: Compare the error with the personalized adaptive dynamic threshold for the subject to identify preliminary anomalies; S5: Construct a temporary behavioral graph centered on the initial outlier, and use a graph neural network model to calculate the graph context outlier score of the outlier. S6: Combine the error, graph context anomaly score, and rule score based on external intelligence to generate a comprehensive risk score. If the score exceeds the threshold, trigger the original alarm. S7: Enrich the original alarms, aggregate the scenarios, and conduct dynamic risk assessments to generate actionable security events and drive the execution of the response workflow; S8: Collect feedback from security operations personnel on the handling of security incidents generated by S7. Cases confirmed as real threats and clear false alarms, along with their corresponding original feature sequences, intermediate model outputs, and contextual information, are stored as labeled samples in the training sample library.

Images