Communication data intelligent safety supervision system based on big data

Abstract

The invention relates to the technical field of big data security analysis and network information security, in particular to a communication data intelligent security supervision system based on big data, which comprises a data acquisition module used for extracting standardized entity behavior feature vectors from multi-source heterogeneous communication data; the baseline modeling module is used for dynamically generating a multi-dimensional behavior baseline portrait based on the historical sequence of the entity behavior feature vectors; the anomaly detection module is used for comparing the real-time entity behavior feature vector with the multi-dimensional behavior baseline portrait so as to calculate a micro anomaly score; the atlas construction module is used for screening the micro-anomaly events according to whether the micro-anomaly score exceeds a preset threshold value or not, and quantifying association confidence among the screened events so as to construct an attack chain atlas; the risk quantification module is used for aggregating the characteristics of the attack chain atlas to determine a systematic risk score; according to the method, the discovery capability and response efficiency of complex attacks such as advanced persistent threats and the like are greatly improved.

Description

TECHNICAL FIELD

[0001] The application relates to the fields of big data security analysis and network information technology, and particularly relates to a communication data intelligent security supervision system based on big data. BACKGROUND

[0002] In the field of network security, a supervision system detects threats by analyzing massive communication data, and most of the existing technologies detect single security alarms in isolation. This mode is insufficient in dealing with complex attacks such as advanced persistent threats (APTs). Such attacks usually manifest as a series of small, scattered and seemingly harmless abnormal behaviors. If these behaviors are analyzed as isolated events, their risk characteristics are not obvious and are easily misjudged as system noise. The existing technologies lack the ability to effectively associate these independent micro abnormal events and reconstruct the complete attack path behind them from a macro perspective, which leads to the inability to accurately quantify the systematic risk of the attack chain, making it difficult for the system to discover and respond to attacks with long duration and complex logic, thus leaving serious security risks. SUMMARY

[0003] To solve the above technical problems, the application provides a communication data intelligent security supervision system based on big data, and specifically, the technical solution of the application is as follows. A communication data intelligent security supervision system based on big data, comprising: a data acquisition module configured to extract standardized entity behavior feature vectors from multi-source heterogeneous communication data; a baseline modeling module configured to dynamically generate a multi-dimensional behavior baseline portrait based on a historical sequence of the entity behavior feature vectors; an anomaly detection module configured to compare real-time entity behavior feature vectors with the multi-dimensional behavior baseline portrait to calculate micro-abnormal scores; a graph construction module configured to filter micro-abnormal events according to whether the micro-abnormal scores exceed a preset threshold, and quantify the correlation confidence between the filtered events to construct an attack chain graph; a risk quantification module configured to aggregate features of the attack chain graph to determine a systematic risk score.

[0004] Preferably, the baseline modeling module generates a multi-dimensional behavior baseline portrait, comprising: acquiring entity behavior feature vectors in an initial observation period and calculating a mean vector; setting the mean vector as an initial behavior baseline portrait; applying an exponential weighted moving average model to iteratively update the baseline at the previous moment using newly acquired entity behavior feature vectors to dynamically update the multi-dimensional behavior baseline portrait.

[0005] Preferably, the anomaly detection module calculates the micro-anomaly score, including: Calculate the statistical distance between the real-time behavioral feature vector and the dynamic behavioral baseline profile to obtain the basic deviation. A Gaussian mixture model is trained based on the timestamps of historical activities of entities to determine the temporal sparseness of the current behavior. Query the pre-defined asset management database to determine the asset importance of the communication source asset and the target asset; The baseline deviation, time rarity, and asset importance are weighted and summed to form a minor anomaly score.

[0006] Preferably, the statistical distance is calculated using Mahalanobis distance, and the covariance matrix required to calculate the Mahalanobis distance is generated together with the multi-dimensional behavioral baseline profile generated by the baseline modeling module.

[0007] Preferably, the atlas construction module filters micro-anomaly events, including: If the micro-anomaly score of a micro-anomaly event exceeds a preset threshold, then the micro-anomaly event is identified as a graph node for constructing the attack chain graph. If the score of a minor anomaly does not exceed the preset threshold, the minor anomaly event will not be identified as a graph node.

[0008] Preferably, the map construction module quantifies the association confidence level, including: The temporal correlation of candidate event pairs is determined based on the exponential decay function; Based on preset rules, the entity association of candidate event pairs is determined; Based on the pre-computed tactical transition matrix, the tactical correlation of candidate event pairs is determined; The temporal correlation, entity correlation, and tactical correlation are weighted and summed, and then mapped using the Sigmoid function to generate the correlation confidence.

[0009] Preferably, the tactical transition matrix is ​​generated by statistical analysis of attack cases in a publicly available threat intelligence database or by mapping tactical sequence relationships in the MITREATT&CK framework.

[0010] Preferably, the risk quantification module determines the systemic risk score, including: Determine the breadth of the attack chain; Calculate the average attack strength of the attack chain; Determine the peak potential impact of the attack chain; Determine the maturity of the attack chain pattern; The attack breadth, average attack strength, peak potential impact, and pattern maturity are multiplied to generate a systemic risk score.

[0011] Preferably, the pattern maturity is calculated by a pre-trained sequence model, which is trained on known attack sample sequences to identify high-risk combined tactical patterns.

[0012] Preferred options also include: The parameter adaptive module is used to dynamically adjust the weight coefficients in the anomaly detection module in response to true positive feedback from alarms. The parameter adaptation module is also used to train and optimize the weights in the graph construction module based on a known dataset of attack samples.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. This system establishes a highly adaptive anomaly detection mechanism that dynamically generates a behavioral baseline profile for each network entity. When assessing anomalies, it innovatively integrates the statistical distance between the behavior and the baseline, temporal rarity, and asset importance to generate a micro-anomaly score. This method upgrades simple statistical deviations into a precise measurement of business risks. By prioritizing high-risk behaviors at key time points and on key assets, and by using Mahalanobis distance to overcome feature interference, it significantly improves the signal-to-noise ratio and detection sensitivity of alarms. 2. This system implements an attack chain graph construction method that closely resembles real attack scenarios. It first efficiently filters massive events through anomaly scores, and then innovatively integrates three orthogonal dimensions—time, entity, and tactical correlation—to quantify the confidence between events. In particular, it introduces a tactical transition matrix based on the ATT&CK framework, integrating prior cybersecurity knowledge into the data-driven analysis process. This enables the accurate reconstruction of complex attack chains spanning long periods of time and different entities, greatly improving the completeness of attack discovery. 3. The system constructs a multi-dimensional systemic risk quantification model that is highly correlated with real business risks. This model does not simply accumulate node scores, but comprehensively evaluates four independent factors: attack breadth, average intensity, peak potential impact determined by the highest value assets involved, and pattern maturity calculated through a sequence model. By multiplying these factors, a final risk score is generated, providing an extremely accurate and interpretable ranking basis for alerts, enabling security operations teams to focus on the most destructive threats. 4. The system possesses continuous self-learning and optimization capabilities through its parameter adaptive module. This module can respond to analysts' true positive feedback on alerts, dynamically fine-tune anomaly detection weights, and retrain and optimize the correlation model constructed from the graph based on known attack sample sets. This allows the system to evolve from a static tool into an intelligent security system capable of adapting to environmental changes and continuously combating new attacks, ensuring its long-term detection accuracy and automated operation and maintenance level. Attached Figure Description

[0014] The present invention will be further explained below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the system of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0016] Example 1: Please see Figure 1 A big data-based intelligent security monitoring system for communication data includes: The data acquisition module is used to extract standardized entity behavior feature vectors from multi-source heterogeneous communication data; The baseline modeling module is used to dynamically generate multi-dimensional behavioral baseline profiles based on the historical sequence of entity behavioral feature vectors. The anomaly detection module is used to compare real-time entity behavior feature vectors with multi-dimensional behavior baseline profiles to calculate micro-anomaly scores. The graph construction module is used to filter micro-anomaly events based on whether the micro-anomaly score exceeds a preset threshold, and to quantify the correlation confidence between the filtered events in order to construct an attack chain graph. The risk quantification module is used to aggregate features of the attack chain graph to determine a systemic risk score.

[0017] This invention provides a big data-based intelligent security monitoring system for communication data. The system aims to achieve macro-level quantification of systemic risks in the attack chain from micro-level detection of individual behavioral anomalies through in-depth analysis of massive amounts of communication data, thereby forming a complete and self-consistent technical closed loop. In this embodiment, the system specifically includes a data acquisition module, a baseline modeling module, an anomaly detection module, a graph construction module, and a risk quantification module; it may also include a tactical identification module, which is used to map the micro-anomaly events output by the anomaly detection module to specific tactical categories in a predefined network attack tactical framework. The data acquisition module aims to transform diverse and sourced raw communication data from the network into structured data that can be uniformly measured and analyzed by subsequent models. This module continuously collects heterogeneous communication data from multiple sources, including network traffic data, DNS request logs, proxy server logs, and authentication logs, through probes deployed at key network nodes. Using a built-in data cleaning and deep protocol analysis engine, the module preprocesses the collected raw data, extracting and quantifying it to form standardized entity behavior feature vectors. The standardized entity behavior feature vector F here refers to a numerical vector with a fixed dimension, which serves as the basic data unit for all subsequent analyses. Its source is multi-source heterogeneous raw communication data. For example, this vector can be set to n dimensions, and specific dimensions may include: the number of uplink / downlink data packets, the entropy of the communication protocol type distribution, the dispersion of communication target IP addresses, the distribution of DNS request types, the authentication failure rate, etc. This process ensures that regardless of the original data format, the input to the downstream module is a unified and standardized behavior record. The baseline modeling module aims to create a dynamically updated behavioral profile for each entity in the network, representing its normal operating status. This module takes as input the historical entity behavior feature vector sequence generated by the data acquisition module and applies it to each independent entity. Dynamically generate its multi-dimensional behavioral baseline profile The multi-dimensional behavioral baseline profile Be here refers to a profile related to the feature vector. The vector of the same dimension serves as a normal reference standard for comparison in subsequent anomaly detection. Its source is the time series of the entity's historical behavioral feature vectors. In this way, the system can adaptively learn the unique behavioral patterns of each entity. The anomaly detection module is designed to identify, in real-time and accurately, minor anomalies that deviate from the normal behavior patterns of an entity; it then converts the real-time entity behavior feature vectors... Compared with the multi-dimensional behavioral baseline profile generated by the baseline modeling module By comparing the results and combining them with contextual information from multiple dimensions such as time and space, a comprehensive micro-anomaly score is calculated. This score is not a simple yes or no judgment, but a continuous numerical value that can finely characterize the degree of abnormality and potential risks of behavior, providing a quantitative basis for subsequent correlation analysis. The graph construction module aims to connect isolated, high-risk micro-anomalies to reconstruct hidden, logically related attack paths in a data-driven manner. During operation, this module uses micro-anomaly scores... Whether the number of micro-anomalies exceeds a preset threshold, the system filters out the massive number of events and retains only the most suspicious events as candidate nodes in the graph. This module quantifies the correlation confidence between the filtered events. When two events show a strong correlation in time, space, and tactical logic, a directed edge is established between them to construct a dynamically evolving attack chain graph. This graph intuitively reveals the evolution process and potential intent of the attack behavior. The risk quantification module, serving as the endpoint of the entire analysis process, aims to conduct a global and systematic risk assessment of the constructed attack chains, outputting a ranking-based decision-making basis strongly correlated with real business risks. This module aggregates the global characteristics of each attack chain graph, such as chain length, the importance of the assets involved, the degree of anomaly at each node, and the maturity of the attack tactics, and applies a multi-factor risk model to determine a quantitative systemic risk score. This score can be directly used to prioritize security alerts, allowing security operations personnel to focus on the most threatening attacks. This invention constructs a complete intelligent security monitoring process from raw data to decision-making. Compared with traditional security systems that detect individual alarms in isolation, this invention can correlate and aggregate small, seemingly harmless anomalies to reveal potential attack chains that are long-lasting and logically complex, and quantitatively assess them from a macro perspective of systemic risk. This greatly improves the ability to detect and respond to complex attacks such as Advanced Persistent Threats (APTs), realizing the transformation from point-based defense to chain-based monitoring.

[0018] Example 2: The baseline modeling module generates a multi-dimensional behavioral baseline profile, including: Collect entity behavior feature vectors within the initial observation period and calculate the mean vector; Set the mean vector as the initial behavioral baseline profile; An exponentially weighted moving average model is applied, and the baseline of the previous time step is iterated using newly collected entity behavior feature vectors to dynamically update the multi-dimensional behavior baseline profile.

[0019] This embodiment is a specific and optimized implementation of the baseline modeling module described in Embodiment 1. Its core lies in using a dynamic iterative method to generate a multi-dimensional behavioral baseline profile, ensuring that the profile can adaptively adjust with the normal changes in entity behavior. To achieve this objective, the baseline modeling module generates a multi-dimensional behavioral baseline profile as follows: The process begins with identifying each new entity in the network. An initial observation period is set, such as 24 hours. Within this period, the system collects all entity behavior feature vectors of the entity. The mean vector of these vectors is then calculated; this mean vector is then set as the initial behavioral baseline profile for the entity. This forms the basis for subsequent dynamic updates; after this initial stage, the system then applies the Exponentially Weighted Moving Average (EWMA) model, utilizing the newly acquired entity behavior feature vectors. Baseline of the previous time step Iterate to dynamically update the multi-dimensional behavioral baseline profile. The introduction of the EWMA model assigns higher weight to recent behavioral data, while the weight of older behavioral data decays exponentially over time. This design allows the baseline profile to retain a memory of the entity’s long-term behavioral patterns and to respond quickly to recent legitimate changes in its behavior, thereby effectively reducing false alarms caused by drift due to normal behavior. By introducing an exponentially weighted moving average model for dynamic updates, the behavioral baseline profile generated by the baseline modeling module in this embodiment has stronger adaptability and timeliness. Compared with the static method that uses a fixed periodic mean as the baseline, this solution can more accurately track the normal behavioral evolution of entities, significantly improve the accuracy of anomaly detection, reduce false alarms caused by normal changes in business models, and enhance the robustness of the system.

[0020] Example 3: The anomaly detection module calculates micro-anomaly scores, including: Calculate the statistical distance between the real-time behavioral feature vector and the dynamic behavioral baseline profile to obtain the basic deviation. A Gaussian mixture model is trained based on the timestamps of historical activities of entities to determine the temporal sparseness of the current behavior. Query the pre-defined asset management database to determine the asset importance of the communication source asset and the target asset; The baseline deviation, time rarity, and asset importance are weighted and summed to form a minor anomaly score.

[0021] This embodiment is a specific implementation of the anomaly detection module described in Embodiment 1. It uses a weighted contextual risk assessment framework to couple statistical anomalies with actual risks of concern in the cybersecurity field, thereby calculating micro-anomaly scores that are more relevant to business operations. ; To achieve this objective, the calculation process of the anomaly detection module is broken down into the following technical steps: Step 1: Calculate the real-time behavior feature vector With dynamic behavioral baseline profile The statistical distance between the two is used to obtain the baseline deviation. The baseline deviation is a numerical value that quantifies the degree of difference between the current behavior and the historical normal pattern. Its purpose is to provide a basic statistical basis for anomaly assessment. Its source is... and Comparison calculation; Step 2: Train a Gaussian Mixture Model (GMM) based on the entity's historical activity timestamps to determine the temporal sparseness of the current action; the temporal sparseness T(·) refers to the time when an evaluation action occurs. The atypical function quantifies the degree to which the occurrence of a behavior deviates from the historical activity time distribution pattern of its entity. It is derived from a probabilistic model trained on historical timestamp data. Specifically, the system maintains a GMM model with activity timestamps for each entity, and calculates its timestamp when a new behavior occurs. probability density under this GMM model and take As a value of time rarity; a database access behavior that occurs at 3 a.m. will have a much higher time rarity than a similar behavior that occurs at 3 p.m. Step 3: Query the pre-defined asset management database (CMDB) to determine the asset importance of the source and target assets in the communication process; asset importance C(·) refers to an assessment of the communication between the two parties. and The criticality function is used to incorporate the business value of an asset into risk assessment. Its source is the predefined asset level in the CMDB. For example, the asset importance of an ordinary employee's PC accessing the development server is lower than that of the PC accessing the core database server. Step four involves weighting and summing the baseline deviation, time scarcity, and asset importance to synthesize a minor anomaly score. This process is achieved through the following core model: entity micro-anomaly scoring function: ; in: : The final output is the dimensionless micro-anomaly score; :calculate and The distance function for the degree of deviation outputs a normalized dimensionless score, which is derived from the real-time behavior feature vector and the dynamic behavior baseline profile. : Temporal rarity function, used to evaluate the timing of an action. The atypicality of the output is reflected in the normalized dimensionless score, which is derived from the timestamp of the behavior and the probability model trained based on historical timestamps. Asset-related importance function for assessing communication source assets. With target assets The criticality is determined by outputting a normalized dimensionless score, which is derived from the source / destination asset identifiers in the communication logs and the asset management database. The dimensionless weight coefficients corresponding to the three terms, and These weights can be initialized by domain experts using the Analytic Hierarchy Process (AHP). By integrating simple behavioral statistical deviations with temporal and spatial contextual risks that have clear safety significance, the system can upgrade the assessment focus from behavioral anomalies to risk anomalies. This embodiment introduces two key risk correction terms—time scarcity and asset importance—and performs weighted synthesis to improve the micro-anomaly score of the output. This not only reflects how different it is from usual, but also more accurately reflects the potential harm of this difference; this design greatly improves the signal-to-noise ratio of alarms, enabling the system to prioritize high-risk abnormal behaviors that occur at critical times and involve critical assets, thereby improving the efficiency and accuracy of security incident response. It should be noted that the effectiveness of this model in assessing asset importance depends on the accuracy and real-time performance of the asset management database (CMDB). In practical deployment, it is recommended to establish a corresponding CMDB update and auditing mechanism, or to adopt a backup strategy when information is missing, such as assigning a default importance level based on information like the network segment where the asset is located or the services it accesses, to ensure the robustness of the model.

[0022] Example 4: The statistical distance is calculated using Mahalanobis distance, and the covariance matrix required to calculate the Mahalanobis distance is generated along with the multi-dimensional behavioral baseline profile generated by the baseline modeling module.

[0023] This embodiment clarifies the specific calculation method of the statistical distance; in this embodiment, the statistical distance is calculated using Mahalanobis distance; Mahalanobis distance is an effective distance metric that takes into account the correlation between various behavioral features. In a multidimensional behavioral feature space, different feature dimensions are often not independent of each other. If a simple Euclidean distance is used, this covariance relationship will be ignored, leading to misjudgment. Mahalanobis distance standardizes the data by introducing the inverse of the covariance matrix, eliminating the interference of correlation between features, making the distance metric more robust and accurate. In one specific implementation, the covariance matrix required to calculate the Mahalanobis distance is generated and dynamically updated simultaneously with the baseline modeling module when it generates a multi-dimensional behavioral baseline profile. This means that when the baseline modeling module updates the mean vector using an exponentially weighted moving average model, it also synchronously and iteratively updates the covariance matrix describing the relationship between features in a similar manner. Thus, the anomaly detection module, when calculating the Mahalanobis distance... At that time, it can always use the current baseline. A perfectly matched, up-to-date covariance matrix ensures the timeliness and accuracy of distance calculations; By employing Mahalanobis distance as a statistical distance metric and generating the required covariance matrix synchronously with the behavioral baseline, this invention can more accurately measure behavioral deviation. This method effectively overcomes the influence of linear correlation between various behavioral feature dimensions. Compared with traditional methods such as Euclidean distance, it can more sensitively detect subtle but safety-critical abnormal patterns in multidimensional space, thereby improving the sensitivity and accuracy of anomaly detection.

[0024] Example 5: The map construction module filters micro-anomaly events, including: If the micro-anomaly score of a micro-anomaly event exceeds a preset threshold, then the micro-anomaly event is identified as a graph node for constructing the attack chain graph. If the score of a minor anomaly does not exceed the preset threshold, the minor anomaly event will not be identified as a graph node.

[0025] This embodiment refines the steps for screening micro-anomaly events in the graph construction module described in Embodiment 1. The purpose is to establish a clear and efficient filtering mechanism to ensure that only events with high risk are eligible to become nodes in the attack chain graph, thereby reducing the complexity of the graph and the overhead of subsequent calculations. For this purpose, the logic for filtering micro-anomaly events in the map construction module is as follows: For each minor anomaly event received from the anomaly detection module, the system will obtain its corresponding minor anomaly score. ; If the micro-abnormality score of this micro-abnormal event Exceeding a preset threshold If the system determines that the event has sufficient analytical value, it will identify the micro-anomaly event as a graph node for constructing the attack chain graph. preset threshold θs This refers to a numerical threshold used to distinguish between high- and low-risk events. Its function is to serve as a basis for event screening. The method for determining it can be based on the statistical distribution of historical data. For example, it can be set as the 95th percentile of the distribution of all abnormal scores to ensure that only the statistically significant abnormalities are considered. Conversely, if the score for minor anomalies is... Not exceeding the preset threshold If the event is considered to have low risk and may be noise or insignificant behavioral fluctuations, then this minor anomaly event will not be identified as a graph node and will be filtered out directly. This embodiment achieves an efficient front-end filtering mechanism by establishing a clear screening threshold based on micro-anomaly scores. This not only greatly reduces the number of nodes entering the graph construction stage and effectively reduces the complexity of subsequent large-scale event pair correlation calculations, saving computing resources, but also makes the constructed attack chain graph clearer and simpler by filtering out a large number of low-risk noise events, and better highlights the real, high-threat attack paths.

[0026] Example 6: The graph construction module quantifies association confidence, including: The temporal correlation of candidate event pairs is determined based on the exponential decay function; Based on preset rules, the entity association of candidate event pairs is determined; Based on the pre-computed tactical transition matrix, the tactical correlation of candidate event pairs is determined; The correlation confidence is generated by weighting and summing the temporal correlation, entity correlation, and tactical correlation, and then mapping them through the Sigmoid function. The tactical transition matrix is ​​generated by statistical analysis of attack cases in a public threat intelligence database or by mapping tactical sequence relationships in the MITREATT&CK framework.

[0027] This embodiment is a specific implementation of the step of quantifying the correlation confidence in the graph construction module described in Embodiment 1, which integrates multi-dimensional information, and further explains the source of the key parameters. Its core idea is that the attacker's continuous actions must have correlations in the three dimensions of time, space and tactics. By quantifying these correlations, the attack path can be reconstructed in a data-driven manner. To achieve this, the map construction module quantizes two candidate event pairs. Confidence of association between At that time, the following core model and abnormal event association confidence function are adopted: ; in: :event arrive The dimensionless correlation confidence level; The Sigmoid function aims to map the internal weighted sum to a probability range between 0 and 1. This model integrates a three-dimensional correlation assessment: Determining the temporal correlation of candidate event pairs based on the exponential decay function: by Item representation; Δt Refers to events With the event Time difference of occurrence ; τ It is a preset time decay constant, which defines the effective time window for correlation. Its value can be set based on the statistical analysis of the time interval of attack steps in historical attack samples. For example, it can be set to 24 hours, which means that the larger the time interval, the more exponentially the probability that the two events belong to the same attack will decrease. Based on preset rules, the entity association of candidate event pairs is determined: by Item representation; entity relevance R(·) refers to an evaluation event and The entity that occurs and A function of the strength of the correlation between them; its source is a set of preset rules, for example: if and If they are the same entity, then Returns 1; if they are in the same predefined network segment, returns 0.5; otherwise, returns 0. This rule is designed to capture common entity association characteristics in attack activities, such as the fact that after an attacker controls a single entity, their subsequent activities are mostly carried out within the entity or in its neighboring network space. Based on the pre-computed tactical transition matrix, the tactical correlation of candidate event pairs is determined: by Item representation; tactical relevance K(·) refers to an evaluation event tactics Transfer to event tactics It is a function of logical probability; its core is a pre-computed tactical transition matrix, which is generated by statistically analyzing a large number of attack cases in a public threat intelligence database to calculate the transition frequency between different attack tactics; or by directly mapping the tactical sequence relationships in the MITREATT&CK framework; for example, lateral movement tactics usually occur after the initial access, so the transition probability from the initial access to lateral movement is... The value is relatively high; The system performs a weighted summation of temporal correlation, entity correlation, and tactical correlation, and then uses a Sigmoid function to map the correlation to generate a correlation confidence score. Weight It can be optimized by training machine learning on known attack sample datasets to maximize the connection confidence of real attack paths; This embodiment quantifies and integrates information from three orthogonal dimensions—time, space, and tactical logic—to establish a correlation confidence assessment model that closely resembles real attack scenarios. In particular, by introducing a tactical transition matrix based on threat intelligence and the ATT&CK framework, prior knowledge in the field of cybersecurity is successfully integrated into the data-driven graph construction process. This enables the system not only to connect events that are close in time and space, but also to discover complex attack steps that conform to attack logic, spanning long periods of time and different entities, greatly improving the completeness and accuracy of attack chain reconstruction.

[0028] Example 7: The risk quantification module determines the systemic risk score, including: Determine the breadth of the attack chain; Calculate the average attack strength of the attack chain; Determine the peak potential impact of the attack chain; Determine the maturity of the attack chain pattern; The attack breadth, average attack strength, peak potential impact, and pattern maturity are multiplied to generate a systemic risk score. The pattern maturity is calculated through a pre-trained sequence model, which is trained on known attack sample sequences to identify high-risk combined tactical patterns.

[0029] This embodiment uses the risk quantification module described in Embodiment 1 to determine a systemic risk score. This is a systematic implementation method; its design concept draws on the multi-factor model in the field of financial engineering, aiming to avoid the one-sidedness of a single indicator and establish a decision model that can comprehensively evaluate the breadth, average strength, peak potential impact and pattern maturity of the attack chain. The risk quantification module, in determining the systemic risk score, includes the following steps: For an attack chain C containing N≥1 nodes, its systematic risk score RC is calculated using the following core model; if no valid attack chain exists, the risk score is 0: ; This model decomposes risk into the product of four orthogonal factors: Determine the breadth of attack in the attack chain: by Item representation; It is an attack chain. The number of nodes in the attack; this reflects the scope of the attack. The more nodes, the wider the attack range, but... log The function makes its growth non-linear, avoiding the overestimation of scores for long chains; Calculate the average attack strength of the attack chain: by Item representation; It is the first in the chain The micro-anomaly score of each node, i.e., the score calculated by the aforementioned module. , It is a dimensionless positional weight pre-defined based on the stage of a node in the attack chain; for example, the stages of the attack chain can be mapped to tactical categories in the MITREATT&CK framework, and different weights can be assigned to different tactics, such as assigning higher weights to nodes in the impact stage and lower weights to nodes in the reconnaissance stage, with the remaining stages having a default weight of 1.0. These weights are designed to reflect the direct harm level of different attack stages. By calculating a weighted average, this metric purely measures the average threat level of nodes in the chain and is correlated with the length of the chain. Decoupling; Determine the peak potential impact of the attack chain: by Item representation; It is the first The importance rating of the assets touched by each node is derived from CMDB; this is used to capture and highlight the highest value assets touched in the attack chain, ensuring that the risk assessment results fully reflect the most severe potential damage caused by the attack. Determine the pattern maturity of the attack chain: by Item representation; It is an attack chain. The complete tactical sequence; It is the matching probability of the tactical sequence with a predefined library of high-risk attack patterns; according to a further limitation of claim 9, the pattern maturity is calculated by a pre-trained sequence model; specifically, the sequence model is trained on a large number of known attack sample sequences to enable it to identify high-risk combined tactical patterns; when a new attack chain appears, its tactical sequence The input is fed into the model, and the model outputs a probability value. The higher the value, the closer the tactical combination of the attack chain is to known, mature, and highly dangerous attack methods; The system multiplies four independent risk factors—attack breadth, average attack strength, peak potential impact, and pattern maturity—to generate a systemic risk score. The product form design can effectively amplify the score of attack chains that simultaneously exhibit high-risk characteristics across multiple dimensions. This embodiment constructs a multi-dimensional, orthogonal systematic risk quantification model. Compared to simply accumulating the anomalies of each node, this scheme can provide a more comprehensive and profound insight into the inherent risk structure of the attack chain. By comprehensively evaluating the breadth, average intensity, peak impact, and pattern maturity of the attack, and especially by introducing a pre-trained sequence model to identify high-risk tactical combinations, the risk scoring becomes more comprehensive. Highly correlated with real business risks, it provides extremely accurate and interpretable prioritization criteria for security alerts, enabling security operations teams to focus their limited resources on the most destructive threats.

[0030] Example 8: The system also includes: The parameter adaptive module is used to dynamically adjust the weight coefficients in the anomaly detection module in response to true positive feedback from alarms. The parameter adaptation module is also used to train and optimize the weights in the graph construction module based on a known dataset of attack samples.

[0031] Based on the system described in Example 1, this embodiment further adds a parameter adaptive module. Its purpose is to introduce a feedback and learning mechanism so that the key parameters of the system can be dynamically optimized according to the actual operational results and new threat data, thereby improving the long-term detection and evaluation accuracy of the system. This parameter adaptive module contains two core functions: In response to true positive feedback from alarms, the weight coefficients in the anomaly detection module are dynamically adjusted. During system operation, security analysts analyze the alarms generated by the system and mark them as true positives or false positives. The parameter adaptation module collects this feedback data with expert labels. Based on this feedback, the system can adopt a gradient adjustment strategy: when a true positive feedback is received, it fine-tunes along the direction that increases the weight with the greatest contribution of the alarm; conversely, when a false positive feedback is received, it fine-tunes along the opposite direction. This adjustment mechanism enables the anomaly detection model to continuously iterate towards greater accuracy. Based on the known attack sample dataset, the weights in the graph construction module are trained and optimized; the parameter adaptation module periodically or when new known attack samples are available, re-executes the optimization of the weights in the graph construction module. The training process aims to maximize the association confidence between nodes belonging to the real attack path in these known attack samples, while minimizing the confidence between unrelated nodes. This ensures that the association model can continuously learn the latest attack techniques and association patterns. By introducing a parameter adaptive module, this invention is no longer a static, one-time configuration system, but a system with adaptive optimization capabilities. It can continuously optimize the key parameters of its internal model based on real-world feedback and new data, thereby ensuring that the system can maintain a high level of detection accuracy and risk assessment precision when facing constantly changing attack methods and network environments, significantly improving the long-term effectiveness and automated operation and maintenance level of the system.

[0032] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention; any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0033] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A communication data intelligent security monitoring system based on big data, characterized in that, include: The data acquisition module is used to extract standardized entity behavior feature vectors from multi-source heterogeneous communication data; The baseline modeling module is used to dynamically generate multi-dimensional behavioral baseline profiles based on the historical sequence of entity behavioral feature vectors. The anomaly detection module is used to compare real-time entity behavior feature vectors with multi-dimensional behavior baseline profiles to calculate micro-anomaly scores. The graph construction module is used to filter micro-anomaly events based on whether the micro-anomaly score exceeds a preset threshold, and to quantify the correlation confidence between the filtered events in order to construct an attack chain graph. The risk quantification module is used to aggregate features of the attack chain graph to determine a systemic risk score.

2. The intelligent security monitoring system for communication data based on big data according to claim 1, characterized in that, The baseline modeling module generates a multi-dimensional behavioral baseline profile, including: Collect entity behavior feature vectors within the initial observation period and calculate the mean vector; Set the mean vector as the initial behavioral baseline profile; An exponentially weighted moving average model is applied, and the baseline of the previous time step is iterated using newly collected entity behavior feature vectors to dynamically update the multi-dimensional behavior baseline profile.

3. The intelligent security monitoring system for communication data based on big data according to claim 1, characterized in that, The anomaly detection module calculates micro-anomaly scores, including: Calculate the statistical distance between the real-time behavioral feature vector and the dynamic behavioral baseline profile to obtain the basic deviation. A Gaussian mixture model is trained based on the timestamps of historical activities of entities to determine the temporal sparseness of the current behavior. Query the pre-defined asset management database to determine the asset importance of the communication source asset and the target asset; The baseline deviation, time rarity, and asset importance are weighted and summed to form a minor anomaly score.

4. The intelligent security monitoring system for communication data based on big data according to claim 3, characterized in that, The statistical distance is calculated using Mahalanobis distance, and the covariance matrix required to calculate the Mahalanobis distance is generated along with the multi-dimensional behavioral baseline profile generated by the baseline modeling module.

5. The intelligent security monitoring system for communication data based on big data according to claim 1, characterized in that, The map construction module filters micro-anomaly events, including: If the micro-anomaly score of a micro-anomaly event exceeds a preset threshold, then the micro-anomaly event is identified as a graph node for constructing the attack chain graph. If the score of a minor anomaly does not exceed the preset threshold, the minor anomaly event will not be identified as a graph node.

6. The intelligent security monitoring system for communication data based on big data according to claim 1, characterized in that, The graph construction module quantifies the association confidence level, including: The temporal correlation of candidate event pairs is determined based on the exponential decay function; Based on preset rules, the entity association of candidate event pairs is determined; Based on the pre-computed tactical transition matrix, the tactical correlation of candidate event pairs is determined; The temporal correlation, entity correlation, and tactical correlation are weighted and summed, and then mapped using the Sigmoid function to generate the correlation confidence.

7. The intelligent security monitoring system for communication data based on big data according to claim 6, characterized in that, The tactical transition matrix is ​​generated by statistical analysis of attack cases in a public threat intelligence database or by mapping tactical sequence relationships in the MITREATT&CK framework.

8. The intelligent security monitoring system for communication data based on big data according to claim 1, characterized in that, The risk quantification module determines the systemic risk score, including: Determine the breadth of the attack chain; Calculate the average attack strength of the attack chain; Determine the peak potential impact of the attack chain; Determine the maturity of the attack chain pattern; The attack breadth, average attack strength, peak potential impact, and pattern maturity are multiplied to generate a systemic risk score.

9. A communication data intelligent security monitoring system based on big data according to claim 8, characterized in that, The pattern maturity is calculated through a pre-trained sequence model, which is trained on known attack sample sequences to identify high-risk combined tactical patterns.

10. A communication data intelligent security monitoring system based on big data according to claim 1, characterized in that, Also includes: The parameter adaptive module is used to dynamically adjust the weight coefficients in the anomaly detection module in response to true positive feedback from alarms. The parameter adaptation module is also used to train and optimize the weights in the graph construction module based on a known dataset of attack samples.

Images