Network information security assessment method and system based on big data analysis

Abstract

The invention discloses a network information security assessment method and system based on big data analysis, and particularly relates to the technical field of data analysis. The method comprises the following steps: collecting multi-source heterogeneous security data, extracting entity multi-dimensional feature information, and constructing a graph data model containing semantic tags; for the entity confusion problem under the shared identifier, an identity recognition and node splitting strategy based on a probabilistic graph model is adopted, and refined recognition and graph structure reconstruction of entities are achieved; according to the method, path analysis is performed in combination with behavior semantic tags, potential attack chains and abnormal behaviors are effectively identified, a structured security assessment report is finally generated, risk level division is performed on entities and assets, and targeted response suggestions are provided, so that the accuracy of entity identification and the precision of attack path analysis are remarkably improved, the false alarm rate is reduced, and the security assessment efficiency is improved. And the intelligent level and the practicability of the network security evaluation system in a complex environment are enhanced.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically to a network information security assessment method and system based on big data analysis. Background Technology

[0002] Network information security assessment based on big data analytics refers to the collection, processing, and analysis of massive amounts of network data using big data technology to uncover potential security threats and risk factors, thereby conducting a comprehensive assessment of the network system's security status. This method enables early warning of attack behaviors, accurate identification of abnormal behaviors, and trend prediction of security incidents, thus improving overall network security protection capabilities and response efficiency.

[0003] The existing technology has the following shortcomings: In graph models, multiple entities (such as terminals, users, and IP addresses) may be indistinguishable, leading to different entities being incorrectly identified as the same object. This confusion can cause the system to mistakenly chain multiple unrelated events into a single attack path when analyzing attack behavior. For example, when multiple terminals access external services through the same public IP address, it may be misjudged as a single host frequently connecting to malicious domains, triggering a false alarm. This problem can lead to security systems improperly blocking innocent terminals, increasing the false alarm rate, obscuring the attack attribution process, and even masking the true attacker's behavior. Summary of the Invention

[0004] The purpose of this invention is to provide a network information security assessment method and system based on big data analysis to address the shortcomings of the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a network information security assessment method based on big data analysis, comprising: Collect network security-related data from multiple heterogeneous data sources, preprocess the data, extract entity feature information, and construct multi-dimensional entity identifiers; Based on the entity features, a graph data model containing multiple nodes and edges is constructed, where nodes represent entities and edges represent behavioral relationships between entities; For nodes with ambiguous entity identifiers, an entity disambiguation mechanism is used to identify and reconstruct entities in the graph model. The entity disambiguation mechanism includes identity recognition and node splitting strategies based on probabilistic graph models, which are used to distinguish behaviors that share the same identifier but belong to different entities. Introduce behavioral context semantic information into the graph model and assign semantic labels to edges to represent different types of behavioral intentions, access purposes, or operational characteristics. Based on the improved graph structure, graph mining and path analysis operations are performed to identify potential attack paths and associated threats. Based on the analysis results, a security assessment report is generated, which classifies different entities and assets into risk levels and provides targeted security response recommendations.

[0006] Preferably, the multiple heterogeneous data sources include: terminal log data, network traffic information, user behavior data, and external threat intelligence data; the terminal log data includes operating system logs, user login logs, security audit logs, and application logs; the multidimensional entity identifier includes feature information of network dimension, user dimension, device dimension, behavior dimension, and security dimension.

[0007] Preferably, the graph data model is a directed attribute graph, wherein the node types include user nodes, terminal nodes, IP address nodes and domain name nodes, and the edge types include communication relationships, access relationships, login relationships, association relationships and potential propagation relationships.

[0008] Preferably, in the entity disambiguation mechanism, after the initial construction of the graph structure, ambiguity detection is performed on each entity node, and candidate ambiguous nodes are identified. The historical behavior records of the candidate ambiguous nodes are then subjected to time-series segmentation and contextual clustering analysis, dividing the node's behavior log into multiple behavior segments according to time. Contextual features of each segment are extracted. If the behavior clusters show high differences after clustering, they are determined to contain multiple potential entities. Identity recognition based on the probabilistic graphical model includes using conditional random fields or Bayesian networks to infer and model the behavioral features of nodes, specifically: Use the current state of the node as the observation variable; Treat potential identity as a latent variable; Based on the relationship edges between a node and its neighbors, construct a probability transition matrix and calculate the posterior probability of each potential identity; If the posterior identity probability distribution of a node satisfies the preset splitting confidence condition, the node splitting strategy is triggered to split the ambiguous node into multiple logical child nodes and redirect the connection relationship of the edges.

[0009] Preferably, the path analysis operation includes extracting the semantic density features and abnormal propagation potential features of each candidate path, and fusing them to form a comprehensive feature vector to construct a multi-dimensional attack determination model.

[0010] Preferably, the semantic density feature extraction method is as follows: extract the contextual semantic content of each edge in the graph, convert the behavioral semantics of each edge into a fixed-length semantic vector, aggregate the semantic vectors of each edge in the path for the candidate path in the graph, calculate the semantic center representation of the path, calculate the semantic vector similarity between all edges in the path and take their average value, use 1 minus the average similarity to represent the semantic diversity of the path, count the number of semantic label types involved in the path and normalize it with the total number of semantic categories to obtain the semantic saturation value, and fuse the semantic diversity and semantic saturation in a weighted manner to obtain the comprehensive semantic density score.

[0011] The preferred method for extracting anomaly propagation potential features is as follows: identify the central nodes with high anomaly scores from the graph structure, denoted as... From the central node Starting from the beginning, traverse the graph structure outwards to construct a k-order adjacency subgraph. , including from Starting from the node and edge reachable in at most k steps: For each edge e on the propagation path, calculate its propagation weight we, and then... Each reachable target node The risk value and path propagation weight are summed to calculate the abnormal propagation potential value.

[0012] Preferably, the comprehensive semantic density score and the anomaly propagation potential value are converted into a comprehensive feature vector. The comprehensive feature vector is used as the input of the machine learning model. The machine learning model uses the prediction of the comprehensive threat score label of each candidate path for each set of comprehensive feature vectors as the prediction objective and the minimization of the sum of prediction errors for the comprehensive threat score labels of all candidate paths as the training objective. The machine learning model is trained until the sum of prediction errors converges and the model training stops. The comprehensive threat score of each candidate path is determined according to the model output. The machine learning model is a multinomial regression model. Based on the comprehensive threat score of each candidate path, the candidate paths are sorted in descending order from high to low, and the comprehensive threat score is compared with a pre-set risk threshold. Paths with a comprehensive threat score greater than or equal to the risk threshold are identified as potential attack paths.

[0013] Preferably, for each candidate path, its comprehensive threat score is obtained and denoted as... Traverse the entity nodes involved in the path, use the overall threat score of the path as a threat contribution factor, and map it to each node in the path. The expression for calculating the entity risk value is as follows: ;in: For entity nodes The risk value, Indicates whether the entity appears in the path middle; τ represents the risk distribution weight of the entity in the path, and τ is the set minimum effective threat score threshold. The risk value of each entity or asset is compared with a gradient risk threshold, which includes a first risk threshold and a second risk threshold, and the first risk threshold is less than the second risk threshold. The risk value of each entity or asset is compared with the first risk threshold and the second risk threshold respectively. If the risk value of each entity or asset is greater than the second risk threshold, the entity or asset is determined to be at a high risk level, and strong response measures such as blocking and isolation are immediately taken; if the risk value of each entity or asset is greater than or equal to the first risk threshold and less than or equal to the second risk threshold, it is determined to be at a medium risk level; monitoring is strengthened, alarms are triggered, and sensitive permissions are restricted; if the risk value of each entity or asset is less than the first risk threshold, it is determined to be at a low risk level.

[0014] This invention also provides a network information security assessment system based on big data analysis, including a data acquisition and preprocessing module, a graph model construction module, a graph structure optimization module, a label setting module, a risk assessment module, and a response decision module; Data acquisition and preprocessing module: Acquires network security-related data from multiple heterogeneous data sources, preprocesses the data, extracts entity feature information, and constructs multi-dimensional entity identifiers; Graph model construction module: Constructs a graph data model containing multiple nodes and edges based on the entity features, where nodes represent entities and edges represent behavioral relationships between entities; Graph structure optimization module: For nodes with ambiguous entity identifiers, an entity disambiguation mechanism is used to identify and reconstruct entities in the graph model. The entity disambiguation mechanism includes identity recognition and node splitting strategies based on probabilistic graph models, which are used to distinguish behaviors that share the same identifier but belong to different entities. Label setting module: Introduces behavioral context semantic information into the graph model, assigns semantic labels to edges to represent different types of behavioral intentions, access purposes or operational characteristics; Risk assessment module: Based on the improved graph structure, it performs graph mining and path analysis operations to identify potential attack paths and related threats; Response Decision Module: Generates security assessment reports based on analysis results, classifies different entities and assets into risk levels, and provides targeted security response recommendations.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention effectively solves the entity confusion problem caused by IP reuse and account sharing by introducing an identity recognition and node splitting strategy based on a probabilistic graphical model, thereby improving the accuracy of entity recognition and the precision of attack chain reconstruction. Simultaneously, by constructing multi-dimensional entity identifiers, semantically enhanced graph models, semantic density analysis, and anomaly propagation potential calculation, it achieves deep modeling of complex network behavior, providing structured support for attack path identification and behavioral intent determination.

[0016] 2. This invention enhances the ability to identify multi-stage, covert attacks by integrating semantic density features and propagation potential features to construct a comprehensive feature vector and introducing a multinomial regression model to intelligently score potential attack paths. The system can map the scoring results to entities and assets, use a gradient risk threshold mechanism to classify risk levels, and generate structured assessment reports and response suggestions, forming a data-driven intelligent assessment and decision-making closed loop. Overall, this invention improves the intelligence, refinement, and automation of cybersecurity assessment, and is suitable for security threat perception, early warning, and response in complex and heterogeneous environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a mind map of the method of the present invention.

[0019] Figure 2 This is a system module diagram of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, please refer to Figure 1 As shown in this embodiment, the network information security assessment method based on big data analysis includes: Collect network security-related data from multiple heterogeneous data sources, preprocess the data, extract entity feature information, and construct multi-dimensional entity identifiers; Based on the entity features, a graph data model containing multiple nodes and edges is constructed, where nodes represent entities and edges represent behavioral relationships between entities; For nodes with ambiguous entity identifiers, an entity disambiguation mechanism is used to identify and reconstruct entities in the graph model. The entity disambiguation mechanism includes identity recognition and node splitting strategies based on probabilistic graph models, which are used to distinguish behaviors that share the same identifier but belong to different entities. Introduce behavioral context semantic information into the graph model and assign semantic labels to edges to represent different types of behavioral intentions, access purposes, or operational characteristics. Based on the improved graph structure, graph mining and path analysis operations are performed to identify potential attack paths and associated threats. Based on the analysis results, a security assessment report is generated, which classifies different entities and assets into risk levels and provides targeted security response recommendations.

[0022] Collect cybersecurity-related data from multiple heterogeneous data sources: Terminal log data sources include: operating system logs, user login logs, security audit logs, application software logs, etc.; example fields include: user ID, login time, login IP, operation type, error code, process ID, etc. Network traffic information sources: network devices such as switches, routers, traffic probes, and SDN controllers; example fields: source IP, destination IP, port, protocol, traffic size, connection duration, number of packets, etc. User behavior data (UEBA) sources: endpoint detection systems, identity authentication systems (such as Active Directory domain controllers), VPN systems, bastion hosts, etc.; example fields: user behavior sequence, access frequency, device switching records, abnormal behavior scores, etc. External threat intelligence (TI) data sources: open-source intelligence platforms (such as VirusTotal, AbuseIPDB), commercial threat intelligence providers, security communities; example fields: malicious IP / domain blacklists, attacker profiles, APT attack chain characteristics, attack techniques (TTPs), file hashes, etc.

[0023] Data collection methods include: Active data collection: Obtain real-time data directly through API interfaces, log subscriptions, and log collection proxies (such as Filebeat).

[0024] Passive data collection: Data packets are acquired and features are extracted through methods such as mirror port monitoring, network packet capture, and traffic replay.

[0025] Scheduling and synchronization mechanism: Data collection is scheduled based on timestamps and data source priorities, and supports breakpoint resume and deduplication of duplicate data.

[0026] The data is preprocessed to extract entity feature information and construct multidimensional entity identifiers: Data cleaning: removing redundant fields and null values; format standardization, such as unifying timestamps to UTC and normalizing IP address formats; outlier detection and correction, such as marking illegal ports and forged fields.

[0027] Data association: Based on timelines, network connection relationships, and user login behavior, multi-source logs are aligned at the event level; for example, user identifiers are aligned between VPN login records and terminal behavior logs to build a unified user view.

[0028] Feature extraction and normalization: Extract structured fields and construct standardized entity attributes: Network attributes: IP address, MAC address, port, protocol type, etc.; User attributes: username, department ID, permission level, device used; Device attributes: device type, operating system, browser version, patch level; Behavioral attributes: access frequency, path sequence, access depth, preferred services, etc.; Perform word vector encoding on text data (such as behavior type, operation command) to support subsequent semantic analysis.

[0029] Multidimensional entity identification (MDI) is a method that integrates entity feature information from multiple dimensions to finely differentiate and identify terminals, users, and devices in a network, thereby improving the accuracy and completeness of network security analysis. This identification system mainly includes the following dimensions: First, at the network dimension, basic communication features such as IP address, MAC address, and port number are extracted to identify the basic identity information of different communication entities in the network. These features constitute the main identity identifiers of entities at the network layer and are the foundation for subsequent behavior tracking and communication relationship modeling.

[0030] Secondly, at the user level, information such as username, user role, and login credentials is used to characterize a user's identity attributes and scope of permissions on the network. This dimension helps distinguish the operational behaviors of different identities and identify potential risky behaviors such as unauthorized access and abnormal logins.

[0031] At the device level, information such as device fingerprints, terminal serial numbers, and operating system versions is collected to identify different types of physical or virtual devices. Even if multiple devices share the same IP address or login account, this dimension can still achieve device-level differentiation, avoiding confusion between different terminal entities.

[0032] The behavioral dimension, starting from dynamic behavioral characteristics, establishes a behavioral profile of a user or device by analyzing the entity's operation patterns, resource access paths, and behavioral deviation scores over time. This dimension is particularly suitable for identifying the behavior of multiple users under the same IP address or account, effectively improving the accuracy of identification in proxy and NAT environments.

[0033] Finally, in terms of security, by combining tagging information from threat intelligence and historical data, we record whether an entity exhibits known malicious behavior, its current risk score, and the security incidents it has participated in. This dimension provides a foundation for subsequent threat identification and attack chain analysis, helping to quickly identify high-risk entities.

[0034] Multidimensional fusion method: Use hash combination keys, clustering algorithms, or graph embedding techniques to map multi-dimensional features into a single entity identifier vector; Construct an "entity identity similarity model" to determine whether the current entity and historical entities are the same object or have an evolutionary relationship; Supports "split" and "merge" operations for entities, and updates their unique identifiers in real time based on behavioral evolution.

[0035] To comprehensively present the relationships and behavioral interactions between entities in the network and enhance the ability to identify, track, and predict cybersecurity threats, the system constructs a graph data model based on preprocessed and feature-extracted multidimensional entity information. This model organizes data in a graph structure, clearly describing the complex behavioral chains and potential attack paths between entities.

[0036] The constructed graph data model is a directed graph or attribute graph, denoted as G = (V, E), where: V is a set of nodes, representing entities in the network with independent identities or behavioral characteristics; E is the set of edges, representing the specific interactive behaviors between different entities; Each node and edge can be attached with attributes to store additional data such as behavioral information, risk tags, and timestamps.

[0037] Based on the entity types modeled by multidimensional features, the nodes in the graph can be subdivided into the following categories, each representing a logical identity: User node (User): represents an individual user with login and access permissions; attribute examples: username, role, organizational unit, behavioral profile, risk level.

[0038] Endpoint / Device: Represents a specific physical or virtual device, including servers, workstations, mobile terminals, etc.; Attribute examples: MAC address, device fingerprint, operating system type, device ID, patch status.

[0039] IP Address Node: Represents the source IP or destination IP entity in network communication; Attribute examples: public network / private network, whether blacklisted, recent communication time, network segment.

[0040] Domain node: Represents the internet domain name or URL being accessed; Attribute examples: resolved IP, domain reputation score, whether it is listed on a threat intelligence blacklist.

[0041] Edges represent a behavioral, communication, or interactive relationship between two nodes and are central to attack path modeling and behavior analysis. The main relationship types include: Communicates With: Describes the network communication behavior between a terminal / IP and another IP or domain name; Attribute examples: communication protocol (TCP / UDP), port number, number of communications, average duration, first / last timestamp.

[0042] Accesses: Describes a user's access behavior to resources or files; Example attributes: operation type (read, modify, upload), whether access is unauthorized, and trigger time.

[0043] Login Relationship (Logs Into): Describes how a user logs into a terminal or system; Attribute examples: Login method (local, VPN, remote desktop), login success / failure, number of exceptions.

[0044] Associated With: Describes the static or historical relationship between two entities, such as the binding of an IP address to a terminal, or the long-term usage relationship between a user and a device; Attribute examples: binding duration, last verification time, trust level.

[0045] Threat Spread To (used for threat path prediction): Describes a potential attack chain, such as how an infected endpoint might spread a threat to another endpoint; Attribute examples: propagation probability, historical infection paths, and estimated scope of impact.

[0046] Entity node initialization: Identify the unique identifier of each type of entity from the cleaned and standardized multidimensional entity feature data; avoid duplicate modeling, such as two user login events pointing to the same user node.

[0047] Behavior edge generation: Based on the log timeline, extract semantically consistent behavior pairs (such as "User A logs into terminal X"); construct directed edges to connect user nodes and terminal nodes, and attach attribute information of the behavior.

[0048] Edge attribute enhancement: Add contextual information to each edge: behavior type, risk label, behavior confidence, and external intelligence cross-validation results; support subsequent semantic behavior analysis and behavior scoring.

[0049] Graph structure optimization: Remove redundant edges and isolated nodes, and merge duplicate relationships; optional graph compression and partitioning strategies can be used to improve graph query efficiency in large-scale environments.

[0050] After the initial graph structure is constructed, the system performs the following ambiguity detection operation on each entity node, and nodes that meet any of the following conditions are marked as ambiguity candidate nodes: The same IP node connects to multiple user nodes with significantly different user behaviors; the login behavior of a certain terminal node overlaps in time or has geographical anomalies; a certain user node initiates operations through multiple devices / IPs in a short period of time; the node's behavioral statistics (such as behavioral vectors) exhibit a multimodal distribution.

[0051] Time series segmentation and contextual clustering analysis are performed on the historical behavior records of candidate ambiguous nodes, using algorithms such as DBSCAN, HDBSCAN, and GMM. Divide the node's behavior log into multiple behavior segments by time; extract the context features of each segment (access frequency, operation category, communication target, etc.); if the behavior clusters are highly different (i.e., the behavior styles are completely different) after clustering, they are determined to contain multiple potential entities.

[0052] To accurately distinguish between multiple entities that a single node may represent, the system introduces a probabilistic graphical model to model and infer node identities. Commonly used models include Conditional Random Fields (CRFs) or Bayesian Networks, specifically: Use the node's current state (such as communication behavior, login history, terminal fingerprint) as the observation variable; Treat the potential identity (logical entity identifier) ​​as a latent variable; Based on the relationship edges between a node and its neighbors, construct a probability transition matrix and calculate the posterior probability of each potential identity.

[0053] For example, if node N is determined to be either entity A or entity B based on its behavioral characteristics, then P(entity A | behavioral sequence, neighbor node relationship) and P(entity B | same conditions) are calculated respectively. If the probability difference between the two is significant (exceeding the preset confidence threshold), then the identity splitting and reconstruction process is triggered.

[0054] After identifying ambiguous nodes, each ambiguous node is split into multiple logical entity nodes, making subsequent graph mining and attack path analysis more accurate. Specifically: The original ambiguous node N is split into Each child node represents a logical entity that is determined to be independent; each child node inherits a specific set of behavioral edges, which are allocated according to time period or behavior type.

[0055] For all incoming and outgoing edges of the original node, reassign them to their corresponding child nodes; ensuring the temporal and semantic continuity of the edges. For example: the login action A → N → resource X is split into A → → Resource X (if this behavior belongs to) (The entity being identified). Update the graph database structure to record the node splitting history; introduce splitting source pointers for reversible tracing; and support avoiding logically obfuscated paths during subsequent graph traversal.

[0056] For example, suppose an IP node IP_192.168.1.100 is associated with multiple communication behaviors: some behaviors are typical office access, with domains belonging to the company's intranet; others include frequent visits to phishing websites, logins from different locations, and communication outside of working hours. Through contextual clustering and probabilistic identity recognition, the system determines that it may be shared by two devices. After performing node splitting, IP_100_A and IP_100_B are generated. The former retains the normal communication path, while the latter is included in the potential attack chain, significantly improving detection accuracy.

[0057] In traditional graph models, edges only indicate whether a certain relationship exists between entities (such as communication, login, or access), but cannot characterize the specific meaning and security significance of the behavior. To enhance the expressive power of graph models, this invention introduces behavioral contextual semantic information into the graph structure and assigns semantic labels to the edges to represent the intent, purpose, sensitivity, or abnormal attributes of the behavior.

[0058] By assigning semantic labels to edges, the following functions can be achieved: It can distinguish between different types of behavioral relationships, such as normal access and abnormal access; reveal the intentions or patterns behind the behavior, such as data collection, lateral movement, privilege escalation, etc.; improve the accuracy of graph mining and attack path identification, and avoid misjudgment or path confusion caused by semantic ambiguity.

[0059] The system defines various semantic tag types, which can be discrete enumeration values ​​or attribute vectors, mainly including: Behavior type label: Indicates the type of operation behavior represented by this edge, such as "access", "communication", "login", "execute", "download", "upload", etc.

[0060] Behavioral intent labels: These indicate the possible intent or purpose of a behavior and are often generated by combining behavioral context with behavioral pattern recognition algorithms, such as "Data Collection", "Reconnaissance", "Lateral Movement", "Privilege Escalation", and "Command and Control (C2)".

[0061] Behavior Traits: Represent some characteristics of behavior at the operational level, such as: Normal / Anomalous, automated / manual triggering, high-frequency / low-frequency access, and time-period sensitivity (working hours / non-working hours); Risk Score: The system scores the behavior based on the behavioral context, with the label being high, medium, low risk or a risk value of 0~1; used for assigning edge weights or weighted scoring of attack paths.

[0062] Build a behavioral pattern library (such as MITRE ATT&CK terminology) and match the operations in the logs with existing behavioral intent templates; Example rule: If there is a sequence of "login → port scanning → large amount of communication" in a short period of time, mark it as a "lateral movement" intent.

[0063] Natural Language Processing (NLP) and Behavior Summary Extraction: Perform text analysis on operation commands, access paths, log texts, etc.; extract keywords and intent phrases, and generate semantic tags.

[0064] Machine learning models generate labels: Training classification models (such as LSTM and Transformer) takes behavioral sequences as input and outputs corresponding semantic categories; it can predict labels for "unknown attack patterns".

[0065] Enhanced external threat intelligence: If the domains, IPs, files, etc. involved in the behavior have historical tags in the intelligence database (such as phishing, backdoor control), the relevant semantics will be automatically assigned.

[0066] In graph databases (such as Neo4j) or graph computing engines (such as TigerGraph), each edge not only records the source node and the target node, but also comes with a set of attribute fields as labels: These labels can be used in attack path weighting calculation (the more paths there are, the higher the risk of the edge → the more suspicious), abnormal subgraph detection (subgraphs that aggregate high-risk semantic edges), and visualization graph display (different semantic edges are distinguished by color).

[0067] The graph structure with semantic tags has higher interpretability and intelligence in security analysis, specifically manifested in the following ways: More accurate attack chain reconstruction: When traversing the graph, the system prioritizes selecting edges labeled with "attack intent" or "abnormal behavior" for path inference; reducing the risk of misjudging non-malicious behavior being included in the attack chain.

[0068] Path risk scoring and early warning mechanism: The attack probability of a path can be comprehensively assessed by weighting the "risk level" and "behavioral intent" of the edges in the path; and early warning can be given for paths that have not yet caused damage but have high attack characteristics.

[0069] Context-aware analysis: Supports filtering paths in the graph by "behavioral purpose", such as only viewing "privilege escalation related behavior chains"; assists manual personnel in quickly locating core attack segments and key behavioral nodes.

[0070] In the constructed and optimized semantic graph structure, the system identifies potential attack chains and high-risk paths through graph mining algorithms and performs threat assessment by integrating graph features from multiple dimensions. By constructing a multi-dimensional attack determination model, the system enhances its ability to identify complex or covert attacks, specifically: Use improved weighted path traversal algorithms (such as weighted depth-first search, constrained shortest path algorithm); Settings: Starting node (e.g., marked malicious IP, known victim host); path length threshold; node / edge semantic constraints (e.g., only consider high-risk communication); generate candidate path set.

[0071] Features are extracted from nodes and edges on each path, and path aggregation indices are calculated. A sliding window is used to analyze whether the time series paths conform to the attack chain stage evolution logic (such as scanning → penetration → lateral movement → seepage).

[0072] The extracted features include semantic density features and anomaly propagation potential features. The semantic density features are extracted by drawing contextual semantic content from each edge in the graph (i.e., the connection representing user behavior, communication, or access relationships). This semantic information can come from fields such as operation commands, access URLs, and behavior descriptions in the original logs. Through natural language preprocessing, redundant words are cleaned, behavioral keywords are retained, and a standardized text representation is formed.

[0073] The extracted edge semantic text is input into a pre-trained language model (such as Sentence-BERT or RoBERTa), which transforms the behavioral semantics of each edge into a fixed-length semantic vector. This vector captures the deep semantic features of the behavior, providing a computational foundation for subsequent diversity and density analysis.

[0074] For a candidate path in the graph, the semantic vectors of each edge in the path are aggregated to calculate the semantic center representation of the path. The aggregation method uses an average weighting approach, which can assign different weights to different edges based on the importance of the behavior. This step is used to construct a path-level semantic representation, supporting subsequent risk scoring or similar path matching.

[0075] Furthermore, to assess the semantic differences among behaviors within a path, the system calculates the semantic vector similarity between all edges in the path and takes their average value. Then, 1 minus the average similarity is used to represent the semantic diversity of the path. When the behavior types in the path differ significantly and the intentions are varied, this diversity value will increase significantly, indicating that the semantic structure of the path is complex.

[0076] In addition, to determine the breadth of path semantic types, the number of semantic tag types involved in the path (such as "login," "file access," "data upload," etc.) is counted and normalized with the total number of semantic categories defined by the system to obtain a semantic saturation value. This indicator reflects whether the path covers multiple stages of attack behavior and is a key parameter for measuring the semantic coverage capability of the path.

[0077] The semantic diversity and semantic saturation are weighted and fused to obtain the comprehensive semantic density score.

[0078] The method for extracting anomaly propagation potential features is as follows: identify one or more central nodes with high anomaly scores from the graph structure, denoted as... These nodes can be obtained through historical intrusion detection results, abnormal behavior scores, and entity risk scores (such as user / terminal).

[0079] From the central node Starting from the beginning, traverse the graph structure outwards to construct a k-order adjacency subgraph. , including from The nodes and edges that can be reached in at most k steps from the starting point: For each edge e on the propagation path, calculate its propagation weight we, and then... Each reachable target node The risk value and path propagation weight are summed to calculate the abnormal propagation potential value. The expression is: ;in: For nodes Current risk score (such as anomaly score, historical attack labels); Indicates from arrive The cumulative propagation weight is defined as the product of the propagation weights of all edges.

[0080] The comprehensive semantic density score and anomaly propagation potential value are converted into a comprehensive feature vector. The comprehensive feature vector is used as the input to the machine learning model. The machine learning model uses the prediction of the comprehensive threat score label of each candidate path for each set of comprehensive feature vectors as the prediction objective and minimizes the sum of prediction errors for the comprehensive threat score labels of all candidate paths as the training objective. The machine learning model is trained until the sum of prediction errors converges and the model training stops. The comprehensive threat score of each candidate path is determined based on the model output. The machine learning model is a multinomial regression model.

[0081] Based on the comprehensive threat score of each candidate path, the candidate paths are sorted in descending order from high to low, and the comprehensive threat score is compared with a pre-set risk threshold. Paths with a comprehensive threat score greater than or equal to the risk threshold are identified as potential attack paths.

[0082] For each candidate path, the system has obtained its comprehensive threat score, denoted as . The system traverses all entity nodes (including users, terminals, IPs, domains, etc.) involved in the path, using the path score as a threat contribution factor and mapping it to each node within the path. The entity risk value calculation expression is: ;in: For entity nodes The risk value, Indicates whether the entity appears in the path middle; τ represents the risk distribution weight of an entity in the path (e.g., the weight of a relay node is greater than that of an edge node); τ is the set minimum effective threat score threshold.

[0083] Risk value of each entity or asset Its risk level is determined by comparing it with a gradient risk threshold: The risk value of each entity or asset is compared with a gradient risk threshold, which includes a first risk threshold and a second risk threshold, and the first risk threshold is less than the second risk threshold. The risk value of each entity or asset is compared with the first risk threshold and the second risk threshold respectively. If the risk value of each entity or asset is greater than the second risk threshold, the entity or asset is determined to be of high risk level; this indicates that it plays a key role in the attack path, or is associated with multiple high-scoring paths, and its behavior is significantly abnormal; it is recommended to take strong response measures such as blocking and isolation immediately. If the risk value of each entity or asset is greater than or equal to the first risk threshold and less than or equal to the second risk threshold, it is judged as a medium risk level; this indicates that the entity’s behavior has an abnormal trend or is indirectly exposed to the attack chain, and the risk is medium; it is recommended to strengthen monitoring, trigger alarms, and restrict sensitive permissions. If the risk value of each entity or asset is less than the first risk threshold, it is classified as a low-risk level. This indicates that the entity's behavior is generally normal or occasionally abnormal, and there is no direct evidence that it is related to a threat; the status quo can be maintained, only the status can be recorded, or it can be set as a low-priority observation object.

[0084] Risk levels are indicated by colors (such as red / orange / yellow / green) or labels, and are presented visually in the assessment report.

[0085] The system generates a structured security posture assessment report, which includes: Candidate Path Scoring Table: Displays the overall threat score, path ID, starting and target nodes, path length and feature summary of all paths; sorted in descending order of score, highlighting the paths most likely to be the real attack chain.

[0086] High-risk entity list: Based on the distribution of score values, it lists the entity nodes with the highest score contribution in the current environment; it includes entity type, risk level, number of involved paths, and risk distribution map.

[0087] Risk asset impact analysis maps path target nodes and entity affiliations to specific business assets (such as databases, servers, and cloud services); and outputs risk scores and priority reinforcement recommendations for affected assets.

[0088] Based on the comprehensive threat score and risk level, the system generates targeted response recommendations, including: ≥0.85 and classified as high-risk, immediately isolate key nodes within the path (e.g., block IP addresses, disconnect from the network, lock terminals); 0.6≤ If the value is less than 0.85 and the risk level is medium, then enhanced monitoring, triggering of security audits, and restriction of account permissions are required. If the value is less than 0.6 and the risk level is low, no action is required or a low-priority strategy should be followed.

[0089] Example 2, please refer to Figure 2 As shown in the figure, the network information security assessment system based on big data analysis described in this embodiment includes a data acquisition and preprocessing module, a graph model construction module, a graph structure optimization module, a label setting module, a risk assessment module, and a response decision module. Data acquisition and preprocessing module: Acquires network security-related data from multiple heterogeneous data sources, preprocesses the data, extracts entity feature information, and constructs multi-dimensional entity identifiers; Graph model construction module: Constructs a graph data model containing multiple nodes and edges based on the entity features, where nodes represent entities and edges represent behavioral relationships between entities; Graph structure optimization module: For nodes with ambiguous entity identifiers, an entity disambiguation mechanism is used to identify and reconstruct entities in the graph model. The entity disambiguation mechanism includes identity recognition and node splitting strategies based on probabilistic graph models, which are used to distinguish behaviors that share the same identifier but belong to different entities. Label setting module: Introduces behavioral context semantic information into the graph model, assigns semantic labels to edges to represent different types of behavioral intentions, access purposes or operational characteristics; Risk assessment module: Based on the improved graph structure, it performs graph mining and path analysis operations to identify potential attack paths and related threats; Response Decision Module: Generates security assessment reports based on analysis results, classifies different entities and assets into risk levels, and provides targeted security response recommendations.

[0090] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A network information security evaluation method based on big data analysis, characterized in that: The method comprises the following steps: Collecting network security related data from multiple heterogeneous data sources, preprocessing the data, extracting entity feature information, and constructing a multi-dimensional entity identifier; Based on the entity features, a graph data model containing multiple nodes and edges is constructed, where the nodes represent entities and the edges represent the behavior relationships between entities; For nodes with entity identifier ambiguity, an entity disambiguation mechanism is used to identify and reconstruct the entities in the graph model, which includes an identity recognition and node splitting strategy based on a probabilistic graph model, used to distinguish behaviors that share the same identifier but belong to different entities; Behavior context semantic information is introduced into the graph model to assign semantic labels to the edges, representing different types of behavior intent, access purpose, or operation characteristics; Based on the improved graph structure, graph mining and path analysis operations are performed to identify potential attack paths and associated threats; According to the analysis results, a security assessment report is generated, the risk level of different entities and assets is divided, and specific security response suggestions are provided. 2.The network information security assessment method based on big data analysis according to claim 1, characterized in that: The multiple heterogeneous data sources include terminal log data, network traffic information, user behavior data, and external threat intelligence data; the terminal log data includes operating system logs, user login logs, security audit logs, and application program logs; the multi-dimensional entity identifier includes feature information in the network dimension, user dimension, device dimension, behavior dimension, and security dimension. 3.The network information security assessment method based on big data analysis of claim 1, characterized in that: The graph data model is a directed attributed graph, where the node types include user nodes, terminal nodes, IP address nodes, and domain name nodes, and the edge types include communication relationships, access relationships, login relationships, association relationships, and potential propagation relationships. 4.The network information security assessment method based on big data analysis of claim 1, wherein: In the entity disambiguation mechanism, after the initial construction of the graph structure, ambiguity detection is performed on each entity node, and candidate ambiguous nodes are determined. The historical behavior records of the candidate ambiguous nodes are time series segmented and context clustered, and the behavior logs of the nodes are divided into multiple behavior segments according to time; the context features of each segment are extracted; If the difference between the behavior clusters after clustering is high, it is determined that multiple potential entities are included; the identity recognition based on the probabilistic graph model includes using conditional random fields or Bayesian networks to model the behavior features of the nodes, specifically: The current state of the node is used as the observation variable; The potential identity is used as the hidden variable; According to the relationship edges between the node and its neighbors, a probability transition matrix is constructed to calculate the posterior probability of each potential identity; If the posterior probability distribution of the node meets the preset splitting confidence condition, the node splitting strategy is triggered to split the ambiguous node into multiple logical sub-nodes, and the connection relationship of the edges is redirected.

5. The network information security assessment method based on big data analysis according to claim 4, characterized in that: The path analysis operation includes extracting the semantic density features and abnormal propagation potential features of each candidate path, and fusing them into a comprehensive feature vector to construct a multi-dimensional attack judgment model.

6. The network information security assessment method based on big data analysis according to claim 5, characterized in that: The extraction method of the semantic density feature is: extracting the context semantic content in each edge of the graph, converting the behavior semantics of each edge into a fixed-length semantic vector, aggregating the semantic vectors of the edges in the candidate path of the graph, calculating the semantic center representation of the path, calculating the semantic vector similarity between all edges in the path, and taking the average value, using 1 minus the average similarity to represent the semantic diversity of the path, counting the number of semantic label categories involved in the path, and normalizing it with the total number of semantic categories to obtain the semantic saturation value, and the semantic diversity and the semantic saturation are fused in a weighted manner to obtain the comprehensive semantic density score.

7. The network information security assessment method based on big data analysis according to claim 6, characterized in that: The extraction method of the anomaly propagation potential feature is: a center node with a high anomaly score is identified from the graph structure, denoted as , starting from the center node , the k-order adjacent subgraph is constructed by traversing the graph structure outward , containing nodes and edges reachable from in at most k steps: for each edge e on the propagation path, the propagation weight we is calculated, and the risk value of each target node reachable from is accumulated with the path propagation weight to calculate the anomaly propagation potential value. 8.The network information security assessment method based on big data analysis of claim 7, wherein: The comprehensive semantic density score and the abnormal propagation potential value are converted into a comprehensive feature vector, the comprehensive feature vector is used as the input of a machine learning model, the machine learning model takes each set of comprehensive feature vectors as the prediction target to predict the comprehensive threat score value label of each candidate path, minimizes the sum of prediction errors of all comprehensive threat score value labels of each candidate path as the training target, trains the machine learning model until the sum of prediction errors converges, and stops model training, and determines the comprehensive threat score value of each candidate path according to the model output result, wherein the machine learning model is a polynomial regression model; According to the obtained comprehensive threat score value of each candidate path, the candidate paths are sorted in descending order from high to low, and the comprehensive threat score value is compared with the pre-set risk threshold value, and the path with a comprehensive threat score value greater than or equal to the risk threshold value is taken as a potential attack path. 9.The network information security assessment method based on big data analysis of claim 8, characterized in that: For each candidate path, its comprehensive threat score value is obtained, denoted as ; the entity nodes involved in the path are traversed, and the path comprehensive threat score value is mapped to each node in the path as a threat contribution factor, and the entity risk value calculation expression is: ; wherein: is the risk value of the entity node , indicates whether the entity appears in the path ; is the risk distribution weight of the entity in the path, and τ is the set minimum threat effective score threshold. The risk value of each entity or asset obtained is compared with the gradient risk threshold value, the gradient risk threshold value includes a first risk threshold value and a second risk threshold value, and the first risk threshold value is less than the second risk threshold value, and the risk value of each entity or asset is compared with the first risk threshold value and the second risk threshold value respectively; If the risk value of each entity or asset is greater than the second risk threshold value, it is determined that the entity or asset is of high risk level, and strong response measures such as ban and isolation are immediately taken; if the risk value of each entity or asset is greater than or equal to the first risk threshold value and less than or equal to the second risk threshold value, it is determined to be of medium risk level; the monitoring is strengthened, the alarm is triggered, and the sensitive permission is limited; if the risk value of each entity or asset is less than the first risk threshold value, it is determined to be of low risk level.

10. A network information security assessment system based on big data analysis, for implementing the network information security assessment method based on big data analysis according to any one of claims 1-9, characterized in that: The system comprises a data acquisition and preprocessing module, a graph model construction module, a graph structure optimization module, a label setting module, a risk assessment module, and a response decision module. The data acquisition and preprocessing module acquires network security related data from multiple heterogeneous data sources, pre-processes the data, extracts entity feature information, and constructs a multi-dimensional entity identifier; The graph model construction module constructs a graph data model comprising multiple nodes and edges based on the entity features, wherein the nodes represent entities and the edges represent the behavior relationship between entities; The graph structure optimization module optimizes the graph data model to obtain an optimized graph data model; A graph structure optimization module: for nodes with entity identification ambiguity, an entity disambiguation mechanism is used to identify and reconstruct entities in the graph model, which includes an identity recognition based on a probabilistic graph model and a node splitting strategy to distinguish behaviors that share the same identifier but belong to different entities; A label setting module: introduces behavior context semantic information in the graph model, and gives semantic labels to the edges to represent different types of behavior intentions, access purposes or operation characteristics; A risk assessment module: based on the improved graph structure, graph mining and path analysis operations are performed to identify potential attack paths and associated threats; A response decision module: generates a security assessment report based on the analysis results, classifies the risk levels of different entities and assets, and provides targeted security response suggestions.

Images