Multi-party joint complex APT attack detection method supporting privacy protection

By using privacy computing and graph deep learning algorithms, a multi-party collaborative platform was built, which solved the problems of accuracy and privacy protection in the detection of complex APT attacks, and achieved efficient and secure APT attack detection and data analysis.

CN119030775AInactive Publication Date: 2024-11-26CHINESE PEOPLES LIBERATION ARMY UNIT 61660

Patent Information

Application Number
CN202411187575.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies have shortcomings in dealing with complex APT attacks, including insufficient detection accuracy, inadequate privacy protection in multi-party data sharing and analysis, and insufficient resource utilization. In particular, they are prone to leakage of sensitive information in multi-party joint intelligence sharing.

Method used

By combining privacy-preserving computation and graph deep learning algorithms, a multi-party collaborative platform is constructed through data anonymization, homomorphic encryption, and secure multi-party computation. This platform performs data preprocessing and encrypted analysis, and utilizes graph attention networks and variational autoencoders to identify anomalous behavior.

Benefits of technology

It significantly improves the accuracy and efficiency of APT attack detection, protects data privacy, optimizes resource utilization, reduces costs, enhances system flexibility and scalability, supports real-time data analysis, and strengthens network security defense capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030775A_ABST
    Figure CN119030775A_ABST
Patent Text Reader

Abstract

The invention provides a multi-party joint complex APT attack detection method supporting privacy protection. The method comprises the steps of data collection and preprocessing, application of a privacy protection technology, construction of a multi-party joint cooperation platform, multi-party data analysis and APT detection. Compared with the prior art, by integrating the advanced privacy protection technology, the graph deep learning algorithm and the multi-party joint cooperation mechanism, the APT attack detection accuracy and efficiency are remarkably improved, and meanwhile a significant progress is made in the aspect of data privacy protection. According to the method, resource utilization is optimized, the cost of a single organization is reduced, and the flexibility and expandability of the system are improved. Besides, real-time data analysis is supported, the response speed of network security threats is greatly improved, potential loss and influence are effectively reduced, and a more comprehensive and efficient solution is provided for network security defense.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of privacy computing technology, and in particular to multi-party secure computing and complex APT attack detection, and proposes a multi-party joint complex APT attack detection method that supports privacy protection. Background Art

[0002] With the development of network technology and the popularization of its application, network security issues are becoming increasingly prominent. In particular, enterprises, government agencies and other large organizations are facing more and more network threats. APT (Advanced Persistent Threat) attacks are a complex form of network attacks, which are usually launched by senior hackers with clear goals, hidden attack methods and long duration. Such attacks are often difficult to effectively defend against through traditional network security means. In such scenarios, data privacy protection has become an important issue. Especially in the field of network security, how to protect the privacy of user data when analyzing and handling potential security threats has become a key issue. Therefore, it is particularly important to develop a method that can detect APT attacks and analyze data without directly sharing the original data. This not only meets the current privacy protection standards, but also effectively responds to complex network security threats.

[0003] Among existing technologies, privacy computing technologies, such as homomorphic encryption, secure multi-party computing (SMC), and federated learning, provide new solutions for protecting data privacy. These technologies allow data processing and analysis without directly sharing the original data. Deep learning technology has made significant progress in image recognition, natural language processing, and other fields. Applying deep learning to network security, especially in complex attack detection, shows great potential.

[0004] Traditional network security solutions (such as firewalls, intrusion detection systems, etc.) are usually unable to effectively identify and defend against APT attacks because these attacks are very similar in behavior to regular network traffic. In addition, in scenarios where multiple parties share data, how to conduct effective data analysis and processing without exposing sensitive information is a major challenge facing existing technologies.

[0005] The importance of privacy protection in APT attack detection is becoming increasingly prominent, especially in the context of multi-party joint intelligence sharing. The traditional threat intelligence sharing model may lead to the leakage of sensitive information (such as user data and corporate secrets), which in turn leads to legal and compliance risks. In the process of sharing threat intelligence between enterprises and organizations, a large amount of sensitive data is involved. This data includes but is not limited to personal identity information, internal corporate communications, customer data, etc. If this information is not handled properly, it may lead to serious privacy leakage problems.

[0006] Currently, the APT attack multi-data source detection technology mainly faces the following challenges:

[0007] 1. Weak ability to deal with long-term and covert attacks: The long-term and covert nature of APT attacks makes them difficult to be quickly identified by traditional security methods. Traditional methods often rely on known attack patterns or features, while APT attacks usually have more complex and changing behavior patterns.

[0008] By using privacy computing technology, data from different sources can be integrated while protecting data privacy, which allows for more comprehensive observation and analysis of attack behaviors and enhances the detection capabilities of long-term and covert attacks.

[0009] 2. Joint sharing and analysis by multiple parties is difficult: Data held by different entities (such as different companies or institutions) have different focuses, and may not fully reveal the full picture of APT attacks when viewed individually.

[0010] Through multi-party joint query, these data sources can be integrated to generate a comprehensive "big picture", thus providing a more comprehensive perspective to observe and analyze potential APT attacks.

[0011] 3. Insufficient comprehensive ability to analyze multi-party data: "A comprehensive network behavior map is formed by integrating multi-party data. This map can reveal complex interactions and connections across multiple networks, systems, and domains, which is crucial for detecting long-term and covert APT attacks. Current methods and technologies are insufficient to analyze more detailed and comprehensive attack behavior patterns, which in turn leads to insufficient accuracy and timeliness of detection.

[0012] 4. Data isolation leads to insufficient analysis capabilities: In existing technologies, due to concerns about data privacy and security, each party often stores and processes data independently, resulting in the inability to effectively utilize cross-institutional or cross-domain data resources. The parties can also jointly analyze data while ensuring data privacy and security, which limits the comprehensiveness and depth of attack detection.

[0013] The prior art includes the following:

[0014] 1. Signature-based APT attack detection method:

[0015] Signature-based APT attack detection is one of the earliest network security technologies, which mainly relies on a predefined malware signature database. The principle of this method is to identify threats by comparing network traffic or system files with the characteristic signatures of known malicious code. The advantage of this method is that signature detection is efficient in identifying known malware and can quickly match and block known threats.

[0016] 2. APT attack detection method based on behavioral analysis:

[0017] The APT attack detection method based on behavioral analysis is a more advanced detection method. Instead of simply looking for known malicious signatures, it identifies potential threats by monitoring the behavior patterns of networks and systems. This method is based on an assumption: even if the signature of malware can change, its behavior pattern (such as accessing specific files, modifying system configuration) is relatively stable. Its advantage is that behavioral analysis can identify new or variant malware that are detected by traditional signatures.

[0018] 3. APT attack detection method based on abnormal activities:

[0019] Anomalous activity detection methods identify potential APT attacks by establishing baselines of normal behavior for networks and systems, and then monitoring deviations from these baselines. This approach typically requires collecting and analyzing large amounts of historical data to determine what is "normal." The advantage is that anomaly detection can reveal unusual activities that may indicate an APT attack is in progress.

[0020] The disadvantages of the prior art are as follows:

[0021] 1. Limitations of signature-based APT attack detection methods: With the continuous evolution of APT attack strategies, signature-based methods cannot effectively detect new or customized malware. They lack global threat perception capabilities, lack multi-party joint sharing of threat intelligence, and lack the ability to identify unknown attacks. They lack cooperative sharing of sensitive data and are unable to organize analysis and detect large-scale attacks. In multiple and complex demand scenarios, balancing the ability to respond to threats efficiently while fully considering user privacy is something that most existing technologies cannot do.

[0022] 2. Limitations of behavioral analysis-based APT attack detection methods: Behavioral analysis may produce a high false alarm rate because normal network or system activities may sometimes be misjudged as malicious behavior. In addition, a large amount of data and complex algorithms are required to accurately define and identify "normal behavior" and "abnormal behavior." There is insufficient consideration of the trade-off between data collection and privacy, and there are limitations in applicable scenarios. There is a lack of balance between the detailed nature of data and the protection of user privacy. Such technologies do not consider secure cooperation mechanisms and sharing scales in collaboration. In summary, while implementing advanced behavioral analysis APT attack detection, there is little comprehensive consideration of privacy protection and multi-party collaboration to establish a secure, efficient and balanced network security ecosystem.

[0023] 3. Limitations of APT attack detection methods based on abnormal activities: Establishing an effective normal behavior baseline is a challenge because network environments and user behaviors often change. In addition, similar to behavioral analysis, anomaly detection may also produce false positives. There is a historical data collected and analyzed during the abnormal activity detection process that exposes sensitive information. There is a lack of awareness of potential threats by the entire ecosystem. Summary of the invention

[0024] The present invention aims to address the shortcomings of existing network security technologies in dealing with complex APT attacks, and at the same time solve the privacy protection problem in multi-party data sharing analysis. By combining privacy computing and graph deep learning algorithms, the present invention provides a new solution for efficient and privacy-friendly APT attack detection.

[0025] The present invention is a multi-party joint complex APT attack detection method supporting privacy protection, comprising the following steps:

[0026] Step 1: Data collection and preprocessing, including:

[0027] Collecting raw data from networks and systems, the raw data including network traffic, system logs, and application logs;

[0028] Collecting network traffic data from network devices and security devices, the network traffic data including packet, TCP and / or UDP traffic statistics;

[0029] Collecting system logs from the operating system, wherein the system logs include system events, warnings, and error messages;

[0030] Collecting log information from the application, the log information including transaction logs, user activities, and system calls;

[0031] Perform data preprocessing and use data cleaning methods to process the collected data, including removing irrelevant data and unifying the data format;

[0032] Use data mining techniques and statistical analysis and pattern recognition to extract key features from pre-processed data for subsequent APT attack detection;

[0033] Step 2: Apply privacy protection technology, including:

[0034] Apply data anonymization technology to de-identify personal or organizational identification information in the data;

[0035] Apply homomorphic encryption technology to encrypt the extracted feature data;

[0036] Deploy secure multi-party computation for joint data analysis between participants without revealing the content of their respective data;

[0037] Step 3: Build a multi-party joint collaboration platform, including:

[0038] Install Docker on all participating servers and create separate Docker containers for each microservice;

[0039] Deploy and configure a Kubernetes cluster, including at least one master node and multiple worker nodes, and use Kubernetes' Pods to run and manage the Docker containers;

[0040] Integrate OAuth and OpenID Connect protocols to implement user authentication and authorization mechanisms;

[0041] Use Kubernetes monitoring tools, such as Prometheus and Grafana, to monitor the health and performance of microservices;

[0042] Step 4: Multi-party data analysis and APT detection, including:

[0043] The attribute-based encryption (ABE) method is used to encrypt the data;

[0044] Incorporating encrypted data into a privacy-preserving computing framework that uses secure multi-party computing or fully homomorphic encryption technology, allowing multiple partners to jointly analyze data without revealing the content of their own data;

[0045] Extract useful features from large-scale graph data and identify abnormal behaviors, deploy deep learning models, use graph attention networks and variational autoencoders;

[0046] The relationship between nodes is dynamically learned through the attention mechanism, and the variational autoencoder is used to learn the potential high-dimensional features by reconstructing the input data to identify abnormal patterns in the network.

[0047] Furthermore, after step 4, the method further includes:

[0048] Step 5: Provide feedback, adjust, and optimize the deep learning model and data processing flow in the APT attack detection system.

[0049] Furthermore, the data cleaning method described in step 1 includes:

[0050] Identify and remove duplicate rows or records in a data set;

[0051] Delete records or columns containing missing values, or fill missing values ​​using mean, median, mode or other statistical methods, or ignore missing values ​​without processing;

[0052] Correcting data entry errors, data corruption or outliers, including by performing validity checks on the data and using statistical tests to detect and address outliers;

[0053] Clean text data, including: converting text to lowercase or uppercase to eliminate differences caused by uppercase and lowercase, deleting leading and trailing spaces and special characters in the text, and performing spelling check and grammar correction on the text.

[0054] Further, in step 1, the use of data mining technology and the use of statistical analysis and pattern recognition include:

[0055] Association analysis, exploring frequent item sets through the Apriori algorithm or FP-Growth algorithm to identify IP addresses that often appear together;

[0056] Cluster analysis, which divides the data into groups with similar characteristics through K-means or DBSCAN to identify events with the same attack pattern;

[0057] Classification techniques, which classify data into predefined categories by training models using decision trees, support vector machines, or neural network algorithms;

[0058] Anomaly detection, identifying outliers in data through statistics-based thresholding methods or machine learning-based isolation forest algorithms.

[0059] Furthermore, in step 1, extracting key features from the preprocessed data includes:

[0060] Monitor network traffic for deviations from historical baselines, focusing on abnormal upload and download volumes of data and activities on uncommon ports;

[0061] Identify failed login attempts, unusual login times or locations, and unusual privilege escalation attempts by analyzing security logs;

[0062] Detecting anomalous behavior in system command and application logs, including unauthorized access and system configuration changes;

[0063] Analyze the usage of command line tools and system calls to find traces of backdoors and malware;

[0064] Use statistical methods and machine learning algorithms to identify and predict unusual deviations in network and system behavior;

[0065] Monitor DNS requests to identify access to new or known malicious domains and analyze unusual usage of common network protocols;

[0066] Implement user behavior analysis and device-to-device behavior correlation to detect sudden access to sensitive data or unusual activity across multiple devices.

[0067] Furthermore, in step 2, the data anonymization technology is a k-anonymity technology, which specifically includes:

[0068] Identify and classify data sets: identify sensitive information such as name, address, and phone number, and classify data attributes into sensitive and non-sensitive attributes;

[0069] Determine the appropriate k value based on the sensitivity of the data and the privacy protection requirements;

[0070] In the data processing stage, direct identifiers such as names and social security numbers are first suppressed, and then quasi-identifiers such as age and zip code are processed through generalization methods, converting quasi-identifiers from specific values ​​to a wider range, or locally suppressing records that cannot be generalized to meet k-anonymity;

[0071] Perform k-anonymity verification to ensure that each set of quasi-identifier attributes in the dataset is repeated at least k times;

[0072] The processed data is released after ensuring that it does not contain any information that can be used to directly or indirectly identify an individual.

[0073] Furthermore, in step 2, the applying homomorphic encryption technology to encrypt the extracted feature data specifically includes:

[0074] Choose a homomorphic encryption scheme;

[0075] Split large data sets into smaller units;

[0076] Generate a key pair, use the public key to encrypt the feature data and convert it into ciphertext form;

[0077] On encrypted data, perform addition or multiplication operations directly in the ciphertext state;

[0078] After completing the encryption calculation, use the private key to decrypt the result, obtain the analysis result, and verify the result.

[0079] Furthermore, in step 2, the deployment of secure multi-party computing specifically includes:

[0080] Determine the applicable SMC protocol;

[0081] All parties negotiate and set up secure communication protocols and network environment;

[0082] Each participant uses his or her own private data to generate the necessary encrypted data or encryption function. If homomorphic encryption is used, each participant encrypts their own data.

[0083] Exchange encrypted or obfuscated data via a secure communication layer;

[0084] After the data exchange is completed, the participants jointly perform the predefined computing tasks;

[0085] The calculation results are output through the decryption step, and each participant obtains the required analysis results without exposing private data.

[0086] Furthermore, step five specifically includes:

[0087] The system collects and analyzes the performance of the model in actual operation, including performance indicators such as false detection rate, missed detection rate and false alarm rate. The data feedback comes from the comparison and analysis of actual detection results with the known threat database, as well as user feedback through real-time monitoring;

[0088] Based on the performance indicators, optimize the model parameters using parameter adjustment, including adjusting the learning rate, modifying the network layer structure, optimizing regularization techniques and batch size;

[0089] Optimize the data processing process based on the feedback from the detection results, including improving the data preprocessing steps, adjusting the data standardization method, or redesigning the feature engineering method to capture the key signals of APT attacks;

[0090] Use adaptive learning rate adjustment and use neural network structure to enhance the generalization ability and accuracy of the model;

[0091] After the adjustments are verified and tested, they are deployed and updated to the production environment.

[0092] By integrating advanced privacy protection technology, graph deep learning algorithms and multi-party joint collaboration mechanisms, the present invention significantly improves the accuracy and efficiency of APT attack detection compared to existing technologies, while making significant progress in protecting data privacy. It not only optimizes resource utilization and reduces the cost of a single organization, but also improves the flexibility and scalability of the system. In addition, the present invention supports real-time data analysis, greatly improves the response speed of network security threats, effectively reduces potential losses and impacts, and provides a more comprehensive and efficient solution for network security defense. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a schematic diagram of the principle of the present invention. DETAILED DESCRIPTION

[0094] In order to better understand the purpose, technical solution and function of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings, but the present invention can be implemented in a variety of different ways as defined and covered by the claims. The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0095] like Figure 1 As shown, a specific embodiment of the present invention includes:

[0096] Step 1: Data collection and preprocessing

[0097] Collect raw data from the network and system, including network traffic, system logs and application logs; collect network traffic data from network devices (such as routers, switches) and security devices (such as firewalls, intrusion detection systems), including data packets, TCP / UDP traffic statistics, etc.; collect system logs from the operating system, including system events, warnings, error messages, etc.; collect log information from applications (such as databases, Web servers), which may include transaction logs, user activities, system calls, etc.

[0098] Perform data preprocessing and use data cleaning to process the collected data, including removing irrelevant data and unifying the data format. The specific data cleaning method includes the following steps:

[0099] 1. Remove duplicate data: Identify and remove duplicate rows or records in a dataset, usually caused by data entry errors or data merging problems. Use library functions in programming languages, such as the drop_duplicates() method in the pandas library to accomplish this.

[0100] 2. Handle missing values, including:

[0101] Deletion: If there are too many missing values ​​or they cannot be filled reasonably, delete the records or columns containing missing values;

[0102] Filling: Fill missing values ​​using mean, median, mode or other statistical methods;

[0103] Ignore: Ignore missing values ​​and do not process them.

[0104] 3. Correct data errors and outliers: Correction is required for data entry errors, data corruption or outliers. Common methods include:

[0105] Data validation: Check the validity of data, such as checking whether the age field contains negative numbers.

[0106] Outlier Detection: Detect and handle outliers using statistical tests.

[0107] 4. Text data cleaning: For text data, the following cleaning operations can be performed:

[0108] Case Conversion: Convert text to lowercase (or uppercase) to eliminate differences caused by uppercase and lowercase.

[0109] Remove spaces and special symbols: Delete leading and trailing spaces, special characters, etc. in the text.

[0110] Semantic Correction: Spell check and grammar correction of text.

[0111] Using data mining techniques and using statistical analysis and pattern recognition, first, association analysis, through algorithms such as Apriori or FP-Growth, explores frequent item sets and identifies patterns such as IP addresses that often appear together. Next, cluster analysis, such as K-means or DBSCAN, is used to divide the data into groups with similar characteristics in order to identify events with the same attack pattern. Classification technology, through decision trees, support vector machines (SVM) or neural networks, trains models to classify data into predefined categories, such as normal or attack traffic. Finally, anomaly detection, through techniques such as statistical-based threshold methods or machine learning-based isolation forest algorithms, identifies anomalies in the data, such as abnormal peaks in network traffic.

[0112] Extract key features from the preprocessed data for subsequent APT attack detection. The features mainly include the following:

[0113] First, monitor network traffic for deviations from historical baselines, paying special attention to abnormal upload and download volumes of data and activities on uncommon ports, which may indicate data leaks or attempts to evade regular monitoring. Second, analyze security logs to identify failed login attempts, unusual login times or locations, and abnormal privilege escalation attempts; detect abnormal behavior in system command and application logs, such as unauthorized access or system configuration changes. Further, analyze the use of command line tools and system calls to find traces of backdoors and malware. Use statistical methods and machine learning algorithms (such as support vector machines, random forests, and neural networks) to determine and predict abnormal deviations in network and system behavior. In addition, monitor DNS requests to identify access to new domain names or known malicious domain names, and analyze abnormal use of commonly used network protocols. Finally, implement user behavior analysis and device-to-device behavior correlation to detect sudden sensitive data access or abnormal activities between multiple devices, which are key clues to identify potential coordinated attacks or lateral movement in APT attacks.

[0114] Step 2: Application of privacy protection technology

[0115] Data anonymization: Apply data anonymization technology and adopt k-anonymity to ensure that personal or organizational identification information is de-identified in the data to prevent leakage.

[0116] First, the dataset needs to be carefully identified and classified: sensitive information such as name, address, phone number, etc. is identified, and data attributes are divided into sensitive and non-sensitive attributes. Next, according to the sensitivity of the data and the need for privacy protection, the appropriate k value is determined, which will directly affect the strength of privacy protection and the practicality of the data. In the data processing stage, direct identifiers such as name and social security number are first suppressed, and then quasi-identifiers such as age and zip code are processed by generalization methods, converting them from specific values ​​to a wider range. If necessary, local suppression is performed on records that cannot be generalized to meet k-anonymity. After that, k-anonymity verification is performed to ensure that each set of quasi-identifier attributes in the dataset is repeated at least k times to prevent attackers from identifying individuals through partial information. Finally, the processed data is released after ensuring that it does not contain any information that can be used to directly or indirectly identify individuals, thereby ensuring that both individual privacy is protected and the partial use value of the data is maintained.

[0117] Apply homomorphic encryption: Use homomorphic encryption technology to encrypt the extracted feature data. Use addition and multiplication to ensure that certain calculations can still be performed on the data in an encrypted state, providing protection for subsequent collaborative analysis.

[0118] First, you need to choose a suitable homomorphic encryption scheme. Common libraries include Microsoft's SEAL or IBM's HElib. Then divide the large data set into smaller units to optimize computational efficiency. Subsequently, generate the necessary key pairs, use the public key to encrypt the feature data, and convert it into ciphertext form. On the encrypted data, perform the required addition or multiplication operations directly in the ciphertext state to ensure that the confidentiality of the data is not leaked. After completing the encryption calculation, use the private key to decrypt the result, obtain the final analysis result, and verify the result to ensure that the decrypted result is consistent with the result of the same operation on the original data. This process not only protects the privacy of the data, but also allows complex data processing operations to be performed directly on the encrypted data.

[0119] Secure Multi-party Computation (SMC) deployment: Deploy SMC technology when necessary to allow joint data analysis between participants without revealing the content of their respective data.

[0120] First, it is necessary to determine the applicable SMC protocol. Here, Garbled Circuits and HomomorphicEncryption are selected. Users can choose the most appropriate method based on the sensitivity of the data and the computing requirements. Next, the parties involved must negotiate and set up a secure communication protocol and network environment to ensure the security and integrity of the data during transmission. Each participant uses its own private data to generate the necessary encrypted data or encryption functions. In the case of homomorphic encryption, each data is encrypted. Subsequently, these encrypted or obfuscated data are exchanged through a secure communication layer. After the data exchange is completed, the participants jointly perform predefined computing tasks, so that their original data or intermediate calculation results will not be leaked. Finally, the calculation results are output through appropriate decryption steps, and each participant can obtain the required analysis results without exposing their own private data.

[0121] Step 3: Build a multi-party joint collaboration platform

[0122] Distributed platform architecture: Adopt microservices and containerization technologies, and use Docker and Kubernetes to build a flexible and scalable data sharing platform.

[0123] Secure communication protocol: Combined with TLS / SSL encryption to ensure the security of data transmission, and implement authentication and authorization mechanisms based on OAuth and OpenID Connect.

[0124] Graph deep learning integration: Integrate graph embedding and graph neural network technology, use graph embedding algorithm Node2Vec and graph convolutional network (GCN) to analyze and learn complex network structures.

[0125] First, install Docker on all participating servers and create independent Docker containers for each microservice (authentication service, database service, data processing service, etc.). Next, deploy and configure the Kubernetes cluster, including at least one master node and multiple worker nodes, and use Kubernetes's Pods to run and manage these Docker containers to ensure high availability and load balancing of services. Microservices communicate through defined APIs, and use Kubernetes' service discovery and network functions to ensure connectivity between containers. For the security of data transmission, all communications between services are encrypted with TLS / SSL and use the HTTPS protocol. In addition, integrate OAuth and OpenID Connect protocols to implement a secure user authentication and authorization mechanism, allowing users to log in to the platform securely and access resources according to permissions. Finally, use Kubernetes' monitoring tools, Prometheus and Grafana, to monitor the health and performance of microservices, ensure the stable operation of the system and respond to any problems in a timely manner.

[0126] Step 4: Multi-party data analysis and APT detection

[0127] Encrypted Data Analysis: Combine attribute encryption techniques (such as CP-ABE) and privacy-based computing frameworks to perform secure analysis on encrypted data.

[0128] Application of deep learning models: Use deep learning models such as the graph attention model (GAT) and autoencoder to extract features and identify abnormal behaviors from large-scale graph data.

[0129] First, attribute-based encryption (ABE) is used to encrypt the data, ensuring that only entities with specific attributes can decrypt and access the data, thereby providing refined access control and protecting data privacy. Then, the encrypted data is incorporated into a privacy-preserving computing framework using secure multi-party computing (SMC) or fully homomorphic encryption (FHE) technology, so that multiple partners can jointly analyze the data without revealing the content of their respective data. Furthermore, useful features are extracted from large-scale graph data and abnormal behaviors are identified. Deep learning models are deployed, using graph attention networks (GATs) and variational autoencoders (VAEs). The relationship between nodes is dynamically learned through the attention mechanism, and the variational autoencoder is used to learn its potential high-dimensional features by reconstructing the input data, thereby effectively identifying abnormal patterns in the network. This series of operations are completed in the state of data encryption, ensuring the security and privacy protection of the entire analysis process, thereby improving the detection efficiency and accuracy of APT attacks.

[0130] In addition, you can continue to step five: provide feedback, adjust, and optimize the deep learning model and data processing flow in the APT attack detection system.

[0131] First, the system collects and analyzes the performance of the model in actual operation, including key performance indicators such as false detection rate, missed detection rate, and false alarm rate. These data feedbacks come from the comparative analysis of actual detection results and known threat databases, as well as user feedback obtained through real-time monitoring. Next, according to these performance indicators, the model parameters are optimized using parameter adjustments. This includes adjusting the learning rate, modifying the network layer structure, optimizing regularization techniques and batch size, etc. At the same time, the data processing process is also optimized based on the feedback of the detection results, including improving the data preprocessing steps, adjusting the data normalization method, or redesigning the feature engineering method to more effectively capture the key signals of APT attacks. In addition, adaptive learning rate adjustment is used, and more complex neural network structures such as deep residual networks are used to enhance the generalization ability and accuracy of the model. Finally, after verification and testing, these adjustments will be deployed and updated to the production environment to ensure the continuous improvement of the APT attack detection system and adapt to the new threat environment.

[0132] The present invention relates to the following technologies:

[0133] 1. Privacy protection

[0134] Technologies such as homomorphic encryption and secure multi-party computation (SMC) are used to perform analysis without exposing the original data.

[0135] Apply data anonymization and differential privacy technologies to further protect personal and organizational data privacy.

[0136] 2. Joint collaboration among multiple parties

[0137] Establish a multi-party joint collaboration framework to share APT attack information and intelligent resources.

[0138] Develop sharing protocols, data format standardization, and security mechanisms to ensure data security and privacy.

[0139] 3. Graph Deep Learning Algorithms

[0140] Use graph deep learning algorithms to analyze the jointly generated "big graph" to identify and predict complex APT attack patterns. Improve the accuracy and efficiency of detection, and process and analyze large-scale network data.

[0141] The above technologies specifically involve:

[0142] 1. Integration of privacy protection technology

[0143] Data anonymization: k-anonymity technology is used to anonymize the data to ensure that the identity information of individuals or organizations is de-identified in the data to prevent information leakage.

[0144] Homomorphic encryption technology: Homomorphic encryption is used to encrypt feature data, allowing addition and multiplication operations to be performed on encrypted data to ensure the privacy and security of data during the calculation process.

[0145] Secure Multi-party Computing (SMC) technology: Deployment of SMC technology allows multiple parties to conduct joint data analysis without revealing the content of their respective data.

[0146] 2. Construction of a multi-party joint collaboration platform

[0147] Microservices and containerization technologies: Use Docker and Kubernetes to build a data sharing platform to improve the flexibility and scalability of the system.

[0148] Secure data transmission: TLS / SSL encryption technology is used to ensure data transmission security, and authentication and authorization mechanisms based on OAuth and OpenID Connect are implemented to protect privacy and security during data transmission.

[0149] Graph Deep Learning Integration: Integrate graph embedding and graph neural network techniques, such as Node2Vec and Graph Convolutional Network (GCN), for analysis and learning of complex network structures.

[0150] 3. Multi-party data analysis and APT detection

[0151] Attribute encryption technology and privacy-based computing framework: Combine attribute encryption technology (such as CP-ABE) to perform secure analysis on encrypted data and ensure privacy protection during data analysis.

[0152] Application of deep learning models: Deep learning models such as the graph attention model (GAT) and autoencoders are used to extract features and identify abnormal behaviors from large-scale graph data, improving the accuracy and efficiency of APT attack detection.

[0153] Here is how the above techniques work:

[0154] 1. Application of privacy protection technology

[0155] Data anonymization: Through k-anonymity technology, after removing direct and indirect identifiers, the information of each entity cannot be associated with less than k entities in the data set, thereby protecting personal privacy.

[0156] Homomorphic encryption technology: Homomorphic encryption allows specific operations (such as addition and multiplication) to be performed on encrypted data, and the encrypted results are obtained. After decryption, these results are the same as the results of the same operation directly on the plaintext. In this way, data can be processed without decrypting the data, ensuring data privacy.

[0157] Secure Multi-Party Computation (SMC) technology: SMC technology allows multiple parties to collaboratively compute a function without revealing the contents of their respective data. Each party can only see the final result of the computation and cannot know the input of other parties, thus protecting the data privacy of all parties.

[0158] 2. Construction of a multi-party joint collaboration platform

[0159] Microservices and containerization technologies: Microservice architecture decomposes applications into a set of small services, each of which runs in its own independent environment and collaborates through lightweight communication mechanisms. Containerization such as Docker and Kubernetes provides a way to isolate applications from the environment, enhancing the flexibility and efficiency of application deployment, expansion and management.

[0160] Secure data transmission: TLS / SSL encryption protocols ensure the security of data during transmission, while authentication and authorization mechanisms such as OAuth and OpenID Connect ensure that only authorized users can access data, enhancing the security of the data exchange process.

[0161] 3. Multi-party data analysis and APT detection

[0162] Attribute encryption and privacy-based computing framework: Attribute encryption allows the decryption rights of encrypted text to be controlled based on the user's attributes. Combined with a privacy-based computing framework, encrypted data can be analyzed while ensuring privacy protection.

[0163] Application of graph deep learning algorithms: Graph embedding algorithms such as Node2Vec and graph convolutional networks (GCN) can learn low-dimensional representations of nodes from complex network structures. Deep learning models such as graph attention models (GAT) and autoencoders further analyze these representations for feature extraction and abnormal behavior identification, thereby effectively detecting APT attacks.

[0164] The following is a specific application case: network security enhancement of a regional bank.

[0165] background:

[0166] A regional bank wanted to build a system to enhance its detection capabilities against sophisticated APT attacks while ensuring the privacy of customer data and transaction information.

[0167] System Configuration:

[0168] Server configuration: Use a server with an Intel Xeon Gold 6230 processor, 128GB RAM and 4TB SSD storage.

[0169] Network environment: The bank's internal network bandwidth is 10Gbps and the external Internet connection is 1Gbps.

[0170] Software and Tools:

[0171] Operating System: The server runs on Ubuntu 20.04LTS.

[0172] Data processing and analysis tools: Use Python 3.8 with NumPy and Pandas libraries for data processing, and use TensorFlow and PyTorch for building and training deep learning models.

[0173] Implementation steps:

[0174] Step 1: Data collection and preprocessing

[0175] First, network traffic data is collected using a network monitoring tool based on Zeek (formerly Bro). Second, system logs from network devices and servers are collected through a Syslog server.

[0176] In data preprocessing, we first use Python scripts to clean the data and remove irrelevant log entries, such as regular network traffic and known safe connection records. Then, we apply the principal component analysis (PCA) method to extract features from the data and select key features such as the number of login attempts and data packet size.

[0177] Step 2: Application of privacy protection technology

[0178] First, apply the k-anonymity algorithm to sensitive feature data (such as customer account information) to ensure privacy. Then use an LWE-based homomorphic encryption library, such as Microsoft SEAL, to encrypt the feature data.

[0179] Step 3: Build and implement a multi-party joint collaboration platform

[0180] First, Apache Kafka is used as a message queue to support real-time transmission and processing of encrypted data. Secondly, graph deep learning algorithms are deployed on the platform, and DGL (Deep Graph Library) is used to build a model based on graph convolutional network (GCN).

[0181] Step 4: Multi-party data analysis and APT detection

[0182] First, the analysis model of homomorphic encrypted data is run, and the encrypted data is processed using secure multi-party computing technology. Secondly, the graph neural network model is used to detect abnormal patterns and potential APT attack behaviors.

[0183] Step 5: Result feedback and system optimization

[0184] The results are transmitted to the bank’s security team through a secure internal network. Finally, based on the actual test results and feedback from the bank team, the deep learning model and data processing process are adjusted and optimized.

[0185] By integrating advanced privacy protection technology, graph deep learning algorithms and multi-party joint collaboration mechanisms, this invention significantly improves the accuracy and efficiency of APT attack detection compared to existing technologies, and at the same time makes significant progress in protecting data privacy. It not only optimizes resource utilization and reduces the cost of a single organization, but also improves the flexibility and scalability of the system. In addition, the solution supports real-time data analysis, greatly improves the response speed of network security threats, effectively reduces potential losses and impacts, and provides a more comprehensive and efficient solution for network security defense.

[0186] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-party joint complex APT attack detection method supporting privacy protection, characterized in that: The steps include: Step 1: Data collection and preprocessing, including: Collecting raw data from networks and systems, the raw data including network traffic, system logs, and application logs; Collecting network traffic data from network devices and security devices, the network traffic data including packet, TCP and / or UDP traffic statistics; Collecting system logs from the operating system, wherein the system logs include system events, warnings, and error messages; Collecting log information from the application, the log information including transaction logs, user activities, and system calls; Perform data preprocessing and use data cleaning methods to process the collected data, including removing irrelevant data and unifying the data format; Use data mining techniques and statistical analysis and pattern recognition to extract key features from pre-processed data for subsequent APT attack detection; Step 2: Apply privacy protection technology, including: Apply data anonymization technology to de-identify personal or organizational identification information in the data; Apply homomorphic encryption technology to encrypt the extracted feature data; Deploy secure multi-party computation for joint data analysis between participants without revealing the content of their respective data; Step 3: Build a multi-party joint collaboration platform, including: Install Docker on all participating servers and create separate Docker containers for each microservice; Deploy and configure a Kubernetes cluster, including at least one master node and multiple worker nodes, and use Kubernetes' Pods to run and manage the Docker containers; Integrate OAuth and OpenID Connect protocols to implement user authentication and authorization mechanisms; Use Kubernetes monitoring tools, such as Prometheus and Grafana, to monitor the health and performance of microservices; Step 4: Multi-party data analysis and APT detection, including: The attribute-based encryption (ABE) method is used to encrypt the data; Incorporating encrypted data into a privacy-preserving computing framework that uses secure multi-party computing or fully homomorphic encryption technology, allowing multiple partners to jointly analyze data without revealing the content of their own data; Extract useful features from large-scale graph data and identify abnormal behaviors, deploy deep learning models, use graph attention networks and variational autoencoders; The relationship between nodes is dynamically learned through the attention mechanism, and the variational autoencoder is used to learn the potential high-dimensional features by reconstructing the input data to identify abnormal patterns in the network.

2. According to the privacy protection-supporting multi-party joint complex APT attack detection method of claim 1, it is characterized in that: After step 4, it also includes: Step 5: Provide feedback, adjust, and optimize the deep learning model and data processing flow in the APT attack detection system.

3. A multi-party joint complex APT attack detection method supporting privacy protection according to claim 1 or 2, characterized in that: The data cleaning method described in step 1 includes: Identify and remove duplicate rows or records in a data set; Delete records or columns containing missing values, or fill missing values ​​using mean, median, mode or other statistical methods, or ignore missing values ​​without processing; Correcting data entry errors, data corruption or outliers, including by performing validity checks on the data and using statistical tests to detect and address outliers; Clean text data, including: converting text to lowercase or uppercase to eliminate differences caused by uppercase and lowercase, deleting leading and trailing spaces and special characters in the text, and performing spelling check and grammar correction on the text.

4. According to claim 3, a multi-party joint complex APT attack detection method supporting privacy protection is characterized in that: In step 1, the use of data mining techniques and statistical analysis and pattern recognition includes: Association analysis, exploring frequent item sets through the Apriori algorithm or FP-Growth algorithm to identify IP addresses that often appear together; Cluster analysis, which divides the data into groups with similar characteristics through K-means or DBSCAN to identify events with the same attack pattern; Classification techniques, which classify data into predefined categories by training models using decision trees, support vector machines, or neural network algorithms; Anomaly detection, identifying outliers in data through statistics-based thresholding methods or machine learning-based isolation forest algorithms.

5. According to claim 4, a multi-party joint complex APT attack detection method supporting privacy protection is characterized in that: In step 1, extracting key features from the preprocessed data includes: Monitor network traffic for deviations from historical baselines, focusing on abnormal upload and download volumes of data and activities on uncommon ports; Identify failed login attempts, unusual login times or locations, and unusual privilege escalation attempts by analyzing security logs; Detecting abnormal behavior in system command and application logs, including unauthorized access and system configuration changes; Analyze the usage of command line tools and system calls to find traces of backdoors and malware; Use statistical methods and machine learning algorithms to identify and predict unusual deviations in network and system behavior; Monitor DNS requests to identify access to new or known malicious domains and analyze unusual usage of common network protocols; Implement user behavior analysis and device-to-device behavior correlation to detect sudden access to sensitive data or unusual activity across multiple devices.

6. According to claim 5, a multi-party joint complex APT attack detection method supporting privacy protection is characterized in that: In step 2, the data anonymization technology is k-anonymity technology, which specifically includes: Identify and classify data sets: identify sensitive information such as name, address, and phone number, and classify data attributes into sensitive and non-sensitive attributes; Determine the appropriate k value based on the sensitivity of the data and the privacy protection requirements; In the data processing stage, direct identifiers such as names and social security numbers are first suppressed, and then quasi-identifiers such as age and zip code are processed through generalization methods, converting quasi-identifiers from specific values ​​to a wider range, or locally suppressing records that cannot be generalized to meet k-anonymity; Perform k-anonymity verification to ensure that each set of quasi-identifier attributes in the dataset is repeated at least k times; The processed data is released after ensuring that it does not contain any information that can be used to directly or indirectly identify an individual.

7. The method for detecting complex APT attacks by multiple parties supporting privacy protection according to claim 6, characterized in that: In step 2, the application of homomorphic encryption technology to encrypt the extracted feature data specifically includes: Choose a homomorphic encryption scheme; Split large data sets into smaller units; Generate a key pair, use the public key to encrypt the feature data and convert it into ciphertext form; On encrypted data, perform addition or multiplication operations directly in the ciphertext state; After completing the encryption calculation, use the private key to decrypt the result, obtain the analysis result, and verify the result.

8. The method for detecting complex APT attacks by multiple parties supporting privacy protection according to claim 7, characterized in that: In step 2, the deployment of secure multi-party computing specifically includes: Determine the applicable SMC protocol; All parties negotiate and set up secure communication protocols and network environment; Each participant uses his or her own private data to generate the necessary encrypted data or encryption function. If homomorphic encryption is used, each participant encrypts their own data. Exchange encrypted or obfuscated data via a secure communication layer; After the data exchange is completed, the participants jointly perform the predefined computing tasks; The calculation results are output through the decryption step, and each participant obtains the required analysis results without exposing private data.

9. The method for detecting complex APT attacks by multiple parties supporting privacy protection according to claim 2, characterized in that: Step 5 specifically includes: The system collects and analyzes the performance of the model in actual operation, including performance indicators such as false detection rate, missed detection rate and false alarm rate. The data feedback comes from the comparison and analysis of actual detection results with the known threat database, as well as user feedback through real-time monitoring; Based on the performance indicators, optimize the model parameters using parameter adjustment, including adjusting the learning rate, modifying the network layer structure, optimizing regularization techniques and batch size; Optimize the data processing process based on the feedback from the detection results, including improving the data preprocessing steps, adjusting the data standardization method, or redesigning the feature engineering method to capture the key signals of APT attacks; Use adaptive learning rate adjustment and use neural network structure to enhance the generalization ability and accuracy of the model; After the adjustments are verified and tested, they are deployed and updated to the production environment.

Citation Information

Patent Citations

  • Node autonomous method, system and device of node cluster and electronic equipment

    CN112035215A

  • Traditional Chinese medicine prescription data privacy protection method based on K-anonymity

    CN113051619A

  • APT detection method of distributed system

    CN114679332A

  • Enterprise information security management system

    CN117640250A

Cited By

  • Method and system for performing network information security analysis by using big data

    CN119835086A