Network Data Leakage Monitoring System and Method

Abstract

The present invention relates to a network data leakage monitoring system and method. The system includes a data collection module, a data preprocessing module, an anomaly detection module, a risk assessment module, and an alarm module. The data collection module is used to collect network data from network data sources in real time; the data preprocessing module cleans and standardizes the collected network data to obtain preprocessed network data; the anomaly detection module uses a pre-trained neural network model to identify anomaly patterns and calculate confidence scores based on the preprocessed network data; the risk assessment module performs real-time assessment of potential data leakage risks based on the type of anomaly patterns and confidence scores, and generates a comprehensive risk assessment result; the alarm module generates and sends an alarm message when the comprehensive risk assessment result exceeds a preset threshold. The system can detect and evaluate network data leakage risks in real time and accurately, and provide effective security alarms.

Description

Technical Field

[0001] The present invention relates to the field of computer network technology, and particularly to a network data leakage monitoring system and method. Background Art

[0002] In the prior art, network data leakage monitoring systems usually implement the monitoring of network data through modules such as real-time data collection, data preprocessing, and anomaly detection. These systems use rule matching or simple statistical analysis methods to identify potential abnormal patterns, such as sudden increases in data transfer volume or frequent access to sensitive data, etc., in order to issue alarms in a timely manner when potential data leakage occurs.

[0003] However, there are some main problems in the prior art. These methods often rely on predefined rules and thresholds and are difficult to cope with complex and changeable network environments. In addition, traditional statistical analysis and rule matching methods often show low detection accuracy and high false alarm rates when facing advanced persistent threats and zero-day attacks. In addition, existing systems also lack an effective mechanism in risk assessment and are difficult to accurately evaluate the actual risk level of the detected abnormal patterns, resulting in insufficient reliability of alarm information.

[0004] Therefore, there is an urgent need for an improved network data leakage monitoring system that can solve the above problems. Summary of the Invention

[0005] The present application provides a network data leakage monitoring system and method to improve the reliability of network data leakage monitoring.

[0006] The present application provides a network data leakage monitoring system, including:

[0007] A data collection module for real-time collecting network data from network data sources;

[0008] A data preprocessing module for cleaning and standardizing the network data collected by the data collection module to obtain preprocessed network data;

[0009] An anomaly detection module for obtaining abnormal patterns of network data and confidence scores of the abnormal patterns according to the preprocessed network data by using a pre-trained neural network model, wherein the abnormal patterns include abnormal increases in data transfer volume, abnormal access frequencies of specific sensitive data, abnormal network connection behaviors, and network behaviors matching known attack patterns;

[0010] A risk assessment module for performing real-time assessment on potential data leakage risks according to the types and confidence scores of the abnormal patterns to obtain a comprehensive risk assessment result;

[0011] An alarm module, configured to generate and send an alarm message when the comprehensive risk assessment result exceeds a preset threshold, where the alarm message includes a description of the abnormal mode, a risk level, and a recommended countermeasure.

[0012] Furthermore, the neural network model used in the anomaly detection module includes an input layer, a feature extraction layer, a feature fusion layer, a graph convolutional network layer, an abnormal pattern recognition layer, and an anomaly verification layer; wherein, the input layer is configured to receive the preprocessed network data; the feature extraction layer includes a plurality of parallel convolutional neural network sub-layers, and the input of each convolutional neural network sub-layer is the preprocessed data provided by the input layer; each convolutional neural network sub-layer processes the input data to obtain a feature vector; the feature fusion layer is implemented by an attention mechanism and is configured to fuse the feature vectors provided by a plurality of convolutional neural network sub-layers to generate a fused comprehensive feature vector; the graph convolutional network layer is implemented by a graph convolutional network and is configured to process and capture the complex dependencies and topological relationships between the fused comprehensive feature vectors in a non-Euclidean manner to generate a high-dimensional feature vector; the abnormal pattern recognition layer is implemented by a recurrent neural network and is configured to perform time series analysis on the high-dimensional feature vector to identify potential abnormal patterns and assign a preliminary confidence score to each potential abnormal pattern; the anomaly verification layer is implemented by a generative adversarial network and obtains an abnormal pattern and a confidence score according to the potential abnormal patterns and the preliminary confidence scores provided by the abnormal pattern recognition layer.

[0013] Furthermore, the feature extraction layer includes the following plurality of parallel convolutional neural network sub-layers:

[0014] The first convolutional neural network sub-layer is configured to extract traffic features within a short time from the input preprocessed network data, and includes a first convolutional layer, a first pooling layer, and a first activation layer; wherein, the convolutional kernel size used in the first convolutional layer is 3×3, and the stride is 1; the pooling kernel size used in the first pooling layer is 2×2, and the stride is 2; the first activation layer uses a ReLU activation function;

[0015] The second convolutional neural network sub-layer is configured to extract behavior features within a long time from the input preprocessed network data, and includes a second convolutional layer, a second pooling layer, and a second activation layer; wherein, the convolutional kernel size used in the second convolutional layer is 5×5, and the stride is 1; the pooling kernel size used in the second pooling layer is 2×2, and the stride is 2; the second activation layer uses a Leaky ReLU activation function;

[0016] The third convolutional neural network sub-layer is used to extract the frequency features of specific sensitive data access from the preprocessed network data of the input, including a third convolutional layer, a third pooling layer, and a third activation layer; wherein, the convolutional kernel size adopted by the third convolutional layer is 1×1, and the stride is 1; the third pooling layer adopts global max pooling; the third activation layer adopts the Sigmoid activation function;

[0017] The fourth convolutional neural network sub-layer is used to extract the abnormal network connection behavior features from the preprocessed network data of the input, including a fourth convolutional layer, a fourth pooling layer, and a fourth activation layer; wherein, the convolutional kernel size adopted by the fourth convolutional layer is 3×3, and the stride is 1; the fourth pooling layer adopts stochastic pooling; the fourth activation layer adopts the ELU activation function.

[0018] Furthermore, the feature fusion layer includes a feature weighting module, a feature fusion module, and a feature transformation module; wherein, the feature weighting module uses a fully connected layer and the Softmax function to calculate the weights of each feature vector, obtaining multiple weighted feature vectors; the feature fusion module combines multiple weighted feature vectors in a concatenated manner to generate multiple initially fused comprehensive feature vectors; the feature transformation module performs a linear transformation on each initially fused comprehensive feature vector through a fully connected layer and the ReLU activation function to obtain the fused comprehensive feature vectors.

[0019] Furthermore, the graph convolutional network layer includes a graph construction module, a graph convolutional layer, and a feature transformation module; wherein, the graph construction module is used to construct a graph structure representing the relationships between the comprehensive feature vectors, generating an adjacency matrix; the graph convolutional layer is used to perform a convolutional operation on the input comprehensive feature vectors and the adjacency matrix to generate initially high-dimensional feature vectors; the feature transformation module is used to transform the initially high-dimensional feature vectors output by the graph convolutional layer through a fully connected layer and a non-linear activation function to obtain high-dimensional feature vectors.

[0020] Furthermore, the anomaly pattern recognition layer includes a time series data preprocessing module, a recurrent neural network layer, an anomaly pattern detection module, and an identification output module; wherein, the time series data preprocessing module is used to segment the high-dimensional feature vectors in chronological order to form multiple time series segments; the recurrent neural network layer is used to process the time series segments to capture the dependencies and patterns in the time series segments; wherein, the recurrent neural network layer includes a first recurrent neural network and a second recurrent neural network, the first recurrent neural network is implemented using long short-term memory units, and the second recurrent neural network is implemented using gated recurrent units; the anomaly pattern detection module detects potential anomaly patterns and assigns preliminary confidence scores based on the output of the recurrent neural network layer; the identification output module is used to output the detected potential anomaly patterns and the preliminary confidence scores of the potential anomaly patterns.

[0021] Furthermore, the anomaly verification layer includes a generator network, a discriminator network, an anomaly verification module, and a verification output module; wherein, the generator network is used to generate new feature samples based on the potential anomaly patterns; the discriminator network is used to discriminate between the generated new feature samples and the real feature samples to evaluate their authenticity; the anomaly verification module is used to comprehensively verify the potential anomaly patterns and assign confidence scores based on the results of the generator network and the discriminator network; the verification output module is used to output the verified anomaly patterns and the confidence scores of the anomaly patterns.

[0022] Furthermore, the risk assessment module calculates the comprehensive risk assessment result according to the following formula 1:

[0023]

[0024] wherein, is the comprehensive risk assessment result; is the total number of detected anomaly patterns; is the th risk weight of the anomaly pattern type; is the th confidence score of the anomaly pattern; is the th severity score of the anomaly pattern, with a scoring range of 1 to 10, set by the system based on historical data and expert judgment; is the th frequency of the anomaly pattern, indicating the number of times the pattern is detected within a certain period of time; is the th duration length of the anomaly pattern; is a specific threshold for normalizing the th duration length of the anomaly pattern.

[0025] Furthermore, the data acquisition module includes:

[0026] A deep packet inspection engine for deeply analyzing the content of network packets and extracting traffic information and packet features;

[0027] A real-time traffic monitoring unit for monitoring real-time traffic in the network, including HTTP requests, FTP transfers, email communications, and instant messages;

[0028] A data mirroring module for real-time mirroring of network data streams without affecting the normal operation of the network for subsequent analysis;

[0029] A data caching module for temporarily storing the collected network data for quick access before the data preprocessing module performs cleaning and standardization processing on the data.

[0030] This application provides a method for monitoring network data leakage, including:

[0031] Real-time collection of network data from network data sources;

[0032] Cleaning and standardizing the network data collected in the data collection step to obtain preprocessed network data;

[0033] According to the preprocessed network data, using a pre-trained neural network model, obtaining abnormal patterns of the network data and confidence scores of the abnormal patterns, where the abnormal patterns include abnormal increases in data transfer volume, abnormal access frequencies of specific sensitive data, abnormal network connection behaviors, and network behaviors matching known attack patterns;

[0034] According to the types and confidence scores of the abnormal patterns, performing real-time assessment of potential data leakage risks to obtain a comprehensive risk assessment result;

[0035] When the comprehensive risk assessment result exceeds a preset threshold, generating and sending an alarm message, where the alarm message includes a description of the abnormal pattern, a risk level, and recommended countermeasures.

[0036] The beneficial effects of this application mainly include: (1) The anomaly detection module uses a pre-trained neural network model to accurately identify various anomaly patterns, including abnormal increases in data transfer volume, abnormal access frequencies of specific sensitive data, abnormal network connection behaviors, and behaviors matching known attack patterns. Through the deep learning ability of the neural network model, the accuracy and coverage of anomaly detection are significantly improved, reducing false positives and false negatives. (2) The risk assessment module comprehensively evaluates the potential data leakage risks based on the types and confidence scores of the detected anomaly patterns. The real-time assessment function ensures that the system can promptly identify and quantify risks, generate comprehensive risk assessment results, and provide intuitive risk assessment information for managers to facilitate decision-making. (3) The alarm module generates and sends alarm information containing descriptions of anomaly patterns, risk levels, and recommended countermeasures when the comprehensive risk assessment result exceeds a preset threshold. Multi-level alarms and detailed countermeasure recommendations can help security personnel quickly take appropriate defensive measures and reduce the harm of data leakage. (4) Through the collaborative work of multiple modules, this system forms a complete security monitoring chain from data collection to anomaly detection, then to risk assessment and alarm. The professional design and high integration of each module ensure the overall performance and reliability of the system, and can provide continuous and stable security protection in complex and changing network environments. Description of the Drawings

[0037] Figure 1 is a schematic diagram of a network data leakage monitoring system provided by the first embodiment of this application.

[0038] Figure 2 is a flowchart of a network data leakage monitoring method provided by the second embodiment of this application. Detailed Embodiments

[0039] Many specific details are set forth in the following description in order to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of this application. Therefore, this application is not limited by the specific embodiments disclosed below.

[0040] The first embodiment of this application provides a network data leakage monitoring system. Please refer to Figure 1 , which is a schematic diagram of the first embodiment of this application. The following will describe in detail a network data leakage monitoring system provided by the first embodiment of this application in combination with Figure 1 .

[0041] The network data leakage monitoring includes a data collection module 101, a data preprocessing module 102, an anomaly detection module 103, a risk assessment module 104, and an alarm module 105.

[0042] The data acquisition module 101 is used to collect network data from network data sources in real time.

[0043] The data acquisition module 101 plays a crucial role in the network data leakage monitoring system. It is responsible for collecting network data from network data sources in real time to ensure the timeliness and accuracy of subsequent analysis. The implementation of the data acquisition module 101 includes the following aspects:

[0044] First of all, the data acquisition module 101 uses a deep packet inspection engine to deeply analyze the content of network packets. The deep packet inspection engine extracts various characteristic information of the packets by parsing each network packet, including but not limited to source address, destination address, packet size, transport protocol, timestamp, etc. This characteristic information provides the basic data for subsequent data preprocessing and anomaly detection.

[0045] Secondly, the data acquisition module 101 is equipped with a real-time traffic monitoring unit for monitoring various real-time traffic in the network. The real-time traffic monitoring unit can capture various types of network traffic such as HTTP requests, FTP transfers, email communications, and instant messages. By capturing these traffic in real time, the system can ensure comprehensive coverage of various possible network activities and avoid potential risks of data leakage.

[0046] To ensure that the normal operation of the network is not affected during the acquisition process, the data acquisition module 101 also includes a data mirroring module. The data mirroring module can mirror the network data stream in real time without interfering with network communication. The mirrored data stream is copied for subsequent analysis and processing, while the original data stream continues to maintain normal transmission and processing. This non-intrusive data acquisition method ensures the transparency and security of the system.

[0047] During the data acquisition process, in order to improve the efficiency of data processing, the data acquisition module 101 also sets up a data caching module. The data caching module is used to temporarily store the collected network data so that the data can be quickly accessed and processed when needed. The data caching module adopts an efficient caching strategy and can effectively manage the cache space to ensure the efficient transfer of data between acquisition and preprocessing.

[0048] In addition, the implementation of the data acquisition module 101 also takes into account the scalability and adaptability of the system. The module is designed with a configurable structure, allowing users to perform customized settings according to specific network environments and requirements. For example, users can configure different acquisition rules and filtering conditions to capture only specific types of network data. This flexibility enables the data acquisition module 101 to adapt to various different application scenarios, thereby improving the practicality of the system.

[0049] Through the above-mentioned various technical means and functional designs, the data acquisition module 101 ensures the real-time, accurate, and comprehensive acquisition of network data from network data sources, providing a reliable data basis for subsequent data preprocessing, anomaly detection, risk assessment, and warning.

[0050] The data preprocessing module 102 is used to clean and standardize the network data collected by the data acquisition module to obtain preprocessed network data.

[0051] The data preprocessing module 102 plays a key role in the network data leakage monitoring system. By cleaning and standardizing the network data obtained by the data acquisition module 101, it ensures the accuracy and effectiveness of subsequent analysis.

[0052] First of all, the data preprocessing module 102 includes a data cleaning unit. The main task of this unit is to remove noise data, repair damaged data packets, and fill in missing data to ensure the integrity and quality of the data. Noise data is usually irrelevant or misleading information, and the cleaning unit will screen it according to predefined rules and filtering conditions to remove this useless data. At the same time, for data packets that may be damaged during transmission, the cleaning unit will try to repair or reconstruct these data to ensure the accuracy of the data. In addition, for some missing data caused by network jitter or other reasons, the cleaning unit will fill it through interpolation or other data recovery techniques to make the data set more complete.

[0053] After the cleaning is completed, the data preprocessing module 102 also includes a data standardization unit. The function of this unit is to unify the data from different sources into a consistent format for subsequent processing and analysis. The standardization process includes operations such as timestamp synchronization, protocol standardization, and data field alignment. Timestamp synchronization is to ensure that the time records of all data packets are on the same time basis, thus ensuring the timeliness of the data. Protocol standardization is to convert data packets of different protocols into a unified representation for subsequent unified processing. Data field alignment is to make the field names and field orders of different data sources consistent, ensuring that each field has a clear and unified meaning in the data set.

[0054] In addition, the data preprocessing module 102 is also provided with a data deduplication unit for detecting and deleting duplicate data packets to avoid redundant data affecting subsequent analysis. This process is usually achieved through hash algorithms or other fast comparison algorithms to ensure the efficient identification and deletion of duplicate data packets.

[0055] Another important component is the feature extraction unit, which extracts key features from the data after cleaning and normalization. Feature extraction is to convert the original data into more representative and analyzable feature data, including but not limited to packet size, transmission speed, source address, destination address, etc. These features will serve as the basis for the subsequent anomaly detection module 103 to perform pattern recognition and analysis. The feature extraction unit can adopt various algorithms and technologies, such as principal component analysis (PCA), clustering analysis, etc., to ensure that the extracted features can accurately reflect the essence of the data and the characteristics of abnormal behaviors.

[0056] The preprocessed network data includes a variety of features and metrics extracted from network traffic. These data are processed by the data acquisition module and the data preprocessing module to ensure their quality and consistency, thus providing reliable inputs for the training and inference of the neural network model. The preprocessed network data usually includes the following:

[0057] 1. Basic network traffic features:

[0058] Packet size: The size of each packet (in bytes).

[0059] Number of packets: The number of packets captured within a certain time window.

[0060] Transmission rate: The data transmission rate within a specific time period (in bytes per second).

[0061] Traffic direction: The transmission direction of the packet (inbound or outbound).

[0062] 2. Time features:

[0063] Timestamp: The timestamp when each packet is captured, used to record the arrival time of the packet.

[0064] Time interval: The time interval between consecutive packets.

[0065] 3. Network protocol features:

[0066] Transport protocol: The type of transport protocol used by the packet (such as TCP, UDP, ICMP, etc.).

[0067] Application protocol: The type of application layer protocol used by the packet (such as HTTP, FTP, DNS, etc.).

[0068] 4. Connection features:

[0069] Source IP address and port: The source IP address and source port number of the packet.

[0070] Destination IP address and port: The destination IP address and destination port number of the packet.

[0071] Connection duration: The duration from connection establishment to disconnection.

[0072] 5. Content features:

[0073] HTTP requests and responses: The method, URL, response status code, etc. of the HTTP request.

[0074] DNS queries and responses: The domain name of the DNS query, the IP address of the response, etc.

[0075] FTP commands and responses: The commands and response codes in the FTP session.

[0076] 6. Statistical features:

[0077] Packets per second (PPS): The number of packets received or sent per second.

[0078] Bytes per second (BPS): The number of bytes received or sent per second.

[0079] Number of sessions: The number of active sessions within a certain time window.

[0080] 7. Behavioral features:

[0081] Packet arrival pattern: The arrival pattern of packets, such as burst transmission, periodic transmission, etc.

[0082] Access frequency: The access frequency to specific targets (such as sensitive data or specific services).

[0083] 8. Security-related features:

[0084] Encryption status: Whether the packets are transmitted after encryption.

[0085] Attack signature: Whether it contains known attack signatures or malicious code fragments.

[0086] These preprocessed network data provide a comprehensive view of network activities, covering all aspects from basic traffic statistics to advanced protocol analysis.

[0087] Through the collaborative work of each of the above units, the data preprocessing module 102 can efficiently and accurately process the collected network data, ensure the quality and consistency of the data, and provide a reliable basis for subsequent anomaly detection, risk assessment, and alarm generation.

[0088] Anomaly detection module 103 is used to obtain the anomaly patterns of network data and the confidence scores of the anomaly patterns according to the preprocessed network data by using a pre-trained neural network model, where the anomaly patterns include abnormal increase in data transfer volume, abnormal access frequency of specific sensitive data, abnormal network connection behavior, and network behavior matching known attack patterns.

[0089] Anomaly detection module 103 is a core component of the network data leakage monitoring system, responsible for detecting anomaly patterns from the preprocessed network data and calculating the confidence scores of each anomaly pattern. This module uses a pre-trained neural network model to ensure efficient and accurate identification of various potential threats.

[0090] In specific implementation, anomaly detection module 103 first receives the preprocessed network data provided by data preprocessing module 102. These data have been cleaned and standardized to ensure the quality and consistency of the input data. After receiving the preprocessed data, anomaly detection module 103 processes it through a series of neural network layers, which can include convolutional neural network (CNN), recurrent neural network (RNN), or other deep learning models. These neural network models have been pre-trained with a large amount of historical network data and can identify complex anomaly patterns.

[0091] The first step of anomaly detection module 103 is to perform feature extraction and pattern recognition on the input data through a neural network model. The convolutional layer of the model can identify local features in the data, such as abnormal increase in data transfer volume, abnormal access frequency of specific sensitive data, etc. The convolutional layer scans the data through filters to extract low-level features, and then performs dimensionality reduction and feature aggregation through the pooling layer. This step can effectively extract important features in the data and reduce the data volume.

[0092] Next, the recurrent neural network layer or long short-term memory (LSTM) layer further processes these features to capture the time series information and long-term dependencies in the data. This is particularly important for detecting persistent abnormal network connection behavior or network behavior matching known attack patterns. The recurrent neural network can remember and utilize the sequential information before and after through its recurrent structure to identify complex anomaly patterns.

[0093] After completing feature extraction and pattern recognition, anomaly detection module 103 calculates the confidence scores of each detected anomaly pattern. The confidence scores are calculated through the output layer of the neural network and represent the probability that each anomaly pattern is determined to be a real anomaly. The calculation of the confidence scores involves the activation function and fully connected layer of the neural network. Through these calculation steps, the system can give an accuracy assessment of each anomaly pattern.

[0094] Finally, the anomaly detection module 103 outputs the detected anomaly patterns and their confidence scores. The output anomaly patterns include an abnormal increase in data transfer volume, an abnormal frequency of access to specific sensitive data, abnormal network connection behaviors, and network behaviors matching known attack patterns. These output results will be passed to the risk assessment module 104 for further analysis and processing.

[0095] Through the above steps, the anomaly detection module 103 can efficiently and accurately detect anomaly patterns in network data and provide a confidence score for each anomaly pattern.

[0096] Furthermore, the neural network model used in the anomaly detection module includes an input layer, a feature extraction layer, a feature fusion layer, a graph convolutional network layer, an anomaly pattern recognition layer, and an anomaly verification layer; wherein, the input layer is used to receive the preprocessed network data; the feature extraction layer includes multiple parallel convolutional neural network sub-layers, and the input of each convolutional neural network sub-layer is the preprocessed data provided by the input layer; each convolutional neural network sub-layer processes the input data to obtain a feature vector; the feature fusion layer is implemented using an attention mechanism and is used to fuse the feature vectors provided by multiple convolutional neural network sub-layers to generate a fused comprehensive feature vector; the graph convolutional network layer is implemented using a graph convolutional network and is used to process and capture the complex dependencies and topological relationships between the fused comprehensive feature vectors in a non-Euclidean manner to generate high-dimensional feature vectors; the anomaly pattern recognition layer is implemented using a recurrent neural network and is used to perform time series analysis on the high-dimensional feature vectors to identify potential anomaly patterns and assign a preliminary confidence score to each potential anomaly pattern; the anomaly verification layer is implemented using a generative adversarial network and obtains the anomaly patterns and confidence scores based on the potential anomaly patterns and preliminary confidence scores provided by the anomaly pattern recognition layer.

[0097] The neural network model used by the anomaly detection module includes an input layer, a feature extraction layer, a feature fusion layer, a graph convolutional network layer, an anomaly pattern recognition layer, and an anomaly verification layer.

[0098] First, the input layer is designed to receive the preprocessed network data. This data comes from the data preprocessing module and has been cleaned and standardized to ensure the quality and consistency of the input data. After receiving this data, the input layer passes it to multiple parallel convolutional neural network sub-layers.

[0099] The feature extraction layer consists of multiple parallel convolutional neural network sub-layers. The input of each convolutional neural network sub-layer is the preprocessed network data, and these sub-layers process the input data independently. Each sub-layer extracts features at different levels through a series of convolutional operations. The convolutional operations include scanning the input data with filters to extract local features, and then performing dimensionality reduction and feature aggregation through pooling operations. This structure can effectively capture the spatial features in the data and generate a set of feature vectors, with each convolutional neural network sub-layer corresponding to a feature vector.

[0100] Then, these feature vectors are passed to the feature fusion layer. The feature fusion layer adopts an attention mechanism to fuse the feature vectors provided by multiple convolutional neural network sub-layers. The attention mechanism calculates the importance weights of each feature vector, dynamically adjusts the contributions of each feature vector, and generates a fused comprehensive feature vector. This method ensures that the most important feature information is retained during the fusion process, improving the quality and robustness of the feature representation.

[0101] The fused comprehensive feature vector then enters the graph convolutional network layer. The main function of the graph convolutional network layer is to process and capture the complex dependencies and topological relationships between feature vectors in a non-Euclidean manner. The graph convolutional network adopts a graph structure, where nodes represent feature vectors and edges represent the relationships between them. Through a series of graph convolutional operations, these feature vectors perform information propagation and aggregation in the graph structure, generating high-dimensional feature vectors. The graph convolutional operations can effectively capture the global relationships and context information between feature vectors, further enhancing the feature representation ability.

[0102] The high-dimensional feature vectors are then passed to the anomaly pattern recognition layer. This layer adopts a recurrent neural network (RNN) structure, especially long short-term memory network (LSTM) or gated recurrent unit (GRU). The recurrent neural network processes the temporal information in the high-dimensional feature vectors through time series analysis. It can capture the temporal dependencies in the data and identify potential anomaly patterns. Each identified anomaly pattern is assigned a preliminary confidence score, indicating the likelihood that the pattern is detected as an anomaly.

[0103] Finally, the anomaly verification layer adopts a generative adversarial network (GAN) structure to verify the potential anomaly patterns and the preliminary confidence scores provided by the anomaly pattern recognition layer. The generative adversarial network consists of a generator and a discriminator. The generator attempts to generate anomaly patterns similar to the real data, while the discriminator attempts to distinguish between the generated anomaly patterns and the real anomaly patterns. Through this adversarial training, the generative adversarial network can improve the detection accuracy of anomaly patterns and the reliability of the confidence scores. Ultimately, the anomaly verification layer outputs the verified anomaly patterns and their final confidence scores.

[0104] Furthermore, the feature extraction layer includes the following multiple parallel convolutional neural network sub-layers:

[0105] The first convolutional neural network sub-layer is used to extract traffic features within a short period from the preprocessed network data input, and includes a first convolutional layer, a first pooling layer, and a first activation layer; wherein, the convolutional kernel size adopted by the first convolutional layer is 3×3, and the stride is 1; the pooling kernel size adopted by the first pooling layer is 2×2, and the stride is 2; the first activation layer adopts the ReLU activation function;

[0106] The second convolutional neural network sub-layer is used to extract behavior features within a long period from the preprocessed network data input, and includes a second convolutional layer, a second pooling layer, and a second activation layer; wherein, the convolutional kernel size adopted by the second convolutional layer is 5×5, and the stride is 1; the pooling kernel size adopted by the second pooling layer is 2×2, and the stride is 2; the second activation layer adopts the Leaky ReLU activation function;

[0107] The third convolutional neural network sub-layer is used to extract frequency features of specific sensitive data access from the preprocessed network data input, and includes a third convolutional layer, a third pooling layer, and a third activation layer; wherein, the convolutional kernel size adopted by the third convolutional layer is 1×1, and the stride is 1; the third pooling layer adopts global max pooling; the third activation layer adopts the Sigmoid activation function;

[0108] The fourth convolutional neural network sub-layer is used to extract abnormal network connection behavior features from the preprocessed network data input, and includes a fourth convolutional layer, a fourth pooling layer, and a fourth activation layer; wherein, the convolutional kernel size adopted by the fourth convolutional layer is 3×3, and the stride is 1; the fourth pooling layer adopts random pooling; the fourth activation layer adopts the ELU activation function.

[0109] The feature extraction layer includes multiple parallel convolutional neural network sub-layers, and each sub-layer is responsible for extracting different types of features from the preprocessed network data input. Specifically, these sub-layers include the first convolutional neural network sub-layer, the second convolutional neural network sub-layer, the third convolutional neural network sub-layer, and the fourth convolutional neural network sub-layer.

[0110] First, the first convolutional neural network sub-layer is designed to extract traffic features within a short time from the preprocessed network data. This sub-layer includes a first convolutional layer, a first pooling layer, and a first activation layer. The first convolutional layer uses a convolutional kernel of size 3×3 with a stride of 1, and by scanning the input data, it extracts local traffic features. These features are reduced in dimension and aggregated through the first pooling layer. The pooling kernel size is 2×2 with a stride of 2, which reduces the data dimension while retaining important features. Subsequently, the first activation layer uses the ReLU activation function to convert the linear output to non-linear, enhancing the network's expressive power and the training effect of the model.

[0111] The second convolutional neural network sub-layer is designed to extract behavior features over a long time from the preprocessed network data. This sub-layer includes a second convolutional layer, a second pooling layer, and a second activation layer. The second convolutional layer uses a convolutional kernel of size 5×5 with a stride of 1, and through a larger receptive field, it captures behavior features within a long time range. The second pooling layer also uses a 2×2 pooling kernel with a stride of 2 for dimensionality reduction. To adapt to the complexity of long-term behavior features, the second activation layer uses the Leaky ReLU activation function, which allows for small negative value outputs, solves the "dying ReLU" problem of the standard ReLU, and enhances the robustness of the model.

[0112] The third convolutional neural network sub-layer is designed to extract frequency features of specific sensitive data access from the preprocessed network data. This sub-layer includes a third convolutional layer, a third pooling layer, and a third activation layer. The third convolutional layer uses a 1×1 convolutional kernel with a stride of 1, mainly for extracting features of each data packet without changing in the spatial dimension. The third pooling layer uses global max pooling to reduce the entire feature map to a single value, which highlights the most prominent features. To map the output to between 0 and 1, the third activation layer uses the Sigmoid activation function, which is particularly suitable for representing frequency features.

[0113] The fourth convolutional neural network sub-layer is designed to extract abnormal network connection behavior features from the preprocessed network data. This sub-layer includes a fourth convolutional layer, a fourth pooling layer, and a fourth activation layer. The fourth convolutional layer uses a 3×3 convolutional kernel with a stride of 1, and by scanning the input data, it extracts local features. The fourth pooling layer uses stochastic pooling, which randomly selects some pooling units, increasing the model's robustness and preventing overfitting. The fourth activation layer uses the ELU (Exponential Linear Unit) activation function, which can not only reduce the vanishing gradient problem but also provide higher scalability in the negative value region to adapt to complex network connection behavior features.

[0114] Through these parallel convolutional neural network sub - layers, the feature extraction layer can extract features of different time scales and different types from the pre - processed network data. These features include traffic features in a short time, behavior features in a long time, frequency features of specific sensitive data access, and abnormal network connection behavior features, providing rich information for subsequent feature fusion and anomaly detection.

[0115] The following is the reference implementation code for the input layer and the feature extraction layer in the neural network model. The code uses the popular deep learning framework Keras.

[0116] import tensorflow as tf

[0117] from tensorflow.keras.layers import Input, Conv2D, MaxPooling2D,GlobalMaxPooling2D, Dense, Flatten, LeakyReLU, ELU, concatenate

[0118] from tensorflow.keras.models import Model

[0119] # Input layer, used to receive pre - processed network data

[0120] # Assume the input data is a three - dimensional tensor with the shape (time steps, number of features, 1)

[0121] input_layer = Input(shape=(None, None, 1), name='input_layer')

[0122] # The first convolutional neural network sub - layer, used to extract traffic features in a short time

[0123] # Includes a convolutional layer, a pooling layer, and an activation layer

[0124] conv1 = Conv2D(filters = 32, kernel_size=(3, 3), strides=(1, 1),padding='same', name='conv1')(input_layer)

[0125] pool1 = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='same', name='pool1')(conv1)

[0126] act1 = tf.keras.layers.ReLU(name='act1')(pool1)

[0127] # The second convolutional neural network sub-layer for extracting behavioral features over a long period of time

[0128] # Including a convolutional layer, a pooling layer, and an activation layer

[0129] conv2 = Conv2D(filters=64, kernel_size=(5, 5), strides=(1, 1),padding='same', name='conv2')(input_layer)

[0130] pool2 = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), padding='same', name='pool2')(conv2)

[0131] act2 = LeakyReLU(alpha=0.1, name='act2')(pool2)

[0132] # The third convolutional neural network sub-layer for extracting the frequency features of specific sensitive data accesses

[0133] # Including a convolutional layer, a pooling layer, and an activation layer

[0134] conv3 = Conv2D(filters=128, kernel_size=(1, 1), strides=(1, 1),padding='same', name='conv3')(input_layer)

[0135] pool3 = GlobalMaxPooling2D(name='pool3')(conv3)

[0136] act3 = tf.keras.layers.Activation('sigmoid', name='act3')(pool3)

[0137] # The fourth convolutional neural network sub-layer for extracting abnormal network connection behavior features

[0138] # It includes a convolutional layer, a pooling layer, and an activation layer

[0139] conv4 = Conv2D(filters=256, kernel_size=(3, 3), strides=(1, 1),padding='same', name='conv4')(input_layer)

[0140] # Assume we use a custom random pooling layer

[0141] class RandomPooling2D(tf.keras.layers.Layer):

[0142] def __init__(self, pool_size=(2, 2), strides=(2, 2), padding='same', **kwargs):

[0143] super(RandomPooling2D, self).__init__(**kwargs)

[0144] self.pool_size = pool_size

[0145] self.strides = strides

[0146] self.padding = padding

[0147] def call(self, inputs):

[0148] batch, height, width, channels = tf.shape(inputs)

[0149] pooled_height = height / / self.pool_size[0]

[0150] pooled_width = width / / self.pool_size[1]

[0151] pooled = tf.image.random_crop(inputs, [batch, pooled_height,pooled_width, channels])

[0152] return pooled

[0153] pool4 = RandomPooling2D(pool_size=(2, 2), strides=(2, 2), padding='same', name='pool4')(conv4)

[0154] act4 = ELU(name='act4')(pool4)

[0155] # Build the model

[0156] model = Model(inputs=input_layer, outputs=feature_vectors, name='network_data_leakage_detection_model')

[0157] # Print the model summary

[0158] model.summary()

[0159] Furthermore, the feature fusion layer includes a feature weighting module, a feature fusion module, and a feature transformation module; wherein, the feature weighting module uses a fully connected layer and a Softmax function to calculate the weights of each feature vector, obtaining multiple weighted feature vectors; the feature fusion module combines multiple weighted feature vectors in a concatenated manner to generate multiple initially fused comprehensive feature vectors; the feature transformation module performs a linear transformation on each initially fused comprehensive feature vector through a fully connected layer and a ReLU activation function to obtain the fused comprehensive feature vectors.

[0160] The feature fusion layer includes a feature weighting module, a feature fusion module, and a feature transformation module. These modules work together to generate more representative comprehensive feature vectors from multiple feature vectors, enhancing the detection ability and accuracy of the system.

[0161] First, the main function of the feature weighting module is to calculate the weights of each feature vector through a fully connected layer and the Softmax function. Specifically, the input feature vector first passes through a fully connected layer, which generates a weight vector. Then, the Softmax function is used to normalize this weight vector so that the sum of all weights is 1. This step ensures that the contributions of different feature vectors can be dynamically adjusted according to their importance. In this way, the system can automatically learn and identify which features are more important in detecting data leakage, and thus assign them higher weights.

[0162] Next, the feature fusion module is responsible for combining the weighted feature vectors to generate multiple preliminary integrated feature vectors after fusion. In the specific implementation process, the feature fusion module uses the splicing method to splice all the weighted feature vectors in a specific dimension. The splicing operation combines multiple feature vectors into a larger vector, retaining the information of all original features. This method can not only effectively integrate features from different sources, but also enhance the richness of feature representation without losing information.

[0163] Finally, the feature transformation module performs a linear transformation on each preliminary integrated feature vector through a fully connected layer and the ReLU activation function. The role of the fully connected layer is to perform a linear transformation on the spliced preliminary integrated feature vector to generate a new feature representation. After the fully connected layer, the ReLU activation function is used to perform a non-linear processing on the result of the linear transformation. The ReLU activation function can introduce non-linear factors, improve the expression ability of the model, and at the same time solve the problem of gradient disappearance. Through this step, the preliminary integrated feature vectors are transformed into more abstract and high-dimensional integrated feature vectors, which will be used for further analysis in the subsequent graph convolutional network layer and anomaly pattern recognition layer.

[0164] The feature weighting module dynamically adjusts the weights of feature vectors through a fully connected layer and the Softmax function to ensure that important features receive higher attention. The feature fusion module combines the weighted feature vectors by splicing to generate preliminary integrated feature vectors. The feature transformation module further performs a linear transformation on the preliminary integrated feature vectors through a fully connected layer and the ReLU activation function to generate the final integrated feature vectors. These steps work together to ensure that the feature fusion layer can effectively integrate and enhance feature representation, providing strong support for subsequent anomaly detection.

[0165] The following is the detailed code for implementing the feature fusion layer in the neural network model, including the feature weighting module, the feature fusion module, and the feature transformation module.

[0166] import tensorflow as tf

[0167] from tensorflow.keras.layers import Dense, Softmax, Concatenate, ReLU

[0168] # Example tensor of input feature vectors

[0169] # Assume there are 4 feature vectors, each with a length of 128

[0170] feature_vector1 = tf.random.normal(shape=(1, 128))

[0171] feature_vector2 = tf.random.normal(shape=(1, 128))

[0172] feature_vector3 = tf.random.normal(shape=(1, 128))

[0173] feature_vector4 = tf.random.normal(shape=(1, 128))

[0174] # Feature weighting module

[0175] # Use a fully connected layer to calculate the weights of each feature vector and normalize through the Softmax function

[0176] def weight_features(feature_vector):

[0177] # The fully connected layer generates weights

[0178] dense_layer = Dense(units=128, activation=None)

[0179] weighted_vector = dense_layer(feature_vector)

[0180] # Calculate the Softmax of the weight vector

[0181] softmax_layer = Softmax(axis=-1)

[0182] weighted_vector = softmax_layer(weighted_vector)

[0183] return weighted_vector

[0184] # Calculate the weights for each feature vector

[0185] weighted_feature1 = weight_features(feature_vector1)

[0186] weighted_feature2 = weight_features(feature_vector2)

[0187] weighted_feature3 = weight_features(feature_vector3)

[0188] weighted_feature4 = weight_features(feature_vector4)

[0189] # Feature fusion module

[0190] # Concatenate the weighted feature vectors along a specific dimension to generate a preliminary combined feature vector

[0191] concatenated_features = Concatenate(axis=-1)([weighted_feature1,weighted_feature2, weighted_feature3, weighted_feature4])

[0192] # Feature transformation module

[0193] # Use a fully connected layer to perform a linear transformation on the concatenated preliminary combined feature vector and apply the ReLU activation function for non - linear processing

[0194] dense_transform = Dense(units=512, activation=None)(concatenated_features)

[0195] relu_transform = ReLU()(dense_transform)

[0196] # The final fused combined feature vector

[0197] fused_feature_vector = relu_transform

[0198] # Print the shape of the fused comprehensive feature vector to ensure correct implementation

[0199] print(fused_feature_vector.shape)

[0200] Through the above detailed code, the design and function of the feature fusion layer can be fully understood and implemented. The feature weighting module, feature fusion module, and feature transformation module work together to ensure that the feature fusion layer can effectively integrate and enhance the feature representation, providing strong support for subsequent anomaly detection.

[0201] Furthermore, the graph convolutional network layer includes a graph construction module, a graph convolutional layer, and a feature transformation module; wherein, the graph construction module is used to construct a graph structure representing the relationship between the comprehensive feature vectors and generate an adjacency matrix; the graph convolutional layer is used to perform a convolutional operation on the input comprehensive feature vectors and the adjacency matrix to generate a preliminary high-dimensional feature vector; the feature transformation module is used to transform the preliminary high-dimensional feature vector output by the graph convolutional layer through a fully connected layer and a non-linear activation function to obtain a high-dimensional feature vector.

[0202] In the network data leakage monitoring system, the graph convolutional network layer is an important component for processing and capturing the complex dependencies and topological relationships between comprehensive feature vectors. The implementation of the graph convolutional network layer includes a graph construction module, a graph convolutional layer, and a feature transformation module, and these modules work together to extract a higher-level feature representation from the input comprehensive feature vectors.

[0203] First of all, the graph construction module is responsible for constructing a graph structure representing the relationship between comprehensive feature vectors. This module generates an adjacency matrix by analyzing the relationship between comprehensive feature vectors. The adjacency matrix is a binary matrix, and the elements in it represent whether there is a connection between feature vectors. Specifically, the graph construction module can determine the connection relationship between feature vectors according to the similarity measure between feature vectors (such as Euclidean distance, cosine similarity, etc.) and generate the corresponding adjacency matrix. This step ensures that the relationship between comprehensive feature vectors can be fully considered and utilized in subsequent convolutional operations.

[0204] Next, the graph convolutional layer performs a convolution operation on the input comprehensive feature vector and the adjacency matrix. The main function of the graph convolutional layer is to propagate and aggregate information of the feature vector in the graph structure through graph convolution operations. In the specific implementation process, the graph convolutional layer receives the comprehensive feature vector and the adjacency matrix as inputs. Through multiple graph convolution operations, it performs a weighted average on the neighbor information of each feature vector and combines the result with its own feature to generate a preliminary high-dimensional feature vector. This graph convolution operation can effectively capture the global dependencies and context information between feature vectors, making the generated feature vector have stronger expressive power and robustness.

[0205] Finally, the feature transformation module transforms the preliminary high-dimensional feature vector output by the graph convolutional layer through a fully connected layer and a non-linear activation function. The role of the fully connected layer is to perform a linear transformation on the preliminary high-dimensional feature vector to generate a new feature representation. After the fully connected layer, a non-linear activation function is used to perform non-linear processing on the result of the linear transformation. Commonly used activation functions include ReLU, Leaky ReLU, ELU, etc. These activation functions can introduce non-linear factors, improve the expressive power of the model, and solve the problem of gradient disappearance. Through the feature transformation module, the preliminary high-dimensional feature vector is transformed into a more abstract and high-dimensional feature representation, and these high-dimensional feature vectors will be used for further analysis in the subsequent anomaly pattern recognition layer.

[0206] Through the above detailed implementation steps, the graph convolutional network layer can efficiently process and capture the complex dependencies and topological relationships between comprehensive feature vectors. The graph construction module generates the adjacency matrix to ensure that the relationships between feature vectors can be fully represented in the graph structure. The graph convolutional layer propagates and aggregates information of the feature vector in the graph structure through graph convolution operations to generate a preliminary high-dimensional feature vector. The feature transformation module performs further linear transformation and non-linear processing on the preliminary high-dimensional feature vector through a fully connected layer and a non-linear activation function to generate the final high-dimensional feature vector. These steps work together to ensure that the graph convolutional network layer can effectively improve the feature representation and provide strong support for subsequent anomaly detection.

[0207] The following is the detailed code for implementing the graph convolutional network layer in the neural network model, including the graph construction module, the graph convolutional layer, and the feature transformation module.

[0208] import tensorflow as tf

[0209] from tensorflow.keras.layers import Dense, ReLU

[0210] import numpy as np

[0211] # Assume the input comprehensive feature vector is a two-dimensional tensor of shape (num_nodes, feature_dim)

[0212] # num_nodes represents the number of nodes, and feature_dim represents the dimension of each feature vector

[0213] input_features = tf.random.normal(shape=(10, 128))

[0214] # Graph construction module

[0215] # Generate a random adjacency matrix of shape (num_nodes, num_nodes)

[0216] # The elements of the adjacency matrix represent whether there is a connection between nodes, 1 means connected, 0 means not connected

[0217] adjacency_matrix = np.random.randint(0, 2, size=(10, 10))

[0218] # Convert the adjacency matrix to a tensor

[0219] adjacency_matrix = tf.convert_to_tensor(adjacency_matrix, dtype=tf.float32)

[0220] # Graph convolution layer

[0221] # Define a function for the graph convolution layer that takes the comprehensive feature vector and the adjacency matrix as inputs

[0222] def graph_convolution(features, adjacency, output_dim):

[0223] # Feature transformation matrix, initially random values

[0224] W = tf.Variable(tf.random.truncated_normal([features.shape[1],output_dim], stddev=0.1))

[0225] # Perform graph convolution operation, A_hat = D^(-1 / 2) * A * D^(-1 / 2) is the normalized adjacency matrix

[0226] D = tf.linalg.diag(tf.reduce_sum(adjacency, axis = 1))

[0227] D_inv_sqrt = tf.linalg.diag(1.0 / tf.sqrt(tf.reduce_sum(adjacency, axis = 1)+1e - 5))

[0228] A_hat = tf.matmul(tf.matmul(D_inv_sqrt, adjacency), D_inv_sqrt)

[0229] # Feature convolution, H' = A_hat * H * W

[0230] features_transformed = tf.matmul(tf.matmul(A_hat, features), W)

[0231] return features_transformed

[0232] # Perform graph convolution operation to generate preliminary high - dimensional feature vectors

[0233] graph_conv_output = graph_convolution(input_features, adjacency_matrix, output_dim = 256)

[0234] # Feature transformation module

[0235] # Use a fully - connected layer to linearly transform the preliminary high - dimensional feature vectors output by the graph convolution layer and use the ReLU activation function for non - linear processing

[0236] dense_transform = Dense(units = 512, activation = None)(graph_conv_output)

[0237] relu_transform = ReLU()(dense_transform)

[0238] # Final high - dimensional feature vectors

[0239] high_dimensional_features = relu_transform

[0240] # Print the shape of the high-dimensional feature vector to ensure correct implementation

[0241] print(high_dimensional_features.shape)

[0242] Through the above code, the design and function of the graph convolutional network layer can be fully understood and implemented. The graph construction module, graph convolutional layer, and feature transformation module work together to ensure that the graph convolutional network layer can effectively process and capture the complex dependencies and topological relationships between comprehensive feature vectors, providing strong support for subsequent anomaly detection.

[0243] Furthermore, the anomaly pattern recognition layer includes a time series data preprocessing module, a recurrent neural network layer, an anomaly pattern detection module, and a recognition output module; wherein, the time series data preprocessing module is used to segment the high-dimensional feature vector in chronological order to form multiple time series segments; the recurrent neural network layer is used to process the time series segments to capture the dependencies and patterns in the time series segments; wherein, the recurrent neural network layer includes a first recurrent neural network and a second recurrent neural network, the first recurrent neural network is implemented using long short-term memory units, and the second recurrent neural network is implemented using gated recurrent units; the anomaly pattern detection module detects potential anomaly patterns and assigns preliminary confidence scores based on the output of the recurrent neural network layer; the recognition output module is used to output the detected potential anomaly patterns and the preliminary confidence scores of the potential anomaly patterns.

[0244] In the network data leakage monitoring system, the anomaly pattern recognition layer plays a key role. This layer effectively identifies potential anomaly patterns and assigns confidence scores through the collaborative work of the time series data preprocessing module, recurrent neural network layer, anomaly pattern detection module, and recognition output module.

[0245] First of all, the main function of the time series data preprocessing module is to segment the high-dimensional feature vector in chronological order to form multiple time series segments. This step analyzes the input high-dimensional feature vector and divides the continuous data into segments of fixed length according to timestamps or other time markers. For example, if the high-dimensional feature vector is the network traffic feature generated once per second, the feature vectors within one minute can be divided into sixty time series segments. This process ensures that subsequent time series analysis can be carried out based on the accurate chronological order.

[0246] Next, the recurrent neural network layer is used to process these time series segments to capture the dependencies and patterns within them. The recurrent neural network layer consists of two different recurrent neural networks. The first recurrent neural network is implemented using long short-term memory units (LSTM). LSTM can effectively capture the long-term dependencies in time series data and avoid the vanishing gradient problem in traditional RNNs. Each time series segment is input into the LSTM unit, and through the processing of multiple layers of LSTM units, features reflecting the time series dependencies are extracted.

[0247] After the LSTM layer, the processed feature vectors are passed to the second recurrent neural network. The second recurrent neural network is implemented using gated recurrent units (GRU). GRU is similar to LSTM, but its structure is more concise and its computational efficiency is higher. Through the processing of GRU units, the dependencies and patterns in time series data are further extracted and enhanced. GRU can effectively capture short-term and medium-term dependencies and, together with LSTM, comprehensively analyze various dependencies in time series data.

[0248] Subsequently, the anomaly pattern detection module detects potential anomaly patterns based on the output of the recurrent neural network layer. This module analyzes the feature vectors generated by the LSTM and GRU layers to identify possible abnormal behaviors. For example, an abnormal sudden increase in network traffic in a certain time series segment or frequent access to sensitive data may be marked as an anomaly pattern. For each detected anomaly pattern, the anomaly pattern detection module assigns a preliminary confidence score. The confidence score reflects the likelihood that the pattern is determined to be an anomaly, and the higher the score, the greater the likelihood of an anomaly.

[0249] Finally, the recognition output module is used to output the detected potential anomaly patterns and their preliminary confidence scores. This module organizes and formats the results of the anomaly pattern detection module for output. The output information includes a detailed description of each potential anomaly pattern, the occurrence time, and the preliminary confidence score. This information will be passed to other parts of the system for further analysis and response measures.

[0250] Through these steps, the anomaly pattern recognition layer can efficiently and accurately identify potential anomaly patterns and assign confidence scores to each pattern.

[0251] The following is the detailed code for implementing the anomaly pattern recognition layer in the neural network model, including the time series data preprocessing module, the recurrent neural network layer, the anomaly pattern detection module, and the recognition output module.

[0252] import tensorflow as tf

[0253] from tensorflow.keras.layers import LSTM, GRU, Dense

[0254] import numpy as np

[0255] # Time series data preprocessing module

[0256] # Segment the high-dimensional feature vectors into multiple time series segments in chronological order

[0257] def preprocess_time_series(features, segment_length):

[0258] num_segments = features.shape[0] / / segment_length

[0259] time_series_segments = []

[0260] for i in range(num_segments):

[0261] segment = features[i*segment_length:(i+1)*segment_length]

[0262] time_series_segments.append(segment)

[0263] return np.array(time_series_segments)

[0264] # Assume the input high-dimensional feature vector is a two-dimensional tensor with a shape of (600, 128)

[0265] # Where 600 represents the time step and 128 represents the feature dimension

[0266] input_features = tf.random.normal(shape=(600, 128))

[0267] # Segment the high-dimensional feature vector into segments every minute (60 seconds)

[0268] segment_length = 60

[0269] time_series_segments = preprocess_time_series(input_features, segment_length)

[0270] # Convert time series segments to TensorFlow tensors

[0271] time_series_segments = tf.convert_to_tensor(time_series_segments, dtype=tf.float32)

[0272] # Recurrent neural network layer

[0273] # The first recurrent neural network layer is implemented using LSTM

[0274] lstm_layer = LSTM(units=64, return_sequences=True, name='lstm_layer')

[0275] lstm_output = lstm_layer(time_series_segments)

[0276] # The second recurrent neural network layer is implemented using GRU

[0277] gru_layer = GRU(units=64, return_sequences=False, name='gru_layer')

[0278] gru_output = gru_layer(lstm_output)

[0279] # Anomaly pattern detection module

[0280] # Detect potential anomaly patterns and assign preliminary confidence scores based on the output of the recurrent neural network layer

[0281] class AnomalyDetectionModule(tf.keras.layers.Layer):

[0282] def __init__(self, units=1):

[0283] super(AnomalyDetectionModule, self).__init__()

[0284] self.dense = Dense(units, activation='sigmoid')

[0285] def call(self, inputs):

[0286] confidence_scores = self.dense(inputs)

[0287] return confidence_scores

[0288] # Create an instance of the anomaly detection module

[0289] anomaly_detection_module = AnomalyDetectionModule(units=1)

[0290] # Perform anomaly detection on the output of the GRU layer

[0291] anomaly_scores = anomaly_detection_module(gru_output)

[0292] # Recognition output module

[0293] # Output the detected potential anomaly patterns and their preliminary confidence scores

[0294] class RecognitionOutputModule(tf.keras.layers.Layer):

[0295] def __init__(self):

[0296] super(RecognitionOutputModule, self).__init__()

[0297] def call(self, scores):

[0298] # For simplicity, directly return the anomaly scores here. In actual applications, further processing and formatting of the output may be required

[0299] return scores

[0300] # Create an instance of the recognition output module

[0301] recognition_output_module = RecognitionOutputModule()

[0302] # Obtain the final output anomaly patterns and their confidence scores

[0303] final_output = recognition_output_module(anomaly_scores)

[0304] # Print the shape of the final output to ensure correct implementation

[0305] print(final_output.shape)

[0306] Through the above detailed code, the design and function of the anomaly pattern recognition layer can be fully understood and implemented. The time series data preprocessing module, recurrent neural network layer, anomaly pattern detection module, and recognition output module work together to ensure that the anomaly pattern recognition layer can efficiently and accurately identify potential anomaly patterns and assign confidence scores, providing support for subsequent anomaly verification and processing.

[0307] Furthermore, the anomaly verification layer includes a generator network, a discriminator network, an anomaly verification module, and a verification output module; wherein, the generator network is used to generate new feature samples based on potential anomaly patterns; the discriminator network is used to discriminate between the generated new feature samples and real feature samples and evaluate their authenticity; the anomaly verification module is used to comprehensively consider the results of the generator network and the discriminator network to verify potential anomaly patterns and assign confidence scores; the verification output module is used to output the verified anomaly patterns and the confidence scores of the anomaly patterns.

[0308] In the network data leakage monitoring system, the anomaly verification layer plays a crucial role. Through the collaborative work of the generator network, discriminator network, anomaly verification module, and verification output module, this layer ensures further verification of the detected potential anomaly patterns and the assignment of confidence scores, thereby improving the accuracy and reliability of anomaly detection.

[0309] First of all, the main function of the generator network is to generate new feature samples based on potential anomaly patterns. The generator network usually adopts the architecture of a generative adversarial network (GAN), receives potential anomaly patterns as input, and through the transformation of a multi-layer neural network, generates new feature samples similar to real feature samples. The design purpose of the generator network is to capture the feature distribution of potential anomaly patterns and verify the authenticity of these patterns through the generated samples.

[0310] Next, the discriminator network is used to discriminate between the newly generated feature samples and the real feature samples to evaluate their authenticity. The discriminator network is also part of the generative adversarial network (GAN). It takes as input the samples generated by the generator network and the real feature samples, and outputs a probability value representing the authenticity of the input samples. After training, the discriminator network can effectively distinguish between real samples and generated samples, providing a reliable evaluation basis for anomaly verification.

[0311] The anomaly verification module combines the results of the generator network and the discriminator network to verify potential anomaly patterns and assign confidence scores. Specifically, the anomaly verification module evaluates each potential anomaly pattern by analyzing the output results of the discriminator network. If the discriminator network believes that the generated samples are very similar to the real samples, it indicates that the authenticity of the potential anomaly pattern is high, and the anomaly verification module will assign a high confidence score to it. On the contrary, if the generated samples are quite different from the real samples, the confidence score of the potential anomaly pattern will be low. In this way, the anomaly verification module can effectively screen out real anomaly patterns and quantify their confidence levels.

[0312] Finally, the verification output module is used to output the verified anomaly patterns and their confidence scores. This module organizes and formats the results of the anomaly verification module for output, including the detailed description, occurrence time, and final confidence score of each verified anomaly pattern. The output information will be passed to other parts of the system for further analysis and response measures.

[0313] Through the above detailed implementation steps, the anomaly verification layer can efficiently verify potential anomaly patterns and assign confidence scores. The generator network generates new feature samples based on potential anomaly patterns, the discriminator network evaluates these samples, the anomaly verification module combines the results of both for verification, and the verification output module outputs the final anomaly patterns and their confidence scores.

[0314] The following is the detailed code for implementing the anomaly verification layer in the neural network model, including the generator network, discriminator network, anomaly verification module, and verification output module.

[0315] import tensorflow as tf

[0316] from tensorflow.keras.layers import Dense, LeakyReLU,BatchNormalization, Input

[0317] from tensorflow.keras.models import Model

[0318] # Generator Network

[0319] # Generate new feature samples based on potential anomaly patterns

[0320] def build_generator(input_dim, output_dim):

[0321] input_layer = Input(shape=(input_dim,))

[0322] x = Dense(128)(input_layer)

[0323] x = LeakyReLU(alpha=0.2)(x)

[0324] x = BatchNormalization(momentum=0.8)(x)

[0325] x = Dense(256)(x)

[0326] x = LeakyReLU(alpha=0.2)(x)

[0327] x = BatchNormalization(momentum=0.8)(x)

[0328] x = Dense(512)(x)

[0329] x = LeakyReLU(alpha=0.2)(x)

[0330] x = BatchNormalization(momentum=0.8)(x)

[0331] output_layer = Dense(output_dim, activation='tanh')(x)

[0332] return Model(input_layer, output_layer, name='Generator')

[0333] # Discriminator Network

[0334] # Discriminate between the generated new feature samples and the real feature samples to evaluate their authenticity

[0335] def build_discriminator(input_dim):

[0336] input_layer = Input(shape=(input_dim,))

[0337] x = Dense(512)(input_layer)

[0338] x = LeakyReLU(alpha=0.2)(x)

[0339] x = Dense(256)(x)

[0340] x = LeakyReLU(alpha=0.2)(x)

[0341] output_layer = Dense(1, activation='sigmoid')(x)

[0342] return Model(input_layer, output_layer, name='Discriminator')

[0343] # Anomaly Validation Module

[0344] # Combine the results of the generator network and the discriminator network to validate potential anomaly patterns and assign confidence scores

[0345] class AnomalyValidationModule(tf.keras.layers.Layer):

[0346] def __init__(self, generator, discriminator):

[0347] super(AnomalyValidationModule, self).__init__()

[0348] self.generator = generator

[0349] self.discriminator = discriminator

[0350] def call(self, potential_anomaly):

[0351] # The generator generates new feature samples

[0352] generated_sample = self.generator(potential_anomaly)

[0353] # Evaluate the discriminator on the generated sample

[0354] real_score = self.discriminator(potential_anomaly)

[0355] fake_score = self.discriminator(generated_sample)

[0356] # Calculate the confidence score

[0357] confidence_score = real_score - fake_score

[0358] return confidence_score

[0359] # Validation output module

[0360] # Output the verified anomaly patterns and their confidence scores

[0361] class ValidationOutputModule(tf.keras.layers.Layer):

[0362] def __init__(self):

[0363] super(ValidationOutputModule, self).__init__()

[0364] def call(self, confidence_scores):

[0365] # For simplicity, directly return the confidence scores here. In actual applications, further processing and formatting of the output may be required

[0366] return confidence_scores

[0367] # Assume the input potential anomaly pattern is a 2D tensor with shape (10, 100)

[0368] # where 10 represents the number of samples and 100 represents the dimension of the potential anomaly pattern

[0369] potential_anomalies = tf.random.normal(shape=(10, 100))

[0370] # Build the generator and discriminator

[0371] generator = build_generator(input_dim=100, output_dim=128)

[0372] discriminator = build_discriminator(input_dim=128)

[0373] # Create an instance of the anomaly validation module

[0374] anomaly_validation_module = AnomalyValidationModule(generator,discriminator)

[0375] # Validate the potential anomaly patterns and generate confidence scores

[0376] confidence_scores = anomaly_validation_module(potential_anomalies)

[0377] # Create an instance of the validation output module

[0378] validation_output_module = ValidationOutputModule()

[0379] # Obtain the final output anomaly patterns and their confidence scores

[0380] final_output = validation_output_module(confidence_scores)

[0381] # Print the shape of the final output to ensure correct implementation

[0382] print(final_output.shape)

[0383] Through the above detailed code, the design and function of the anomaly verification layer can be fully understood and implemented. The generator network generates new feature samples based on potential anomaly patterns, the discriminator network evaluates these samples, the anomaly verification module comprehensively verifies the results of both, and outputs the final anomaly pattern and its confidence score through the verification output module, thereby improving the detection accuracy and reliability of the network data leakage monitoring system.

[0384] In the network data leakage monitoring system, the training of the neural network model is a multi-stage and iterative process, involving a series of steps from data collection to model deployment. This process ensures that the model can effectively learn and adapt to the dynamic changes of network data to achieve optimized anomaly detection and protection. The following are the specific steps of model training:

[0385] First of all, the system needs to collect various data during network operation, including network traffic data, access logs, user behavior records, etc. These data are collected in real time through the data acquisition module and processed by the data preprocessing module for cleaning, denoising, normalization, etc., to ensure the quality and consistency of the data and lay a solid foundation for subsequent model training.

[0386] In the feature extraction layer training stage, the system uses multiple parallel convolutional neural network sub-layers to process the preprocessed network data. Each convolutional neural network sub-layer focuses on extracting different types of features, such as traffic features in a short period of time, behavior features in a long period of time, frequency features of specific sensitive data access, abnormal network connection behavior features, etc. In this way, the system can comprehensively capture various complex patterns in network data and generate multiple feature vectors.

[0387] In the feature fusion layer training stage, the system adopts an attention mechanism to weight and fuse the feature vectors extracted by multiple convolutional neural network sub-layers. The feature weighting module uses a fully connected layer and the Softmax function to calculate the weight of each feature vector and weight the feature vectors according to the weight. The weighted feature vectors are concatenated through the feature fusion module to generate multiple preliminary comprehensive feature vectors. Then, the feature transformation module performs linear transformation and non-linear processing on the preliminary comprehensive feature vectors through a fully connected layer and the ReLU activation function to generate the final fused comprehensive feature vectors.

[0388] In the graph convolutional network layer training stage, the system first uses the graph construction module to generate a graph structure representing the relationship between comprehensive feature vectors and generate an adjacency matrix. The graph convolutional layer performs a convolutional operation on the input comprehensive feature vectors and the adjacency matrix, captures the complex dependencies and topological relationships between the feature vectors through the graph convolutional operation, and generates preliminary high-dimensional feature vectors. Then, the feature transformation module performs further linear transformation and non-linear processing on the preliminary high-dimensional feature vectors output by the graph convolutional layer to generate high-dimensional feature vectors.

[0389] During the training phase of the abnormal pattern recognition layer, the system uses a recurrent neural network layer to perform time series analysis on high-dimensional feature vectors. The first recurrent neural network layer is implemented using long short-term memory units (LSTMs), which can capture long-term dependencies in time series data. The second recurrent neural network layer is implemented using gated recurrent units (GRUs) to further extract and enhance the dependencies and patterns in the time series data. The abnormal pattern detection module detects potential abnormal patterns and assigns preliminary confidence scores based on the output of the recurrent neural network layer.

[0390] During the training phase of the abnormal verification layer, the system uses a generative adversarial network (GAN) to verify potential abnormal patterns. The generator network generates new feature samples based on the potential abnormal patterns, and the discriminator network discriminates between the generated samples and real samples to evaluate their authenticity. The abnormal verification module synthesizes the results of the generator network and the discriminator network to verify the potential abnormal patterns and assigns final confidence scores. The verification output module outputs the verified abnormal patterns and their confidence scores.

[0391] After training, the model needs to undergo rigorous evaluation and testing. Cross-validation and various performance metrics (such as accuracy, response time, etc.) are used to verify the effectiveness and reliability of the model, ensuring that its performance in actual applications meets the system requirements.

[0392] The model verified through evaluation and testing will be deployed into the network data leakage monitoring system to process network traffic and user behavior data in real time, and dynamically detect and protect potential data leakage risks based on the results predicted by the model.

[0393] After deployment, the system will continuously monitor the performance and actual application effects of the model, collect new operation data for iterative update and optimization of the model, ensuring that the model can adapt to environmental changes and new threats, and continuously provide an efficient and accurate network data leakage protection solution.

[0394] Through this detailed training and implementation process, the network data leakage monitoring system can make full use of the powerful learning and prediction capabilities of neural networks, continuously optimize and improve the detection and protection performance of the system, and achieve efficient network security protection.

[0395] The risk assessment module 104 performs real-time assessment of potential data leakage risks based on the type and confidence score of the abnormal pattern, and obtains a comprehensive risk assessment result.

[0396] The risk assessment module 104 plays a crucial role in the network data leakage monitoring system. It is responsible for analyzing and evaluating the detected abnormal patterns to determine the severity of potential data leakage risks and generating a comprehensive risk assessment result. This module utilizes the abnormal patterns and their confidence scores provided by the anomaly detection module 103, through complex algorithms and multi-dimensional analysis, to ensure the accuracy and real-time nature of the risk assessment.

[0397] First, the risk assessment module 104 receives inputs from the anomaly detection module 103, which include the detected abnormal patterns and their corresponding confidence scores. Each abnormal pattern is attached with a confidence score, indicating the credibility of that abnormal pattern. The risk assessment module needs to comprehensively consider this information and conduct a detailed analysis of each abnormal pattern.

[0398] In the specific implementation process, the risk assessment module 104 first assigns a risk weight to each type of abnormal pattern. The risk weight is predefined according to the type of abnormal pattern, representing the degree of impact of this type of abnormal pattern on the overall network security. For example, an abnormal increase in data transfer volume may have a relatively high risk weight, while an abnormal frequency of accessing specific sensitive data may have an even higher weight because the latter may directly involve the leakage of sensitive information.

[0399] Next, the risk assessment module calculates the comprehensive risk value for each abnormal pattern. This calculation process not only considers the confidence score and risk weight of the abnormal pattern but also comprehensively considers other relevant factors, such as the access frequency of the abnormal pattern, the duration length, the user impact coefficient, the priority coefficient, and the detection difficulty coefficient, etc. The access frequency reflects the number of times the abnormal pattern is detected within a certain period of time, the duration length represents the duration of the abnormal behavior, the user impact coefficient evaluates the potential impact of the abnormal pattern on users, the priority coefficient determines the priority order for handling this abnormal pattern, and the detection difficulty coefficient reflects the difficulty level of detecting this abnormal pattern.

[0400] By integrating these factors, the risk assessment module can calculate a weighted comprehensive risk value to ensure the comprehensiveness and accuracy of the assessment result. For example, the calculation of the comprehensive risk value can be achieved through a polynomial formula that sums up all relevant parameters with weights to obtain the final risk assessment result. The higher the calculated comprehensive risk value, the greater the potential data leakage risk.

[0401] Finally, the risk assessment module aggregates the comprehensive risk values of all detected abnormal patterns to generate an overall comprehensive risk assessment result. This result not only provides an overall assessment of the current network security status but also can divide the risk levels according to the specific comprehensive risk values, such as low risk, medium risk, and high risk, etc., for subsequent decision-making and handling.

[0402] The risk assessment module 104 ensures high accuracy and real-time performance in the assessment of potential data leakage risks through the above detailed and explicit steps.

[0403] Furthermore, the risk assessment module calculates the comprehensive risk assessment result according to Formula 1 as follows:

[0404]

[0405] Where, is the comprehensive risk assessment result; is the total number of detected abnormal patterns; is the risk weight of the th type of abnormal pattern; is the confidence score of the th type of abnormal pattern; is the severity score of the th type of abnormal pattern, with a scoring range of 1 to 10, set by the system based on historical data and expert judgment; is the frequency of the th type of abnormal pattern, indicating the number of times this pattern is detected within a certain period of time; is the duration length of the th type of abnormal pattern; is a specific threshold used to normalize the duration length of the th type of abnormal pattern.

[0406] In the network data leakage monitoring system, the risk assessment module is responsible for calculating the comprehensive risk assessment result based on the detected abnormal patterns. This module uses Formula (1) for calculation to ensure a comprehensive assessment of different types of abnormal patterns, thereby providing an accurate risk score. The following is a detailed description of this formula and the meaning and calculation method of each term.

[0407]

[0408] Comprehensive risk assessment result :

[0409] This is the finally calculated comprehensive risk assessment result, used to represent the overall risk level in the current network state. It is obtained by accumulating the risk values of all detected abnormal patterns.

[0410] Total number of detected abnormal patterns :

[0411] This is the total number of all abnormal patterns detected by the system within a specific time window.

[0412] The th type of abnormal pattern risk weight :

[0413] This is a preset constant used to represent the risk weight of the th anomaly pattern. The weight is usually set based on experience and historical data, indicating the degree of impact of this anomaly pattern on the overall network security. The higher the weight, the greater the threat of this type of anomaly pattern to network security.

[0414] The th anomaly pattern's confidence score :

[0415] This is the confidence score assigned by the model to the th anomaly pattern after the system detects it. The range is usually between 0 and 1. The higher the confidence score, the greater the likelihood that this anomaly pattern is detected as a real anomaly.

[0416] The th anomaly pattern's severity score :

[0417] This is a score set based on historical data and expert judgment, used to represent the severity of the th anomaly pattern. The score range is from 1 to 10. The higher the score, the greater the potential harm of this anomaly pattern to the system.

[0418] The th anomaly pattern's frequency :

[0419] This is the number of times the th anomaly pattern is detected within a certain period of time. The higher the frequency, the more frequently this anomaly pattern appears, and the greater the possible impact on the system.

[0420] The th anomaly pattern's duration length :

[0421] This is the duration length of each occurrence of the th anomaly pattern, and the unit can be seconds, minutes, or other appropriate time units. The longer the duration, the longer the impact time of this anomaly pattern on the system.

[0422] Specific threshold :

[0423] This is the threshold used to normalize the duration length of the th anomaly pattern, and the unit is the same as . Through normalization, it is ensured that anomaly patterns with different durations can be treated fairly in the evaluation.

[0424] The calculation methods for each parameter of the formula are as follows:

[0425] 1. Determination of risk weight :

[0426] It can be set based on the analysis of historical security events, expert experience, and the evaluation of different types of abnormal patterns. For example, the risk weight for a cyber attack may be higher than that for a general access anomaly.

[0427] 2. Calculation of confidence score :

[0428] It is output by the anomaly detection module according to the pre-trained neural network model, indicating the credibility of the currently detected abnormal pattern. The specific calculation method is the output score of the detection model.

[0429] 3. Setting of severity score :

[0430] It is set by analyzing historical data, event impacts, and expert evaluations. For abnormal patterns that have caused serious consequences in history, the score will be set higher.

[0431] 4. Calculation of frequency :

[0432] Statistically count the number of times the th abnormal pattern is detected within a specific time window.

[0433] 5. Measurement of duration length :

[0434] Each time an abnormal pattern is detected, record its start and end times and calculate its duration length.

[0435] 6. Setting of specific threshold :

[0436] Set a benchmark value according to historical data and experience for normalization processing. For example, if most anomalies last for 5 minutes, then = 300 seconds can be set.

[0437] Through these steps and calculations, the system can accurately evaluate the overall risk level in the current network environment and take corresponding protective measures.

[0438] The alarm module 105 is used to generate and send an alarm message when the comprehensive risk assessment result exceeds a preset threshold. The alarm message includes the abnormal pattern description, risk level, and recommended countermeasures.

[0439] The alarm module 105 plays a crucial role in the network data leakage monitoring system. When the comprehensive risk assessment result exceeds the preset threshold, this module is responsible for generating and sending alarm information so as to take countermeasures in a timely manner to prevent the further spread and damage of data leakage. Its implementation requires the combination of various technical means to ensure the accuracy, timeliness and operability of the alarm information.

[0440] The alarm module 105 first receives the comprehensive risk assessment result from the risk assessment module 104. When the comprehensive risk assessment result exceeds the preset threshold, the alarm module starts to operate. The preset threshold is defined according to the system's security policy and specific application environment, and can be a fixed value or adjusted according to historical data and dynamic analysis results.

[0441] When generating the alarm information, the alarm module 105 will describe in detail the detected abnormal patterns, including information such as the type of abnormal pattern, specific manifestations and detection time. For example, if an abnormal increase in data transmission volume is detected, the alarm information will include a detailed description of this abnormal pattern, such as the specific value of the abnormal transmission volume, the source and target addresses involved, the transmission protocol, etc.

[0442] At the same time, the alarm information also includes the risk level. The risk level is determined based on the comprehensive risk assessment result and can be divided into multiple levels such as low, medium, and high. Each level represents a different degree of risk severity. Low risk may only require monitoring and recording, while high risk requires immediate countermeasures. The division of the risk level makes the alarm information more operable and helps relevant personnel take corresponding actions according to the level of the risk.

[0443] In addition, the alarm module 105 also provides recommended countermeasures. These measures are predefined operation plans according to the type of abnormal pattern and the risk level. For example, for high-risk abnormal data transmission behaviors, the recommended countermeasures may include immediately isolating the affected network nodes, conducting detailed traffic analysis, checking system logs to find more clues, etc. Providing specific countermeasures can help relevant personnel react quickly and avoid delaying the processing time due to information asymmetry.

[0444] The alarm module 105 not only generates alarm information but also is responsible for sending this information through multiple channels. The sending channels can include email, text message, instant messaging applications, system notifications, etc. Selecting appropriate sending channels can ensure that the alarm information can be quickly conveyed to relevant personnel, so as to take necessary countermeasures in a timely manner. Different sending channels can be configured according to the actual application scenario to meet the needs in different environments.

[0445] To ensure the integrity and traceability of alarm information, the alarm module 105 also records the detailed information of each alarm, including the generation time, sending channel, recipient, and feedback situation, etc. These records not only contribute to subsequent security audits and analyses but also can serve as an important reference basis for improving the system security policy and optimizing the alarm mechanism.

[0446] Through the above detailed implementation method, the alarm module 105 can effectively generate and send alarm information, ensuring that the network data leakage monitoring system can take timely countermeasures when detecting potential risks.

[0447] In the above embodiments, a network data leakage monitoring system is provided. Correspondingly, the present application also provides a network data leakage monitoring method. Please refer to Figure 2 , which is a flowchart of an embodiment of a network data leakage monitoring method of the present application. Since this embodiment, that is, the second embodiment, is basically similar to the first embodiment, the description is relatively simple. For related parts, refer to the partial description of the first embodiment. The method embodiments described below are only illustrative.

[0448] A network data leakage monitoring method provided by the second embodiment of the present application includes:

[0449] Step S201: Real-time collect network data from network data sources;

[0450] Step S202: Clean and standardize the network data collected in the data collection step to obtain preprocessed network data;

[0451] Step S203: According to the preprocessed network data, use a pre-trained neural network model to obtain the abnormal patterns of the network data and the confidence scores of the abnormal patterns. Among them, the abnormal patterns include an abnormal increase in data transmission volume, an abnormal access frequency of specific sensitive data, abnormal network connection behaviors, and network behaviors matching known attack patterns;

[0452] Step S204: According to the types and confidence scores of the abnormal patterns, conduct real-time assessment of potential data leakage risks to obtain a comprehensive risk assessment result;

[0453] Step S205: When the comprehensive risk assessment result exceeds a preset threshold, generate and send an alarm message, where the alarm message includes a description of the abnormal pattern, a risk level, and recommended countermeasures.

[0454] Although the present application is disclosed above with preferred embodiments, it is not used to limit the present application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application should be defined by the scope defined in the claims of the present application.

Claims

1. A network data leakage monitoring system, characterized in that: include: A data collection module is used to collect network data from network data sources in real time; A data preprocessing module is used to clean and standardize the network data collected by the data collection module to obtain preprocessed network data; An anomaly detection module, used to obtain an abnormal pattern of network data and a confidence score of the abnormal pattern based on the pre-processed network data using a pre-trained neural network model, wherein the abnormal pattern includes an abnormal increase in data transmission volume, an abnormal frequency of access to specific sensitive data, an abnormal network connection behavior, and a network behavior matching a known attack pattern; A risk assessment module, which performs real-time assessment of potential data leakage risks based on the type and confidence score of the abnormal pattern to obtain a comprehensive risk assessment result; An alarm module is used to generate and send an alarm message when the comprehensive risk assessment result exceeds a preset threshold, wherein the alarm message includes an abnormal pattern description, a risk level, and a recommended response measure; The risk assessment module calculates the comprehensive risk assessment result according to the following formula (1): in, The results of the comprehensive risk assessment; is the total number of abnormal patterns detected; For the Risk weights for each type of abnormal pattern; For the confidence scores of the anomaly patterns; For the A severity score for each abnormal pattern, ranging from 1 to 10, set by the system based on historical data and expert judgment; For the The frequency of an abnormal pattern, which indicates the number of times the pattern is detected within a certain period of time; For the the duration of the abnormal pattern; is a specific threshold used to normalize the the duration of the abnormal pattern; The neural network model used in the anomaly detection module includes an input layer, a feature extraction layer, a feature fusion layer, a graph convolutional network layer, an abnormal pattern recognition layer and an abnormal verification layer; wherein the input layer is used to receive the preprocessed network data; the feature extraction layer includes multiple parallel convolutional neural network sublayers, and the input of each convolutional neural network sublayer is the preprocessed data provided by the input layer; each convolutional neural network sublayer processes the input data to obtain a feature vector; the feature fusion layer is implemented by an attention mechanism, which is used to fuse the feature vectors provided by multiple convolutional neural network sublayers to generate a fused comprehensive feature vector; the graph convolutional network layer is implemented by a graph convolutional network, which is used to process and capture the complex dependencies and topological relationships between the fused comprehensive feature vectors in a non-Euclidean way to generate a high-dimensional feature vector; the abnormal pattern recognition layer is implemented by a recurrent neural network, which is used to perform time series analysis on the high-dimensional feature vector, identify potential abnormal patterns, and assign a preliminary confidence score to each potential abnormal pattern; the abnormal verification layer is implemented by a generative adversarial network, and obtains abnormal patterns and confidence scores according to the potential abnormal patterns and preliminary confidence scores provided by the abnormal pattern recognition layer.

2. The network data leakage monitoring system according to claim 1, characterized in that: The feature extraction layer includes the following multiple parallel convolutional neural network sublayers: The first convolutional neural network sublayer is used to extract short-term traffic features from the input preprocessed network data, including the first convolution layer, the first pooling layer, and the first activation layer; wherein the convolution kernel size used in the first convolution layer is 3×3, and the step size is 1; the pooling kernel size used in the first pooling layer is 2×2, and the step size is 2; the first activation layer uses the ReLU activation function; The second convolutional neural network sublayer is used to extract long-term behavioral features from the input preprocessed network data, including a second convolutional layer, a second pooling layer, and a second activation layer; wherein the convolution kernel size used in the second convolutional layer is 5×5, and the step size is 1; the pooling kernel size used in the second pooling layer is 2×2, and the step size is 2; the second activation layer uses a Leaky ReLU activation function; The third convolutional neural network sublayer is used to extract the frequency characteristics of specific sensitive data access from the input preprocessed network data, including a third convolution layer, a third pooling layer and a third activation layer; wherein the convolution kernel size used in the third convolution layer is 1×1 and the step size is 1; the third pooling layer uses global maximum pooling; and the third activation layer uses a Sigmoid activation function; The fourth convolutional neural network sublayer is used to extract abnormal network connection behavior features from the input preprocessed network data, including a fourth convolutional layer, a fourth pooling layer and a fourth activation layer; wherein the convolution kernel size used in the fourth convolutional layer is 3×3 and the step size is 1; the fourth pooling layer uses random pooling; and the fourth activation layer uses an ELU activation function.

3. The network data leakage monitoring system according to claim 1, characterized in that: The feature fusion layer includes a feature weighting module, a feature fusion module and a feature transformation module; wherein the feature weighting module uses a fully connected layer and a Softmax function to calculate the weight of each feature vector to obtain multiple weighted feature vectors; the feature fusion module combines multiple weighted feature vectors in a splicing manner to generate multiple fused preliminary comprehensive feature vectors; the feature transformation module performs a linear transformation on each preliminary comprehensive feature vector through a fully connected layer and a ReLU activation function to obtain a fused comprehensive feature vector.

4. The network data leakage monitoring system according to claim 1, characterized in that: The graph convolution network layer includes a graph construction module, a graph convolution layer and a feature transformation module; wherein the graph construction module is used to construct a graph structure representing the relationship between the comprehensive feature vectors and generate an adjacency matrix; the graph convolution layer is used to perform a convolution operation on the input comprehensive feature vector and the adjacency matrix to generate a preliminary high-dimensional feature vector; the feature transformation module is used to transform the preliminary high-dimensional feature vector output by the graph convolution layer through a fully connected layer and a nonlinear activation function to obtain a high-dimensional feature vector.

5. The network data leakage monitoring system according to claim 1, characterized in that: The abnormal pattern recognition layer includes a time series data preprocessing module, a recurrent neural network layer, an abnormal pattern detection module and a recognition output module; wherein the time series data preprocessing module is used to segment the high-dimensional feature vector in time sequence to form multiple time series segments; the recurrent neural network layer is used to process the time series segments to capture the dependencies and patterns in the time series segments; wherein the recurrent neural network layer includes a first layer of recurrent neural network and a second layer of recurrent neural network, the first layer of recurrent neural network is implemented using long short-term memory units, and the second layer of recurrent neural network is implemented using gated recurrent units; the abnormal pattern detection module detects potential abnormal patterns and assigns preliminary confidence scores based on the output of the recurrent neural network layer; the recognition output module is used to output the detected potential abnormal patterns and preliminary confidence scores of the potential abnormal patterns.

6. The network data leakage monitoring system according to claim 1, characterized in that: The anomaly verification layer includes a generator network, a discriminator network, an anomaly verification module and a verification output module; wherein the generator network is used to generate new feature samples according to potential anomaly patterns; the discriminator network is used to discriminate between the generated new feature samples and the real feature samples, and evaluate their authenticity; the anomaly verification module is used to integrate the results of the generator network and the discriminator network, verify the potential anomaly patterns and assign confidence scores; the verification output module is used to output the verified anomaly patterns and the confidence scores of the anomaly patterns.

7. The network data leakage monitoring system according to claim 1, characterized in that: The data acquisition module comprises: Deep packet inspection engine, used to deeply analyze the content of network data packets and extract traffic information and data packet characteristics; Real-time traffic monitoring unit, used to monitor real-time traffic in the network, including HTTP requests, FTP transfers, email communications, and instant messages; Data mirroring module, used to mirror network data flow in real time without affecting the normal operation of the network for subsequent analysis; The data cache module is used to temporarily store the collected network data so as to quickly access the data before the data preprocessing module performs cleaning and standardization processing.

8. A network data leakage monitoring method, characterized in that: include: Collect network data from network data sources in real time; Clean and standardize the collected network data to obtain pre-processed network data; According to the preprocessed network data, using a pre-trained neural network model, obtaining abnormal patterns of the network data and confidence scores of the abnormal patterns, wherein the abnormal patterns include abnormal increase in data transmission volume, abnormal frequency of access to specific sensitive data, abnormal network connection behavior, and network behavior matching a known attack pattern; Based on the type and confidence score of the abnormal pattern, a real-time assessment of the potential data leakage risk is performed to obtain a comprehensive risk assessment result; When the comprehensive risk assessment result exceeds a preset threshold, generate and send an alarm message, the alarm message including an abnormal pattern description, risk level and recommended response measures; Wherein, the potential data leakage risk is evaluated in real time according to the type and confidence score of the abnormal pattern to obtain a comprehensive risk assessment result, including: The comprehensive risk assessment result is calculated according to the following formula (1): in, The results of the comprehensive risk assessment; is the total number of abnormal patterns detected; For the Risk weights for each type of abnormal pattern; For the confidence scores of the anomaly patterns; For the A severity score for each abnormal pattern, ranging from 1 to 10, set by the system based on historical data and expert judgment; For the The frequency of an abnormal pattern, which indicates the number of times the pattern is detected within a certain period of time; For the the duration of the abnormal pattern; is a specific threshold used to normalize the the duration of the abnormal pattern; The neural network model includes an input layer, a feature extraction layer, a feature fusion layer, a graph convolutional network layer, an abnormal pattern recognition layer and an abnormal verification layer; wherein the input layer is used to receive the preprocessed network data; the feature extraction layer includes multiple parallel convolutional neural network sublayers, and the input of each convolutional neural network sublayer is the preprocessed data provided by the input layer; each convolutional neural network sublayer processes the input data to obtain a feature vector; the feature fusion layer is implemented by an attention mechanism, which is used to fuse the feature vectors provided by multiple convolutional neural network sublayers to generate a fused comprehensive feature vector; the graph convolutional network layer is implemented by a graph convolutional network, which is used to process and capture the complex dependencies and topological relationships between the fused comprehensive feature vectors in a non-Euclidean way to generate a high-dimensional feature vector; the abnormal pattern recognition layer is implemented by a recurrent neural network, which is used to perform time series analysis on the high-dimensional feature vector, identify potential abnormal patterns, and assign a preliminary confidence score to each potential abnormal pattern; the abnormal verification layer is implemented by a generative adversarial network, which obtains abnormal patterns and confidence scores according to the potential abnormal patterns and preliminary confidence scores provided by the abnormal pattern recognition layer.

Images