Real-time analyzing and monitoring method for network security information data
By training high-concurrency behavior recognition models and combining multimodal real-time datasets to identify and predict high-concurrency threats, the problem of insufficient real-time monitoring accuracy in the prior art is solved, and more accurate and efficient network security threat identification and defense are achieved.
Patent Information
- Application Number
- CN202411094664.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-10
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art has poor adaptability when facing diversified real-time log data, and the accuracy of real-time monitoring still needs to be improved, making it difficult to effectively identify and deal with high concurrency threats.
By obtaining multimodal historical log datasets, training a high-concurrency behavior recognition model, combining real-time network traffic, server operation and application logs, integrating device fingerprint and behavior feature information, using convolutional neural networks, long-term memory networks, and self-attention The mechanism performs high concurrent behavior identification and prediction, and generates defense strategies.
It significantly improves the monitoring accuracy of network security threats, can accurately identify and distinguish different devices and users, reduce false alarms and missed reports, improves the accuracy and efficiency of real-time monitoring, and generates effective defense strategies to deal with high concurrent attacks.
Smart Images

Figure CN118972127A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data monitoring, and in particular to a real-time analysis and monitoring method for network security information data. Background Art
[0002] In modern society, the rapid development of information technology has made the network an infrastructure for social economy and daily life. However, with the popularization of network technology, network security issues are becoming increasingly serious. In order to meet these challenges, network security protection technology has continued to advance, from traditional firewalls and antivirus software to modern intrusion detection systems, intrusion prevention systems, and security information and event management systems. These technologies help defend against and detect various network threats by monitoring network traffic, analyzing system logs, and detecting abnormal behaviors. In these protection measures, log data plays a key role.
[0003] Log data records various activities of network devices, servers, and applications, and is the basis of network security analysis. By analyzing log data, security experts can identify potential threats, track attack paths, audit security events, and conduct post-event forensics. In the modern network environment, high-concurrency threats pose a serious threat to websites and network resources. High-concurrency threats include not only crawler behaviors, but also distributed denial of service (DDoS) attacks, brute force cracking, and other automated attacks. These attacks can steal sensitive data, consume bandwidth resources, and even cause server overloads, affecting normal user access.
[0004] In existing research, a Chinese patent with a publication number of CN118200190B discloses a network performance monitoring and maintenance method, system and medium based on artificial intelligence. The specific steps include: obtaining network operation monitoring data, and generating network resource allocation demand data based on this data processing, and then generating network resource allocation strategy data. Network resources are allocated according to the strategy, and operation performance indicator monitoring data is obtained, and then the network resource allocation performance evaluation index is calculated. At the same time, the server operation log data is obtained, and the server load balancing performance evaluation index is calculated by combining the network resource allocation performance evaluation index and the server log data. An attack test is performed on the network after resource allocation, network attack response data is obtained and a security performance evaluation index is generated; a fault test is performed, fault diagnosis and repair data is obtained and a fault repair performance evaluation index is generated. Finally, according to the network resource allocation performance evaluation index, the server load balancing performance evaluation index, the security performance evaluation index and the fault repair performance evaluation index, a comprehensive network evaluation index is generated by comprehensive processing, and the corresponding maintenance strategy is matched.
[0005] A Chinese patent with publication number CN112419130B discloses an emergency response system and method based on network security monitoring and data analysis, in which the data acquisition module is responsible for collecting status information and user information of the system to be monitored, and the data management module classifies, processes and stores the collected data. The inspection module regularly checks these data, and the analysis module analyzes the inspection results to determine whether the activation of services, software and security policies reaches the preset threshold. The policy management module formulates or updates network access policies and monitoring policies based on the analysis results. The monitoring module monitors the system in real time in combination with these policies and generates or updates the corresponding monitoring model. When an anomaly is found, the response module issues an alarm and automatically processes it according to the preset response strategy, generates an anomaly report, and optimizes the response strategy model by learning the operation data of the management personnel so that similar anomalies can be directly handled when they occur again.
[0006] There is also a Chinese patent with publication number CN116933283A that discloses a monitoring method and system for big data security baselines based on machine learning. The method first obtains the original data of the data platform, including log data, user behavior data, and network traffic data, and generates corresponding feature vectors. Then, these feature vectors are trained through machine learning algorithms to build a security baseline model. The system also obtains the access data of the platform, and determines the outliers in the access data based on the security baseline model, so as to formulate corresponding security protection strategies. This process includes preprocessing of the original data, feature extraction, and generation of a security baseline model using classification and regression methods, and adjusting the model when obtaining updated data. Based on the outliers of the access data, the system determines whether security protection measures need to be taken, and formulates a protection strategy based on the mapping relationship between outlier attributes and security policies.
[0007] However, in the above-mentioned prior art, when faced with diverse real-time log data, the adaptability of the existing methods is poor, and the accuracy of real-time monitoring still needs to be improved.
[0008] Therefore, a real-time analysis and monitoring method for network security information data is proposed. Summary of the invention
[0009] The object of the present invention is to provide a real-time analysis and monitoring method for network security information data, which first obtains a multimodal historical log data set and trains a high-concurrency behavior recognition model to identify potential high-concurrency behaviors. Then, network traffic logs, server operation logs and application logs are obtained in real time and integrated into a first integrated data. The first integrated data is integrated by IP address to form a first multimodal real-time data set. As another data integration method, the device fingerprint and behavior feature information of each request in the first integrated data are extracted, and then a second multimodal real-time data set is generated in combination with this information. The two real-time data sets are respectively input into the high-concurrency behavior recognition model to obtain a high-concurrency behavior recognition result. If a high-concurrency behavior is identified, an early warning is issued, and malicious identification is performed on the two data sets to obtain a malicious identification result. If a high-concurrency behavior is not identified, a high-concurrency prediction is performed first, and then the real-time log data is re-acquired and high-concurrency behavior recognition is performed. Finally, a defense strategy is generated according to the high-concurrency behavior recognition result, the malicious identification result and the high-concurrency prediction result, including real-time adjustment of network configuration, real-time adjustment of firewall rules, restriction of attack source IP, dynamic allocation of computing resources and dynamic allocation of bandwidth resources. By implementing this method, the monitoring accuracy of network security threats can be improved.
[0010] To achieve the above object, the present invention provides the following technical solutions:
[0011] A real-time analysis and monitoring method for network security information data, comprising:
[0012] S1. Obtain a multimodal historical log data set, and train a high-concurrency behavior recognition model according to the multimodal historical log data set to recognize the occurrence of high-concurrency behaviors;
[0013] S2, obtaining real-time network traffic logs, real-time server operation logs and real-time application logs to obtain first integrated data;
[0014] S3, integrating the first integrated data according to the IP address to obtain a first multimodal real-time data set; extracting the device fingerprint information and the behavior characteristic information of each request in the first integrated data; combining the device fingerprint information and the behavior characteristic information, integrating the first integrated data to obtain a second multimodal real-time data set;
[0015] S4, inputting the first multimodal real-time data set and the second multimodal real-time data set into the high-concurrency behavior recognition model respectively to obtain a high-concurrency behavior recognition result;
[0016] S5. If the high-concurrency behavior recognition result is yes, issue an early warning; perform malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior recognition result, obtain malicious identification results, and execute step S7; if the high-concurrency behavior recognition result is no, execute step S6 first, and then repeat steps S2 to S4;
[0017] S6. Predicting the first multimodal real-time data set and the second multimodal real-time data set respectively to obtain a high-concurrency prediction result;
[0018] S7. Generate a corresponding defense strategy according to the high-concurrency behavior identification result, the malicious identification result and the high-concurrency prediction result.
[0019] Furthermore, the multimodal historical log data set specifically comprises: obtaining historical network traffic logs, historical server operation logs and historical application logs, and integrating them to obtain the multimodal historical log data set.
[0020] Furthermore, the first integrated data is integrated according to the IP address to obtain the first multimodal real-time data set, including:
[0021] According to a predefined log template, template matching is performed on the real-time network traffic log, the real-time server operation log and the real-time application log to obtain first template log data and first non-template log data; for the first template log data, direct parsing and feature extraction are performed to obtain second template log data; for the first non-template log data, an adaptive parsing method is used to dynamically adjust the template or use natural language processing technology to analyze to obtain second non-template log data; the second template log data and the second non-template log data are integrated according to IP addresses to form the first multimodal real-time data set.
[0022] Further, extracting the device fingerprint information and the behavior characteristic information of each request in the first integrated data; and integrating the first integrated data by combining the device fingerprint information and the behavior characteristic information includes:
[0023] Extract browser information in the request, including browser type, version, plug-in, font and screen resolution; extract operating system information, including operating system type, version and language setting; extract request header information, including User-Agent, Accept-Language and Content-Encoding in the HTTP request header; extract hardware information, including the device's CPU type, memory size and time zone; calculate the number of requests, access path sequence, access mode, operation type and response time for each IP address within a specified time period; generate a device fingerprint feature vector and a behavior feature vector after standardization of the device fingerprint information and the behavior feature information, and then concatenate the device fingerprint feature vector and the behavior feature vector to form a comprehensive feature vector; perform similar user identification based on the comprehensive feature vector, identify M similar users, and obtain similar user tags; integrate the first integrated data according to the similar user tags to obtain the second multimodal real-time data set.
[0024] Furthermore, the high-concurrency behavior recognition model includes:
[0025] The input layer is used to receive multimodal historical log data sets and perform preprocessing and normalization;
[0026] A feature extraction layer, used for extracting local features from the multimodal historical log data set using a convolutional neural network; comprising multiple convolutional layers and pooling layers;
[0027] Long short-term memory network layer, composed of multiple LSTM units, used to process time series data;
[0028] A fusion layer, used to fuse the features extracted by the feature extraction layer and the long short-term memory network layer;
[0029] The attention mechanism layer is used to calculate feature importance weights through the self-attention mechanism;
[0030] The rule judgment layer is used to obtain rule features according to predefined rules. The rule features are binary variables, indicating whether the predefined rules are met.
[0031] The classification layer, including the fully connected layer, activation function and Dropout layer, is used to classify the fused features;
[0032] The output layer, including the Softmax layer, is used to convert the output of the classification layer into a probability distribution.
[0033] Furthermore, the predefined rules include at least one of request frequency rules, data packet size rules, connection duration rules, access path rules, error rate rules, user agent rules, geographic location rules, port scanning rules and protocol usage rules.
[0034] Furthermore, after obtaining the malicious identification result and before executing the S6 step, the method further includes: if the malicious identification result is yes, issuing a warning.
[0035] Further, performing malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior identification result includes:
[0036] According to the high-concurrency behavior recognition result, a high-concurrency IP corresponding to the first multimodal real-time data set and a high-concurrency user corresponding to the second multimodal real-time data set are obtained; according to the high-concurrency behavior recognition result, high-concurrency users that meet the specified conditions are filtered out to obtain a high-concurrency tag; according to the high-concurrency tag, the first multimodal real-time data set and the second multimodal real-time data set corresponding to the high-concurrency tag are obtained to obtain a first multimodal tag data set and a second multimodal tag data set; the specified condition is: there is only one IP address and the IP address is in the IP list of the high-concurrency IP;
[0037] Performing request frequency analysis on the first multimodal label data set and the second multimodal label data set;
[0038] Construct a historical multi-source threat intelligence database, and establish a blacklist based on the historical multi-source threat intelligence, including malicious IP addresses and domain names; extract multi-mode rules based on the historical multi-source threat intelligence, including behavior mode rules, feature mode rules, communication mode rules, geographic location mode rules and time mode rules; the multi-mode rules are updated in real time;
[0039] The behavior pattern rules include abnormal request frequency and access path pattern;
[0040] The characteristic pattern rules include IP address pattern, domain name pattern and file hash pattern;
[0041] The communication mode rules include JA3 fingerprint mode and data packet feature mode;
[0042] The geographic location pattern rules include geographic location change patterns and IP geographic location matching;
[0043] The time pattern rules include time period abnormality patterns and time interval patterns;
[0044] The multimodal rules and the blacklist are applied to the first multimodal labeled dataset and the second multimodal labeled dataset to detect malicious behavior.
[0045] Further, respectively predicting the first multimodal real-time data set and the second multimodal real-time data set includes:
[0046] The first multimodal real-time data set and the second multimodal real-time data set are respectively input into the trained high-concurrency prediction model for prediction to obtain a high-concurrency prediction result; the high-concurrency prediction model is trained based on the LSTM model according to the multimodal historical log data set, and is used to predict the probability value of high-concurrency behavior occurring in a specified time period in the future; the multimodal historical log data set is corrected and integrated according to the same user.
[0047] Furthermore, the defense strategy includes: real-time adjustment of network configuration, real-time adjustment of firewall rules, limiting attack source IP, and dynamically allocating computing resources and bandwidth resources.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] 1. The present invention obtains detailed features of the user's device by extracting device fingerprint information, so that different devices can be accurately identified and distinguished. At the same time, the extracted behavioral feature information reveals the user's activity pattern. Combining these two types of information helps to form an accurate user portrait, which can identify and distinguish different devices. Even if the user changes the IP address or uses a proxy, the system can still accurately identify based on the unique fingerprint of the device, thereby improving the ability to prevent fraud and disguised behavior. Thereby more effectively identifying and distinguishing different activities of the same user. This comprehensive analysis method significantly improves the accuracy of recognition, so that potential abnormal behaviors can be better captured and responded to in real-time monitoring. When faced with diverse real-time log data, it helps to improve the accuracy of real-time monitoring.
[0050] 2. The high-concurrency behavior recognition model provided by the present invention provides a high-concurrency behavior recognition method by combining convolutional neural networks, long short-term memory networks, self-attention mechanisms, and rule judgment mechanisms. Specifically, the preprocessing and normalization of the input layer ensure the quality and consistency of the data; the feature extraction layer extracts local features through CNN, which helps to capture the key patterns of the data; the LSTM layer processes time series data and identifies temporal dependencies; the fusion layer and the attention mechanism layer enhance the model's ability to recognize important features, allowing the model to focus on key features and optimize performance; the rule judgment layer enhances the model's rule matching ability by introducing predefined rules; and the classification layer accurately classifies the fused features. Finally, the output layer converts the classification results into probability distributions. This multi-level, multi-dimensional comprehensive analysis method can effectively improve the performance of the model in a complex data environment, reduce false positives and false negatives, and improve the accuracy and reliability of real-time monitoring of network security information data.
[0051] 3. The present invention effectively identifies high-concurrency IPs and users by inputting the first multimodal real-time data set and the second multimodal real-time data set into the high-concurrency behavior recognition model respectively. In this process, some users may be identified as high-concurrency IPs and high-concurrency users at the same time. To avoid redundant analysis, the present invention excludes users who are repeatedly identified through a filtering mechanism, thereby reducing subsequent repeated identification and analysis. This strategy optimizes the monitoring process, improves analysis efficiency and accuracy, reduces false positives, and ensures that system resources are efficiently utilized. By analyzing the request frequency and combining it with the constructed blacklist, malicious IPs and domain names can be identified in a timely manner. The introduced multi-mode rules, including behavior patterns, feature patterns, communication patterns, geographic location patterns, and time patterns, cover a variety of malicious behavior patterns from abnormal request frequencies to geographic location matching. The real-time update of these rules ensures that the monitoring system can adapt to new threats. The comprehensive application of these rules and blacklists significantly improves the detection capability of malicious behaviors, reduces missed reports and false positives, and improves the accuracy of real-time network security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 A flow chart of a method for real-time analysis and monitoring of network security information data provided by an embodiment of the present invention;
[0053] Figure 2 A structural diagram of a high-concurrency behavior recognition model provided by an embodiment of the present invention;
[0054] Figure 3 A data flow chart for obtaining high-concurrency IPs and high-concurrency users from first integrated data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] Embodiment 1:
[0057] A large e-commerce platform A wants to protect its website from cyber attacks, such as DDoS attacks and malicious crawlers. The platform has a large amount of user access and transaction data, so monitoring and timely response to high concurrent behaviors are crucial. In order to ensure the security and stability of the platform, a real-time analysis and monitoring method for network security information data is deployed, such as Figure 1 As shown, including:
[0058] S1. Obtain a multimodal historical log data set, and train a high-concurrency behavior recognition model according to the multimodal historical log data set to recognize the occurrence of high-concurrency behaviors;
[0059] Furthermore, the multimodal historical log data set specifically comprises: obtaining historical network traffic logs, historical server operation logs and historical application logs, and integrating them to obtain the multimodal historical log data set.
[0060] The consistency of historical log data and real-time analysis log data is guaranteed. At the same time, log data from different sources provides a multi-dimensional perspective on system operation, which can more comprehensively reflect system status and activities, thereby capturing more behavioral characteristics and abnormal patterns, and improving the accuracy and reliability of high-concurrency behavior recognition models. In addition, historical log data records past system behaviors and security events, providing rich sample data for training models. Through comprehensive analysis of multimodal data, potential security threats can be more accurately identified and targeted defense strategies can be generated, improving the system's security protection level and real-time monitoring capabilities.
[0061] S2, obtaining real-time network traffic logs, real-time server operation logs and real-time application logs to obtain first integrated data;
[0062] S3. Two data integration methods are provided. The first method is: integrating the first integrated data by IP address to obtain a first multimodal real-time data set; extracting device fingerprint information and behavior feature information of each request in the first integrated data; another data integration method is: combining the device fingerprint information and the behavior feature information to integrate the first integrated data to obtain a second multimodal real-time data set;
[0063] These two data integration methods provide different levels of data processing and analysis perspectives. The first method integrates data by IP address, making it easier to understand the activities of each IP from a global perspective. The second method further improves the details and accuracy of the data by combining device fingerprints and behavioral feature information, so that multimodal real-time datasets can better support complex behavior recognition tasks. These two integration methods complement each other and can effectively improve the performance and accuracy of high-concurrency behavior recognition models. In the first multimodal real-time dataset obtained by the first integration method, one IP address corresponds to one user by default; in the second multimodal real-time dataset obtained by the second integration method, one user corresponds to at least one IP and can correspond to multiple IPs.
[0064] Furthermore, the first integrated data is integrated according to the IP address to obtain the first multimodal real-time data set, including:
[0065] According to a predefined log template, template matching is performed on the real-time network traffic log, the real-time server operation log and the real-time application log to obtain first template log data and first non-template log data; for the first template log data, direct parsing and feature extraction are performed to obtain second template log data; for the first non-template log data, an adaptive parsing method is used to dynamically adjust the template or use natural language processing technology to analyze to obtain second non-template log data; the second template log data and the second non-template log data are integrated according to IP addresses to form the first multimodal real-time data set.
[0066] This method can improve the flexibility and accuracy of parsing in the real-time data parsing process. Through template matching, common log formats can be quickly processed to ensure efficient parsing; and for non-template log data, adaptive parsing and natural language processing technology can dynamically respond to various complex and non-standardized log formats, ensuring the comprehensiveness and accuracy of data parsing, thereby improving the quality of multimodal real-time data sets and providing a reliable data foundation for subsequent behavior recognition and defense strategy generation.
[0067] In this embodiment, historical network traffic logs, historical server operation logs, and historical application logs are also subjected to log template matching, parsing, and feature extraction according to the above method to obtain a multimodal historical log data set.
[0068] Further, extracting the device fingerprint information and the behavior characteristic information of each request in the first integrated data; and integrating the first integrated data by combining the device fingerprint information and the behavior characteristic information includes:
[0069] The device fingerprint information includes browser information, operating system information, request header information and hardware information;
[0070] The behavior characteristic information includes the number of requests, access path sequence, access mode, operation type and response time of each IP address within a specified time period;
[0071] Specifically, extract browser information in the request, including browser type, version, plug-in, font and screen resolution; extract operating system information, including operating system type, version and language setting; extract request header information, including User-Agent, Accept-Language and Content-Encoding in the HTTP request header; extract hardware information, including the device's CPU type, memory size and time zone; calculate the number of requests, access path sequence, access mode, operation type and response time for each IP address within a specified time period; generate a device fingerprint feature vector and a behavior feature vector after standardization of the device fingerprint information and the behavior feature information, and then concatenate the device fingerprint feature vector and the behavior feature vector to form a comprehensive feature vector; perform similar user identification based on the comprehensive feature vector, identify M similar users, and obtain similar user tags; integrate the first integrated data according to the similar user tags to obtain the second multimodal real-time data set. Among them, M refers to the number of users identified in the first integrated data based on the device fingerprint information and behavior feature information. These users have similar behavior characteristics or device configurations within a specific time period and are therefore regarded as the same user or users with similar user tags.
[0072] The extracted browser information, operating system information, request header and hardware information are used to build a detailed device fingerprint for each request; by calculating the number of requests, access paths, operation modes and other behavioral feature information, similar users can be identified through the combination of device fingerprint information and behavioral feature information, and then potential IP camouflage behaviors can be identified and revealed. Even if the attacker uses different IP addresses for camouflage, they can be accurately identified. The second multimodal real-time data set integrated by users can help to more comprehensively detect and prevent camouflage attacks, improve overall security protection capabilities, reduce false positives and false negatives, and improve the accuracy and effectiveness of network security monitoring.
[0073] S4, inputting the first multimodal real-time data set and the second multimodal real-time data set into the high-concurrency behavior recognition model respectively to obtain a high-concurrency behavior recognition result;
[0074] Furthermore, if Figure 2 As shown, the high-concurrency behavior recognition model includes:
[0075] The input layer is used to receive multimodal historical log data sets and perform preprocessing and normalization;
[0076] A feature extraction layer, used for extracting local features from the multimodal historical log data set using a convolutional neural network; comprising multiple convolutional layers and pooling layers;
[0077] Long short-term memory network layer, composed of multiple LSTM units, used to process time series data;
[0078] A fusion layer, used to fuse the features extracted by the feature extraction layer and the long short-term memory network layer;
[0079] The attention mechanism layer is used to calculate feature importance weights through the self-attention mechanism;
[0080] The rule judgment layer is used to obtain rule features according to predefined rules. The rule features are binary variables, indicating whether the predefined rules are met.
[0081] The classification layer, including the fully connected layer, activation function and Dropout layer, is used to classify the fused features;
[0082] The output layer, including the Softmax layer, is used to convert the output of the classification layer into a probability distribution.
[0083] Combining convolutional neural networks and long short-term memory networks, it can effectively extract and process the spatiotemporal features in log data, enhance the importance of features through fusion layers and attention mechanisms, and provide additional rule feature information in combination with the rule judgment layer. The role of the rule judgment layer is to systematically introduce specific rule information into the model to enhance the model's ability to recognize specific behavior patterns and provide useful feature information to improve the final classification and decision results. Finally, accurate behavior recognition is performed at the classification layer. The overall model has high flexibility and accuracy, can more effectively identify high-concurrency behaviors, and improve the security and protection capabilities of the system.
[0084] Furthermore, the predefined rules include at least one of request frequency rules, data packet size rules, connection duration rules, access path rules, error rate rules, user agent rules, geographic location rules, port scanning rules and protocol usage rules.
[0085] Among them, the request frequency rule is specifically to monitor whether the number of requests from a single IP address in a unit time exceeds the set threshold. The packet size rule is specifically to check whether the size of the packet sent by a single IP address is within the specified range, and identify packets of abnormal size. The connection duration rule is specifically to analyze whether the connection time of a single IP address is abnormal, including whether the duration is too long (exceeding the first preset threshold) or too short (below the second preset threshold). The access path rule is specifically to track whether a single IP address frequently accesses a specific path or resource, such as a login page or a sensitive data interface. The error rate rule is specifically to count the number of error response codes generated by a single IP address request, such as 404 or 500, to identify abnormally high error rates. The user agent rule is specifically to review the User-Agent field in the request header and identify abnormal values that do not conform to the norm. The geographic location rule is specifically to monitor the geographic location activities of a single IP address, especially requests initiated from multiple different geographic locations in a short period of time. The port scanning rule is specifically to observe whether a single IP address attempts to connect to multiple different ports to identify port scanning behavior. The protocol usage rule is specifically to detect the type of network protocol used by a single IP address in a unit time, such as TCP or UDP, to discover abnormal behavior using multiple protocols.
[0086] By covering a variety of rules, the system can accurately monitor network activities and detect anomalies from multiple angles. Each rule targets different abnormal behaviors or attack patterns, such as request frequency rules are used to detect abnormal traffic, packet size rules identify packet anomalies, and port scan rules are specifically used to detect port scan behavior. The diversity of rules ensures that the system can flexibly adapt to new attack patterns and ensure comprehensive monitoring of various potential threats. In addition, these rules can identify anomalies in real time, providing more accurate support for protecting high-concurrency monitoring of the network.
[0087] S5. If the high-concurrency behavior recognition result is yes, issue an early warning; perform malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior recognition result, obtain malicious identification results, and execute step S7; if the high-concurrency behavior recognition result is no, execute step S6 first, and then repeat steps S2 to S4;
[0088] Further, performing malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior identification result includes:
[0089] like Figure 3As shown, according to the high-concurrency behavior recognition result, the high-concurrency IP corresponding to the first multimodal real-time data set and the high-concurrency user corresponding to the second multimodal real-time data set are obtained; according to the high-concurrency behavior recognition result, the high-concurrency users that meet the specified conditions are filtered out to obtain a high-concurrency tag; according to the high-concurrency tag, the corresponding first multimodal real-time data set and the second multimodal real-time data set are obtained to obtain a first multimodal tag data set and a second multimodal tag data set; the specified condition is: there is only one IP address and the IP address is in the IP list of the high-concurrency IP;
[0090] Performing request frequency analysis on the first multimodal label data set and the second multimodal label data set;
[0091] Construct a historical multi-source threat intelligence database, and establish a blacklist based on the historical multi-source threat intelligence, including malicious IP addresses and domain names; extract multi-mode rules based on the historical multi-source threat intelligence, including behavior mode rules, feature mode rules, communication mode rules, geographic location mode rules and time mode rules; the multi-mode rules are updated in real time;
[0092] The behavior pattern rules include abnormal request frequency and access path pattern;
[0093] The characteristic pattern rules include IP address pattern, domain name pattern and file hash pattern;
[0094] The communication mode rules include JA3 fingerprint mode and data packet feature mode;
[0095] The geographic location pattern rules include geographic location change patterns and IP geographic location matching;
[0096] The time pattern rules include time period anomaly patterns and time interval patterns;
[0097] The multimodal rules and the blacklist are applied to the first multimodal labeled dataset and the second multimodal labeled dataset to detect malicious behavior.
[0098] In this embodiment, the high-concurrency behavior recognition result includes a first recognition result obtained from the first multimodal label data set, and also includes a second recognition result obtained from the second multimodal label data set;
[0099] Table 1. High Concurrency IP Example Table
[0100] IP address Source dataset 61.216.156.222 The first multimodal labeled dataset 113.195.143.39 The first multimodal labeled dataset 47.117.125.229 The first multimodal labeled dataset 117.34.25.11 The first multimodal labeled dataset 61.178.149.237 The first multimodal labeled dataset 59.66.19.53 The first multimodal labeled dataset 8.218.55.24 The first multimodal labeled dataset ... ...
[0101] According to the first identification result, several high-concurrency IPs can be obtained, and a high-concurrency IP list can be obtained, as shown in Table 1. To protect privacy, the specific IP addresses have been anonymized.
[0102] Table 2: Example table of high concurrent users
[0103] user IP address Source dataset User A 61.216.156.222 Second multimodal labeled dataset User A 113.195.143.39 Second multimodal labeled dataset User B 47.117.125.229 Second multimodal labeled dataset User B 117.34.25.11 Second multimodal labeled dataset User C 61.178.149.237 Second multimodal labeled dataset User D 59.66.19.53 Second multimodal labeled dataset User E 183.195.106.118 Second multimodal labeled dataset ... ... ...
[0104] Table 3, Filter judgment table
[0105] user IP address Source dataset Whether to filter User A 61.216.156.222 Second multimodal labeled dataset no User A 113.195.143.39 Second multimodal labeled dataset no User B 47.117.125.229 Second multimodal labeled dataset no User B 117.34.25.11 Second multimodal labeled dataset no User C 61.178.149.237 Second multimodal labeled dataset yes User D 59.66.19.53 Second multimodal labeled dataset yes User E 183.195.106.118 Second multimodal labeled dataset no ... ... ... ...
[0106] The identified high-concurrency users are shown in Table 2. The IP address 183.195.106.118 corresponding to user E in Table 2 is not in the IP address in Table 1. Therefore, although the user has only one IP address, it does not meet the specified requirements. Users A and B both correspond to multiple IP addresses, so they also do not meet the specified requirements. The filtering results are shown in Table 3.
[0107] According to the high-concurrency behavior recognition result, the high-concurrency IP corresponding to the first multimodal real-time data set and the high-concurrency user corresponding to the second multimodal real-time data set are obtained, wherein each of the identified high-concurrency IPs corresponds to only one user, and each of the identified high-concurrency users may involve multiple IPs. The users that are repeatedly identified are filtered out to obtain high-concurrency tags.
[0108] At the same time, this method not only utilizes the malicious IP and domain names in the blacklist, but also covers a variety of behavior patterns, characteristic patterns, communication patterns, geographic location patterns, and time pattern rules, thereby enhancing the ability to detect complex attack behaviors. Behavior pattern rules can reveal abnormal request frequencies and access paths, characteristic pattern rules can identify suspicious IP addresses and domain names, communication pattern rules help detect specific fingerprint patterns, and geographic location pattern rules and time pattern rules further improve the accurate identification of abnormal activities. The combined use of these rules and blacklists can effectively improve the accuracy and timeliness of malicious behavior detection.
[0109] Furthermore, after obtaining the malicious identification result and before executing the S6 step, the method further includes: if the malicious identification result is yes, issuing a warning.
[0110] This early warning method helps improve the timeliness of monitoring and respond to security incidents in a timely manner. By issuing an immediate warning at the early stage of detecting malicious activities, system administrators can mobilize resources and deploy necessary defense measures more quickly. This not only shortens the response time to security incidents, but also greatly reduces potential damage and recovery costs.
[0111] S6. Predicting the first multimodal real-time data set and the second multimodal real-time data set respectively to obtain a high-concurrency prediction result;
[0112] Further, respectively predicting the first multimodal real-time data set and the second multimodal real-time data set includes:
[0113] The first multimodal real-time data set and the second multimodal real-time data set are respectively input into the trained high-concurrency prediction model for prediction to obtain a high-concurrency prediction result; the high-concurrency prediction model is trained based on the LSTM model according to the multimodal historical log data set, and is used to predict the probability value of high-concurrency behavior occurring in a specified time period in the future; the multimodal historical log data set is corrected and integrated according to the same user.
[0114] Specifically, multimodal historical log data is collected from multiple data sources, each of which contains fields that can uniquely identify a user (including user ID, device ID, session ID, and IP address);
[0115] According to the predefined log template, the historical network traffic log, the historical server operation log, and the historical application log template are matched to obtain the third template log data and the third non-template log data; for the third template log data, directly parse and extract features to obtain the fourth template log data; for the third non-template log data, an adaptive parsing method is used to dynamically adjust the template or use natural language processing technology for analysis to obtain the fourth non-template log data; finally, the fourth template log data and the fourth non-template log data are integrated to form the second multimodal historical data set. The second multimodal historical log data is preprocessed, and the preprocessing includes data cleaning, processing missing values, outliers and duplicate data, and ensuring the consistency of time sequence through timestamp alignment;
[0116] Use multiple user identifiers (including user ID, device ID, session ID, and IP address) to associate the same user data in different data sources;
[0117] Detect abnormal usage patterns of IP addresses, identify situations where a large number of different users log in to the same IP address in a short period of time or where the geographical locations are significantly different, to identify IP spoofing behavior;
[0118] Conduct behavioral feature analysis by comparing current data with historical behavior patterns to detect records that are significantly inconsistent with the user's historical behavior, and use geographic location verification technology to confirm the rationality of user behavior through the geographic location corresponding to the IP address;
[0119] In the event of abnormal behavior or IP spoofing, multiple identifiers including user ID, device ID and session ID are integrated to cross-verify and correct the data;
[0120] Further verify suspicious data through automated review mechanisms to ensure the accuracy and consistency of final data;
[0121] Through the above steps, we ensure that each user's data is accurately integrated and a true label is generated, and finally obtain the third multimodal historical log data, which provides a reliable data foundation for training high-concurrency prediction models.
[0122] Preprocessing the third multimodal historical log data set, wherein the preprocessing includes data cleaning, feature extraction, and data normalization processing; converting the preprocessed third multimodal historical log data set into a time series format to create input and output pairs; dividing the converted time series data into a training set and a test set;
[0123] Build an LSTM model, train the LSTM model using the training dataset, use optimization algorithms and loss functions to minimize prediction errors, and use the validation set to evaluate model performance in real time during the training process, select the best model structure and hyperparameters, and prevent overfitting;
[0124] After training, the performance of the LSTM model is evaluated on the test set to verify its generalization ability and performance on unseen data; finally, the tuned and verified LSTM model is saved and deployed for real-time prediction of the input multimodal real-time data set, thereby obtaining a high-concurrency prediction result, which is used to predict the probability value of high-concurrency behavior occurring in a specified time period in the future.
[0125] This prediction method can timely predict the probability of high-concurrency behavior in the future. It enhances the reliability and stability of real-time monitoring of log data. It can identify possible high-concurrency events in advance. By integrating the modified multimodal historical log data set and classifying them according to the same users, the high-concurrency prediction model can more accurately learn and simulate user behavior and system response patterns. This integration helps to eliminate noise and redundancy in the data, providing cleaner and more consistent data input to the LSTM model, thereby enhancing the reliability and accuracy of the prediction results.
[0126] S7. Generate a corresponding defense strategy according to the high-concurrency behavior identification result, the malicious identification result and the high-concurrency prediction result.
[0127] Furthermore, the defense strategy includes: real-time adjustment of network configuration, real-time adjustment of firewall rules, limiting attack source IP, and dynamically allocating computing resources and bandwidth resources.
[0128] The defense strategy effectively enhances network security. Through real-time adjustment of network configuration and instant update of firewall rules, it can effectively limit the potential attack source IP addresses. In addition, the strategy of dynamically allocating computing resources and bandwidth resources not only improves the response speed of the system, but also enhances the ability to resist sudden traffic attacks, thereby ensuring the stable operation of the network environment and data security. These measures together constitute a multi-level, dynamic defense mechanism, which greatly improves the defense effect against external high-concurrency threats.
[0129] In order to verify the effectiveness of a real-time analysis and monitoring method for network security information data provided in this embodiment, two groups of comparative experiments were designed. The difference between the two groups of comparative experiments lies in the data integration method of the first integrated data. The integration method of the benchmark experiment mentioned in the method is to integrate according to IP addresses and users respectively, obtain high-concurrency IPs and high-concurrency users, and then filter out the high-concurrency users with only one IP address. Comparative experiment one is to use only IP address integration, and the integrated data is used for subsequent high-concurrency behavior identification, malicious identification and high-concurrency prediction; comparative experiment two is to integrate for users, and the integrated data is used for subsequent high-concurrency behavior identification, malicious identification and high-concurrency prediction;
[0130] Table 4. Comparative experimental data table
[0131]
[0132] The accuracy rates in Table 4 are based on the actual labeling results obtained after a detailed analysis of the real-time log data. In this process, security experts first classify the events and identify them as normal, abnormal, or malicious. Then, security analysts conduct a detailed manual review of all suspected malicious events to confirm their nature and label the events as "normal", "abnormal", or "malicious" accordingly. As shown in Table 4, overall, the baseline experiment shows the highest accuracy in all three indicators, indicating that it performs well in real-time analysis and monitoring scenarios. Comparative Experiments 1 and 2 show a decline in various indicators, especially in high-concurrency recognition accuracy.
[0133] The real-time analysis and monitoring method of network security information data can effectively identify and predict high-concurrency behaviors and their potential malicious activities by comprehensively analyzing multimodal historical log data sets and multimodal real-time log data sets. The high-concurrency behavior identification model and multi-level data integration and analysis ensure comprehensive monitoring of network traffic and server operation status. The data is refined and classified by two integration methods, namely, integration by IP address and integration by similar users, to achieve more accurate analysis and detection effects. Specifically, on the one hand, integration by IP address helps to identify cluster behaviors from the same IP and reveal possible large-scale traffic attacks or DDoS attacks. On the other hand, integration by similar users can identify abnormal behavior patterns disguised by different proxy IPs by analyzing the user's device fingerprints and behavioral characteristics, thereby effectively identifying and responding to IP disguises. This dual integration method not only improves the detection capability of complex attack patterns, but also reduces the possibility of false positives and false negatives, thereby providing more accurate security protection. The early warning mechanism and malicious identification steps further enhance the response capability to possible malicious activities. Ultimately, the defense strategy generated based on the identification and prediction results can adjust network security protection measures in real time, effectively reducing the risk of the system being attacked by high-concurrency attacks and improving the defense capabilities against complex attack patterns.
[0134] Embodiment 2:
[0135] There is a large ticketing website B that needs to protect its online system from cyber attacks, such as malicious crawlers and malicious ticket purchase scripts. The website platform handles a large number of user access and transaction requests every day, so monitoring and timely response to high concurrent behaviors are crucial. In order to ensure the security and stability of the website, a real-time analysis and monitoring method for network security information data is deployed, including:
[0136] S1. Obtain a multimodal historical log data set, and train a high-concurrency behavior recognition model according to the multimodal historical log data set to recognize the occurrence of high-concurrency behaviors;
[0137] Furthermore, the multimodal historical log data set specifically comprises: obtaining historical network traffic logs, historical server operation logs and historical application logs, and integrating them to obtain the multimodal historical log data set.
[0138] S2, obtaining real-time network traffic logs, real-time server operation logs and real-time application logs to obtain first integrated data;
[0139] S3, integrating the first integrated data according to the IP address to obtain a first multimodal real-time data set; extracting the device fingerprint information and the behavior characteristic information of each request in the first integrated data; combining the device fingerprint information and the behavior characteristic information, integrating the first integrated data to obtain a second multimodal real-time data set;
[0140] Furthermore, the first integrated data is integrated according to the IP address to obtain the first multimodal real-time data set, including:
[0141] According to a predefined log template, template matching is performed on the real-time network traffic log, the real-time server operation log and the real-time application log to obtain first template log data and first non-template log data; for the first template log data, direct parsing and feature extraction are performed to obtain second template log data; for the first non-template log data, an adaptive parsing method is used to dynamically adjust the template or use natural language processing technology to analyze to obtain second non-template log data; the second template log data and the second non-template log data are integrated according to IP addresses to form the first multimodal real-time data set.
[0142] In this embodiment, historical network traffic logs, historical server operation logs, and historical application logs are also subjected to log template matching, parsing, and feature extraction according to the above method to obtain a multimodal historical log data set.
[0143] Further, extracting the device fingerprint information and the behavior characteristic information of each request in the first integrated data; and integrating the first integrated data by combining the device fingerprint information and the behavior characteristic information includes:
[0144] Extract browser information in the request, including browser type, version, plug-in, font and screen resolution; extract operating system information, including operating system type, version and language setting; extract request header information, including User-Agent, Accept-Language and Content-Encoding in the HTTP request header; extract hardware information, including the device's CPU type, memory size and time zone; calculate the number of requests, access path sequence, access mode, operation type and response time for each IP address within a specified time period; generate a device fingerprint feature vector and a behavior feature vector after standardization of the device fingerprint information and the behavior feature information, and then concatenate the device fingerprint feature vector and the behavior feature vector to form a comprehensive feature vector; perform similar user identification based on the comprehensive feature vector, identify M similar users, and obtain similar user tags; integrate the first integrated data according to the similar user tags to obtain the second multimodal real-time data set.
[0145] S4, inputting the first multimodal real-time data set and the second multimodal real-time data set into the high-concurrency behavior recognition model respectively to obtain a high-concurrency behavior recognition result;
[0146] Furthermore, the high-concurrency behavior recognition model includes:
[0147] The input layer is used to receive multimodal historical log data sets and perform preprocessing and normalization;
[0148] A feature extraction layer, used for extracting local features from the multimodal historical log data set using a convolutional neural network; comprising multiple convolutional layers and pooling layers;
[0149] Long short-term memory network layer, composed of multiple LSTM units, used to process time series data;
[0150] A fusion layer, used to fuse the features extracted by the feature extraction layer and the long short-term memory network layer;
[0151] The attention mechanism layer is used to calculate feature importance weights through the self-attention mechanism;
[0152] The rule judgment layer is used to obtain rule features according to predefined rules. The rule features are binary variables, indicating whether the predefined rules are met.
[0153] The classification layer, including the fully connected layer, activation function and Dropout layer, is used to classify the fused features;
[0154] The output layer, including the Softmax layer, is used to convert the output of the classification layer into a probability distribution.
[0155] Furthermore, the predefined rules include at least one of request frequency rules, data packet size rules, connection duration rules, access path rules, error rate rules, user agent rules, geographic location rules, port scanning rules and protocol usage rules.
[0156] S5. If the high-concurrency behavior recognition result is yes, issue an early warning; perform malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior recognition result, obtain malicious identification results, and execute step S7; if the high-concurrency behavior recognition result is no, execute step S6 first, and then repeat steps S2 to S4;
[0157] Further, performing malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior identification result includes:
[0158] According to the high-concurrency behavior recognition result, a high-concurrency IP corresponding to the first multimodal real-time data set and a high-concurrency user corresponding to the second multimodal real-time data set are obtained; according to the high-concurrency behavior recognition result, high-concurrency users that meet the specified conditions are filtered out to obtain a high-concurrency tag; according to the high-concurrency tag, the first multimodal real-time data set and the second multimodal real-time data set corresponding to the high-concurrency tag are obtained to obtain a first multimodal tag data set and a second multimodal tag data set; the specified condition is: there is only one IP address and the IP address is in the IP list of the high-concurrency IP;
[0159] Performing request frequency analysis on the first multimodal label data set and the second multimodal label data set;
[0160] Construct a historical multi-source threat intelligence database, and establish a blacklist based on the historical multi-source threat intelligence, including malicious IP addresses and domain names; extract multi-mode rules based on the historical multi-source threat intelligence, including behavior mode rules, feature mode rules, communication mode rules, geographic location mode rules and time mode rules; the multi-mode rules are updated in real time;
[0161] The behavior pattern rules include abnormal request frequency and access path pattern;
[0162] The characteristic pattern rules include IP address pattern, domain name pattern and file hash pattern;
[0163] The communication mode rules include JA3 fingerprint mode and data packet feature mode;
[0164] The geographic location pattern rules include geographic location change patterns and IP geographic location matching;
[0165] The time pattern rules include time period anomaly patterns and time interval patterns;
[0166] The multimodal rules and the blacklist are applied to the first multimodal labeled dataset and the second multimodal labeled dataset to detect malicious behavior.
[0167] Furthermore, after obtaining the malicious identification result and before executing the S6 step, the method further includes: if the malicious identification result is yes, issuing a warning.
[0168] S6. Predicting the first multimodal real-time data set and the second multimodal real-time data set respectively to obtain a high-concurrency prediction result;
[0169] Further, respectively predicting the first multimodal real-time data set and the second multimodal real-time data set includes:
[0170] The first multimodal real-time data set and the second multimodal real-time data set are respectively input into the trained high-concurrency prediction model for prediction to obtain a high-concurrency prediction result; the high-concurrency prediction model is trained based on the LSTM model according to the multimodal historical log data set, and is used to predict the probability value of high-concurrency behavior occurring in a specified time period in the future; the multimodal historical log data set is corrected and integrated according to the same user.
[0171] S7. Generate a corresponding defense strategy according to the high-concurrency behavior identification result, the malicious identification result and the high-concurrency prediction result.
[0172] Furthermore, the defense strategy includes: real-time adjustment of network configuration, real-time adjustment of firewall rules, limiting attack source IP, and dynamically allocating computing resources and bandwidth resources.
[0173] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A real-time analysis and monitoring method for network security information data, characterized in that: include: S1. Obtain a multimodal historical log data set, and train a high-concurrency behavior recognition model according to the multimodal historical log data set to recognize the occurrence of high-concurrency behaviors; S2, obtaining real-time network traffic logs, real-time server operation logs and real-time application logs to obtain first integrated data; S3. Integrate the first integrated data according to the IP address to obtain a first multimodal real-time data set; extract device fingerprint information and behavior feature information of each request in the first integrated data; Integrate the first integrated data in combination with the device fingerprint information and the behavior feature information to obtain a second multimodal real-time data set; S4, inputting the first multimodal real-time data set and the second multimodal real-time data set into the high-concurrency behavior recognition model respectively to obtain a high-concurrency behavior recognition result; S5. If the high concurrent behavior identification result is yes, issue an early warning; Perform malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior identification result to obtain a malicious identification result, and execute step S7; If the high concurrency behavior identification result is no, first execute step S6, and then repeat steps S2 to S4; S6. Predicting the first multimodal real-time data set and the second multimodal real-time data set respectively to obtain a high-concurrency prediction result; S7. Generate a corresponding defense strategy according to the high-concurrency behavior identification result, the malicious identification result and the high-concurrency prediction result.
2. A method for real-time analysis and monitoring of network security information data according to claim 1, characterized in that: The multimodal historical log data set specifically comprises: obtaining historical network traffic logs, historical server operation logs and historical application logs, and integrating them to obtain the multimodal historical log data set.
3. The real-time analysis and monitoring method of network security information data according to claim 1 is characterized in that: Integrating the first integrated data according to the IP address to obtain the first multimodal real-time data set includes: According to a predefined log template, template matching is performed on the real-time network traffic log, the real-time server operation log and the real-time application log to obtain first template log data and first non-template log data; for the first template log data, direct parsing and feature extraction are performed to obtain second template log data; for the first non-template log data, an adaptive parsing method is used to dynamically adjust the template or use natural language processing technology to analyze to obtain second non-template log data; the second template log data and the second non-template log data are integrated according to IP addresses to form the first multimodal real-time data set.
4. The real-time analysis and monitoring method of network security information data according to claim 1 is characterized in that: extracting the device fingerprint information and the behavior feature information of each request in the first integrated data; Combining the device fingerprint information and the behavior feature information, integrating the first integrated data includes: Extract browser information in the request, including browser type, version, plug-in, font and screen resolution; extract operating system information, including operating system type, version and language setting; extract request header information, including User-Agent, Accept-Language and Content-Encoding in the HTTP request header; extract hardware information, including the device's CPU type, memory size and time zone; calculate the number of requests, access path sequence, access mode, operation type and response time for each IP address within a specified time period; generate a device fingerprint feature vector and a behavior feature vector after standardization of the device fingerprint information and the behavior feature information, and then concatenate the device fingerprint feature vector and the behavior feature vector to form a comprehensive feature vector; perform similar user identification based on the comprehensive feature vector, identify M similar users, and obtain similar user tags; integrate the first integrated data according to the similar user tags to obtain the second multimodal real-time data set.
5. The real-time analysis and monitoring method of network security information data according to claim 1 is characterized in that: The high-concurrency behavior recognition model includes: The input layer is used to receive multimodal historical log data sets and perform preprocessing and normalization; A feature extraction layer, used for extracting local features from the multimodal historical log data set using a convolutional neural network; comprising multiple convolutional layers and pooling layers; Long short-term memory network layer, composed of multiple LSTM units, used to process time series data; A fusion layer, used to fuse the features extracted by the feature extraction layer and the long short-term memory network layer; The attention mechanism layer is used to calculate feature importance weights through the self-attention mechanism; The rule judgment layer is used to obtain rule features according to predefined rules. The rule features are binary variables, indicating whether the predefined rules are met. The classification layer, including the fully connected layer, activation function and Dropout layer, is used to classify the fused features; The output layer, including the Softmax layer, is used to convert the output of the classification layer into a probability distribution.
6. A method for real-time analysis and monitoring of network security information data according to claim 5, characterized in that: The predefined rules include at least one of request frequency rules, data packet size rules, connection duration rules, access path rules, error rate rules, user agent rules, geographic location rules, port scanning rules and protocol usage rules.
7. A method for real-time analysis and monitoring of network security information data according to claim 1, characterized in that: After obtaining the malicious identification result and before executing the step S6, the method further includes: if the malicious identification result is yes, issuing a warning.
8. The method for real-time analysis and monitoring of network security information data according to claim 1, characterized in that: Performing malicious identification on the first multimodal real-time data set and the second multimodal real-time data set respectively according to the high-concurrency behavior identification result includes: According to the high-concurrency behavior recognition result, a high-concurrency IP corresponding to the first multimodal real-time data set and a high-concurrency user corresponding to the second multimodal real-time data set are obtained; according to the high-concurrency behavior recognition result, high-concurrency users that meet the specified conditions are filtered out to obtain a high-concurrency tag; according to the high-concurrency tag, the first multimodal real-time data set and the second multimodal real-time data set corresponding to the high-concurrency tag are obtained to obtain a first multimodal tag data set and a second multimodal tag data set; the specified condition is: there is only one IP address and the IP address is in the IP list of the high-concurrency IP; Performing request frequency analysis on the first multimodal label data set and the second multimodal label data set; Construct a historical multi-source threat intelligence database, and establish a blacklist based on the historical multi-source threat intelligence, including malicious IP addresses and domain names; extract multi-mode rules based on the historical multi-source threat intelligence, including behavior mode rules, feature mode rules, communication mode rules, geographic location mode rules and time mode rules; the multi-mode rules are updated in real time; The behavior pattern rules include abnormal request frequency and access path pattern; The characteristic pattern rules include IP address pattern, domain name pattern and file hash pattern; The communication mode rules include JA3 fingerprint mode and data packet feature mode; The geographic location pattern rules include geographic location change patterns and IP geographic location matching; The time pattern rules include time period anomaly patterns and time interval patterns; The multimodal rules and the blacklist are applied to the first multimodal labeled dataset and the second multimodal labeled dataset to detect malicious behavior.
9. The method for real-time analysis and monitoring of network security information data according to claim 1, characterized in that: Predicting the first multimodal real-time data set and the second multimodal real-time data set respectively includes: The first multimodal real-time data set and the second multimodal real-time data set are respectively input into the trained high-concurrency prediction model for prediction to obtain a high-concurrency prediction result; the high-concurrency prediction model is trained based on the LSTM model according to the multimodal historical log data set, and is used to predict the probability value of high-concurrency behavior occurring in a specified time period in the future; the multimodal historical log data set is corrected and integrated according to the same user.
10. The real-time analysis and monitoring method of network security information data according to claim 1, characterized in that: The defense strategy includes: real-time adjustment of network configuration, real-time adjustment of firewall rules, limiting attack source IP, dynamic allocation of computing resources and dynamic allocation of bandwidth resources.
Citation Information
Cited By
Multi-source network security log data fusion analysis method and system
CN119854028A
Artificial intelligence-based government and enterprise gateway log analysis method, device and equipment
CN119862567A
Network threat perception defense strategy optimization system and method based on machine learning
CN119996009A
