Intelligent monitoring system and method applied to communication data security

Abstract

The invention discloses an intelligent monitoring system and method applied to communication data security, and particularly relates to the technical field of communication data security, and the method comprises the following steps: collecting a transmission mode and a data transmission behavior of data in a communication network in real time, carrying out the preprocessing of the transmission mode and the data transmission behavior, and generating structured time series data; the method comprises the following steps: vectorizing a transmission mode and a transmission behavior of a current time window, performing clustering analysis in combination with historical data, and respectively identifying groups most similar to the transmission mode and the transmission behavior; time sequence prediction is carried out based on the average frequency of various abnormal activities in the similar group, and potential anomalies in the next time window are deduced; safety monitoring measures are automatically deployed according to a prediction result, and safety early warning information is generated in real time; through combination of the time sequence analysis model and clustering analysis, potential abnormal activities in communication data can be accurately identified and predicted, and a dynamic adjustment mode is adopted to automatically adjust a monitoring strategy according to a prediction result of the abnormal activities.

Description

Technical Field

[0001] The present invention relates to the technical field of communication data security, and more specifically, to an intelligent monitoring system and method for communication data security. Background Art

[0002] With the rapid development of information technology, communication networks have become one of the core infrastructures for daily operations in modern society. Data communications play a vital role in different fields, such as finance, electricity, medical care, transportation, etc. With the increase in data traffic and the increasing diversity of attack methods, the security issues of communication networks have become increasingly prominent and face huge challenges.

[0003] Traditional network security protection mechanisms often seem powerless in the face of increasingly complex network attacks. When faced with high-speed, large-scale, and dynamically changing network environments, it is often difficult to effectively respond to new or complex attack behaviors. Moreover, they usually work based on the currently deployed security monitoring measures to identify known attack models, but have weak recognition capabilities for unknown attacks and new security threats, and are prone to missed reports or false reports, thereby affecting the overall network security protection effect. To this end, the present invention proposes an intelligent monitoring system and method for communication data security in order to solve the above problems. Summary of the invention

[0004] To achieve the above object, the present invention provides the following technical solutions: An intelligent monitoring method for communication data security includes the following steps: Collect the transmission mode and data transmission behavior of data in the communication network in real time, and preprocess it to generate structured time series data; By vectorizing the transmission mode and transmission behavior of the current time window and combining it with historical data for cluster analysis, the most similar groups can be identified. Perform time series forecasting based on the average frequency of various abnormal activities in similar groups to infer potential anomalies in the next time window; Automatically deploy safety monitoring measures based on prediction results and generate safety warning information in real time.

[0005] In a preferred embodiment, the transmission mode includes data flow size, data packet size and type, transmission protocol type, source address, destination address and transmission time interval; Transmission behavior includes the time interval between data request and response, authentication and authorization behavior, data interaction frequency, data flow direction, session duration and cycle, and error and exception handling behavior.

[0006] In a preferred embodiment, vectorizing the transmission mode and transmission behavior of the current time window refers to: The transmission mode and transmission behavior of the current time window are normalized, standardized or encoded respectively to generate a numerical vector that can be used for cluster analysis.

[0007] In a preferred embodiment, the cluster analysis method adopts any one of the preset distance metric-based clustering methods.

[0008] In a preferred embodiment, before inputting the cluster analysis, the cosine similarity is used to calculate the transmission mode and transmission behavior of the current time window, and the state similarity of the transmission mode and transmission behavior of each historical time window in the historical data is respectively calculated. When the state similarity between the transmission mode of the current time window and the historical time window is greater than a preset similarity threshold of one, the historical time window is retained and input into the transmission mode cluster analysis; when the state similarity between the transmission behavior of the current time window and the historical time window is greater than a preset similarity threshold of two, the historical time window is retained and input into the transmission behavior cluster analysis.

[0009] In a preferred embodiment, a preset time series analysis model is used to perform time series prediction based on the average frequency of various abnormal activities in similar groups, and the time series analysis model is a long short-term memory network or an autoregressive integrated moving average model.

[0010] In a preferred embodiment, the autocorrelation coefficient of the transmission behavior or transmission mode time series with itself at lag k is calculated based on the autocorrelation function. , when the autocorrelation coefficient When the autocorrelation coefficient is greater than the preset selection threshold, the autoregressive integral moving average model is used. When it is less than or equal to the preset selection threshold, the long short-term memory network is used.

[0011] In a preferred embodiment, automatically deploying safety monitoring measures according to prediction results refers to: The potential abnormal results of transmission behavior and transmission mode in the next time window are obtained respectively, and the average frequencies of abnormal activities of the same type are merged, and then a weighted sum is performed based on the merged results to obtain the risk coefficient value, and then the risk coefficient value is compared with the preset security level-numeric range division form to obtain the security level corresponding to the numerical range that the risk coefficient value falls into, and the corresponding level of security monitoring measures are automatically deployed according to the security level.

[0012] In a preferred embodiment, an intelligent monitoring system for communication data security includes: The data acquisition module collects the transmission mode and data transmission behavior of data in the communication network in real time, and preprocesses it to generate structured time series data; A vectorization module normalizes, standardizes or encodes the transmission mode and transmission behavior of the current time window to generate a numerical vector that can be used for cluster analysis; Cluster analysis module, based on vectorized transmission mode and transmission behavior in the current time window, and combined with historical data for cluster analysis, to identify the most similar groups; The time prediction module performs time series prediction based on the average frequency of various abnormal activities in similar groups and infers potential anomalies in the next time window; The security deployment module automatically deploys security monitoring measures based on the prediction results and generates security warning information in real time.

[0013] Technical effects and advantages of the present invention: The present invention combines time series analysis models (such as LSTM and ARIMA) with cluster analysis to accurately identify and predict potential abnormal activities in communication data. The system can automatically learn patterns in historical data and predict future security risks based on these patterns, thereby detecting abnormal behavior in advance and avoiding the occurrence of network security threats.

[0014] The present invention adopts a dynamic adjustment method to automatically adjust the monitoring strategy according to the prediction results of abnormal activities. For different network security threats, the system can automatically select the most appropriate monitoring method. For example, if a high-risk abnormal activity is predicted, the system will automatically enhance the protection measures; if the prediction results show that the risk is low, it can be adjusted to a lightweight monitoring mode to optimize resource allocation and improve system operation efficiency.

[0015] Through multi-dimensional analysis of transmission modes and behaviors, the present invention can comprehensively and accurately evaluate various abnormal activities in the network. Compared with traditional detection methods based on single rules or thresholds, the present invention can more deeply identify known and unknown abnormal activities, especially when facing complex and dynamically changing attack behaviors, the system can respond in time and make adaptive adjustments.

[0016] The present invention optimizes the allocation of network security monitoring resources through the weighted summation calculation method of risk coefficient values ​​and the dynamic division of security levels. During high-risk periods, the system can automatically enable more powerful protection measures to ensure network security; during low-risk periods, the system will automatically reduce resource usage, thereby avoiding unnecessary waste of resources and improving overall monitoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings; Figure 1 This is a schematic diagram of a smart monitoring method for communication data security in the present invention.

[0018] Figure 2 This is a schematic diagram of an intelligent monitoring system for communication data security in the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Reference Figure 1 - Figure 2 The following embodiments are obtained: Embodiment 1: With the rapid development of information technology, communication networks have become an indispensable part of modern society. In communication networks, data security and stability are becoming increasingly important, especially in large-scale and complex network environments. Traditional network security monitoring systems mostly rely on rule engines and anomaly detection methods based on fixed thresholds. Although these methods can detect known attacks and abnormal activities, they are weak in identifying unknown and dynamically changing security threats. Especially when facing complex and large-scale data traffic, traditional methods often have false positives and false negatives.

[0021] In this context, intelligent monitoring technology, especially anomaly detection methods based on big data analysis, machine learning, and deep learning, has gradually become a research hotspot in the field of communication data security monitoring. Through in-depth analysis of data transmission patterns and behaviors, potential security threats can be captured more accurately and early warnings can be issued. Therefore, how to intelligently identify abnormal activities and issue accurate warnings through comprehensive analysis of data transmission patterns and behaviors has become an urgent problem to be solved in the current field of communication data security.

[0022] The purpose of the present invention is to provide an intelligent monitoring method for communication data security, which can accurately identify potential security risks and issue early warnings by real-time acquisition and analysis of data transmission patterns and transmission behaviors in communication networks, using cluster analysis and time series prediction technology. Specific objectives include: Enhanced anomaly detection capabilities: Through multi-dimensional analysis of data transmission patterns and behaviors, known and unknown abnormal activities can be identified to overcome the limitations of traditional security detection methods.

[0023] Improve monitoring efficiency and accuracy: Use advanced cluster analysis and time series prediction models to accurately predict abnormal activities in the next time window, so as to deploy security monitoring measures in advance and reduce the possibility of security incidents.

[0024] Dynamically adapt to changes in the network environment: Through learning and analysis based on real-time data, the monitoring strategy can be dynamically adjusted to adapt to the ever-changing network environment and new attack methods.

[0025] Optimize resource allocation: By predicting the average frequency of abnormal activities, reasonably allocate computing and monitoring resources to improve the efficiency of the overall system.

[0026] It should be noted that in communication data security monitoring, known abnormal activities refer to those abnormal patterns that have been identified through historical data accumulation, rule setting or standard security monitoring technology, such as network attacks, malicious traffic, data leakage, etc. These activities can usually be identified and responded to through existing monitoring measures. Their characteristics and behavior patterns can be clearly defined in known attack models, and in existing monitoring systems, they can be detected in real time through rules or fixed thresholds.

[0027] However, unknown abnormal activities are different. They refer to potential threats that have not yet been identified by traditional security monitoring systems. These activities may be new attack methods, complex attack variants, or undiscovered network vulnerabilities. Because the characteristics and behavior patterns of these abnormal activities do not conform to existing security rules or attack models, traditional security detection systems often cannot effectively identify them. Unknown abnormal activities are often used by attackers to evade detection by existing monitoring systems by changing strategies and using covert methods. Therefore, they bring greater risks and are more harmful.

[0028] Calculation and weighted processing of the frequency of various abnormal activities in similar groups: This invention uses similar group analysis to comprehensively consider known and unknown abnormal activities, and dynamically predicts and weights them based on the frequency of abnormal activities to predict the potential security risks of the next time window in advance. Specifically: Calculate the frequency of various abnormal activities in similar groups: First, cluster the data transmission patterns and behaviors of the current time window to identify the groups that are most similar to historical data. Each similar group contains multiple time windows that are highly similar to the current window in transmission patterns and behaviors.

[0029] For each similar group, the frequency of each type of abnormal activity is calculated. These abnormal activities can be known abnormal activities that can be detected by current security monitoring measures, or unknown abnormal activities that cannot be identified by current measures.

[0030] Frequencies of known abnormal activities: These are frequencies that match attack patterns or known anomalies defined by current monitoring measures, such as DDoS attacks, port scans, malware propagation, etc.

[0031] Frequency of unknown abnormal activities: refers to those abnormal activities that are not identified or defined by current monitoring measures, such as new attacks, behavioral disguises, etc.

[0032] By calculating the average frequency of these abnormal activities, we can obtain the average occurrence frequency of each type of abnormal activity in each group.

[0033] Weighted calculation of the risk of abnormal activities: By weighting the frequency of known and unknown abnormal activities, we can reflect the rationality and perfection of security monitoring measures, especially the risk management of unknown abnormal activities. Specifically: The risk values ​​of known abnormal activities are usually relatively stable because the characteristics of these activities have been identified by the monitoring system and existing detection measures can effectively prevent them.

[0034] The risk value of unknown abnormal activities is usually large because they are not easily identified by current security monitoring measures and have potential high risks. In order to better reflect the potential threats of these activities, the weight of unknown abnormal activities can be increased in the present invention, so that the importance of unknown abnormal activities can be more highlighted during prediction.

[0035] The specific weighted calculation formula is: ; Known abnormal activity category The corresponding proportionality coefficient is, Known abnormal activity category The corresponding next time window prediction value, Known abnormal activity category The total number of Unknown abnormal activity category The corresponding next time window prediction value, Unknown abnormal activity category The total number of ; The magnification factor of unknown abnormal activities that cannot be detected by the currently deployed security monitoring measures is set based on the known abnormal activities detected by the currently deployed security monitoring measures. It is the risk coefficient value, and regarding the known abnormal activities monitored by the currently deployed security monitoring measures, there are pre-set proportional coefficients corresponding to all known abnormal activities that can be monitored. It is set according to the impact range brought by the abnormal activity type. To meet the currently deployed security monitoring measures, the sum of the proportional coefficients corresponding to all known abnormal activities that can be monitored is one.

[0036] Through the above calculation formula, the system can adjust the detection strategy according to whether the current security monitoring measures are effective enough. For example, if the risk brought by unknown abnormal activities is high, the system will automatically adjust the security monitoring strategy and upgrade to more advanced detection measures, such as enabling more sophisticated deep learning models, strengthening traffic monitoring, or conducting more in-depth analysis of network behavior. This adaptive adjustment can not only improve the defense capability against unknown threats, but also optimize the allocation of monitoring resources, so that during high-risk periods, the system can deploy more powerful security protection measures to prevent potential security threats in a timely manner. It provides a dynamic and intelligent monitoring solution for communication data security, which can adapt to the ever-changing network environment and ensure the security and stability of the communication network.

[0037] The present invention proposes an intelligent monitoring method for communication data security, comprising the following steps: The transmission mode and data transmission behavior of data in the communication network are collected in real time, and preprocessed to generate structured time series data; in order to ensure that potential security threats in the network can be identified and responded to in a timely manner, key data in the communication network must be obtained in real time. Transmission modes (such as traffic, packet size, protocol, etc.) and transmission behaviors (such as request / response time interval, data interaction frequency, etc.) provide a detailed description of network activities and can reveal potential anomalies in network communications.

[0038] Data preprocessing: Data in communication networks often have problems such as noise, missing values, and outliers, which may affect accuracy if used directly for analysis. Therefore, by preprocessing the data (such as denoising, standardization, missing value supplementation, etc.), the quality of the data can be guaranteed and the effectiveness of subsequent analysis can be ensured.

[0039] Structured time series data: In order to conduct further pattern recognition and prediction, the data needs to be converted into structured time series data. Time series data can reveal the timing characteristics of communication activities and is the basis for subsequent clustering analysis and prediction.

[0040] By vectorizing the transmission mode and behavior of the current time window and combining it with historical data for cluster analysis, the most similar groups can be identified; data transmission mode and behavior are multidimensional, and converting them into vector form is an effective way to express them numerically. This process converts the data into a numerical vector that can be used for cluster analysis by normalizing, standardizing or encoding it.

[0041] Cluster analysis: Cluster analysis can identify the groups in historical data that are most similar to current data based on the similarity of transmission patterns and behaviors. By comparing the similarity of the data in the current time window with each time window in the historical data, the groups that are most similar to the current activity can be identified. By analyzing similar groups, it is possible to find out whether there are potential abnormal patterns, thereby helping to identify security threats.

[0042] Identify similar groups: Through cluster analysis, it is possible to identify the groups in historical data that are most similar to the current transmission patterns and behaviors, ensuring that the security monitoring system can judge potential anomalies based on existing data and patterns.

[0043] Based on the average frequency of various abnormal activities in similar groups, time series prediction is performed to infer potential anomalies in the next time window; similar groups obtained through cluster analysis can help identify which groups have a higher frequency of abnormal activities. The frequency of abnormal activities reflects the probability of a certain type of attack or abnormal event. Analysis of these frequencies can reveal potential security threats.

[0044] Time series forecasting: Based on known historical data and current transmission patterns, time series analysis methods (such as LSTM or ARIMA) are used to predict abnormal activities in future time windows. Time series forecasting helps identify trends in security threats and provide predictive warnings for network security, rather than just reactive monitoring.

[0045] Infer potential anomalies: By analyzing the frequency of abnormal activities in similar groups and combining it with the prediction of time series models, it is possible to foresee abnormal situations that may occur in the next time window, which helps to detect potential threats in advance and take preventive measures.

[0046] Automatically deploy security monitoring measures based on the prediction results and generate security warning information in real time. Based on the prediction of abnormal activities, it is possible to dynamically adjust and automatically deploy appropriate security monitoring measures. For example, firewall rules can be enhanced, deep packet inspection can be enabled, intrusion detection can be added, etc. to deal with predicted potential risks. This automated response improves the flexibility and efficiency of network security defense.

[0047] Generate security warning information in real time: When potential abnormal activities are predicted, the system will generate real-time warnings and promptly notify network administrators or security personnel to intervene. This process can greatly shorten the response time and reduce the losses caused by potential threats.

[0048] Improved security: By combining real-time data collection, prediction of abnormal activities, and automated deployment of security measures, this approach can proactively improve the security of communication networks, reduce the exposure time of security vulnerabilities, and prevent the spread of attacks.

[0049] The transmission mode includes data flow size, data packet size and type, transmission protocol type, source address, destination address and transmission time interval; specifically: Data traffic size and changes: Under normal circumstances, data traffic should follow certain changing patterns, such as peaks and valleys within a specific period of time. Monitoring the range and frequency of these traffic changes can help identify abnormal situations of excessively high or low traffic.

[0050] Packet size and type: Normal data transmission usually has a fixed packet size and a specific format (such as the segment size in the TCP / IP protocol). Any abnormal packet size, frequency, or type that does not conform to this pattern may indicate an attack or malfunction.

[0051] Transport protocol: Different applications or communication services use different protocols (such as HTTP, FTP, DNS, etc.). The protocol type and its behavior (such as TCP handshake, confirmation mechanism, etc.) in normal transmission should be consistent. Any mismatched protocol usage may indicate an attack or illegal access.

[0052] Time interval: Normal data transmission usually has a certain time interval or frequency. For example, some real-time data streams may be transmitted at a fixed time interval. Abnormal time intervals (such as sudden interruptions or too fast requests) may indicate a problem.

[0053] Source and destination addresses: The source and destination addresses of communications usually have regularity, such as the communication flow between specific servers or user devices. When data comes from unknown or unauthorized IP addresses, it can be considered abnormal behavior.

[0054] Transmission behavior includes the time interval between data request and response, identity authentication and authorization behavior, data interaction frequency, data flow direction, session duration and cycle, and error and exception handling behavior. Specifically: Data request and response patterns: In normal communication behavior, data requests and responses follow a certain pattern. For example, after a client requests data, the server will return a response within a certain period of time. Abnormal request patterns (such as frequent requests, no response, etc.) can indicate potential problems.

[0055] Authentication and authorization behavior: Authentication and authorization processes in communication (such as login, token verification, etc.) are usually executed according to established rules. Any behavior that bypasses or tampering with this process may be a security threat.

[0056] Data flow direction: Data in the communication process usually has a clear sender and receiver, and the communication is bidirectional or unidirectional, which meets the application requirements. If the data flow direction is abnormal, it may indicate data leakage or illegal access.

[0057] Session duration and cycle: In normal communication, each session usually has a clear duration and life cycle, and there will be an end mark (such as connection closing). Abnormal session time, or continuous sessions, may indicate an ongoing attack (such as session hijacking).

[0058] Error and exception handling behavior: During the communication process, if there is a problem (such as timeout, connection interruption, etc.), the normal behavior is to have a corresponding error handling mechanism (such as retransmission, disconnection and reconnection, etc.). No error handling or abnormal error handling may be caused intentionally by the attacker.

[0059] Vectorizing the transmission mode and transmission behavior of the current time window refers to: Normalize, standardize or encode the transmission mode and transmission behavior of the current time window to generate a numerical vector that can be used for cluster analysis. Raw data (such as packet size, transmission protocol type, source address, etc.) is often discrete or non-numeric, and machine learning algorithms or clustering algorithms generally require numerical input. Through vectorization, we convert communication data into a numerical representation so that it can be directly input into cluster analysis and prediction models for calculation and analysis.

[0060] The vectorized data can be used to measure and compare the similarity between data through mathematical calculations (such as Euclidean distance, cosine similarity, etc.), which is the basis for clustering analysis or similarity analysis. Standardization and normalization are common data preprocessing methods that can eliminate the scale differences between different features and ensure that each feature is processed under the same standard.

[0061] Standardization: Adjust the mean of the data to 0 and the variance to 1 to make them have the same scale, which helps to avoid the excessive impact of the numerical range of some features on clustering analysis.

[0062] Normalization: Adjust the data to a fixed range (such as [0, 1]) in proportion to avoid the dimension with larger eigenvalues ​​dominating the clustering results, thereby ensuring that each feature can fairly influence the clustering results.

[0063] These processing methods ensure that features from different sources (such as transmission time interval and packet size) are treated fairly, preventing certain features from dominating the model results due to their large values.

[0064] Cluster analysis is essentially about finding similarities in data and grouping them by calculating distance or similarity. After the input data is vectorized, clustering algorithms (such as K-means, DBSCAN, etc.) can more accurately measure the similarity between the current time window and each time window in the historical data.

[0065] Through vectorization, the transmission pattern and behavior of the current time window can be compared and clustered with the data of the historical time window, so as to discover potential pattern anomalies or behavioral anomalies, providing a basis for subsequent risk prediction and security monitoring.

[0066] Vectorized and standardized data can eliminate noise and bias, improve the stability and accuracy of the model. When faced with complex data sets, this processing method can help identify more subtle abnormal behaviors and effectively improve the detection capabilities of unknown attacks or abnormal activities.

[0067] Through unified numerical expression, errors caused by data format problems can be avoided, allowing the model to better extract valuable features from the data and improve the ability to identify unknown threats.

[0068] Through encoding processing (for example, one-hot encoding, label encoding, etc.), some category features (such as protocol type, source address, etc.) are converted into numerical form to further enhance the expressiveness of the features.

[0069] These numerical vectors can better represent the multidimensional information of the data, enabling clustering algorithms to efficiently classify and cluster data in different dimensions, helping to discover potential rules and patterns in the data.

[0070] The clustering analysis method uses any of the preset distance-based clustering methods. The transmission patterns and behaviors of communication data may be distributed in different forms: some transmission behaviors may be stable and periodic, suitable for K-means, while other behaviors may be more complex, with higher noise or density changes, suitable for DBSCAN or K-medoids. Therefore, using any of the distance-based clustering methods, you can flexibly choose the most suitable algorithm according to the data characteristics.

[0071] Algorithm substitutability: In the process of cluster analysis, using different distance measurement methods and clustering algorithms often produces similar clustering results. In the present invention, since the goal is to identify historical data groups that are similar to the current window transmission behavior, and the transmission modes and behavior characteristics of these groups may be diverse, using different distance measurement methods (such as Euclidean distance, Manhattan distance, etc.) to evaluate similarity and using different clustering algorithms for clustering can achieve the purpose better. The network environment and data transmission behavior will change according to time and load, so it is necessary to select a suitable clustering method according to the actual scenario. For example, if the data flow is relatively stable, K-means may be more efficient; when anomalies or new transmission modes occur in the network, DBSCAN can discover unknown clusters and effectively deal with abnormal data.

[0072] Taking K-means as an example, cluster analysis is performed on the transmission mode of data in the communication network: In the present invention, K-means clustering is applied to analyze the data transmission pattern in the communication network to identify the historical group that is most similar to the data transmission pattern in the current time window. The transmission pattern of communication data includes multiple features, such as packet size, transmission protocol type, source address and destination address, transmission time interval, etc. Through cluster analysis of these features, we can divide the transmission pattern in the communication network into several clusters (groups), each cluster represents a similar network communication behavior pattern. This is of great significance for subsequent anomaly detection and prediction.

[0073] Initialize cluster centers (centroids): Assume that you need to perform cluster analysis on the transmission modes in the communication data. Each transmission mode consists of multiple features (such as packet size, transmission protocol type, source and destination addresses, etc.). First, you need to convert these features into numerical data through vectorization and perform standardization or normalization to ensure that each feature is compared on the same scale.

[0074] The K-means algorithm randomly selects K transmission mode instances as the initial cluster centers according to the pre-set number of clusters K. This cluster center represents the "center point" of the initial guess, and assigns each data point in the data set (i.e., each transmission mode) to the closest cluster center.

[0075] Assigning data points (transmission patterns): Each data point (i.e., each transmission pattern) will determine the cluster it belongs to based on the calculated distance to the K cluster centers (usually using Euclidean distance). The smaller the distance of the transmission pattern feature, the more similar it is to the cluster center. This step assigns all transmission patterns to the nearest cluster, and each cluster contains transmission patterns with similar characteristics. For example, transmission patterns with similar packet sizes and transmission protocol types will be assigned to the same cluster, and transmission patterns with similar characteristics such as source and destination addresses will also be classified into the same cluster.

[0076] Update cluster centers: After completing cluster assignment, the K-means algorithm recalculates the center point (center of mass) of each cluster. The new cluster center is the average of the characteristic values ​​of all transmission modes in the cluster. For example, the mean of the packet size and transmission time interval of each cluster will become the characteristic value of the new cluster center.

[0077] The updated cluster center more accurately reflects the overall characteristics of the transmission pattern within the cluster, thereby helping the algorithm to better represent the center position of each cluster.

[0078] Repeated iteration: After updating the cluster center, the K-means algorithm will re-assign the data to clusters. Each data point will recalculate its distance from each cluster center to determine whether it needs to be reassigned to another cluster. This process will continue to iterate until the cluster center position no longer changes significantly or the predetermined number of iterations is reached.

[0079] In K-means clustering, choosing the right number of clusters K is a crucial step. If the K value is too small, it may lead to too broad clustering and fail to effectively distinguish different communication modes; if the K value is too large, it will lead to too fine clusters and may cause overfitting. You can use the elbow rule to select the number of clusters K that optimizes the clustering effect. The specific steps are as follows: Perform K-means clustering: For different cluster numbers K (starting from 1 to the set maximum K value), execute the K-means algorithm and calculate the clustering effect under each K value.

[0080] Calculate the sum of squared errors (SSE) for each K value: The sum of squared errors (SSE) is an important indicator for measuring clustering quality. It represents the sum of the squares of the distances from the data points to their cluster centers.

[0081] Plot the relationship between SSE and K value: Plot the SSE value corresponding to each K value into a curve graph. As the K value increases, the SSE will gradually decrease.

[0082] Find the elbow position: Observe the SSE curve. When the K value increases, the rate of decrease of SSE will gradually slow down. At a certain point on the curve, the amplitude of the decrease of SSE suddenly decreases, forming an "elbow" shape. The K value corresponding to this elbow is the optimal number of clusters, because it avoids over-segmentation while ensuring a reasonable classification effect. Through the elbow rule, a suitable K value can be determined so that the clustering results can effectively distinguish different communication data transmission modes, and will not lead to an overly complex model due to an excessively large K value.

[0083] Similarly, by analyzing the transmission behavior of communication data through K-means clustering, similar transmission behaviors can be classified into one category. I will not go into details here. Cluster analysis can be combined with historical data to explain the risks of the current window, which is important for understanding the current network status and assessing potential security risks. For example, if the behavior of the current window is close to the abnormal group in the historical cluster (that is, the abnormal behavior in the historical data), this indicates that some potential abnormal activities may have occurred in the current window, which is important for understanding the current network status and assessing potential security risks.

[0084] Before inputting the cluster analysis, the transmission mode and transmission behavior of the current time window are calculated using cosine similarity, and the state similarity of the transmission mode and transmission behavior of each historical time window in the historical data is respectively calculated. When the state similarity between the transmission mode of the current time window and the historical time window is greater than the preset similarity threshold of one, the historical time window is retained and input into the transmission mode cluster analysis; when the state similarity between the transmission behavior of the current time window and the historical time window is greater than the preset similarity threshold of two, the historical time window is retained and input into the transmission behavior cluster analysis.

[0085] Cosine similarity is a method to measure the angular similarity between two vectors and is often used to calculate the similarity of high-dimensional data. In the present invention, the transmission mode and transmission behavior are converted into vector form in the time window, and cosine similarity can quantify the similarity between the data in the current time window and the data in the historical time window. Its core advantage is that cosine similarity can effectively eliminate the difference in data size and pay more attention to the directionality of the data (i.e., the similarity of the pattern), thereby improving the accuracy of similarity calculation.

[0086] By setting preset similarity thresholds (such as threshold 1 for transmission mode and threshold 2 for transmission behavior), we can ensure that only historical data with high similarity to the current time window is selected for comparison during cluster analysis. This avoids the interference of irrelevant data due to the large difference between historical data and the current time window, and improves the accuracy and effect of cluster analysis.

[0087] The transmission mode and transmission behavior of communication data belong to two different types of features. The former focuses on the structure of the data packet, the protocol and traffic characteristics during the transmission process, while the latter focuses on the behavioral characteristics of data exchange such as time interval, frequency, and session. Therefore, calculating the similarity of the transmission mode and transmission behavior separately can ensure that the differences between these two types of features are taken into account during clustering, so as to conduct targeted analysis.

[0088] Different similarity thresholds are set for transmission mode and transmission behavior, because they have different importance and variation in communication behavior. For example, transmission mode may focus more on regular features such as traffic fluctuations and protocol usage, which have smaller fluctuations; while transmission behavior may focus more on event response time, interaction frequency, etc., which may have more abnormal fluctuations. Therefore, setting different thresholds can better adjust the sensitivity of the analysis and ensure the effectiveness of data clustering.

[0089] By pre-screening the historical time windows similar to the current time window, only the most relevant historical data is input during cluster analysis, avoiding irrelevant or abnormal data in the historical data affecting the clustering results. This greatly improves the efficiency and accuracy of cluster analysis and reduces the occurrence of errors. The screened similar historical data provides a more accurate "training sample" for subsequent cluster analysis, making the clustering results more consistent with the transmission behavior patterns in the actual communication network, thus facilitating subsequent security risk assessment and identification of abnormal activities.

[0090] The preset time series analysis model is used to perform time series prediction based on the average frequency of various abnormal activities in similar groups. The time series analysis model is a long short-term memory network or an autoregressive integrated moving average model. Take the average frequency of abnormal activities in similar groups of transmission modes as an example: Prediction steps of LSTM model: Data preparation and input: The input data is the average frequency of one type of abnormal activity in a similar group. These data form a time series, in which the input data at each time point represents the frequency of abnormal activity in a certain time window. The data usually needs to be normalized or standardized to ensure that the LSTM network can better learn the data pattern.

[0091] LSTM network architecture: The LSTM model consists of multiple LSTM units, each of which has a "forget gate", "input gate" and "output gate". These structures can effectively capture the dependencies of time series with longer time spans. LSTM learns the long-term and short-term characteristics of time series through these gating mechanisms. In this application, the input of the LSTM network is the frequency of abnormal activities in the historical time window, and the network learns the time dependency of these frequencies.

[0092] Model training and prediction: Training process: Use the abnormal activity frequency data of the historical time window as the training set, and update the network weights through the back propagation algorithm so that the LSTM model can predict the abnormal activity frequency at the next moment. Prediction step: After training, the LSTM network uses the input historical data to predict the abnormal activity frequency of the next time window. The predicted value can be expressed as the trend value of this type of abnormal activity in the next time window.

[0093] The ARIMA (Autoregressive Integrated Moving Average) model can model the average frequency of each type of abnormal activity separately. Since in the present invention, the frequency of abnormal activities of each similar group is an independent time series data, each type of abnormal activity (such as abnormal data packet size, abnormal transmission time interval, etc.) can be used as an independent time series and modeled and predicted separately using the ARIMA model.

[0094] The core idea of ​​the ARIMA model is to model and predict time series data through three main parts: autoregression (AR), differencing (I), and moving average (MA). Specifically, the ARIMA model assumes that future values ​​are determined by past values, past errors (noise), and data differencing. For each type of average frequency of abnormal activity, we need to build an ARIMA model for each time series.

[0095] Modeling steps of ARIMA model: Data preprocessing and stationarity test: Before using ARIMA model, it is necessary to test the stationarity of time series data. Stationarity means that the mean and variance of the data remain constant over time. If the data is non-stationary, it is necessary to perform difference processing (i.e., calculate the difference between the series and its own lagged value) until the data becomes stationary. For example, suppose you want to predict the time series of abnormal packet size, you first need to test the stationarity of the abnormal frequency series of packet size, and perform difference (such as first-order difference) if necessary.

[0096] Choosing the parameters of the ARIMA model: The ARIMA model is usually expressed in the following form: ; p represents the order of the autoregressive (AR) term, which indicates the impact of past data on the current value. d represents the number of differences, which indicates the number of differences that make the time series stable. q represents the order of the moving average (MA) term, which indicates the impact of past errors on the current value. For each type of time series of abnormal activities, the appropriate p and q values ​​can be selected through the ACF (autocorrelation function) and PACF (partial autocorrelation function) graphs. ACF and PACF graphs can help identify the autocorrelation of data, thereby determining the order of the AR and MA parts, which is well known to personnel in the technical field corresponding to the present invention and will not be repeated here.

[0097] Model training and fitting: Use the abnormal activity frequency data of the historical time window (such as the frequency of abnormal data packet size) to train the ARIMA model. During the training process, ARIMA adjusts the model parameters based on the past values ​​and errors of the data to minimize the error between the predicted value and the actual data. This process usually includes fitting the model, optimizing the parameters, and evaluating the model based on the error of the training set.

[0098] Time series forecasting: After training, the ARIMA model can be used to predict the frequency of abnormal activities in future time windows. The predicted value is calculated based on the autoregressive relationship of historical data, differential operations, and past error information. For example, for abnormal packet size, the ARIMA model predicts the frequency of abnormal packet size in future time windows based on historical abnormal data and the model's autoregressive parameters.

[0099] In the present invention, for each type of abnormal activity (e.g., abnormal frequency of data request, abnormal frequency of identity authentication, abnormal frequency of data packet size, etc.), a separate ARIMA model is constructed, and these models are used to predict the frequency of abnormal activities in the next time window. By using ARIMA models to predict different types of abnormal activities, a detailed risk assessment picture can be obtained, providing strong support for the security monitoring system.

[0100] Calculate the autocorrelation coefficient between the transmission behavior or transmission mode time series and itself at lag k based on the autocorrelation function , when the autocorrelation coefficient When the autocorrelation coefficient is greater than the preset selection threshold, the autoregressive integral moving average model is used. When it is less than or equal to the preset selection threshold, the long short-term memory network is used. The calculation formula is: ; It is a time series In lag The autocorrelation coefficient when . It is the observed value of the time series data (i.e., the frequency value of various abnormal activities) at time point t. is the mean of the time series (which can be calculated by the mean of the overall abnormal activity frequency), is the total number of data points, It's a time lag.

[0101] When the autocorrelation coefficient is large, it means that the behavior of the current time window (such as transmission behavior or pattern) is strongly dependent on the previous behavior. In this case, the autoregressive model (AR) is usually selected for prediction because the autoregressive model focuses on the relationship between current data and past data. In the case where the transmission behavior is strongly dependent on the past state, the AR model can effectively capture this sequential correlation and therefore can more accurately predict future abnormal activities.

[0102] When the autocorrelation coefficient is small, it means that there is almost no dependency between the behavior of the current time window and past time points, so a more complex model is used, such as the long short-term memory network (LSTM). LSTM is able to capture long-term dependencies and is suitable when the data does not have simple, immediate past correlations. In this case, LSTM is better able to model the long-term dependencies of the data and adapt to changes in long-term patterns even if there is no significant short-term correlation in the data.

[0103] By selecting the model based on the size of the autocorrelation coefficient, the system can adaptively select the most appropriate model based on the specific characteristics of the data. If the data exhibits obvious short-term dependencies, a simple autoregressive model (AR) can be used; if the data exhibits more complex long-term dependencies, an LSTM network is used for prediction. This approach ensures that the selected model best adapts to the characteristics of the data, thereby improving the accuracy of the prediction while reducing computational complexity.

[0104] Automatically deploying security monitoring measures based on prediction results means: The potential abnormal results of the transmission behavior and transmission mode in the next time window are obtained respectively, and the average frequencies of abnormal activities of the same type are merged. The merging means that the average frequencies of abnormal activities of the types corresponding to the transmission behavior and transmission mode are added according to categories to obtain the merged value of the average frequencies of abnormal activities of each type, and then a weighted sum is performed based on the merged result to obtain the risk coefficient value, and then the risk coefficient value is compared with the preset security level-numeric range division form to obtain the security level corresponding to the numerical range that the risk coefficient value falls into, and the corresponding level of security monitoring measures are automatically deployed according to the security level.

[0105] The mechanism of automatically deploying security monitoring measures is designed to dynamically adjust network security monitoring strategies based on the prediction results of potential abnormal activities in the current time window. This process not only improves the efficiency of security monitoring, but also responds to possible threats more accurately. Specifically, through the following steps, the system can adaptively adjust the monitoring intensity, ensure the rational use of resources, and improve the accuracy of network security protection.

[0106] The system first analyzes the transmission behavior and transmission pattern of the current window based on a time series prediction model (such as LSTM or ARIMA) to predict abnormal activities that may occur in the next time window. These abnormal activities include changes in packet size, traffic, protocol type, etc. For each type of abnormal activity, the system calculates the potential abnormal frequency, which indicates the possibility of such activity occurring in the future. The frequencies of different types of abnormal activities are integrated so that the frequencies of the same type of abnormal activities can reflect the overall potential risk. This merger not only simplifies the data, but also improves the effectiveness of subsequent decisions.

[0107] Based on the combined frequency of abnormal activities, the system will weight the risk of each abnormal activity according to the preset weights and calculate the risk coefficient value. The weighting process takes into account the impact of each type of abnormal activity on the overall security risk. After the risk coefficient value is calculated, the system will compare it with the preset security level-value range division form. This form divides different security levels (for example, high, medium, and low) according to different risk coefficient values. Each security level corresponds to a different risk level and determines the strength of the security measures that need to be taken. For example, if the risk coefficient is very high, indicating a serious security threat, the system will give a "high" security level and require more stringent monitoring measures.

[0108] Automatically deploy security monitoring measures based on security level: Based on the risk factor and security level obtained from the prediction results, the system will automatically adjust the current security monitoring measures. For example: High risk level: may trigger comprehensive security protection measures, such as strengthening traffic monitoring, enabling deep packet inspection, strengthening network isolation, etc. Medium risk level: may trigger certain monitoring upgrades, such as strengthening the review of transmission protocols and conducting more frequent behavioral analysis. Low risk level: maintain existing monitoring measures and maintain normal network traffic monitoring.

[0109] Embodiment 2: An intelligent monitoring system for communication data security, comprising: The data acquisition module collects the transmission mode and data transmission behavior of data in the communication network in real time, and preprocesses it to generate structured time series data; A vectorization module normalizes, standardizes or encodes the transmission mode and transmission behavior of the current time window to generate a numerical vector that can be used for cluster analysis; Cluster analysis module, based on vectorized transmission mode and transmission behavior in the current time window, and combined with historical data for cluster analysis, to identify the most similar groups; The time prediction module performs time series prediction based on the average frequency of various abnormal activities in similar groups and infers potential anomalies in the next time window; The security deployment module automatically deploys security monitoring measures based on the prediction results and generates security warning information in real time.

[0110] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0111] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0112] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An intelligent monitoring method for communication data security, characterized in that: The following steps are involved: Collect the transmission mode and data transmission behavior of data in the communication network in real time, and preprocess it to generate structured time series data; By vectorizing the transmission mode and transmission behavior of the current time window and combining it with historical data for cluster analysis, the most similar groups can be identified. Time series prediction based on the average frequency of various abnormal activities in similar groups to infer potential anomalies in the next time window; Automatically deploy safety monitoring measures based on prediction results and generate safety warning information in real time.

2. The intelligent monitoring method for communication data security according to claim 1, characterized in that: The transmission mode includes data flow size, data packet size and type, transmission protocol type, source address, destination address and transmission time interval; Transmission behavior includes the time interval between data request and response, authentication and authorization behavior, data interaction frequency, data flow direction, session duration and cycle, and error and exception handling behavior.

3. The intelligent monitoring method for communication data security according to claim 2, characterized in that: Vectorizing the transmission mode and transmission behavior of the current time window refers to: The transmission mode and transmission behavior of the current time window are normalized, standardized or encoded respectively to generate a numerical vector that can be used for cluster analysis.

4. The intelligent monitoring method for communication data security according to claim 3 is characterized in that: The cluster analysis method adopts any one of the preset clustering methods based on distance measurement.

5. The intelligent monitoring method for communication data security according to claim 4, characterized in that: Before cluster analysis is input, the transmission mode and transmission behavior of the current time window are calculated using cosine similarity, and the state similarity of the transmission mode and transmission behavior of each historical time window in the historical data is calculated. When the state similarity between the transmission mode of the current time window and the state similarity of the historical time window is greater than the preset similarity threshold of one, the historical time window is retained and input into the transmission mode cluster analysis; When the state similarity between the current time window transmission behavior and the historical time window is greater than a preset similarity threshold of two, the historical time window is retained and input into the transmission behavior cluster analysis.

6. The intelligent monitoring method for communication data security according to claim 5, characterized in that: A preset time series analysis model is used to perform time series prediction based on the average frequency of various abnormal activities in similar groups. The time series analysis model is a long short-term memory network or an autoregressive integrated moving average model.

7. The intelligent monitoring method for communication data security according to claim 6, characterized in that: Calculate the autocorrelation coefficient between the transmission behavior or transmission mode time series and itself at lag k based on the autocorrelation function , when the autocorrelation coefficient When the autocorrelation coefficient is greater than the preset selection threshold, the autoregressive integral moving average model is used. When it is less than or equal to the preset selection threshold, the long short-term memory network is used.

8. The intelligent monitoring method for communication data security according to claim 7, characterized in that: Automatically deploying security monitoring measures based on prediction results means: The potential abnormal results of transmission behavior and transmission mode in the next time window are obtained respectively, and the average frequencies of abnormal activities of the same type are merged, and then a weighted sum is performed based on the merged results to obtain the risk coefficient value, and then the risk coefficient value is compared with the preset security level-numeric range division form to obtain the security level corresponding to the numerical range that the risk coefficient value falls into, and the corresponding level of security monitoring measures are automatically deployed according to the security level.

9. An intelligent monitoring system for communication data security, used to implement an intelligent monitoring method for communication data security as claimed in any one of claims 1 to 8, characterized in that: include: The data acquisition module collects the transmission mode and data transmission behavior of data in the communication network in real time, and preprocesses it to generate structured time series data; A vectorization module normalizes, standardizes or encodes the transmission mode and transmission behavior of the current time window to generate a numerical vector that can be used for cluster analysis; Cluster analysis module, based on vectorized transmission mode and transmission behavior in the current time window, and combined with historical data for cluster analysis, to identify the most similar groups; The time prediction module performs time series prediction based on the average frequency of various abnormal activities in similar groups and infers potential anomalies in the next time window; The security deployment module automatically deploys security monitoring measures based on the prediction results and generates security warning information in real time.

Images