Communication monitoring system based on artificial intelligence

Abstract

The invention discloses a communication monitoring system based on artificial intelligence. The communication monitoring system comprises a server module, and a data acquisition module, a data storage module, a data processing module, a monitoring module and a monitoring display module which are connected with the server module, the data acquisition module is responsible for collecting original data from a communication network; the data storage module is responsible for storing the collected data; the data processing module is responsible for further processing and analyzing the stored data; and the monitoring module is responsible for performing real-time detection and analysis on the processed data by using a deep learning model. According to the invention, a plurality of technical means are adopted to ensure the accuracy and integrity of data, such as data cleaning, noise data identification and processing and the like, so that the robustness and reliability of the system are improved; and the monitoring display module can display monitored data, results and alarm information to a user in a visual and understandable manner.

Description

Technical Field

[0001] The present invention relates to the technical field of communication monitoring, and in particular to a communication monitoring system based on artificial intelligence. Background Art

[0002] With the rapid development of communication technology, the scale of communication networks continues to expand, and the complexity and importance of communication systems are becoming increasingly prominent. In order to ensure the normal operation of communication systems, communication monitoring systems have emerged. These systems are usually based on advanced computer technology, communication technology, electronic technology and automatic control technology. Through real-time monitoring, data collection, processing and analysis of communication equipment and networks, they can achieve comprehensive monitoring and management of communication networks. With the development of artificial intelligence and big data technology, communication monitoring systems have gradually realized intelligent management, which can automatically identify faults, predict equipment status, optimize network configuration, etc.

[0003] For example, some existing communication security monitoring system technologies use a processing module to process data information, an encryption and decoding module to encrypt and decode data information, and a network monitoring module to detect the network. Although it can achieve a certain degree of communication security monitoring, the accuracy and completeness of the data need to be further improved. In addition, this technical solution cannot give intuitive monitoring results. Summary of the invention

[0004] To this end, the present invention provides a communication monitoring system based on artificial intelligence to solve the problem in the prior art that the system is complex and cannot give intuitive monitoring results.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A communication monitoring system based on artificial intelligence, including a server module and a data acquisition module, a data storage module, a data processing module, a monitoring module, and a monitoring display module connected thereto;

[0007] The data acquisition module is responsible for collecting raw data from the communication network and selecting the data source to be collected according to the detection requirements;

[0008] The data storage module is responsible for storing the collected data;

[0009] The data processing module is responsible for further processing and analyzing the stored data to extract useful information and analyze the causes of abnormal values, and transmit the processed data to the monitoring module;

[0010] The monitoring module is responsible for real-time detection and analysis of processed data using a deep learning model; when abnormal behavior is detected, an alarm mechanism is triggered and an alarm message is sent;

[0011] The monitoring and display module is responsible for displaying the monitored data, results and alarm information to the user in an intuitive and easy-to-understand manner.

[0012] Furthermore: the data processing module automatically cleans, extracts features, analyzes and standardizes the data to extract useful feature information.

[0013] Further: the deep learning model identifies DDoS attacks, malware propagation, insider misconduct, network sniffing and man-in-the-middle attacks by analyzing abnormal data packets in network traffic;

[0014] The deep learning model identifies DDoS attacks by analyzing abnormal data packets in network traffic;

[0015] The deep learning model identifies malware propagation behavior by analyzing abnormal file transfer behavior in network traffic;

[0016] The deep learning model identifies these improper behaviors by analyzing abnormal access records in system logs;

[0017] The deep learning model identifies network sniffing and man-in-the-middle attacks by analyzing abnormal transmission patterns in network traffic.

[0018] Further: the data acquisition module transmits the collected data to the data storage module in real time for storage;

[0019] The data processing module reads data from the data storage module and processes and analyzes it, and then transmits the processed data to the monitoring module for real-time monitoring;

[0020] The monitoring module uses the processed data to monitor the communication network in real time and triggers corresponding alarm and response mechanisms based on the monitoring results;

[0021] The monitoring module will also feed back the monitoring results to the data acquisition module and the data processing module so that they can optimize and adjust according to the monitoring results to improve the accuracy and efficiency of the monitoring system.

[0022] Further: the data processing module includes a cleaning module, a feature extraction module, an analysis module and a conversion module, wherein the cleaning module identifies the noise data in the single-dimensional data set through data distribution characteristics and a box plot method, and cleans it;

[0023] The feature extraction module can automatically extract useful features from communication data based on a deep learning algorithm;

[0024] The analysis module uses the association rule Apriori algorithm to explore the association relationship between different features in the communication data;

[0025] The conversion module can convert the data into the format required by the monitoring module.

[0026] Further: The cleaning method of the cleaning module is as follows:

[0027] (1) Using data distribution characteristics to identify noisy data

[0028] Dataset partitioning: Divide the data set into multiple intervals. The number of intervals is determined based on the size and distribution characteristics of the data.

[0029] Determine the data concentration domain: By observing the distribution of data in each interval, identify the interval where the data is mainly concentrated, that is, the data concentration domain; the data in this interval is usually normal data, and the data far away from this interval is noise data;

[0030] (2) Using the box plot method to identify and clean noise data

[0031] Construct a box plot: The box plot can intuitively display the central trend, dispersion and outliers of the data; in the box plot, it contains the first quartile Q1, the third quartile Q3 and the interquartile range IQR;

[0032] Identify outliers: According to the definition of the box plot, outliers are usually defined as data points that are smaller than Q1-3IQR or larger than Q3+3IQR; these data points are far away from the central domain of the data set and are likely to be noise data;

[0033] Cleaning noisy data: For the identified outliers, different processing strategies are adopted, including direct deletion, replacement with the mean or median.

[0034] Further: When cleaning noisy data, caution is required to avoid accidentally deleting normal data or introducing new noise.

[0035] Further: Before cleaning the noisy data, the data needs to be dimensionless processed to eliminate the impact of units on missing value cleaning.

[0036] Further: the features extracted by the feature extraction module include network traffic pattern, data packet size, transmission time, source address and destination address.

[0037] Further: Apply the Apriori algorithm: take the preprocessed data as input, set the minimum support threshold, use the Apriori algorithm to find the frequently occurring extracted feature information; generate association rules from the frequent item sets, and calculate the support and confidence of each rule.

[0038] The present invention has the following advantages:

[0039] 1. The present invention adopts a variety of technical means to ensure the accuracy and integrity of data, such as data cleaning, noise data identification and processing, etc., which improves the robustness and reliability of the system.

[0040] 2. The data processing module provided in the present invention can extract valuable information and patterns through a series of operations such as cleaning, feature extraction, and analysis, thereby providing strong support for subsequent monitoring and analysis; the feature extraction module provided is based on a deep learning algorithm, and can automatically extract useful features from communication data, thereby improving the accuracy and efficiency of feature extraction; the analysis module provided utilizes the association rule Apriori algorithm, and can explore the correlation between different features in the communication data, thereby helping to discover potential communication patterns and abnormal behaviors; the monitoring and display module provided can display the monitored data, results, and alarm information to the user in an intuitive and easy-to-understand manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing the present application; for example, those skilled in the art are capable of easily making conventional adjustments or further optimizations to the addition / reduction / attribution division, specific shapes, positional relationships, connection methods, dimensional ratios, etc. of certain units (components) based on the technical concepts and exemplary drawings disclosed in the present application.

[0042] Figure 1 A schematic diagram of the structure of an artificial intelligence-based communication monitoring system provided for one embodiment of the present invention.

[0043] Figure 2 It is a structural schematic diagram of a data processing module in the communication monitoring system of the present invention. DETAILED DESCRIPTION

[0044] The following is an explanation of the implementation of the present invention by specific specific examples. People familiar with this technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. It should be understood that these embodiments are only to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Technical engineers in this field can make some non-essential improvements and adjustments to the present invention based on the contents of the above invention; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1, a communication monitoring system based on artificial intelligence, including a server module and a data acquisition module, a data storage module, a data processing module, a monitoring module, and a monitoring display module connected thereto.

[0046] The data acquisition module is responsible for collecting raw data from the communication network and selecting the data source to be collected according to the detection requirements, such as sensor data, network traffic data, log files, etc.; an API interface can be used to obtain data from the data source.

[0047] The data storage module is responsible for storing the collected data for subsequent data processing and analysis; the module can use high-performance storage devices and technologies, and select appropriate storage media according to the type and scale of the data, such as SSD hard drives, RAID technology, distributed storage systems, relational databases, NoSQL databases, etc., to ensure data security and reliability; at the same time, the data storage module can also use data backup and recovery technology to prevent data loss or damage.

[0048] In addition, stored data can be indexed to increase the speed and efficiency of data retrieval.

[0049] The data processing module is responsible for further processing and analyzing the stored data to extract useful information; the data processing module can automatically clean, convert and standardize the data to extract useful feature information.

[0050] The monitoring module is responsible for real-time detection and analysis of processed data using a deep learning model; timely detection of abnormal behavior and potential threats; when abnormal behavior is detected, an alarm mechanism is triggered and an alarm message is sent so that timely measures can be taken to respond.

[0051] By analyzing abnormal data packets in network traffic, the deep learning model can identify possible threats such as DDoS attacks and malware propagation.

[0052] DDoS attack: Distributed denial of service (DDoS) attack is a common network attack method. The attacker controls a large number of botnets to send a large number of requests to the target server, causing the server resources to be exhausted and unable to provide services normally. Deep learning models can identify DDoS attacks by analyzing abnormal data packets in network traffic (such as a large number of identical requests from different source addresses).

[0053] Malware propagation: Malware is usually spread through email, malicious websites, etc. Deep learning models can identify malware propagation behaviors by analyzing abnormal file transfer behaviors in network traffic (such as sending a large number of suspicious files to unknown addresses).

[0054] Internal staff misconduct: In the enterprise network, internal staff misconduct (such as leaking sensitive information, illegally accessing unauthorized resources, etc.) may also pose a potential threat. Deep learning models can identify these misconducts by analyzing abnormal access records in system logs (such as frequent logins during non-working hours, access to unauthorized resources, etc.).

[0055] Network sniffing and man-in-the-middle attacks: Network sniffing and man-in-the-middle attacks are two common network attack methods. Attackers steal sensitive information or disrupt the communication process by intercepting and tampering with network traffic. Deep learning models can identify these attacks by analyzing abnormal transmission patterns in network traffic (such as abnormal forwarding of data packets, abnormal decryption of encrypted traffic, etc.).

[0056] The monitoring and display module is responsible for displaying the monitored data, results and alarm information to the user in an intuitive and easy-to-understand manner; the module can use various visualization technologies and tools, such as charts, dashboards, maps, etc.; through these technologies and tools, the monitoring and display module can convert complex data and results into intuitive graphics and images, helping users to quickly understand the status and performance of the communication system and make corresponding decisions and processing.

[0057] The data acquisition module transmits the collected data to the data storage module in real time for storage;

[0058] The data processing module reads data from the data storage module and processes and analyzes it, and then transmits the processed data to the monitoring module for real-time monitoring;

[0059] The monitoring module uses the processed data to monitor the communication network in real time and triggers corresponding alarm and response mechanisms based on the monitoring results;

[0060] The monitoring module can also feed back the monitoring results to the data acquisition module and the data processing module so that they can optimize and adjust according to the monitoring results to improve the accuracy and efficiency of the monitoring system.

[0061] like Figure 2 As shown, the data processing module includes a cleaning module, a feature extraction module, an analysis module and a conversion module. The cleaning module identifies the noise data in the single-dimensional data set through data distribution characteristics and box plot method, and cleans it. The specific method is as follows:

[0062] (1) Using data distribution characteristics to identify noisy data

[0063] Dataset division:

[0064] Divide the data set into multiple intervals. The number of intervals (αn) can be determined based on the size and distribution characteristics of the data. Common division methods include equal-width binning and equal-depth binning.

[0065] Determine the data center domain:

[0066] By observing the distribution of data in each interval, we can identify the interval where the data is mainly concentrated, that is, the data concentration domain; the data within this interval is usually normal data, while the data far away from this interval may be noise data.

[0067] (2) Using the box plot method to identify and clean noise data

[0068] Construct a box plot:

[0069] A box plot is a statistical chart used to describe the distribution characteristics of data. It can intuitively display the central tendency, dispersion and outliers of the data. In a box plot, it usually contains the first quartile (Q1), the third quartile (Q3) and the interquartile range (IQR, i.e. Q3-Q1).

[0070] Identifying outliers:

[0071] According to the definition of the box plot, outliers are usually defined as data points that are smaller than Q1-3IQR or larger than Q3+3IQR. These data points are far away from the center of the data set and are likely to be noise data.

[0072] Clean noisy data:

[0073] Different processing strategies can be adopted for the identified outliers, such as direct deletion, replacement with the mean or median, etc. The specific strategy should be determined according to the characteristics of the data and business needs.

[0074] When cleaning noisy data, caution is required to avoid accidentally deleting normal data or introducing new noise.

[0075] Before data processing, it is usually necessary to perform dimensionless processing (standardization) on the data to eliminate the influence of units on missing value cleaning.

[0076] The feature extraction module can automatically extract useful features from communication data based on deep learning algorithms, including network traffic patterns, packet size, transmission time, source address and destination address, etc. These features are crucial for identifying abnormal patterns and potential threats.

[0077] The analysis module uses the association rule Apriori algorithm to explore the association relationship between different features in the communication data. For example, it analyzes whether certain phone number combinations often appear together, or whether a certain type of network traffic is associated with a specific time period or event. Association analysis can help discover potential communication patterns and abnormal behaviors.

[0078] Apply the Apriori algorithm: Use the preprocessed data as input, set a minimum support threshold (such as 5%), and use the Apriori algorithm to find frequently occurring IP address combinations. For example, you may find that certain specific source IP address and destination IP address combinations appear frequently, or that certain IP address combinations are unusually active in a specific time period.

[0079] Association rule mining: Generate association rules from frequent item sets and calculate the support and confidence of each rule.

[0080] For example, the rule "If the source IP address is A, then the destination IP address is B" has a support of 10% and a confidence of 70%, which means that among all transactions, 10% of the transactions contain both source IP address A and destination IP address B, and among the transactions containing source IP address A, 70% of the transactions also contain destination IP address B.

[0081] Anomaly detection and analysis: Identify potential abnormal communication patterns based on association rules and set thresholds (such as minimum support and confidence requirements). For example, if a certain IP address combination appears much more frequently than normal, or an IP address is unusually active during non-working hours, it may indicate abnormal communication behavior. Further analyze these abnormal patterns to determine whether they are caused by network attacks, equipment failures, abnormal user behavior, etc.

[0082] Assume that a week of network traffic data is collected and preprocessed to obtain partial data as shown in Table 1 below:

[0083] Table 1

[0084] Source IP address Destination IP address Protocol Type Packet size Timestamp 192.168.1.1 192.168.1.2 TCP 500B 2024-10-08 10:00:00 192.168.1.3 192.168.1.4 UDP 300B 2024-10-08 10:01:00 ... ... ... ... ...

[0085] After applying the Apriori algorithm, we may get the following frequent item sets and association rules:

[0086] Frequent itemsets: {(192.168.1.1, 192.168.1.2), (192.168.1.3, 192.168.1.4)}, with support of 6% and 5% respectively.

[0087] Association rule: If the source IP address is 192.168.1.1, then the destination IP address is 192.168.1.2, and the confidence level is 80%; if the source IP address is 192.168.1.3, then the destination IP address is 192.168.1.4, and the confidence level is 75%.

[0088] The conversion module can convert the data into the format required by the monitoring module.

[0089] The present invention adopts a variety of technical means to ensure the accuracy and integrity of data, such as data cleaning, noise data identification and processing, etc., to improve the robustness and reliability of the system.

[0090] The data processing module provided in the present invention can extract valuable information and patterns through a series of operations such as cleaning, feature extraction, and analysis, thereby providing strong support for subsequent monitoring and analysis; the feature extraction module provided is based on a deep learning algorithm, and can automatically extract useful features from communication data, thereby improving the accuracy and efficiency of feature extraction; the analysis module provided utilizes the association rule Apriori algorithm, and can explore the association relationship between different features in the communication data, thereby facilitating the discovery of potential communication patterns and abnormal behaviors.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A communication monitoring system based on artificial intelligence, characterized in that: It includes a server module and a data acquisition module, a data storage module, a data processing module, a monitoring module, and a monitoring display module connected thereto; The data acquisition module is responsible for collecting raw data from the communication network and selecting the data source to be collected according to the detection requirements; The data storage module is responsible for storing the collected data; The data processing module is responsible for further processing and analyzing the stored data to extract useful information and analyze the causes of abnormal values, and transmit the processed data to the monitoring module; The monitoring module is responsible for real-time detection and analysis of processed data using a deep learning model; when abnormal behavior is detected, an alarm mechanism is triggered and an alarm message is sent; The monitoring and display module is responsible for displaying the monitored data, results and alarm information to the user in an intuitive and easy-to-understand manner.

2. The communication monitoring system based on artificial intelligence according to claim 1, characterized in that: The data processing module automatically cleans, extracts features, analyzes and standardizes the data to extract useful feature information.

3. The communication monitoring system based on artificial intelligence according to claim 1, characterized in that: The deep learning model analyzes abnormal data packets in network traffic to identify DDoS attacks, malware propagation, insider misconduct, network sniffing, and man-in-the-middle attacks. The deep learning model identifies DDoS attacks by analyzing abnormal data packets in network traffic; The deep learning model identifies malware propagation behavior by analyzing abnormal file transfer behavior in network traffic; The deep learning model identifies these improper behaviors by analyzing abnormal access records in system logs; The deep learning model identifies network sniffing and man-in-the-middle attacks by analyzing abnormal transmission patterns in network traffic.

4. The communication monitoring system based on artificial intelligence according to claim 1, characterized in that: The data acquisition module transmits the collected data to the data storage module in real time for storage; The data processing module reads data from the data storage module and processes and analyzes it, and then transmits the processed data to the monitoring module for real-time monitoring; The monitoring module uses the processed data to monitor the communication network in real time and triggers corresponding alarm and response mechanisms based on the monitoring results; The monitoring module will also feed back the monitoring results to the data acquisition module and the data processing module so that they can optimize and adjust according to the monitoring results to improve the accuracy and efficiency of the monitoring system.

5. The communication monitoring system based on artificial intelligence according to claim 1, characterized in that: The data processing module includes a cleaning module, a feature extraction module, an analysis module and a conversion module. The cleaning module identifies the noise data in the single-dimensional data set through data distribution characteristics and box plot method, and cleans it; The feature extraction module can automatically extract useful features from communication data based on a deep learning algorithm; The analysis module uses the association rule Apriori algorithm to explore the association relationship between different features in the communication data; The conversion module can convert the data into the format required by the monitoring module.

6. The communication monitoring system based on artificial intelligence according to claim 5, characterized in that: The cleaning method of the cleaning module is as follows: (1) Using data distribution characteristics to identify noisy data Dataset partitioning: Divide the data set into multiple intervals. The number of intervals is determined based on the size and distribution characteristics of the data. Determine the data concentration domain: By observing the distribution of data in each interval, identify the interval where the data is mainly concentrated, that is, the data concentration domain; the data in this interval is usually normal data, and the data far away from this interval is noise data; (2) Using the box plot method to identify and clean noise data Construct a box plot: The box plot can intuitively display the central trend, dispersion and outliers of the data; in the box plot, it contains the first quartile Q1, the third quartile Q3 and the interquartile range IQR; Identify outliers: According to the definition of the box plot, outliers are usually defined as data points that are smaller than Q1-3IQR or larger than Q3+3IQR; these data points are far away from the central domain of the data set and are classified as noise data; Cleaning noisy data: For the identified outliers, different processing strategies are adopted, including direct deletion, replacement with the mean or median.

7. The communication monitoring system based on artificial intelligence according to claim 6, characterized in that: When cleaning noisy data, caution is required to avoid accidentally deleting normal data or introducing new noise.

8. The communication monitoring system based on artificial intelligence according to claim 7, characterized in that: Before cleaning the noisy data, the data needs to be dimensionless processed to eliminate the impact of units on missing value cleaning.

9. The communication monitoring system based on artificial intelligence according to claim 5, characterized in that: The features extracted by the feature extraction module include network traffic pattern, data packet size, transmission time, source address and destination address.

10. The communication monitoring system based on artificial intelligence according to claim 5, characterized in that: Apply the Apriori algorithm: take the preprocessed data as input, set the minimum support threshold, and use the Apriori algorithm to find the frequently occurring extracted feature information; generate association rules from the frequent item sets, and calculate the support and confidence of each rule.

Images