An abnormal data monitoring method and system based on artificial intelligence

Abstract

The present invention is applicable to the field of abnormal data monitoring, and provides an abnormal data monitoring method and system based on artificial intelligence. The system includes: a real-time data stream collection module, a cross-level correlation analysis module, a dynamic baseline model establishment module, an abnormal behavior comparison module, and an abnormal traceability module. Through real-time data acquisition, cross-level analysis, dynamic modeling, abnormal detection, and in-depth threat traceability, this method constructs an efficient and intelligent abnormal monitoring closed loop, showing significant beneficial effects. First of all, this method uses real-time data stream capture and cross-level correlation analysis to ensure comprehensive coverage of interaction behaviors at multiple protocol levels in the network, and combines context information to identify complex abnormal patterns. This multi-dimensional analysis significantly improves the accuracy of detection, effectively reduces false positives and false negatives, and at the same time generates an abnormal marker data set with weights, providing high-quality data support for subsequent processes.

Description

Technical Field

[0001] The present invention belongs to the field of abnormal data monitoring, and particularly relates to an abnormal data monitoring method and system based on artificial intelligence. Background Art

[0002] The field of abnormal data monitoring focuses on identifying potential abnormal behaviors and threat patterns through real-time analysis and processing of massive data to ensure the normal operation of systems and networks. With the continuous increase in the scale and complexity of networks, the task of abnormal monitoring has become increasingly important and complex. The main challenges faced in this field include the need to balance real-time performance and accuracy, adapt to changing behavior patterns in a dynamic environment, and effectively reduce false alarm and missed alarm rates. In addition, the popularity of encrypted traffic and complex cross-protocol layer interactions in modern networks make it difficult for traditional monitoring methods to comprehensively capture abnormal signals. With the introduction of artificial intelligence technology, abnormal data monitoring has gradually evolved from single-level static analysis to multi-level, dynamic, and intelligent directions, enabling in-depth mining of complex threats, analysis of propagation paths, and overall control of the security situation. The research and development in this field not only improve the accuracy and timeliness of abnormal detection but also provide new ideas and technical support for the proactive defense of network security.

[0003] Traditional technologies are often not comprehensive enough in abnormal detection and are difficult to capture complex abnormal behaviors across multiple protocol layers simultaneously, resulting in the omission of some potential threats. At the same time, due to the inability to accurately distinguish the diversity of normal behaviors from real abnormal behaviors, false alarms and missed alarms are relatively common, thus reducing the credibility of detection. In addition, the adaptability of existing methods to dynamic network environments is poor, and static baseline models are difficult to effectively cope with the frequent changes in network traffic patterns, resulting in unstable detection effects. In terms of threat priority assessment, traditional technologies lack clear classification of the severity and risk levels of abnormal signals, leading to low efficiency in the allocation of security resources. More importantly, existing technologies usually only stay at the surface detection of abnormal signals and lack in-depth analysis of deep threat patterns, propagation paths, and root causes, unable to provide a global understanding of threats. Summary of the Invention

[0004] The purpose of the present invention is to provide an abnormal data monitoring method and system based on artificial intelligence, aiming to solve the technical problems existing in the prior art as determined in the background art.

[0005] The present invention is implemented as follows. An abnormal data monitoring method based on artificial intelligence, the method includes:

[0006] Real-time collect data packets from network traffic, including encrypted data and protocol metadata, to generate a real-time data stream containing all relevant encrypted and protocol information;

[0007] Perform cross - layer correlation analysis on real - time data streams, while using context information to understand the associations between different layers, identify the associations and abnormal interaction patterns of data between different layers, mark the analysis results as preliminary abnormal signals, and generate a weighted abnormal - marked data set;

[0008] Establish a dynamic baseline model based on the real - time data stream and the abnormal - marked data set, enhance the accuracy of the baseline model using the correlation analysis results, and adjust the baseline parameters in real - time;

[0009] Compare the real - time collected data with the baseline model, identify deviated abnormal activities, mark the deviated abnormal behaviors, and feedback the deviation identification results to the encrypted log for analysis to obtain abnormal background information;

[0010] Based on the abnormal identification results of the baseline model, analyze the encrypted log, identify potential deep - level threat patterns and abnormal signals, and trace the origin of the threat patterns and abnormal signals to identify the root cause and abnormal path.

[0011] As a further solution of the present invention, the real - time collection of data packets from network traffic, including encrypted data and protocol metadata, to generate a real - time data stream, specifically includes:

[0012] Identify and determine the sources of network traffic to be monitored, and capture data packets from real - time network traffic;

[0013] Classify the captured data packets, distinguish data packets at the application layer, transport layer, and network layer, and parse the data packets to extract encrypted data and protocol metadata;

[0014] Integrate and format the parsed data to generate a structured real - time data stream.

[0015] As a further solution of the present invention, the cross - layer correlation analysis of the real - time data stream, while using context information to understand the associations between different layers, identify the associations and abnormal interaction patterns of data between different layers, and mark the analysis results as preliminary abnormal signals, and generate a weighted abnormal - marked data set, specifically includes:

[0016] Sort the real - time data stream according to the protocol layer and extract context information from the data;

[0017] Perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark the abnormal interaction patterns to generate preliminary abnormal signals, and record the abnormal type, frequency, and severity;

[0018] Assign weights according to the characteristics of each abnormal signal, generate an abnormal labeled dataset with weights, and integrate all analysis and labeling results to output a structured abnormal labeled dataset.

[0019] As a further solution of the present invention, establish a dynamic baseline model based on the real-time data stream and the abnormal labeled dataset, enhance the accuracy of the baseline model using the correlation analysis results, and adjust the baseline parameters in real time, specifically including:

[0020] Extract features from the real-time data stream and the abnormal labeled dataset, establish a dynamic baseline model, and train the dynamic baseline model with the extracted features;

[0021] Dynamically adjust the parameters and structure of the baseline model according to the real-time correlation analysis results.

[0022] As a further solution of the present invention, compare the real-time collected data with the baseline model, identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feedback the deviation recognition results to the encrypted log for analysis to obtain abnormal background information, specifically including:

[0023] Input the real-time collected network data stream into the baseline model, use the baseline model to compare the real-time input data, analyze the differences between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities;

[0024] Mark the detected abnormal deviation behaviors, and record the deviation degree and relevant context information;

[0025] Evaluate the severity of the deviation behavior, and assign an abnormal level to each abnormal activity according to the deviation amplitude and frequency;

[0026] Extract the deviation recognition results and feedback them to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

[0027] As a further solution of the present invention, based on the abnormal recognition results of the baseline model, analyze the encrypted log, identify potential deep threat patterns and abnormal signals, and conduct abnormal traceability on the threat patterns and abnormal signals to identify the root cause and abnormal path, specifically including:

[0028] Collect the encrypted logs related to the deviation recognition results and obtain the complete context information;

[0029] Analyze the encrypted log, identify potential deep threat patterns and abnormal signals, extract potential threat clues from the log, and identify the threat types;

[0030] Conduct traceability analysis on the identified abnormal signals, identify the occurrence path of the abnormal signal, and trace the original source of the abnormality;

[0031] Generate a combined report based on all the identified information.

[0032] Another object of the present invention is to provide an abnormal data monitoring system based on artificial intelligence, and the system includes:

[0033] A real-time data stream collection module, configured to collect data packets in real time from network traffic, including encrypted data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information;

[0034] A cross-layer correlation analysis module, configured to perform cross-layer correlation analysis on the real-time data stream, and at the same time utilize context information to understand the associations between different layers, identify the associations and abnormal interaction patterns of data between different layers, and mark the analysis results as preliminary abnormal signals, generating an abnormal marking data set with weights;

[0035] A dynamic baseline model establishment module, configured to establish a dynamic baseline model according to the real-time data stream and the abnormal marking data set, enhance the accuracy of the baseline model by using the correlation analysis results, and adjust the baseline parameters in real time;

[0036] An abnormal behavior comparison module, configured to compare the real-time collected data with the baseline model, identify deviated abnormal activities, mark the deviated abnormal behaviors, and feedback the deviation recognition results to the encrypted log for analysis to obtain abnormal background information;

[0037] An abnormal traceability module, configured to analyze the encrypted log based on the abnormal recognition results of the baseline model, identify potential deep threat patterns and abnormal signals, and perform abnormal traceability on the threat patterns and abnormal signals to identify the root cause and abnormal path.

[0038] As a further solution of the present invention, the cross-layer correlation analysis module includes:

[0039] A real-time data protocol layer sorting unit, configured to sort the real-time data stream according to the protocol layer and extract context information from the data;

[0040] A protocol layer inter-association analysis unit, configured to perform association analysis between different protocol layers, identify normal and abnormal interaction patterns, mark the abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency and severity;

[0041] An abnormal signal weight assignment unit, configured to assign weights according to the characteristics of each abnormal signal, generate an abnormal marking data set with weights, and integrate all analysis and marking results to output a structured abnormal marking data set.

[0042] As a further solution of the present invention, the abnormal behavior comparison module includes:

[0043] A difference analysis unit, configured to input the network data stream collected in real time into the baseline model, use the baseline model to compare the real-time input data, analyze the differences between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities;

[0044] An abnormal deviation behavior marking unit, configured to mark the detected abnormal deviation behavior, and record the deviation degree and relevant context information;

[0045] An abnormal activity severity evaluation unit, configured to evaluate the severity of the deviation behavior, and assign an abnormal level to each abnormal activity according to the deviation amplitude and frequency;

[0046] A deviation result extraction and encrypted log retrieval unit, configured to extract the deviation identification result and feedback it to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

[0047] As a further solution of the present invention, the abnormal traceability module includes:

[0048] An encrypted log collection unit, configured to collect the encrypted logs related to the deviation identification result and obtain the complete context information;

[0049] A deep threat pattern recognition unit, configured to analyze the encrypted log, identify potential deep threat patterns and abnormal signals, extract potential threat clues from the log, and identify the threat type;

[0050] An abnormal signal traceability unit, configured to perform traceability analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and trace the initial source of the abnormality;

[0051] A comprehensive information integration unit, configured to generate a combined report based on all the identified information.

[0052] The beneficial effects of the present invention are:

[0053] This method constructs an efficient and intelligent abnormal monitoring closed loop through real-time data collection, cross-level analysis, dynamic modeling, abnormal detection and deep threat traceability, presenting significant beneficial effects. First of all, this method uses real-time data stream capture and cross-level correlation analysis to ensure comprehensive coverage of the interaction behaviors of multiple protocol layers in the network, and combines context information to identify complex abnormal patterns. This multi-dimensional analysis significantly improves the accuracy of detection, effectively reduces false positives and false negatives, and at the same time generates an abnormal marking data set with weights, providing high-quality data support for subsequent processes.

[0054] Through the application of a dynamic baseline model, this method demonstrates strong adaptability in the face of network dynamic changes. The real-time comparison mechanism can accurately identify abnormal activities that deviate. Combining severity level assessment and priority allocation optimizes resource allocation, enabling high-risk threats to be handled preferentially and enhancing the system's response efficiency. In addition, the feedback mechanism of anomaly detection results and encrypted logs comprehensively integrates context and background information, laying a foundation for in-depth threat analysis.

[0055] Through the mining of deep threat patterns and the traceability analysis of abnormal behaviors, the root causes and propagation paths of threats are revealed, providing the ability to gain insights into threats across the entire link. The final generated combined report is presented in a structured and visual form, comprehensively supporting security protection decisions from threat types, propagation paths to severity analysis. This closed-loop method not only achieves accurate detection and rapid response to network threats but also provides key support for proactive defense and policy optimization through in-depth analysis, comprehensively enhancing the intelligent level and overall effectiveness of network security management. Brief Description of the Drawings

[0056] Figure 1 Flowchart of a method for monitoring abnormal data based on artificial intelligence provided by an embodiment of the present invention;

[0057] Figure 2 Flowchart of collecting data packets in real time from network traffic and generating a real-time data stream provided by an embodiment of the present invention;

[0058] Figure 3 Flowchart of identifying abnormal interactions and marking the analysis results as preliminary abnormal signals provided by an embodiment of the present invention;

[0059] Figure 4 Flowchart of enhancing the accuracy of the baseline model using the results of correlation analysis and adjusting the baseline parameters in real time provided by an embodiment of the present invention;

[0060] Figure 5 Flowchart of feeding back the deviation identification results to the encrypted log for analysis to obtain abnormal background information provided by an embodiment of the present invention;

[0061] Figure 6 Flowchart of identifying potential deep threat patterns and abnormal signals and tracing the anomalies of the threat patterns and abnormal signals provided by an embodiment of the present invention;

[0062] Figure 7 Structural block diagram of a system for monitoring abnormal data based on artificial intelligence provided by an embodiment of the present invention;

[0063] Figure 8 Structural block diagram of a cross-level correlation analysis module provided by an embodiment of the present invention;

[0064] Figure 9 Block diagram of the abnormal behavior comparison module provided by an embodiment of the present invention;

[0065] Figure 10 Block diagram of the abnormal traceability module provided by an embodiment of the present invention. Detailed implementation manners

[0066] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

[0067] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of this application, the first xx script may be referred to as the second xx script, and similarly, the second xx script may be referred to as the first xx script.

[0068] Figure 1 Flowchart of an abnormal data monitoring method based on artificial intelligence provided by an embodiment of the present invention, as Figure 1 shown, the method includes:

[0069] S100, collect data packets from network traffic in real time, including encrypted data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information;

[0070] In this step, first, identify and determine the source of the network traffic to be monitored. In this link, technicians need to clarify the monitoring scope according to business requirements and security policies, for example, select specific network terminals, server nodes or application traffic as the monitoring objects. Then, capture the data packets in the target traffic through real-time network traffic capture technology. Secondly, classify and analyze the captured data packets. This process involves dividing the data packets into application layer, transport layer and network layer data packets according to the network hierarchical model, so as to fully extract valuable information in each layer. Especially for the analysis of encrypted data and protocol metadata, appropriate tools and algorithms are used to extract necessary information while protecting the sensitivity and privacy of the data. Finally, integrate and format the analyzed data to ensure that the generated real-time data stream has consistency and structured characteristics. This step will use professional data processing tools to clean, format and standardize the data stream to make it adapt to the requirements of subsequent analysis models.

[0071] This step ensures that the monitoring system can quickly respond to dynamic changes in the network and reduce the latency of potential security threats by capturing and processing network traffic data in real time. This real-time nature significantly enhances the agility and practicality of the system, enabling it to adapt to complex and changing network environments. Secondly, by classifying and parsing different types of data packets layer by layer, comprehensive capture of multi-level information can be achieved. In particular, the extraction of encrypted data and protocol metadata provides crucial basic support for analyzing potential threats in encrypted communications. In addition, the integration and formatting process greatly improves the standardization of data, enabling subsequent analysis models to efficiently process and interpret this data, thereby enhancing the accuracy and stability of the overall analysis process. Finally, this process strictly adheres to the principles of data privacy and security protection. Through precise target traffic selection and parsing strategies, sensitive information can be effectively protected, achieving a good balance between security and functionality.

[0072] As Figure 2 shown, the real-time collection of data packets from network traffic, including encrypted data and protocol metadata, to generate a real-time data stream specifically includes:

[0073] S110, identify and determine the sources of network traffic to be monitored, and capture data packets from real-time network traffic;

[0074] S120, classify the captured data packets, distinguish data packets at the application layer, transport layer, and network layer, and parse the data packets to extract encrypted data and protocol metadata;

[0075] S130, integrate and format the parsed data to generate a structured real-time data stream.

[0076] S200, perform cross-layer correlation analysis on the real-time data stream, simultaneously utilize context information to understand the associations between different layers, identify the associations and abnormal interaction patterns of data between different layers, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal label data set;

[0077] In this step, the real-time data stream is sorted according to the protocol layers. This sorting process is based on the OSI model, which hierarchically manages data from the application layer, transport layer to the network layer, and extracts key context information from it, including the timing characteristics of the data, protocol field features, and cross-layer interaction context dependencies. The extraction of this context information ensures a comprehensive understanding of the data characteristics and traffic patterns, providing in-depth support for subsequent analysis. Subsequently, the system performs correlation analysis between different protocol layers, using machine learning algorithms or deep learning models to identify normal interaction patterns (such as expected request-response behaviors) and potential abnormal interaction patterns (such as sudden traffic increases, atypical protocol usage, etc.). For the identified abnormal interaction patterns, the system marks them and attaches detailed information, such as the type of anomaly (such as traffic anomaly, protocol anomaly, etc.), frequency (the number of occurrences or frequency), and severity (the degree of potential threat to the system or network). These information form the basis of the initial anomaly signals.

[0078] Next, the system assigns weights according to the characteristics of each anomaly signal. The weight assignment is comprehensively calculated based on factors such as the importance of anomaly features, occurrence frequency, and relevance in historical data, with the aim of providing a more accurate basis for subsequent model training and anomaly priority ranking. Through such processing, an anomaly labeled dataset with weights is generated. This dataset not only contains the initial anomaly signals, but also integrates the weight information of the anomalies and other analysis marking results, and outputs them in a structured form, providing complete data support for the establishment of the dynamic baseline model and subsequent comparison.

[0079] This step can integrate the multi-layer characteristics of the data through cross-layer correlation analysis, and identify complex anomaly patterns that may not be captured by single-layer analysis. This cross-layer analysis ability enables the system to have higher anomaly detection accuracy, especially in the face of complex protocols and encrypted traffic in modern networks, and can more comprehensively capture global anomaly behaviors. Secondly, through the extraction and parsing of context information, S200 can understand the timing relationship and semantic association between data, providing key support for the accurate identification of abnormal interaction patterns. In addition, the information such as type, frequency, and severity attached to the anomaly signal marking enables the system to classify anomaly events, better support the response decision of the security team, and improve the processing efficiency. The weight assignment mechanism further optimizes the application value of the anomaly signal in subsequent analysis, ensuring that high-priority anomalies can be processed first, thus significantly enhancing the threat recognition ability and resource allocation efficiency of the overall monitoring system.

[0080] Such as Figure 3As shown, the cross - layer correlation analysis of the real - time data stream is performed, while using context information to understand the associations between different layers, identifying the associations and abnormal interaction patterns of data between different layers, and marking the analysis results as preliminary abnormal signals, generating an abnormal - marked data set with weights, specifically including:

[0081] S210, sort the real - time data stream according to the protocol layer and extract context information from the data;

[0082] S220, perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark the abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency, and severity;

[0083] S230, assign weights according to the characteristics of each abnormal signal, generate an abnormal - marked data set with weights, and integrate all analysis and marking results to output a structured abnormal - marked data set.

[0084] S300, establish a dynamic baseline model based on the real - time data stream and the abnormal - marked data set, enhance the accuracy of the baseline model using the correlation analysis results, and adjust the baseline parameters in real - time;

[0085] This step extracts features from the real - time data stream and the abnormal - marked data set. This process uses advanced machine - learning feature - engineering techniques to extract key attributes that can reflect the behavior and characteristics of the data, such as the traffic pattern of data packets, protocol interaction characteristics, time - series patterns, and the degree of deviation from historical data. These features provide the necessary input dimensions for establishing the dynamic baseline model, enabling the model to accurately identify the boundary between normal traffic and abnormal behavior.

[0086] Next, the establishment and training of the dynamic baseline model are the key to this step. The baseline model is designed as a statistic that can adapt to the real - time changing network behavior. By inputting and training on the extracted features, the baseline model can capture the dynamic changes of normal network behavior and form a benchmark for "normal behavior". At the same time, the model uses the information in the abnormal - marked data set to incorporate the marked abnormal signals into the training process to enhance the model's sensitivity and recognition ability to abnormal behavior, thus avoiding false alarms caused by the diversity of normal behavior.

[0087] After the model is built, another important aspect of this step is to dynamically adjust the parameters and structure of the baseline model according to the results of real-time correlation analysis. This means that the baseline model is not fixed, but will be continuously updated with the input of real-time data streams and analysis results. For example, when there are periodic fluctuations or sudden changes in network traffic, the dynamic adjustment mechanism can ensure that the model can reflect the actual situation of the current network environment and achieve accurate capture and continuous adaptation to abnormal deviations by optimizing the thresholds, weights, or algorithm structures of the model in real time.

[0088] Finally, the system uses the dynamic baseline model to compare the real-time collected data to identify abnormal activities that deviate, and marks the relevant information. For example, when the baseline model detects that certain traffic characteristics deviate from the normal behavior range, these traffic flows will be marked as abnormal behaviors, and the deviation identification results will be fed back to the encrypted log for further analysis to obtain the context background of the data and potential abnormal root cause information. This closed-loop feedback mechanism not only improves the adaptability of the model to complex network environments, but also provides key data support for subsequent traceability analysis and threat localization.

[0089] The dynamic baseline model can capture the dynamic change characteristics of network behavior, which makes it show extremely high flexibility and adaptability when dealing with complex and changeable communication patterns in modern networks, avoiding the limitations of static models that are prone to failure in dynamic environments. Secondly, the combination of feature extraction and model training process not only improves the sensitivity of the model to abnormal signals, but also effectively reduces the false alarm rate and missed alarm rate by integrating the information of the abnormal marked data set, thus enhancing the accuracy and reliability of abnormal detection. In addition, the introduction of the dynamic adjustment mechanism enables the system to optimize the model performance according to the changes in real-time data, and can achieve rapid response without manual intervention, which greatly improves the operation efficiency of the system and the timeliness of threat identification. Through the linked analysis with the encrypted log, not only can the identification of a single abnormal behavior be achieved, but also support can be provided for subsequent abnormal background analysis and threat traceability, thus greatly enhancing the global threat perception ability of the entire monitoring method.

[0090] As Figure 4 shown, the establishment of a dynamic baseline model based on real-time data streams and abnormal marked data sets, and the use of correlation analysis results to enhance the accuracy of the baseline model and adjust the baseline parameters in real time specifically include:

[0091] S310, extract features from the real-time data stream and the abnormal marked data set, establish a dynamic baseline model, and train the dynamic baseline model through the extracted features;

[0092] S320, dynamically adjust the parameters and structure of the baseline model according to the results of real-time correlation analysis.

[0093] The S400 compares the real-time collected data with the baseline model, identifies the deviated abnormal activities, marks the deviated abnormal behaviors, and feeds back the deviation identification results to the encrypted log for analysis to obtain the abnormal background information;

[0094] In this step, the real-time collected network data stream will be input into the established and dynamically adjusted baseline model. The baseline model has accumulated an accurate understanding of the normal network behavior patterns during the previous training and real-time updates. Therefore, during the comparison process, the model will analyze the real-time input data one by one based on the known normal behavior characteristics. By comparing the differences between the real-time data and the standard baseline, the system can identify the abnormal activities or behaviors. This analysis includes not only the matching of numerical characteristics (such as packet size, traffic rate, etc.), but also more complex pattern differences (such as the fluctuation characteristics of time series, protocol mismatch situations, etc.), so as to ensure that abnormal activities can be comprehensively captured.

[0095] Next, mark the detected abnormal deviation behaviors. In this step, the system will record the deviation degree and relevant context information of each abnormal behavior. For example, for an obvious traffic anomaly, the system will not only mark its deviation amplitude (such as the percentage by which the packet size exceeds the normal range), but also record its context information (such as the source IP address of the abnormal data, the destination port, the protocol type used, etc.). This detailed marking can not only provide data support for subsequent analysis, but also help the security team quickly locate the root cause of the problem.

[0096] Then, the system evaluates the severity of the deviation behavior. This evaluation is based on the amplitude and frequency of the deviation behavior. For example, a minor single deviation may be marked as a low-level anomaly, while multiple frequent deviations with obvious characteristics will be evaluated as high-level anomalies. This severity evaluation assigns an anomaly level to each abnormal activity (such as low, medium, high or a similar scoring mechanism), which enhances the objectivity and accuracy of the identification while providing priority guidance for subsequent processing, enabling high-risk anomalies to be processed first and ensuring the timeliness and effectiveness of network security.

[0097] Finally, the system feeds back the extracted deviation identification results to the encrypted log and retrieves and analyzes the log data. This feedback process is a closed-loop mechanism. By associating the identification results with the background information in the encrypted log, the system can further extract the corresponding context data (such as the communication behaviors before or after the abnormal activity, the usage history of specific protocols, etc.). The mining of this background information provides more in-depth support for subsequent threat analysis and root cause location, ensuring that abnormal signals are not analyzed in isolation but are comprehensively evaluated in the specific environment where they occur.

[0098] By comparing the real-time data stream with the dynamic baseline model, the system can efficiently capture abnormal activities that deviate from the normal behavior pattern. This real-time and precision ensure that network security incidents can be detected at an early stage, thus effectively reducing the destructiveness of potential threats. Secondly, the marking of deviation behavior and severity assessment greatly improve the ability to classify and prioritize abnormal activities, providing clear handling guidance for the security team. By assigning an anomaly level, the system can help network administrators focus on the most critical high-risk events, thus significantly improving the response efficiency. Furthermore, the correlation analysis of the feedback of deviation results and encrypted logs closely combines abnormal signals with their context information, laying a solid foundation for threat tracing and root cause analysis. Through this mechanism, the system not only identifies abnormal activities but also reveals the background and reasons for their occurrence, enabling the security team to more comprehensively understand the overall picture of the problem and formulate targeted countermeasures.

[0099] As Figure 5 shown, comparing the real-time collected data with the baseline model, identifying abnormal activities that deviate, marking the abnormal behavior that deviates, and feeding back the deviation identification result to the encrypted log for analysis to obtain abnormal background information, specifically including:

[0100] S410. Input the real-time collected network data stream into the baseline model, use the baseline model to compare the real-time input data, analyze the difference between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities;

[0101] S420. Mark the detected abnormal deviation behavior, and record the deviation degree and relevant context information;

[0102] S430. Evaluate the severity of the deviation behavior, and assign an anomaly level to each abnormal activity according to the deviation amplitude and frequency;

[0103] S440. Extract the deviation identification result and feed it back to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

[0104] S500. Based on the anomaly identification result of the baseline model, analyze the encrypted log, identify potential deep threat patterns and abnormal signals, and conduct anomaly tracing on the threat patterns and abnormal signals to identify the root cause and abnormal path.

[0105] This step collects encrypted logs related to the deviation identification results and obtains complete context information through linkage with S400. This process helps the system accurately locate historical activities and background information related to abnormal signals by mining the timestamps, data packet contents, protocol information, and related communication metadata in the encrypted logs. For example, by retrieving the source IP address, target port number, and related device identification involved in the abnormal behavior in the log, context data directly or indirectly related to the abnormal behavior can be obtained. This context information provides comprehensive background support for subsequent analysis, ensuring that abnormal behavior is not analyzed in isolation, but is placed in the environment where it is generated for comprehensive analysis.

[0106] Next, the system conducts an in-depth analysis of the encrypted logs to identify potential deep-level threat patterns and abnormal signals. During this process, the system uses artificial intelligence models to mine and model the data in the encrypted logs, trying to find potential threat characteristics from surface anomalies. For example, by conducting a comprehensive analysis of multiple abnormal interactions, the system may discover a continuous scanning behavior or a precursor to a distributed denial of service (DDoS) attack. In addition, the system can extract more potential threat clues from the logs and further identify threat types, such as data leakage, privilege escalation attacks, or lateral movement threats, thereby providing the security team with more detailed threat classification information.

[0107] The identified abnormal signals are then traced back to the source of the abnormal signal and its original source is identified. The traceability analysis gradually restores the propagation trajectory of the abnormal signal by tracing back the communication links in the encrypted log. For example, the system may locate the starting point of the malicious behavior by tracking the flow of abnormal activities and determining the source IP range of the attacker. At the same time, the traceability analysis can reveal the propagation path of the abnormal signal and determine whether there are other potential targets affected by the abnormal activity. By identifying the "starting point-path-end point" full-link information of abnormal behavior, the system provides more accurate and comprehensive basic data for threat disposal.

[0108] Finally, based on all the identified threat information, the system generates a consolidated report. This report summarizes everything from context information extraction to threat type identification, from source analysis to path attribution in a structured and visual form. The report not only includes a detailed description and background information of each abnormal signal, but also provides the security management team with an overall insight into the deep threat pattern. For example, the report may include a visual threat propagation map, the distribution of threat severity levels, and response recommendations for each threat type. This comprehensive report enables the security team to quickly understand the full picture of the threat and develop targeted security protection measures.

[0109] Through in-depth analysis of encrypted logs and comprehensive extraction of context information, this step can reveal deep threat patterns behind surface anomalies, helping the security team understand the attacker's behavior logic and potential intentions, thereby enhancing threat prediction and defense capabilities. Secondly, the introduction of traceback analysis enables the system not only to identify abnormal signals but also to accurately trace their original sources and propagation paths. This full-link threat analysis capability effectively reduces the blind spots in security protection, making the response to attacks more precise and efficient. In addition, the generation of a consolidated report presents complex security analysis results in a structured and visual way, significantly reducing the complexity of information transmission, helping the team quickly grasp key information and formulate countermeasures. Most importantly, this step provides closed-loop support for the abnormal data monitoring system through comprehensive threat identification and traceback analysis, enabling the system to advance from real-time detection to in-depth analysis and decision support, thus comprehensively enhancing the overall protection ability and processing efficiency of network security.

[0110] As Figure 6 shown, based on the anomaly recognition results of the baseline model, analyze the encrypted logs, identify potential deep threat patterns and abnormal signals, and conduct anomaly traceback on the threat patterns and abnormal signals to identify the root cause and abnormal path, specifically including:

[0111] S510, collect encrypted logs related to the deviation recognition results and obtain complete context information;

[0112] S520, analyze the encrypted logs, identify potential deep threat patterns and abnormal signals, extract potential threat clues from the logs, and identify threat types;

[0113] S530, conduct traceback analysis on the identified abnormal signals, identify the occurrence path of the abnormal signal, and trace the original source of the anomaly;

[0114] S540, generate a consolidated report based on all the identified information.

[0115] Figure 7 The structural block diagram of an abnormal data monitoring system based on artificial intelligence provided by an embodiment of the present invention, as Figure 7 shown, the system includes:

[0116] A real-time data stream collection module 100, configured to collect data packets in real time from network traffic, including encrypted data and protocol metadata, and generate a real-time data stream containing all relevant encrypted and protocol information;

[0117] The cross - layer correlation analysis module 200 is used to perform cross - layer correlation analysis on real - time data streams, while using context information to understand the associations between different layers, identify the associations and abnormal interaction patterns of data between different layers, mark the analysis results as preliminary abnormal signals, and generate an abnormal label dataset with weights;

[0118] The dynamic baseline model establishment module 300 is used to establish a dynamic baseline model based on real - time data streams and the abnormal label dataset, enhance the accuracy of the baseline model using the correlation analysis results, and adjust the baseline parameters in real - time;

[0119] The abnormal behavior comparison module 400 is used to compare the real - time collected data with the baseline model, identify deviated abnormal activities, mark the deviated abnormal behaviors, and feedback the deviation identification results to the encrypted log for analysis to obtain abnormal background information;

[0120] The abnormal traceability module 500 is used to analyze the encrypted log based on the abnormal identification results of the baseline model, identify potential deep - level threat patterns and abnormal signals, and perform abnormal traceability on the threat patterns and abnormal signals to identify the root cause and abnormal path.

[0121] As Figure 8 shown, the cross - layer correlation analysis module 200 includes:

[0122] The real - time data protocol layer sorting unit 210 is used to sort the real - time data stream according to the protocol layer and extract context information from the data;

[0123] The protocol layer - to - layer association analysis unit 220 is used to perform association analysis between different protocol layers, identify normal and abnormal interaction patterns, mark the abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency, and severity;

[0124] The abnormal signal weight assignment unit 230 is used to assign weights according to the characteristics of each abnormal signal, generate an abnormal label dataset with weights, and integrate all analysis and marking results to output a structured abnormal label dataset.

[0125] As Figure 9 shown, the abnormal behavior comparison module 400 includes:

[0126] The difference analysis unit 410 is used to input the real - time collected network data stream into the baseline model, use the baseline model to compare the real - time input data, analyze the differences between the real - time data and the data of the normal behavior pattern, and identify potential abnormal activities;

[0127] An abnormal deviation behavior marking unit 420 is used to mark the detected abnormal deviation behavior, record the deviation degree and relevant context information;

[0128] An abnormal activity severity assessment unit 430 is used to evaluate the severity of the deviation behavior, and assign an abnormal level to each abnormal activity according to the deviation amplitude and frequency;

[0129] A deviation result extraction and encrypted log retrieval unit 440 is used to extract the deviation identification result and feedback it to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

[0130] As Figure 10 shown, the abnormal traceability module 500 includes:

[0131] An encrypted log collection unit 510 is used to collect the encrypted logs related to the deviation identification result and obtain the complete context information;

[0132] A deep threat pattern recognition unit 520 is used to analyze the encrypted log, identify potential deep threat patterns and abnormal signals, extract potential threat clues from the log, and identify the threat type;

[0133] An abnormal signal traceability unit 530 is used to perform traceability analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and trace the original source of the abnormality;

[0134] A comprehensive information integration unit 540 is used to generate a combined report based on all the identified information.

[0135] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown in sequence according to the indication of the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily need to be executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages does not necessarily need to be sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0136] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0137] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0138] The above embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

[0139] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An abnormal data monitoring method based on artificial intelligence, characterized in that: The method comprises: Collect packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; Perform cross-level correlation analysis on real-time data streams, use contextual information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Establish a dynamic baseline model based on real-time data streams and abnormal labeled data sets, use correlation analysis results to enhance the accuracy of the baseline model, and adjust baseline parameters in real time; Compare the real-time collected data with the baseline model to identify and mark the deviated abnormal activities, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; Based on the anomaly identification results of the baseline model, the encrypted logs are analyzed to identify potential deep-level threat patterns and abnormal signals, and the threat patterns and abnormal signals are traced to their sources to identify the root causes and abnormal paths. The cross-level correlation analysis of the real-time data stream is performed, and the context information is used to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling data set, which specifically includes: Sort the real-time data stream according to the protocol hierarchy and extract contextual information from the data; Conduct correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record abnormal types, frequencies, and severity; Assign weights according to the characteristics of each abnormal signal to generate a weighted abnormal labeling dataset, integrate all analysis and labeling results, and output a structured abnormal labeling dataset; The real-time collected data is compared with the baseline model to identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information, including: Input the network data stream collected in real time into the baseline model, use the baseline model to compare the real-time input data, analyze the difference between the real-time data and the data of normal behavior patterns, and identify potential abnormal activities; Mark the detected abnormal deviation behavior and record the degree of deviation and related context information; Assess the severity of deviant behavior and assign an anomaly level to each anomalous activity based on the magnitude and frequency of the deviation; The deviation identification results are extracted and fed back to the encrypted log, the encrypted log is searched, and the corresponding background information is read.

2. The method according to claim 1, characterized in that The real-time collection of data packets from network traffic, including encrypted data and protocol metadata, and the generation of real-time data streams specifically include: Identify and determine the source of network traffic that needs to be monitored and perform packet capture of real-time network traffic; Classify captured data packets, distinguish between application layer, transport layer and network layer packets, and parse data packets to extract encrypted data and protocol metadata; Aggregate and format parsed data to generate structured real-time data streams.

3. The method according to claim 1, characterized in that The method of establishing a dynamic baseline model based on the real-time data stream and the abnormal labeled data set, enhancing the accuracy of the baseline model using the correlation analysis results, and adjusting the baseline parameters in real time specifically includes: Extract features from real-time data streams and anomaly labeled data sets, establish a dynamic baseline model, and train the dynamic baseline model using the extracted features; Dynamically adjust the parameters and structure of the baseline model based on the real-time correlation analysis results.

4. The method according to claim 1, characterized in that: The anomaly identification results based on the baseline model analyze the encrypted logs to identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths, including: Collect encrypted logs related to deviation identification results and obtain full context information; Analyze encrypted logs to identify potential deep-level threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; Conduct source tracing analysis on the identified abnormal signals, identify the occurrence path of the abnormal signals, and trace the original source of the abnormalities; Based on all the information identified, a consolidated report is generated.

5. An abnormal data monitoring system based on artificial intelligence, characterized in that: The system comprises: A real-time data stream collection module is used to collect data packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; The cross-level correlation analysis module is used to perform cross-level correlation analysis on real-time data streams, and use context information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Dynamic baseline model building module, used to build a dynamic baseline model based on real-time data streams and abnormal labeled data sets, enhance the accuracy of the baseline model using correlation analysis results, and adjust baseline parameters in real time; The abnormal behavior comparison module is used to compare the real-time collected data with the baseline model, identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; The anomaly tracing module is used to analyze encrypted logs based on the anomaly identification results of the baseline model, identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths; Wherein, the cross-level correlation analysis module includes: A real-time data protocol layer arrangement unit is used to arrange the real-time data stream according to the protocol layer and extract context information from the data; The protocol layer correlation analysis unit is used to perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency and severity; Anomaly signal weight assignment unit, used to assign weights according to the characteristics of each anomaly signal, generate anomaly labeled data set with weights, integrate all analysis and labeling results, and output a structured anomaly labeled data set; The abnormal behavior comparison module includes: A difference analysis unit is used to input the network data stream collected in real time into the baseline model, compare the real-time input data with the baseline model, analyze the difference between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities; An abnormal deviation behavior marking unit is used to mark the detected abnormal deviation behavior and record the degree of deviation and related context information; An abnormal activity severity assessment unit is used to assess the severity of deviant behavior and assign an abnormality level to each abnormal activity based on the magnitude and frequency of the deviation; The deviation result extraction and encrypted log retrieval unit is used to extract the deviation identification result and feed it back to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

6. The system according to claim 5, characterized in that The abnormality tracing module includes: An encrypted log collection unit, used to collect encrypted logs related to deviation identification results and obtain complete context information; A deep threat pattern recognition unit is used to analyze encrypted logs, identify potential deep threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; The abnormal signal tracing unit is used to perform tracing analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and track the original source of the abnormality; The comprehensive information integration unit is used to generate a consolidated report based on all the identified information.

Images