Abnormal data monitoring method and system based on artificial intelligence

Abstract

The invention is suitable for the field of abnormal data monitoring, and provides an abnormal data monitoring method and system based on artificial intelligence, and the system comprises a real-time data flow collection module, a cross-level correlation analysis module, a dynamic baseline model building module, an abnormal behavior comparison module, and an abnormal traceability module. According to the method, through real-time data acquisition, cross-hierarchy analysis, dynamic modeling, anomaly detection and deep threat tracing, an efficient and intelligent anomaly monitoring closed loop is constructed, and remarkable beneficial effects are presented. According to the method, firstly, real-time data flow capture and cross-level correlation analysis are utilized to ensure that interaction behaviors of multiple protocol levels in a network are comprehensively covered, and a complex abnormal mode is recognized in combination with context information; according to the multi-dimensional analysis, the detection accuracy is remarkably improved, false alarm and missing alarm are effectively reduced, meanwhile, an abnormal mark data set with weights is generated, and high-quality data support is provided for follow-up processes.

Description

Technical Field

[0001] The present invention belongs to the field of abnormal data monitoring, and in particular relates to an abnormal data monitoring method and system based on artificial intelligence. Background Art

[0002] The field of data monitoring focuses on identifying potential abnormal behaviors and threat patterns through real-time analysis and processing of massive data to ensure the normal operation of systems and networks. As the scale and complexity of networks continue to increase, the task of anomaly monitoring becomes increasingly important and complex. The main challenges faced by this field include the need to strike a balance between real-time and accuracy, adapt to changing behavior patterns in a dynamic environment, and effectively reduce false positives and false negatives. In addition, the popularity of encrypted traffic in modern networks and the complex interactions across protocol layers make it difficult for traditional monitoring methods to fully capture abnormal signals. With the introduction of artificial intelligence technology, abnormal data monitoring has gradually developed from a single-level static analysis to a multi-level, dynamic and intelligent direction, which can achieve in-depth mining of complex threats, propagation path analysis, and global control of security situations. Research and development in this field not only improves the accuracy and timeliness of anomaly detection, but also provides new ideas and technical support for active defense of network security.

[0003] Traditional technologies are often not comprehensive enough in anomaly detection, and it is difficult to capture complex abnormal behaviors across multiple protocol layers at the same time, resulting in some potential threats being ignored. At the same time, due to the inability to accurately distinguish between the diversity of normal behaviors and true abnormal behaviors, false positives and missed positives are common, which reduces the credibility of detection. In addition, existing methods have poor adaptability to dynamic network environments, and static baseline models are difficult to effectively cope with the frequent changes in network traffic patterns, resulting in instability in detection results. In terms of threat priority assessment, traditional technologies lack clear classification of the severity and risk level of abnormal signals, resulting in inefficient allocation of security resources. More importantly, existing technologies usually only stay at the surface detection of abnormal signals, lack in-depth analysis of deep-level threat patterns, propagation paths and root causes, and cannot provide a global understanding of threats. Summary of the invention

[0004] The purpose of the present invention is to provide an abnormal data monitoring method and system based on artificial intelligence, aiming to solve the technical problems existing in the prior art identified in the background technology.

[0005] The present invention is implemented as follows: an abnormal data monitoring method based on artificial intelligence, the method comprising: Collect packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; Perform cross-level correlation analysis on real-time data streams, use contextual information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Establish a dynamic baseline model based on real-time data streams and abnormal labeled data sets, use correlation analysis results to enhance the accuracy of the baseline model, and adjust baseline parameters in real time; Compare the real-time collected data with the baseline model to identify and mark the deviated abnormal activities, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; Based on the anomaly identification results of the baseline model, the encrypted logs are analyzed to identify potential deep-level threat patterns and abnormal signals, and the threat patterns and abnormal signals are traced to identify the root causes and abnormal paths.

[0006] As a further solution of the present invention, the real-time collection of data packets from network traffic, including encrypted data and protocol metadata, to generate real-time data streams specifically includes: Identify and determine the source of network traffic that needs to be monitored and perform packet capture of real-time network traffic; Classify captured data packets, distinguish between application layer, transport layer and network layer packets, and parse data packets to extract encrypted data and protocol metadata; Aggregate and format parsed data to generate structured real-time data streams.

[0007] As a further solution of the present invention, the cross-level correlation analysis of the real-time data stream is performed, and the association between different levels is understood by using context information, the association between data at different levels and the abnormal interaction mode are identified, and the analysis results are marked as preliminary abnormal signals to generate an abnormal marked data set with weights, which specifically includes: Sort the real-time data stream according to the protocol hierarchy and extract contextual information from the data; Conduct correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record abnormal types, frequencies, and severity; Assign weights according to the characteristics of each abnormal signal to generate a weighted abnormal labeling dataset, and integrate all analysis and labeling results to output a structured abnormal labeling dataset.

[0008] As a further solution of the present invention, the dynamic baseline model is established according to the real-time data stream and the abnormal label data set, the accuracy of the baseline model is enhanced by using the correlation analysis results, and the baseline parameters are adjusted in real time, which specifically includes: Extract features from real-time data streams and anomaly labeled data sets, establish a dynamic baseline model, and train the dynamic baseline model using the extracted features; Dynamically adjust the parameters and structure of the baseline model based on the real-time correlation analysis results.

[0009] As a further solution of the present invention, the real-time collected data is compared with the baseline model, the deviated abnormal activities are identified, and the deviated abnormal behaviors are marked, and the deviation identification results are fed back to the encrypted log for analysis to obtain abnormal background information, which specifically includes: Input the network data stream collected in real time into the baseline model, use the baseline model to compare the real-time input data, analyze the difference between the real-time data and the data of normal behavior patterns, and identify potential abnormal activities; Mark the detected abnormal deviation behavior and record the degree of deviation and related context information; Assess the severity of deviant behavior and assign an anomaly level to each anomalous activity based on the magnitude and frequency of the deviation; The deviation identification results are extracted and fed back to the encrypted log, the encrypted log is searched, and the corresponding background information is read.

[0010] As a further solution of the present invention, the anomaly identification result based on the baseline model analyzes the encrypted logs to identify potential deep-level threat patterns and abnormal signals, and traces the threat patterns and abnormal signals to identify the root causes and abnormal paths, specifically including: Collect encrypted logs related to deviation identification results and obtain full context information; Analyze encrypted logs to identify potential deep-level threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; Conduct source tracing analysis on the identified abnormal signals, identify the occurrence path of the abnormal signals, and trace the original source of the abnormalities; Based on all the information identified, a consolidated report is generated.

[0011] Another object of the present invention is to provide an abnormal data monitoring system based on artificial intelligence, the system comprising: A real-time data stream collection module is used to collect data packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; The cross-level correlation analysis module is used to perform cross-level correlation analysis on real-time data streams, and use context information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Dynamic baseline model building module, used to build a dynamic baseline model based on real-time data streams and abnormal labeled data sets, enhance the accuracy of the baseline model using correlation analysis results, and adjust baseline parameters in real time; The abnormal behavior comparison module is used to compare the real-time collected data with the baseline model, identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; The anomaly tracing module is used to analyze encrypted logs based on the anomaly identification results of the baseline model, identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths.

[0012] As a further solution of the present invention, the cross-level correlation analysis module includes: A real-time data protocol layer arrangement unit is used to arrange the real-time data stream according to the protocol layer and extract context information from the data; The protocol layer correlation analysis unit is used to perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency and severity; The abnormal signal weight assignment unit is used to assign weights according to the characteristics of each abnormal signal, generate an abnormal labeling data set with weights, integrate all analysis and labeling results, and output a structured abnormal labeling data set.

[0013] As a further solution of the present invention, the abnormal behavior comparison module includes: A difference analysis unit is used to input the network data stream collected in real time into the baseline model, compare the real-time input data with the baseline model, analyze the difference between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities; An abnormal deviation behavior marking unit is used to mark the detected abnormal deviation behavior and record the degree of deviation and related context information; An abnormal activity severity assessment unit is used to assess the severity of deviant behavior and assign an abnormality level to each abnormal activity based on the magnitude and frequency of the deviation; The deviation result extraction and encrypted log retrieval unit is used to extract the deviation identification result and feed it back to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

[0014] As a further solution of the present invention, the abnormality tracing module includes: An encrypted log collection unit, used to collect encrypted logs related to deviation identification results and obtain complete context information; A deep threat pattern recognition unit is used to analyze encrypted logs, identify potential deep threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; The abnormal signal tracing unit is used to perform tracing analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and track the original source of the abnormality; The comprehensive information integration unit is used to generate a consolidated report based on all the identified information.

[0015] The beneficial effects of the present invention are: This method builds an efficient and intelligent anomaly monitoring closed loop through real-time data collection, cross-level analysis, dynamic modeling, anomaly detection and deep threat tracing, showing significant beneficial effects. First, this method uses real-time data stream capture and cross-level correlation analysis to ensure comprehensive coverage of interactive behaviors at multiple protocol levels in the network, and combines contextual information to identify complex anomaly patterns. This multi-dimensional analysis significantly improves the accuracy of detection, effectively reduces false positives and false negatives, and generates a weighted anomaly labeling data set to provide high-quality data support for subsequent processes.

[0016] Through the application of dynamic baseline models, this method demonstrates strong adaptability in the face of dynamic changes in the network. The real-time comparison mechanism can accurately identify deviated abnormal activities. Combined with severity level assessment and priority allocation, it optimizes resource allocation, enables high-risk threats to be dealt with first, and improves the response efficiency of the system. In addition, the feedback mechanism of anomaly detection results and encrypted logs fully integrates context and background information, laying the foundation for in-depth threat analysis.

[0017] Through the mining of deep threat patterns and the tracing and analysis of abnormal behaviors, the root causes and propagation paths of threats are revealed, providing full-link threat insight capabilities. The final generated combined report is presented in a structured and visual form, from threat types, propagation paths to severity analysis, to fully support security protection decisions. This closed-loop approach not only achieves accurate detection and rapid response to network threats, but also provides key support for active defense and strategy optimization through in-depth analysis, comprehensively improving the intelligence level and overall effectiveness of network security management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of an abnormal data monitoring method based on artificial intelligence provided by an embodiment of the present invention; Figure 2 A flowchart of collecting data packets from network traffic in real time and generating real-time data streams provided by an embodiment of the present invention; Figure 3 A flowchart of identifying abnormal interactions and marking analysis results as preliminary abnormal signals provided by an embodiment of the present invention; Figure 4 A flowchart of using correlation analysis results to enhance the accuracy of a baseline model and adjust baseline parameters in real time provided by an embodiment of the present invention; Figure 5 A flowchart of feeding back the deviation identification result to the encrypted log for analysis to obtain abnormal background information provided by an embodiment of the present invention; Figure 6 A flowchart for identifying potential deep-level threat patterns and abnormal signals, and tracing the abnormalities of threat patterns and abnormal signals provided by an embodiment of the present invention; Figure 7 A structural block diagram of an abnormal data monitoring system based on artificial intelligence provided by an embodiment of the present invention; Figure 8 A structural block diagram of a cross-level correlation analysis module provided in an embodiment of the present invention; Fig. 9 A structural block diagram of an abnormal behavior comparison module provided in an embodiment of the present invention; Fig.10 A structural block diagram of an anomaly tracing module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0021] Figure 1 A flowchart of an abnormal data monitoring method based on artificial intelligence provided by an embodiment of the present invention is as follows: Figure 1 As shown, the method includes: S100, collects packets in real time from network traffic, including encryption data and protocol metadata, and generates a real-time data stream containing all relevant encryption and protocol information; In this step, first, identify and determine the source of network traffic that needs to be monitored. This step requires technicians to clarify the monitoring scope based on business needs and security policies, such as selecting specific network terminals, server nodes or application traffic as the monitoring object. Then, the data packets in the target traffic are captured through real-time network traffic capture technology. Secondly, the captured data packets are classified and parsed. This process involves dividing the data packets into application layer, transport layer and network layer data packets according to the network layer model, so as to fully extract valuable information in each layer. In particular, for the parsing of encrypted data and protocol metadata, appropriate tools and algorithms are used to extract the necessary information while protecting the sensitivity and privacy of the data. Finally, the parsed data is integrated and formatted to ensure that the generated real-time data stream has consistency and structured characteristics. This step uses professional data processing tools to clean, transform and standardize the data stream to adapt it to the subsequent analysis model requirements.

[0022] This step ensures that the monitoring system can respond quickly to dynamic changes in the network and reduce the lag time of potential security threats by capturing and processing network traffic data in real time. This real-time performance significantly enhances the agility and practicality of the system, enabling it to adapt to complex and changing network environments. Secondly, by classifying and parsing different types of data packets by layer, it is possible to capture comprehensive multi-level information, especially the extraction of encrypted data and protocol metadata, which provides key basic support for analyzing potential threats in encrypted communications. In addition, the integration and formatting process greatly improves the standardization of data, allowing subsequent analysis models to efficiently process and interpret this data, thereby improving the accuracy and stability of the overall analysis process. Finally, this process strictly adheres to the principles of data privacy and security protection, and ensures that sensitive information can be effectively protected through precise target traffic selection and parsing strategies, achieving a good balance between security and functionality.

[0023] like Figure 2 As shown, the real-time collection of data packets from network traffic, including encrypted data and protocol metadata, generates a real-time data stream, specifically including: S110, identifying and determining the source of network traffic that needs to be monitored, and capturing data packets of real-time network traffic; S120, classifying the captured data packets, distinguishing data packets at the application layer, transport layer, and network layer, and parsing the data packets to extract encrypted data and protocol metadata; S130, integrating and formatting the parsed data to generate a structured real-time data stream.

[0024] S200 performs cross-level correlation analysis on real-time data streams, uses context information to understand the associations between different levels, identifies the associations and abnormal interaction patterns between data at different levels, and marks the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; In this step, the real-time data stream is organized according to the protocol layer. This organization process is based on the OSI model, which manages the data from the application layer, transport layer to the network layer in a hierarchical manner, and extracts key contextual information from it, including the timing characteristics of the data, the characteristics of the protocol fields, and the contextual dependencies of cross-layer interactions. The extraction of this contextual information ensures a comprehensive understanding of the data characteristics and traffic patterns, providing in-depth support for subsequent analysis. Subsequently, the system will perform correlation analysis between different protocol layers, and use machine learning algorithms or deep learning models to identify normal interaction patterns (such as expected request-response behavior) and potential abnormal interaction patterns (such as traffic surges, atypical protocol usage, etc.). For the identified abnormal interaction patterns, the system marks them and attaches detailed information, such as the type of anomaly (such as traffic anomaly, protocol anomaly, etc.), frequency (number of occurrences or frequency), and severity (the degree of potential threat to the system or network). This information forms the basis of preliminary abnormal signals.

[0025] Next, the system assigns weights based on the features of each abnormal signal. The weights are calculated based on the importance of the abnormal features, the frequency of occurrence, the correlation in historical data, and other factors. The purpose is to provide a more accurate basis for subsequent model training and abnormal priority sorting. Through this process, a weighted abnormal labeling dataset is generated. This dataset not only contains preliminary abnormal signals, but also integrates the abnormal weight information and other analysis and labeling results, and is output in a structured form, providing complete data support for the establishment of the dynamic baseline model and subsequent comparison.

[0026] This step integrates the multi-layer characteristics of the data through cross-layer correlation analysis, and identifies complex abnormal patterns that may not be captured by single-layer analysis. This cross-layer analysis capability enables the system to have higher anomaly detection accuracy, especially in the face of complex protocols and encrypted traffic in modern networks, and can more comprehensively capture global abnormal behaviors. Secondly, through the extraction and analysis of contextual information, S200 can understand the temporal relationship and semantic association between data, providing key support for the accurate identification of abnormal interaction patterns. In addition, the type, frequency, and severity information attached to the abnormal signal tag enables the system to classify abnormal events, better support the response decision of the security team, and improve processing efficiency. The weight allocation mechanism further optimizes the application value of abnormal signals in subsequent analysis, ensuring that high-priority anomalies can be processed first, thereby significantly enhancing the threat identification capability and resource allocation efficiency of the overall monitoring system.

[0027] like Figure 3 As shown, the cross-level correlation analysis of the real-time data stream is performed, and the association between different levels is understood by using context information, the association and abnormal interaction patterns of data between different levels are identified, and the analysis results are marked as preliminary abnormal signals to generate an abnormal labeling data set with weights, which specifically includes: S210, sorting the real-time data stream according to the protocol layer and extracting context information from the data; S220, performing correlation analysis between different protocol layers, identifying normal and abnormal interaction patterns, marking abnormal interaction patterns, generating preliminary abnormal signals, and recording abnormal types, frequencies, and severity; S230, assigning weights according to the features of each abnormal signal, generating an abnormal labeling data set with weights, and integrating all analysis and labeling results to output a structured abnormal labeling data set.

[0028] S300, establishes a dynamic baseline model based on the real-time data stream and the abnormal labeled data set, uses the correlation analysis results to enhance the accuracy of the baseline model, and adjusts the baseline parameters in real time; This step extracts features from real-time data streams and anomaly labeled data sets. This process uses advanced machine learning feature engineering techniques to extract key attributes that reflect data behavior and characteristics, such as data packet traffic patterns, protocol interaction characteristics, time series patterns, and the degree of deviation from historical data. These features provide the necessary input dimensions for building a dynamic baseline model, enabling the model to accurately identify the boundaries between normal traffic and abnormal behavior.

[0029] Next, the establishment and training of the dynamic baseline model is the key to this step. The baseline model is designed as a statistic that can adapt to real-time changes in network behavior. By inputting and training the extracted features, the baseline model can capture the dynamic changes of normal network behavior and form a benchmark for "normal behavior." At the same time, the model will use the information in the abnormal labeled data set to incorporate the labeled abnormal signals into the training process to enhance the model's sensitivity and recognition ability to abnormal behavior, thereby avoiding false positives due to the diversity of normal behavior.

[0030] After the model is built, dynamically adjusting the parameters and structure of the baseline model based on the results of real-time correlation analysis is another important step. This means that the baseline model is not fixed, but will be continuously updated with the input and analysis results of real-time data streams. For example, when network traffic has periodic fluctuations or sudden changes, the dynamic adjustment mechanism can ensure that the model can reflect the actual situation of the current network environment by optimizing the model's thresholds, weights, or algorithm structure in real time, and achieve accurate capture and continuous adaptation of abnormal deviations.

[0031] Finally, the system uses the dynamic baseline model to compare the data collected in real time to identify deviated abnormal activities and mark the relevant information. For example, when the baseline model detects that certain traffic characteristics deviate from the range of normal behavior, these flows will be marked as abnormal behavior, and the deviation identification results will be fed back to the encrypted log for further analysis to obtain the context of the data and potential abnormal root cause information. This closed-loop feedback mechanism not only improves the model's adaptability to complex network environments, but also provides key data support for subsequent source tracing analysis and threat location.

[0032] The dynamic baseline model can capture the dynamic characteristics of network behavior, which makes it extremely flexible and adaptable when dealing with complex and changeable communication modes in modern networks, avoiding the limitation that static models are prone to failure in dynamic environments. Secondly, the combination of feature extraction and model training process not only improves the sensitivity of the model to abnormal signals, but also effectively reduces the false alarm rate and missed alarm rate by integrating the information of abnormal labeled data sets, thereby enhancing the accuracy and reliability of anomaly detection. In addition, the introduction of dynamic adjustment mechanism enables the system to optimize the model performance according to the changes in real-time data, and achieve rapid response without human intervention, which greatly improves the system's operating efficiency and the timeliness of threat identification. Through the linkage analysis with encrypted logs, it can not only realize the identification of single abnormal behavior, but also provide support for subsequent abnormal background analysis and threat tracing, thereby greatly improving the global threat perception capability of the entire monitoring method.

[0033] like Figure 4As shown, the dynamic baseline model is established based on the real-time data stream and the abnormal labeled data set, the accuracy of the baseline model is enhanced by using the correlation analysis results, and the baseline parameters are adjusted in real time, specifically including: S310, extracting features from the real-time data stream and the abnormality labeling data set, establishing a dynamic baseline model, and training the dynamic baseline model using the extracted features; S320, dynamically adjusting the parameters and structure of the baseline model according to the real-time correlation analysis result.

[0034] S400 compares the real-time collected data with the baseline model, identifies the deviated abnormal activities, marks the deviated abnormal behaviors, and feeds the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; In this step, the network data stream collected in real time will be input into the baseline model that has been established and dynamically adjusted. The baseline model has accumulated an accurate understanding of normal network behavior patterns during previous training and real-time updates. Therefore, during the comparison process, the model will analyze the real-time input data one by one based on the known normal behavior characteristics. By comparing the differences between real-time data and the standard baseline, the system can identify abnormal activities or behaviors. This analysis not only includes matching numerical features (such as packet size, traffic rate, etc.), but also covers more complex pattern differences (such as fluctuation characteristics of time series, protocol mismatches, etc.), thereby ensuring that abnormal activities can be fully captured.

[0035] Next, the detected abnormal deviation behavior is marked. In this step, the system records the degree of deviation of each abnormal behavior and related context information. For example, for an obvious traffic anomaly, the system will not only mark its deviation magnitude (such as the percentage of the data packet size exceeding the normal range), but also record its context information (such as the source IP address of the abnormal data, the target port, the type of protocol used, etc.). This detailed marking can not only provide data support for subsequent analysis, but also help the security team quickly locate the root cause of the problem.

[0036] The system then conducts a severity assessment of the deviation behavior. This assessment is based on the magnitude and frequency of the deviation behavior. For example, a minor single deviation may be marked as a low-level anomaly, while multiple frequent deviations with obvious characteristics will be assessed as a high-level anomaly. This severity assessment assigns an anomaly level (such as low, medium, high, or a similar scoring mechanism) to each abnormal activity, which not only enhances the objectivity and accuracy of identification, but also provides priority guidance for subsequent processing, so that high-risk anomalies can be handled first, ensuring the timeliness and effectiveness of network security.

[0037] Finally, the system feeds back the extracted deviation identification results to the encrypted log and retrieves and analyzes the log data. This feedback process is a closed-loop mechanism. By associating the identification results with the background information in the encrypted log, the system can further extract the corresponding contextual data (such as communication behaviors before or after the abnormal activity, the usage history of specific protocols, etc.). This mining of background information provides more in-depth support for subsequent threat analysis and root cause location, ensuring that abnormal signals are not analyzed in isolation, but are comprehensively evaluated in the specific environment in which they occur.

[0038] By comparing the real-time data stream with the dynamic baseline model, the system can efficiently capture abnormal activities that deviate from normal behavior patterns. This real-time and accuracy ensures that network security incidents can be discovered at an early stage, thereby effectively reducing the destructiveness of potential threats. Secondly, the marking and severity assessment of deviant behaviors greatly improves the ability to classify and prioritize abnormal activities, providing clear processing guidance for security teams. By assigning abnormal levels, the system can help network administrators focus on the most critical high-risk events, thereby greatly improving response efficiency. Furthermore, the feedback of deviation results and the correlation analysis of encrypted logs closely combine abnormal signals with their contextual information, laying a solid foundation for threat tracing and root cause analysis. Through this mechanism, the system not only identifies abnormal activities, but also reveals the background and reasons for their occurrence, allowing security teams to more comprehensively understand the full picture of the problem and formulate targeted countermeasures.

[0039] like Figure 5 As shown, the real-time collected data is compared with the baseline model to identify deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information, including: S410, inputting the network data stream collected in real time into the baseline model, using the baseline model to compare the real-time input data, analyzing the difference between the real-time data and the data of the normal behavior pattern, and identifying potential abnormal activities; S420, marking the detected abnormal deviation behavior, recording the degree of deviation and related context information; S430, evaluating the severity of the deviant behavior and assigning an abnormality level to each abnormal activity based on the magnitude and frequency of the deviation; S440, extract the deviation recognition result and feed it back to the encrypted log, search the encrypted log, and read the corresponding background information.

[0040] S500, based on the anomaly identification results of the baseline model, analyzes the encrypted logs to identify potential deep-level threat patterns and abnormal signals, and traces the threat patterns and abnormal signals to identify the root causes and abnormal paths.

[0041] This step collects encrypted logs related to the deviation identification results and obtains complete context information through linkage with S400. This process helps the system accurately locate historical activities and background information related to abnormal signals by mining the timestamps, data packet contents, protocol information, and related communication metadata in the encrypted logs. For example, by retrieving the source IP address, target port number, and related device identification involved in the abnormal behavior in the log, context data directly or indirectly related to the abnormal behavior can be obtained. This context information provides comprehensive background support for subsequent analysis, ensuring that abnormal behavior is not analyzed in isolation, but is placed in the environment where it is generated for comprehensive analysis.

[0042] Next, the system conducts an in-depth analysis of the encrypted logs to identify potential deep-level threat patterns and abnormal signals. During this process, the system uses artificial intelligence models to mine and model the data in the encrypted logs, trying to find potential threat characteristics from surface anomalies. For example, by conducting a comprehensive analysis of multiple abnormal interactions, the system may discover a continuous scanning behavior or a precursor to a distributed denial of service (DDoS) attack. In addition, the system can extract more potential threat clues from the logs and further identify threat types, such as data leakage, privilege escalation attacks, or lateral movement threats, thereby providing the security team with more detailed threat classification information.

[0043] The identified abnormal signals are then traced back to the source of the abnormal signal and its original source is identified. The traceability analysis gradually restores the propagation trajectory of the abnormal signal by tracing back the communication links in the encrypted log. For example, the system may locate the starting point of the malicious behavior by tracking the flow of abnormal activities and determining the source IP range of the attacker. At the same time, the traceability analysis can reveal the propagation path of the abnormal signal and determine whether there are other potential targets affected by the abnormal activity. By identifying the "starting point-path-end point" full-link information of abnormal behavior, the system provides more accurate and comprehensive basic data for threat disposal.

[0044] Finally, based on all the identified threat information, the system generates a consolidated report. This report summarizes everything from context information extraction to threat type identification, from source analysis to path attribution in a structured and visual form. The report not only includes a detailed description and background information of each abnormal signal, but also provides the security management team with an overall insight into the deep threat pattern. For example, the report may include a visual threat propagation map, the distribution of threat severity levels, and response recommendations for each threat type. This comprehensive report enables the security team to quickly understand the full picture of the threat and develop targeted security protection measures.

[0045] Through in-depth analysis of encrypted logs and comprehensive extraction of contextual information, this step can reveal the deep threat patterns behind the surface anomalies, helping the security team understand the behavioral logic and potential intentions of attackers, thereby improving threat prediction and defense capabilities. Secondly, the introduction of source tracing analysis enables the system to not only identify abnormal signals, but also accurately trace their original sources and propagation paths. This full-link threat analysis capability effectively reduces the blind spots of security protection and makes the response to attacks more accurate and efficient. In addition, the generation of a combined report presents complex security analysis results in a structured and visual manner, greatly reducing the complexity of information transmission, helping the team quickly grasp key information and formulate response strategies. Most importantly, this step provides closed-loop support for the abnormal data monitoring system through comprehensive threat identification and source tracing analysis, enabling the system to advance from real-time detection to in-depth analysis and decision support, thereby comprehensively improving the overall protection capabilities and processing efficiency of network security.

[0046] like Figure 6 As shown in the figure, the anomaly identification result based on the baseline model analyzes the encrypted logs, identifies potential deep-level threat patterns and abnormal signals, and traces the threat patterns and abnormal signals to identify the root causes and abnormal paths, including: S510, collecting encrypted logs related to the deviation identification results and obtaining complete context information; S520, analyzing the encrypted logs, identifying potential deep-level threat patterns and abnormal signals, extracting potential threat clues from the logs, and identifying threat types; S530, performing source tracing analysis on the identified abnormal signal, identifying the occurrence path of the abnormal signal, and tracing the initial source of the abnormality; S540, generating a combined report based on all the identified information.

[0047] Figure 7 A structural block diagram of an abnormal data monitoring system based on artificial intelligence provided by an embodiment of the present invention, such as Figure 7 As shown, the system comprises: A real-time data stream collection module 100 is used to collect data packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; The cross-level correlation analysis module 200 is used to perform cross-level correlation analysis on the real-time data stream, and use context information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling data set; A dynamic baseline model building module 300 is used to build a dynamic baseline model based on the real-time data stream and the abnormal labeled data set, enhance the accuracy of the baseline model using the correlation analysis results, and adjust the baseline parameters in real time; The abnormal behavior comparison module 400 is used to compare the real-time collected data with the baseline model, identify the deviated abnormal activities, mark the deviated abnormal behaviors, feed back the deviation identification results to the encrypted log for analysis, and obtain abnormal background information; The anomaly tracing module 500 is used to analyze the encrypted logs based on the anomaly identification results of the baseline model, identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths.

[0048] like Figure 8 As shown, the cross-level correlation analysis module 200 includes: A real-time data protocol layer arrangement unit 210 is used to arrange the real-time data stream according to the protocol layer and extract context information from the data; The protocol layer correlation analysis unit 220 is used to perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record abnormal types, frequencies, and severity; The abnormal signal weight assignment unit 230 is used to assign weights according to the characteristics of each abnormal signal, generate an abnormal labeling data set with weights, and integrate all analysis and labeling results to output a structured abnormal labeling data set.

[0049] like Fig. 9 As shown, the abnormal behavior comparison module 400 includes: The difference analysis unit 410 is used to input the network data stream collected in real time into the baseline model, compare the real-time input data with the baseline model, analyze the difference between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities; An abnormal deviation behavior marking unit 420 is used to mark the detected abnormal deviation behavior and record the degree of deviation and related context information; An abnormal activity severity assessment unit 430 is used to assess the severity of the deviant behavior and assign an abnormality level to each abnormal activity based on the magnitude and frequency of the deviation; The deviation result extraction and encrypted log retrieval unit 440 is used to extract the deviation identification result and feed it back to the encrypted log, search the encrypted log, and read the corresponding background information.

[0050] like Fig.10 As shown, the abnormality tracing module 500 includes: The encrypted log collection unit 510 is used to collect the encrypted logs related to the deviation identification results and obtain the complete context information; A deep threat pattern recognition unit 520 is used to analyze the encrypted logs, identify potential deep threat patterns and abnormal signals, extract potential threat clues from the logs, and identify threat types; The abnormal signal tracing unit 530 is used to perform a traceability analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and track the initial source of the abnormality; The comprehensive information integration unit 540 is used to generate a combined report based on all the identified information.

[0051] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0052] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0053] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0054] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An abnormal data monitoring method based on artificial intelligence, characterized in that: The method comprises: Collect packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; Perform cross-level correlation analysis on real-time data streams, use contextual information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Establish a dynamic baseline model based on real-time data streams and abnormal labeled data sets, use correlation analysis results to enhance the accuracy of the baseline model, and adjust baseline parameters in real time; Compare the real-time collected data with the baseline model to identify and mark the deviated abnormal activities, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; Based on the anomaly identification results of the baseline model, the encrypted logs are analyzed to identify potential deep-level threat patterns and abnormal signals, and the threat patterns and abnormal signals are traced to identify the root causes and abnormal paths.

2. The method according to claim 1, characterized in that The real-time collection of data packets from network traffic, including encrypted data and protocol metadata, and the generation of real-time data streams specifically include: Identify and determine the source of network traffic that needs to be monitored and perform packet capture of real-time network traffic; Classify captured data packets, distinguish between application layer, transport layer and network layer packets, and parse data packets to extract encrypted data and protocol metadata; Aggregate and format parsed data to generate structured real-time data streams.

3. The method according to claim 2, characterized in that The cross-level correlation analysis of the real-time data stream is performed, and context information is used to understand the associations between different levels, identify the associations and abnormal interaction patterns of data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling data set, which specifically includes: Sort the real-time data stream according to the protocol layer and extract contextual information from the data; Conduct correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record abnormal types, frequencies, and severity; Assign weights according to the characteristics of each abnormal signal to generate a weighted abnormal labeling dataset, and integrate all analysis and labeling results to output a structured abnormal labeling dataset.

4. The method according to claim 3, characterized in that The method of establishing a dynamic baseline model based on the real-time data stream and the abnormal labeled data set, enhancing the accuracy of the baseline model using the correlation analysis results, and adjusting the baseline parameters in real time specifically includes: Extract features from real-time data streams and anomaly labeled data sets, establish a dynamic baseline model, and train the dynamic baseline model using the extracted features; Dynamically adjust the parameters and structure of the baseline model based on the real-time correlation analysis results.

5. The method according to claim 4, characterized in that The real-time collected data is compared with the baseline model to identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information, including: Input the network data stream collected in real time into the baseline model, use the baseline model to compare the real-time input data, analyze the difference between the real-time data and the data of normal behavior patterns, and identify potential abnormal activities; Mark the detected abnormal deviation behavior and record the degree of deviation and related context information; Assess the severity of deviant behavior and assign an anomaly level to each anomalous activity based on the magnitude and frequency of the deviation; The deviation identification results are extracted and fed back to the encrypted log, the encrypted log is searched, and the corresponding background information is read.

6. The method according to claim 5, characterized in that The anomaly identification results based on the baseline model analyze the encrypted logs to identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths, including: Collect encrypted logs related to deviation identification results and obtain full context information; Analyze encrypted logs to identify potential deep-level threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; Conduct source tracing analysis on the identified abnormal signals, identify the occurrence path of the abnormal signals, and trace the original source of the abnormalities; Based on all the information identified, a consolidated report is generated.

7. An abnormal data monitoring system based on artificial intelligence, characterized in that: The system comprises: A real-time data stream collection module is used to collect data packets from network traffic in real time, including encryption data and protocol metadata, and generate a real-time data stream containing all relevant encryption and protocol information; The cross-level correlation analysis module is used to perform cross-level correlation analysis on real-time data streams, and use context information to understand the associations between different levels, identify the associations and abnormal interaction patterns between data at different levels, and mark the analysis results as preliminary abnormal signals to generate a weighted abnormal labeling dataset; Dynamic baseline model building module, used to build a dynamic baseline model based on real-time data streams and abnormal labeled data sets, enhance the accuracy of the baseline model using correlation analysis results, and adjust baseline parameters in real time; The abnormal behavior comparison module is used to compare the real-time collected data with the baseline model, identify the deviated abnormal activities, mark the deviated abnormal behaviors, and feed the deviation identification results back to the encrypted log for analysis to obtain abnormal background information; The anomaly tracing module is used to analyze encrypted logs based on the anomaly identification results of the baseline model, identify potential deep-level threat patterns and abnormal signals, and trace the threat patterns and abnormal signals to identify the root causes and abnormal paths.

8. The system according to claim 7, characterized in that The cross-level correlation analysis module includes: A real-time data protocol layer arrangement unit is used to arrange the real-time data stream according to the protocol layer and extract context information from the data; The protocol layer correlation analysis unit is used to perform correlation analysis between different protocol layers, identify normal and abnormal interaction patterns, mark abnormal interaction patterns, generate preliminary abnormal signals, and record the abnormal type, frequency and severity; The abnormal signal weight assignment unit is used to assign weights according to the characteristics of each abnormal signal, generate an abnormal labeling data set with weights, integrate all analysis and labeling results, and output a structured abnormal labeling data set.

9. The system according to claim 8, characterized in that The abnormal behavior comparison module includes: A difference analysis unit is used to input the network data stream collected in real time into the baseline model, compare the real-time input data with the baseline model, analyze the difference between the real-time data and the data of the normal behavior pattern, and identify potential abnormal activities; An abnormal deviation behavior marking unit is used to mark the detected abnormal deviation behavior and record the degree of deviation and related context information; An abnormal activity severity assessment unit is used to assess the severity of deviant behavior and assign an abnormality level to each abnormal activity based on the magnitude and frequency of the deviation; The deviation result extraction and encrypted log retrieval unit is used to extract the deviation identification result and feed it back to the encrypted log, retrieve the encrypted log, and read the corresponding background information.

10. The system according to claim 9, characterized in that The abnormality tracing module includes: An encrypted log collection unit, used to collect encrypted logs related to deviation identification results and obtain complete context information; A deep threat pattern recognition unit is used to analyze encrypted logs, identify potential deep threat patterns and abnormal signals, extract potential threat clues from logs, and identify threat types; The abnormal signal tracing unit is used to perform tracing analysis on the identified abnormal signal, identify the occurrence path of the abnormal signal, and track the original source of the abnormality; The comprehensive information integration unit is used to generate a consolidated report based on all the identified information.

Images