System anomaly detection method and device based on correlation analysis, equipment and medium

Abstract

The invention provides a system anomaly detection method and device based on correlation analysis, equipment and a medium, and relates to the field of network security. The method comprises the steps of obtaining first system data of a network system, performing feature extraction on the first system data to obtain various data features, and performing feature fusion on the various data features to generate a comprehensive feature vector; generating an association rule based on the comprehensive feature vector by using an association analysis algorithm, the association rule being used for representing an association relationship between the data features and an association relationship between the data features and system behaviors; after the association relationship is determined, performing model training based on the comprehensive feature vector to obtain an association analysis model; monitoring operation of the network system, acquiring second system data of the network system in real time, and constructing a data feature vector based on the second system data; and inputting the data feature vector into a correlation analysis model, identifying a correlation relationship between the system behavior and the second system data through the correlation analysis model, and detecting the system abnormal behavior.

Description

Technical Field

[0001] The present application relates to the field of network security, and in particular to a system anomaly detection method, device, computer equipment and readable storage medium based on correlation analysis. Background Art

[0002] With the rapid development of information technology, network security issues are becoming increasingly prominent. Vulnerabilities in software systems have become the main target of attackers. Vulnerability detection technology is one of the important means to ensure system security. Vulnerability detection methods in related technologies mainly include static analysis, dynamic analysis, and symbolic execution.

[0003] Static analysis techniques look for potential vulnerabilities by examining software code or binary files. This method does not require running the program and mainly relies on pattern matching and rule bases. Although static analysis can find known vulnerabilities, it is limited in detecting unknown vulnerabilities because the rule base it relies on often cannot cover all possible vulnerability types. Dynamic analysis techniques monitor the behavior of the program while it is running and detect abnormal activities. This method can capture vulnerabilities in the actual running environment, but its coverage is limited by the quality and scope of the test cases. In addition, dynamic analysis has weak detection capabilities for unknown vulnerabilities because it mainly relies on known attack patterns and behavioral characteristics. Symbolic execution technology uses the input of the program as symbolic variables and discovers potential vulnerabilities by simulating the program execution path. This method can systematically explore various execution paths of the program, but its computational complexity is high and it is prone to path explosion problems. At the same time, symbolic execution still has challenges in detecting unknown vulnerabilities because the constraint solver it relies on may not fully cover all complex program logic.

[0004] Therefore, how to effectively detect unknown vulnerabilities becomes an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present application provides a system anomaly detection method, apparatus, computer device and readable storage medium based on association analysis, which realizes the identification of potential abnormal association relationships and thus discovers unknown vulnerabilities.

[0006] In a first aspect, an embodiment of the present application provides a system anomaly detection method based on association analysis, comprising: Acquire first system data of the network system, perform feature extraction on the first system data to obtain multiple data features, and perform feature fusion on the multiple data features to generate a comprehensive feature vector; Generate association rules based on the comprehensive feature vector using an association analysis algorithm, wherein the association rules are used to represent the association relationship between the data features and between the data features and system behavior; After determining the association relationship, performing model training based on the comprehensive feature vector to obtain an association analysis model; Monitoring the operation of the network system, acquiring second system data of the network system in real time, and constructing a data feature vector based on the second system data; The data feature vector is input into the association analysis model, and the association relationship between the system behavior and the second system data is identified through the association analysis model to detect abnormal system behavior.

[0007] In a second aspect, an embodiment of the present application provides a system anomaly detection device based on association analysis, comprising: A data acquisition module, used for acquiring first system data of a network system, performing feature extraction on the first system data to obtain a plurality of data features, and performing feature fusion on the plurality of data features to generate a comprehensive feature vector; An association rule generation module, used to generate association rules based on the comprehensive feature vector using an association analysis algorithm, wherein the association rules are used to represent the association relationship between the data features and between the data features and system behavior; A model training module, used to perform model training based on the comprehensive feature vector after determining the association relationship, so as to obtain an association analysis model; The data acquisition module is further used to monitor the operation of the network system, acquire the second system data of the network system in real time, and construct a data feature vector based on the second system data; The detection module is used to input the data feature vector into the association analysis model, identify the association relationship between the system behavior and the second system data through the association analysis model, and detect abnormal system behavior.

[0008] In a third aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be executed on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the first aspect.

[0012] The embodiments of the present application can realize dynamic correlation analysis of abnormal system behaviors. By monitoring the behavioral data of the system during operation, dynamic correlation analysis can be performed to identify potential abnormal correlation relationships, that is, the process and technical details of detecting and identifying potential vulnerabilities during system operation, thereby discovering unknown vulnerabilities.

[0013] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of the process of a system anomaly detection method based on association analysis according to an embodiment of the present application is shown; Figure 2 A structural block diagram of a system anomaly detection device based on correlation analysis according to an embodiment of the present application is shown; Figure 3 A structural block diagram of a computer device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0016] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0017] In conjunction with the accompanying drawings, the following detailed description is given of the method, device, computer equipment and readable storage medium for detecting anomalies based on correlation analysis provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0018] The present application embodiment provides a system anomaly detection method based on association analysis, such as Figure 1 As shown, the method includes: Step 101 , obtaining first system data of a network system, performing feature extraction on the first system data to obtain a plurality of data features, and performing feature fusion on the plurality of data features to generate a comprehensive feature vector.

[0019] Step 102: Generate association rules based on the comprehensive feature vector using an association analysis algorithm. The association rules are used to represent the association relationship between data features and between data features and system behaviors.

[0020] Step 103, after determining the association relationship, model training is performed based on the comprehensive feature vector to obtain an association analysis model.

[0021] In the above steps 101 to 103, the first system data (i.e., sample data) of the acquired network system is mined and analyzed using association analysis algorithms, such as Apriori, FP-Growth, etc. Specifically, representative data features, such as data prone to vulnerabilities, are extracted from the first system data to construct a comprehensive feature vector. Association rules are generated using an association analysis algorithm, and the association rules are used to reveal the potential association relationship between each data feature and between the data feature and the system behavior. Furthermore, based on the comprehensive feature vector with a determined association relationship, model training is performed to obtain an association analysis model to identify normal behavior patterns and abnormal behavior patterns.

[0022] In one embodiment, the association analysis model can be a machine learning model or a deep learning model, etc. The machine learning models include decision trees, random forests, support vector machines, etc., and the deep learning models include Autoencoders, GraphNeural Networks, etc. By performing association analysis with a deep learning model, complex patterns and associations in the data can be automatically learned to improve the ability to detect unknown vulnerabilities.

[0023] In the embodiment of the present application, an association analysis algorithm is used to analyze the association relationship between data and data, and between data and system behavior, and then an association analysis model is trained based on the comprehensive feature vector of the analyzed association relationship. When the system behavior is subsequently detected, the association analysis model is directly used for automatic identification, rather than searching in the association rules. Automatic identification is performed through the association analysis model, thereby improving detection accuracy and detection efficiency.

[0024] Step 104 , monitor the operation of the network system, obtain the second system data of the network system in real time, and construct a data feature vector based on the second system data.

[0025] Step 105: input the data feature vector into the association analysis model, identify the association relationship between the system behavior and the second system data through the association analysis model, and detect abnormal system behavior.

[0026] In the above steps 104 and 105, the real-time system data during the operation of the network system, that is, the second system data, is monitored to construct a data feature vector. This application can introduce stream processing technologies such as Apache Kafka and Apache Flink to achieve real-time data collection and processing, ensuring that the system can quickly respond to and detect the latest security threats.

[0027] The data feature vector is input into the association analysis model to determine whether the second system data conforms to the normal behavior of the system, so as to detect the abnormal behavior of the system. The association analysis model is a pre-trained machine learning model used to identify the association relationship between the system behavior and the second system data. That is, the abnormal behavior of the system is detected by analyzing the association relationship. For example, if the association analysis model identifies that the score of the association relationship between the second system data and a certain system behavior is less than a preset threshold, the system behavior is output as abnormal behavior; if the association analysis model identifies that the score of the association relationship between the second system data and a certain system behavior is greater than or equal to the preset threshold, the system behavior is output as normal behavior.

[0028] The embodiments of the present application can realize dynamic correlation analysis of abnormal system behaviors. By monitoring the behavioral data of the system during operation, dynamic correlation analysis can be performed to identify potential abnormal correlation relationships, that is, the process and technical details of detecting and identifying potential vulnerabilities during system operation, thereby discovering unknown vulnerabilities.

[0029] Compared with the static analysis technology in the related art, the solution of the embodiment of the present application does not rely on predefined known vulnerability patterns and rules, and can adapt to changing attack methods and emerging vulnerability types. Compared with the dynamic analysis technology in the related art, the coverage of the solution of the embodiment of the present application will not be limited by the test cases and rule base, and can detect all potential vulnerabilities of the program as comprehensively as possible. Compared with the symbolic execution technology in the related art, the solution of the embodiment of the present application is not computationally complex and will not limit its feasibility in practical applications.

[0030] The system anomaly detection method based on association analysis in the embodiment of the present application can be applied to the following scenarios and achieve corresponding effects: (1) Enterprise information system security: Protect the enterprise's internal information system from threats from unknown vulnerabilities and new types of attacks, and ensure the security of enterprise data and business.

[0031] (2) Cloud computing environment: Real-time monitoring of cloud computing platforms and virtual machines to detect and defend against potential security threats and ensure the safe operation of the cloud environment.

[0032] (3) Internet of Things (IoT) Security: Monitor and protect IoT devices and networks to prevent security risks caused by unknown vulnerabilities and ensure the normal operation of IoT systems.

[0033] (4) Industrial Control System (ICS) Security: Monitor various devices and network traffic in industrial control systems, detect and respond to potential security threats, and ensure the safety and continuity of industrial production.

[0034] In one embodiment of the present application, generating association rules based on the comprehensive feature vector includes: generating a target frequent item set according to the comprehensive feature vector; and determining the association rules based on the target frequent item set.

[0035] In this embodiment, the target frequent item set is first generated according to the comprehensive feature vector, and then the association rule is determined based on the target frequent item set. In one embodiment of the present application, the step of generating the target frequent item set according to the comprehensive feature vector specifically includes: Generate the i-th candidate item set, calculate the support of the items in the i-th candidate item set, filter out the items whose support is greater than the preset support threshold, form the i-th frequent item set, repeat the steps of generating candidate item sets until a frequent item set is formed, until a larger candidate item set cannot be generated; when i=1, scan each element in the comprehensive feature vector, take an element as an item, and generate the i-th candidate item set; when i is greater than 1, connect multiple items in the i-1th frequent item set to generate the i-th candidate item set; The intersection of all frequent itemsets is taken as the target frequent itemset.

[0036] In this embodiment, item scanning is performed to generate a candidate item set, support calculation is performed on the items in the candidate item set, items whose support is greater than a preset support threshold are screened out to form a frequent item set, and the steps of generating candidate item sets to form a frequent item set are repeated until a larger candidate item set cannot be generated, thereby obtaining multiple frequent item sets, and the intersection of all frequent item sets is used as the target frequent item set. Specifically, when generating the first frequent item set, each element (one element is one item) in the comprehensive feature vector is scanned to generate the first candidate item set, thereby obtaining the first frequent item set; when generating frequent item sets after the first frequent item set, multiple items in the previous frequent item set are connected to generate the current candidate item set, thereby obtaining the current frequent item set.

[0037] The definitions mentioned above include: Transaction: Each row of data in a data set is called a transaction. Item: An element in a transaction. Itemset: A collection of items. Frequent Itemset: An item set whose support is greater than or equal to the minimum support threshold. Support: The frequency with which an item set appears in all transactions.

[0038] Exemplarily, the process of generating the target frequent itemsets includes: Step 1: Generate candidate item sets Starting from a single entry in the comprehensive feature vector, progressively larger sets of candidate items are generated.

[0039] First scan: Calculate the support of each single item to form a candidate 1-item set (C1).

[0040] Filter frequent 1-item sets: Filter out items whose support is greater than or equal to the minimum support threshold to form frequent 1-item sets (L1).

[0041] Step 2: Generate a larger itemset Generate candidate k-itemsets using frequent (k-1) itemsets.

[0042] Connection operation: connect frequent (k-1) item sets two by two to generate candidate k item sets (C k ). For example, if {A,B} and {A,C} are frequent 2-item sets, then {A,B,C} is a candidate 3-item set.

[0043] Pruning operation: Check whether all subsets of the candidate k-item set are frequent. If any subset is not frequent, delete the candidate item set.

[0044] Step 3: Scan the dataset and calculate support Scan the data set and calculate the support for each candidate item set.

[0045] Support calculation: traverse all transactions, count the number of times each candidate item set appears, and calculate its support.

[0046] Step 4: Filter frequent itemsets Filter out candidate item sets whose support is greater than or equal to the minimum support threshold to form frequent k-item sets (L k ).

[0047] Step 5: Repeat steps 2 to 4 until no larger candidate item set can be generated.

[0048] Assume there is a transaction database as shown in Table 1: Table 1

[0049] First scan (generating frequent 1-item set L1) Candidate 1-item set C1: {A}, {B}, {C}, {D} Support calculation: {A}:4, {B}:4, {C}:4, {D}:3.

[0050] Filter support ≥ 3 (assuming the minimum support threshold is 3) and obtain the frequent 1-item set L1: {A}, {B}, {C}, {D}.

[0051] Second scan (generating frequent 2-item set L2) The concatenation operation generates C2: {A,B},{A,C},{A,D},{B,C},{B,D},{C,D}.

[0052] Support calculation: {A,B}:3, {A,C}:3, {A,D}:2, {B,C}:3, {B,D}:3, {C,D}:3.

[0053] Filter support ≥ 3 and obtain the frequent 2-item set L2: {A,B},{A,C},{B,C},{B,D},{C,D}.

[0054] The third scan (generating frequent 3-item set L3) The join operation generates C3: {A,B,C},{B,C,D}.

[0055] Support calculation: {A,B,C}:2, {B,C,D}:3.

[0056] Filter support ≥ 3 and obtain the frequent 3-item set L3: {B, C, D}.

[0057] When no larger candidate item set can be generated, the algorithm terminates. Finally, the target frequent item set is the union of L1, L2, and L3.

[0058] In one embodiment of the present application, the step of determining association rules based on the target frequent item set specifically includes: for any frequent item set in the target frequent item set, generating candidate rules from the frequent item set; calculating the confidence of each candidate rule, and screening out candidate rules whose confidence is greater than a preset confidence threshold as association rules.

[0059] In this embodiment, association rule generation is to find rules with strong association based on the generated target frequent item sets. The process mainly includes calculating indicators such as confidence and lift to determine the significance of the rules. The following are the detailed steps of association rule generation: Step 1: Define basic concepts (1) Association rules: in the form of A B, where A and B are item sets, and A B = empty set.

[0060] (2) Support: Rule A The support of B is the item set A The frequency with which B appears in transactions.

[0061] Support(A B) = The transaction contains A Number of B / Total number of transactions (3) Confidence: Rule A The confidence of B is the item set A The ratio of the frequency of B to the frequency of item set A.

[0062] Confidence(A B) =Support (A B) / Support (A) (4) Lift: Rule A The lift of B is the ratio of the confidence score to the independent probability of B.

[0063] Lift(A B) =Confidence (A B) / Support(B) Step 2: Generate rules Step (1): Generate candidate rules from frequent itemsets From each frequent item set L k (where k is greater than or equal to 2) to generate all possible candidate rules.

[0064] For example, for the frequent itemset {A,B,C}, the following candidate rules are generated: A B,C,B A, C, C A, B, A, B C、A,C B, B, C A Step (2): Calculate confidence For each candidate rule A B, calculate its confidence.

[0065] If the confidence is greater than or equal to a preset minimum confidence threshold, the rule is considered to be a meaningful rule.

[0066] For the frequent itemset {A,B,C}, rules A,B C, the confidence is calculated as follows: Confidence(A,B C)=Support(A,B,C) / Support(A,B) Step (3): Pruning Prune the rules whose confidence is below the threshold.

[0067] Step (4): Lift calculation (optional) For the selected association rules, calculate their lift, which represents the correlation between item sets.

[0068] A rule with a lift greater than 1 indicates that there is a positive correlation between A and B.

[0069] A rule with a lift of 1 means that A and B are independent of each other.

[0070] A rule with a lift less than 1 indicates that there is a negative correlation between A and B.

[0071] Assume that there are the following frequent item sets and support (the total number of transactions is 5) as shown in Table 2: Table 2

[0072] Generate rules from frequent itemsets: For frequent itemsets {A,B}, candidate rules: A B and B A, calculate confidence: Confidence(A B)=Support(A,B) / Support(A)=3 / 4=0.75 Confidence(B A)=Support(A,B) / Support(B)=3 / 4=0.75 For the frequent itemset {A,B,C}, candidate rules: A,B C、A,C B and B, C A, calculate confidence: Confidence(A,B C)=Support(A,B,C) / Support(A,B)=2 / 3 0.675 Confidence(A,C B)=Support(A,B,C) / Support(A,C)=2 / 3 0.675 Confidence(B,C A)=Support(A,B,C) / Support(B,C)=2 / 3 0.675 Screening and pruning: Assuming the minimum confidence threshold is 0.7, only A B and B A is a valid rule and the rest are pruned.

[0073] Through the above method, association rules between data and data with strong correlation, and between data and system behavior are obtained.

[0074] In one embodiment of the present application, based on the comprehensive feature vector, the machine learning model is trained to obtain an association analysis model, including: Divide the comprehensive feature vector into training set and test set; Determine that the machine learning model is a decision tree model, and use the training set to train the decision tree model until the training termination condition is met to obtain a trained decision tree model; Use the trained decision tree model to predict the test set and obtain the predicted behavior label; The model evaluation index is calculated according to the real behavior label and the predicted behavior label, and the parameters of the decision tree model are adjusted according to the model evaluation index to obtain the association analysis model.

[0075] In this embodiment, the steps of model training specifically include: Step 1: Data Preparation Use the data features extracted during frequent item set and association rule mining to ensure that the data features contain enough information to describe system behavior, such as system log events, traffic patterns, runtime data, user behavior, etc. Standardize or normalize the data features to ensure that the dimensions of different features are consistent.

[0076] The data features are randomly divided into training set and test set. Common division ratios are 70% training set and 30% test set, or 80% training set and 20% test set.

[0077] Step 2: Decision tree model training The decision tree algorithm is a supervised learning algorithm suitable for classification and regression tasks. It predicts data by building a tree model. Each node of the tree represents a feature, each branch represents a value of the feature, and the leaf node represents the category label.

[0078] The CART (Classification and Regression Tree) decision tree model is used to train the decision tree model using the training set. The optimal features are selected recursively for splitting until the stopping condition (such as the maximum depth or the minimum number of samples) is met. The specific steps are as follows: (1) Initialize the root node: contains all training samples.

[0079] (2) Select the optimal feature splitting point: According to a certain splitting criterion (such as Gini coefficient, information gain, chi-square test, etc.), select the optimal feature and its splitting point to divide the training set into two subsets.

[0080] (3) Recursive splitting: Repeat step (2) for each subset until the stopping condition is met.

[0081] (4) Construct a decision tree: Organize the split points and leaf nodes formed by the above process into a tree structure to form a decision tree model.

[0082] Step 3: Model Evaluation Model evaluation indicators include: accuracy, precision, recall and F1 score. Accuracy refers to the ratio of correctly classified samples to the total number of samples, Accuracy = correctly classified samples / total number of samples; Precision refers to the ratio of correctly predicted positive samples to all predicted positive samples, Precision = true positive samples / (true positive samples + false positive samples); Recall refers to the ratio of correctly predicted positive samples to all actually positive samples, Recall = true positive samples / (true positive samples + false negative samples); F1 Score refers to the harmonic average of precision and recall, F1 Score = 2 × Precision × Recall / (Precision + Recall).

[0083] Use the trained decision tree model to predict the test set to obtain the predicted behavior labels, and calculate evaluation indicators such as accuracy, precision, recall, and F1 score based on the true behavior labels and predicted behavior labels.

[0084] The model hyperparameters can be tuned based on the evaluation indicators to obtain the association analysis model. The hyperparameters of the decision tree include: maximum depth (max_depth), minimum number of sample splits (min_samples_split), minimum number of leaf node samples (min_samples_leaf), and maximum number of features (max_features).

[0085] Through the above method, an accurate association analysis model is obtained, and the trained association analysis model can be subsequently deployed to the production environment for real-time detection and classification.

[0086] It should be noted that the embodiment of the present application can also retrain the model regularly, combining the latest system data and association rules to optimize model performance.

[0087] In one embodiment of the present application, a method of acquiring system data includes: the system data includes first system data and / or second system data, and different types of system data are acquired, and the different types of system data include one or more of log data, traffic data, operation data, and user behavior data; Among them, log data includes at least one of the following: log type, timestamp, log level, event ID, event description; traffic data includes at least one of the following: IP address, port number, protocol type, packet size, timestamp; operation data includes at least one of the following: process ID, thread ID, CPU usage, memory usage, file operation; user behavior data includes at least one of the following: user ID, operation type, operation time, access path.

[0088] In this embodiment, the potential vulnerabilities of the system are comprehensively analyzed from the code level, runtime behavior level, data flow level and user behavior level. Various data collected when the system is running include but are not limited to log data, traffic data, operation data, user behavior data, etc. Log data includes log type, timestamp, log level, event ID, event description, etc., and the extracted data features include event frequency, event sequence, etc. Traffic data IP address, port number, protocol type, packet size, timestamp, etc., and the extracted data features include traffic frequency, traffic direction (inbound / outbound), traffic pattern, etc. Operation data includes process ID, thread ID, CPU usage, memory usage, file operation, etc., and the extracted data features include process behavior pattern, resource consumption pattern, etc. User behavior data includes user ID, operation type, operation time, access path, etc., and the extracted data features include user operation sequence, access frequency, etc.

[0089] It should be noted that after the above system data are collected, the system data are pre-processed, including denoising and formatting, so as to provide a high-quality data source for subsequent correlation analysis.

[0090] The embodiment of the present application, by receiving the above-mentioned different types of data, realizes multimodal data fusion, constructs a comprehensive data set, makes the data richer, improves the detection accuracy and coverage, and can obtain a more accurate model when conducting model training. When performing detection based on real-time data and multimodal association analysis technology, it can identify more complex and hidden vulnerability patterns, and has the advantages of lower false alarm rate and higher monitoring degree.

[0091] It is worth noting that the embodiments of the present application can collect and analyze users' operating habits and behavior patterns to identify abnormal user behavior and discover potential security threats in a timely manner.

[0092] In one embodiment of the present application, the method further includes: visually displaying the detection results of the system behavior so that management personnel can view and analyze the detection results.

[0093] In this embodiment, the present application provides a visualization and interactive interface, using visualization libraries such as D3.js and Plotly to build an interactive vulnerability detection result display interface, allowing users to intuitively view and analyze the detection results of system behavior.

[0094] You can also design a real-time monitoring dashboard that integrates multiple monitoring indicators and alarm functions to provide comprehensive security situation awareness capabilities.

[0095] In one embodiment of the present application, the method further includes: after detecting abnormal behavior of the system, issuing abnormal alarm information; obtaining feedback information given by the management personnel to the abnormal alarm information, the feedback information including abnormal confirmation information or abnormal false alarm information.

[0096] In this embodiment, after identifying the abnormal association relationship and determining that there is abnormal system behavior, the alarm mechanism will be triggered and abnormal alarm information will be issued. The abnormal alarm information may include the system abnormal behavior and / or the second system data related to the system abnormal behavior, notifying the management personnel to conduct timely investigation and processing, thereby ensuring the security of the system.

[0097] After knowing the abnormal alarm, the manager will give feedback. The feedback information given by the manager to the abnormal alarm information is obtained, and the feedback information includes abnormal confirmation information or abnormal false alarm information. In this way, the feedback information of the manager can be received, and subsequent processing can be performed based on the feedback information.

[0098] In one embodiment of the present application, the method further includes: if the feedback information is abnormal false alarm information, determining new association rules related to abnormal behavior of the system, and updating the association analysis model according to the system data involved in the new association rules.

[0099] In this embodiment, if the feedback information is abnormal false alarm information, new association rules related to the abnormal behavior of the system are added to the association rule library, and the association analysis model is updated according to the system data involved in the new association rule to achieve dynamic update of the association analysis model and ensure the timeliness and accuracy of the model.

[0100] In one embodiment, a reinforcement learning algorithm, such as Q-learning, Deep Q-Network, etc., can be introduced to automatically adjust the detection strategy and parameter configuration according to the detection results and environmental changes, thereby improving the system's adaptability and detection efficiency.

[0101] In one embodiment of the present application, the method further includes: if the feedback information is abnormal false alarm information, storing new association rules related to abnormal behavior of the system into an association rule library; The method also includes: verifying the validity of the association rules in the association rule base based on historical system data and preset test data.

[0102] In this embodiment, the obtained association rules are stored in an association rule library for subsequent use.

[0103] When abnormal false alarm information is determined based on user feedback, new association rules related to the abnormal behavior of the system are stored in the association rule library, that is, the association rule library is updated to improve the system's detection capabilities for new attacks and unknown vulnerabilities.

[0104] In one embodiment, the validity of the association rules in the association rule base is verified based on the historical system data and the preset test data. Specifically, the historical system data refers to the data generated and collected in a certain period of time in the past, which usually includes past network security events, attack patterns, vulnerability information, etc. When verifying and updating the association rule base, the historical system data is mainly used to: help identify new or variant attack patterns by analyzing the attack patterns and vulnerability types in the historical system data, and evaluate the accuracy and efficiency of the association rule base in dealing with historical attack patterns, so as to determine whether it needs further optimization.

[0105] Preset test data is a data set created or selected for specific testing purposes, used to verify the functionality, performance, security, etc. of software or systems. Test data containing specific attack patterns or vulnerability types can be generated by simulating real attack scenarios, or testing the boundary conditions of software or systems to discover potential vulnerabilities or errors. When verifying and updating the association rule base, preset test data is mainly used to: ensure that the association rule base can correctly identify and respond to attack patterns and vulnerability types in the preset test data, and evaluate the performance of the association rule base in handling new attack patterns and vulnerability types through testing of a large amount of preset test data.

[0106] Validating the association rule base using historical system data and preset test data can ensure that the association rule base is effective in responding to new attack patterns and vulnerability types.

[0107] In one embodiment of the present application, distributed computing and stream processing are performed, that is, distributed computing frameworks such as Apache Spark and Hadoop are used to implement parallelization and distributed processing of association analysis tasks, thereby improving the processing speed and scalability of the system.

[0108] As a specific implementation of the above-mentioned system anomaly detection method based on association analysis, the embodiment of the present application provides a system anomaly detection device based on association analysis. Figure 2 As shown, the system anomaly detection device 200 based on association analysis includes: a data acquisition module 201, an association rule generation module 202, a model training module 203 and a detection module 204.

[0109] The data acquisition module 201 is used to acquire first system data of the network system, extract features from the first system data to obtain multiple data features, and fuse the multiple data features to generate a comprehensive feature vector; An association rule generation module 202 is used to generate association rules based on the comprehensive feature vector using an association analysis algorithm, wherein the association rules are used to represent the association relationship between various data features and between data features and system behaviors; A model training module 203 is used to perform model training based on the comprehensive feature vector after determining the association relationship to obtain an association analysis model; The data acquisition module 201 is further used to acquire the second system data when the network system is running, and to construct a data feature vector based on the second system data; The detection module 204 is used to input the data feature vector into the association analysis model, identify the association relationship between the system behavior and the second system data through the association analysis model, and detect abnormal system behavior.

[0110] Furthermore, the detection module 204 is specifically configured to: If the association analysis model identifies that the score of the association relationship between the second system data and a system behavior is less than a preset threshold, it is determined that the system behavior is an abnormal system behavior; If the association analysis model identifies that the score of the association relationship between the second system data and a system behavior is greater than or equal to a preset threshold, it is determined that the system behavior is a normal system behavior.

[0111] Furthermore, the model building module is specifically used to: generate a target frequent item set according to the comprehensive feature vector; and determine an association rule based on the target frequent item set.

[0112] Furthermore, the model building module is specifically used to: generate the ith candidate item set, calculate the support of the items in the ith candidate item set, screen out the items whose support is greater than a preset support threshold to form the ith frequent item set, repeat the steps of generating candidate item sets until a frequent item set is formed until a larger candidate item set can be generated; wherein, when i=1, scan each element in the comprehensive feature vector, take one element as an item, and generate the ith candidate item set; when i is greater than 1, perform a connection operation on multiple items in the i-1th frequent item set to generate the ith candidate item set; and take the intersection of all frequent item sets as the target frequent item set.

[0113] Furthermore, the model building module is specifically used to: for any frequent item set in the target frequent item set, generate candidate rules from the frequent item set; calculate the confidence of each candidate rule, and screen out candidate rules with confidence greater than a preset confidence threshold as association rules.

[0114] Furthermore, the model building module is specifically used to: Divide the comprehensive feature vector into training set and test set; Determine that the machine learning model is a decision tree model, and use the training set to train the decision tree model until the training termination condition is met to obtain a trained decision tree model; Use the trained decision tree model to predict the test set and obtain the predicted behavior label; The model evaluation index is calculated according to the real behavior label and the predicted behavior label, and the parameters of the decision tree model are adjusted according to the model evaluation index to obtain the association analysis model.

[0115] Furthermore, the methods for obtaining system data include: The system data includes the first system data and / or the second system data, and different types of system data are obtained. The different types of system data include one or more of log data, traffic data, operation data, and user behavior data; wherein the log data includes at least one of the following: log type, timestamp, log level, event ID, and event description; the traffic data includes at least one of the following: IP address, port number, protocol type, data packet size, and timestamp; the operation data includes at least one of the following: process ID, thread ID, CPU usage, memory usage, and file operation; the user behavior data includes at least one of the following: user ID, operation type, operation time, and access path.

[0116] Furthermore, the device also includes: a display module, which is used to visually display the detection results of the system behavior so that management personnel can view and analyze the detection results.

[0117] Furthermore, the device also includes: an alarm module, which is used to issue abnormal alarm information after detecting abnormal behavior of the system; The data acquisition module 201 is further used to obtain feedback information given by management personnel to abnormal alarm information, wherein the feedback information includes abnormal confirmation information or abnormal false alarm information.

[0118] Furthermore, the device also includes: an updating module, which is used to determine new association rules related to abnormal behavior of the system if the feedback information is abnormal false alarm information, and update the association analysis model according to the system data involved in the new association rules.

[0119] Further, the device also includes: a storage module, which is used to store new association rules related to abnormal behavior of the system into an association rule library if the feedback information is abnormal false alarm information; The device also includes: a verification module, which is used to verify the validity of the association rules in the association rule base based on historical system data and preset test data.

[0120] The system anomaly detection device 200 based on correlation analysis in the embodiment of the present application can be a computer device, or a component in a computer device, such as an integrated circuit or a chip. Figure 1To avoid repetition, the various processes implemented in the embodiment of the system anomaly detection method based on association analysis are not described here.

[0121] The present application also provides a computer device, such as Figure 3 As shown, the computer device 300 includes a processor 301 and a memory 302. The memory 302 stores programs or instructions that can be executed on the processor 301. When the program or instruction is executed by the processor 301, the various steps of the above-mentioned system anomaly detection method based on association analysis are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0122] The memory 302 can be used to store software programs and various data. The memory 302 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 302 may include a volatile memory or a non-volatile memory, or the memory 302 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 302 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0123] The processor 301 may include one or more processing units; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 301.

[0124] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned system anomaly detection method based on association analysis is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0125] An embodiment of the present application also provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned system anomaly detection method embodiment based on association analysis, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0126] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0127] An embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned system anomaly detection method embodiment based on association analysis, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0128] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0129] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A system anomaly detection method based on association analysis, characterized in that: include: Acquire first system data of the network system, perform feature extraction on the first system data to obtain multiple data features, and perform feature fusion on the multiple data features to generate a comprehensive feature vector; Generate association rules based on the comprehensive feature vector using an association analysis algorithm, wherein the association rules are used to represent the association relationship between the data features and between the data features and system behavior; After determining the association relationship, performing model training based on the comprehensive feature vector to obtain an association analysis model; Monitoring the operation of the network system, acquiring second system data of the network system in real time, and constructing a data feature vector based on the second system data; The data feature vector is input into the association analysis model, and the association relationship between the system behavior and the second system data is identified through the association analysis model to detect abnormal system behavior.

2. The method according to claim 1, characterized in that The abnormal behavior of the detection system includes: If the association analysis model identifies that the score of the association relationship between the second system data and a system behavior is less than a preset threshold, it is determined that the system behavior is an abnormal system behavior; If the association analysis model identifies that the score of the association relationship between the second system data and a system behavior is greater than or equal to a preset threshold, it is determined that the system behavior is a normal system behavior.

3. The method according to claim 1, characterized in that The generating of association rules based on the comprehensive feature vector comprises: Generate a target frequent item set according to the comprehensive feature vector; The association rule is determined based on the target frequent item set.

4. The method according to claim 3, characterized in that The step of generating a target frequent item set according to the comprehensive feature vector comprises: Generate an i-th candidate item set, calculate the support of the items in the i-th candidate item set, screen out the items whose support is greater than a preset support threshold, form an i-th frequent item set, repeat the steps of generating candidate item sets until a frequent item set is formed, until a larger candidate item set cannot be generated; wherein, when i=1, scan each element in the comprehensive feature vector, take one of the elements as an item, and generate the i-th candidate item set; when i is greater than 1, perform a connection operation on multiple items in the i-1th frequent item set to generate the i-th candidate item set; The intersection of all frequent item sets is taken as the target frequent item set.

5. The method according to claim 3, characterized in that: Determining the association rule based on the target frequent item set includes: For any frequent item set in the target frequent item set, generating a candidate rule from the frequent item set; The confidence of each candidate rule is calculated, and the candidate rules whose confidence is greater than a preset confidence threshold are screened out as the association rules.

6. The method according to claim 2, characterized in that The step of training the machine learning model based on the comprehensive feature vector to obtain the association analysis model includes: Dividing the comprehensive feature vector into a training set and a test set; Determining that the machine learning model is a decision tree model, and training the decision tree model using the training set until a training termination condition is met, thereby obtaining the trained decision tree model; Predict the test set using the trained decision tree model to obtain a predicted behavior label; A model evaluation index is calculated according to the real behavior label and the predicted behavior label, and parameters of the decision tree model are adjusted according to the model evaluation index to obtain the association analysis model.

7. The method according to claim 2, characterized in that Ways to obtain system data include: The system data includes the first system data and / or the second system data, and different types of system data are obtained. The different types of system data include one or more of log data, traffic data, operation data, and user behavior data; Among them, the log data includes at least one of the following: log type, timestamp, log level, event ID, event description; the traffic data includes at least one of the following: IP address, port number, protocol type, data packet size, timestamp; the operation data includes at least one of the following: process ID, thread ID, CPU usage, memory usage, file operation; the user behavior data includes at least one of the following: user ID, operation type, operation time, access path.

8. The method according to claim 1, characterized in that The method further comprises: The detection results of system behavior are displayed visually for management personnel to view and analyze the detection results.

9. The method according to claim 1, characterized in that: The method further comprises: After detecting abnormal behavior of the system, issuing abnormal alarm information; Acquire feedback information given by the management personnel to the abnormal alarm information, wherein the feedback information includes abnormal confirmation information or abnormal false alarm information.

10. The method according to claim 9, characterized in that The method further comprises: If the feedback information is abnormal false alarm information, a new association rule related to the abnormal behavior of the system is determined, and the association analysis model is updated according to the system data involved in the new association rule.

11. The method according to claim 9, characterized in that The method further comprises: If the feedback information is abnormal false alarm information, then storing the new association rules related to the abnormal behavior of the system into the association rule library; The method further comprises: Based on historical system data and preset test data, the association rules in the association rule base are verified for validity.

12. A system anomaly detection device based on correlation analysis, characterized in that: include: A data acquisition module, used for acquiring first system data of a network system, performing feature extraction on the first system data to obtain a plurality of data features, and performing feature fusion on the plurality of data features to generate a comprehensive feature vector; An association rule generation module, used to generate association rules based on the comprehensive feature vector using an association analysis algorithm, wherein the association rules are used to represent the association relationship between the data features and between the data features and system behavior; A model training module, used to perform model training based on the comprehensive feature vector after determining the association relationship, so as to obtain an association analysis model; The data acquisition module is further used to monitor the operation of the network system, acquire the second system data of the network system in real time, and construct a data feature vector based on the second system data; The detection module is used to input the data feature vector into the association analysis model, identify the association relationship between the system behavior and the second system data through the association analysis model, and detect abnormal system behavior.

13. A computer device, characterized in that: It includes a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the system anomaly detection method based on association analysis as described in any one of claims 1 to 11 are implemented.

14. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the system anomaly detection method based on association analysis as described in any one of claims 1 to 11 are implemented.

Images