Abnormal behavior detection method and device based on large model and log big data

Abstract

The invention discloses an abnormal behavior detection method and device based on a large model and log big data, and relates to the technical field of abnormal detection. The method comprises the steps of processing target log data by utilizing a pre-training large model to obtain a corresponding target cue word template, and performing abnormal behavior detection on a target information system corresponding to the target log data by utilizing the pre-training large model and the target cue word template. Therefore, according to the scheme, the generalization ability of the pre-training large model can be fully utilized during abnormal behavior detection, so that abnormal behavior detection can be carried out in various different scenes, and higher universality is achieved.

Description

Technical Field

[0001] The present application relates to the field of anomaly detection technology, and in particular to a method and device for detecting abnormal behavior based on large models and log big data. Background Art

[0002] As digital transformation deepens, businesses and organizations increasingly rely on complex information systems to support daily operations, increasing the complexity of data security and compliance management. To improve data security, it's necessary to detect whether users of information systems are engaging in unusual or irregular behavior and take timely countermeasures.

[0003] Currently, there are mainly the following methods for detecting abnormal and illegal data operations in the field of information security.

[0004] The first category is traditional rule-based security approaches, which rely on a set of detailed, pre-defined rules and a database of known attack patterns to identify threats. For example, an intrusion detection system (IDS) monitors network traffic and system events in real time, compares this information with a predefined database of malicious behavior signatures, and identifies and flags anomalous behavior based on the comparison results. Security information and event management (SIEM) tools go a step further, integrating IDS functionality while also aggregating, correlating, and analyzing log information from various sources. Using rule matching, they identify potential security incidents, enabling rapid response by operations and maintenance personnel.

[0005] The second category involves methods based on statistics and machine learning. These methods utilize mathematical statistical principles and machine learning algorithms to analyze the characteristics and patterns of historical data and identify anomalous behavior based on the analysis results. Statistical methods can identify anomalous behavior by setting thresholds or applying statistical tests. Supervised learning methods (such as support vector machines and decision trees) learn the corresponding characteristics and patterns from labeled datasets to identify anomalous behavior. Unsupervised learning methods (such as cluster analysis and autoencoders) can autonomously discover structures and patterns in data and are therefore suitable for identifying unknown anomalous behavior. These models can continuously improve their detection accuracy and robustness as training data becomes more abundant and algorithms are optimized.

[0006] The third category is user and entity behavior analytics (UEBA). This approach collects and analyzes information such as user login records, resource access frequency, and operational sequences to establish a baseline of normal behavior for each user or entity. It then identifies any deviations from this baseline as anomalous behavior. This approach is particularly effective in detecting anomalous behavior within information systems and long-term, planned anomalous behavior.

[0007] Each of the above methods is effective in specific scenarios, but has certain limitations. When used to detect abnormal behavior in various scenarios, their accuracy is low and their versatility is poor. It is difficult to provide comprehensive, timely and accurate anomaly detection capabilities. Summary of the Invention

[0008] In order to improve the versatility of abnormal behavior detection technology, this application discloses the following technical solutions:

[0009] The first aspect of the present application provides a method for detecting abnormal behavior based on a large model and log big data, comprising:

[0010] Get target log data;

[0011] Process the target log data according to the pre-trained large model to obtain a target prompt word template;

[0012] Abnormal behavior detection is performed on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template.

[0013] Optionally, obtaining target log data includes:

[0014] Obtain original user behavior logs of multiple users;

[0015] Preprocessing the original user behavior log to obtain a preprocessed user behavior log, wherein the preprocessing includes at least one of data standardization, data cleaning, and correlation aggregation;

[0016] Feature extraction is performed on the preprocessed user behavior log to obtain target log data.

[0017] Optionally, extracting features from the preprocessed user behavior log to obtain target log data includes:

[0018] Converting the pre-processed user behavior log corresponding to each user into time series data based on a target time window;

[0019] Analyze the time series data of each user to obtain a user behavior baseline corresponding to each user;

[0020] Comparing the user behavior baselines of different users to obtain user comparison features;

[0021] Comparing the user behavior baselines of different categories of users to obtain user group comparison features;

[0022] For each of the users, extracting discrete behavior features and continuous behavior features from the user's time series data according to the user behavior baseline of the user to obtain the discrete behavior features and continuous behavior features of the user;

[0023] The user comparison feature, the user group comparison feature, the discrete behavior feature and the continuous behavior feature of the user constitute the target log data.

[0024] Optionally, converting the pre-processed user behavior log corresponding to each user into time series data based on the target time window includes:

[0025] For each of the users, the pre-processed user behavior log corresponding to the user is converted based on multiple target time windows of different lengths to obtain multiple time series data corresponding to the user, each of which is converted based on one of the target time windows.

[0026] Optionally, before performing abnormal behavior detection on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template, the method further includes:

[0027] Performing knowledge injection into the pre-trained large model according to the positive samples and the negative samples to obtain a large model after knowledge injection;

[0028] The detecting abnormal behavior of the target information system corresponding to the target log data by using the pre-trained large model and the target prompt word template includes:

[0029] Abnormal behavior detection is performed on the target information system corresponding to the target log data using the large model after knowledge injection and the target prompt word template.

[0030] Optionally, processing the target log data according to the pre-trained large model to obtain a target prompt word template includes:

[0031] Process the target log data according to the pre-trained large model to obtain an initial prompt word template;

[0032] Determining an evaluation index of the initial prompt word template according to the target log data;

[0033] If the evaluation index does not meet the target condition, updating the initial prompt word template according to the pre-trained large model and the target log data;

[0034] Return to the step of determining the evaluation index of the initial prompt word template based on the target log data until the initial prompt word template whose evaluation index meets the target condition is obtained, wherein the initial prompt word template whose evaluation index meets the target condition is used as the target prompt word template.

[0035] Optionally, the evaluation indicator includes at least one of precision, recall and F1 score.

[0036] Optionally, updating the initial prompt word template according to the pre-trained large model and the target log data includes at least one of the following:

[0037] Adjusting the time range corresponding to the initial prompt word template according to the pre-trained large model and the target log data;

[0038] Adding context information to the initial prompt word template according to the pre-trained large model and the target log data;

[0039] The keywords contained in the initial prompt word template are adjusted according to the pre-trained large model and the target log data.

[0040] Optionally, also include:

[0041] The target prompt word template is updated regularly according to a preset update cycle.

[0042] Optionally, the performing abnormal behavior detection on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template includes:

[0043] Obtaining log data to be detected generated by the target information system;

[0044] Extracting prompt information from the log data to be detected;

[0045] Mapping the prompt information to the target prompt word template to obtain the target prompt word;

[0046] The target prompt word and the log data to be detected are input into the pre-trained large model to detect whether there is abnormal behavior in the log data to be detected.

[0047] The second aspect of the present application provides an abnormal behavior detection device based on a large model and log big data, comprising:

[0048] An acquisition unit, used for acquiring target log data;

[0049] A processing unit, configured to process the target log data according to a pre-trained large model to obtain a target prompt word template;

[0050] The detection unit is used to use the pre-trained large model and the target prompt word template to perform abnormal behavior detection on the target information system corresponding to the target log data.

[0051] The beneficial effect of this solution lies in: it uses a pre-trained large model to process target log data, obtains the corresponding target prompt word template, and then uses the pre-trained large model and the target prompt word template to detect abnormal behavior in the target information system corresponding to the target log data. As a result, this solution can fully utilize the generalization capabilities of the pre-trained large model when detecting abnormal behavior, making it capable of detecting abnormal behavior in a variety of different scenarios and having greater versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0053] Figure 1 This is a flow chart of an abnormal behavior detection method based on a large model and log big data provided by an embodiment of the present application;

[0054] Figure 2 This is a flowchart of obtaining target log data provided by an embodiment of the present application;

[0055] Figure 3 This is a flowchart of another abnormal behavior detection method based on a large model and log big data provided by an embodiment of the present application;

[0056] Figure 4 This is a flow chart of a method for obtaining a target log template provided by an embodiment of the present application;

[0057] Figure 5 This is a structural diagram of an abnormal behavior detection device based on a large model and log big data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0059] This embodiment provides a method for detecting abnormal behavior based on a large model and log data. Figure 1 , is a flowchart of the method, which may include the following steps.

[0060] S101, obtaining target log data.

[0061] In step S101, for any target information system that needs to be subjected to abnormal behavior detection, original user behavior logs of the information system in a recent period of time may be collected, and target log data may be obtained based on these original user behavior logs.

[0062] The specific collection period of original user behavior logs can be set as needed without any limit. For example, you can collect original user behavior logs within the last month, the last 80 days, the last week, etc.

[0063] S102: Process the target log data according to the pre-trained large model to obtain a target prompt word template.

[0064] The pre-trained large model used in this embodiment can be any open source large language model without limitation.

[0065] S103: Detect abnormal behavior of the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template.

[0066] In step S103, the method for obtaining target log data can be used to obtain the log data to be detected that is different from the target log data and generated by the target information system. Then, the pre-trained large model and the target prompt word template are used to process the log data to be detected to determine whether there is abnormal behavior in the log data to be detected.

[0067] Exemplarily, the target log data can be obtained based on the original user behavior logs of the target information system in the past week, and the log data to be detected can be obtained based on the original user behavior logs generated by the target information system in other time periods. For example, in the next three months, every day after a day, the log data to be detected for that day can be obtained based on the original user behavior logs of that day, and then the pre-trained large model and the target prompt word template can be used to detect whether there is any abnormal behavior in the log data to be detected for that day.

[0068] The method for obtaining the log data to be detected based on the original user behavior log is the same as the method for obtaining the target log data based on the original user behavior log, and will not be described in detail.

[0069] In this embodiment, the pre-trained large model can use unsupervised learning methods to identify abnormal behavior. Specifically, through unsupervised learning methods, the pre-trained large model does not rely on labeled data to learn the difference between normal and abnormal behavior. Instead, it can observe the distribution pattern of the data to discover instances that deviate from normal behavior and identify these instances as abnormal behavior. The use of unsupervised learning methods can enable the pre-trained large model to still have the ability to identify various abnormal behaviors even in small sample learning scenarios.

[0070] The beneficial effects of this embodiment are:

[0071] This embodiment combines a large pre-trained model with massive amounts of log data, leveraging its generalization and ability to learn complex patterns. This model, combined with analysis of log data, derives appropriate target prompt word templates. The pre-trained model and target prompt word templates are then used to achieve a deep understanding of the log data and accurately identify abnormal behavior. This technology not only rapidly identifies various types of abnormal behavior in various scenarios, reducing false positives and false negatives, but also maintains efficient operation on large datasets, providing a more comprehensive, timely, and accurate data protection mechanism to effectively combat external attacks and internal abuse.

[0072] In addition, during the training of large models, unsupervised learning is used, which leverages unlabeled data to enhance the model's learning capabilities. This approach helps the model learn patterns of normal behavior from large amounts of unlabeled log data, thereby assisting the model in identifying abnormal behavior.

[0073] In some optional embodiments, see Figure 2 , the target log data can be obtained based on the original user behavior log as follows:

[0074] Obtain original user behavior logs of multiple users;

[0075] Preprocessing the original user behavior log to obtain a preprocessed user behavior log, wherein the preprocessing includes at least one of data standardization, data cleaning, and correlation aggregation;

[0076] Perform feature extraction on the preprocessed user behavior log to obtain target log data.

[0077] like Figure 2 As shown, the original user behavior logs may include but are not limited to virtual private network (VPN) logs, office automation (OA) logs, traffic logs, and terminal logs generated when users access the target information system. These original user behavior logs may contain key information such as the access time, access object, access permission, user identity, and access results of each user each time they access the target system within the corresponding time period.

[0078] The original user behavior logs can be obtained from the database or file server of the target information system.

[0079] In the preprocessing step, the original behavior logs can be first standardized to unify the field naming, data type, and encoding format of the original behavior logs to ensure consistency and comparability of the original behavior logs obtained from different sources.

[0080] The standardized logs can then be cleaned to remove invalid data and fill in missing values to ensure data quality. Specific cleaning methods can be found in related existing technologies and will not be detailed here.

[0081] Finally, the standardized and cleaned logs can be correlated and aggregated to obtain the target log data. This correlation and aggregation can include, for each user of the target information system, filtering out all logs related to the user's access behavior from the standardized and cleaned logs, and forming these logs as the log set corresponding to the user. For each user, the log set corresponding to the user is the preprocessed user behavior log obtained after preprocessing in this embodiment.

[0082] In this embodiment, feature extraction is performed on the pre-processed user behavior log to obtain target log data in the following manner:

[0083] A1, converts the pre-processed user behavior log corresponding to each user into time series data based on the target time window;

[0084] A2: Analyze each user's time series data to obtain the corresponding user behavior baseline;

[0085] A3, compare the user behavior baselines of different users to obtain user comparison features;

[0086] A4, compare the user behavior baselines of different categories of users to obtain user group comparison features;

[0087] A5. For each user, extract discrete behavior features and continuous behavior features from the user's time series data based on the user's user behavior baseline to obtain the user's discrete behavior features and continuous behavior features;

[0088] Among them, user comparison features, user group comparison features, user discrete behavior features and continuous behavior features constitute the target log data.

[0089] The target time window can be of any length, such as 1 hour, 30 minutes, 2 hours, 24 hours, etc., without limitation. In step A1, the time period covered by the original user behavior log can be divided into multiple sub-time periods according to the target time window. Then, for each user, the log data generated in each sub-time period is segmented from the user's pre-processed user behavior log. The log data of multiple sub-time periods constitute the user's time series data. In other words, the user's pre-processed user behavior log can be divided into multiple data according to the target time window, and these data constitute the user's time series data.

[0090] As an example, assuming that the target time window is 1 hour, and the original user behavior log includes the behavior log within the last week, then in step A1, the behavior log of the first hour in the last week can be obtained from the preprocessed user behavior log of user A as one data, the behavior log of the second hour as the second data, and so on, until the last hour in the last week (i.e., the last hour). The multiple data obtained in this way constitute a time series data of user A.

[0091] In some optional embodiments, in order to conduct more comprehensive detection, the time series data obtained in step A1 may include three types of data: original sequence data, content sequence data, and event sequence data. Each data in the original sequence data may directly include data in the corresponding sub-time period in the pre-processed user behavior log, the content sequence data may include data related to access content in the corresponding sub-time period in the pre-processed user behavior log, and the event sequence data may include data related to access events in the corresponding sub-time period in the pre-processed user behavior log.

[0092] Data related to accessed content may include the name of the accessed file, and data related to access events may include user access behaviors, such as downloading information, modifying information, deleting information, adding information, and the like.

[0093] In combination with the above example, the pre-processed user behavior log of user A can be divided by hour, and the multiple data obtained constitute the original sequence data; the pre-processed user behavior log of user A and the data related to the accessed content can be divided by hour, and the multiple data obtained constitute the content sequence data; the pre-processed user behavior log of user A and the data related to the access event can be divided by hour, and the multiple data obtained constitute the event sequence data.

[0094] In some embodiments, multiple target time windows of different lengths may be set. Correspondingly, the above-mentioned step of converting to obtain time series data may include:

[0095] For each user, the pre-processed user behavior log corresponding to the user is converted based on multiple target time windows of different lengths to obtain multiple time series data corresponding to the user. Each time series data is converted based on a target time window.

[0096] As some examples, a target time window of 1 hour, a target time window of 12 hours, and a target time window of 30 minutes can be set. Based on the target time window of 1 hour, the preprocessed user behavior log of user A is processed in the aforementioned manner to obtain the time series data of user A in units of 1 hour; based on the target time window of 12 hours, the preprocessed user behavior log of user A is processed in the aforementioned manner to obtain the time series data of user A in units of 12 hours; based on the target time window of 30 minutes, the preprocessed user behavior log of user A is processed in the aforementioned manner to obtain the time series data of user A in units of 30 minutes.

[0097] Using multiple target time windows of different lengths allows the pre-trained large model to detect abnormal behavior. This allows the pre-trained large model to focus on the user's short-term behavior through a shorter target time window, while also enhancing contextual awareness through a longer target time window, thereby better understanding the context in which the behavior occurs. This is crucial for identifying behaviors that may appear normal in the short term but exhibit abnormalities over a longer time frame.

[0098] In step A2, the pre-trained large model can be used to analyze the time series data of each user to obtain a user behavior baseline for each user. When the user's time series data includes multiple time series data, all of the user's time series data can be input into the pre-trained large model to obtain the user behavior baseline for the user. For example, the user's original sequence data, content sequence data, and event sequence data can all be input into the pre-trained large model to obtain the user behavior baseline, or multiple time series data corresponding to multiple different target time windows can all be input into the pre-trained large model to obtain the user behavior baseline for the user.

[0099] The method of using a pre-trained large model to analyze the user's time series data to obtain the user behavior baseline can be found in the relevant existing technology and will not be described in detail.

[0100] A user's user behavior baseline can characterize the normal access behavior performed by the user. For example, a user's user behavior baseline may include accessing the system with group administrator privileges, accessing files A and B on the system, accessing the system between 12:00 and 15:00 every day, and adding or modifying data to the accessed files during each access.

[0101] In step A3, for each user, the user behavior baseline of the user may be compared with the user behavior baselines of multiple other users to obtain the user comparison features of the user.

[0102] Exemplarily, the user behavior baseline of user A is compared with the user behavior baseline of user B, and the user behavior baseline of user A is compared with the user behavior baseline of user C, and the user comparison feature of user A is obtained based on the comparison results.

[0103] For each user, several users in the target information system who interact with the user most frequently may be used as comparison objects.

[0104] Combined with the above example, it is found through statistics that for user A, users B and C are the two users with the highest interaction frequency with user A among all users, so a comparison is performed in the above manner to determine the user comparison feature of user A.

[0105] Interactions between users may include but are not limited to sending messages to each other, transferring files to each other, etc.

[0106] User comparison features may include the similarity between the user's user behavior baseline and the user behavior baselines of other users used for comparison. In the aforementioned example, user comparison features for user A may include the similarity between the behavior baselines of user A and user B, and the similarity between the behavior baselines of user A and user C.

[0107] The principle of analyzing the similarity of user behavior baselines of different users is:

[0108] Frequently interacting users generally have similar behaviors. For example, if two users interact frequently, there's a high probability that their system access times overlap. Therefore, if the behavior baselines of two frequently interacting users differ significantly, it can be assumed that one of these users' baselines may contain abnormal behavior. Therefore, incorporating the aforementioned user comparison features into the target log data can help detect such abnormal behavior and improve the accuracy of detection results.

[0109] In step A4, the user categories can be divided according to the groups to which the users belong, that is, users in each group belong to the same category, and users in different groups belong to different categories. The group to which each user belongs can be pre-set by the system administrator of the target information system.

[0110] For each user, the group to which the user belongs can be determined, and then the group behavior baseline of the group is compared with the group behavior baseline of each other group, and the user group comparison feature of the user is obtained based on the comparison result.

[0111] For example, user A belongs to group 1. There are 10 groups in the system. The group behavior baseline of group 1 can be compared with the group behavior baselines of the other 9 groups to obtain the user group comparison feature of user A.

[0112] The group behavior baseline of each group may be a set of user behavior baselines of all users in the group.

[0113] User group comparison features may include similarity of group behavior baselines. Based on the above example, user group comparison features for user A may include the following similarities:

[0114] Similarity of group behavior baselines between Group 1 and Group 2;

[0115] Similarity of group behavior baselines between Group 1 and Group 3;

[0116] Similarity of group behavior baselines between Group 1 and Group 4;

[0117] Similarity of group behavior baselines between Group 1 and Group 5;

[0118] Similarity of group behavior baselines between Group 1 and Group 6;

[0119] Similarity of group behavior baselines between Group 1 and Group 7;

[0120] Similarity of group behavior baselines between Group 1 and Group 8;

[0121] Similarity of group behavior baselines between group 1 and group 9.

[0122] The principle of analyzing user group comparison characteristics is:

[0123] Different groups generally correspond to different functional departments in an enterprise, so the behavioral baselines of users in different groups should generally be quite different. If the group behavioral baselines of two groups are relatively similar, it means that the user behavior baselines of one or more users in the group may contain abnormal behaviors. Therefore, introducing the above-mentioned user group comparison features in the target log data can help detect such abnormal behaviors and improve the accuracy of the detection results.

[0124] In step A5, for each user, the user behavior baseline and the user's time series data can be compared to extract the user's discrete behavior features and continuous behavior features within each target time window. The discrete behavior features of each target time window can include the number of logins, the number of file accesses, etc. The continuous behavior features of each target time window can include the average login duration, the average file access interval, etc.

[0125] Introducing discrete behavior features and continuous behavior features into target log data is conducive to comprehensively reflecting the user's behavioral characteristics at different time scales.

[0126] For example, if the target time window is 1 hour, the number of logins and file accesses of User A in each hour can be extracted as discrete behavioral features of User A, and the average login duration and average file access interval of User A in each hour can be extracted as continuous behavioral features of User A.

[0127] The above steps of extracting discrete behavior features and continuous behavior features can be performed using a pre-trained large model.

[0128] For each user, the target log data of the user may include the user comparison features, user group comparison features, discrete behavior features, and continuous behavior features of the user obtained according to the above steps A1 to A5.

[0129] See Figure 4 The method of obtaining the target prompt word template in step S102 may include the following steps:

[0130] B1, process the target log data according to the pre-trained large model to obtain the initial prompt word template;

[0131] B2, determine the evaluation index of the initial prompt word template based on the target log data;

[0132] B3, when the evaluation index does not meet the target conditions, update the initial prompt word template according to the pre-trained large model and target log data;

[0133] B4, returning to the step of determining the evaluation index of the initial prompt word template according to the target log data, until an initial prompt word template whose evaluation index meets the target condition is obtained, wherein the initial prompt word template whose evaluation index meets the target condition is used as the target prompt word template.

[0134] In step B1, the target log data may be divided into two parts, one as a training data set and the other as a test data set. The training data set is then processed using a pre-trained large model to obtain an initial prompt word template.

[0135] For example, target log data of 100 users in a target information system is obtained in S101 . In step B1 , the target log data of 50 users may be used as a training data set, and the target log data of the other 50 users may be used as a test data set.

[0136] For the target log data of each user in the test data set, a labeling result of each target log data in the test data set can be determined by manual labeling, and the labeling result indicates whether there is abnormal behavior in the corresponding target log data.

[0137] Optionally, when processing a training dataset with a pre-trained large model, a default prompt word can be generated based on the training dataset and a pre-designed default prompt word template, and then the default prompt word and the training dataset are input into the pre-trained large model to generate an initial prompt word template.

[0138] As some examples, the generated initial prompt word template may be "detect abnormal login behavior of user X within time period Y".

[0139] The default prompt word templates can be designed by relevant users, and these templates can contain the key information and instructions required for the model to perform tasks.

[0140] The method for generating default prompt words based on the training data set and the pre-designed default prompt word template can be referred to the method for generating target prompt words based on the log data to be detected and the target prompt word template, and will not be described in detail.

[0141] In step B2, a test prompt word can be generated based on the currently obtained initial prompt word template and the test data set. The method of generating a test prompt word based on the test data set and the pre-designed initial prompt word template can be referred to the method of generating a target prompt word based on the log data to be detected and the target prompt word template, and will not be repeated here.

[0142] The test data set and test prompt words can then be input into the pre-trained large model to use the pre-trained large model to detect abnormal behavior of the target log data in the test data set, and obtain the detection results corresponding to each target log data in the test data set. The detection results indicate whether there is abnormal behavior in the target log data.

[0143] After obtaining the detection results, the detection results of each target log data can be compared with the corresponding annotation results, and the evaluation index at this time can be calculated based on the comparison results.

[0144] In some embodiments, the evaluation metric may include at least one of precision, recall, and F1 score.

[0145] Accuracy refers to the ratio of the number of samples correctly classified by the model to the total number of samples, which can be calculated using the following formula (1): Accuracy = (TP + TN) / (TP + TN + FP + FN), (1).

[0146] Among them, TP (True Positives) is the number of true positive classes correctly predicted as positive classes, TN (True Negatives) is the number of true negative classes correctly predicted as negative classes, FP (False Positives) is the number of false positive classes incorrectly predicted as positive classes, and FN (False Negatives) is the number of false negative classes incorrectly predicted as negative classes.

[0147] In this embodiment, TP can be the number of target log data in the test data set whose labeling result is no abnormal behavior and whose detection result is also no abnormal behavior; TN can be the number of target log data in the test data set whose labeling result is abnormal behavior and whose detection result is also abnormal behavior; FP can be the number of target log data in the test data set whose labeling result is abnormal behavior and whose detection result is no abnormal behavior; FN can be the number of target log data in the test data set whose labeling result is no abnormal behavior and whose detection result is abnormal behavior.

[0148] In the task of detecting abnormal and illegal operations in data, accuracy reflects the proportion of normal and abnormal operations that the model correctly identifies among all judgments. A high accuracy means that the classification of operations (normal or abnormal) is generally correct. However, in cases of data imbalance (for example, where normal operations far outnumber abnormal and illegal operations), relying solely on accuracy may mask an inadequate assessment of anomaly detection capabilities.

[0149] Recall measures the proportion of positive samples identified by the classifier that are actually positive, that is, the proportion of samples that are correctly identified among all samples that are actually positive. A higher recall means that the classifier misses fewer positive samples. Recall can be calculated using the following formula (2): Recall = TP / (TP + FN), (2).

[0150] In the task of detecting data anomaly and illegal operations, recall measures the proportion of all actual anomaly and illegal operations identified by the system. Considering that missed anomaly and illegal operations may lead to serious security risks or compliance issues, a high recall rate means that the system can effectively reduce the number of missed anomalies.

[0151] The F1 score is the harmonic mean of precision and recall, intended to provide a comprehensive metric that balances precision and recall. When both precision and recall are important, the F1 score is an alternative metric. The F1 score can be calculated using formula (3): F1 score = (2*TP) / (2*TP + FP + FN), (3).

[0152] In B3, the target condition may include an evaluation metric greater than a corresponding evaluation metric threshold. For example, the accuracy rate may be greater than an accuracy threshold, the recall rate may be greater than a recall threshold, and the F1 score may be greater than an F1 score threshold. The above thresholds can be set as needed and are not limited.

[0153] In step B3, the target log data included in the training data set may be input into the pre-trained large model, so that the pre-trained large model updates the initial prompt word template in at least one of the following ways:

[0154] Adjust the time range corresponding to the initial prompt word template based on the pre-trained large model and target log data;

[0155] Add contextual information to the initial prompt word template based on the pre-trained large model and target log data;

[0156] Adjust the keywords contained in the initial prompt word template based on the pre-trained large model and target log data.

[0157] In the above example, assuming that the generated initial prompt word template is "Detect abnormal login behavior of user X in time period Y", in step B3, "time period Y" can be adjusted to "time period Z" based on the pre-trained large model and the target log data contained in the training dataset, and a new initial prompt word template "Detect abnormal login behavior of user X in time period Z" is obtained;

[0158] Alternatively, based on the pre-trained large model and the target log data contained in the training dataset, the keyword "login" can be adjusted to "download" to obtain a new initial prompt word template "Detect abnormal download behavior of user X in time period Y";

[0159] Alternatively, based on the pre-trained large model and the target log data contained in the training dataset, the context information "continuous logins with a login interval of less than 5 minutes" can be added to obtain a new initial prompt word template "Detect abnormal login behavior of user X who logs in continuously within time period Y with a login interval of less than 5 minutes."

[0160] In step B4, after obtaining a new initial prompt word template, step B2 can be executed again based on the new initial prompt word template, and the loop can be iterated several times until the evaluation index obtained based on a certain initial prompt word template meets the target condition. At this time, the initial prompt word template obtained this time can be output as the target prompt word template.

[0161] Obtaining the target cue word template in this way ensures that the pre-trained model accurately understands abnormal behavior in the data and flexibly adjusts its detection strategy based on different data characteristics. This helps improve the accuracy and efficiency of anomaly detection and reduce false positives and missed negatives.

[0162] In some embodiments, see Figure 2 Before using the pre-trained large model and target prompt word template to detect abnormal behavior of the target information system corresponding to the target log data, the following steps can also be performed:

[0163] Inject knowledge into the pre-trained large model based on positive samples and negative samples to obtain a large model after knowledge injection;

[0164] Correspondingly, using the pre-trained large model and the target prompt word template to detect abnormal behavior of the target information system corresponding to the target log data may include:

[0165] The large model after knowledge injection and the target prompt word template are used to detect abnormal behavior of the target information system corresponding to the target log data.

[0166] The method of using a large model after knowledge injection to detect abnormal behavior is the same as the method of using a pre-trained large model to detect abnormal behavior, so it will not be repeated here.

[0167] Positive samples can include target log data without abnormal behavior, and negative samples can include target log data with abnormal behavior. When performing knowledge injection, the above samples and corresponding sample prompt words can be input into the pre-trained large model together, thereby injecting knowledge into the large model and realizing effective data abnormal operation detection.

[0168] The sample prompt word corresponding to the sample is used to indicate whether there is abnormal behavior in the sample. For example, the sample prompt word of a positive sample can indicate that there is no abnormal behavior in the sample, and the sample prompt word of a negative sample can indicate that there is abnormal behavior in the sample.

[0169] The specific method of knowledge injection can be found in the relevant existing technology and will not be described in detail.

[0170] The large model is pre-trained on a wide range of massive corpora and can understand and learn complex language patterns and behavioral characteristics. Using the above-mentioned positive and negative samples for knowledge injection can enhance the generalization ability of the pre-trained large model when processing the target log data corresponding to the target information system, so that the model after knowledge injection can identify and adapt to unprecedented threat types.

[0171] In some optional embodiments, the method of this embodiment may further include: regularly updating the target prompt word template according to a preset update cycle.

[0172] The update cycle can be set as needed, such as every month, every week, etc.

[0173] In this embodiment, target log data may be obtained in the manner of step S101 every time an update cycle passes, and the target log data may be processed in the manner of step S102 to obtain an updated target prompt word template.

[0174] Each time the target prompt word template is updated, the original target prompt word template before the update can be used as the initial prompt word template when step B2 is first executed, and the evaluation index of the initial prompt word template can be directly determined based on the newly obtained target log data without executing step B1 again.

[0175] Optionally, using a pre-trained large model and a target prompt word template to detect abnormal behavior of the target information system corresponding to the target log data may include:

[0176] Obtain the log data to be tested generated by the target information system;

[0177] Generate target prompt words according to the log data to be detected and the target prompt word template;

[0178] The target prompt word and the log data to be detected are input into the pre-trained large model to detect whether there is abnormal behavior in the log data to be detected.

[0179] The method for generating the target prompt word according to the log data to be detected and the target prompt word template may be:

[0180] Extract prompt information from the log data to be detected;

[0181] Map the prompt information to the target prompt word template to obtain the target prompt word.

[0182] In this embodiment, the original user behavior logs generated by the target information system can be obtained, and the log data to be detected can be obtained based on these original user behavior logs. Among them, the time period corresponding to the log data to be detected and the time period corresponding to the target log data can be different. For example, the target log data is obtained based on the original user behavior logs in the past week, and the log data to be detected can be obtained based on the original user behavior logs of each day in the future to detect whether there is abnormal behavior in each day in the future. In other words, every day in the future, the corresponding log data to be detected can be obtained based on the current original user behavior log. The method for obtaining the data to be detected based on the original user behavior log is consistent with the aforementioned method for obtaining the target log data based on the original user behavior log, and will not be repeated.

[0183] Like the target log data, there can be multiple log data to be detected, and each log data to be detected corresponds to a user of the target information system.

[0184] In this embodiment, for each log data to be detected, the target prompt word corresponding to the log data to be detected can be generated according to the above method, and then the log data to be detected and the corresponding target prompt word are input into the pre-trained large model to obtain the detection result corresponding to the log data to be detected, that is, Figure 3 The output shown in the figure can also be used to input multiple log data to be detected and the corresponding target prompt words, so that the pre-trained large model can output multiple detection results of multiple log data to be detected, for example, output n detection results corresponding to n log data to be detected, that is, Figure 3 Output 1 to output n are shown, and n can be any integer, such as 5, 12, etc., without limitation.

[0185] For each log data to be detected, when extracting prompt information from the log data to be detected, refer to Figure 3 , information related to the task description and information related to the format declaration can be directly extracted from the log data to be detected as prompt information, or the original user behavior log used to generate the log data to be detected can be read to obtain the above prompt information from these original user behavior logs.

[0186] The information related to the task description may include the user corresponding to the log data to be detected, the time period covered by the log data to be detected, and what type of abnormal behavior needs to be detected in the log data to be detected.

[0187] For example, information extracted from a log data to be detected and related to the task description may include: User A, from 12:00 to 24:00 on January 1, 2021, detected abnormal deletion behavior. Abnormal deletion behavior refers to abnormal deletion of data from the target information system.

[0188] The information related to the format declaration may include various information that can define the format of the detection results output by the pre-trained large model, as well as information that can indicate the format of the input log data to be detected.

[0189] This information can be obtained by analyzing the log data to be detected, or by analyzing the original user behavior log used to generate the log data to be detected.

[0190] Exemplarily, the format of the log data to be detected can be analyzed to obtain information that can indicate the format of the input log data to be detected, such as "the input data includes the number of user logins every two hours." The original user behavior log used to generate the log data to be detected can be analyzed to obtain information that can limit the format of the detection results output by the pre-trained large model, such as "please output the detection results in digital form, 1 indicates abnormal behavior, and 0 indicates no abnormal behavior."

[0191] Mapping the prompt information to the target prompt word template means determining the position for inserting the prompt information in the target prompt word template, and then inserting the prompt information into the corresponding position of the target prompt word template to obtain the target prompt word consisting of the target prompt word template and the prompt information at the corresponding position.

[0192] For information related to the task description, keywords of the same type can be identified in the target prompt word template, and the locations of these keywords can be determined as the locations for inserting information related to the task description. When inserting information related to the task description, the inserted information can replace the keywords originally located at that location.

[0193] Combined with the above example of information related to the task description, assuming that the target prompt word template is "Detect abnormal login behavior of user X within time period Y", the insertion position in the template can be the location of "user X", "time period Y" and "abnormal login behavior". After inserting the information related to the task description, you can obtain: "Detect user A's abnormal deletion behavior between 12:00 and 24:00 on January 1, 2021".

[0194] For information related to the format declaration, it can be determined that this information should be inserted at the end of the target prompt word template. Combining the above example of information related to the format declaration, after inserting the information related to the task description and the information related to the format declaration into the target prompt word template, the following target prompt word can be obtained:

[0195] "Detect user A's abnormal deletion behavior between 12:00 and 24:00 on January 1, 2021. Please output the detection results in digital form, 1 indicates abnormal behavior, and 0 indicates no abnormal behavior."

[0196] By generating target prompt words in the above manner, the target prompt words can provide an accurate description of the task and clear data format specification instructions for the process of detecting abnormal behavior of the pre-trained large model, which can not only improve the processing efficiency and accuracy of the pre-trained large model, but also obtain detection results that meet specific format requirements.

[0197] The present application also provides an abnormal behavior detection device based on a large model and log big data, see Figure 5 , the device may include the following units.

[0198] An obtaining unit 501 is used to obtain target log data;

[0199] Processing unit 502, configured to process target log data according to the pre-trained large model to obtain a target prompt word template;

[0200] The detection unit 503 is configured to perform abnormal behavior detection on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template.

[0201] Optionally, when obtaining target log data, the obtaining unit 501 is configured to:

[0202] Obtain original user behavior logs of multiple users;

[0203] Preprocessing the original user behavior log to obtain a preprocessed user behavior log, wherein the preprocessing includes at least one of data standardization, data cleaning, and correlation aggregation;

[0204] Perform feature extraction on the preprocessed user behavior log to obtain target log data.

[0205] Optionally, when the obtaining unit 501 extracts features from the pre-processed user behavior log and obtains target log data, it is used to:

[0206] Convert the pre-processed user behavior logs corresponding to each user into time series data based on the target time window;

[0207] Analyze each user's time series data to obtain the corresponding user behavior baseline for each user;

[0208] Compare the user behavior baselines of different users to obtain user comparison features;

[0209] Compare the user behavior baselines of different categories of users to obtain user group comparison features;

[0210] For each user, discrete behavior feature extraction and continuous behavior feature extraction are performed on the user's time series data based on the user's user behavior baseline to obtain the user's discrete behavior feature and continuous behavior feature;

[0211] Among them, user comparison features, user group comparison features, user discrete behavior features and continuous behavior features constitute the target log data.

[0212] Optionally, when the obtaining unit 501 converts the pre-processed user behavior log corresponding to each user into time series data based on the target time window, it is used to:

[0213] For each user, the pre-processed user behavior log corresponding to the user is converted based on multiple target time windows of different lengths to obtain multiple time series data corresponding to the user. Each time series data is converted based on a target time window.

[0214] Optionally, before the detection unit 503 uses the pre-trained large model and the target prompt word template to perform abnormal behavior detection on the target information system corresponding to the target log data, it is further configured to:

[0215] Inject knowledge into the pre-trained large model based on positive samples and negative samples to obtain a large model after knowledge injection;

[0216] When the detection unit 503 uses the pre-trained large model and the target prompt word template to detect abnormal behavior of the target information system corresponding to the target log data, it is used to:

[0217] The large model after knowledge injection and the target prompt word template are used to detect abnormal behavior of the target information system corresponding to the target log data.

[0218] Optionally, when the processing unit 502 processes the target log data according to the pre-trained large model and obtains the target prompt word template, it is used to:

[0219] Process the target log data according to the pre-trained large model to obtain the initial prompt word template;

[0220] Determine the evaluation index of the initial prompt word template based on the target log data;

[0221] If the evaluation index does not meet the target conditions, the initial prompt word template is updated according to the pre-trained large model and target log data;

[0222] Return to the step of determining the evaluation index of the initial prompt word template according to the target log data until an initial prompt word template whose evaluation index meets the target condition is obtained, wherein the initial prompt word template whose evaluation index meets the target condition is used as the target prompt word template.

[0223] Optionally, the evaluation metric includes at least one of precision, recall, and F1 score.

[0224] Optionally, when the processing unit 502 updates the initial prompt word template according to the pre-trained large model and the target log data, it is configured to perform at least one of the following:

[0225] Adjust the time range corresponding to the initial prompt word template based on the pre-trained large model and target log data;

[0226] Add contextual information to the initial prompt word template based on the pre-trained large model and target log data;

[0227] Adjust the keywords contained in the initial prompt word template based on the pre-trained large model and target log data.

[0228] Optionally, the processing unit 502 is further configured to:

[0229] The target prompt word template is updated regularly according to the preset update cycle.

[0230] Optionally, when the detection unit 503 uses the pre-trained large model and the target prompt word template to detect abnormal behavior of the target information system corresponding to the target log data, it is used to:

[0231] Obtain the log data to be tested generated by the target information system;

[0232] Generate target prompt words according to the log data to be detected and the target prompt word template;

[0233] The target prompt word and the log data to be detected are input into the pre-trained large model to detect whether there is abnormal behavior in the log data to be detected.

[0234] The method for generating the target prompt word according to the log data to be detected and the target prompt word template may be:

[0235] Extract prompt information from the log data to be detected;

[0236] Map the prompt information to the target prompt word template to obtain the target prompt word.

[0237] The working principle of the abnormal behavior detection device based on large models and log big data provided in this embodiment can be found in the relevant steps of the abnormal behavior detection method based on large models and log big data provided in the previous embodiment, and will not be repeated here.

[0238] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0239] For the convenience of description, the above systems or devices are described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0240] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0241] Finally, it should be noted that, in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0242] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting abnormal behavior based on a large model and log data, characterized in that: include: Get target log data; Process the target log data according to the pre-trained large model to obtain a target prompt word template; Abnormal behavior detection is performed on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template.

2. The method according to claim 1, characterized in that The obtaining of target log data includes: Obtain original user behavior logs of multiple users; Preprocessing the original user behavior log to obtain a preprocessed user behavior log, wherein the preprocessing includes at least one of data standardization, data cleaning, and correlation aggregation; Feature extraction is performed on the preprocessed user behavior log to obtain target log data.

3. The method according to claim 2, characterized in that The step of extracting features from the pre-processed user behavior log to obtain target log data includes: Converting the pre-processed user behavior log corresponding to each user into time series data based on a target time window; Analyze the time series data of each user to obtain a user behavior baseline corresponding to each user; Comparing the user behavior baselines of different users to obtain user comparison features; Comparing the user behavior baselines of different categories of users to obtain user group comparison features; For each of the users, extracting discrete behavior features and continuous behavior features from the time series data of the user according to the user behavior baseline of the user to obtain discrete behavior features and continuous behavior features of the user; The user comparison feature, the user group comparison feature, the user's discrete behavior feature and the continuous behavior feature constitute the target log data.

4. The method according to claim 3, characterized in that The converting the pre-processed user behavior log corresponding to each user into time series data based on the target time window includes: For each of the users, the pre-processed user behavior log corresponding to the user is converted based on multiple target time windows of different lengths to obtain multiple time series data corresponding to the user, each of which is converted based on one of the target time windows.

5. The method according to claim 1, wherein Before performing abnormal behavior detection on the target information system corresponding to the target log data using the pre-trained large model and the target prompt word template, the method further includes: Performing knowledge injection into the pre-trained large model according to the positive samples and the negative samples to obtain a large model after knowledge injection; The detecting abnormal behavior of the target information system corresponding to the target log data by using the pre-trained large model and the target prompt word template includes: Abnormal behavior detection is performed on the target information system corresponding to the target log data using the large model after knowledge injection and the target prompt word template.

6. The method according to claim 1, characterized in that The step of processing the target log data according to the pre-trained large model to obtain a target prompt word template includes: Process the target log data according to the pre-trained large model to obtain an initial prompt word template; Determining an evaluation index of the initial prompt word template according to the target log data; If the evaluation index does not meet the target condition, updating the initial prompt word template according to the pre-trained large model and the target log data; Return to the step of determining the evaluation index of the initial prompt word template based on the target log data until the initial prompt word template whose evaluation index meets the target condition is obtained, wherein the initial prompt word template whose evaluation index meets the target condition is used as the target prompt word template.

7. The method according to claim 6, characterized in that The evaluation index includes at least one of precision, recall and F1 score.

8. The method according to claim 6, characterized in that The updating of the initial prompt word template according to the pre-trained large model and the target log data includes at least one of the following: Adjusting the time range corresponding to the initial prompt word template according to the pre-trained large model and the target log data; Adding context information to the initial prompt word template according to the pre-trained large model and the target log data; The keywords contained in the initial prompt word template are adjusted according to the pre-trained large model and the target log data.

9. The method according to claim 1, characterized in that The detecting abnormal behavior of the target information system corresponding to the target log data by using the pre-trained large model and the target prompt word template includes: Obtaining log data to be detected generated by the target information system; Extracting prompt information from the log data to be detected; Mapping the prompt information to the target prompt word template to obtain the target prompt word; The target prompt word and the log data to be detected are input into the pre-trained large model to detect whether there is abnormal behavior in the log data to be detected.

10. An abnormal behavior detection device based on a large model and log big data, characterized in that: include: An acquisition unit, used for acquiring target log data; A processing unit, configured to process the target log data according to a pre-trained large model to obtain a target prompt word template; The detection unit is used to use the pre-trained large model and the target prompt word template to perform abnormal behavior detection on the target information system corresponding to the target log data.

Images