Methods, apparatus, devices, and readable storage media for determining log operation behavior

Abstract

This application relates to the field of network security and provides a method, apparatus, device, and readable storage medium for determining log operation behaviors. The method for determining log operation behaviors includes: extracting numerical and content features from historical log data; determining the distance between historical log data based on the numerical and content features; clustering the historical log data based on the distance and preset neighborhood parameters to obtain the operation behaviors corresponding to each historical log data; constructing an operation behavior table based on the numerical features, content features, and the operation behaviors corresponding to each historical log data; and determining the operation behaviors of new log data based on the operation behavior table. By clustering historical log data to obtain a table containing the operation behaviors of each historical log data, and then determining the operation behaviors of new log data based on the operation behavior table, the processing efficiency of log operation behaviors is improved.

Description

Technical Field

[0001] This application relates to the field of network security, specifically to a method, apparatus, device, and readable storage medium for determining log operation behavior. Background Technology

[0002] Using traditional statistical methods to classify user log activity is inefficient and prone to misjudgments. Auditors categorize user account login logs and backend operation logs using traditional statistical methods. This method is relatively easy to use with small amounts of data, but when the data volume increases dramatically, it becomes difficult and inefficient to classify user log activity manually.

[0003] Traditional density-based clustering (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) algorithms are relatively inefficient, requiring re-clustering of all data whenever new log data appears. Traditional K-means clustering algorithms require pre-selection of the number of clusters / categories. In massive log datasets, due to the uncertainty and complexity of log operations, the number of clusters / categories is highly random, making pre-selection of clusters / categories unreliable in ensuring classification accuracy. The mean-shift algorithm suffers from a fixed window size / radius; in complex scenarios with massive log datasets, inappropriate window size / radius settings can lead to poor classification performance. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for determining log operation behavior, in order to solve the technical problem of low efficiency in existing methods for determining log operation behavior.

[0005] In a first aspect, embodiments of this application provide a method for determining log operation behavior, including:

[0006] Extract numerical and content features from historical log data;

[0007] Based on the numerical features and the content features, the distance between each of the historical log data is determined;

[0008] Based on the distance and preset neighborhood parameters, the historical log data are clustered to obtain the operation behavior corresponding to each historical log data.

[0009] Based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data, an operation behavior table is constructed.

[0010] Based on the operation behavior table, determine the operation behavior for the new log data.

[0011] In one embodiment, the numerical feature includes operation time and operation type; the content feature includes an operation content vector; determining the distance between each of the historical log data based on the numerical feature and the content feature includes:

[0012] Determine the similarity between the vectors containing the operations described;

[0013] Based on the similarity and the normalized operation time and operation type, the distance between each of the historical log data is determined.

[0014] In one embodiment, the preset neighborhood parameters include a preset number and a preset distance; the step of clustering the historical log data based on the distance and the preset neighborhood parameters to obtain the corresponding operation behavior for each historical log data includes:

[0015] Based on the distance between each of the historical log data, the core data in each of the historical log data is determined. The number of log data in the neighborhood of the core data is greater than the preset number. The neighborhood of the core data is determined with the core data as the center and the preset distance as the radius.

[0016] Based on the neighborhood of the core data, normal log data and abnormal log data are determined in each of the historical log data. The operation behavior corresponding to the normal log data is a normal operation, and the operation behavior corresponding to the abnormal log data is an abnormal operation if it is not in the neighborhood of the core data.

[0017] In one embodiment, constructing the operation behavior table based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data includes:

[0018] A normal cluster is determined based on each normal log data, and the distance between each normal log data in the normal cluster is less than or equal to the preset distance.

[0019] An anomaly cluster is determined based on the anomaly log data, wherein the distance between the anomaly log data in the anomaly cluster is less than or equal to the preset distance;

[0020] The operation behavior table is determined to include the operation time, the operation type, the operation content vector, and the operation tag. The operation tag includes the normal cluster, the abnormal cluster, and abnormal log data.

[0021] In one embodiment, determining the operation behavior for new log data based on the operation behavior table includes:

[0022] Based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data, the operation label corresponding to the new log data is determined;

[0023] If the operation tag corresponding to the new log data is the normal cluster, then the operation behavior corresponding to the new log data is determined to be a normal operation.

[0024] In one embodiment, after determining the operation tag corresponding to the new log data based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data, the process includes:

[0025] If the operation tag corresponding to the new log data is the abnormal log data, and the new log data is within the neighborhood of the abnormal log data, the new log data and the abnormal log data are merged to obtain a new abnormal cluster.

[0026] If the operation label corresponding to the new log data is the anomaly cluster, and the number of abnormal log data in the anomaly cluster is greater than the first number, clustering processing is performed on the new log data and each of the historical log data to obtain the operation behavior classification result.

[0027] Secondly, embodiments of this application provide a log operation behavior determination device, comprising:

[0028] The feature extraction module is used to extract numerical and content features from historical log data;

[0029] The distance determination module is used to determine the distance between each of the historical log data based on the numerical features and the content features;

[0030] The clustering processing module is used to perform clustering processing on each of the historical log data based on the distance and preset neighborhood parameters to obtain the operation behavior corresponding to each of the historical log data.

[0031] The operation behavior table construction module is used to construct an operation behavior table based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data.

[0032] The operation behavior classification module is used to determine the operation behavior of new log data based on the operation behavior table.

[0033] Thirdly, embodiments of this application provide a device including a processor and a memory storing a computer program, wherein the processor executes the computer program to implement the log operation behavior determination method described in the first aspect.

[0034] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the log operation behavior determination method described in the first aspect.

[0035] The log operation behavior determination method, apparatus, device, and readable storage medium provided in this application extract features from historical log data, determine the distance between each historical log data based on the extracted numerical and content features, cluster each historical log data based on the distance and preset neighborhood parameters to obtain the operation behavior corresponding to each historical log data, construct an operation behavior table based on the obtained numerical and content features and the operation behavior corresponding to each historical log data, and finally determine the operation behavior of new log data through the operation behavior table, thereby improving the processing efficiency of log operation behavior. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is one of the flowcharts illustrating the log operation behavior determination method provided in the embodiments of this application;

[0038] Figure 2 This is a second flowchart illustrating the method for determining log operation behavior provided in this application embodiment;

[0039] Figure 3 This is the third flowchart illustrating the log operation behavior determination method provided in the embodiments of this application;

[0040] Figure 4 This is a schematic diagram of the structure of the log operation behavior determination device provided in the embodiments of this application;

[0041] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] Reference Figure 1 , Figure 1 This is one of the flowcharts illustrating the method for determining log operation behavior in this application. The method for determining log operation behavior provided in this application may include:

[0044] Step 100: Extract numerical and content features from historical log data;

[0045] Specifically, the method for determining log operation behavior provided in this application involves feature extraction of the log operation data to be processed, as follows:

[0046] The operation time and operation type of historical log data are used as numerical features, and the operation content (vector) of historical log data is used as content features. The operation time and operation content are transformed, and the operation time can be converted into different log formats according to different statistical period requirements. Natural language processing is used to segment the operation content, and word2vec is used to vectorize the segmented content to construct the text similarity feature vector of the operation content.

[0047] By extracting features from historical log data, we can obtain the behavioral characteristics of users interacting with historical log data. The specific features and feature values ​​are as follows:

[0048] Extract fields such as user account, operation time, operation type, and operation content from historical log data. Add a unique identifier field for each historical log record to distinguish them. Group the data according to user account (hereinafter referred to as Zhang San). Table 1 below shows a portion of the historical log operation data records of account Zhang San at a certain time (e.g., June 1, 2023). The log operations are denoted as p1-p11. The operation time field is denoted as x, and the operation type field is denoted as y. The operation type codes in Table 1 have the following meanings: 1 indicates user login, 2 indicates data query, 3 indicates exporting a data list, 4 indicates deleting data, and 5 indicates modifying data.

[0049] p1 2023-06-01 09:01:25 1 Zhang San logs into system A p2 2023-06-01 09:06:04 2 Zhang San queries system b p3 2023-06-01 10:18:21 3 Zhang San logs into system A p4 2023-06-01 11:12:12 1 Query P5 2023-06-01 12:06:01 3 Zhang San logs into system A p6 2023-06-01 13:00:11 1 Query System p7 2023-06-01 13:30:05 2 Zhang San logs into system A p8 2023-06-01 15:19:01 2 Query System p9 2023-06-01 23:01:01 1 Zhang San logs into system A p10 2023-06-01 23:30:28 2 Login p11 2023-06-02 00:02:10 4 Login System

[0050] Table 1

[0051] The specific details of feature preprocessing are as follows: Operation time and content are converted. Operation time can be processed into different log formats according to different statistical periods. For example, when analyzing logs on a daily basis, data from one day (from 0:00 to 24:00) is selected and processed into decimal form in hourly units. When using a monthly statistical period, the data can be converted into second-level timestamps. Due to the large span of time differences, the time differences need to be normalized, that is, the time field is normalized to between 0 and 1 based on the selected time period offset. For example, the minimum operation time in June 2023 is 2023-06-01 09:01:25, corresponding to a second-level timestamp of 1685581264. The maximum operation time is 2023-06-30 23:59:00, corresponding to a timestamp of 1688140740. The process of calculating the normalized value corresponding to the operation time 2023-06-15 10:25:11 is as follows: convert it to a timestamp of 1686795911, substitute it into the normalization formula, and you will get a normalized time value of 0.47.

[0052] For the text features of the operation content, natural language processing techniques are used to segment the operation content into words, and then word2vec is used to vectorize the operation content to construct a text similarity feature vector. In summary, the normalized historical log data format is shown in Table 2 below. Each log operation record in Table 2 corresponds to one historical log data and also one data point.

[0053] p1 9 1 [[1，2]，[3，4]] p2 9.1 2 [[2，2]，[3，4]] p3 10.3 3 [[1，3]，[3，4]] p4 11.2 1 [[1，2]，[3，4]] P5 12.1 3 [[4，2]，[3，4]] p6 13 1 [[2，5]，[3，4]] p7 13.5 2 [[2，2]，[3，4]] p8 15.3 2 [[1，2]，[3，4]] p9 23 1 [[5，6]，[3，4]] p10 23.5 2 [[6，7]，[3，4]] p11 24 4 [[3，2]，[3，4]]

[0054] Table 2

[0055] Step 200: Based on the numerical features and the content features, determine the distance between each of the historical log data;

[0056] The log operation behavior determination method provided in this application embodiment may further include:

[0057] Step 210: Determine the similarity between each of the operation content vectors;

[0058] Step 220: Based on the similarity and the normalized operation time and operation type, determine the distance between each of the historical log data.

[0059] Specifically, as shown in Tables 1 and 2, each historical log data corresponds to a data point. This application provides a method for calculating the neighborhood distance between data points based on operational behavior characteristics. As shown in Formula 1, the calculation result of Formula 1 is the neighborhood distance. Assuming one historical log data is (p1, x1, y1, A) and another is (p2, x2, y2, B), where p is the operation record identifier, x represents the operation time, y represents the operation type, and A and B represent the operation content vector matrices, the distance between the two historical log data is calculated as shown in Formula 1.

[0060]

[0061] sim(A,B) represents the similarity between two operation content vectors. The higher the similarity, the closer the two historical log data are. The sigmoid function is used to convert the similarity to between 0 and 1. 1-sigmoid represents the distance change. The K value is used to adjust the weight of the operation content in calculating the distance between historical log data. It can be adjusted and optimized according to the classification results.

[0062] Step 300: Based on the distance and preset neighborhood parameters, perform clustering processing on each of the historical log data to obtain the operation behavior corresponding to each of the historical log data;

[0063] Step 400: Based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data, construct an operation behavior table;

[0064] Specifically, this application embodiment sets the neighborhood parameters Eps and MinPts (i.e., the preset neighborhood parameters in this embodiment) of DBSCAN, as well as the distance calculation formula. Eps describes the neighborhood distance threshold of historical log data, and MinPts describes the threshold for the number of historical log data entries in the neighborhood of Eps. Using the DBSCAN clustering algorithm proposed in this application embodiment, cluster analysis is performed on the operation behavior characteristics of Zhang San's account, such as operation time, operation type, and operation content, to obtain the operation behavior corresponding to each historical log data. This determines which operations are routine and which are abnormal. The obtained operation behavior determination results are recorded and stored to generate a log operation behavior table for user Zhang San, so that the operation behavior of new log operation data of Zhang San can be determined based on the operation behavior table later.

[0065] Step 400: Based on the operation behavior table, determine the operation behavior for the new log data.

[0066] Specifically, the operation behaviors corresponding to Zhang San's new log operation data are determined using the aforementioned feature extraction and cluster analysis methods, resulting in label values ​​for the new log operation data. These label values ​​include normal operations, abnormal operations, and a normal operation cluster. The normal cluster contains normal log operation data. The operation behavior features and corresponding label values ​​of the new log data are then updated in the operation behavior table.

[0067] This embodiment extracts features from historical log data, determines the distance between each historical log data based on the extracted numerical and content features, clusters each historical log data based on the distance and preset neighborhood parameters, obtains the corresponding operation behavior for each historical log data, then constructs an operation behavior table based on the obtained numerical and content features and the corresponding operation behavior for each historical log data, and finally determines the operation behavior for new log data through the operation behavior table, thereby improving the processing efficiency of log operation behavior.

[0068] Reference Figure 2 , Figure 2 This is a second flowchart illustrating the method for determining log operation behavior in this application. In one embodiment, the method for determining log operation behavior provided in this application may further include:

[0069] Step 310: Determine the core data in each of the historical log data based on the distance between each of the historical log data. The number of log data in the neighborhood of the core data is greater than the preset number. The neighborhood of the core data is determined with the core data as the center and the preset distance as the radius.

[0070] Step 320: Based on the neighborhood of the core data, determine the normal log data and abnormal log data in each of the historical log data. The operation behavior corresponding to the normal log data is a normal operation, and the operation behavior corresponding to the abnormal log data is an abnormal operation if it is not in the neighborhood of the core data.

[0071] The log operation behavior determination method provided in this application embodiment may further include:

[0072] Step 410: Determine normal clusters based on each normal log data, wherein the distance between each normal log data in the normal cluster is less than or equal to the preset distance;

[0073] Step 420: Determine an anomaly cluster based on each of the anomaly log data, wherein the distance between each anomaly log data in the anomaly cluster is less than or equal to the preset distance;

[0074] Step 430: Determine that the operation behavior table includes the operation time, the operation type, the operation content vector, and the operation tag, wherein the operation tag includes the normal cluster, the abnormal cluster, and abnormal log data.

[0075] Specifically, as shown in Table 2 above, each historical log data corresponds to a data point. Historical log data of users over a past period is filtered based on the operation time field. The average distance between each historical log data is calculated as the preset distance Eps in the preset neighborhood parameters. Then, based on Eps, core data points n are sampled from each historical log data, and these core points P1, P2, ..., P... are calculated. n Points Pts1, Pts2, ..., Pts within a preset distance range of Eps n The average value of these points is calculated using the following formula 2, which serves as the preset number MinPts in the preset neighborhood parameters.

[0076]

[0077] First, find an unvisited point p4 (11.2, 1, [[1, 2], [3, 4]]). If this point is a core (data) point, create a new cluster C1, add the points in the neighborhood of this core point to this cluster, and traverse the data points in its neighborhood. If a point p3 (10.3, 3, [[4, 2], [3, 4]]) in the neighborhood of this core point is also a core point, then add the points in the neighborhood of p3 (10.3, 3, [[4, 2], [3, 4]]) to cluster C1. A core point is defined as follows: if the number of data points within Eps of a given point exceeds Minpts, then this point is a core point. This process continues until cluster C1 stops expanding and until all points are processed. Points are marked as processed: two lists are set up: the unvisited list stores unprocessed points, and the visited list stores processed points. Each time a point is processed, it is removed from the unvisited list and added to the visited list, thereby changing the processing status of the data point.

[0078] In the improved DBSCN algorithm proposed in this application, the definition of quantity reachability is as follows: if a point a is within the neighborhood of point b, then b is directly quantity reachable from core point a; if point c is also within the neighborhood of point b, then point c is indirectly quantity reachable from point a, and the quantities of a and c are connected. The definitions of outlier points / outlier clusters are as follows: if a point is not within the neighborhood of a core point and is quantity unreachable from any core point, a point that is neither a core point nor a boundary point is called an outlier point. If the distance between two outlier points is within Eps, then the two outlier points are clustered into an outlier cluster.

[0079] User log operation behaviors are complex, and abnormal operations may exist even within normal clusters (for example, a scenario where a large number of abnormal operations are performed within a certain period; if these operations are frequent and concentrated in time, they may be aggregated into a normal cluster in DSCCAN classification). Anomalies and anomaly clusters may also contain some normal operations (such as specific maintenance operations). Therefore, the concepts of normal clusters, anomalies, or anomaly clusters in DBSCAN are related to high and low density areas and are different from the actual concepts of normal and abnormal operations. Therefore, further verification and judgment are needed based on the operation behavior determination results of this application to ultimately determine which user operations are non-routine or abnormal.

[0080] p1 2023-06-01 09:06:25 1 Zhang San [[1，2]，[3，4]] <![CDATA[C1]]> p2 2023-06-01 09:01:04 2 Zhang San [[2，2]，[3，4]] <![CDATA[C1]]> p3 2023-06-01 10:18:21 3 Zhang San [[1，3]，[3，4]] <![CDATA[C1]]> p4 2023-06-01 11:12:12 1 Zhang San [[1，2]，[3，4]] <![CDATA[C1]]> P5 2023-06-01 12:06:01 3 Zhang San [[4，2]，[3，4]] <![CDATA[C1]]> p6 2023-06-01 13:00:11 1 Zhang San [[2，5]，[3，4]] <![CDATA[C1]]> p7 2023-06-01 13:30:05 2 Zhang San [[2，2]，[3，4]] <![CDATA[C1]]> p8 2023-06-01 15:19:01 2 Zhang San [[1，2]，[3，4]] <![CDATA[C1]]> p9 2023-06-01 23:01:01 1 Zhang San [[5，6]，[3，4]] <![CDATA[UC1]]> p10 2023-06-01 23:30:28 2 Zhang San [[6，7]，[3，4]] <![CDATA[UC1]]> p11 2023-06-01 24:02:10 4 Zhang San [[3，2]，[3，4]] <![CDATA[UP1]]>

[0081] Table 3

[0082] Table 3 shows the user Zhang San's operational behavior over a specific time period. The table includes fields such as a unique identifier for each historical log entry, log operation time, operation type, user name, operation content, and label value. The label value represents the operational behavior determined by the cluster analysis. Normal cluster label values ​​are C1, C2, C3, ..., C... n This indicates that the abnormal clusters are represented by UC1, UC2, UC3, ..., UC... n Anomalies are indicated by UP1, UP2, ..., UP n express.

[0083] This embodiment uses an improved DBSCAN quantitative clustering algorithm to store the results of log operation behavior determination, thereby improving the efficiency of log operation behavior determination.

[0084] Reference Figure 3 , Figure 3 This is the third flowchart illustrating the method for determining log operation behavior in this application. In one embodiment, the method for determining log operation behavior provided in this application may further include:

[0085] Step 510: Based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data, determine the operation label corresponding to the new log data;

[0086] Step 520: If the operation tag corresponding to the new log data is the normal cluster, determine that the operation behavior corresponding to the new log data is a normal operation.

[0087] The log operation behavior determination method provided in this application embodiment may further include:

[0088] Step 530: If the operation tag corresponding to the new log data is the abnormal log data, and the new log data is within the neighborhood of the abnormal log data, then the new log data and the abnormal log data are merged to obtain a new abnormal cluster.

[0089] Step 540: If the operation label corresponding to the new log data is the abnormal cluster, and the number of abnormal log data in the abnormal cluster is greater than the first number, perform clustering processing on the new log data and each of the historical log data to obtain the operation behavior classification result.

[0090] Specifically, the operation behavior corresponding to newly generated log operation data (i.e., new log data in this embodiment) is determined through an operation behavior table, and the operation behavior table is incrementally updated. The process of determining the operation behavior is the process of cluster analysis. First, the new log data needs to be converted into the corresponding data format. For example, when user Zhang San generates new log operation data, the calculation is first performed based on the normal cluster points mentioned above. If the number of data points corresponding to the new log data is equal to that of a certain normal cluster (C1 in this example), then it is classified as the current normal cluster C1; if the data points corresponding to the new log data do not belong to any existing normal cluster, then it is further checked whether they belong to an anomaly or are in the neighborhood of an anomaly cluster UC1; if they are in the neighborhood of an anomaly cluster UC1, then they are classified as an anomaly cluster UC1; if they belong to an anomaly, then they are merged with an anomaly cluster UP1 to form a new anomaly cluster UC2; if they belong to neither, then they are classified as a new anomaly cluster UP2.

[0091] The specific process for updating the operation behavior table is as follows:

[0092] Anomalies within a past period are filtered based on the operation time field. Using a custom distance formula, the average distance between two anomalies is calculated as the initial radius threshold value, uEps. Then, based on uEps, n anomalies uP1, uP2, ..., uP1 are sampled. n Calculate the number of outliers uPts1, uPts2, ..., uPts within the radius threshold range. n The average value of these points is taken as the initial value uMinPts for the threshold of the number of outlier clusters. The formula for calculating uMinPts is as follows:

[0093] As shown in Equation 3;

[0094]

[0095] Calculate the custom distance between the new log data and the extracted core points. If the distance between the new log data point and a certain core point is less than or equal to the neighborhood radius Eps, then add the new data to the neighborhood of that core point.

[0096] If new log data is classified into the neighborhood of a core point, check if the number of points within the radius of the new data point's neighborhood exceeds MinPts. If it does, mark it as a core point and add the data points within its neighborhood to the cluster.

[0097] If new data is not added to the neighborhood of any core point (i.e., all core points are unreachable and the data is not within the neighborhood of other outliers or outlier clusters), it is marked as an outlier. If the distance between a new outlier and a known outlier is less than the radius uEps, the two are merged into a new outlier cluster; if the distance is greater than the radius uEps, it is classified as a new outlier. If a point in a known outlier cluster is less than the radius uEps, it is merged into that outlier cluster. When the number of points in an outlier cluster exceeds the outlier cluster count threshold uMinPts after updating, in order to ensure the accuracy of clustering, the above clustering steps need to be repeated for the entire dataset.

[0098] The log operation behavior determination method provided in this embodiment uses an operation behavior table to determine the log operation behavior of new log data, reducing the operation of re-clustering and improving the efficiency of log operation behavior determination.

[0099] refer to Figure 4 , Figure 4 This is a schematic diagram of the log operation behavior determination device in the embodiments of this application. The log operation behavior determination device provided in the embodiments of this application is described below. The log operation behavior determination device described below and the log operation behavior determination method described above can be referred to each other.

[0100] Feature extraction module 401 is used to extract numerical features and content features from historical log data;

[0101] The distance determination module 402 is used to determine the distance between each of the historical log data based on the numerical features and the content features;

[0102] Clustering processing module 403 is used to perform clustering processing on each of the historical log data based on the distance and preset neighborhood parameters to obtain the operation behavior corresponding to each of the historical log data.

[0103] The operation behavior table construction module 404 is used to construct an operation behavior table based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data.

[0104] The operation behavior classification module 405 is used to determine the operation behavior of new log data based on the operation behavior table.

[0105] Optionally, the numerical features include operation time and operation type; the content features include an operation content vector; the distance determination module includes:

[0106] A similarity determination unit is used to determine the similarity between each of the operation content vectors;

[0107] The distance determination unit is used to determine the distance between each of the historical log data based on the similarity and the normalized operation time and operation type.

[0108] Optionally, the preset neighborhood parameters include a preset number and a preset distance; the clustering processing module includes:

[0109] The core data determination unit is used to determine the core data in each of the historical log data based on the distance between each of the historical log data. The number of log data in the neighborhood of the core data is greater than the preset number. The neighborhood of the core data is determined with the core data as the center and the preset distance as the radius.

[0110] The log data classification unit is used to determine normal log data and abnormal log data in each of the historical log data based on the neighborhood of the core data. The operation behavior corresponding to the normal log data is a normal operation, and the operation behavior corresponding to the abnormal log data is an abnormal operation if it is not in the neighborhood of the core data.

[0111] Optionally, the operation behavior table construction module includes:

[0112] A normal cluster determination unit is used to determine a normal cluster based on each of the normal log data, wherein the distance between each normal log data in the normal cluster is less than or equal to the preset distance.

[0113] An anomaly cluster determination unit is used to determine an anomaly cluster based on each of the anomaly log data, wherein the distance between each anomaly log data in the anomaly cluster is less than or equal to the preset distance.

[0114] An operation behavior table construction unit is used to determine that the operation behavior table includes the operation time, the operation type, the operation content vector, and the operation tag, wherein the operation tag includes the normal cluster, the abnormal cluster, and abnormal log data.

[0115] Optionally, the operation behavior classification module includes:

[0116] The first operation label determination unit is used to determine the operation label corresponding to the new log data based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data.

[0117] The second operation label determination unit is used to determine that the operation behavior corresponding to the new log data is a normal operation when the operation label corresponding to the new log data is the normal cluster.

[0118] Optionally, the log operation behavior determination device further includes:

[0119] The data merging module is used to merge the new log data and the abnormal log data to obtain a new abnormal cluster when the operation tag corresponding to the new log data is the abnormal log data and the new log data is within the neighborhood of the abnormal log data.

[0120] The data clustering module is used to perform clustering processing on the new log data and each of the historical log data when the operation label corresponding to the new log data is the abnormal cluster and the number of abnormal log data in the abnormal cluster is greater than a first number, so as to obtain the operation behavior classification result.

[0121] Figure 5 An example is a schematic diagram of the physical structure of a device, such as... Figure 5 As shown, the device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call a computer program in the memory 530 to execute the steps of the log operation behavior determination method.

[0122] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program thereon, and when the computer program is executed by a processor, the computer is able to perform the steps of the log operation behavior determination method provided in the above embodiments.

[0124] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the log operation behavior determination method described in various embodiments or some parts of embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining log operation behavior, characterized in that, include: Extract numerical and content features from historical log data; Based on the numerical features and the content features, the distance between each of the historical log data is determined; Based on the distance and preset neighborhood parameters, the historical log data are clustered to obtain the operation behavior corresponding to each historical log data. Based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data, an operation behavior table is constructed. Based on the operation behavior table, determine the operation behavior for the new log data; The numerical features include operation time and operation type; the content features include operation content vectors; determining the distance between each historical log data based on the numerical features and the content features includes: Determine the similarity between the vectors containing the operations described; Based on the similarity and the normalized operation time and operation type, the distance between each of the historical log data is determined. The preset neighborhood parameters include a preset number and a preset distance; the clustering process based on the distance and preset neighborhood parameters to obtain the corresponding operation behavior for each historical log data includes: Based on the distance between each historical log data, the core data in each historical log data is determined. The number of log data in the neighborhood of the core data is greater than the preset number. The neighborhood of the core data is determined with the core data as the center and the preset distance as the radius. Based on the neighborhood of the core data, normal log data and abnormal log data are determined in each of the historical log data. The operation behavior corresponding to the normal log data is a normal operation, and the operation behavior corresponding to the abnormal log data is an abnormal operation if it is not in the neighborhood of the core data.

2. The method for determining log operation behavior according to claim 1, characterized in that, The step of constructing an operation behavior table based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data includes: A normal cluster is determined based on each normal log data, and the distance between each normal log data in the normal cluster is less than or equal to the preset distance. An anomaly cluster is determined based on the anomaly log data, wherein the distance between the anomaly log data in the anomaly cluster is less than or equal to the preset distance; The operation behavior table is determined to include the operation time, the operation type, the operation content vector, and the operation tag. The operation tag includes the normal cluster, the abnormal cluster, and abnormal log data.

3. The method for determining log operation behavior according to claim 1, characterized in that, The operation behavior for determining new log data based on the operation behavior table includes: Based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data, the operation label corresponding to the new log data is determined; If the operation tag corresponding to the new log data is the normal cluster, then the operation behavior corresponding to the new log data is determined to be a normal operation.

4. The method for determining log operation behavior according to claim 3, characterized in that, After determining the operation tag corresponding to the new log data based on the preset neighborhood parameters and the distance between the new log data and each of the historical log data, the process includes: If the operation tag corresponding to the new log data is the abnormal log data, and the new log data is within the neighborhood of the abnormal log data, the new log data and the abnormal log data are merged to obtain a new abnormal cluster. If the operation label corresponding to the new log data is the anomaly cluster, and the number of abnormal log data in the anomaly cluster is greater than the first number, clustering processing is performed on the new log data and each of the historical log data to obtain the operation behavior classification result.

5. A device for determining log operation behavior, characterized in that, include: The feature extraction module is used to extract numerical and content features from historical log data; The distance determination module is used to determine the distance between each of the historical log data based on the numerical features and the content features; The clustering processing module is used to perform clustering processing on each of the historical log data based on the distance and preset neighborhood parameters to obtain the operation behavior corresponding to each of the historical log data. The operation behavior table construction module is used to construct an operation behavior table based on the numerical features, the content features, and the operation behaviors corresponding to each of the historical log data. The operation behavior classification module is used to determine the operation behavior of new log data based on the operation behavior table; The numerical features include operation time and operation type; the content features include an operation content vector; and the distance determination module includes: A similarity determination unit is used to determine the similarity between each of the operation content vectors; The distance determination unit is used to determine the distance between each of the historical log data based on the similarity and the normalized operation time and operation type. The preset neighborhood parameters include a preset number and a preset distance; the clustering processing module includes: The core data determination unit is used to determine the core data in each of the historical log data based on the distance between each of the historical log data. The number of log data in the neighborhood of the core data is greater than the preset number. The neighborhood of the core data is determined with the core data as the center and the preset distance as the radius. The log data classification unit is used to determine normal log data and abnormal log data in each of the historical log data based on the neighborhood of the core data. The operation behavior corresponding to the normal log data is a normal operation, and the operation behavior corresponding to the abnormal log data is an abnormal operation if it is not in the neighborhood of the core data.

6. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the log operation behavior determination method according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the log operation behavior determination method as described in any one of claims 1 to 4.

Images