A user anomaly monitoring method and device, electronic equipment and storage medium
By performing sequence partitioning and dynamic baseline generation on log data, combined with outlier detection methods, the problems of missing labeled samples and neglect of white noise in existing technologies are solved, achieving more accurate user anomaly monitoring.
Patent Information
- Application Number
- CN202310412127.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-14
- Publication Date
- 2026-08-25
- Estimated Expiration
- 2043-04-14
AI Technical Summary
Existing technologies cannot detect user anomalies when labeled abnormal data samples are missing, and ignoring white noise data leads to inaccurate detection results.
The log data to be inspected is divided into time-series data and white noise data by sequence partitioning, and targeted anomaly detection methods are adopted. Dynamic baseline and outlier detection are combined. The Seq2Seq method is used to generate dynamic baseline and the 3-Sigma algorithm is used for white noise detection.
It improves the accuracy of anomaly detection, avoids the problem of missing labeled sample data, and enhances the ability to monitor white noise data.
Smart Images

Figure CN116450482B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of anomaly monitoring technology, and in particular to a user anomaly monitoring method, device, electronic device, and storage medium. Background Technology
[0002] User and entity behavior analysis technology can analyze and monitor user behavior for anomalies, targeting users and entities. This allows for the rapid detection of suspicious and illegal activities by internal users, providing reliable data for security analysts. Currently, anomaly monitoring can be performed by analyzing log data, alerting security analysts when anomalies are detected in the data under inspection.
[0003] In user behavior analysis using log data, supervised algorithms are typically used for anomaly detection. However, machine learning cannot be performed when labeled anomaly data samples are missing, thus hindering anomaly detection. Furthermore, related technologies often neglect white noise data in logs, leading to inaccurate anomaly detection results. Therefore, a method for accurately detecting user anomalies that is not limited by sample data is needed. Summary of the Invention
[0004] In view of the above, this application provides a neurostimulator and its control method to overcome the deficiencies in the related technologies. The technical solution of this application is as follows:
[0005] According to an embodiment of the first aspect of this application, a user anomaly monitoring method is provided, comprising:
[0006] Obtain the log data to be inspected from the target user, divide the log data to be inspected into sequences to obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set, but there is no time-series relationship between the data in the white noise data set.
[0007] Based on the deviation between the data in the time series dataset and the dynamic baseline, abnormal operation monitoring is performed on the time series data to be inspected. The dynamic baseline is predicted based on historical time series data.
[0008] Anomaly detection is performed on the white noise data to be tested based on an outlier detection method. The standard deviation used in the outlier detection method is calculated based on historical white noise data.
[0009] Optionally, the historical time-series data and the historical white noise data are the historical time-series data and historical white noise data of the target user.
[0010] Optionally, it also includes: standardizing the log data to be inspected, the historical time-series data, and the historical white noise data to obtain standardized new data, and predicting the dynamic baseline based on the standardized new data.
[0011] Optionally, the step of performing sequence partitioning on the log data to be inspected includes:
[0012] The log data to be inspected is subjected to stationary sequence detection to obtain the stationary sequence in the log data to be inspected.
[0013] Autocorrelation detection was performed on the stationary sequence to obtain the probability distribution of the statistic;
[0014] If the probability distribution of the statistic is greater than a preset threshold, the log data to be inspected is determined to be white noise data and is included in the white noise data set to be inspected.
[0015] If the probability distribution of the statistic is less than or equal to a preset threshold, the log data to be inspected is determined to be time-series data and is included in the time-series data set to be inspected.
[0016] Optionally, the dynamic baseline is generated by learning the historical distribution of the target user's historical time-series data using the Seq2Seq method, and predicting the dynamic baseline based on the learning results.
[0017] Optionally, the outlier detection method for monitoring abnormal operations in the white noise data to be tested includes:
[0018] When the white noise data to be tested follows a normal distribution, abnormal operation monitoring is performed using the 3-Sigma algorithm.
[0019] When the white noise data to be tested does not follow a normal distribution, abnormal operation monitoring is performed using a preset threshold.
[0020] Optional, also includes:
[0021] An alarm is triggered if the monitoring results of the abnormal operation monitoring indicate that the number of abnormal operations reaches a preset threshold; and / or,
[0022] An alarm is triggered when the monitoring results of the abnormal operation monitoring show that a user in the preset key user list has engaged in abnormal operation.
[0023] According to an embodiment of the second aspect of this application, a user anomaly monitoring device is provided, comprising:
[0024] The sequence partitioning unit is used to acquire the log data to be inspected from the target user, and to partition the log data to be inspected into a sequence to obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set, but there is no time-series relationship between the data in the white noise data set.
[0025] An anomaly monitoring unit is used to monitor abnormal operations in the time series data to be inspected based on the deviation between the data in the time series data set and the dynamic baseline, wherein the dynamic baseline is predicted based on historical time series data.
[0026] The anomaly monitoring unit is also used to monitor abnormal operations in the white noise data to be tested based on the outlier detection method, wherein the standard deviation used by the outlier detection method is calculated based on historical white noise data.
[0027] According to an embodiment of the third aspect of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the abnormal user monitoring method as described in the first aspect.
[0028] According to an embodiment of the fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the abnormal user monitoring method as described in the first aspect.
[0029] In the technical solution provided in this application, after obtaining the log data to be detected from the target user, the log data to be detected is first divided into different data sets based on whether there is a temporal relationship between the data. Based on a dynamic baseline predicted in advance from historical time-series data, the deviation between the time-series data to be detected and the dynamic baseline can accurately detect abnormal operations contained in the time-series data to be detected. Simultaneously, an outlier detection method can accurately detect abnormal operations contained in the white noise data to be detected. Applying the technical solution of this application, time-series data and white noise data in the log data to be detected can be distinguished through sequence partitioning, and targeted anomaly detection methods can be applied to data in different data sets, improving the accuracy of anomaly detection. Furthermore, by predicting the dynamic baseline based on historical log data and calculating the standard deviation used in the outlier detection method, and applying it to the anomaly detection of the log data to be detected, the problem of not being able to use supervised algorithms for monitoring due to the lack of labeled abnormal sample data is avoided.
[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the embodiments of this application. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0032] Figure 1 This is a flowchart illustrating a user anomaly monitoring method according to an exemplary embodiment of this application;
[0033] Figure 2 This is a flowchart illustrating a user anomaly monitoring method according to an exemplary embodiment of this application;
[0034] Figure 3 This is a schematic diagram of a user anomaly monitoring device shown in an exemplary embodiment of this application;
[0035] Figure 4 This is a schematic diagram of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation
[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0037] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0038] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0039] Logs are a way to record various events in a computer system, such as user logins, network activity, and security incidents. Log files are typically stored in the system and can be used by administrators and other users to diagnose problems and monitor system status. User and Entity Behavior Analytics (UEBA) is a security analysis method based on machine learning and data analytics. It improves security and reduces risk by modeling and analyzing the behavior of users and entities (such as devices and applications). Examples of potential security threats include unusual login behavior, data breaches, and insider threats. UEBA typically requires large amounts of data and currently analyzes user and entity behavior based on network, system, and application logs. These logs can be used to build models that identify differences between normal and abnormal behavior.
[0040] In related technologies, when performing user behavior analysis using log data as samples, supervised machine learning algorithms are typically employed. This involves training the model using known input and output samples, enabling the model to predict the output of new input samples. However, in the absence of labeled outlier data samples, it is impossible to train the model using corresponding data samples from the known dataset, thus hindering anomaly detection.
[0041] Meanwhile, current log analysis technologies typically do not focus on white noise data within logs. White noise data lacks interdependencies, and historical data has no impact on future developments; such sequences are unpredictable. Therefore, related technologies usually only analyze log data with temporal relationships. However, some special types of logs, such as 4A audit logs (i.e., unified security management platform solutions), contain a significant amount of white noise data. Ignoring user behavior data within this white noise data can lead to inaccurate anomaly detection results, impacting the effectiveness of anomaly monitoring.
[0042] To address the aforementioned issues, this application proposes a user anomaly monitoring method, apparatus, electronic device, and storage medium. It can distinguish between time-series data and white noise data in the log data to be inspected through sequence partitioning, and employs targeted anomaly monitoring methods for data in different datasets, thereby improving the accuracy of anomaly monitoring. Simultaneously, it predicts a dynamic baseline based on historical log data, calculates the standard deviation used in outlier detection methods, and applies it to anomaly monitoring of the log data to be inspected, avoiding the problem of being unable to use supervised algorithms for monitoring due to the lack of labeled anomaly sample data. The embodiments of this application will be described in detail below.
[0043] Figure 1This is a flowchart illustrating an exemplary embodiment of a user anomaly monitoring method, the method comprising:
[0044] S101, Obtain the log data to be inspected from the target user, perform sequence partitioning on the log data to be inspected to obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set, but no time-series relationship between the data in the white noise data set.
[0045] S1021, Based on the deviation between the data in the time series data set and the dynamic baseline, abnormal operation monitoring is performed on the time series data to be inspected, wherein the dynamic baseline is predicted based on historical time series data;
[0046] S1022, Based on the outlier detection method, abnormal operation monitoring is performed on the white noise data to be tested. The standard deviation used by the outlier detection method is calculated based on historical white noise data.
[0047] The following will be coordinated Figure 2 The specific process shown here provides a brief explanation of a user anomaly monitoring method illustrated in an exemplary embodiment:
[0048] S201, Obtain the log data to be inspected for the target user.
[0049] In one exemplary embodiment, the target user is a user who needs anomaly monitoring, such as an operator of a server, database, or other network device. When these systems or network devices are running, they record the user's operations in a log. In one exemplary embodiment, the log can record relevant information for each operation in the system, specifically including the user account, operation time, operation content, and IP address. In practical use, technicians can set the information to be recorded in the log according to actual needs; this application does not impose any restrictions on this.
[0050] In one exemplary embodiment, the range of log data to be inspected and the range of historical log data can be customized. For example, nearly 7 days of log data can be used as the log data to be inspected to monitor abnormal operations; nearly 30 days of log data can be used as historical log data for analyzing and learning user behavior data. The more historical log data used for machine learning, the higher the prediction accuracy of the obtained dynamic baseline, and the closer the standard deviation used in outlier detection is to user characteristics, thereby improving the effectiveness of abnormal operation monitoring. Those skilled in the art can define the range of log data to be inspected and historical log data according to actual needs; this application does not impose any limitations on this.
[0051] S202, standardize the acquired log data.
[0052] In one exemplary embodiment, the raw data recorded in the acquired logs can be analyzed before data processing to classify the user's specific operations. For example, operation types that need to be focused on can be defined, and log information of that operation type can be extracted. This embodiment will take the user's sensitive operations as an example for detailed explanation.
[0053] Sensitive user actions can include a variety of actions, such as accessing encrypted information or changing operation permissions. During data processing, logs containing relevant actions can be extracted, and the number of times a user performs sensitive actions within a certain time period can be counted to obtain the sensitive action statistics shown in Table 1. The first column is the user name, the second column onwards is the time period (taking one day as an example), and the subsequent values are the number of sensitive actions that occurred on that day.
[0054] User1 0 0 0 121 …… 59 32 285 User2 40 125 84 17 …… 365 0 75 User3 15 0 0 21 …… 25 14 9 …… …… …… …… …… …… …… …… ……
[0055] Table 1
[0056] To ensure the normal convergence of the trained model and obtain a better machine learning model, the statistically derived dataset can be standardized. Here, data standardization refers to converting data into values within a certain standard range, making the values of different features comparable. In this embodiment, the mean and variance of the time series for each user are calculated during standardization, resulting in a corresponding data series as new data. Subsequent method steps are then implemented based on this standardized new data. In practical use, technicians can choose specific data standardization methods as needed, such as Z-score standardization or linear proportional standardization. Different standardization methods will have different impacts on the model training results, and this application does not impose any restrictions on this.
[0057] It should be noted that the data objects for standardization processing in this embodiment may include the log data to be inspected, and may also include historical log data. Specifically, historical log data may include historical time-series data and historical white noise data. Standardizing historical log data makes the value ranges of the log data similar, making it easier for machine learning models to converge during training, thereby improving the speed of constructing a dynamic baseline based on historical log data. It also makes the influence of data features on the model more balanced, thereby improving the accuracy of the constructed dynamic baseline. Simultaneous standardization of both the log data to be inspected and historical log data unifies the ranges of the two data, making them comparable and ensuring the accuracy of anomaly detection.
[0058] S203, determine whether the log data to be inspected is white noise data, that is, perform sequence partitioning on the log data to be inspected.
[0059] In an exemplary embodiment, after obtaining the log data to be inspected from the target user, it is necessary to perform sequence partitioning of the log data to be inspected based on the temporal relationship between the data to obtain a time-series data set to be inspected and a white noise data set to be inspected. The data in the time-series data set have a temporal relationship, while the data in the white noise data set do not have a temporal relationship.
[0060] Specifically, since white noise data is a stationary sequence, we can first perform stationary sequence detection on the acquired log data to be inspected to obtain the stationary sequence in the log data to be inspected, and then perform autocorrelation detection on the obtained stationary sequence. If there is no autocorrelation between the data in the stationary sequence, it can be determined to be white noise data.
[0061] In one exemplary embodiment, the unit root is first used to detect whether the log data to be tested is a stationary series. For example, it can be initially assumed that the time series to be tested has a unit root, i.e., the log data to be tested is a non-stationary series. Then, the lag order is determined based on the characteristics of the time series. Next, the time series is used as the dependent variable, and the lagged time series is used as the independent variable to construct a regression model. The constructed regression model is then used to calculate the t-statistic, the probability value p corresponding to the t-statistic, and the significance test statistics at different confidence levels. In one exemplary embodiment, if the value of p is less than 0.05 and the value of t is less than the significance test statistic at a 5% confidence level, the null hypothesis of the existence of a unit root can be rejected, i.e., the log data to be tested is considered a stationary series. It is understandable that technicians may choose appropriate stationary sequence test methods according to different situations, such as KPSS (Kwiatkowski-Phillips-Schmidt-Shin) test, PP (Phillips-Perron) test, and DF-GLS (Dickey-Fuller Generalized Least Squares) test, etc., and may also make adaptive adjustments to the values used for judgment in specific test methods according to actual needs. This application does not impose any restrictions on this.
[0062] After identifying the stationary sequence, autocorrelation detection is required to separate the final white noise data. In an exemplary embodiment, it can be initially assumed that the data in the stationary sequence does not exhibit autocorrelation. Then, an order is selected, and the statistic of the stationary sequence at that order is calculated. The probability of this statistic distributed in a chi-square with k degrees of freedom is then taken as p, where k is the lag order of the autocorrelation function. If the value of p is less than or equal to a preset threshold, the null hypothesis that no autocorrelation exists can be rejected, meaning the data exhibits autocorrelation, and it is classified into the time series data set. If the value of p is greater than the preset threshold, the null hypothesis that no autocorrelation exists can be affirmed, meaning the data does not exhibit autocorrelation, and it is classified into the white noise data set. It is understood that the smaller the preset threshold, the higher the requirement for autocorrelation between data. Technicians can adjust this threshold according to the log data to be inspected; for example, it can be 0.05, 0.01, etc. This application does not impose any limitations on this.
[0063] It should be noted that the sequence partitioning method described here can be used not only to partition the log data to be inspected, but also to partition historical log data. The sequence partitioning method used in this embodiment comprehensively considers factors such as the characteristics of the data in the time series and the sample size of the user log data, which can accurately determine the stationarity of the sequence and further accurately partition the time series data and white noise data, thus helping to ensure the accuracy of the anomaly monitoring results.
[0064] S2041, if the log data to be inspected is not white noise data, i.e., the log data to be inspected is time-series data, then based on the deviation of the data from the dynamic baseline, abnormal operation monitoring is performed on the data. The dynamic baseline is predicted based on historical time-series data. The dynamic baseline is a method for time series prediction that uses intelligent algorithms for deep learning based on historical data to accurately predict the value at each time point within a future period. The predicted value is then used as the baseline, and anomalies are detected by comparing the deviation of the actual value from the baseline.
[0065] In one exemplary embodiment, the mean absolute error (MAE) can be obtained based on the user's historical log data. The tolerance for prediction deviation can be controlled by control parameters, and the upper and lower boundaries of the dynamic baseline are defined based on the MAE and the control parameters. If the log data to be inspected exceeds the upper and lower boundaries of the dynamic baseline, it is determined to be abnormal data. It is understood that technicians can adjust the values of the aforementioned control parameters, thereby adjusting the upper and lower boundaries of the dynamic baseline to meet different anomaly monitoring needs; this application does not impose any limitations on this.
[0066] In an exemplary embodiment, a dynamic baseline can be generated by learning the historical distribution of the target user's historical time-series data using a Seq2Seq method and predicting the dynamic baseline based on the learning results. Seq2Seq (Sequence to Sequence) is a learning method for natural language processing, currently mainly used in machine translation, speech recognition, and automatic summarization. This method mainly consists of two parts: an encoder and a decoder. The encoder encodes the input information, converting it into a vector form, and then the decoder decodes this vector, gradually restoring it to the output sequence. When generating each target element, this method inputs its previous element and the current vector into a recurrent neural network model, then generates the next element based on the model's output, repeating this process until the entire output sequence is generated. This application uses log data for anomaly monitoring, which requires the simultaneous prediction of a large number of sequences. Compared with other traditional sequence analysis models, such as the Autoregressive Model (AR) and the Autoregressive Integrated Moving Average (ARIMA) model, the Seq2Seq method can achieve one-step fitting and batch prediction without the need for batch calculation of multiple time series, thus reducing computation time. At the same time, it does not require specific parameter adjustments for each time series, simplifying data analysis and improving the work efficiency of security analysts.
[0067] S2042, If the log data to be inspected is white noise data, then based on the outlier detection method, abnormal operation monitoring is performed on the data. The standard deviation used by the outlier detection method is calculated based on historical white noise data.
[0068] In an exemplary embodiment, the white noise data to be tested can be further divided, and different methods can be used for anomaly monitoring. When the white noise data to be tested follows a normal distribution, anomaly monitoring can be performed using the 3-Sigma algorithm. The 3-Sigma algorithm, also known as the 3σ criterion, states that if the data follows a normal distribution, an outlier is defined as a value whose deviation from the mean exceeds three times the standard deviation. Here, the mean and standard deviation are calculated separately for historical white noise data. When the white noise data to be tested does not follow a normal distribution, it can be compared with a preset threshold to determine whether it is an anomaly. The preset threshold can be dynamic and calculated based on historical log data. For example, if the maximum value in the historical log data is 200, and 80% of that value is taken as the preset threshold, then white noise data exceeding 160 that does not follow a normal distribution is considered anomaly data. Alternatively, the preset threshold can be a fixed value defined by a technician. In practical use, specific settings can be made as needed to meet different anomaly monitoring requirements; this application does not impose any limitations on this. The anomaly monitoring method for white noise data in this embodiment can further divide the white noise data and use different methods to monitor anomalies, making the anomaly monitoring of the white noise data under test more comprehensive and accurate.
[0069] The user anomaly monitoring method in this application embodiment can distinguish between time-series data and white noise data in the log data to be inspected by sequence partitioning, and adopt targeted anomaly monitoring methods for data in different data sets, thereby improving the accuracy of anomaly monitoring. At the same time, it predicts dynamic baselines based on historical log data, calculates the standard deviation used by outlier detection methods, and uses it for anomaly monitoring of the log data to be inspected, avoiding the problem that supervised algorithms cannot be used for monitoring due to the lack of labeled anomaly sample data.
[0070] In one exemplary embodiment, the historical time-series data and the historical white noise data are the historical time-series data and historical white noise data of the target user. That is, the construction of the dynamic baseline and the calculation of the standard deviation used for outlier detection are based on the historical log data of the target user. Therefore, each user has a dynamic baseline and values used in outlier detection tailored to their individual characteristics, allowing the anomaly detection method to better fit user characteristics and thus detect abnormal operations for that user, improving the accuracy of anomaly detection. In another exemplary embodiment, the historical log data of all users can also be uniformly analyzed to obtain a universal dynamic baseline and values used in outlier detection, which can then be used to monitor abnormal operations of all users. This approach simplifies the preparation work before conducting anomaly detection and saves computer resources.
[0071] S205, an alarm is triggered when the monitoring results of abnormal operation indicate that abnormal operation has occurred.
[0072] In one exemplary embodiment, specific abnormal operation types can be prioritized. Specifically, an alarm can be triggered when the monitoring results of abnormal operation detection show that the number of abnormal operations reaches a preset threshold; an alarm can be triggered when the monitoring results of abnormal operation detection show that a user on a preset key user list has engaged in abnormal operation; or both of the above situations can be monitored simultaneously, or a combination of both, i.e., an alarm can be triggered when the monitoring results show that the number of abnormal operations by a user on the preset key user list reaches a preset threshold. In practical use, the types of abnormal operations to be prioritized can be selected according to needs, and specific alarm rules can be set; alternatively, no distinction can be made between abnormal operation types, i.e., an alarm can be triggered whenever any abnormal operation is detected. This embodiment of the application allows for alarms to be triggered for specific types of abnormal operations, enabling security analysts to quickly identify key anomalies and handle them promptly, thus improving work efficiency.
[0073] Based on the user anomaly monitoring method provided in this application, this application also provides a user anomaly monitoring device, see [link to relevant documentation]. Figure 3 ,include:
[0074] The sequence partitioning unit 31 is configured to acquire the log data to be inspected from the target user, partition the log data to be inspected into a sequence, and obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set and no time-series relationship between the data in the white noise data set.
[0075] Anomaly monitoring unit 32 is configured to monitor abnormal operations in the time series data to be inspected based on the deviation between the data in the time series data set and the dynamic baseline, wherein the dynamic baseline is predicted based on historical time series data.
[0076] The anomaly monitoring unit 32 is also configured to monitor abnormal operations in the white noise data to be tested based on an outlier detection method, wherein the standard deviation used by the outlier detection method is calculated based on historical white noise data.
[0077] Optionally, the historical time-series data and the historical white noise data are the historical time-series data and historical white noise data of the target user.
[0078] Optionally, the user anomaly monitoring device further includes a processing unit 33, configured to: perform standardization processing on the log data to be inspected, the historical time-series data, and the historical white noise data to obtain standardized new data, and predict the dynamic baseline based on the standardized new data.
[0079] Optionally, sequence partitioning unit 31 is specifically configured as follows:
[0080] The log data to be inspected is subjected to stationary sequence detection to obtain the stationary sequence in the log data to be inspected.
[0081] Autocorrelation detection was performed on the stationary sequence to obtain the probability distribution of the statistic;
[0082] If the probability distribution of the statistic is greater than a preset threshold, the log data to be inspected is determined to be white noise data and is included in the white noise data set to be inspected.
[0083] If the probability distribution of the statistic is less than or equal to a preset threshold, the log data to be inspected is determined to be time-series data and is included in the time-series data set to be inspected.
[0084] Optionally, the dynamic baseline is generated by learning the historical distribution of the target user's historical time-series data using the Seq2Seq method, and predicting the dynamic baseline based on the learning results.
[0085] Optionally, the anomaly monitoring unit 32 is specifically configured as follows:
[0086] When the white noise data to be tested follows a normal distribution, abnormal operation monitoring is performed using the 3-Sigma algorithm.
[0087] When the white noise data to be tested does not follow a normal distribution, abnormal operation monitoring is performed using a preset threshold.
[0088] Optionally, the user anomaly monitoring device further includes an alarm unit 34, configured to:
[0089] An alarm is triggered if the monitoring results of the abnormal operation monitoring indicate that the number of abnormal operations reaches a preset threshold; and / or,
[0090] An alarm is triggered when the monitoring results of the abnormal operation monitoring show that a user in the preset key user list has engaged in abnormal operation.
[0091] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.
[0092] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0093] Accordingly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the control method described in any of the above embodiments.
[0094] refer to Figure 4 At the hardware level, the electronic device includes a processor 402, an internal bus 404, a network interface 406, memory 408, and non-volatile memory 410, and may also include other hardware required for business operations. The processor 402 reads the corresponding computer program from the non-volatile memory 410 into the memory 408 and then runs it, forming a device at the logical level for dividing log data into sequences and monitoring user anomalies separately. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.
[0095] Accordingly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method as described in any of the above embodiments.
[0096] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.
[0097] The above description is only a specific implementation of the embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the embodiments of this application, and these improvements and modifications should also be considered as the protection scope of the embodiments of this application.
Claims
1. A method for monitoring user anomalies, characterized in that, include: Obtain the log data to be inspected from the target user, divide the log data to be inspected into sequences to obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set, but there is no time-series relationship between the data in the white noise data set. Based on the deviation between the data in the time series dataset and the dynamic baseline, abnormal operation monitoring is performed on the time series data to be inspected. The dynamic baseline is predicted based on historical time series data. Anomaly detection is performed on the white noise data to be tested based on an outlier detection method. The standard deviation used in the outlier detection method is calculated based on historical white noise data.
2. The method according to claim 1, characterized in that, The historical time-series data and the historical white noise data are the historical time-series data and historical white noise data of the target user.
3. The method according to claim 1, characterized in that, Also includes: The log data to be inspected, the historical time-series data, and the historical white noise data are standardized to obtain standardized new data, and the dynamic baseline is predicted based on the standardized new data.
4. The method according to claim 1, characterized in that, The step of performing sequence partitioning on the log data to be inspected includes: The log data to be inspected is subjected to stationary sequence detection to obtain the stationary sequence in the log data to be inspected. Autocorrelation detection was performed on the stationary sequence to obtain the probability distribution of the statistic; If the probability distribution of the statistic is greater than a preset threshold, the log data to be inspected is determined to be white noise data and is included in the white noise data set to be inspected. If the probability distribution of the statistic is less than or equal to a preset threshold, the log data to be inspected is determined to be time-series data and is included in the time-series data set to be inspected.
5. The method according to claim 1, characterized in that, The dynamic baseline is generated by learning the historical distribution of the target user's historical time-series data using the Seq2Seq method and predicting the dynamic baseline based on the learning results.
6. The method according to claim 1, characterized in that, The outlier detection method is used to monitor abnormal operations in the white noise data to be tested, including: When the white noise data to be tested follows a normal distribution, abnormal operation monitoring is performed using the 3-Sigma algorithm. When the white noise data to be tested does not follow a normal distribution, abnormal operation monitoring is performed using a preset threshold.
7. The method according to claim 1, characterized in that, Also includes: An alarm is triggered when the monitoring results of the abnormal operation monitoring show that the number of abnormal operations reaches a preset threshold. And / or, An alarm is triggered when the monitoring results of the abnormal operation monitoring show that a user in the preset key user list has engaged in abnormal operation.
8. A user anomaly monitoring device, characterized in that, include: The sequence partitioning unit is used to acquire the log data to be inspected from the target user, and to partition the log data to be inspected into a sequence to obtain a time-series data set to be inspected and a white noise data set to be inspected. There is a time-series relationship between the data in the time-series data set, but there is no time-series relationship between the data in the white noise data set. An anomaly monitoring unit is used to monitor abnormal operations in the time series data to be inspected based on the deviation between the data in the time series data set and the dynamic baseline, wherein the dynamic baseline is predicted based on historical time series data. The anomaly monitoring unit is also used to monitor abnormal operations in the white noise data to be tested based on the outlier detection method, wherein the standard deviation used by the outlier detection method is calculated based on historical white noise data.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the user anomaly monitoring method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the user anomaly monitoring method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Abnormity detection method and device, computer equipment and storage medium
CN110888788A
Abnormity recognition device based on big data and artificial intelligence
CN114781715A