Anomaly detection method and related apparatus

Abstract

The application discloses an abnormality detection method and related device. The method comprises: extracting user behavior feature data according to a user login log, wherein the user behavior feature data is used as a detection standard for whether a current user login behavior is abnormal based on a preset time window; establishing a preset abnormality detection model based on the user behavior feature data, wherein the preset abnormality detection model is obtained by unsupervised machine learning training using multiple sets of data, and each set of data in the multiple sets of data comprises: small sample data, wherein the small sample data comprises data of user abnormal login behavior; and detecting a user abnormal login behavior by using the preset abnormality detection model. The application solves the technical problems of high false alarm rate of abnormality detection and difficulty in constructing samples. The application realizes accurate monitoring of user abnormal behavior, and the small sample data can be used for training of the model based on an unsupervised training mode.

Description

Technical Field

[0001] This application relates to the field of cybersecurity, and more specifically, to an anomaly detection method and related apparatus. Background Technology

[0002] User login logs record information such as login time, login device, and login IP. This data often records attacker behavior and can be used to detect abnormal behavior.

[0003] Anomaly detection for user login behavior mainly focuses on detecting logins from different locations, but the false positive rate is relatively high. Furthermore, since anomaly login behavior does not occur frequently in real-world network environments, training a detection model requires a large amount of sample data.

[0004] There is currently no effective solution to the problems of high false alarm rate and difficulty in constructing samples in related technologies for anomaly detection. Summary of the Invention

[0005] The main objective of this application is to provide an anomaly detection method and related apparatus to solve the problems of high false alarm rate and difficulty in constructing samples for anomaly detection.

[0006] To achieve the above objectives, according to one aspect of this application, an anomaly detection method is provided.

[0007] The anomaly detection method according to this application includes: extracting user behavior feature data from user login logs, wherein the user behavior feature data is used as a detection standard for whether the current user login behavior is abnormal based on a preset time window; establishing a preset anomaly detection model based on the user behavior feature data, wherein the preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning, and each set of data includes: small sample data, wherein the small sample data includes data on abnormal user login behavior; and detecting abnormal user login behavior through the preset anomaly detection model.

[0008] Furthermore, user behavior feature data is extracted based on user login logs. The user behavior feature data is used as a detection standard for whether the current user login behavior is abnormal based on a preset time window, including: extracting user behavior feature data based on a preset time window of 1 day as the detection standard for whether the current user login behavior is abnormal, and detecting and analyzing whether the user login behavior within that day is abnormal.

[0009] Furthermore, the step of extracting user behavior feature data based on user login logs also includes: establishing a corresponding user profile information table for each user, and statistically analyzing the user login time period, login method, and user login location for each user; calculating the frequency of occurrence of each preset user feature value for the user login time period, login method, and user login location, wherein the login method includes a triple <login system, login browser, application used for login>; and marking whether the log entry is a frequently used login time, a frequently used login method is abnormal, a frequently used login location, and an IP address is abnormal, based on the user profile information table and the frequency of occurrence.

[0010] Furthermore, the preset user feature values ​​extracted from the user behavior feature data include at least one of the following: the total number of logins by the user in a given day, the number of source IPs for logins by the account in a given day, the number of cities for logins by the account in a given day, the number of devices used for logins by the account in a given day, the number of browsers used for logins by the account in a given day, the number of login entry points for the account in a given day, the proportion of logins marked as frequently used login times in a given day, the proportion of logins marked as abnormal frequently used login methods in a given day, the proportion of logins marked as frequently used login locations in a given day, and the proportion of logins marked as abnormal IP addresses in a given day.

[0011] Furthermore, based on the user behavior feature data, a preset anomaly detection model is established. This preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning. Each set of data includes: small sample data, which includes data on abnormal user login behavior; and an autoencoder using a sparse autoencoder, where the unsupervised machine learning employs an encoder and a decoder. Data is judged to be abnormal when the output matrix is ​​greater than a preset threshold after being input into the autoencoder via an input matrix.

[0012] Furthermore, the autoencoder further includes: minimizing the error between the output reconstructed data and the original data by adjusting the parameters of the encoder and the decoder; and reconstructing the original data through training using the autoencoder.

[0013] Furthermore, the reconstruction of the original data through training using the autoencoder further includes: adding a dropout layer when building the model to prevent overfitting, and / or adding a sparse penalty term to the loss function of the autoencoder.

[0014] To achieve the above objectives, according to another aspect of this application, an anomaly detection device is provided.

[0015] The anomaly detection device according to this application includes: an extraction module, used to extract user behavior feature data based on user login logs, wherein the user behavior feature data is used as a detection standard for whether the current user login behavior is abnormal based on a preset time window; an unsupervised model building module, used to build a preset anomaly detection model based on the user behavior feature data, wherein the preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning, and each set of data includes: small sample data, wherein the small sample data includes data on abnormal user login behavior; and an anomaly detection module, used to detect abnormal user login behavior through the preset anomaly detection model.

[0016] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program is configured to execute the method at runtime.

[0017] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method.

[0018] In the embodiments of this application, the anomaly detection method and related apparatus extract user behavior feature data from user login logs. By establishing a preset anomaly detection model based on the user behavior feature data, the method achieves the purpose of detecting abnormal user login behavior through the preset anomaly detection model. This achieves the technical effect of accurately monitoring abnormal user login behavior and training the detection model with a small sample, thereby solving the technical problems of high false alarm rate and difficulty in constructing samples for anomaly detection. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0020] Figure 1 This is a schematic diagram of the hardware structure of an anomaly detection method according to an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating an anomaly detection method according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the structure of an anomaly detection device according to an embodiment of this application;

[0023] Figure 4 This is a flowchart illustrating an anomaly detection method according to a preferred embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0027] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0028] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linking," and "socketing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0029] During their research, the inventors discovered that the most commonly used method for detecting abnormal user login behavior is location-based login detection. This involves remembering the cities a user frequently logs into and triggering protective measures such as CAPTCHA or SMS verification when the login location changes. However, since the login location is generally determined based on the IP address, this method is highly prone to false alarms when the user's broadband provider has NAT exits in multiple locations across the country, or when the user is using a VPN or proxy.

[0030] To improve the accuracy of abnormal behavior detection, in the embodiments of this application, user behavior features are extracted from multiple dimensions in combination with user profiles for subsequent anomaly detection.

[0031] Furthermore, in real-world network environments, abnormal login behavior is infrequent, leading to a highly imbalanced distribution of normal and abnormal samples, resulting in a lack of abnormal data when constructing anomaly detection models. This patent addresses this problem by employing an unsupervised learning algorithm. Unsupervised learning algorithms have advantages such as not relying on a large number of abnormal samples as training data and being able to detect unknown abnormal behaviors.

[0032] Therefore, in order to address the shortcomings of simply detecting logins from different locations, this patent analyzes user login behavior habits and constructs user profiles. By combining users' infrequent login locations, infrequent login times, and infrequent login behaviors, user login behavior features are generated within a one-day time window for subsequent anomaly detection model building, achieving multi-dimensional anomaly detection and improving accuracy.

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] like Figure 1 As shown, the embodiment of the anomaly detection method of this application includes: user 100 and server 200. The login log in the server 200 records relevant information such as user login time, login device, and login IP. This data often records the attacker's attack behavior, and analyzing the log can be used as an effective means of detecting malicious login.

[0035] like Figure 2As shown, the method includes the following steps S201 to S203:

[0036] Step S201: Extract user behavior feature data from user login logs, wherein the user behavior feature data is based on a preset time window as the detection standard for whether the current user login behavior is abnormal.

[0037] Step S202: Based on the user behavior feature data, establish a preset anomaly detection model, wherein the preset anomaly detection model is obtained by training multiple sets of data through unsupervised machine learning, and each set of data includes: small sample data, wherein the small sample data includes data on abnormal user login behavior;

[0038] Step S203: Detect abnormal user login behavior using the preset anomaly detection model.

[0039] As can be seen from the above description, this application achieves the following technical effects:

[0040] By extracting user behavior feature data from user login logs, and establishing a preset anomaly detection model based on the user behavior feature data, the method achieves the goal of detecting abnormal user login behavior through the preset anomaly detection model. This achieves the technical effect of accurately monitoring abnormal user login behavior and training the detection model with a small sample, thereby solving the technical problems of high false alarm rate and difficulty in constructing samples for anomaly detection.

[0041] In step S201 above, user behavior feature data is extracted from the user login log in the server's log file.

[0042] As an optional implementation, the user behavior feature data is used as a detection standard for whether the current user's login behavior is abnormal, based on a preset time window.

[0043] As a preferred implementation, user behavior feature data includes, but is not limited to: the total number of logins by the user in a given day, the number of source IPs for logins by the account in a given day, the number of cities for logins by the account in a given day, the number of devices used for logins by the account in a given day, the number of browsers used for logins by the account in a given day, the number of login entry points for the account in a given day, the proportion of logins marked as "frequently used login times" in a given day, the proportion of logins marked as "frequently used login methods are abnormal" in a given day, the proportion of logins marked as "frequently used login locations" in a given day, and the proportion of logins marked as "abnormal IPs" in a given day.

[0044] In step S202 above, a preset anomaly detection model is established based on the user behavior feature data.

[0045] As an optional implementation, the preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning. Unsupervised learning algorithms do not rely on a large number of anomalous samples as training data and can still discover unknown anomalous behaviors.

[0046] As an optional implementation, each of the multiple datasets includes a small sample dataset, which comprises data on abnormal user login behavior. Due to the infrequent nature of attacks and the scarcity of labeled data, this paper employs an unsupervised learning algorithm. The adopted unsupervised anomaly detection algorithm does not rely on prior attack knowledge, thus allowing the system to discover rare and previously undetected threats.

[0047] In step S203 above, abnormal user login behavior is detected using the preset anomaly detection model. After the autoencoder of the preset anomaly detection model is constructed, the feature matrix of the log to be tested is constructed and input into the autoencoder. After passing through the autoencoder, the reconstruction error is calculated. If it exceeds a certain threshold, it is considered abnormal data.

[0048] As an optional implementation method, user login behavior features are generated using a one-day time window for subsequent anomaly detection model building, thereby achieving multi-dimensional anomaly detection and improving accuracy.

[0049] As an optional implementation, user login behavior features are generated using a one-day time window for subsequent unsupervised anomaly detection model building, enabling multi-dimensional anomaly detection and improving accuracy.

[0050] In this preferred embodiment, user behavior feature data is extracted from user login logs. The user behavior feature data is used as a detection standard for whether the current user login behavior is abnormal based on a preset time window, including: extracting user behavior feature data based on a preset time window of 1 day as the detection standard for whether the current user login behavior is abnormal, and detecting and analyzing whether the user login behavior within that day is abnormal.

[0051] In specific implementation, a one-day time window is used to analyze whether user login behavior is abnormal within that day. The preset user feature values ​​in the extracted user behavior feature data include at least one of the following: the total number of logins by the user within the day, the number of source IPs for logins by the account within the day, the number of cities for logins by the account within the day, the number of devices used for logins by the account within the day, the number of browsers used for logins by the account within the day, the number of login entry points for the account within the day, the proportion of logins marked as "frequently used login times" within the day, the proportion of logins marked as "frequently used login methods are abnormal" within the day, the proportion of logins marked as "frequently used login locations" within the day, and the proportion of logins marked as "abnormal IPs" within the day.

[0052] As a preferred embodiment, the step of extracting user behavior feature data based on user login logs further includes: establishing a corresponding user profile information table for each user, and statistically analyzing the user login time period, login method, and user login location for each user; calculating the frequency of occurrence of each preset user feature value for the user login time period, login method, and user login location, wherein the login method includes a triple <login system, login browser, application used for login>; and marking whether the log entry is a frequently used login time, a frequently used login method is abnormal, a frequently used login location, and an IP address is abnormal, based on the user profile information table and the frequency of occurrence.

[0053] In practice, a user profile is created for each user, and login time periods are statistically analyzed for each user. Login methods can be represented by a triple <login system, login browser, application used>, and the user's login location. The frequency of each of these features is then calculated.

[0054] It is understandable that the values ​​of the above features include: the total number of logins for the user in a given day, the number of source IPs for logins for the account in a given day, the number of cities for logins for the account in a given day, the number of devices used for logins for the account in a given day, the number of browsers used for logins for the account in a given day, the number of login entry points for the account in a given day, the proportion of logins marked as frequently used login times in a given day, the proportion of logins marked as abnormal frequently used login methods in a given day, the proportion of logins marked as frequently used login locations in a given day, and the proportion of logins marked as abnormal IP addresses in a given day.

[0055] Based on the user profile information table above, we can mark whether the log entry is a frequently logged-in time, whether the frequently logged-in method is abnormal, and whether the frequently logged-in location is common.

[0056] It should be noted that for the logs to be tested, the time of the current log occurrence is first converted into a certain time period, and then the "User Profile Table" is checked. If the threshold of the time period is higher than 0.1, the log can be determined to be a login at a common time.

[0057] For the logs to be tested, the common login methods are determined. The login method (<login system, login browser, which application to use to log in>) is determined to be a common login method for the user. If the threshold of the login method is higher than 0.1, the log can be determined to be a common login method.

[0058] The commonly used login locations are used to determine whether the login location is a user's commonly used login location for the log to be tested. If the threshold of the login location is higher than 0.1, the log can be determined to be a commonly used login location.

[0059] The IP address is considered abnormal, and there are two scenarios for determining this. If an IP address logs into a large number of accounts with a fixed access pattern (device, system, browser), and the login failure frequency is high, the IP address can be marked as abnormal. If an IP address has never appeared in a user's history and the access pattern is not present in the user's records, the user can be asked to confirm again; if the user confirms it is abnormal, it is marked as abnormal.

[0060] In a preferred embodiment, a preset anomaly detection model is established based on the user behavior feature data. This preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning. Each set of data includes: small sample data, which includes data on abnormal user login behavior; and an autoencoder using a sparse autoencoder, where the unsupervised machine learning employs an encoder and a decoder. Data is judged to be abnormal when the output matrix is ​​greater than a preset threshold after being input into the autoencoder via an input matrix.

[0061] In practice, due to the infrequent nature of attacks and the scarcity of labeled data, this paper employs an unsupervised learning algorithm. Furthermore, unsupervised anomaly detection algorithms do not rely on prior attack knowledge, allowing the system to discover rare and previously undetected threats.

[0062] Preferably, a sparse autoencoder can be used. This sparse mode autoencoder is an improvement on the traditional autoencoder, where some neurons are inactive to better extract features. By adjusting the parameters of the encoder and decoder, the error between the reconstructed output data and the original data can be minimized. The autoencoder can be trained to reconstruct the original data.

[0063] As a preferred embodiment, the reconstruction of the original data through training by the autoencoder further includes: adding a dropout layer when building the model to prevent overfitting, and / or adding a sparse penalty term to the loss function of the autoencoder.

[0064] As an alternative implementation, a dropout layer can be added when building the model to prevent overfitting.

[0065] As an alternative implementation, in order to better extract features, a sparse penalty term can be added to the loss function, which can make some neurons in an inactive state.

[0066] In a preferred embodiment, the autoencoder further includes: adjusting the parameters of the encoder and the decoder to minimize the error between the output reconstructed data and the original data; and reconstructing the original data through training using the autoencoder.

[0067] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0068] According to an embodiment of this application, an anomaly detection device for implementing the above method is also provided, such as... Figure 3 As shown, the device includes:

[0069] The extraction module 301 is used to extract user behavior feature data based on user login logs, wherein the user behavior feature data is based on a preset time window as the detection standard for whether the current user login behavior is abnormal.

[0070] The unsupervised model building module 302 is used to build a preset anomaly detection model based on the user behavior feature data. The preset anomaly detection model is trained by unsupervised machine learning using multiple sets of data. Each set of data includes: small sample data, which includes data on abnormal user login behavior.

[0071] The anomaly detection module 303 is used to detect abnormal user login behavior through the preset anomaly detection model.

[0072] In the extraction module 301 of this application, user behavior feature data is extracted from the user login log in the server's log file.

[0073] As an optional implementation, the user behavior feature data is used as a detection standard for whether the current user's login behavior is abnormal, based on a preset time window.

[0074] As a preferred implementation, user behavior feature data includes, but is not limited to: the total number of logins by the user in a given day, the number of source IPs for logins by the account in a given day, the number of cities for logins by the account in a given day, the number of devices used for logins by the account in a given day, the number of browsers used for logins by the account in a given day, the number of login entry points for the account in a given day, the proportion of logins marked as "frequently used login times" in a given day, the proportion of logins marked as "frequently used login methods are abnormal" in a given day, the proportion of logins marked as "frequently used login locations" in a given day, and the proportion of logins marked as "abnormal IPs" in a given day.

[0075] The unsupervised model building module 302 of this application establishes a preset anomaly detection model based on the user behavior feature data.

[0076] As an optional implementation, the preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning. Unsupervised learning algorithms do not rely on a large number of anomalous samples as training data and can still discover unknown anomalous behaviors.

[0077] As an optional implementation, each of the multiple datasets includes a small sample dataset, which comprises data on abnormal user login behavior. Due to the infrequent nature of attacks and the scarcity of labeled data, this paper employs an unsupervised learning algorithm. The adopted unsupervised anomaly detection algorithm does not rely on prior attack knowledge, thus allowing the system to discover rare and previously undetected threats.

[0078] The anomaly detection module 303 of this application detects abnormal user login behavior through the preset anomaly detection model. After the autoencoder of the preset anomaly detection model is constructed, for the log to be tested, the feature matrix is ​​constructed and input into the autoencoder. After passing through the autoencoder, the reconstruction error is calculated. If it exceeds a certain threshold, it is considered abnormal data.

[0079] As an optional implementation method, user login behavior features are generated using a one-day time window for subsequent anomaly detection model building, thereby achieving multi-dimensional anomaly detection and improving accuracy.

[0080] As an optional implementation, user login behavior features are generated using a one-day time window for subsequent unsupervised anomaly detection model building, enabling multi-dimensional anomaly detection and improving accuracy.

[0081] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0082] To better understand the above process, the following explanation of the technical solutions is provided in conjunction with preferred embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention.

[0083] In this embodiment, user login behavior features are extracted, and sparse autoencoders are used to detect abnormal user login behavior based on the small sample data obtained from the extraction.

[0084] User login behavior characteristics may include: extracting user login behavior characteristics, taking one day as a time window, extracting the number of source IPs for login of the account on that day, the number of cities logged in by the account on that day, the number of devices used for login of the account on that day, the number of browsers used for login of the account on that day, the number of login entry points for the account on that day, the proportion of logins marked as "frequent login time" on that day, the proportion of logins marked as "frequent login method abnormal" on that day, the proportion of logins marked as "frequent login location" on that day, and the proportion of logins marked as "IP abnormal" on that day.

[0085] like Figure 4 The diagram shown is a flowchart of the intelligent dialogue interaction method in an embodiment of this application. The specific process includes the following steps:

[0086] Step S401, User login behavior characteristics.

[0087] Step S402: Analyze whether user login behavior is abnormal within a 1-day time window.

[0088] We analyze user login behavior within a 1-day time window to determine if it is abnormal. We extract the following data for the day: total number of logins, number of source IPs for the account, number of cities logged in, number of devices used, number of browsers used, number of login entry points, percentage of logins marked as "frequently used login times," percentage of logins marked as "frequently used login methods are abnormal," percentage of logins marked as "frequently used login locations," and percentage of logins marked as "abnormal IP addresses."

[0089] Create a user profile for each user, and statistically analyze the user's login time period, login method (represented by a triple <login system, login browser, application used to log in>), and login location for each user. Calculate the frequency of each of these features.

[0090] Based on the user profile information table above, we can mark whether the log entry is a frequently logged-in time, whether the frequently logged-in method is abnormal, and whether the frequently logged-in location is common.

[0091] Step S403, unsupervised anomaly detection.

[0092] Because attacks are infrequent and labeled data is scarce, this paper uses an unsupervised learning algorithm. Furthermore, unsupervised anomaly detection algorithms do not rely on prior attack knowledge, allowing the system to discover rare and previously undetected threats.

[0093] This system uses a sparse autoencoder, an improvement on traditional autoencoders that deactivates some neurons to better extract features. The anomaly detection algorithm is as follows:

[0094] A sparse encoder consists of two parts: an encoder and a decoder. The encoder's processing from the input layer to the hidden layer is shown in formula (1), where W... ij Here, j1 is the weight matrix, j1 is the offset, and s is the activation function.

[0095] H = g q1 (X)=s(W ij X+j1) (1)

[0096] The decoding process from the hidden layer to the reconstruction layer is as shown in formula (2), where W... jk Here, j is the weight matrix, j2 is the offset, and s is the activation function.

[0097] Y = g q2 (H)=s(W jk H+j2) (2)

[0098] By adjusting the parameters of the encoder and decoder, the error between the reconstructed output data and the original data can be minimized (Equation (3)). The autoencoder can reconstruct the original data through training. A dropout layer can be added when building the model to prevent overfitting. In the formula, Y... r A matrix composed of real labels, X r is the output value of the model, and N is the length of the elements in the dataset.

[0099]

[0100] To better extract features, a sparse penalty term can be added to the loss function, which can make some neurons in an inactive state, as shown in formula (4). is the average activity of hidden neuron j, r is the sparsity parameter, and m is used to control the weights of sparsity.

[0101]

[0102]

[0103] After the autoencoder is built, for the logs to be tested, after constructing the feature matrix X...(r) The input is fed into an autoencoder, and the result after the autoencoder is Y. (r) The reconstruction error is calculated using formula (3), and data exceeding a certain threshold is considered abnormal data.

[0104] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An anomaly detection method, characterized in that, include: Based on user login logs, user behavior feature data is extracted, including: establishing a corresponding user profile information table for each user, and statistically analyzing the user's login time period, login method, and login location for each user. The login method includes a triple <login system, login browser, which application to use for login>; Calculate the frequency of occurrence of each preset user characteristic value; Based on the user profile information table and the frequency of occurrence, the log entry is marked as to whether it is a common login time, a common login method, a common login location, or an IP address. The user behavior feature data is based on a preset time window of 1 day as the detection standard for whether the current user's login behavior is abnormal. The preset user characteristic values ​​must include at least one of the following: Extract the following data for the user: total number of logins within the day, number of source IPs for logins within the day, number of cities logged in within the day, number of devices used for logins within the day, number of browsers used for logins within the day, number of login entry points for the account within the day, percentage of logins marked as frequently used login times within the day, percentage of logins marked as abnormal frequently used login methods within the day, percentage of logins marked as frequently used login locations within the day, and percentage of logins marked as abnormal IP addresses within the day. Based on the user behavior feature data, a preset anomaly detection model is established. The preset anomaly detection model is trained using multiple sets of data through unsupervised machine learning. Each set of data includes: small sample data, which includes data on abnormal user login behavior. The unsupervised machine learning employs a sparse autoencoder, wherein the autoencoder includes an encoder and a decoder. The encoder's processing from the input layer to the hidden layer is shown in formula (1), where W... ij It is a weight matrix. It's the offset. It is an activation function: （1） The decoding process from the hidden layer to the reconstruction layer is shown in formula (2), where W jk It is a weight matrix ， j2 is the offset. It is an activation function （2） By adjusting the parameters of the encoder and decoder, the error between the reconstructed output data and the original data is minimized, as shown in formula (3). The autoencoder reconstructs the original data through training. A dropout layer is added when building the model to prevent overfitting. In the formula, Y r A matrix composed of real labels, X r is the output value of the model, and N is the length of the elements in the dataset; (3） Add a sparse penalty term to the loss function to make some neurons in an inactive state, as shown in formula (4). It is the average activity level of hidden neuron j. It is a sparsity parameter. Weights used to control sparsity; （4） After the autoencoder is constructed, for the log to be tested, after the feature matrix X(r) is constructed, it is input into the autoencoder. After passing through the autoencoder, Y(r) is obtained. The reconstruction error is calculated using formula (3). If it is greater than a certain threshold, it is considered abnormal data.

2. An anomaly detection device, characterized in that, include: The extraction module is used to extract user behavior feature data based on user login logs, including: building a corresponding user profile information table for each user, and statistically analyzing the user's login time period, login method, and login location for each user. The login method includes a triple <login system, login browser, which application to use for login>; Calculate the frequency of occurrence of each preset user characteristic value; Based on the user profile information table and the frequency of occurrence, the log entry is marked as to whether it is a common login time, a common login method, a common login location, or an IP address. The user behavior feature data is based on a preset time window of 1 day as the detection standard for whether the current user's login behavior is abnormal. The preset user characteristic values ​​must include at least one of the following: Extract the following data for the user: total number of logins within the day, number of source IPs for logins within the day, number of cities logged in within the day, number of devices used for logins within the day, number of browsers used for logins within the day, number of login entry points for the account within the day, percentage of logins marked as frequently used login times within the day, percentage of logins marked as abnormal frequently used login methods within the day, percentage of logins marked as frequently used login locations within the day, and percentage of logins marked as abnormal IP addresses within the day. An unsupervised model building module is used to build a preset anomaly detection model based on the user behavior feature data. The preset anomaly detection model is trained by unsupervised machine learning using multiple sets of data. Each set of data includes: small sample data, which includes data on abnormal user login behavior. By adjusting the parameters of the encoder and decoder, the error between the reconstructed output data and the original data is minimized, as shown in formula (3). The autoencoder reconstructs the original data through training. A dropout layer is added when building the model to prevent overfitting. In the formula, Y r A matrix composed of real labels, X r is the output value of the model, and N is the length of the elements in the dataset; (3） Add a sparse penalty term to the loss function to make some neurons in an inactive state, as shown in formula (4). It is the average activity level of hidden neuron j. It is a sparsity parameter. Weights used to control sparsity; （4）； The anomaly detection module is used to detect abnormal user login behavior through the preset anomaly detection model. For the log to be tested, after constructing the feature matrix X(r), it is input into the autoencoder. After passing through the autoencoder, Y(r) is obtained. The reconstruction error is calculated using formula (3). If it is greater than a certain threshold, it is considered abnormal data.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method of claim 1 at runtime.

4. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of claim 1.

Images