Density clustering-based detection method and device for abnormal login behavior of email accounts
Through the detection method based on density clustering, the DBSCAN algorithm and information gain algorithm are used to identify abnormal login behavior of mailbox accounts, which solves the problem of misjudgment and relying on external threat intelligence databases in the existing technology, and achieves high accuracy and rapid detection effect.
Patent Information
- Application Number
- CN202110673803.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-17
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-06-17
AI Technical Summary
The existing technology has many misjudgments and is inaccurate when detecting abnormal login behavior of email account, and cannot effectively identify hidden abnormal login behavior. It relies on external threat intelligence database and is not updated in time, which affects accuracy.
Using density clustering-based detection method, the target feature information is extracted for feature processing and encoding by obtaining and cleaning the login log information, the DBSCAN algorithm and information gain algorithm are used for density clustering, and outliers are identified as abnormal points, and anomaly detection is constructed. The model depends only on account history for judgment.
It improves the accuracy and flexibility of detection, reduces false alarms, and can quickly identify abnormal login behaviors. It has simple configuration and fast processing speed. It does not rely on external threat intelligence databases, making it convenient for operation and maintenance personnel to verify.
Smart Images

Figure CN115499142B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for detecting abnormal login behavior of an email account based on density clustering. Background Art
[0002] The theft of an email account may cause the loss of important information in the mailbox and the leakage of sensitive data, thus causing huge losses to users and even organizations. Abnormal login of an email account is often the beginning and an important sign of the theft of an email account. Therefore, it is particularly important to detect and discover abnormal login behavior of email accounts in a timely manner, and then take warning and disposal measures.
[0003] The existing method for detecting abnormal logins to email accounts is mainly to issue a login alarm by judging the difference between the user's login location information and the last login location when the account is logged in. This method has a large number of false positives. For example, users who have configured a proxy to send and receive emails, or who log in to their email accounts through overseas servers will be identified as abnormal logins. Summary of the invention
[0004] The present invention aims to solve the problem of many misjudgments and inaccuracy in the prior art.
[0005] In view of this, in a first aspect, the present invention provides a method for detecting abnormal login behavior of an email account based on density clustering, comprising:
[0006] Acquire login log information, and extract target feature information from the login log information after cleaning the login log information;
[0007] Performing feature processing on the target feature information to obtain corresponding feature values;
[0008] A target feature value is determined from the feature value by a feature selection algorithm, the target feature value is input into a DBSCAN algorithm model for density clustering to obtain outliers, and the outliers are determined as abnormal points.
[0009] Preferably, the target characteristic information includes: login time information, login IP information, login email account name, account login status information, and login protocol information.
[0010] Preferably, the feature processing of the target feature information to obtain the corresponding feature value includes: encoding and normalizing the login time information and the login IP information to obtain the corresponding login time feature value and login IP feature value.
[0011] Preferably, encoding and normalizing the login time information includes: extracting the year, month, and day information of the login time, and determining whether it is a working day / holiday, encoding and normalizing the login time based on preset rules to obtain a login time coding value, and using the judgment result information of whether it is a working day / holiday and the login time coding value as the login time feature value.
[0012] Preferably, the preset rule is: converting the specific time when the login behavior occurs into the number of seconds between it and the midnight of the day as the code of the time, and normalizing the code to obtain the login time code value.
[0013] Preferably, encoding and normalizing the login IP information includes: parsing the extracted login IP field to obtain the latitude and longitude, country, region, and operator of the IP address; extracting the four segments of the IP address and converting them into decimal numbers respectively, and marking them as p1, p2, p3, and p4 from left to right, assigning different weights a, b, c, and d to the four values of p1, p2, p3, and p4, and calculating the weighted coding value IP_code of the IP address, where IP_code=a*p1+b*p2+c*p3+d*p4; uniformly encoding and normalizing the parsed operator field to obtain the operator field coding value; using the weighted coding value of the IP address and the operator field coding value as the login IP feature value.
[0014] Preferably, after the target feature information is processed to obtain the corresponding feature value, the method further includes: selecting and determining the target time window according to the recent behavior characteristics of the email account to be detected, the number of valid user login logs and the detection requirements.
[0015] Preferably, the feature selection algorithm is an information gain algorithm model.
[0016] Preferably, the login log information includes: imap protocol login log, web protocol login log, pop protocol login log.
[0017] In a second aspect, the present invention provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the density clustering-based email account abnormal login behavior detection method provided in the first aspect above.
[0018] According to the density clustering-based mailbox account abnormal login behavior detection method and device provided by the present invention, the login logs of each mail protocol in the mail system are first cleaned to extract features such as login time and login IP address; then feature encoding, feature selection and other processing are performed, and a sliding time window is used to control and select the time period used for user login behavior modeling; then an abnormal detection model based on density clustering is constructed. This method of the present invention does not rely on third-party information libraries such as external threat intelligence libraries, and only uses the historical login records of the account itself as a basis for judgment. It has the advantages of flexible configuration, high accuracy, and fast processing speed. It can quickly find account login records with abnormal login features and issue alarm information, which is convenient for operation and maintenance personnel to further verify and discover hidden abnormal login behaviors. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 A flow chart of a method for detecting abnormal login behavior of an email account based on density clustering provided in an embodiment of the present invention;
[0021] Figure 2 A flow chart of a method for detecting abnormal login behavior of an email account based on density clustering provided in an embodiment of the present invention;
[0022] Figure 3 This is a diagram showing the clustering effect of an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0024] The theft of an email account may cause the loss of important information in the mailbox and the leakage of sensitive data, thus causing huge losses to users and even organizations. Abnormal login of an email account is often the beginning and an important sign of the theft of an email account. Therefore, it is particularly important to detect and discover abnormal login behavior of email accounts in a timely manner, and then take warning and disposal measures.
[0025] Accurately detecting abnormal login behavior of email accounts is difficult. First, abnormal behavior of email accounts is often hidden. Although some obvious abnormal behaviors can be identified and intercepted by security products such as email gateways, some hidden abnormal login behaviors are difficult to discover and identify. Second, the volume of email system login logs is large, and it is difficult for operation and maintenance personnel to conduct in-depth manual analysis of massive login logs. In addition, most login logs are normal data, and only a very small number of logs are login anomalies. Therefore, there is no need to manually analyze all login logs. Third, the login logs of the email system are actually unlabeled data. Unless user feedback is obtained, the detection accuracy of abnormal login behaviors cannot be confirmed, whether they are detected by security products or manually identified, which increases the difficulty of using machine learning algorithms to train models to detect abnormal login behaviors of email accounts.
[0026] There are two main methods for detecting abnormal logins to email accounts. The first is to give a login alarm by judging the difference between the user's login location and the last login location when logging in to an account. This method has a large number of false positives. For example, the user's configuration of a proxy to send and receive emails, and the behavior of logging in to the email account through an overseas server will be identified as abnormal login. The second type of existing mainstream methods is to judge the login IP. For example, by judging whether the login IP is a problem IP in the threat intelligence library, it can be checked whether the current account login is an abnormal operation. The accuracy of this method is limited by the accuracy of the threat intelligence library information. If the information in the threat intelligence library itself is inaccurate, it is difficult to use it to detect abnormal login behavior of email accounts; on the other hand, if the threat intelligence library information is not updated in a timely manner, it will also affect the accuracy of abnormal login judgment.
[0027] Therefore, the first embodiment of the present invention provides a method for detecting abnormal login behavior of email accounts based on density clustering, such as Figure 1 FIG. 1 is a flowchart of a method for detecting abnormal login behavior of an email account based on density clustering according to an embodiment of the present invention. The method comprises the following steps:
[0028] S101, obtain login log information, and extract target feature information from the login log information after cleaning the login log information. The login log information includes: imap protocol login log, web protocol login log, pop protocol login log, that is, the scope of log cleaning includes login logs of various mail protocols, such as imap protocol login log, web protocol login log, pop protocol login log, and some useless information is cleaned to improve the efficiency of subsequent processing. After the login log information is cleaned, the target feature information required for subsequent machine learning is extracted from the cleaned log data. The target feature information may include: login time information, login IP information, login email account name, account login status information, login protocol information and other key fields. The login time information is the time period information of the email account login, the login IP information is the IP address of the email account login, the account login status is login success or login failure, and the login protocol represents the login protocol of the email account.
[0029] S102, feature processing is performed on the target feature information to obtain a corresponding feature value. Feature processing is performed on the target feature information to obtain a corresponding feature value, which can be conveniently used as an input for a subsequent algorithm. Here, the feature processing of the target feature information to obtain a corresponding feature value may include encoding and normalizing the login time information and the login IP information to obtain a corresponding login time feature value and a login IP feature value, so as to be conveniently used as an input for a subsequent algorithm.
[0030] As a preferred embodiment, encoding and normalizing the login time information includes: extracting the year, month, and day information of the login time, and judging whether it is a working day or a holiday, encoding and normalizing the login time based on a preset rule to obtain a login time code value, and using the judgment result information of whether it is a working day or a holiday and the login time code value as the login time feature value. The preset rule is: converting the specific time when the login behavior occurs into the number of seconds between it and the zero o'clock time of the day as the code of the time, and normalizing the code to obtain the login time code value. For example, there are 86400 seconds in a day, and converting the specific time when the login behavior occurs into the number of seconds between it and the zero o'clock time of the day as the code of the time; after encoding the login time in this way, the login time of users on different dates and the login of different users at the same time can be normalized, which is convenient for the machine learning algorithm to process.
[0031] As a preferred embodiment, encoding and normalizing the login IP information includes: parsing the extracted login IP field to obtain the latitude and longitude, country, region, and operator of the IP address; extracting the four segments of the IP address and converting them into decimal numbers respectively, and marking them as p1, p2, p3, and p4 from left to right, assigning different weights a, b, c, and d to the four values of p1, p2, p3, and p4, and calculating the weighted coding value IP_code of the IP address, where IP_code=a*p1+b*p2+c*p3+d*p4; uniformly encoding and normalizing the parsed operator field to obtain the operator field coding value; using the weighted coding value of the IP address and the operator field coding value as the login IP feature value. For example, the login IP field extracted from an email account is parsed to obtain the latitude and longitude, country, region, and operator of the IP address, thereby judging that the login IP address belongs to domestic / foreign information. The four segments of the IP address are extracted and converted into decimal numbers respectively, and marked as p1, p2, p3, and p4 from left to right. Different weights a, b, c, and d are assigned to the four values of p1, p2, p3, and p4, and the weighted coding value IP_code of the IP address is calculated: IP_code=a*p1+b*p2+c*p3+d*p4. The reasons for weighted encoding of IP addresses are: first, encoding IP addresses can normalize the logins of users at different IP addresses and different users at the same IP address, which is convenient for subsequent processing by machine learning algorithms; and the values of different IP address segments have a very different impact on the actual IP. For example, if the first segment of IP addresses differs by 10, it may mean that the two IP addresses belong to different countries, while the fourth segment of IP addresses differs by 10, which may just be two different IP addresses in the same LAN, with a small difference. Therefore, it is necessary to weight the IP addresses. In actual operation, when assigning weights, a can be assigned a very large number, and b, c, and d can be assigned smaller numbers in turn, so as to widen the gap in the impact of the four segment address values on the encoding value. For example, a, b, c, and d are assigned values of 256, 64, 16, and 2 respectively (the assignment can be adjusted according to the subsequent machine learning effect). When uniformly encoding the operator field, assuming that the parsed operator field includes 48 different values after deduplication: Beijing Mobile, Liaoning Unicom, China Science and Technology Network, etc., all operator fields are numerically encoded and normalized to the range of 0-1 to obtain the operator field encoding value. Finally, the weighted encoding value of the IP address and the operator field encoding value are used as the login IP feature value to facilitate the input of the subsequent algorithm.
[0032] S103, determine the target feature value from the feature value through the feature selection algorithm, input the target feature value into the DBSCAN algorithm model for density clustering to obtain outliers, determine the outliers as abnormal points, and warn the abnormal information. Here, the feature selection algorithm is preferably an information gain algorithm model. Thus, through the feature selection of the information gain algorithm model, the feature dimension can be reduced, and then several features with better clustering effect can be selected to participate in the subsequent machine learning calculation, which can effectively improve the calculation efficiency and reduce the difficulty of subsequent density clustering parameter tuning.
[0033] Therefore, the density clustering-based mailbox account abnormal login behavior detection method and device provided in this embodiment first cleans the login logs of each email protocol in the email system, extracts login time, login IP address and other features; then performs feature encoding, feature selection and other processing; and then builds an abnormal detection model based on density clustering. This method of the present invention does not rely on third-party information libraries such as external threat intelligence libraries, and only uses the historical login records of the account itself as the basis for judgment. It has the advantages of flexible configuration, high accuracy and fast processing speed. It can quickly find account login records with abnormal login features and issue alarm information, which is convenient for operation and maintenance personnel to further verify and discover hidden abnormal login behaviors.
[0034] like Figure 2 FIG. 1 is a flowchart of a method for detecting abnormal login behavior of an email account based on density clustering according to another embodiment of the present invention. The method comprises the following steps:
[0035] S201, obtain login log information, and extract target feature information from the login log information after cleaning the login log information. The login log information includes: imap protocol login log, web protocol login log, pop protocol login log, that is, the scope of log cleaning includes login logs of various mail protocols, such as imap protocol login log, web protocol login log, pop protocol login log, and some useless information is cleaned to improve the efficiency of subsequent processing. After the login log information is cleaned, the target feature information required for subsequent machine learning is extracted from the cleaned log data. The target feature information may include: login time information, login IP information, login email account name, account login status information, login protocol information and other key fields. The login time information is the time period information of the email account login, the login IP information is the IP address of the email account login, the account login status is login success or login failure, and the login protocol represents the login protocol of the email account.
[0036] S202, feature processing is performed on the target feature information to obtain a corresponding feature value. Feature processing is performed on the target feature information to obtain a corresponding feature value, which can be conveniently used as an input for a subsequent algorithm. Here, the feature processing of the target feature information to obtain a corresponding feature value may include encoding and normalizing the login time information and the login IP information to obtain a corresponding login time feature value and a login IP feature value, so as to be conveniently used as an input for a subsequent algorithm.
[0037] As a preferred embodiment, encoding and normalizing the login time information includes: extracting the year, month, and day information of the login time, and judging whether it is a working day or a holiday, encoding and normalizing the login time based on a preset rule to obtain a login time code value, and using the judgment result information of whether it is a working day or a holiday and the login time code value as the login time feature value. The preset rule is: converting the specific time when the login behavior occurs into the number of seconds between it and the zero o'clock time of the day as the code of the time, and normalizing the code to obtain the login time code value. For example, there are 86400 seconds in a day, and converting the specific time when the login behavior occurs into the number of seconds between it and the zero o'clock time of the day as the code of the time; after encoding the login time in this way, the login time of users on different dates and the login of different users at the same time can be normalized, which is convenient for the machine learning algorithm to process.
[0038] As a preferred embodiment, encoding and normalizing the login IP information includes: parsing the extracted login IP field to obtain the latitude and longitude, country, region, and operator of the IP address; extracting the four segments of the IP address and converting them into decimal numbers respectively, and marking them as p1, p2, p3, and p4 from left to right, assigning different weights a, b, c, and d to the four values of p1, p2, p3, and p4, and calculating the weighted coding value IP_code of the IP address, where IP_code=a*p1+b*p2+c*p3+d*p4; uniformly encoding and normalizing the parsed operator field to obtain the operator field coding value; using the weighted coding value of the IP address and the operator field coding value as the login IP feature value. For example, the login IP field extracted from an email account is parsed to obtain the latitude and longitude, country, region, and operator of the IP address, thereby judging that the login IP address belongs to domestic / foreign information. The four segments of the IP address are extracted and converted into decimal numbers respectively, and marked as p1, p2, p3, and p4 from left to right. Different weights a, b, c, and d are assigned to the four values of p1, p2, p3, and p4, and the weighted coding value IP_code of the IP address is calculated: IP_code=a*p1+b*p2+c*p3+d*p4. The reasons for weighted encoding of IP addresses are: first, encoding IP addresses can normalize the logins of users at different IP addresses and different users at the same IP address, which is convenient for subsequent processing by machine learning algorithms; and the values of different IP address segments have a very different impact on the actual IP. For example, if the first segment of IP addresses differs by 10, it may mean that the two IP addresses belong to different countries, while the fourth segment of IP addresses differs by 10, which may just be two different IP addresses in the same LAN, with a small difference. Therefore, it is necessary to weight the IP addresses. In actual operation, when assigning weights, a can be assigned a very large number, and b, c, and d can be assigned smaller numbers in turn, so as to widen the gap in the impact of the four segment address values on the encoding value. For example, a, b, c, and d are assigned values of 256, 64, 16, and 2 respectively (the assignment can be adjusted according to the subsequent machine learning effect). When uniformly encoding the operator field, assuming that the parsed operator field includes 48 different values after deduplication: Beijing Mobile, Liaoning Unicom, China Science and Technology Network, etc., all operator fields are numerically encoded and normalized to the range of 0-1 to obtain the operator field encoding value. Finally, the weighted encoding value of the IP address and the operator field encoding value are used as the login IP feature value to facilitate the input of the subsequent algorithm.
[0039] S203, select and determine the target time window according to the recent behavior characteristics of the email account to be detected, the number of valid user login logs and the detection requirements. For example, if it is known that the user has not had any job adjustment, business trips, etc. in the past year, the time window can be selected for the past year; if the user has adjusted his job since June 2020, resulting in a significant change in email login and usage habits, then the time window can be selected after June 2020. In this way, a sliding time window can be used to control and select the time period used for user login behavior modeling, reducing the subsequent calculation amount and thus improving the accuracy of detection.
[0040] S204, determine the target feature value from the feature value through the feature selection algorithm, input the target feature value into the DBSCAN algorithm model for density clustering to obtain outliers, determine the outliers as abnormal points, and warn the abnormal information. Here, the feature selection algorithm is preferably an information gain algorithm model. Thus, through the feature selection of the information gain algorithm model, the feature dimension can be reduced, and then several features with better clustering effect can be selected to participate in the subsequent machine learning calculation, which can effectively improve the calculation efficiency and reduce the difficulty of subsequent density clustering parameter tuning.
[0041] like Figure 3 As shown, it is a clustering effect diagram of an example in an embodiment of the present invention. After the target feature information is processed to obtain the corresponding feature value, the three feature fields of login time, weighted coding value IP_code, and operator are selected from the feature value through the feature selection algorithm, and the DBSCAN clustering algorithm model is used to cluster the successful login records of a mailbox account on weekdays within a year. The black outliers on the left side of the figure are the abnormal login points. It can be seen that only three features are selected to quickly find suspected abnormal login behaviors, which greatly reduces the manual detection scope of operation and maintenance personnel.
[0042] Therefore, the density clustering-based mailbox account abnormal login behavior detection method and device provided in this embodiment first cleans the login logs of each email protocol in the email system, extracts login time, login IP address and other features; then performs feature encoding, feature selection and other processing, adopts sliding time window control and selects the time period used for user login behavior modeling; then builds an abnormal detection model based on density clustering. This method of the present invention does not rely on third-party information libraries such as external threat intelligence libraries, and only uses the historical login records of the account itself as the basis for judgment. It has the advantages of flexible configuration, high accuracy and fast processing speed. It can quickly find account login records with abnormal login features and issue alarm information, which is convenient for operation and maintenance personnel to further verify and discover hidden abnormal login behaviors.
[0043] This embodiment also provides an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the above-mentioned density clustering-based email account abnormal login behavior detection method.
[0044] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the embodiment of the above-mentioned method. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0045] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.
Claims
1. A method for detecting abnormal login behavior of email accounts based on density clustering, characterized in that: include: Obtain login log information, and extract target feature information from the login log information after cleaning the login log information, wherein the target feature information includes login time information, login IP information, login email account name, account login status information, and login protocol information; Performing feature processing on the target feature information to obtain corresponding feature values; Determine a target feature value from the feature value by a feature selection algorithm, input the target feature value into a DBSCAN algorithm model for density clustering to obtain an outlier, and determine the outlier as an abnormal point; the feature processing of the target feature information to obtain a corresponding feature value includes: encoding and normalizing the login time information and the login IP information to obtain a corresponding login time feature value and a login IP feature value; The encoding and normalization processing of the login time information includes: extracting the year, month, and day information of the login time, and judging whether it is a working day or a holiday, encoding and normalizing the login time based on a preset rule to obtain a login time encoding value, and using the judgment result information of whether it is a working day or a holiday and the login time encoding value as the login time feature value; The preset rule is: convert the specific time when the login behavior occurs into the number of seconds between it and the midnight of the day as the code of the time, and normalize the code to obtain the login time code value; After the target feature information is processed to obtain the corresponding feature value, the method further includes: selecting and determining the target time window according to the recent behavior characteristics of the email account to be detected, the number of valid user login logs and the detection requirements.
2. The method according to claim 1, characterized in that The encoding and normalization of the login IP information includes: parsing the extracted login IP field to obtain the latitude and longitude, country, region, and operator of the IP address; extracting the four segments of the IP address and converting them into decimal numbers respectively, and marking them as p1, p2, p3, and p4 from left to right, assigning different weights a, b, c, and d to the four values of p1, p2, p3, and p4, and calculating the weighted coding value IP_code of the IP address, where IP_code=a*p1+b*p2+c*p3+d*p4; uniformly encoding and normalizing the parsed operator field to obtain the operator field coding value; using the weighted coding value of the IP address and the operator field coding value as the login IP feature value.
3. The method according to claim 1, characterized in that: The feature selection algorithm is an information gain algorithm model.
4. The method according to claim 1, characterized in that: The login log information includes: imap protocol login log, web protocol login log, pop protocol login log.
5. An electronic device, characterized in that: include: At least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the density clustering-based email account abnormal login behavior detection method described in any one of claims 1-4.
