An abnormal access detection method and device

Abstract

The present disclosure provides an abnormal access detection method and apparatus, which relate to the field of computer technologies, and particularly to the field of big data. The specific implementation solution is as follows: determining target users who access a first business line within a first time period; obtaining access resource information corresponding to the user identifier of each of the target users; clustering the user identifiers based on the access resource information to determine a plurality of user clusters after clustering; and detecting the user clusters to determine abnormal user clusters with abnormal access. By using the access resource information of users as clustering features for clustering and discovering abnormal access teams through the clustering results, it has stronger timeliness compared with the manual mining and analysis methods, and at the same time can discover abnormal access teams that are not easily found.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to the field of big data technologies. Background Art

[0002] Web crawler traffic refers to the automatic scraping of network traffic controlled by a script according to certain rules, which is different from the way normal users obtain information traffic. Therefore, it belongs to cheating traffic, or abnormal traffic.

[0003] To maintain the security of World Wide Web information, it is necessary to detect web crawler traffic. Summary of the Invention

[0004] The present disclosure provides an abnormal access detection method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, there is provided an abnormal access detection method, the method including:

[0006] Determining target users who access a first service line within a first time period;

[0007] Obtaining access resource information corresponding to the user identifier of each of the target users; the access resource information represents the access resources used by the target user when initiating an access request;

[0008] Clustering the user identifiers based on the access resource information to determine multiple user clusters after clustering;

[0009] Detecting the user clusters to determine abnormal user clusters with abnormal access.

[0010] According to a second aspect of the present disclosure, there is provided an abnormal access detection apparatus, the apparatus including:

[0011] A target user determination module, configured to determine target users who access a first service line within a first time period;

[0012] An information acquisition module, configured to obtain access resource information corresponding to the user identifier of each of the target users; the access resource information represents the access resources used by the target user when initiating an access request;

[0013] A first clustering module, configured to cluster the user identifiers based on the access resource information to determine multiple user clusters after clustering;

[0014] A detection module, configured to detect the user clusters to determine abnormal user clusters with abnormal access.

[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to perform an abnormal access detection method.

[0019] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform an abnormal access detection method.

[0020] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program, and the computer program implements an abnormal access detection method when executed by a processor.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0023] Figure 1 is a schematic flowchart of an abnormal access detection method provided by an embodiment of the present disclosure;

[0024] Figure 2 is another schematic flowchart of an abnormal access detection method provided by an embodiment of the present disclosure;

[0025] Figure 3 is yet another schematic flowchart of an abnormal access detection method provided by an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of an abnormal access detection method provided by an embodiment of the present disclosure;

[0027] Figure 5 is a block diagram of a device for implementing the abnormal access detection method of the embodiment of the present disclosure;

[0028] Figure 6 is a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0030] Web crawler traffic refers to the automatic capture of network traffic controlled by a script according to certain rules, which is different from the way normal users obtain information traffic. Therefore, it belongs to cheating traffic, or abnormal traffic.

[0031] To maintain the security of the information on the World Wide Web, it is necessary to detect web crawler traffic.

[0032] Since crawler behavior usually has a team nature, in related technologies, an artificial method is used to analyze and discover teams with abnormal access. However, the more advanced the crawler behavior, the more dispersed it is in terms of resource usage. For example, using an IP pool, cracking and invading multiple accounts, etc. It is difficult to intuitively discover abnormal teams through log traffic, resulting in a lack of understanding of the specific behavior patterns of abnormal teams and the inability to effectively associate the resource pools used by the teams. Subsequently, teams using the same resource pool in related services cannot be located and tracked in a timely manner.

[0033] To solve the above technical problems, the present disclosure provides an abnormal access detection method and device.

[0034] In one embodiment of the present disclosure, an abnormal access detection method is provided. The method includes:

[0035] Determine the target users who access the first service line within the first time period;

[0036] Obtain the access resource information corresponding to the user identifier of each of the target users;

[0037] Cluster the user identifiers based on the access resource information to determine multiple user clusters after clustering;

[0038] Detect the user clusters to determine the abnormal user clusters with abnormal access.

[0039] Anomalous access has the following characteristics: The teams with anomalous access usually associate with a unified resource pool and frequently change the accessed resources from the resource pool to avoid detection. In the embodiments of the present disclosure, considering the above characteristics of anomalous access, the access resource information of users is used as clustering features for clustering. Thus, users with similar access resource information will be clustered into one category. For the teams with anomalous access, since the same resource pool is adopted, clustering based on the access resource information can cluster the identifiers of the user accounts used by the teams with anomalous access into one category. After clustering, it is easy to identify whether each user cluster is an anomalous user cluster, that is, to mine the teams with anomalous access through the clustering results. Compared with the method of manual mining and analysis, it has stronger timeliness and can also mine the teams with anomalous access that are not easily discovered.

[0040] The following provides a detailed introduction to the anomalous access detection method provided by the embodiments of the present disclosure.

[0041] See Figure 1 , Figure 1 which is a schematic flowchart of an anomalous access detection method provided by the embodiments of the present disclosure. As Figure 1 shown, the method may include the following steps:

[0042] S101: Determine the target users who access the first business line within the first time period.

[0043] In the embodiments of the present disclosure, traffic logs are obtained and log standardization is performed. For example, data cleaning, field extraction, and database storage are performed in sequence.

[0044] Based on the standardized logs, the users who access the business line within a specific time period can be determined. For the sake of convenience of description, taking the users who access the first business line within the first time period as an example, they are denoted as target users.

[0045] S102: Obtain the access resource information corresponding to the user identifier of each target user, where the access resource information characterizes the access resources used by the target user when initiating an access request.

[0046] Among them, the user identifier may be UID (User Identification), that is, the numerical value generated by the network side when the user registers, which can be used as the unique identifier of the user.

[0047] Through the standardized logs, the access resource information corresponding to the user identifier of each target user can also be obtained, where the access resource information characterizes the access resources used by the target user when initiating an access request.

[0048] As an example, an IP address (Internet Protocol Address) is an essential resource when a user initiates an access request. Therefore, an IP address can be used as access resource information. For the sake of convenience in description, IP will be used to represent the IP address hereinafter.

[0049] S103: Cluster user identities based on access resource information to determine multiple user clusters after clustering.

[0050] In the embodiments of the present disclosure, the access resource information of each user identity can be represented by a feature vector.

[0051] Before clustering, the feature vector can be standardized. After the standardization process, a relevant clustering algorithm is used for clustering.

[0052] As an example, first, the elbow method provided by the k-means clustering algorithm is used to determine the number of categories, and then the k-means clustering model is used to cluster user identities.

[0053] Among them, the k-means clustering algorithm may include the following steps:

[0054] 1) Determine the number of categories k, and select the initial k samples as the initial cluster centers.

[0055] 2) Calculate the distance from each sample in the dataset to the k cluster centers, and assign it to the class corresponding to the cluster center with the minimum distance.

[0056] 3) Recalculate the cluster centers for each category.

[0057] 4) Repeat steps 2-3 until the termination condition is reached.

[0058] In the embodiments of the present disclosure, the input of the k-means clustering model is the standardized feature vector of the access resource information of each user identity.

[0059] After clustering, multiple user identity clusters can be obtained, which can also be understood as user clusters.

[0060] S104: Detect user clusters to determine abnormal user clusters with abnormal access.

[0061] In the embodiments of the present disclosure, since an abnormal access team (such as a web crawler team) will create resource pools such as IP pools to avoid online detection and frequently change access resources, while normal access users do not frequently change access resources, so clustering based on access resource information can accurately detect abnormal access teams using the same resource pool.

[0062] Therefore, by detecting user clusters, the abnormal user clusters of abnormal access can be determined.

[0063] In the embodiments of the present disclosure, specific rules can be set. For example, if the number of average IPs of a user cluster exceeds a set value, or the overall traffic scale of the user cluster is large, then the user cluster is considered an abnormal user cluster.

[0064] It can be seen that in the embodiments of the present disclosure, the access resource information of users is used as clustering features for clustering, so that users with similar access resource information will be clustered into one category. For an abnormal access team, since the same resource pool is used, clustering based on access resource information can cluster the identifiers of user accounts used by the abnormal access team into one category. After clustering, it is easy to identify whether each user cluster is an abnormal user cluster, that is, to mine the abnormal access team through the clustering result. Compared with the method of manual mining and analysis, it has stronger timeliness and can also mine abnormal access teams that are not easily discovered.

[0065] In one embodiment of the present disclosure, the access resource information may include one or more of: the number of distinct IPs, the number of distinct IP network segments, the number of distinct user identity cache identifiers, and the number of distinct browser user agents.

[0066] Among them, the user identity cache identifier can be a cookie, which is data generated by a website to identify the user identity and stored on the user's local terminal. The IP network segment is the IPC, that is, part of the fields in the IP address. The browser user agent (UserAgent, UA) is used to identify the browser client information.

[0067] It can be seen that in the embodiments of the present disclosure, considering that an abnormal access team frequently changes the IP, user identity cache identifier, and browser user agent when accessing the business line, the above access resource information is used as the basis for clustering, and then the abnormal access team is mined according to the clustering result.

[0068] In one embodiment of the present disclosure, the target user can be a user whose number of accesses to the first business line in the first time period is greater than a set value.

[0069] Since the number of accesses of abnormal access users is large, preliminary screening can be performed according to the number of accesses to obtain users with a large number of accesses. These users may be involved in abnormal access and are used as target users.

[0070] It can be seen that in the embodiments of the present disclosure, preliminary screening is performed according to the number of accesses, reducing the amount of data participating in clustering and further improving the efficiency of abnormal access detection.

[0071] In one embodiment of the present disclosure, on the basis of Figure 1 the method shown, it may further include:

[0072] Mark the access resource information of the abnormal user cluster as abnormal resource information;

[0073] Mark the requests that use the abnormal resource information detected online for access as abnormal access requests.

[0074] Specifically, since the user accounts in the abnormal user cluster are the accounts used by the abnormal access team, the corresponding access resources also belong to the resource pool created by the abnormal access team. Therefore, mark these access resource information as abnormal resource information. Then, in the subsequent detection process, if a request using the abnormal resource information for access is detected, it can be directly marked as an abnormal access request.

[0075] It can be seen that in the embodiments of the present disclosure, clustering is performed based on access resource information to mine the abnormal access team, and the access resource information used by the abnormal access team is marked, which helps to understand the specific behavior patterns of the abnormal access team and effectively associate the resource pool used by the abnormal access team. When an abnormal access team using the same resource pool appears in the relevant business subsequently, it can be located and tracked in a timely manner.

[0076] As an example, first obtain the daily traffic data of a certain business line in the standardized log, perform feature dimension aggregation based on UID, and obtain the distinct IP count, distinct IPC count, distinct cookie count, and distinct UA count of UID as clustering features. Screen the UIDs with the request count greater than 1000. Finally, 4039 UID feature vectors of this business line are screened out. Therefore, the corresponding access resource information can be represented as 4039 four-dimensional feature vectors.

[0077] Standardize the UID feature vectors, perform clustering through the k-means clustering algorithm, and obtain the category labels of each UID. Then, detect and identify each user cluster obtained by clustering, and finally locate multiple typical abnormal access teams.

[0078] Furthermore, mark the access resource information of the abnormal user cluster as abnormal resource information for online detection, so as to locate and track the abnormal access team using the same resource pool in a timely manner.

[0079] In an embodiment of the present disclosure, abnormal risk pattern mining can also be performed based on clustering, locate the abnormal features of the risk pattern, and improve the online detection rules. For details, see Figure 2 , Figure 2 which is another process schematic diagram of the abnormal access detection method provided by the embodiments of the present disclosure. The method may include:

[0080] S201: Determine the candidate IPs accessing the second business line within the second time period.

[0081] Specifically, based on the standardized logs, the IPs accessing the business line during a specific period can be determined, that is, the IPs used by the users accessing the business line.

[0082] For the sake of convenience of description, taking the IPs accessing the second business line during the second period as an example, they are denoted as candidate IPs.

[0083] S202: Obtain the first time-series access sequence of each candidate IP. The first time-series access sequence includes the access times of the candidate IP in each sub-period during the second period.

[0084] Based on the standardized logs, the first time-series access sequence of each candidate IP can be further obtained. The first time-series access sequence includes the access times of the candidate IP in each sub-period during the second period.

[0085] As an example, if the second period is one day and each sub-period is 1 minute, then the first time-series access sequence can be represented as a 1440-dimensional feature vector, and each value represents the access times of the candidate IP for the second business line in the corresponding sub-period.

[0086] S203: Based on the first time-series access sequence, screen out the target IPs that meet the preset abnormal access characteristics from the candidate IPs.

[0087] In the embodiments of the present disclosure, the abnormal access characteristics can be set according to the detection experience of abnormal traffic. For example, the number of access requests of normal users is usually not stable, usually having peaks and valleys, that is, the access volume is higher during the day and lower at night, while the abnormal access is controlled by a script and is usually stable throughout the day.

[0088] Therefore, in an embodiment of the present disclosure, it can be determined whether the first time-series access sequence of the candidate IP is a time-series stationary sequence. If so, it is determined that the candidate IP meets the abnormal access characteristics, and the candidate IP is determined as the target IP.

[0089] It can be seen that in the embodiments of the present disclosure, considering that the access requests of normal IPs are time-series stationary, while the access requests of abnormal IPs are not time-series stationary, thus if it is determined that the time-series access sequence of the candidate IP belongs to the time-series stationary sequence, it is determined that the candidate IP meets the abnormal access characteristics. Abnormal IPs can be efficiently screened out.

[0090] S204: Cluster the target IPs based on the time-series access sequences of each target IP to determine multiple IP clusters after clustering.

[0091] Subsequently, cluster the target IPs based on the time-series access sequences to mine the common characteristics of the abnormal IP clusters.

[0092] S205: Mine abnormal IP features based on IP clusters and update the abnormal access detection rules deployed online based on the abnormal IP features.

[0093] Specifically, abnormal IP features can be mined more intuitively through clustering, that is, the features shared by the entire abnormal IP cluster. Updating the online detection rules according to these features can improve the accuracy of detecting abnormal access traffic online.

[0094] As an example, the business feedback shows that some abnormal IPs have not been detected. After confirmation, there is a type of traffic that persists throughout the day but is low-frequency and bypasses the online detection rules. It is necessary to locate the abnormal features of this type of low-frequency traffic to improve the online detection rules.

[0095] Specifically, obtain the standardized log of the traffic data of this business line in one day. Aggregate the feature dimensions based on IP, obtain the time-series request sequence of the IP as the clustering feature, and filter the IPs with the number of requests greater than 30,000. The base number of requests can be configured according to the specific business scenario. Finally, 580 IPs are filtered out, and 121 stationary time-series sequences are obtained using the ADF (Augmented Dickey-Fuller) test. Finally, the input of the clustering algorithm can be represented by 121 feature vectors of 1440 dimensions.

[0096] Standardize the above feature vectors and use the k-means clustering algorithm for clustering to determine the class label to which each IP belongs.

[0097] As an example, it is finally determined that the abnormal IPs are all of the IDC (Internet Data Center) type. The Internet Data Center has complete equipment (including high-speed Internet access bandwidth, high-performance local area network, secure and reliable computer room environment, etc.) and a professionally managed service platform, and frequently changes user nicknames. The generation time of the user nicknames is very new. Then, these features can be used to further improve the online detection rules.

[0098] It can be seen that in the embodiments of the present disclosure, IPs that meet the abnormal access characteristics are clustered based on the time-series access sequence, so that abnormal IP features can be mined more intuitively through clustering, that is, the features shared by the entire abnormal IP cluster. Updating the online detection rules according to these features can improve the accuracy of detecting abnormal access traffic online.

[0099] In an embodiment of the present disclosure, the online misjudgment results can also be corrected based on clustering. For details, see Figure 3 , Figure 3 which is another process schematic diagram of the abnormal access detection method provided by the embodiments of the present disclosure. The method may include:

[0100] S301: Determine non-natural person identifiers with the number of business accesses in the third time period greater than a preset threshold.

[0101] Specifically, based on the standardized logs, non-natural person identifiers accessing the business line during a specific time period can be determined. For ease of description, take the non-natural person identifiers accessing the third business line in the third time period as an example.

[0102] Among them, the non-natural person identifiers can include one or more of IP, browser user agent, and client fingerprint. The client fingerprint can be a JA3 fingerprint.

[0103] It can be seen that in the embodiments of the present disclosure, the non-natural person identifiers can cover various types of information, including IP, browser user agent, and client fingerprint. When a user (natural person) accesses the business line, these non-natural person identifier information will be generated. By counting the access requests, the sequential access sequence corresponding to the non-natural person identifier can be efficiently determined.

[0104] S302: Determine the second sequential access sequence corresponding to the non-natural person identifier. The second sequential access sequence includes the number of accesses of the non-natural person identifier in each sub-time period within the third time period.

[0105] Based on the standardized logs, the second sequential access sequence corresponding to each non-natural person identifier can be further obtained. As an example, if the third time period is one day and each sub-time period is 1 minute, then the second sequential access sequence can be represented as a 1440-dimensional feature vector, and each value represents the number of accesses of the non-natural person identifier to the second business line in the corresponding sub-time period.

[0106] It is easy to understand that in the embodiments of the present disclosure, the number of accesses of the non-natural person identifier to the business line is essentially the number of accesses of the user using the non-natural person identifier to the business line.

[0107] S303: Based on the second sequential access sequence, cluster the non-natural person identifiers to determine multiple non-natural person identifier clusters after clustering, and the clustered sequential access sequence of each non-natural person identifier cluster after clustering.

[0108] Subsequently, cluster the non-natural person identifiers based on the second sequential access sequence to obtain multiple non-natural person identifier clusters. In addition, the clustered sequential access sequence of each non-natural person identifier cluster after clustering can be determined.

[0109] S304: Determine whether the clustered sequential access sequence of the non-natural person identifier cluster conforms to the preset natural person access characteristics. If so, mark the non-natural person identifier cluster as a non-abnormal access identifier cluster.

[0110] Subsequently, sequentially determine whether the clustered sequential access sequences of each non-natural person identifier cluster conform to the natural person access characteristics.

[0111] For example, if the clustered time-series access sequence is non-stationary with peak and trough periods, it can be determined to conform to the access characteristics of natural persons.

[0112] S305: Based on the non-abnormal access identifier cluster, correct the misjudgment of the abnormal identifiers detected online.

[0113] Specifically, it is inevitable that misjudgments will occur when using the detection rules configured online. For example, during business testing, the business test traffic is quite different from the access traffic of normal users. Therefore, the business test traffic is easily identified as crawler traffic. However, when testers conduct business tests, the generated traffic also conforms to the access characteristics of natural persons, that is, the time-series behavior throughout the day will remain consistent with that of normal people.

[0114] Therefore, if the abnormal identifier detected online belongs to the non-abnormal access identifier cluster, then this abnormal identifier is a misjudgment, and it is corrected.

[0115] It can be seen that in the embodiments of the present disclosure, non-natural person identifiers are clustered according to the time-series access sequence, and then it is determined whether the time-series access sequence conforms to the access characteristics of natural persons. If it conforms, it means that these non-natural person identifiers do not belong to abnormal access identifiers. If these non-natural person identifiers are detected as abnormal identifiers through the detection rules deployed online, it can be determined that the online detection is a misjudgment, and then the misjudgment can be corrected, further improving the detection rules and enhancing the accuracy of detecting abnormal access traffic.

[0116] As an example, non-natural person identifiers with the number of requests per business line output greater than 30,000 are obtained, including IP, UA, and JA3. Then the time-series access sequence is counted, and finally a total of 13,354 time-series access sequences are output. The input of the clustering algorithm can be represented by 13,354 feature vectors with 1440 dimensions.

[0117] Standardize the above-mentioned feature vectors, use the k-means clustering algorithm for clustering, and determine the class label to which each non-natural person identifier belongs. Then determine whether each clustering cluster conforms to the normal time-series characteristics, that is, the access characteristics of natural persons. If the clustering cluster conforms to the access characteristics of natural persons, it means that this clustering cluster does not belong to the abnormal access group. Based on this, the misjudgment of the abnormal identifiers detected online can be corrected.

[0118] For ease of understanding, the following further describes the abnormal access detection method provided by the embodiments of the present disclosure in conjunction with the accompanying drawings Figure 4 , and further illustrate the abnormal access detection method provided by the embodiments of the present disclosure.

[0119] See Figure 4 , Figure 4A schematic diagram of the abnormal access detection method provided by the embodiments of the present disclosure. First, obtain the standardized business traffic logs, and then determine the clustering ID (i.e., the clustering object) and the corresponding clustering features according to different scenarios. The clustering objects may include UID, IP, non-natural person identifiers, etc.; the clustering features may include: the number of distinct IPs, the number of distinct UAs, the time-series access sequence, etc. Subsequently, preprocess the clustering features and then perform clustering through a clustering algorithm.

[0120] When the clustering object is UID and the clustering features are one or more of the number of distinct IPs, the number of distinct IPCs, the number of distinct cookies, and the number of distinct UAs, abnormal access teams can be mined.

[0121] When the clustering object is IP and the clustering feature is the time-series access sequence, abnormal IP features can be mined to improve the online detection rules.

[0122] When the clustering object is a non-natural person identifier and the clustering feature is the time-series access sequence, online misjudgments can be corrected.

[0123] See Figure 5 , Figure 5 A schematic structural diagram of the abnormal access detection device provided by the embodiments of the present disclosure. The device may include:

[0124] A target user determination module 501, configured to determine target users who access the first service line within the first time period.

[0125] An information acquisition module 502, configured to acquire access resource information corresponding to the user identifier of each said target user.

[0126] A first clustering module 503, configured to cluster the user identifiers based on the access resource information to determine multiple user clusters after clustering.

[0127] A detection module 504, configured to detect the user clusters to determine abnormal user clusters with abnormal access.

[0128] It can be seen that in the embodiments of the present disclosure, the access resource information of users is used as the clustering feature for clustering, so that users with similar access resource information will be clustered into one category. For abnormal access teams, since the same resource pool is used, clustering based on the access resource information can cluster the identifiers of the user accounts used by the abnormal access teams into one category. After clustering, it is easy to identify whether each user cluster is an abnormal user cluster, that is, to mine the abnormal access teams through the clustering results. Compared with the manual mining and analysis methods, it has stronger timeliness and can also mine abnormal access teams that are not easily discovered.

[0129] In one embodiment of the present disclosure, the access resource information includes one or more of the following: the deduplication count of Internet Protocol (IP) addresses, the deduplication count of IP network segments, the deduplication count of user identity cache identifiers, and the deduplication count of browser user agents.

[0130] It can be seen that in the embodiment of the present disclosure, considering that the abnormal access team frequently changes IP addresses, user identity cache identifiers, and browser user agents when accessing the business line, the above access resource information is used as the basis for clustering, and then the abnormal access team is mined according to the clustering result.

[0131] In one embodiment of the present disclosure, the target user is a user whose number of accesses to the first business line within the first time period is greater than a set value.

[0132] It can be seen that in the embodiment of the present disclosure, preliminary screening is performed based on the number of accesses, reducing the amount of data participating in clustering and further improving the efficiency of abnormal access detection.

[0133] In one embodiment of the present disclosure, it further includes:

[0134] A first marking module, configured to mark the access resource information of the abnormal user cluster as abnormal resource information;

[0135] A second marking module, configured to mark a request that uses the abnormal resource information detected online as an abnormal access request.

[0136] It can be seen that in the embodiment of the present disclosure, clustering is performed based on the access resource information to mine the abnormal access team, and the access resource information used by the abnormal access team is marked, which helps to understand the specific behavior pattern of the abnormal access team and effectively associate the resource pool used by the abnormal access team. When an abnormal access team using the same resource pool appears in a related business later, it can be located and tracked in a timely manner.

[0137] In one embodiment of the present disclosure, it further includes:

[0138] A candidate IP determination module, configured to determine candidate IPs for accessing the second business line within the second time period;

[0139] A first sequence acquisition module, configured to acquire a first time-sequential access sequence for each candidate IP, where the first time-sequential access sequence includes the number of accesses of the candidate IP in each sub-time period within the second time period;

[0140] A screening module, configured to screen out target IPs that meet the preset abnormal access characteristics from the candidate IPs based on the first time-sequential access sequence;

[0141] A second clustering module, configured to cluster the target IPs based on the time-series access sequences of the respective target IPs, and determine multiple IP clusters after clustering;

[0142] A feature mining module, configured to mine abnormal IP features based on the IP clusters, and update the online detection rules for abnormal access traffic based on the abnormal IP features.

[0143] It can be seen that in the embodiments of the present disclosure, the IPs that meet the abnormal access characteristics are clustered based on the time-series access sequences, so that the abnormal IP features, that is, the features common to the entire abnormal IP cluster, can be more intuitively mined through clustering. Updating the online detection rules according to these features can improve the accuracy of online detection of abnormal access traffic.

[0144] In an embodiment of the present disclosure, the screening module is specifically configured to:

[0145] Judge whether the first time-series access sequence of the candidate IP is a time-series stationary sequence. If so, determine that the candidate IP meets the preset abnormal access characteristics, and determine the candidate IP as the target IP.

[0146] It can be seen that in the embodiments of the present disclosure, considering that the access requests of normal IPs are time-series stationary, while the access requests of abnormal IPs are not time-series stationary, so if it is determined that the time-series access sequence of the candidate IP belongs to the time-series stationary sequence, it is determined that the candidate IP meets the abnormal access characteristics. Abnormal IPs can be efficiently screened out.

[0147] In an embodiment of the present disclosure, it further includes:

[0148] An identification determination module, configured to determine non-natural person identifications with the number of business accesses greater than a preset threshold within a third time period;

[0149] A second sequence determination module, configured to determine the second time-series access sequence corresponding to the non-natural person identification, where the second time-series access sequence includes the access times of the non-natural person identification in each sub-time period within the third time period;

[0150] A third clustering module, configured to cluster the non-natural person identifications based on the second time-series access sequence, determine multiple non-natural person identification clusters after clustering, and the clustered time-series access sequences of each non-natural person identification cluster after clustering;

[0151] A marking module, configured to judge whether the clustered time-series access sequence of the non-natural person identification cluster meets the preset natural person access characteristics. If so, mark the non-natural person identification cluster as a non-abnormal access identification cluster;

[0152] A correction module, configured to correct the misjudgment of the abnormal identifications detected online based on the non-abnormal access identification clusters.

[0153] It can be seen that in the embodiments of the present disclosure, IPs that conform to abnormal access characteristics are clustered based on the time-series access sequence, so that the abnormal IP characteristics, that is, the characteristics common to the entire abnormal IP cluster, can be more intuitively mined through clustering. Updating the online detection rules according to these characteristics can improve the accuracy of detecting abnormal access traffic online.

[0154] In one embodiment of the present disclosure, the non-natural person identifier includes one or more of an IP, a browser user agent, and a client fingerprint.

[0155] It can be seen that in the embodiments of the present disclosure, the non-natural person identifier can cover various types of information, including IP, browser user agent, and client fingerprint. When a user (natural person) accesses a service line, these non-natural person identifier information will be generated. By counting the access requests, the time-series access sequence corresponding to the non-natural person identifier can be efficiently determined.

[0156] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0157] Figure 6 FIG. shows a schematic block diagram of an exemplary electronic device 600 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0158] As Figure 6 shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 602 or the computer program loaded from the storage unit 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0159] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as a keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as a disk, optical disc, etc.; and communication unit 609, such as a network card, modem, wireless communication transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0160] Computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 601 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 601 executes the various methods and processes described above, such as the abnormal access detection method. For example, in some embodiments, the abnormal access detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the abnormal access detection method described above can be executed. Alternatively, in other embodiments, computing unit 601 can be configured to execute the abnormal access detection method by any other suitable means (e.g., by means of firmware).

[0161] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), system-on-chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0162] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0163] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0165] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0166] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0167] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0168] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. An abnormal access detection method, including: Determining target users who access the first business line within the first time period; The target users are those whose number of accesses to the first business line within the first time period is greater than a set value; Obtaining access resource information corresponding to the user identifier of each target user; The access resource information characterizes the access resources used by the target user when initiating an access request; The access resource information includes one or more of the number of distinct Internet Protocol addresses (IPs), the number of distinct IP networks, the number of distinct user identity cache identifiers, and the number of distinct browser user agents; Clustering the user identifiers based on the access resource information to determine multiple user clusters after clustering; Detecting the user clusters to determine abnormal user clusters with abnormal accesses.

2. The method according to claim 1, further including: Marking the access resource information of the abnormal user clusters as abnormal resource information; Marking requests for access using the abnormal resource information detected online as abnormal access requests.

3. The method according to claim 1, further including: Determining candidate IPs that access the second business line within the second time period; Obtaining the first time-series access sequence of each candidate IP, where the first time-series access sequence includes the number of accesses of the candidate IP in each sub-time period within the second time period; Based on the first time-series access sequence, screening out target IPs that meet the preset abnormal access characteristics from the candidate IPs; Clustering the target IPs based on the time-series access sequences of the respective target IPs to determine multiple IP clusters after clustering; Mining abnormal IP characteristics based on the IP clusters and updating the abnormal access detection rules deployed online based on the abnormal IP characteristics.

4. The method according to claim 3, wherein, The step of screening out target IPs that meet the preset abnormal access characteristics from the candidate IPs based on the first time-series access sequence includes: Judging whether the first time-series access sequence of the candidate IP is a time-series stationary sequence. If so, determining that the candidate IP meets the preset abnormal access characteristics and determining the candidate IP as a target IP.

5. The method according to any one of claims 1-4, further including: Determining non-natural person identifiers with the number of business accesses greater than a preset threshold within the third time period; Determining the second time-series access sequence corresponding to the non-natural person identifier, where the second time-series access sequence includes the number of accesses of the non-natural person identifier in each sub-time period within the third time period; Based on the second time-series access sequence, clustering the non-natural person identifiers to determine multiple non-natural person identifier clusters after clustering and the clustered time-series access sequences of each non-natural person identifier cluster after clustering; Judging whether the clustered time-series access sequence of the non-natural person identifier cluster meets the preset natural person access characteristics. If so, marking the non-natural person identifier cluster as a non-abnormal access identifier cluster; Based on the non-abnormal access identifier cluster, correcting misjudgments of abnormal identifiers detected online.

6. The method according to claim 5, wherein, The non-natural person identifiers include one or more of IP, browser user agent, and client fingerprint.

7. An abnormal access detection device, comprising: A target user determination module, configured to determine a target user who accesses a first service line within a first time period; The target user is a user whose number of accesses to the first service line within the first time period is greater than a set value; An information acquisition module, configured to acquire access resource information corresponding to the user identifier of each target user; The access resource information characterizes the access resources used by the target user when initiating an access request; The access resource information includes one or more of the de-duplication count of the Internet protocol address IP, the de-duplication count of the IP network segment, the de-duplication count of the user identity cache identifier, and the de-duplication count of the browser user agent; A first clustering module, configured to cluster the user identifiers based on the access resource information to determine multiple user clusters after clustering; A detection module, configured to detect the user clusters to determine abnormal user clusters with abnormal accesses.

8. The device according to claim 7, further comprising: A first marking module, configured to mark the access resource information of the abnormal user cluster as abnormal resource information; A second marking module, configured to mark a request that uses the abnormal resource information for access detected online as an abnormal access request.

9. The device according to claim 7, further comprising: A candidate IP determination module, configured to determine candidate IPs that access a second service line within a second time period; A first time sequence acquisition module, configured to acquire a first time sequence access sequence of each candidate IP, where the first time sequence access sequence includes the access counts of the candidate IP in each sub-time period within the second time period; A screening module, configured to screen out target IPs that conform to preset abnormal access characteristics from the candidate IPs based on the first time sequence access sequence; A second clustering module, configured to cluster the target IPs based on the time sequence access sequences of the respective target IPs to determine multiple IP clusters after clustering; A feature mining module, configured to mine abnormal IP features based on the IP clusters and update the abnormal access detection rules deployed online based on the abnormal IP features.

10. The device according to claim 9, wherein, The screening module is specifically configured to: Determine whether the first time sequence access sequence of the candidate IP is a time sequence stationary sequence. If so, determine that the candidate IP conforms to the preset abnormal access characteristics and determine the candidate IP as a target IP.

11. The device according to any one of claims 7-10, further comprising: An identifier determination module, configured to determine non-natural person identifiers whose service access counts within a third time period are greater than a preset threshold; A second time sequence determination module, configured to determine a second time sequence access sequence corresponding to the non-natural person identifier, where the second time sequence access sequence includes the access counts of the non-natural person identifier in each sub-time period within the third time period; A third clustering module, configured to cluster the non-natural person identifiers based on the second time sequence access sequence to determine multiple non-natural person identifier clusters after clustering, and the clustered time sequence access sequence of each non-natural person identifier cluster after clustering; A third marking module, configured to determine whether the clustering time-series access sequence of the non-natural-person identification clusters conforms to a preset natural-person access feature. If so, mark the non-natural-person identification clusters as non-abnormal access identification clusters; A correction module, configured to correct misjudgments of abnormal identifications detected online based on the non-abnormal access identification clusters.

12. The apparatus according to claim 11, wherein, the non-natural-person identifications include one or more of an IP, a browser user agent, and a client fingerprint.

13. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, the computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.

Images