Flow detection methods, devices and electronic equipment

Abstract

This disclosure provides a traffic detection method, apparatus, and electronic device, relating to the field of data processing technology, specifically to the fields of Internet technology and machine learning technology. The specific implementation scheme is as follows: First, traffic time-series information of a first user on a target service is acquired; correlation data is determined between the first traffic time-series information and N second traffic time-series information of a second user on M services in pre-acquired target data. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and a threshold number of items within the neighborhood. The M services include the target service; based on the correlation data and the K classification sets, the classification set to which the first traffic time-series information belongs is determined, thus obtaining the traffic category of the first user. The traffic category indicates whether the first user's traffic is abnormal.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to the fields of Internet technology and machine learning technology, specifically to a traffic detection method, device, and electronic device. Background Technology

[0002] In recent years, internet technology has developed rapidly. In order to maintain the healthy development of internet services, it is necessary to detect abnormal traffic.

[0003] Currently, abnormal traffic detection for business operations is typically based on configuring dimensional statistical rules using a strategy engine. Summary of the Invention

[0004] This disclosure provides a flow detection method, apparatus, and electronic device.

[0005] According to a first aspect of this disclosure, a traffic detection method is provided, comprising:

[0006] Obtain the first user's first traffic time-series information on the target business;

[0007] The correlation data between the first traffic time-series information and the N second traffic time-series information of the second user in M ​​services in the pre-acquired target data is determined. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services. M is a positive integer, and N and K are both integers greater than 1.

[0008] Based on the correlation data and the K classification sets, the classification set to which the first traffic time series information belongs is determined, and the traffic category of the first user is obtained. The traffic category indicates whether the traffic of the first user is abnormal traffic.

[0009] According to a second aspect of this disclosure, a flow detection device is provided, comprising:

[0010] The first acquisition module is used to acquire the first traffic timing information of the first user on the target service;

[0011] The first determining module is used to determine the correlation data between the first traffic time-series information and the N second traffic time-series information of the second user on M services in the pre-acquired target data. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services, where M is a positive integer and N and K are both integers greater than 1.

[0012] The second determining module is used to determine the classification set to which the first traffic time series information belongs based on the correlation data and the K classification sets, and to obtain the traffic category of the first user. The traffic category indicates whether the traffic of the first user is abnormal traffic.

[0013] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0014] At least one processor; and

[0015] A memory that is communicatively connected to at least one processor; wherein,

[0016] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform any of the methods in the first aspect.

[0017] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform any of the methods in the first aspect.

[0018] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the methods in the first aspect.

[0019] The present disclosure solves the problem of low accuracy in category detection of business traffic in related technologies and improves the accuracy of category detection of business traffic.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a schematic flowchart of the traffic detection method according to the first embodiment of this disclosure;

[0023] Figure 2 This is a schematic diagram illustrating the clustering of N second traffic time-series information using DBSCAN;

[0024] Figure 3 This is a time-series trend chart of normal traffic.

[0025] Figure 4 It is a time-series trend chart of abnormal traffic;

[0026] Figure 5 This is a schematic diagram illustrating the process of determining the traffic category for the first user;

[0027] Figure 6 This is a schematic diagram of the flow detection device according to the second embodiment of the present disclosure;

[0028] Figure 7 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] First Embodiment

[0031] like Figure 1 As shown, this disclosure provides a traffic detection method, including the following steps:

[0032] Step S101: Obtain the first traffic timing information of the first user on the target service.

[0033] In this embodiment, the traffic detection method relates to the field of data processing technology, particularly to the fields of Internet technology and machine learning technology, and can be widely applied in business traffic detection scenarios. The traffic detection method of this disclosure embodiment can be executed by the traffic detection device of this disclosure embodiment. The traffic detection device of this disclosure embodiment can be configured in any electronic device to execute the traffic detection method of this disclosure embodiment.

[0034] The first user can be an Internet user, whose identifier can be an Internet Protocol (IP), a network camera (IP Camera, IPC), a user agent (User-Agent, UA), etc.

[0035] The target business can be any internet business, such as a search engine business; no specific limitation is made here.

[0036] The first traffic time series information can be the time series sequence of the business traffic of the first user, which can be a time series sequence of 1 minute, 5 minutes, 10 minutes or 30 minutes, corresponding to 1440-dimensional, 288-dimensional, 144-dimensional and 30-dimensional feature vectors respectively.

[0037] The first user's first traffic time series information can be one, such as a 10-minute time series. The first user's first traffic time series information can also be multiple, such as aggregating the first user's traffic on the target business according to 1 minute and 10 minutes respectively, to obtain a 1-minute time series and a 10-minute time series.

[0038] The number of first users can be one. In an application scenario, when the first user is detected, the traffic of the first user can be obtained from the target business line based on the identifier of the first user, such as the IP address. For example, the traffic data of the first user from 0:00 to 24:00 of the whole day can be obtained, and the traffic can be aggregated in 10-minute intervals to obtain the first traffic time sequence information.

[0039] The number of users can be multiple. For example, in another application scenario, the traffic of the target business line for one day can be obtained. Since the traffic time series of the business throughout the day is usually the behavior of multiple users, time series aggregation can be performed based on multiple user identifiers such as IP, IPC, and UA with a request scale. Depending on the specific scenario, the time series sequences of 1 minute, 5 minutes, 10 minutes, and 30 minutes of the relevant ID dimensions can be obtained, resulting in feature vectors of 1440 dimensions, 288 dimensions, 144 dimensions, and 30 dimensions, respectively. Thus, the first traffic time series information of multiple users in multiple time series dimensions can be obtained.

[0040] Step S102: Determine the correlation data between the first traffic time-series information and the N second traffic time-series information of the second user on M services in the pre-acquired target data. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services.

[0041] Where M is a positive integer, and N and K are both integers greater than 1.

[0042] In this step, the target data can be the training data for machine learning models such as density-based clustering (DBSCAN) algorithms.

[0043] The training data can be generated based on historical online log data from M services, including the target service. One or more days of online log data from the M services can be obtained. For the traffic of these service lines, time-series sequences can be generated based on ID identifiers such as IPC, IP, and UA, yielding second traffic time-series information. For each second user (which can be the user corresponding to the ID identifier in the training data), it can include time-series sequences across multiple time dimensions, such as 1-minute and 10-minute time-series sequences. Therefore, the matrix dimensions in the generated training data are 1000*1440 and 1000*144, respectively.

[0044] Where 1000 represents the number of the second user, i.e., the number of training samples, 1440 represents the feature dimension of the 1-minute time series, and 144 represents the feature dimension of the 10-minute time series.

[0045] Alternatively, the training data could include only a second traffic time series information in one time dimension, such as a 10-minute time series. However, by aggregating user traffic from multiple time dimensions, the user's time series characteristics can be expressed more accurately, thereby improving the accuracy of traffic category detection.

[0046] Furthermore, when M is greater than 1, training data is obtained by acquiring historical online log data from multiple services, and then traffic category detection is performed on the target service based on the training data. This can improve the generalization of the machine learning model and enable it to be applied to multiple services.

[0047] Furthermore, once the second traffic time-series information of the training samples in the business is obtained, DBSCAN can be used to cluster N second traffic time-series information to aggregate similar traffic time-series information together to form clusters.

[0048] Specifically, N second-order flow time series information can be input into DBSCAN, with a pre-set neighborhood radius such as e = 0.3 and a neighborhood quantity threshold, where the neighborhood quantity threshold represents the minimum number of samples required to form a cluster. The type of each point, such as whether it is a core point or an edge point, can be determined by scanning (i.e., comparing the cosine distance or Pearson correlation coefficient between each second-order flow time series information). One point represents one second-order flow time series information.

[0049] Extract one unprocessed point from N points. If the extracted point is a core point, find all objects density-connected to that point to form a cluster. Density adjacency means that the distance between two points is less than or equal to the neighborhood radius. If the extracted point is an edge point, exit the loop for finding density-connected points and continue searching for the next point until all points have been processed and all generated clusters are output, meeting the density requirement.

[0050] The following is a diagram illustrating the clustering of N second-order traffic time-series information using DBSCAN. Figure 2 As shown, through scanning, point A can be identified as the core point, points B and C as boundary points, and point D as a noise point. For each point, all points with density adjacent to that point can be found using the neighborhood radius e. When the number of points exceeds a threshold (which can be 3) within the neighborhood, a cluster is formed. Figure 2 It can be seen that, except for point D (which is a noise point), all the other samples form a cluster.

[0051] The above process can be called the DBSCAN training process. Upon completion of training, it can output K classification sets, such as 10 clusters and 18 noise points. Each cluster and each noise point can be considered a classification set. Noise points refer to points that cannot form clusters.

[0052] Correspondingly, the category attributes of the classification set can be manually labeled based on the trend of the traffic time series, such as a set of second traffic time series information with the category attribute of normal traffic or a set of second traffic time series information with the category attribute of abnormal traffic. The time series trend of normal traffic is usually represented as... Figure 3 As shown, the time series trend of abnormal traffic is usually expressed as... Figure 4 As shown.

[0053] Subsequently, traffic detection can be performed based on the trained DBSCAN. Specifically, the first traffic time series information or a time series matrix composed of the first traffic time series information can be input into the DBSCAN model. The DBSCAN model can determine the Pearson correlation coefficient between the first traffic time series information and the second traffic time series information in the training data, thus obtaining the correlation data.

[0054] Step S103: Based on the correlation data and the K classification sets, determine the classification set to which the first traffic time series information belongs, and obtain the traffic category of the first user. The traffic category indicates whether the traffic of the first user is abnormal traffic.

[0055] In this step, based on the correlation data, the second traffic time series information that is closest to the first traffic time series information can be determined. The classification set corresponding to the second traffic time series information can be determined as the classification set to which the first traffic time series information belongs. Based on the category attribute to which the classification set belongs, the traffic category of the first user can be determined.

[0056] The traffic category can be abnormal traffic or normal traffic. If the category attribute of the first traffic time series information to which the category set belongs is normal traffic, then the traffic of the first user is determined to be normal traffic. If the category attribute of the first traffic time series information to which the category set belongs is abnormal traffic, then the traffic of the first user is abnormal traffic, and the first user is an abnormal user.

[0057] like Figure 5 As shown, there are K classification sets: classification set 1, classification set 2, classification set 3, and classification set 4. Classification set 1 belongs to the category attribute of normal traffic, classification sets 2 and 3 belong to the category attribute of abnormal traffic, and classification set 4 represents noise points, which can also be classified as abnormal traffic. If the first user's first traffic time-series information (denoted by A1) is closest to the second user's second traffic time-series information (denoted by B1) in classification set 2 (i.e., has the best correlation), then the first user's traffic is determined to be abnormal traffic.

[0058] Correspondingly, upon detecting the traffic category of the first user, actions can be taken regarding the traffic of the first user or the first user themselves. For example, if the traffic category of the first user is abnormal traffic, the posting of abnormal traffic can be prohibited or the account of the first user can be banned. Conversely, if the traffic category of the first user is normal traffic, normal traffic can be recalled.

[0059] In this embodiment, by aggregating the traffic of M services into N sets of second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood, K classification sets are obtained. Based on the K classification sets and the correlation data between the first and second traffic time-series information, the traffic of the first user is detected to determine the traffic category of the first user. This improves the accuracy of service traffic category detection and enhances the generalization of traffic detection.

[0060] Optionally, obtaining the first user's first traffic timing information on the target service includes at least one of the following:

[0061] Obtain the first online log data of the target service, and statistically analyze the first online log data to obtain the first traffic time sequence information of the first user on the target service;

[0062] Obtain the risk traffic data of the target service, perform statistics on the risk traffic data, and obtain the first traffic time sequence information of the first user on the target service. The risk traffic data is the risk traffic detected from the online log data of the target service through the risk traffic feature set configured under the target service.

[0063] In this embodiment, in one application scenario, abnormal traffic can be detected for a target service. Specifically, the first online log data of the target service can be acquired on a daily basis. This first online log data is then statistically analyzed to aggregate time-series sequences (e.g., 10-minute intervals) based on multiple user IDs within the target service's online traffic, resulting in a time-series matrix. This matrix is ​​then input into a DBSCAN model, which outputs user traffic tags, such as the category set to which the user belongs. This allows for the detection of abnormal online traffic for the target service.

[0064] In another application scenario, the target business line typically detects abnormal traffic by configuring rule sets, i.e., risk traffic feature sets. If a traffic pattern matches a risk traffic feature set, it is determined to be risky traffic. However, due to imperfections in the configuration of risk traffic feature sets, the identified risky traffic may be misjudged. Therefore, traffic inspection can be performed on the risky traffic data of the target business. The purpose is to recall normal traffic from the risky traffic data to the business line, thereby reducing the probability of false positives through two inspections and improving the user experience.

[0065] Specifically, risky traffic data for the target business can be acquired, and statistical analysis can be performed on this data. Based on multiple user IDs within this risky traffic data, time-series sequences (e.g., 10-minute intervals) can be aggregated to obtain a time-series sequence matrix. This matrix is ​​then input into the DBSCAN model, which outputs user traffic tags, indicating the category set to which the user belongs. Thus, when the traffic tag indicates normal traffic, it can be retrieved from the risky traffic data, thereby achieving the retrieval of normal traffic and further improving the accuracy of online traffic category detection for the target business.

[0066] Optionally, for a second user, the number of second traffic time-series information of the second user includes at least two. The at least two second traffic time-series information of the second user are obtained by aggregating traffic from different time dimensions based on the identifier of the second user. One time dimension corresponds to one second traffic time-series information. The number of first traffic time-series information of the first user is the same as the number of second traffic time-series information of the second user, and at least two first traffic time-series information and at least two second traffic time-series information correspond one-to-one in the time dimension.

[0067] In this embodiment, during model training, for each training sample, time series sequences under multiple time dimensions can be aggregated, such as time series sequences of 1 minute, 5 minutes, 10 minutes and 30 minutes. Based on the time series sequences of the training samples under each time dimension, multiple classification sets can be trained, and these multiple classification sets can be aggregated to obtain K classification sets.

[0068] Correspondingly, during prediction, the time series sequence of the first user in the corresponding time dimension can be obtained, and the Pearson correlation coefficient between the time series sequences of the first user and the second user in the same time dimension can be determined. In this way, the traffic category of the first user can be evaluated from multiple time dimensions, further improving the accuracy of business traffic category detection.

[0069] Optionally, step S103 specifically includes:

[0070] For each time dimension, based on the correlation data between the first traffic time series information and the second traffic time series information under the time dimension and the classification set under the time dimension in the K classification sets, the classification set to which the first traffic time series information under the time dimension belongs is determined;

[0071] Based on the category set to which the first traffic time-series information belongs under each time dimension and the category attributes of the category set, the traffic category of the first user is determined.

[0072] In this embodiment, for each time dimension, the traffic category of the first user under that time dimension can be determined according to the above-described traffic category determination method. For example, the category set under the one-minute time dimension includes category set A1, category set B1, and category set C1. Based on the correlation data under this time dimension and these category sets, the category set to which the first traffic time series information belongs is determined to be A1. The category set under the 10-minute time dimension includes category set A2, category set B2, and category set C2. Based on the correlation data under this time dimension and these category sets, the category set to which the first traffic time series information belongs is determined to be B2. The category set under the 30-minute time dimension includes category set A3, category set B3, and category set C3. Based on the correlation data under this time dimension and these category sets, the category set to which the first traffic time series information belongs is determined to be B3.

[0073] Then, based on the category attributes of category set A1, category set B2 and category set B3, three category labels for the first user can be determined, such as two category labels for normal traffic and one category label for abnormal traffic. The category corresponding to the same category label with a number exceeding a certain threshold can be determined as the traffic category of the first user.

[0074] In this way, by aggregating user traffic from multiple time dimensions, the temporal characteristics of users can be expressed more accurately, thereby improving the accuracy of traffic category detection.

[0075] Optionally, the K classification sets include a first target classification set, which is a classification set that did not form a cluster when clustering the N second traffic time-series information. After step S103, the method further includes:

[0076] Obtain the second online log data from the M services;

[0077] By statistically analyzing the second online log data, the third user's third traffic time-series information on the M services is obtained;

[0078] If, based on the third traffic time-series information and the N second traffic time-series information, the classification set to which the third traffic time-series information belongs is determined to be the first target classification set, then a first traffic feature shared by the second traffic time-series information and the third traffic time-series information under the first target classification set is determined.

[0079] Add the first traffic feature to the risk traffic feature set configured under the M services.

[0080] In this implementation, abnormal risk features can be mined. Abnormal risk features refer to common characteristics of abnormal traffic, such as: "All IPs are from Internet Data Centers (IDCs), and a certain identifier field is frequently changed." This allows the identifier field to be generated relatively recently, thereby bypassing the detection of the risk traffic feature set deployed on the target business line. Therefore, it is necessary to continuously mine new abnormal risk features to improve the risk traffic feature set and enhance its accuracy in detecting abnormal traffic.

[0081] In one alternative implementation, the abnormal risk features can be mined by utilizing the characteristics of the noise points (such as the category attribute of the noise points being abnormal traffic) during DBSCAN model training. The classification set corresponding to the noise points is the first target classification set.

[0082] Specifically, second-line log data from M business lines can be obtained on a daily basis. Statistics can be performed on the second-line log data to aggregate time-series sequences, such as 10-minute sequences, based on the multi-user IDs of these business lines, to obtain third-line traffic time-series information.

[0083] Then, the third traffic time series information can be input into the DBSCAN model. The DBSCAN model can determine the classification set to which the third traffic time series information belongs based on the third traffic time series information and the N second traffic time series information. The specific determination method is similar to that for determining the classification set to which the first traffic time series information belongs, and will not be elaborated here.

[0084] When the category attribute of the classification set to which the third traffic time series information belongs is a noise point, that is, the classification set is the first target classification set, the first traffic feature shared by the second traffic time series information and the third traffic time series information under the first target classification set can be determined. Specifically, the first traffic feature shared by the user traffic data corresponding to the second traffic time series information and the user traffic data corresponding to the third traffic time series information can be determined. This first traffic feature can be a traffic feature other than the time series feature, such as a field feature.

[0085] Correspondingly, the first traffic feature can be added to the risk traffic feature set. This enables abnormal traffic detection of business traffic with the first traffic feature, thereby improving the accuracy of abnormal traffic detection.

[0086] Optionally, the method further includes:

[0087] Obtain the abnormal users of the M services and the second traffic characteristics of the abnormal users;

[0088] By statistically analyzing the third online log data on the M services, the fourth user's fourth traffic time-series information on the M services is obtained;

[0089] Target traffic time series information that matches the second traffic feature is selected from the fourth traffic time series information;

[0090] The target traffic time-series information is clustered to obtain a second target classification set. The second target classification set is the classification set of clusters formed when the target traffic time-series information is clustered. The second target classification set includes the traffic time-series information of the abnormal user.

[0091] Identify the third traffic feature shared by the traffic time-series information under the second target classification set;

[0092] The third traffic feature is added to the risk traffic feature set configured under the M services.

[0093] In this implementation, abnormal risk characteristics can be mined by training a DBSCAN model based on M abnormal users from various services. The identifier of an abnormal user can be an IP address.

[0094] Specifically, the business risk traffic feature set detected some abnormal IPs. Through business feedback, some abnormal IPs were found to have gone undetected. It was confirmed that there is a type of traffic that bypasses the risk traffic feature set continuously throughout the day but at low frequency (i.e., the second traffic feature). It is necessary to locate the abnormal characteristics of this type of low-frequency traffic.

[0095] It is possible to perform statistics on the third online log data of M services, and based on the multi-user IDs of these service lines, aggregate time-series sequences such as 10-minute intervals to obtain the fourth traffic time-series information.

[0096] Next, target traffic time series information matching the second traffic characteristic can be filtered from the fourth traffic time series information. For example, if the second traffic characteristic is continuous throughout the day but low frequency, IPs with a request count greater than a certain threshold, such as 20,000, can be filtered. The base number of requests can be configured according to the specific business scenario to discover IPs of the same magnitude as the IPs reported as abnormal by the business. The output IP count is 580. Using the time series stationarity test, 121 stationary time series sequences are obtained. These 121 time series sequences are the target traffic time series information matching the second traffic characteristic, and a time series sequence matrix with a dimension of 121*144 can be output.

[0097] The target traffic time-series information can be input into the DBSCAN model for clustering, resulting in multiple classification sets. These multiple classification sets can include a second target classification set, which can include the traffic time-series information of abnormal users. In addition, it can also include other similar abnormal IPs that have not been detected.

[0098] A third traffic feature shared by the traffic time-series information under the second target classification set can be determined. Specifically, this can be achieved by determining the third traffic feature shared by the user traffic data corresponding to the traffic time-series information under the second target classification set. This third traffic feature can be a traffic feature other than the time-series feature, such as a field feature.

[0099] The third traffic feature is added to the risk traffic feature set configured for the M services. This enables abnormal traffic detection for service traffic exhibiting the third traffic feature, thereby improving the accuracy of abnormal traffic detection.

[0100] Second Embodiment

[0101] like Figure 6 As shown, this disclosure provides a flow detection device 600, comprising:

[0102] The first acquisition module 601 is used to acquire the first traffic timing information of the first user on the target service;

[0103] The first determining module 602 is used to determine the correlation data between the first traffic time-series information and the N second traffic time-series information of the second user on M services in the pre-acquired target data. The target data also includes K classification sets to which the N second traffic time-series information belongs. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services, where M is a positive integer and N and K are both integers greater than 1.

[0104] The second determining module 603 is used to determine the classification set to which the first traffic time series information belongs based on the correlation data and the K classification sets, and to obtain the traffic category of the first user. The traffic category indicates whether the traffic of the first user is abnormal traffic.

[0105] Optionally, the first acquisition module 601 is specifically used for:

[0106] Obtain the first online log data of the target service, and statistically analyze the first online log data to obtain the first traffic time sequence information of the first user on the target service;

[0107] Obtain the risk traffic data of the target service, perform statistics on the risk traffic data, and obtain the first traffic time sequence information of the first user on the target service. The risk traffic data is the risk traffic detected from the online log data of the target service through the risk traffic feature set configured under the target service.

[0108] Optionally, for a second user, the number of second traffic time-series information of the second user includes at least two. The at least two second traffic time-series information of the second user are obtained by aggregating traffic from different time dimensions based on the identifier of the second user. One time dimension corresponds to one second traffic time-series information. The number of first traffic time-series information of the first user is the same as the number of second traffic time-series information of the second user, and at least two first traffic time-series information and at least two second traffic time-series information correspond one-to-one in the time dimension.

[0109] Optionally, the second determining module 603 is specifically used for:

[0110] For each time dimension, based on the correlation data between the first traffic time series information and the second traffic time series information under the time dimension and the classification set under the time dimension in the K classification sets, the classification set to which the first traffic time series information under the time dimension belongs is determined;

[0111] Based on the category set to which the first traffic time-series information belongs under each time dimension and the category attributes of the category set, the traffic category of the first user is determined.

[0112] Optionally, the K classification sets include a first target classification set, which is a classification set that did not form a cluster when clustering the N second traffic time-series information. The device further includes:

[0113] The second acquisition module is used to acquire the second online log data on the M services;

[0114] The first statistics module is used to perform statistics on the second online log data to obtain the third user's third traffic time-series information on the M services;

[0115] The third determining module is used to determine a first traffic feature common to the second traffic time series information and the third traffic time series information when the classification set to which the third traffic time series information belongs is determined to be the first target classification set based on the third traffic time series information and the N second traffic time series information.

[0116] The first addition module is used to add the first traffic feature to the risk traffic feature set configured under the M services.

[0117] Optionally, the device further includes:

[0118] The third acquisition module is used to acquire abnormal users of the M services and the second traffic characteristics of the abnormal users;

[0119] The second statistics module is used to statistically analyze the third online log data on the M services to obtain the fourth traffic time sequence information of the fourth user on the M services.

[0120] The filtering module is used to filter out target traffic time-series information that matches the second traffic feature from the fourth traffic time-series information;

[0121] The clustering module is used to cluster the target traffic time-series information to obtain a second target classification set. The second target classification set is the classification set of clusters formed when the target traffic time-series information is clustered. The second target classification set includes the traffic time-series information of the abnormal user.

[0122] The fourth determining module is used to determine the third traffic feature shared by the traffic time-series information under the second target classification set;

[0123] The second adding module is used to add the third traffic feature to the risk traffic feature set configured under the M services.

[0124] The flow detection device 600 provided in this disclosure can implement all the processes implemented in the flow detection method embodiments and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0125] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0126] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0127] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0128] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0129] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0130] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the flow detection method. For example, in some embodiments, the flow detection method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the flow detection method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the flow detection method by any other suitable means (e.g., by means of firmware).

[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0136] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0137] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0138] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A flow detection method, comprising: Obtain the first user's first traffic time-series information on the target business; The correlation data between the first traffic time-series information and the N second traffic time-series information of the second user in M ​​services in the pre-acquired target data is determined. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services. M is a positive integer, and N and K are both integers greater than 1. Based on the correlation data and the K classification sets, the classification set to which the first traffic time series information belongs is determined, and the traffic category of the first user is obtained. The traffic category indicates whether the traffic of the first user is abnormal traffic. The target data is the training data for the machine learning model. The training data is generated based on historical online log data from M services. For the historical online log data from the M services, a time series is generated based on the identifier of the second user to obtain the second traffic time series information. For a second user, the number of second traffic time series information for the second user includes at least two. The at least two second traffic time series information for the second user are obtained by aggregating traffic from different time dimensions based on the identifier of the second user. One time dimension corresponds to one second traffic time series information. The number of first traffic time series information for the first user is the same as the number of second traffic time series information for the second user, and at least two first traffic time series information correspond one-to-one with at least two second traffic time series information in the time dimension.

2. The method according to claim 1, wherein, The acquisition of the first user's first traffic time-series information on the target service includes at least one of the following: Obtain the first online log data of the target service, and statistically analyze the first online log data to obtain the first traffic time sequence information of the first user on the target service; Obtain the risk traffic data of the target service, perform statistics on the risk traffic data, and obtain the first traffic time sequence information of the first user on the target service. The risk traffic data is the risk traffic detected from the online log data of the target service through the risk traffic feature set configured under the target service.

3. The method according to claim 1, wherein, The step of determining the category set to which the first traffic time-series information belongs, based on the correlation data and the K category sets, to obtain the traffic category of the first user includes: For each time dimension, based on the correlation data between the first traffic time series information and the second traffic time series information under the time dimension and the classification set under the time dimension in the K classification sets, the classification set to which the first traffic time series information under the time dimension belongs is determined; Based on the category set to which the first traffic time-series information belongs under each time dimension and the category attributes of the category set, the traffic category of the first user is determined.

4. The method according to claim 1, wherein, The K classification sets include a first target classification set, which is a classification set that did not form clusters when clustering the N second traffic time-series information. After determining the classification set to which the first traffic time-series information belongs based on the correlation data and the K classification sets to obtain the traffic category of the first user, the method further includes: Obtain the second online log data from the M services; By statistically analyzing the second online log data, the third user's third traffic time-series information on the M services is obtained; If, based on the third traffic time-series information and the N second traffic time-series information, the classification set to which the third traffic time-series information belongs is determined to be the first target classification set, then a first traffic feature shared by the second traffic time-series information and the third traffic time-series information under the first target classification set is determined. Add the first traffic feature to the risk traffic feature set configured under the M services.

5. The method according to claim 1, further comprising: Obtain the abnormal users of the M services and the second traffic characteristics of the abnormal users; By statistically analyzing the third online log data on the M services, the fourth user's fourth traffic time-series information on the M services is obtained; Target traffic time series information that matches the second traffic feature is selected from the fourth traffic time series information; The target traffic time-series information is clustered to obtain a second target classification set. The second target classification set is the classification set of clusters formed when the target traffic time-series information is clustered. The second target classification set includes the traffic time-series information of the abnormal user. Identify the third traffic feature shared by the traffic time-series information under the second target classification set; The third traffic feature is added to the risk traffic feature set configured under the M services.

6. A flow detection device, comprising: The first acquisition module is used to acquire the first traffic timing information of the first user on the target service; The first determining module is used to determine the correlation data between the first traffic time-series information and the N second traffic time-series information of the second user on M services in the pre-acquired target data. The target data also includes K classification sets to which the N second traffic time-series information belong. The K classification sets are obtained by clustering the N second traffic time-series information according to the neighborhood radius and the number threshold within the neighborhood of the second traffic time-series information. The M services include the target services, where M is a positive integer and N and K are both integers greater than 1. The second determining module is used to determine the classification set to which the first traffic time series information belongs based on the correlation data and the K classification sets, and to obtain the traffic category of the first user. The traffic category indicates whether the traffic of the first user is abnormal traffic. The target data is the training data for the machine learning model. The training data is generated based on historical online log data from M services. For the historical online log data from the M services, a time series is generated based on the identifier of the second user to obtain the second traffic time series information. For a second user, the number of second traffic time series information for the second user includes at least two. The at least two second traffic time series information for the second user are obtained by aggregating traffic from different time dimensions based on the identifier of the second user. One time dimension corresponds to one second traffic time series information. The number of first traffic time series information for the first user is the same as the number of second traffic time series information for the second user, and at least two first traffic time series information correspond one-to-one with at least two second traffic time series information in the time dimension.

7. The apparatus according to claim 6, wherein, The first acquisition module is specifically used for: Obtain the first online log data of the target service, and statistically analyze the first online log data to obtain the first traffic time sequence information of the first user on the target service; Obtain the risk traffic data of the target service, perform statistics on the risk traffic data, and obtain the first traffic time sequence information of the first user on the target service. The risk traffic data is the risk traffic detected from the online log data of the target service through the risk traffic feature set configured under the target service.

8. The apparatus according to claim 6, wherein, The second determining module is specifically used for: For each time dimension, based on the correlation data between the first traffic time series information and the second traffic time series information under the time dimension and the classification set under the time dimension in the K classification sets, the classification set to which the first traffic time series information under the time dimension belongs is determined; Based on the category set to which the first traffic time-series information belongs under each time dimension and the category attributes of the category set, the traffic category of the first user is determined.

9. The apparatus according to claim 6, wherein, The K classification sets include a first target classification set, which is a classification set that did not form a cluster when the N second traffic time-series information were clustered. The device further includes: The second acquisition module is used to acquire the second online log data on the M services; The first statistics module is used to perform statistics on the second online log data to obtain the third user's third traffic time-series information on the M services; The third determining module is used to determine a first traffic feature common to the second traffic time series information and the third traffic time series information when the classification set to which the third traffic time series information belongs is determined to be the first target classification set based on the third traffic time series information and the N second traffic time series information. The first addition module is used to add the first traffic feature to the risk traffic feature set configured under the M services.

10. The apparatus according to claim 6, further comprising: The third acquisition module is used to acquire abnormal users of the M services and the second traffic characteristics of the abnormal users; The second statistics module is used to statistically analyze the third online log data on the M services to obtain the fourth traffic time sequence information of the fourth user on the M services. The filtering module is used to filter out target traffic time-series information that matches the second traffic feature from the fourth traffic time-series information; The clustering module is used to cluster the target traffic time-series information to obtain a second target classification set. The second target classification set is the classification set of clusters formed when the target traffic time-series information is clustered. The second target classification set includes the traffic time-series information of the abnormal user. The fourth determining module is used to determine the third traffic feature shared by the traffic time-series information under the second target classification set; The second adding module is used to add the third traffic feature to the risk traffic feature set configured under the M services.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images