Cumulative theft detection method and device for network traffic, computer equipment and readable storage medium

Abstract

This application relates to a method, apparatus, computer device, and readable storage medium for detecting cumulative data theft in network traffic. The method includes: acquiring access data of a target object within the current time period; extracting features from the access data according to a preset time granularity to obtain feature values ​​of multiple access behavior features of the target object; determining the current anomaly score of the target object based on the feature values ​​of the multiple access behavior features; obtaining a cumulative data theft anomaly index based on the current anomaly score and historical anomaly scores of the target object; and determining the cumulative data theft behavior detection result of the target object based on the cumulative data theft anomaly index. This method can effectively monitor suspected cumulative data theft behavior that may exist in network traffic data.

Description

Technical Field

[0001] This application relates to the field of information security technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting the cumulative theft of network traffic. Background Technology

[0002] With the development of information technology, many business systems store a large amount of valuable data, and data theft is a problem that must be faced on the modern Internet. Currently, commonly used defense and detection methods include: monitoring and tracking the flow of sensitive and important data in the network to identify potential data leakage; collecting and analyzing access information and historical access information to obtain access risk assessment values; analyzing real-time browsing information based on access risk assessment values ​​and historical browsing information to determine whether there are anomalies; calculating silent period difference factors based on traffic data within a time window to construct abnormal time series, detecting data transmission behavior, identifying abnormal transmission deviations, and calculating protocol switching rates based on traffic data sets to capture typical low-frequency, intermittent communication patterns in APT (Advanced Persistent Threat) attacks.

[0003] However, current common data theft detection methods are often limited to monitoring highly sensitive data, short-term high-frequency access, download volume detection, and web crawler detection, which easily overlook the time-cumulative effect of network data theft. Furthermore, the features used for detection are relatively simple, and there is a lack of effective detection methods for long-term cumulative data theft scenarios in the network. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problem that the above methods cannot effectively detect long-term cumulative data theft in networks by providing a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting cumulative data theft in network traffic.

[0005] Firstly, this application provides a method for detecting cumulative network traffic theft. The method includes:

[0006] Obtain access data of the monitored object within the current time period;

[0007] According to a preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object;

[0008] Based on the feature values ​​of the multiple access behavior characteristics, the current anomaly score of the detected object is determined;

[0009] Based on the current and historical abnormal scores of the detected object, the cumulative theft anomaly index is obtained;

[0010] Based on the cumulative theft anomaly index, the cumulative theft behavior detection result of the detected object is determined.

[0011] In one embodiment, there are multiple objects to be detected, and determining the current anomaly score of the detected objects based on the feature values ​​of the multiple access behavior characteristics includes:

[0012] Obtain the target weight of each of the aforementioned access behavior features within the current time period;

[0013] For each detected object, the feature values ​​of each access behavior feature of the detected object are weighted and aggregated according to the target weight to obtain the current anomaly score of the detected object.

[0014] In one embodiment, obtaining the target weight of each of the access behavior features within the current time period includes:

[0015] Obtain the initial weights of each of the aforementioned access behavior features;

[0016] For each access behavior feature, the initial weight of the access behavior feature is updated based on the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period, to obtain the target weight.

[0017] In one embodiment, updating the initial weight of the access behavior feature based on the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period to obtain the target weight includes:

[0018] If, based on the distribution information, the distribution difference of each detected object on the access behavior feature is greater than a first threshold, the initial weight of the access behavior feature is increased to obtain the target weight.

[0019] If, based on the distribution information, the distribution difference of each detected object on the access behavior feature is less than a second threshold, the initial weight of the access behavior feature is reduced to obtain the target weight.

[0020] In one embodiment, the step of weighted aggregation of the feature values ​​of each access behavior feature of the detected object according to the target weight to obtain the current anomaly score of the detected object includes:

[0021] Determine a first type of feature and a second type of feature among the various access behavior features; the first type of feature is a feature in which the order of magnitude difference of the feature values ​​of different detection objects under this type of feature is greater than a difference threshold, and the second type of feature is a feature in which the order of magnitude difference of the feature values ​​of different detection objects under this type of feature is less than the difference threshold;

[0022] The feature values ​​of the first type of feature and the feature values ​​of the second type of feature are respectively subjected to feature transformation processing to obtain the first transformed feature value and the second transformed feature value;

[0023] Based on the first transformation feature value and the second transformation feature value, the processed feature value of each access behavior feature is obtained;

[0024] Based on the target weight, the processed feature values ​​of each access behavior feature are weighted and aggregated to obtain the current anomaly score of the detected object.

[0025] In one embodiment, feature transformation processing is performed on the feature values ​​of the first type of feature to obtain a first transformed feature value, including:

[0026] Determine the maximum feature value of each detection object under the first type of feature, and perform feature transformation processing on the feature value of each detection object under the first type of feature based on the maximum feature value to obtain the first transformed feature value of each detection object under the first type of feature;

[0027] The feature values ​​of the second type of feature are subjected to feature transformation processing to obtain the second transformed feature values, including:

[0028] Determine the average feature value of each detection object under the second type of feature, and perform feature transformation processing on the feature value of each detection object under the second type of feature based on the average feature value to obtain the second transformed feature value of each detection object under the second type of feature.

[0029] In one embodiment, the historical anomaly score includes anomaly scores from multiple historical time periods; the step of obtaining the cumulative theft anomaly index based on the current anomaly score and the historical anomaly score of the detected object includes:

[0030] Obtain the first weight of the current time period and the second weight of each of the historical time periods; the first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with the time distance of the historical time period relative to the current time period;

[0031] Based on the first weight and the second weight, the current abnormal score and the historical abnormal scores of each of the historical time periods are weighted and aggregated to obtain the cumulative theft abnormality index.

[0032] Secondly, this application also provides a device for detecting the cumulative theft of network traffic. The device includes:

[0033] The data acquisition module is used to acquire access data of the detected object within the current time period;

[0034] The feature extraction module is used to extract features from the access data according to a preset time granularity to obtain feature values ​​of multiple access behavior features of the detected object;

[0035] The score determination module is used to determine the current anomaly score of the detected object based on the feature values ​​of the multiple access behavior features;

[0036] The score fusion module is used to obtain the cumulative theft anomaly index based on the current abnormal score and historical abnormal score of the detected object;

[0037] An anomaly detection module is used to determine the cumulative theft behavior detection result of the detected object based on the cumulative theft anomaly index.

[0038] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0039] Obtain access data of the monitored object within the current time period;

[0040] According to a preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object;

[0041] Based on the feature values ​​of the multiple access behavior characteristics, the current anomaly score of the detected object is determined;

[0042] Based on the current and historical abnormal scores of the detected object, the cumulative theft anomaly index is obtained;

[0043] Based on the cumulative theft anomaly index, the cumulative theft behavior detection result of the detected object is determined.

[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0045] Obtain access data of the monitored object within the current time period;

[0046] According to a preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object;

[0047] Based on the feature values ​​of the multiple access behavior characteristics, the current anomaly score of the detected object is determined;

[0048] Based on the current and historical abnormal scores of the detected object, the cumulative theft anomaly index is obtained;

[0049] Based on the cumulative theft anomaly index, the cumulative theft behavior detection result of the detected object is determined.

[0050] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0051] Obtain access data of the monitored object within the current time period;

[0052] According to a preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object;

[0053] Based on the feature values ​​of the multiple access behavior characteristics, the current anomaly score of the detected object is determined;

[0054] Based on the current and historical abnormal scores of the detected object, the cumulative theft anomaly index is obtained;

[0055] Based on the cumulative theft anomaly index, the cumulative theft behavior detection result of the detected object is determined.

[0056] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for detecting cumulative network traffic theft extracts features from the access data of the target object within the current time period according to a preset time granularity, obtaining feature values ​​of the target object under multiple access behavior features; determines the current abnormal score of the target object based on the feature values ​​under multiple access behavior features; obtains the cumulative theft anomaly index based on the current abnormal score and historical abnormal scores of the target object; and determines the cumulative theft behavior detection result of the target object based on the cumulative theft anomaly index. This method starts by analyzing the network access behavior of the target, and uses feature extraction and multi-feature anomaly detection to detect potential cumulative data theft. It can effectively detect suspected cumulative data theft in network traffic data, solving the technical problems of most current traffic detection methods that only focus on highly sensitive data, have short detection time periods, detect single behavioral features, and lack effective detection methods for suspected cumulative data theft in network traffic data. Furthermore, by combining access behavior features with time granularity to generate multiple detection feature selection methods closely related to cumulative data theft, it considers the cumulative effect of time and the diversity of data theft behavior, and evaluates the access behavior of the target from a more comprehensive perspective, which can improve the accuracy of identifying data theft behavior of the target. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating a method for detecting cumulative network traffic theft in one embodiment;

[0058] Figure 2 This is a flowchart illustrating the steps for determining the anomaly score of a detected object in one embodiment.

[0059] Figure 3 This is a flowchart illustrating the steps for determining the cumulative theft anomaly index in one embodiment;

[0060] Figure 4 This is a flowchart illustrating a method for detecting cumulative network traffic theft in another embodiment;

[0061] Figure 5 This is a structural block diagram of a network traffic cumulative theft detection device in one embodiment;

[0062] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0065] Accumulation of data voyeur refers to the act of a risky entity repeatedly and persistently extracting large volumes of data from a system via multiple channels online, potentially posing data security risks such as information leakage. For business systems in various industries, intruders not only steal highly confidential business information through network attacks, but sometimes also repeatedly and persistently extract data that may seem less important individually, but whose accumulated volume can yield crucial information about the business system. Therefore, preventing accumulation of data voyeur attacks is essential. However, most current data security defenses and technologies focus on protecting critical and sensitive data, often neglecting the significant cumulative effect of relatively non-critical data.

[0066] Based on this, this application proposes a multi-feature anomaly detection algorithm that combines time granularity. Starting from the network access behavior of the detected object, it extracts and processes a large number of features of the detected object's access behavior over a long period of time, uses the anomaly detection algorithm to obtain the risk score of the detected object within the time period, sets an appropriate threshold to determine whether there is a suspected risk of cumulative data theft, extracts and displays the performance and evidence of the network access behavior of the detected risky object, and designs an anomaly monitoring device based on this algorithm model, which can effectively detect possible cumulative data theft behavior in the network traffic monitoring system.

[0067] In one embodiment, such as Figure 1 As shown, a method for detecting cumulative network traffic theft is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to terminals, and to systems including terminals and servers, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. The server can be a standalone server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0068] Step S110: Obtain access data of the detected object within the current time period.

[0069] The detection target refers to the entity that accesses network traffic data, such as the source IP and account in the traffic storage log.

[0070] Specifically, a log audit table of network traffic data of the monitored object within the current time period can be collected as raw access data. Further preprocessing of the raw access data yields access data to ensure data integrity and consistency and improve data quality. Preprocessing may include at least one of the following: data cleaning, attribute alignment, and filling in missing data.

[0071] For example, set the time period to one week, and determine the start and end times of this period (e.g., from midnight on Monday to midnight on the following Monday). Collect and audit the log records of network traffic data as raw access data, perform data cleaning, store the access record attributes in the prescribed format, and fill in missing and blank values ​​to maintain data format consistency and facilitate subsequent analysis.

[0072] In some embodiments, the detection object can be determined by: acquiring access data of multiple candidate detection objects within the current time period; counting the number of access records of each candidate detection object based on the access data of each candidate detection object; and selecting candidate detection objects from multiple candidate detection objects whose number of access records is greater than a threshold as detection objects.

[0073] Specifically, the system can count the number of access records for multiple candidate detection objects within the current time period, set a threshold to filter out detection objects with low access volume within the current time period, and select other detection objects with more than the threshold for subsequent testing. For example, setting a threshold N=1000, the system can count the number of access records for all source IPs in the audit log within the current time period, filter out source IPs with fewer than 1000 access records, and record the remaining source IPs for subsequent analysis.

[0074] Step S120: According to the preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object.

[0075] The preset time granularity can be full time, working hours, nighttime, weekdays, and rest days, etc.

[0076] Among them, access behavior characteristics can include the number of access records, the amount of data pulled, the number of APIs accessed, the number of access IPs, the number of cities where the source IP (i.e. the object being detected) is located, and the number of returned data tags.

[0077] In practice, feature extraction is performed on the access data according to each time granularity to obtain feature values ​​of multiple access behavior features of the detected object at different time granularities.

[0078] In this step, combining multiple access behavior features of the target object with time granularity allows for a more comprehensive analysis of the target object's access behavior.

[0079] Step S130: Determine the current anomaly score of the detected object based on the feature values ​​of multiple access behavior features.

[0080] In practice, each access behavior feature can be weighted, and the feature values ​​of each access behavior feature can be weighted according to the target weight of each feature to obtain multiple weighted access behavior features. Multi-feature anomaly detection is then performed using these multiple weighted access behavior features to determine the current anomaly score of the detected object in the current time period.

[0081] Specifically, anomaly detection can be performed on each weighted access behavior feature using an anomaly detection model. For example, anomaly detection models can be built based on methods such as linear regression (LR), Isolation Forest (IForest), principal component analysis (PCA), and one-class support vector machine (OCSVM).

[0082] Furthermore, in some embodiments, after obtaining the current outlier score, the outlier score can be transformed into a value between 0 and 100 to enhance the readability and comparability of the outlier score.

[0083] Step S140: Obtain the cumulative theft anomaly index based on the current anomaly score and historical anomaly score of the detected object.

[0084] In practice, a weighted aggregation method can be used to obtain the cumulative theft anomaly index by aggregating the current and historical anomaly scores of the detected object. Specifically, a first weight can be assigned to the current time period, and a second weight can be assigned to each historical time period. Based on the first and second weights, the current anomaly score and the historical anomaly scores of each historical time period are weighted and aggregated to obtain the cumulative theft anomaly index. The first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with the time proximity of the historical time period to the current time period.

[0085] In this step, by comparing historical anomaly scores from historical event cycles, we can identify and assess the abnormal behavioral trends of the detected object, thereby enhancing the accuracy of the cumulative theft anomaly index.

[0086] Step S150: Determine the detection result of the cumulative theft behavior of the detected object based on the cumulative theft anomaly index.

[0087] In practice, an appropriate index threshold can be set to determine whether a detected object exhibits suspected cumulative theft behavior. For example, if the index threshold is set to 80, and the cumulative theft anomaly index is greater than the index threshold of 80, then it is determined that the detected object exhibits suspected cumulative theft behavior. Conversely, if the index is less than the threshold, then it is determined that no cumulative theft behavior exists.

[0088] Furthermore, in some embodiments, after determining that a detected object has suspected cumulative data theft based on the cumulative data theft anomaly index, access behavior tracing and evidence collection can be performed on the detected object to display the changing trend of the anomaly index over multiple time periods. For example, a line graph can be used to display the time series of the anomaly index changes of the detected object over multiple time periods. Several significant access behavior features that contribute most to the anomaly index of the detected object are extracted. For example, the ten significant features contributing most to the anomaly index can be found by calculating the product of feature weights and feature values. These significant features are compared and evaluated with corresponding features of other detected objects and displayed using a histogram. The original attributes (such as accessed API, access destination IP, region, etc.) of these significant features are traced and evidence collected by querying the traffic log audit table, recording the access behavior data of the detected object. This embodiment performs risk evidence collection and data tracing on the identified suspected cumulative data theft objects, visually displaying the abnormal access behavior of risk objects and providing a reference for further in-depth analysis of whether risk objects exhibit abnormal behavior.

[0089] In the aforementioned method for detecting cumulative data theft in network traffic, features are extracted from the access data of the target object within the current time period according to a preset time granularity, resulting in feature values ​​of the target object under multiple access behavior features. Based on the feature values ​​under multiple access behavior features, the current anomaly score of the target object is determined. Based on the current anomaly score and historical anomaly scores of the target object, a cumulative data theft anomaly index is obtained. Based on the cumulative data theft anomaly index, the detection result of the target object's cumulative data theft behavior is determined. This method starts by analyzing the network access behavior of the target object and detects potential cumulative data theft behavior through feature extraction and multi-feature anomaly detection. It can effectively monitor suspected cumulative data theft behavior that may exist in network traffic data, solving the technical problems of most current traffic detection methods that only focus on highly sensitive data, have short detection time periods, and single detection behavior features, lacking effective detection methods for suspected cumulative data theft behavior that may exist in network traffic data. Furthermore, by combining access behavior features with time granularity to generate multiple detection feature selection methods closely related to cumulative data theft, the cumulative effect of time and the diversity of data theft behavior are considered, allowing for a more comprehensive evaluation of the target object's access behavior and improving the accuracy of identifying the target object's data theft behavior.

[0090] In an exemplary embodiment, there are multiple objects to be detected. Step S130 above determines the current anomaly score of the detected objects based on the feature values ​​of multiple access behavior characteristics, including:

[0091] Step S131: Obtain the target weight of each access behavior feature within the current time period;

[0092] Step S132: For each detected object, the feature values ​​of each access behavior feature of the detected object are weighted and aggregated according to the target weight to obtain the current abnormal score of the detected object.

[0093] In practice, the larger the feature value of the access behavior feature, the higher the degree of abnormality. A linear regression model can be used to determine the current abnormality score of each detected object within the current time period. That is, by assigning weights to each access behavior feature, for each detected object, the feature values ​​of each access behavior feature are weighted and aggregated using the target weight of each access behavior feature within the current time period to obtain the current abnormality score of the detected object. This can be expressed by the formula:

[0094]

[0095] in, This represents the current anomaly score for any detected object. Indicates access behavior characteristics The target weight, Indicates access behavior characteristics eigenvalues, This represents the total number of access behavior features. For index, Set to 1.

[0096] Furthermore, in some embodiments, to intuitively reflect the degree of abnormality of the detected object, the current abnormality score can be transformed into a value SC between 0 and 100. The transformation process can be represented by the following formula:

[0097] .

[0098] In this embodiment, by assigning different weights to each access behavior feature, and performing weighted aggregation on the feature values ​​of each access behavior feature of the detected object according to the target weight, the current anomaly score of the detected object is obtained. This makes the obtained anomaly score more accurately reflect the real anomaly situation, thereby improving the accuracy and effectiveness of detection.

[0099] In an exemplary embodiment, step S131 above, obtaining the target weight of each access behavior feature in the current time period, includes: obtaining the initial weight of each access behavior feature; for each access behavior feature, updating the initial weight of the access behavior feature according to the distribution information of the feature values ​​of each detection object on the access behavior feature in the current time period, to obtain the target weight.

[0100] In the specific implementation, the target weights assigned to each access behavior feature can be obtained iteratively based on the initial weights. First, the importance of each access behavior feature in the cumulative data theft is analyzed based on experience and historical data, thereby assigning an initial weight to each access behavior feature. And make the total weight sum equal to 1, that is Then, for each access behavior feature, the weight of that access behavior feature is updated based on the distribution information of the feature values ​​of each detected object on that access behavior feature within the current time period. Finally, the updated weights are multiplied by the same coefficient so that the sum of the weights remains unchanged.

[0101] Furthermore, in an exemplary embodiment, updating the initial weight of the access behavior feature based on the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period to obtain the target weight includes: increasing the initial weight of the access behavior feature when the distribution information determines that the distribution difference of each detected object on the access behavior feature is greater than a first threshold to obtain the target weight; and decreasing the initial weight of the access behavior feature when the distribution information determines that the distribution difference of each detected object on the access behavior feature is less than a second threshold to obtain the target weight.

[0102] In specific implementation, based on the distribution information of feature values ​​of each detected object in access behavior features within the current time period, the principle for updating the initial weights of access behavior features is as follows: increase the weights of access behavior features with large distribution differences, and decrease the weights of access behavior features with small distribution differences. That is, if the distribution difference of each detected object in access behavior features is determined to be greater than a first threshold, the initial weights of the access behavior features are increased; if the distribution difference of each detected object in access behavior features is determined to be less than a second threshold, the initial weights of the access behavior features are decreased.

[0103] Specifically, the weight update process for each access behavior feature can be expressed by the formula:

[0104]

[0105]

[0106] in, Indicates the first Among the access behavior characteristics, the first one is the Large eigenvalues, Indicates the first Initial weights for each access behavior feature, This represents the target weight of each access behavior feature.

[0107] In the above embodiments, by increasing the weight of access behavior features with large distribution differences and decreasing the weight of access behavior features with small distribution differences, the initial weight of each access behavior feature is updated based on the distribution information of the feature values ​​of each detected object on each access behavior feature within the current time period. This method of updating and iterating the weight of access behavior features using a decay factor can improve the accuracy of subsequent anomaly detection results.

[0108] In one exemplary embodiment, such as Figure 2As shown, in step S132 above, the feature values ​​of each access behavior feature of the detected object are weighted and aggregated according to the target weight to obtain the current anomaly score of the detected object, including:

[0109] Step S210: Determine the first type of feature and the second type of feature among the various access behavior features; the first type of feature is the feature whose magnitude difference in feature value under this type of feature is greater than the difference threshold, and the second type of feature is the feature whose magnitude difference in feature value under this type of feature is less than the difference threshold.

[0110] Step S220: Perform feature transformation processing on the feature values ​​of the first type of feature and the feature values ​​of the second type of feature respectively to obtain the first transformed feature value and the second transformed feature value;

[0111] Step S230: Based on the first transformation feature value and the second transformation feature value, obtain the processed feature value of each access behavior feature;

[0112] Step S240: Based on the target weight, perform weighted aggregation on the processed feature values ​​of each access behavior feature to obtain the current anomaly score of the detected object.

[0113] In the specific implementation, considering the differences among different detection objects on the same access behavior feature, feature processing is required before weighted aggregation of the feature values ​​of each access behavior feature of the detection object. Specifically, features whose magnitude difference in feature values ​​of different detection objects under this type of feature is greater than a difference threshold are first-class features, and features whose magnitude difference in feature values ​​of different detection objects under this type of feature is less than a difference threshold are second-class features. For the first-class and second-class features, different feature transformation methods are used to perform feature transformation processing to obtain first-transformed feature values ​​and second-transformed feature values. The first-transformed feature values ​​and second-transformed feature values ​​are used as the processed feature values ​​of each access behavior feature. Finally, all the obtained processed feature values ​​are normalized and scaled to the [0, 1] interval for subsequent analysis. According to the target weight, the normalized feature values ​​are weighted and aggregated to obtain the current anomaly score of the detection object.

[0114] Furthermore, in an exemplary embodiment, performing feature transformation processing on the feature values ​​of the first type of features to obtain first transformed feature values ​​includes: determining the maximum feature value of each detection object under the first type of features, and performing feature transformation processing on the feature values ​​of each detection object under the first type of features based on the maximum feature value to obtain the first transformed feature value of each detection object under the first type of features.

[0115] In practical implementation, considering the order-of-magnitude differences in the feature values ​​of different detection objects in the first type of feature, feature processing is required to compress these differences. Specifically, the first type of feature may include multiple access behavior features. For any access behavior feature in the first type of feature, the maximum feature value of each detection object under that access behavior feature can be determined. Based on all feature values ​​of each detection object under that access behavior feature and the maximum feature value, the first transformed feature value of each detection object under that access behavior feature is determined. The specific transformation process can be expressed by the formula:

[0116]

[0117] in, and Each represents a different access behavior feature of a detected object in the first type of features (e.g., ...). The first transformed eigenvalue and the eigenvalue before transformation. This indicates the characteristics of each detected object in this access behavior. The largest eigenvalue.

[0118] In some embodiments, feature transformation processing is performed on the feature values ​​of the second type of features to obtain second transformed feature values, including: determining the average feature value of each detected object under the second type of features, and performing feature transformation processing on the feature values ​​of each detected object under the second type of features based on the average feature value to obtain the second transformed feature value of each detected object under the second type of features.

[0119] In specific implementations, the second type of feature may also include multiple access behavior features. For any one of the access behavior features in the second type of feature... For each detected object, this access behavior characteristic The eigenvalues ​​are averaged to obtain the average eigenvalue. For any detected object c, the eigenvalue of that detected object c is used in the context of the access behavior characteristics. The difference between the feature values ​​and the average feature values ​​is taken, and the absolute value of the difference is used as the feature value of the detection object c in the access behavior characteristics. The second transformation eigenvalue. It can be expressed by the formula: ,in, It is the average characteristic value.

[0120] It is understood that in some embodiments, in addition to being classified into the first type of feature and the second type of feature, each access behavior feature may also be classified into other features and transformed using other feature transformation methods (such as regularization). This application does not make specific limitations on this.

[0121] In the above embodiments, the obtained access features are processed by feature engineering to normalize the access behavior features, highlight the differences in behavior features of the subjects, and make the behavior features better reflect the degree of abnormality of the access behavior; and through feature engineering of access behavior features, the first type of features are compressed by an order of magnitude, and the second type of features are subtracted from the average feature value, so that the obtained model detection results are more interpretable.

[0122] In one exemplary embodiment, the historical anomaly score includes anomaly scores for multiple historical time periods; such as Figure 3 As shown, step S140 above obtains the cumulative theft anomaly index based on the current anomaly score and historical anomaly score of the detected object, including:

[0123] Step S310: Obtain the first weight of the current time period and the second weight of each historical time period;

[0124] Step S320: Based on the first weight and the second weight, perform weighted aggregation on the current abnormal score and the historical abnormal scores of each historical time period to obtain the cumulative theft abnormal index.

[0125] In practice, the first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with its distance from the current time period. That is, the closer to the current time, the greater the weight; the farther away from the current time, the smaller the weight. For example, let the current time period be... The previous 5 time periods were respectively The weights for each time period can then be set as follows: The process of weighting and aggregating the current anomaly score and the historical anomaly scores of each historical time period based on the first weight of the current time period and the second weight of each historical time period to obtain the cumulative theft anomaly index can be expressed as follows:

[0126]

[0127] in, It's object detection going forward. Abnormal scores for each time period This represents the anomaly score for the current time period. If the detected object is not within the model's detection range during a certain time period, the anomaly score is 0.

[0128] In this embodiment, when determining the cumulative theft anomaly index for the current time period, in addition to using the anomaly score for the current time period, the anomaly scores from previous periods are also combined for calculation. This method of calculating the cumulative theft anomaly index of the detected object in the current time period by combining the anomaly scores of the current time period and historical time periods is equivalent to increasing the detection time range. Furthermore, it takes into account the decay effect of time, determining the weights according to the fact that the closer to the current time, the greater the weight, and the farther away from the current time, the smaller the weight. This can further improve the accuracy of the determined cumulative theft anomaly index and improve the reliability of subsequent detection results.

[0129] The method proposed in this application can be applied to a data flow risk monitoring system to detect suspected cumulative data theft by source IPs in network traffic data. Data flow detection refers to the real-time monitoring and analysis of data flows within a network, collecting information about how data flows through the network. Its purpose is to obtain the propagation paths and trends of data at various levels during the process, and to detect potential data leaks, data forgery, and other abnormal behaviors.

[0130] In one embodiment, to facilitate understanding of the embodiments of this application by those skilled in the art, the following will be described in conjunction with the appendix. Figure 4 This application will be further described below. This embodiment includes the following steps:

[0131] Step S401: Obtain access data of multiple candidate detection objects within the current time period;

[0132] Step S402: Based on the access data of each candidate detection object, count the number of access records for each candidate detection object;

[0133] Step S403: Select multiple detection objects from multiple candidate detection objects whose number of access records is greater than the threshold;

[0134] Step S404: For each detected object, feature extraction is performed on the access data according to a preset time granularity to obtain feature values ​​of multiple access behavior features of the detected object.

[0135] Step S405: For each access behavior feature, update the initial weight of the access behavior feature based on the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period to obtain the target weight.

[0136] Step S406: Determine the first type of feature and the second type of feature among each access behavior feature; the first type of feature is the feature whose magnitude difference in feature value under this type of feature is greater than the difference threshold, and the second type of feature is the feature whose magnitude difference in feature value under this type of feature is less than the difference threshold.

[0137] Step S407: Perform feature transformation processing on the feature values ​​of the first type of feature and the feature values ​​of the second type of feature respectively to obtain the first transformed feature value and the second transformed feature value;

[0138] Step S408: Based on the first transformation feature value and the second transformation feature value, obtain the processed feature value of each access behavior feature;

[0139] Step S409: Based on the target weight, perform weighted aggregation on the processed feature values ​​of each access behavior feature to obtain the current anomaly score of the detected object;

[0140] Step S410: Based on the first weight of the current time period and the second weight of each historical time period, perform weighted aggregation on the current abnormal score and the historical abnormal scores of each historical time period to obtain the cumulative theft abnormal index; the first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with the time distance of the historical time period relative to the current time period.

[0141] Step S411: Conduct source tracing and evidence collection on the access behavior of the detected object, and display the changing trend of the abnormal index of the detected object over multiple time periods.

[0142] The method proposed in this application addresses the problem that most current traffic detection methods focus only on highly sensitive data, have short detection cycles, and detect only single behavioral characteristics, lacking effective methods for detecting potential cumulative data theft in network traffic data. Starting from analyzing the network access behavior of the actors, it uses feature engineering and multi-feature anomaly detection methods to detect potential cumulative data theft, considering the cumulative effect of time and the diversity of data theft behaviors. This provides a more comprehensive assessment of the actors' access behavior and improves the accuracy of the model in identifying data theft. For risk objects identified by the model as potentially exhibiting cumulative data theft, risk forensics and data tracing are performed. This not only visually demonstrates the abnormal access behavior of the risk objects but also provides a reference for further in-depth analysis of whether abnormal behavior exists.

[0143] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0144] Based on the same inventive concept, this application also provides a network traffic cumulative theft detection device for implementing the aforementioned network traffic cumulative theft detection method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more network traffic cumulative theft detection device embodiments provided below can be found in the limitations of the network traffic cumulative theft detection method described above, and will not be repeated here.

[0145] In one embodiment, such as Figure 5 As shown, a network traffic cumulative theft detection device is provided, comprising:

[0146] Data acquisition module 510 is used to acquire access data of the detected object within the current time period;

[0147] The feature extraction module 520 is used to extract features from the access data according to a preset time granularity to obtain feature values ​​of multiple access behavior features of the detected object;

[0148] The score determination module 530 is used to determine the current anomaly score of the detected object based on the feature values ​​of multiple access behavior characteristics;

[0149] The score fusion module 540 is used to obtain the cumulative theft anomaly index based on the current and historical anomaly scores of the detected object;

[0150] The anomaly detection module 550 is used to determine the detection result of the cumulative theft behavior of the detected object based on the cumulative theft anomaly index.

[0151] In one embodiment, there are multiple detection objects. The score determination module 530 is also used to obtain the target weight of each access behavior feature in the current time period. For each detection object, the feature values ​​of each access behavior feature of the detection object are weighted and aggregated according to the target weight to obtain the current abnormal score of the detection object.

[0152] In one embodiment, the score determination module 530 is further configured to obtain the initial weights of each access behavior feature; for each access behavior feature, the initial weights of the access behavior feature are updated according to the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period to obtain the target weight.

[0153] In one embodiment, the score determination module 530 is further configured to increase the initial weight of the access behavior feature to obtain a target weight when the distribution difference of each detected object on the access behavior feature is greater than a first threshold, based on the distribution information; and to decrease the initial weight of the access behavior feature to obtain a target weight when the distribution difference of each detected object on the access behavior feature is less than a second threshold, based on the distribution information.

[0154] In one embodiment, the score determination module 530 is further configured to determine a first type of feature and a second type of feature among various access behavior features; the first type of feature is a feature whose magnitude difference in feature values ​​of different detection objects under this type of feature is greater than a difference threshold, and the second type of feature is a feature whose magnitude difference in feature values ​​of different detection objects under this type of feature is less than the difference threshold; feature transformation processing is performed on the feature values ​​of the first type of feature and the feature values ​​of the second type of feature respectively to obtain a first transformed feature value and a second transformed feature value; based on the first transformed feature value and the second transformed feature value, the processed feature value of each access behavior feature is obtained; according to the target weight, the processed feature value of each access behavior feature is weighted and aggregated to obtain the current abnormal score of the detection object.

[0155] In one embodiment, the score determination module 530 is further configured to determine the maximum feature value of each detection object under the first type of feature, perform feature transformation processing on the feature value of each detection object under the first type of feature based on the maximum feature value, and obtain a first transformed feature value of each detection object under the first type of feature; determine the average feature value of each detection object under the second type of feature, and perform feature transformation processing on the feature value of each detection object under the second type of feature based on the average feature value, and obtain a second transformed feature value of each detection object under the second type of feature.

[0156] In one embodiment, the historical anomaly score includes anomaly scores from multiple historical time periods; the score fusion module 540 is further configured to obtain a first weight for the current time period and a second weight for each historical time period; the first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with the time proximity of the historical time period to the current time period; based on the first weight and the second weight, the current anomaly score and the historical anomaly scores of each historical time period are weighted and aggregated to obtain the cumulative theft anomaly index.

[0157] Each module in the aforementioned network traffic accumulation theft detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0158] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data used in the cumulative theft detection process of network traffic. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for cumulative theft detection of network traffic.

[0159] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0161] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0164] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0165] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting cumulative metering fraud of network traffic, characterized in that, The method includes: Acquire access data of the detected object within the current time period; there are multiple detected objects. According to a preset time granularity, feature extraction is performed on the access data to obtain feature values ​​of multiple access behavior features of the detected object; Identify a first type of feature and a second type of feature among various access behavior features; the first type of feature is a feature whose magnitude difference in feature values ​​of different detection objects under this type of feature is greater than a difference threshold, and the second type of feature is a feature whose magnitude difference in feature values ​​of different detection objects under this type of feature is less than the difference threshold; Based on the maximum feature value of each detected object under the first type of feature, feature transformation processing is performed on the feature value of the detected object under the first type of feature to obtain a first transformed feature value; based on the average feature value of each detected object under the first type of feature, feature transformation processing is performed on the feature value of the detected object under the second type of feature to obtain a second transformed feature value; Based on the first transformation feature value and the second transformation feature value, the processed feature values ​​of the plurality of access behavior features are obtained, and the current anomaly score of the detected object is determined according to the processed feature values ​​of the plurality of access behavior features. Based on the current and historical abnormal scores of the detected object, the cumulative theft anomaly index is obtained; Based on the cumulative theft anomaly index, the cumulative theft behavior detection result of the detected object is determined.

2. The method according to claim 1, characterized in that, Determining the current anomaly score of the detected object based on the processed feature values ​​of the multiple access behavior features includes: Obtain the target weight of each of the aforementioned access behavior features within the current time period; For each detected object, the processed feature values ​​of each access behavior feature of the detected object are weighted and aggregated according to the target weight to obtain the current anomaly score of the detected object.

3. The method according to claim 2, characterized in that, The step of obtaining the target weight of each of the access behavior features within the current time period includes: Obtain the initial weights of each of the aforementioned access behavior features; For each access behavior feature, the initial weight of the access behavior feature is updated based on the distribution information of the feature values ​​of each detected object on the access behavior feature within the current time period, to obtain the target weight.

4. The method according to claim 3, characterized in that, The step of updating the initial weights of the access behavior features based on the distribution information of the feature values ​​of each detected object on the access behavior features within the current time period to obtain the target weights includes: If, based on the distribution information, the distribution difference of each detected object on the access behavior feature is greater than a first threshold, the initial weight of the access behavior feature is increased to obtain the target weight. If, based on the distribution information, the distribution difference of each detected object on the access behavior feature is less than a second threshold, the initial weight of the access behavior feature is reduced to obtain the target weight.

5. The method according to claim 1, characterized in that, The historical anomaly score includes anomaly scores from multiple historical time periods; the process of obtaining the cumulative theft anomaly index based on the current anomaly score and historical anomaly score of the detected object includes: Obtain the first weight of the current time period and the second weight of each of the historical time periods; the first weight is greater than the second weight, and the weight of the historical time period is negatively correlated with the time distance of the historical time period relative to the current time period; Based on the first weight and the second weight, the current abnormal score and the historical abnormal scores of each historical time period are weighted and aggregated to obtain the cumulative theft abnormality index.

6. The method according to claim 1, characterized in that, The detection target is determined in the following way: Acquire access data for multiple candidate detection objects within the current time period; Based on the access data of each candidate detection object, count the number of access records for each candidate detection object; From the multiple candidate detection objects, candidate detection objects with more than a threshold number of access records are selected as detection objects.

7. The method according to claim 1, characterized in that, After determining the cumulative theft behavior detection result of the detected object based on the cumulative theft anomaly index, the method further includes: The cumulative theft anomaly index of the detected object over multiple time periods is displayed using a line graph.

8. A device for detecting the cumulative theft of network traffic, characterized in that, The device includes: The data acquisition module is used to acquire access data of the detected objects within the current time period; there are multiple detected objects. The feature extraction module is used to extract features from the access data according to a preset time granularity to obtain feature values ​​of multiple access behavior features of the detected object; A score determination module is used to determine a first type of feature and a second type of feature among various access behavior features. The first type of feature is a feature where the order of magnitude difference of the feature values ​​of different detection objects under this type of feature is greater than a difference threshold, and the second type of feature is a feature where the order of magnitude difference of the feature values ​​of different detection objects under this type of feature is less than the difference threshold. Based on the maximum feature value of each detection object under the first type of feature, feature transformation processing is performed on the feature value of the detection object under the first type of feature to obtain a first transformed feature value. Based on the average feature value of each detection object under the first type of feature, feature transformation processing is performed on the feature value of the detection object under the second type of feature to obtain a second transformed feature value. Based on the first transformed feature value and the second transformed feature value, the processed feature value of the multiple access behavior features is obtained, and the current anomaly score of the detection object is determined according to the processed feature value of the multiple access behavior features. The score fusion module is used to obtain the cumulative theft anomaly index based on the current abnormal score and historical abnormal score of the detected object; An anomaly detection module is used to determine the cumulative theft behavior detection result of the detected object based on the cumulative theft anomaly index.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the cumulative theft detection method for network traffic as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cumulative theft detection method for network traffic as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method for detecting the cumulative theft of network traffic as described in any one of claims 1 to 7.

Images