App traffic anomaly detection method and system based on multi-scale time series

Abstract

The application relates to an APP traffic anomaly detection method and system based on a multi-scale time sequence and belongs to the field of mobile internet traffic monitoring.The system comprises a traffic data acquisition module, a traffic data preprocessing module, a multi-scale time sequence decomposition module and an abnormal traffic detection module.The network traffic of a target APP is packet captured through the traffic data acquisition module, classified according to API calling, resource loading and advertisement request types, and the APP running state is recorded as context information; the traffic data acquisition module groups the collected traffic data according to scenes and extracts statistical features based on a set time window; the multi-scale time sequence decomposition module performs wavelet transformation and empirical mode decomposition on the feature vector sequence, extracts high-frequency components and low-frequency components of the traffic respectively, and captures short-term bursts and long-term trend changes.The application designs an abnormal detection method according to the APP traffic characteristics, and can more accurately detect APP traffic anomalies.

Description

Technical Field

[0001] This invention belongs to the field of mobile internet traffic monitoring, and relates to a method and system for detecting abnormal APP traffic based on multi-scale time series. Background Technology

[0002] With the rapid development of network technology, network systems are becoming increasingly complex, and the types of network devices are also increasing, leading to frequent network traffic anomalies. Network traffic anomalies refer to network traffic behavior deviating from normal patterns, which can cause significant harm to the network and the computers on it. Therefore, monitoring network traffic behavior and promptly detecting anomalies is crucial for improving network reliability and availability.

[0003] Traditional methods for detecting network traffic anomalies primarily analyze overall network traffic in PC or server environments, rarely considering the unique characteristics of application (APP) traffic on mobile devices. However, APP traffic often exhibits characteristics distinct from network traffic, such as context-driven API call frequencies, request patterns across different time scales, and various types of resource loading. Traditional methods struggle to fully capture these unique features, resulting in suboptimal detection performance.

[0004] App traffic exhibits unique behavioral patterns, such as periodic server access and specific API call frequencies. These characteristics may be more pronounced in multi-scale time series analysis. Compared to network traffic detection, app traffic detection can more sensitively capture abnormal traffic at the application layer, such as sudden bursts of requests within a short period or abuse of specific APIs. These are often potential indicators of fraud or malicious activity.

[0005] Several invention patents have been developed to address the limitations of existing network traffic anomaly detection technologies in effectively capturing short-term and long-term anomaly patterns and distinguishing between sudden, persistent, and trend-based anomalies. Patent CN106411597A discloses a network traffic anomaly detection method and system. This method uses extracted time series data, composed of traffic data samples, as samples for model training and classification. It considers the temporal continuity and correlation of network traffic changes, introducing temporal information into the detection and classification of anomaly traffic. However, this patent still has the issue of further optimizing the neural network model structure to improve its generalization ability and interpretability. Patent CN116232761A discloses a shapelet-based network anomaly traffic detection method and system. This method uses shapelet time series data processing technology to obtain shapelet sequences for network traffic anomaly detection, which can improve the detection rate and reduce the false positive and false negative rates. However, this patent still has the issue of further optimizing the shapelet sequence generation method to improve the interpretability and reproducibility of the shapelet sequences.

[0006] Existing technologies have the following drawbacks: they cannot effectively capture short-term and long-term abnormal patterns in network traffic, nor can they comprehensively detect abnormal situations; they cannot distinguish between sudden, persistent, and trend-based anomalies, nor can they accurately identify anomaly types; their detection accuracy is low, and they cannot fully utilize historical information to analyze current data; they lack specialized detection methods for APP traffic, and cannot perform anomaly detection based on the characteristics of APP traffic; there is room for optimization in model structure and parameter selection, and their generalization ability and interpretability need to be improved. Summary of the Invention

[0007] Given that existing technologies suffer from limitations such as the inability to comprehensively detect anomalies in app traffic, the inability to simultaneously capture short-term and long-term anomaly patterns, and the inability to distinguish between sudden, persistent, and trend-based anomalies, this invention provides a method and system for detecting app traffic anomalies based on multi-scale time series analysis to address these issues.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for detecting anomalies in app traffic based on multi-scale time series analysis includes the following steps:

[0010] Step 1, Traffic Data Collection: Capture network traffic of the target application (APP) and categorize it according to API calls, resource loading, and advertising request types;

[0011] Step 2, Traffic Data Preprocessing: Identify the APP running scenario based on the request content and Uniform Resource Locator (URL), group the traffic data by scenario, divide the time window based on scenario characteristics, and calculate the (Application Programming Interface, API) call frequency, request size distribution, response time, request type distribution, and request type switching frequency within each time window;

[0012] Step 3, Multi-scale time series decomposition: Combine the statistical features of each time window into a feature vector sequence, decompose it into high-frequency components and low-frequency components through wavelet transform, and perform empirical mode decomposition on the low-frequency components to obtain multiple intrinsic mode functions;

[0013] Step 4: Calculate the anomaly score for high-frequency components using the Isolation Forest algorithm, and calculate the anomaly score for the intrinsic mode functions of low-frequency components using a Long Short-Term Memory network. Use the high and low frequency anomaly scores to determine the traffic risk level and anomaly type.

[0014] Furthermore, the identification of the APP running scenario in step 2 is as follows: matching preset scenario tags based on keywords or domain names in the request URL, wherein the scenario tags are registration, login, payment, advertising, or resource loading;

[0015] The time window division rule in step 2 is as follows: a smaller time window is used for high-density scenarios and a longer time window is used for low-density scenarios. The high-density scenarios include resource loading or advertising requests, and the low-density scenarios include payment or login.

[0016] In step 3, the db4 wavelet basis is used to perform discrete wavelet transform on the feature vector sequence, decomposing it into high-frequency components and low-frequency components.

[0017] The method of using high-frequency and low-frequency anomaly scores in step 4 is as follows: if both the high-frequency anomaly score and the low-frequency anomaly score exceed a preset threshold, it is determined to be a persistent anomaly; if only the high-frequency score exceeds the threshold, it is determined to be a sudden anomaly; if only the low-frequency score exceeds the threshold, it is determined to be a trend anomaly.

[0018] An app traffic anomaly detection system based on multi-scale time series analysis includes:

[0019] The traffic data collection module captures network traffic from the app, categorizes it by request type, and performs packet capture.

[0020] The traffic data preprocessing module identifies the APP's running scenarios and divides them into time windows, extracting the traffic statistics features of each window;

[0021] The multi-scale time series decomposition module decomposes the feature vector sequence into high-frequency and low-frequency components through wavelet transform and empirical mode decomposition.

[0022] The abnormal traffic detection module includes a high-frequency component anomaly detection module, a low-frequency component anomaly detection module, and an anomaly judgment module. It uses the isolated forest algorithm and the Long Short-Term Memory (LSTM) network model to calculate the anomaly score, and the anomaly judgment module outputs the anomaly type and risk level.

[0023] Furthermore, the traffic data preprocessing module further includes a scene recognition unit, which associates preset scene tags by matching URL keywords or domain names;

[0024] In the multi-scale time series decomposition module, the decomposition of low-frequency components uses empirical mode decomposition to generate multiple intrinsic mode functions;

[0025] The isolated forest algorithm of the high-frequency component anomaly detection module determines the anomaly score by calculating the path length of data points; the shorter the path length, the higher the anomaly score.

[0026] The LSTM model of the low-frequency component anomaly detection module calculates the anomaly score by the residual between the predicted value and the actual value. The larger the residual, the higher the anomaly score.

[0027] Furthermore, the traffic data preprocessing module divides the traffic data into different APP usage scenarios, assuming each scenario has a traffic volume of C. k Let k represent the scene number. The resulting traffic dataset D, containing n different scene identifications, is represented by the following formula:

[0028] D = {C1,C2,…,C} n}

[0029] Where C1 represents traffic data from the registration scenario; C2 represents traffic data from the login scenario; C n This indicates other identifiable scene traffic data;

[0030] For each scenario C k The characteristics of the scene are used to divide the time window in order to capture the traffic characteristics within the scene. Let scene C be... k The time window length is Δt k Then, scenario C k Divide into m time windows {W k,1 W k,2 ,...,W k,m}:

[0031]

[0032] Among them, W k，j Representing scenario C k The j-th time window, the length of the time window Δt k It can be adjusted according to the needs of the scenario;

[0033] Calculate W for each time window k，j Various traffic statistics characteristics.

[0034] Furthermore, the calculation of each time window W k，j The specific characteristics of the various traffic statistics are as follows:

[0035] Calculate the API call frequency f during this time period. API Request size distribution μ s and σ s 2 Response time μ r and σ r 2 Request type distribution: API request P(T1), resource loading request P(T2), advertising request P(T3), other requests P(T4), request type switching frequency f 切换 ;

[0036] Scene C k W k，j The various flow characteristics within the time window are combined to form a feature vector x. k，j This can be expressed as the following formula:

[0037]

[0038] Scene C k The feature vectors of all time windows are combined into a feature vector sequence X. k (t):

[0039] X k (t)=[x1,x2,...,x m ]

[0040] The feature sequences from multiple time windows across all scenarios are combined into a single time series X(t):

[0041] X(t) = [X1(t), X2(t), ..., X n [(t)]=[x1,x2,...,x T ]

[0042] Where T is the total number of time windows and n represents the number of scene categories. This time series provides input for the subsequent multi-scale time series decomposition module.

[0043] Furthermore, the multi-scale time series decomposition module divides the time window W k，j The various flow characteristics are combined into a feature vector; the feature vectors of all time windows are combined into a feature vector sequence; the feature vectors of each time window are arranged sequentially to form a feature vector sequence X; wavelet decomposition is performed on the above feature vector sequence X to obtain the high-frequency and low-frequency components of the flow data; discrete wavelet transform is performed on the flow feature sequence X(t), and first-level decomposition is performed using the db4 wavelet basis to obtain the low-frequency component L(t) and the high-frequency component H(t). The calculation formula of the wavelet transform is as follows:

[0044]

[0045] Where N represents the number of sample points in the flow characteristic sequence; n represents the discrete-time index, i.e., the sample point position of the input flow characteristic sequence X(n); X(n) represents the input flow characteristic sequence; g(tn) represents the high-pass filter coefficient in the db4 wavelet basis, h(tn) represents the low-pass filter coefficient in the wavelet transform; and t is the time index of the output signal.

[0046] Furthermore, the high-frequency component anomaly detection module determines the anomaly S. H The calculation is performed using the following formula:

[0047]

[0048] Among them, AnomalyScore(H i ) is the Isolation Forest algorithm for data points H i The abnormal score, where n is the number of data points.

[0049] Furthermore, the low-frequency component anomaly detection module uses empirical mode decomposition to decompose the low-frequency component L(t) into multiple intrinsic mode functions (IMFs) to capture subtle trends and patterns;

[0050] Let the input of the model be IMF. k (t), the output is the outlier score of this IMF. Abnormal score S L It is obtained by weighting the outlier scores of each IMF, as shown in the following formula:

[0051]

[0052] Where, α k Weights for each IMF, represents the outlier score for the IMF, k is the number of IMFs, and K is the total number of IMFs.

[0053] Furthermore, based on the anomaly detection results of high-frequency and low-frequency components, the anomaly judgment module further evaluates the traffic risk for each time window; for each time window t, it performs a comprehensive judgment process on the anomaly scores of high-frequency and low-frequency components, and obtains the high-frequency anomaly threshold T by training the model through centralized data. H With low-frequency anomaly threshold T L Based on the degree to which the score exceeds the threshold, different risk levels are defined; high-frequency and low-frequency anomaly types are integrated to generate traffic anomalies, persistent anomaly patterns, and long-term trend anomaly types.

[0054] The beneficial effects of this invention are as follows:

[0055] It can simultaneously capture short-term and long-term abnormal patterns in APP traffic, achieving comprehensive detection of anomalies; it can distinguish between sudden, persistent, and trend-based anomalies, accurately identifying anomaly types and providing knowledge support for network security situation assessment and immune decision-making; it fully utilizes historical information to analyze current data, improving the accuracy of anomaly detection; it has designed anomaly detection methods specifically for the characteristics of APP traffic, enabling more precise detection of APP traffic anomalies; and it employs advanced time series analysis methods such as wavelet transform and empirical mode decomposition, improving the model's generalization ability and interpretability.

[0056] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0057] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein...

[0058] Figure 1 This is a flowchart of an app traffic anomaly detection method based on multi-scale time series according to an embodiment of the present invention;

[0059] Figure 2 This is an architecture diagram of an APP traffic anomaly detection system based on multi-scale time series according to an embodiment of the present invention;

[0060] Figure 3 This is a data processing flowchart of an APP traffic anomaly detection system based on multi-scale time series according to an embodiment of the present invention. Detailed Implementation

[0061] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0062] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0063] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0064] Please see Figure 1 This is a flowchart of the APP traffic anomaly detection method based on multi-scale time series according to an embodiment of the present invention;

[0065] The method includes the following steps:

[0066] Step 1: The traffic data collection module captures the APP's traffic packets and categorizes the traffic data according to different types such as API calls, resource loading, and advertising requests.

[0067] Step 2: The traffic data preprocessing module uses the information recorded in the data acquisition module to identify the APP scenario to which each request belongs based on relevant information in the request (such as specific characters, request content, etc.). The data is grouped according to different scenarios. Different application scenarios may have different traffic behaviors and timing characteristics. Therefore, the time window is adjusted according to the scenario category to capture these characteristics more accurately. Then, the requests to each scenario are segmented according to the defined time window. Smaller time windows are used for higher density scenarios to capture rapidly changing behaviors, while larger time windows are used for low density scenarios to reduce data fragmentation. The statistical characteristics of API call frequency, request size distribution, response time, request type distribution, and request type switching frequency are calculated within each time window.

[0068] Step 3: The multi-scale time series decomposition module combines the statistical features calculated by the traffic data preprocessing module into a feature vector sequence. Each time window generates a feature vector sequence, and the feature vector sequences of all time windows are combined into a time series. The traffic characteristics of different application scenarios may be reflected at different time scales. Wavelet transform can clearly separate these characteristics, which is convenient for further analysis. Therefore, wavelet transform is performed on the time series to decompose it into high-frequency components that reflect short-term fluctuations and low-frequency components that reflect global trends. The low-frequency components still contain change patterns at multiple time scales. Therefore, empirical mode decomposition is applied to the low-frequency components for further decomposition to obtain multiple intrinsic mode functions. Each intrinsic mode function represents the change of the sequence at a specific time scale, which is convenient for capturing detailed trends and patterns.

[0069] Step 4: The abnormal traffic detection module is divided into two sub-modules: a high-frequency component anomaly detection module and a low-frequency component anomaly detection module. The high-frequency component anomaly detection module receives high-frequency components from wavelet decomposition and uses an isolated forest algorithm to calculate the "isolation degree" of samples to give an anomaly score for the high-frequency components. The low-frequency component anomaly detection module receives multiple intrinsic mode functions (IMFs) from empirical mode decomposition and uses a long short-term memory (LSTM) network to give an anomaly score for the low-frequency components. Finally, the high-frequency and low-frequency anomaly scores are combined to give a risk level and anomaly type. For each time window, if both the high-frequency and low-frequency anomaly scores exceed a set threshold, the traffic in that window is considered abnormal.

[0070] Please see Figure 2 This is an architecture diagram of an APP traffic anomaly detection system based on multi-scale time series according to an embodiment of the present invention.

[0071] The system includes: a traffic data acquisition module, which captures and classifies APP network traffic according to request type; a traffic data preprocessing module, which identifies APP operating scenarios, divides time windows, and extracts traffic statistical features for each window; a multi-scale time series decomposition module, which decomposes the feature vector sequence into high-frequency and low-frequency components through wavelet transform and empirical mode decomposition; and an abnormal traffic detection module, including a high-frequency component anomaly detection module, a low-frequency component anomaly detection module, and an anomaly judgment module, which calculates anomaly scores using the isolated forest algorithm and the Long Short-Term Memory (LSTM) network model, and outputs the anomaly type and risk level through the comprehensive judgment module. Further, the specific methods for capturing and classifying APP traffic in step 1 include:

[0072] Capture network traffic from the target app, obtain network request data packets, and preprocess them.

[0073] Data is categorized based on different request types (such as API calls, resource loading, ad requests, etc.) for subsequent processing. API requests typically access backend server API endpoints, and their URLs contain specific API paths (e.g., / api / v1 / user). Resource loading requests usually refer to requests for resources such as images, videos, and Cascading Style Sheets (CSS), and their URLs contain static resource paths (e.g., .jpg, .css, .mp4, etc.). Ad requests can be identified by domain name or specific URL patterns (e.g., ad service provider domain name, keywords ads, track, etc.). Requests not belonging to the previous three categories are classified as other requests.

[0074] The processing steps of the traffic data preprocessing module include:

[0075] While capturing traffic data, the system identifies and records the app's operational status information based on the request content and URL. This includes, but is not limited to, scenarios such as registration, login, payment, advertising, and resource loading. For example, login status is usually accompanied by login requests (such as keywords like auth, login, and token), and the requested URL is generally fixed on a specific authentication server. Content loading status involves multiple resource requests, such as URL requests for resources like images, videos, and text. The collected traffic data is marked and stored according to different application scenarios and timestamps to ensure that the time series data in subsequent analysis has accurate time and scenario information.

[0076] In step 2, the scene recognition and grouping methods include:

[0077] Step 2.1: The traffic data preprocessing module receives the raw traffic data and corresponding context information from the traffic data acquisition module. Using these context tags, the traffic data is divided into different app usage scenarios. Let the traffic for each scenario be C. k (Where k represents the scene number), resulting in a traffic dataset D containing n different scene identifications:

[0078] D = {C1,C2,...,C} n}

[0079] Where C1 represents traffic data from the registration scenario; C2 represents traffic data from the login scenario; C n This indicates other identifiable scene traffic data;

[0080] Traffic data within each scenario will be analyzed subsequently based on independent characteristics and patterns;

[0081] Step 2.2, for each scenario C k The traffic data preprocessing module divides the time window according to the characteristics of the scenario. The purpose of the time window design is to group requests within the scenario into appropriate time periods in order to capture the traffic characteristics of the scenario. Let scenario C be an example. k The time window length is Δt k Then, scenario C can be... k Divide into m time windows {W k,1 W k,2 ,…,W k,m}:

[0082]

[0083] Among them, W k，j Representing scenario C k The j-th time window, the length of the time window Δt k The window size can be adjusted according to the needs of the scenario. For example, for loading scenarios, a shorter window can be selected to capture frequent requests, while for payment scenarios, a longer window can be selected to cover the entire payment process.

[0084] Step 2.3, in each time window W k，j In the middle, calculate the statistical characteristics of various traffic flows within this time period:

[0085] W in each time window k，j In the calculation, the API call frequency f within this time period is calculated. API Request size distribution μ s and σ s 2 Response time μ r and σ r 2Request type distribution P(T1), P(T2), P(T3), P(T4) represent API requests, resource loading requests, advertising requests, and other requests, respectively, with request type switching frequency f. 切换 .

[0086] Scene C k W k，j The various flow characteristics within the time window are combined to form a feature vector x. k，j :

[0087]

[0088] Scene C k The feature vectors of all time windows are combined into a feature vector sequence X. k (t):

[0089] X k (t)=[x1,x2,...,x m ]

[0090] Different scenarios correspond to different time windows. This feature vector sequence represents the traffic characteristic changes of that scenario within its specific time window. The feature sequences of multiple time windows in all scenarios are combined into a single time series X(t):

[0091] X(t) = [X1(t), X2(t), ..., X n [(t)]=[x1,x2,...,x T ]

[0092] Where T is the total number of time windows and n represents the number of scene categories. This time series represents the changes in traffic characteristics of captured traffic under different time windows and provides input for the subsequent multi-scale time series decomposition module.

[0093] API call frequency: Calculation time window W k，j The number of API calls in the time window. k，j If n API calls occur, then the API call frequency is:

[0094]

[0095] Request size distribution: The size of each request within a statistical time window. The mean μ is used. s and variance σ s 2 This indicates the distribution characteristics of request sizes:

[0096]

[0097] Among them, s iIndicates the size of the i-th request;

[0098] Response time: Calculate the response time of each request within the time window and calculate its average value μ. r and variance σ r 2

[0099]

[0100] Where, r i This represents the response time for the i-th request;

[0101] Request type distribution: Request type distribution represents the proportion of different request types (such as API requests, resource loading requests, and ad requests) within a certain time window. Assume there are p types of request types (e.g., API requests, resource loading requests, ad requests, etc.) within a time window. For the k-th request type, its proportion P(T) k The formula for calculating ) is:

[0102]

[0103] Where, n k Let P(T1) represent the number of requests of type k, P(T2) represent API requests, P(T3) represent resource loading requests, and P(T3) represent advertising requests. N is the total number of requests within the time window.

[0104] Request type switching frequency: Calculate the number of times the request type switches within a time window. It is defined as the number of switches in different request type sequences. Assuming there are m switches within the window, the switching frequency is:

[0105]

[0106] Furthermore, in step 3, the implementation process of the multi-scale time series decomposition module includes:

[0107] Step 3.1: Set the time window W k，j The various flow characteristics are combined to form a feature vector. Let the feature vector of the j-th window be x. k，j If the above characteristics are included, such as API call frequency, request size distribution, response time, request type distribution, and request type switching frequency, then:

[0108]

[0109] The feature vectors of all time windows are combined into a single feature vector sequence, representing the changes in traffic characteristics of the scenario across different time windows. The feature vectors of each time window are arranged sequentially to form the feature vector sequence X:

[0110] X = [x1, x2, ..., x T ]

[0111] Where T is the total number of time windows, the multi-scale time series decomposition module performs wavelet decomposition on the above feature vector sequence X to obtain the high-frequency and low-frequency components of the traffic data.

[0112] X = L(t) + H(t)

[0113] Where L(t) represents the low-frequency component and H(t) represents the high-frequency component.

[0114] Step 3.2: Perform Discrete Wavelet Transform (DWT) on the flow characteristic sequence X(t), using the db4 (Daubechies 4) wavelet basis for first-order decomposition to obtain the low-frequency component L(t) and the high-frequency component H(t). The calculation formula for the wavelet transform is as follows:

[0115]

[0116] Where N represents the number of sample points in the flow characteristic sequence; n represents the discrete-time index, i.e., the sample point position of the input flow characteristic sequence X(n); X(n) represents the input flow characteristic sequence; g(tn) represents the high-pass filter coefficient in the db4 wavelet basis, h(tn) represents the low-pass filter coefficient in the wavelet transform; and t is the time index of the output signal.

[0117] Step 3.3: Analyze the low-frequency component L(t) extracted from the flow data using Empirical Mode Decomposition (EMD): Extract the local extrema of L(t) and construct upper and lower envelopes; calculate the mean of the upper and lower envelopes and subtract it from L(t) to obtain a new signal h(t); determine whether h(t) satisfies the conditions of the Intrinsic Mode Function (IMF). If it does, extract it as the first IMF; if h(t) does not satisfy the IMF conditions, continue to extract its extrema, calculate the envelope and subtract it, until an IMF that satisfies the conditions is obtained; repeat this process until all IMFs are obtained; finally, the low-frequency component L(t) is decomposed into multiple IMFs, representing the flow changes at different time scales.

[0118] Step 4, the implementation process of the abnormal traffic detection module, includes:

[0119] Step 4.1: Receive the high-frequency component data H(t) after wavelet transform; use the Isolation Forest algorithm for high-frequency component anomaly detection. H(t) is the high-frequency component data after wavelet transform. The Isolation Forest algorithm constructs multiple decision trees and calculates the anomaly score based on the splitting path of each data point in the tree. Anomaly score S H The calculation is performed using the following formula:

[0120]

[0121] Among them, AnomalyScore(H i ) is the Isolation Forest algorithm for data points H i The abnormal score, where n is the number of data points.

[0122] Step 4.2: Receive the low-frequency component L(t) obtained from wavelet decomposition, and further decompose the low-frequency component L(t) using empirical mode decomposition. The low-frequency component L(t) is decomposed into multiple intrinsic mode functions (IMFs), namely IMF1(t), ..., IMF2(t). k (t); represents changes in flow over a specific time scale to capture subtle trends and patterns.

[0123] For each IMF, a Long Short-Term Memory (LSTM) network is used for anomaly detection in the time series data. The model input is the IMF. k (t), the output is the outlier score of this IMF. Abnormal score S L It is obtained by weighting the outlier scores of each IMF, as shown in the following formula:

[0124]

[0125] Where, α k Weights for each IMF, represents the outlier score for the IMF, k is the number of IMFs, and K is the total number of IMFs.

[0126] The processing steps of the high- and low-frequency component anomaly detection modules in the abnormal flow detection module include:

[0127] 1) High-frequency component anomaly detection module: Receives high-frequency component data H(t) after wavelet transform; employs the Isolation Forest algorithm for high-frequency component anomaly detection. Specifically, assuming H(t) is the high-frequency component data after wavelet transform, the Isolation Forest algorithm constructs multiple decision trees. Within a tree, the smaller the number of splits h(t) required for a data point t to be isolated (i.e., split to a single leaf node), the more anomalous t is. The anomaly score formula is calculated as follows:

[0128] First, calculate the average path length h(t) of x across all trees:

[0129]

[0130] Where, n t It is the total number of trees, h i (t) is the path length of point t in the i-th tree;

[0131] Then, the anomaly score S is calculated by normalizing the path length. H (t):

[0132]

[0133] Where c(n) is the normalization factor, representing the expected path length of the tree:

[0134]

[0135] H(n) is the nth harmonic number, S H The value of (t) ranges from [0, 1]. The closer it is to 1, the faster point t is isolated, and the more likely it is to be an anomaly.

[0136] 2) Low-frequency component anomaly detection module: This module receives low-frequency component L(t) obtained from wavelet decomposition. L(t) represents flow changes over a long time scale and reflects the overall trend of the flow data. Anomalies in the low-frequency component manifest as significant deviations, such as gradual accumulation or systematic anomalies. The module then uses EMD to decompose the low-frequency component L(t) to obtain multiple intrinsic mode functions, namely IMF1(t), IMF2(t), ..., IMF... k EMD decomposes the low-frequency component L(t) into IMFs at different time scales. These IMFs do not depend on preset parameters and can adapt to the complexity of traffic data. For each IMF... k (t) uses a Long Short-Term Memory (LSTM) network for anomaly detection in time series. LSTM has the ability to capture long-term dependencies in time series, making it suitable for anomaly detection scenarios. LSTM can learn its temporal dynamic features and give the anomaly score S of the IMF. IMFK :

[0137]

[0138] in, These are the values ​​predicted by LSTM, IMF. k (t) represents the actual value. A higher score indicates more anomalous behavior at that point in time. A global anomalous score S is then derived by combining the anomalous scores from various IMFs. L (t):

[0139]

[0140] Based on the anomaly detection results of high-frequency and low-frequency components, the integrated module further assesses the traffic risk for each time window.

[0141] Step 4.3: For each time window t, calculate the high-frequency component anomaly score S. H and low-frequency component abnormal score S L A comprehensive judgment process is performed. The high-frequency anomaly threshold TH and the low-frequency anomaly threshold TL are obtained by training the model in the dataset. The risk level is divided according to the degree to which the score exceeds the threshold.

[0142] Step 4.4: Integrate the high-frequency and low-frequency anomaly types to generate specific anomaly types: High-frequency anomaly score and low-frequency anomaly score: judged as sudden traffic anomaly, such as short-term distributed denial-of-service (DDoS) attacks or instantaneous traffic spikes; High scores for both high-frequency and low-frequency anomalies: judged as persistent anomaly patterns, such as application hijacking or continuous malicious code activity; Low-frequency anomaly score and high-frequency anomaly score: judged as long-term trend anomalies, such as data leakage or resource abuse.

[0143] Please see Figure 3 This is a data processing flowchart of an APP traffic anomaly detection system based on multi-scale time series data according to an embodiment of the present invention. The system first captures network traffic of the target APP through a traffic data acquisition module, classifying it according to API calls, resource loading, and advertising request types, while recording the APP's running status as context information. Next, a traffic data preprocessing module groups the collected traffic data by scenario and extracts statistical features based on a set time window. Subsequently, a multi-scale time series decomposition module performs wavelet transform and empirical mode decomposition on the feature vector sequence, extracting high-frequency and low-frequency components of the traffic to capture short-term bursts and long-term trend changes. An anomaly traffic detection module uses the isolated forest algorithm to detect anomalies in the high-frequency components and simultaneously uses a Long Short-Term Memory (LSTM) network to detect anomalies in the low-frequency components. Finally, the detection results from both methods are combined for evaluation, and the anomaly type is determined.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting abnormal APP traffic based on multi-scale time series, characterized in that, Includes the following steps: Step 1, Traffic Data Collection: Capture network traffic of the target application (APP) and categorize it according to API calls, resource loading, and advertising request types; Step 2, Traffic Data Preprocessing: Identify the APP running scenario based on the request content and Uniform Resource Locator (URL), group the traffic data by scenario, divide the time window based on scenario characteristics, and calculate various traffic statistics characteristics for each time window. The traffic statistics characteristics include Application Programming Interface (API) call frequency, request size distribution, response time, request type distribution, and request type switching frequency. Step 3, Multi-scale time series decomposition: Combine the traffic statistics features of each time window into a feature vector sequence, decompose it into high-frequency components and low-frequency components through wavelet transform, and perform empirical mode decomposition on the low-frequency components to obtain multiple intrinsic mode functions; Step 4: Use the isolated forest algorithm to calculate the high-frequency anomaly score for the high-frequency components, and use the long short-term memory network to calculate the low-frequency anomaly score for the intrinsic mode functions of the low-frequency components. Use the high and low frequency anomaly scores to determine the traffic risk level and anomaly type. The specific steps of grouping traffic data by scenario and dividing time windows based on scenario characteristics are as follows: Traffic data is divided into different app usage scenarios, with each scenario's traffic set to [value]. C k , k Indicate the scene number, and obtain the content. n Traffic datasets identified in different scenarios D This can be expressed as the following formula: in, C 1 indicates traffic data related to the registration scenario; C 2 indicates traffic data related to the login scenario; C n This indicates other identifiable scene traffic data; For each scenario C k The characteristics of the scene are used to divide the time window in order to capture the traffic characteristics within the scene. C k The time window length is Δt k Then the scene C k Divided into m Time window : in, W k，j Representing a scene C k The Middle j A time window, the length of the time window Δt k Adjust according to the needs of the scenario; Calculate each time window W k，j Various traffic statistics characteristics.

2. The method for detecting abnormal APP traffic based on multi-scale time series as described in claim 1, characterized in that: The identification of the APP running scenario in step 2 is as follows: matching preset scenario tags based on keywords or domain names in the request URL, wherein the scenario tags are registration, login, payment, advertising, or resource loading; The time window division rule in step 2 is as follows: a smaller time window is used for high-density scenarios and a longer time window is used for low-density scenarios. The high-density scenarios include resource loading or advertising requests, and the low-density scenarios include payment or login. In step 3, the db4 wavelet basis is used to perform discrete wavelet transform on the feature vector sequence, decomposing it into high-frequency components and low-frequency components. The method of using high-frequency and low-frequency anomaly scores in step 4 is as follows: if both the high-frequency anomaly score and the low-frequency anomaly score exceed a preset threshold, it is determined to be a persistent anomaly; if only the high-frequency score exceeds the threshold, it is determined to be a sudden anomaly; if only the low-frequency score exceeds the threshold, it is determined to be a trend anomaly.

3. A multi-scale time series-based APP traffic anomaly detection system, characterized in that, include: The traffic data collection module captures network traffic from the app, categorizes it by request type, and performs packet capture. The traffic data preprocessing module identifies the APP running scenario based on the request content and Uniform Resource Locator, groups the traffic data by scenario, divides time windows based on scenario characteristics, and calculates the traffic statistics characteristics of each item in each time window. The traffic statistics characteristics include API call frequency, request size distribution, response time, request type distribution, and request type switching frequency. The multi-scale time series decomposition module combines the traffic statistics features of each time window into a feature vector sequence, decomposes it into high-frequency and low-frequency components through wavelet transform, and performs empirical mode decomposition on the low-frequency components to obtain multiple intrinsic mode functions. The abnormal traffic detection module includes a high-frequency component anomaly detection module, a low-frequency component anomaly detection module, and an anomaly judgment module. The high-frequency component anomaly detection module uses the isolated forest algorithm to calculate the high-frequency anomaly score for the high-frequency components. The low-frequency component anomaly detection module uses a long short-term memory (LSTM) network to calculate the low-frequency anomaly score for the intrinsic mode functions of the low-frequency components. The anomaly judgment module comprehensively judges and outputs the anomaly type and risk level by using the high-frequency and low-frequency anomaly scores. The specific steps of grouping traffic data by scenario and dividing time windows based on scenario characteristics are as follows: Traffic data is divided into different app usage scenarios, with each scenario's traffic set to [value]. C k , k Indicate the scene number, and obtain the content. n Traffic datasets identified in different scenarios D This can be expressed as the following formula: in, C 1 indicates traffic data related to the registration scenario; C 2 indicates traffic data related to the login scenario; C n This indicates other identifiable scene traffic data; For each scenario C k The characteristics of the scene are used to divide the time window in order to capture the traffic characteristics within the scene. C k The time window length is Δt k Then the scene C k Divided into m Time window : in, W k，j Representing a scene C k The Middle j A time window, the length of the time window Δt k Adjust according to the needs of the scenario; Calculate each time window W k，j Various traffic statistics characteristics.

4. The APP traffic anomaly detection system based on multi-scale time series as described in claim 3, characterized in that: The traffic data preprocessing module further includes a scene recognition unit, which associates preset scene tags by matching URL keywords or domain names; In the multi-scale time series decomposition module, the decomposition of low-frequency components uses empirical mode decomposition to generate multiple intrinsic mode functions; The isolated forest algorithm of the high-frequency component anomaly detection module determines the high-frequency anomaly score by calculating the path length of data points. The shorter the path length, the higher the high-frequency anomaly score. The LSTM model of the low-frequency component anomaly detection module calculates the low-frequency anomaly score by the residual between the predicted value and the actual value. The larger the residual, the higher the low-frequency anomaly score.

5. The APP traffic anomaly detection system based on multi-scale time series as described in claim 4, characterized in that: The calculation for each time window W k，j The various traffic statistics characteristics are as follows: calculating the API call frequency within this time window. f API Request the mean of the size distribution μ s and variance σ s 2 Mean response time μ r and variance σ r 2 Request type distribution and request type switching frequency f 切换 The request types are distributed as follows: API requests P ( T 1) Resource loading request P ( T 2) ， Advertising requests P ( T 3) Other requests P ( T 4); Scene C k of W k，j The various flow characteristics within the time window are combined to form a feature vector. x k，j This can be expressed as the following formula: Scene C k The feature vectors of all time windows are combined into a feature vector sequence. X k ( t ) ： The feature sequences of multiple time windows in all scenarios are combined into a single time series. X ( t ): in, T It is the total number of time windows. n This indicates the number of scene categories divided, and this time series provides input for the subsequent multi-scale time series decomposition module.

6. The APP traffic anomaly detection system based on multi-scale time series as described in claim 4, characterized in that: The multi-scale time series decomposition module divides the time window W k，j The various flow characteristics are combined into a feature vector; the feature vectors of all time windows are combined into a feature vector sequence; the feature vectors of each time window are arranged sequentially to form a feature vector sequence. X For the above feature vector sequence X Wavelet decomposition is performed to obtain the high-frequency and low-frequency components of the traffic data; the traffic feature sequence is analyzed. X ( t Perform discrete wavelet transform and use the db4 wavelet basis for first-order decomposition to obtain the low-frequency components. L ( t and high-frequency components H ( t The formula for calculating wavelet transform is as follows: in, N The number of sample points representing the flow characteristic sequence; n This represents the discrete-time index, i.e., the input flow characteristic sequence. The location of the sample points; Represents the input flow characteristic sequence; This represents the high-pass filter coefficients in the db4 wavelet basis. This represents the low-pass filter coefficients in the wavelet transform; t The output signal is indexed by time.

7. The APP traffic anomaly detection system based on multi-scale time series as described in claim 4, characterized in that: The high-frequency component anomaly detection module determines the high-frequency anomaly score. S H The calculation is performed using the following formula: in, AnomalyScore ( H i (This refers to the Isolation Forest algorithm for data points) H i High-frequency abnormal scores, n This represents the number of data points.

8. The APP traffic anomaly detection system based on multi-scale time series as described in claim 4, characterized in that: The low-frequency component anomaly detection module uses empirical mode decomposition to detect low-frequency components. L(t) It is decomposed into multiple Intrinsic Mode Functions (IMFs) to capture subtle trends and patterns; For each IMF, a Long Short-Term Memory (LSTM) network is used for anomaly detection in the time series data. Let the model input be... IMF k ( t The output is the low-frequency anomaly score of the IMF. Low-frequency anomaly score S L It is obtained by weighting the low-frequency anomaly scores of each IMF, as shown in the following formula: in, α k Weights for each IMF, The score for low-frequency anomalies in the IMF. k For the number of IMFs, K This represents the total number of all IMFs.

9. The APP traffic anomaly detection system based on multi-scale time series as described in claim 4, characterized in that: Based on the anomaly detection results for high-frequency and low-frequency components, the anomaly detection module further assesses the traffic risk for each time window; for each time window... t The high-frequency anomaly scores and low-frequency anomaly scores are combined for judgment and processing, and the high-frequency anomaly threshold is obtained by training the model in the dataset. T H Low-frequency anomaly threshold T L The risk levels are divided according to the degree to which the scores exceed the threshold; By integrating high-frequency and low-frequency anomaly types, we can generate traffic anomalies, persistent anomaly patterns, and long-term trend anomaly types.

Images