Abnormal flow detection method and device, equipment and medium

Abstract

The invention relates to the technical field of cloud monitoring, particularly can be applied to a financial network scene, and discloses an abnormal traffic detection method, device and equipment and a medium, and the method comprises the steps: obtaining network traffic collected in real time in a proxy gateway cluster; performing feature extraction of at least one dimension on the network traffic to obtain a multi-dimensional traffic feature; analyzing the multi-dimensional traffic characteristics by using a pre-deployed detection model in the proxy gateway cluster to obtain an abnormal traffic analysis result; and feeding back the traffic analysis result to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts a triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result. Known attack features can be learned from network traffic and generalized to be applied to an unknown scene as a trigger condition, and potential risks in the network traffic can be effectively intercepted.

Description

Technical Field

[0001] The present invention relates to the field of cloud monitoring technology, and can be specifically applied to financial network scenarios, and in particular to a method, device, equipment and medium for detecting abnormal traffic. Background Art

[0002] With the rapid development of the internet, it has become deeply integrated into every aspect of social life. From core business operations to personal social interaction and entertainment, all rely heavily on the internet. At the same time, cybersecurity attacks and defenses are becoming increasingly fierce, and cyberattacks are becoming increasingly sophisticated. Abnormal traffic detection has become a critical task in network security, especially in financial scenarios. Financial transactions contain large amounts of sensitive data. Abnormal traffic detection can maintain the real-time nature of transactions and ensure the efficient and stable transmission of transaction data.

[0003] In related technologies, abnormal traffic detection methods primarily rely on predefined rules and feature matching. By analyzing known attack behaviors, representative features are extracted, compiled into rules, and stored in the detection system. As network traffic passes through, the proxy network cluster scans the traffic data piece by piece, comparing it against pre-stored rules. Once a match is found, the traffic is identified as abnormal. However, these pre-stored rules are based on known attack patterns and feature libraries. Faced with the ever-increasing number of location-based threats and highly concealed advanced persistent threats, it is difficult to extract universal attack features and formulate rules, making it impossible to effectively identify potential risks. Summary of the Invention

[0004] The present invention provides a method, device, computer equipment and medium for detecting abnormal traffic to solve the technical problem that related technologies are difficult to extract common attack features and form rules from emerging location threats and highly concealed advanced persistent threats, and are unable to effectively identify potential risks.

[0005] In a first aspect, a method for detecting abnormal traffic is provided, comprising:

[0006] Get real-time collected network traffic in the proxy gateway cluster;

[0007] Extracting features of at least one dimension from the network traffic to obtain multi-dimensional traffic features;

[0008] Analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results;

[0009] The abnormal traffic analysis result is fed back to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

[0010] In a second aspect, a device for detecting abnormal traffic is provided, comprising:

[0011] The first acquisition module is used to obtain the network traffic collected in real time in the proxy gateway cluster;

[0012] An extraction module, configured to extract features of at least one dimension of the network traffic to obtain multi-dimensional traffic features;

[0013] An analysis module is used to analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results;

[0014] The adjustment module is used to feed back the abnormal traffic analysis result to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering conditions of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

[0015] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned abnormal traffic detection method when executing the computer program.

[0016] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned abnormal traffic detection method are implemented.

[0017] In the scheme implemented by the above-mentioned abnormal traffic detection method, device, computer equipment and storage medium, the network traffic collected in real time in the proxy gateway cluster can be obtained through the server; the network traffic is subjected to feature extraction of at least one dimension to obtain multi-dimensional traffic features; the multi-dimensional traffic features are analyzed using the detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results; the traffic analysis results are fed back to the proxy gateway cluster so that the proxy gateway cluster dynamically adjusts the triggering conditions for intercepting abnormal traffic in the proxy server according to the abnormal traffic analysis results on the client side. In the present invention, by pre-deploying the detection model in the gateway proxy cluster, the abnormal traffic features hidden in the network traffic can be accurately excavated, and the abnormal traffic features can be parsed as triggering conditions for intercepting abnormal traffic. In this way, known attack features can be learned from the network traffic and generalized to unknown scenarios as triggering conditions, which can effectively intercept potential risks in the network traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 1 is a schematic diagram of an application environment of a method for detecting abnormal traffic in an embodiment of the present invention;

[0020] Figure 2 1 is a flow chart of a method for detecting abnormal traffic in one embodiment of the present invention;

[0021] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S10;

[0022] Figure 4 is a flow chart of a method for detecting abnormal traffic in another embodiment of the present invention;

[0023] Figure 5 yes Figure 2 A schematic flow chart of a specific implementation of step S40;

[0024] Figure 6 1 is a schematic structural diagram of an abnormal flow detection device according to an embodiment of the present invention;

[0025] Figure 7 is a structural diagram of a computer device in one embodiment of the present invention;

[0026] Figure 8 FIG. 2 is another structural diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] The abnormal flow detection method provided by the embodiment of the present invention can be applied in the following situations: Figure 1In an application environment, the network traffic collected in real time in the proxy gateway cluster is obtained through the server; the network traffic is subjected to feature extraction of at least one dimension to obtain multi-dimensional traffic features; the multi-dimensional traffic features are analyzed using the detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results; the traffic analysis results are fed back to the proxy gateway cluster so that the proxy gateway cluster dynamically adjusts the triggering conditions for intercepting abnormal traffic in the proxy server according to the abnormal traffic analysis results on the client. In the present invention, by pre-deploying the detection model in the gateway proxy cluster, the abnormal traffic features hidden in the network traffic can be accurately mined, and the abnormal traffic features can be parsed as triggering conditions for intercepting abnormal traffic, so that known attack features can be learned from the network traffic and generalized to unknown scenarios as triggering conditions, which can effectively intercept potential risks in the network traffic. Among them, the client can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0029] See also Figure 2 As shown, Figure 2 A flow chart of a method for detecting abnormal traffic provided by an embodiment of the present invention includes the following steps:

[0030] S10: Obtain the network traffic collected in real time in the proxy gateway cluster.

[0031] The abnormal traffic detection method provided by this invention can be applied to security protection scenarios in various network environments. Its core approach is to proactively detect potential threats and implement defensive measures by identifying abnormal patterns in network traffic. Taking the financial sector as an example, the abnormal traffic detection process can monitor internal traffic between financial databases and server clusters to prevent attackers from breaching the boundaries and spreading within the internal network. For example, in scenarios such as bank transfers and securities trading, risks such as high-frequency trading fraud and account theft are common. An abnormal traffic detection system can establish a baseline for user transaction behavior and analyze characteristics such as transaction frequency, amount, and time period. Once an anomaly is detected, the system immediately triggers an alarm and intercepts the transaction. For another example, financial data centers store vast amounts of customer information and transaction data, facing threats such as database tampering and ransomware attacks. An abnormal traffic detection system can conduct in-depth analysis of network traffic between internal servers, parsing query statements and monitoring port connections. If a large number of suspicious queries are detected within a short period of time, or if non-standard ports are accessed frequently, the attack can be promptly blocked to prevent data leakage.

[0032] In this embodiment, the proxy gateway cluster is a distributed system composed of multiple proxy servers that centrally manages network traffic forwarding, filtering, load balancing, and security control. Its core function is to act as an intermediary node for network communications, forwarding client requests to backend services and centrally processing network traffic. It offers high availability, scalability, and security.

[0033] Typically, a proxy gateway cluster consists of a load balancer, gateway server nodes, and a control plane. Specifically, in a proxy gateway cluster, client requests first reach the load balancer, which then distributes the request to a gateway node based on an algorithm. The gateway node processes the network traffic and forwards it to the backend service. The response traffic returns to the client along the reverse path, and the gateway node reports the network traffic to the control plane.

[0034] In specific implementation scenarios, the proxy gateway cluster can collect network traffic in real time through a variety of methods. As a collection method, the gateway nodes in the proxy gateway cluster have built-in log modules, which can record the metadata of network traffic in real time, such as source IP, destination port, request method, response status code, and push it to the centralized storage or streaming processing platform through the log forwarding tool. Figure 3 As shown, step S10, i.e. obtaining the network traffic collected in real time in the proxy gateway cluster, includes the following steps:

[0035] S11: configuring a log file for recording network traffic in the proxy gateway cluster, so as to define preset parameter fields for collecting network traffic through the log file.

[0036] S12: When a network request flows through the proxy gateway cluster, the network request is parsed, and preset parameter fields obtained by parsing are stored in a log file according to a field format.

[0037] In step S11, the log file configured in the proxy gateway cluster will record the parameter field of the network traffic. The parameter field must meet the integrity condition, that is, it must cover the key attributes of the entire life cycle of the network traffic to ensure that the complete link of the traffic can be traced. The parameter field also needs to meet the standard condition, that is, it must be stored in a unified structured format to facilitate cross-cluster and cross-system parsing and aggregation. The parameter field also needs to meet the business adaptation condition, that is, it must reserve extended fields in addition to the basic fields, such as business tags and user IDs, to support customized collection in different scenarios. Specific parameter fields include but are not limited to time dimension fields, such as request time, processing time, upstream response time, etc., client information fields, such as source IP and port, user agent, request source, etc., proxy forwarding and target service fields, such as target IP and port, service name / identifier, forwarding status, etc., request and response content fields, such as request method, response status code, traffic size, etc., network connection fields, such as connection ID, connection status, number of concurrent requests, etc.

[0038] Furthermore, to flexibly define the preset parameter fields in log files, the preset parameter fields for collecting network traffic can be defined in conjunction with business scenarios. For example, the user identity field can be defined in conjunction with the session ID field to analyze traffic anomalies for specific users. The preset parameter fields for collecting network traffic can also be expanded to reflect business scenarios. For example, the business tag field can be used to assign log collection accuracy based on the business line to which the network traffic belongs, according to business priority. Another example is the request source channel field, which defines the anomaly channel based on the source of the network traffic.

[0039] As a second collection method, gateway nodes in a proxy gateway cluster can also configure traffic mirroring at the load balancer or network switch level, copying the original traffic and sending it to a dedicated collection node for real-time network traffic collection. As a third collection method, nodes in a proxy gateway cluster can also provide interfaces that allow external systems to periodically pull traffic statistics.

[0040] In step S12, parsing the network request includes network layer parsing, through which the five-tuple information of the network request, such as the source IP, target IP, and port number, can be extracted to obtain the network source and forwarding target of the request. Parsing the network request also includes transport layer parsing, through which the protocol characteristics can be analyzed and the connection establishment time, timeout status, etc. can be recorded. Parsing the network request also includes application layer parsing, through which the request content can be parsed according to the protocol type, including first protocol type request parsing and second protocol type request parsing. The first protocol type request can extract the request method, request body length, protocol header, etc., while the second protocol type request can extract the query domain name, response IP, etc. Preset parameter fields are further extracted from the parsing results and the format is standardized. It should be noted that if the parsing results contain suspicious content, the corresponding parameter fields can be marked as abnormal.

[0041] In this embodiment, the field format is a storage format of a preset parameter field in a log file. Different types of parameter fields can be stored in corresponding field positions in the log file through field classification and format planning.

[0042] S20: Extract features of at least one dimension of the network traffic to obtain multi-dimensional traffic features.

[0043] In this embodiment, network traffic feature extraction is the basis for anomaly detection and attack identification. By preprocessing network traffic and performing multi-dimensional analysis, the network traffic behavior can be comprehensively characterized. Here, the multi-dimensional traffic features include at least one of statistical traffic features, protocol traffic features, and content traffic features.

[0044] Specifically, time information can be extracted from network traffic in the quantity dimension to obtain statistical traffic characteristics, where the statistical traffic characteristics include at least one of request frequency, number of requests, response time, and request interval time; and / or traffic behavior analysis can be performed on network traffic in the communication rule dimension to obtain protocol traffic characteristics, where the protocol traffic characteristics include at least one of request method, request header information, uniform resource locator structure, and request identifier; and / or semantic information mining can be performed on network traffic in the data carrier dimension to obtain content traffic characteristics, where the content traffic characteristics include at least one of keywords, malicious code fragments, and sensitive information in the request content.

[0045] It can be understood that extracting time information from network traffic in the quantitative dimension is equivalent to a quantification process for traffic behavior. Among them, the request frequency is the number of network requests per unit time, which is used to reflect the density of network traffic. The time can be set through a sliding window to count the number of requests within the window. The number of requests is the total number of requests in a specified time period, which is used for trend analysis and capacity planning. The total number of requests can be accumulated by minutes, hours, and days to draw a request volume trend chart. The response time is the time interval from the client to the response received, which is used to reflect service availability and performance. Specifically, it can be calculated by recording the difference between the sending time and the receiving time of each request. The request interval feature is the time interval between two requests, which is used to reflect the timing law of the request. The difference between the current request time and the previous request time can be calculated according to the request sequence to obtain the interval time series.

[0046] It can be understood that analyzing the traffic behavior of network traffic in the dimension of communication rules is equivalent to the process of identifying the communication protocol pattern, wherein the request method is a field in the application layer protocol that identifies the request type, which is used to reflect the client behavior pattern and potential risks, and can be calculated based on the proportion of business requests. The request header information contains metadata such as the client environment and request attributes. Abnormalities in key fields can reveal potential risks, which can be obtained by parsing the target field through the Uniform Resource Locator header. The Uniform Resource Locator structure includes the path, parameters and encoding format of the URI to reflect resource access behavior. Abnormal structures can identify attack intentions, which can be obtained through semantic matching through grammatical analysis and set encoding deformation. The request identifier is used to track sessions or requests. Its abnormal use can reveal attack patterns and can be extracted from the request header or identifier.

[0047] It can be understood that the semantic information mining of network traffic in the data carrier dimension is equivalent to the analysis process of traffic load semantics. Among them, the keywords in the request content are character sequences with business or security semantics in the traffic load. Their frequency of occurrence and context can reveal the nature of the traffic and can be obtained through regular expression matching, word frequency statistics and semantic analysis, keyword deformation processing, etc. Malicious code fragments are executable or destructive instructions contained in the traffic, which are common in script injection, binary features and command injection processes. They can be obtained through feature library matching, behavioral sandbox analysis, and code syntax detection. Sensitive information is data involving privacy or security, including personal information, financial data, business sensitive data, etc., which can be obtained through rule engine detection, data classification and grading, and encryption and desensitization detection.

[0048] S30: Analyze the multi-dimensional traffic characteristics using the detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results.

[0049] In this embodiment, the proxy gateway cluster acts as an intermediate node between the client and the target server, and all traffic entering and leaving the network must be forwarded through it. To ensure the security and reliability of network traffic, a model for traffic anomaly detection can be pre-deployed in the proxy gateway cluster. The detection model can be deployed on a gateway node specified in the proxy gateway cluster, eliminating the need to deploy a separate detection module on each business server, reducing architectural complexity. The detection model can also be deployed on each gateway node in the proxy gateway cluster, ensuring direct connection between the detection model and the network traffic collected by each gateway node, reducing data transmission latency.

[0050] Specifically, multidimensional traffic features can be converted into numerical vectors, aggregated according to a set time window, and then input into the detection model. This detection model can be deployed in a multi-layered model configuration. At the first layer, a lightweight rule engine or machine learning model can be deployed to quickly identify obvious anomalies in multidimensional traffic features, such as a large number of requests from a single IP address within a short period of time. At the second layer, a fine-grained analysis model can be deployed to analyze time series patterns or perform unsupervised anomaly detection on traffic using deep learning models to identify hidden attacks, such as slow CC attacks. At the third layer, an expert model can be deployed to analyze user behavior for specific scenarios, such as financial transactions, by combining business rules with graph neural networks to detect complex anomalies such as account theft. Detection results from different layers can then be fused through model integration. As one fusion method, weights can be assigned to models at different layers based on their accuracy, and the weighted summation of detection results from different layers can be used to generate anomaly traffic analysis results. Another fusion method is to use the output of the model at the previous layer as input to the model at the current layer, with the output of the model at the final layer serving as the anomaly traffic analysis result.

[0051] As a judgment process for abnormal traffic analysis results, the output of the detection model is usually a score value obtained through feature matching. At this time, the abnormal traffic analysis results can be combined with historical data and business needs to set dynamic threshold conditions. The dynamic threshold is used to determine whether the score value output by the detection model meets the threshold conditions and determine the abnormal traffic analysis results.

[0052] As another judgment process for abnormal traffic analysis results, the output of the detection model is usually the abnormal category label obtained through heterogeneous feature matching. At this time, the abnormal traffic analysis results can comprehensively evaluate the risk level of the abnormal category based on the impact range, attack intensity, business sensitivity, etc., and determine the abnormal traffic analysis results based on the risk level of the abnormal category.

[0053] In actual application scenarios, the combination of proxy gateway clusters and detection models can achieve global linkage for real-time identification of abnormal traffic. Considering the differences in network traffic characteristics in different business scenarios, specific detection models need to be trained in combination with category labels in the corresponding business scenarios. Figure 4 As shown, before step S30, the method further includes the following steps:

[0054] S50: Obtain network traffic with different category labels collected by the proxy gateway cluster.

[0055] S60: Inputting the network traffic features obtained by extracting the network traffic with different category labels into the network model for training, and capturing the temporal dependency features of the network traffic during the training process.

[0056] S70: Learning the mapping relationship between network traffic on different category labels according to the timing dependency feature, so as to construct a detection model through the mapping relationship.

[0057] In step S50, a proxy gateway cluster acts as a checkpoint for network traffic and can be deployed at network entry points or key nodes to collect all inbound and outbound network traffic. The collected network traffic contains a wealth of information, from transport layer protocols to network layer IP addresses and application layer content. Each network flow record includes the source IP address, destination IP address, port number used, protocol type, and key application layer information, such as the Uniform Resource Locator (URL) of the web page accessed and the response status code returned by the server. In other words, network traffic can be used to fully reconstruct the entire network interaction process.

[0058] As a collection method, gateway nodes in a proxy gateway cluster can leverage underlying technologies to directly capture network traffic from the network core layer, reducing data transmission losses and ensuring efficient collection. Another collection method is to utilize a layered collaborative architecture deployed within the proxy gateway cluster to capture a more comprehensive set of network traffic layer by layer. Specifically, a layered collaborative architecture can be deployed within the proxy gateway cluster. This layered collaborative collection architecture includes a traffic access layer, a traffic aggregation layer, and a traffic tagging layer. The traffic access layer controls the collection of basic network traffic by each gateway node in the proxy gateway cluster. The traffic aggregation layer correlates the basic network traffic of different gateway nodes based on business domains / regions. The traffic tagging layer maps labels to these correlated network traffic based on a category tag system. The layered collaborative architecture then performs multi-layer link processing on the network traffic collected by each gateway node in the proxy gateway cluster, resulting in network traffic with different category tags. For example, to implement real-time compliance tag injection in a financial scenario, the access layer of the layered collaborative architecture can detect identity information contained in the request body, the aggregation layer can record user information transmission, and the table layer can automatically trigger compliance audit tags and encrypt and store the relevant fields.

[0059] In step S60, the network model training process dynamically adjusts the window size of network traffic based on the category label. This allows the global attention mechanism to calculate the dependencies between network traffic at any time step and capture the temporal dependency characteristics of network traffic. The core of this process is to convert the time dimension into a computable model input. Through an architecture and training strategy adapted to temporal characteristics, the network model learns the evolution logic of network traffic along the time axis, enabling timely identification of anomalies in network traffic and the discovery of complex attack patterns hidden in time series, enabling dynamic analysis and prediction of network traffic.

[0060] In step S70, the time-dependent features can be vectorized and expressed as a time correlation matrix or state transition model. The time correlation matrix can be used to calculate the correlation between the features and labels at each time step. For example, the correlation coefficient between the number of abnormal port scans in the past hour and the network attack label is 0.7. The state transition matrix can record the evolution path from normal traffic to abnormal labels, such as the state transition probability between normal access, abnormal login, and privilege escalation, and is used to predict subsequent labels.

[0061] Specifically, the attention mechanism can be used to convert different category labels into semantic vectors for weighted interaction with time-dependent features to obtain feature weights of different category labels; the mapping relationship of network traffic on different category labels can be learned based on the feature weights of different category labels; the loss value of the mapping relationship is calculated based on a pre-constructed loss function, and the network model parameters are adjusted through the loss value to construct a detection model. The detection model is used to output the category label mapped to the target network traffic based on the target network traffic with unknown category labels.

[0062] S40: Feedback the abnormal traffic analysis result to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition for intercepting abnormal traffic in the proxy server according to the abnormal traffic analysis result.

[0063] In this embodiment, the abnormal traffic analysis result is the abnormal detection information in the same format output by the monitoring model, including but not limited to core fields such as the abnormality type, trigger threshold, impact range, recommended interception action, and generation time contained in the network traffic.

[0064] Specifically, the proxy gateway cluster can translate the anomaly detection information into rules based on the abnormal traffic analysis results, and use the translated rules to dynamically adjust the triggering conditions for abnormal traffic interception in the proxy server. For example, if the abnormal traffic analysis results detect that a certain IP frequently accesses an interface, an IP blacklist ban rule can be generated to dynamically adjust the triggering conditions for IP access in the proxy server. For another example, if the abnormal traffic analysis results detect that the path access frequency exceeds a threshold, a path speed limit policy can be generated to dynamically adjust the triggering conditions for path access in the proxy server.

[0065] In the specific implementation scenario, every time the detection model identifies a valid anomaly, it will generate an abnormal traffic analysis result and feed it back to the proxy gateway cluster, so that the proxy gateway cluster can adjust its strategy according to the current anomaly. Figure 5 As shown, in step S40, the abnormal traffic analysis result is fed back to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result, including the following steps:

[0066] S41: Feedback the abnormal traffic analysis result to the proxy gateway cluster through the coordinated deployment of a preset service interface and / or a message queue, so that the proxy gateway cluster obtains a feature identifier of the abnormal traffic.

[0067] S42: Dynamically adjust the triggering condition for intercepting abnormal traffic in the proxy server according to the characteristic identifier of the abnormal traffic.

[0068] In step S41, the preset service interface is a standardized interactive channel provided by the service interface layer. By calling the preset service interface, the abnormal traffic analysis results output by the detection model can be fed back to the proxy gateway cluster. The message queue is a data channel for asynchronous real-time communication. Usually, the abnormal traffic analysis results output by the detection model are fed back to the proxy gateway cluster based on the publish-subscribe model. The specific collaborative deployment process can deploy a message queue client and an interface call component on each gateway node in the proxy gateway cluster to ensure that each gateway node can access both channels at the same time. After the detection model outputs the abnormal traffic analysis results, the push channel can be marked according to the urgency of the feature. For the abnormal traffic analysis results of urgent features, they can be pushed through the message queue first. For the abnormal traffic analysis results of regular features, they can be regularly pulled by the gateway provided by the service interface.

[0069] It is understood that when the detection model outputs the analysis results of abnormal traffic, it will encapsulate the identified abnormal features into structured feature identifiers, which serve as the basis for subsequent interception strategy adjustments. Accordingly, after receiving the abnormal traffic analysis results, the proxy gateway cluster can obtain the feature identifiers of the abnormal traffic. These feature identifiers can include basic attributes such as the anomaly type, specific feature value, and timestamp, as well as risk attributes such as risk level, scope of impact, and historical trigger counts. They can also include contextual attributes such as the associated business scenario, user ID, and source of the problem.

[0070] In step S42, the characteristic identification of abnormal traffic determines the direction in which the proxy server adjusts the interception strategy. Different interception strategies can be adopted according to the types of different characteristic identifications, and the triggering conditions for interception of abnormal traffic in the proxy server can be adjusted accordingly. For example, IP-type characteristic identification can directly add the IP to the gateway blacklist, triggering the interception condition of "rejecting all requests". For example, behavioral-type characteristic identification can set a frequency threshold for the uniform resource identifier separately, and trigger interception when the frequency threshold is exceeded, or add additional verification. For example, protocol-type characteristic identification can add protocol verification rules at the gateway layer, and directly discard data packets that do not meet the rules, or record such characteristics for updating the normal traffic baseline.

[0071] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0072] In one embodiment, a device for detecting abnormal flow is provided, which corresponds to the abnormal flow detection method in the above embodiment. Figure 6 As shown, the abnormal traffic detection device includes a first acquisition module 101, an extraction module 102, an analysis module 103 and an adjustment module 104. The functional modules are described in detail as follows:

[0073] The first acquisition module 101 is used to obtain the network traffic collected in real time in the proxy gateway cluster;

[0074] An extraction module 102 is configured to extract features of at least one dimension of the network traffic to obtain multi-dimensional traffic features;

[0075] An analysis module 103 is configured to analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results;

[0076] The adjustment module 104 is configured to feed back the abnormal traffic analysis result to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition for intercepting abnormal traffic in the proxy server according to the abnormal traffic analysis result.

[0077] In one embodiment, the first acquisition module 101 is specifically configured to:

[0078] Configuring a log file for recording network traffic in the proxy gateway cluster, so as to define preset parameter fields for collecting network traffic through the log file;

[0079] When a network request flows through the proxy gateway cluster, the network request is parsed, and the preset parameter fields obtained by parsing are stored in a log file according to the field format.

[0080] In one embodiment, the extraction module 102 is specifically configured to:

[0081] Extracting time information from the network traffic in a quantity dimension to obtain statistical traffic characteristics, wherein the statistical traffic characteristics include at least one of request frequency, number of requests, response time, and request interval; and / or

[0082] Performing traffic behavior analysis on the network traffic based on the communication rules dimension to obtain protocol traffic characteristics, wherein the protocol traffic characteristics include at least one of a request method, request header information, a uniform resource locator structure, and a request identifier; and / or

[0083] Semantic information mining is performed on the network traffic in the data carrier dimension to obtain content traffic characteristics, where the content traffic characteristics include at least one of keywords, malicious code fragments, and sensitive information in the request content.

[0084] In one embodiment, the apparatus further comprises:

[0085] A second acquisition module is configured to acquire network traffic with different category labels collected by the proxy gateway cluster before analyzing the multi-dimensional traffic characteristics using the detection model pre-deployed in the proxy gateway cluster to obtain traffic analysis results;

[0086] A training module is used to input the network traffic features obtained by extracting the network traffic with different category labels into the network model for training, and capture the temporal dependency features of the network traffic during the training process;

[0087] A construction module is used to learn the mapping relationship of network traffic on different category labels based on the timing dependency features, so as to build a detection model through the mapping relationship.

[0088] In one embodiment, the second acquisition module is specifically configured to:

[0089] Deploy a layered collaborative architecture in the proxy gateway cluster. The layered collaborative collection architecture includes a traffic access layer, a traffic aggregation layer, and a traffic marking layer. The traffic access layer is used to control each gateway node in the proxy gateway cluster to collect basic network traffic. The traffic aggregation layer is used to time-sequence the basic network traffic of different gateway nodes according to the business domain / region. The traffic marking layer is used to perform label mapping on the time-sequence-related network traffic according to the category label system.

[0090] According to the layered collaborative architecture, multi-layer link processing is performed on the network traffic collected by each gateway node in the proxy gateway cluster to obtain network traffic with different category labels.

[0091] In one embodiment, the building module is specifically configured to:

[0092] Using the attention mechanism to convert the different category labels into semantic vectors and perform weighted interaction with the temporal dependency features to obtain feature weights of the different category labels;

[0093] Learning the mapping relationship between network traffic on different category labels according to the feature weights of the different category labels;

[0094] The loss value of the mapping relationship is calculated according to a pre-constructed loss function, so as to adjust the parameters of the network model through the loss value and construct a detection model. The detection model is used to output the category label mapped by the target network traffic based on the target network traffic with unknown category label.

[0095] In one embodiment, the adjustment module 104 is specifically configured to:

[0096] Feedback of the abnormal traffic analysis results to the proxy gateway cluster through the coordinated deployment of a preset service interface and / or a message queue, so that the proxy gateway cluster obtains a characteristic identifier of the abnormal traffic;

[0097] The triggering conditions for intercepting abnormal traffic in the proxy server are dynamically adjusted according to the characteristic identifier of the abnormal traffic.

[0098] The present invention provides a device for detecting abnormal traffic. By pre-deploying a detection model in a gateway proxy cluster, it can accurately mine the abnormal traffic features hidden in network traffic. The abnormal traffic features can be parsed as trigger conditions for intercepting abnormal traffic. In this way, known attack features can be learned from network traffic and generalized to unknown scenarios as trigger conditions, which can effectively intercept potential risks in network traffic.

[0099] The specific definition of the abnormal traffic detection device can be found in the definition of the abnormal traffic detection method above and will not be repeated here. The various modules in the above-mentioned abnormal traffic detection device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0100] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a method for detecting abnormal traffic.

[0101] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a method for detecting abnormal traffic.

[0102] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0103] Get real-time collected network traffic in the proxy gateway cluster;

[0104] Extracting features of at least one dimension from the network traffic to obtain multi-dimensional traffic features;

[0105] Analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results;

[0106] The abnormal traffic analysis result is fed back to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

[0107] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0108] Get real-time collected network traffic in the proxy gateway cluster;

[0109] Extracting features of at least one dimension from the network traffic to obtain multi-dimensional traffic features;

[0110] Analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results;

[0111] The abnormal traffic analysis result is fed back to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

[0112] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0113] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0114] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0115] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for detecting abnormal traffic, characterized in that: include: Get real-time collected network traffic in the proxy gateway cluster; Extracting features of at least one dimension from the network traffic to obtain multi-dimensional traffic features; Analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results; The abnormal traffic analysis result is fed back to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering condition of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

2. The abnormal flow detection method according to claim 1, characterized in that: The method of obtaining the network traffic collected in real time in the proxy gateway cluster includes: Configuring a log file for recording network traffic in the proxy gateway cluster, and defining preset parameter fields for collecting network traffic through the log file; When a network request flows through the proxy gateway cluster, the network request is parsed, and the preset parameter fields obtained by parsing are stored in a log file according to the field format.

3. The abnormal flow detection method according to claim 1, wherein: The extracting features of at least one dimension of the network traffic to obtain multi-dimensional traffic features includes: Extracting time information from the network traffic in a quantity dimension to obtain statistical traffic characteristics, wherein the statistical traffic characteristics include at least one of request frequency, number of requests, response time, and request interval; and / or Performing traffic behavior analysis on the network traffic based on the communication rules dimension to obtain protocol traffic characteristics, wherein the protocol traffic characteristics include at least one of a request method, request header information, a uniform resource locator structure, and a request identifier; and / or Semantic information mining is performed on the network traffic in the data carrier dimension to obtain content traffic characteristics, where the content traffic characteristics include at least one of keywords, malicious code fragments, and sensitive information in the request content.

4. The abnormal flow detection method according to claim 1, wherein: Before analyzing the multi-dimensional traffic characteristics using the detection model pre-deployed in the proxy gateway cluster to obtain the traffic analysis results, the method further includes: Obtain network traffic with different category labels collected by the proxy gateway cluster; The network traffic features obtained by extracting the network traffic with different category labels are input into the network model for training, and the temporal dependency features of the network traffic are captured during the training process; The mapping relationship between network traffic and different category labels is learned according to the temporal dependency features, so as to construct a detection model through the mapping relationship.

5. The abnormal flow detection method according to claim 4, characterized in that: The network traffic with different category labels collected by the proxy gateway cluster is obtained, including: Deploy a layered collaborative architecture in the proxy gateway cluster. The layered collaborative collection architecture includes a traffic access layer, a traffic aggregation layer, and a traffic marking layer. The traffic access layer is used to control each gateway node in the proxy gateway cluster to collect basic network traffic. The traffic aggregation layer is used to time-sequence the basic network traffic of different gateway nodes according to the business domain / region. The traffic marking layer is used to perform label mapping on the time-sequence-related network traffic according to the category label system. According to the layered collaborative architecture, multi-layer link processing is performed on the network traffic collected by each gateway node in the proxy gateway cluster to obtain network traffic with different category labels.

6. The abnormal flow detection method according to claim 4, characterized in that: The learning of the mapping relationship between network traffic on different category labels based on the temporal dependency features to build a detection model includes: Using the attention mechanism to convert the different category labels into semantic vectors and perform weighted interaction with the temporal dependency features to obtain feature weights of the different category labels; Learning the mapping relationship between network traffic on different category labels according to the feature weights of the different category labels; The loss value of the mapping relationship is calculated according to a pre-constructed loss function, so as to adjust the parameters of the network model through the loss value and construct a detection model. The detection model is used to output the category label mapped by the target network traffic based on the target network traffic with unknown category label.

7. The abnormal flow detection method according to any one of claims 1 to 6, characterized in that: Feeding back the abnormal traffic analysis result to the proxy gateway cluster so that the proxy gateway cluster dynamically adjusts the triggering condition for intercepting abnormal traffic in the proxy server according to the abnormal traffic analysis result, includes: Feedback of the abnormal traffic analysis results to the proxy gateway cluster through the coordinated deployment of a preset service interface and / or a message queue, so that the proxy gateway cluster obtains a characteristic identifier of the abnormal traffic; The triggering conditions for intercepting abnormal traffic in the proxy server are dynamically adjusted according to the characteristic identifier of the abnormal traffic.

8. An abnormal flow detection device, characterized in that: include: The first acquisition module is used to obtain the network traffic collected in real time in the proxy gateway cluster; An extraction module, configured to extract features of at least one dimension of the network traffic to obtain multi-dimensional traffic features; An analysis module is used to analyze the multi-dimensional traffic characteristics using a detection model pre-deployed in the proxy gateway cluster to obtain abnormal traffic analysis results; The adjustment module is used to feed back the abnormal traffic analysis result to the proxy gateway cluster, so that the proxy gateway cluster dynamically adjusts the triggering conditions of abnormal traffic interception in the proxy server according to the abnormal traffic analysis result.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the abnormal traffic detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the abnormal traffic detection method according to any one of claims 1 to 7 are implemented.

Images