Network traffic processing method and apparatus, and storage medium and electronic device

Abstract

The present invention relates to the field of software development. Disclosed are a network traffic processing method and apparatus, and a storage medium and an electronic device. The method comprises: receiving network traffic data forwarded by a smart gateway device; performing feature extraction processing on the network traffic data, so as to obtain network traffic features; on the basis of the network traffic features, determining N abnormal events comprised in the network traffic data, and abnormal-event scores respectively corresponding to the N abnormal events; on the basis of the abnormal-event scores respectively corresponding to the N abnormal events, determining an abnormality processing strategy for the network traffic data; and sending the abnormality processing strategy to the smart gateway device, such that the smart gateway device performs abnormality elimination on the network traffic data on the basis of the abnormality processing strategy. The present invention solves the technical problem in the prior art of the flexibility and accuracy being poor due to abnormal events being detected and eliminated on the basis of manual or preset rules and threshold values.

Description

Network traffic processing method, device, storage medium and electronic device Technical Field

[0001] The present invention relates to the field of software development, and in particular to a network traffic processing method, device, storage medium and electronic equipment. Background Art

[0002] The gateway management technology in related technologies relies on pure manual support, making predictions based on the real-time network traffic data of the online business system, and executing expansion and contraction, network switching and other solutions. Technical issues

[0003] Specifically, related technologies rely on manual or preset rules and thresholds to detect anomalies during network traffic data transmission, which lacks flexibility and accuracy. Furthermore, manual intervention is required to correct anomalies in network traffic data, which is time-consuming, labor-intensive, and error-prone. Furthermore, the reliance on manual or preset rules and thresholds to detect anomalies makes it difficult to adapt to the dynamic changes and complexity of the network.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Technical Solutions

[0005] The embodiments of the present invention provide a network traffic processing method, device, storage medium and electronic device to at least solve the technical problem in related technologies that rely on manual or preset rules and thresholds to detect and eliminate abnormal events, which lacks flexibility and accuracy.

[0006] According to one aspect of an embodiment of the present invention, a network traffic processing method is provided, comprising: receiving network traffic data forwarded by an intelligent gateway device; performing feature extraction processing on the network traffic data to obtain network traffic features; determining, based on the network traffic features, N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, respectively, wherein N is an integer greater than or equal to 1; determining an abnormality handling strategy for the network traffic data according to the abnormal event scores corresponding to the N abnormal events, wherein the abnormality handling strategy includes an abnormality exclusion order for the N abnormal events, and an abnormality exclusion method corresponding to the N abnormal events; and sending the abnormality handling strategy to the intelligent gateway device, for the intelligent gateway device to perform abnormality exclusion on the network traffic data based on the abnormality handling strategy.

[0007] Optionally, the determining of N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events based on the network traffic characteristics includes: based on the network traffic characteristics, using a network traffic anomaly detection model to determine the N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein the network traffic anomaly detection model is obtained through machine learning based on a historical traffic data set, and the historical traffic data set includes multiple historical traffic data, as well as abnormal events and abnormal time scores corresponding to the multiple historical traffic data.

[0008] Optionally, based on the network traffic characteristics, a network traffic anomaly detection model is used to determine N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, including: based on the network traffic characteristics, the network traffic anomaly detection model is used to determine the abnormal event scores corresponding to M abnormal events, where M is an integer greater than or equal to 2; judging whether there is an abnormal event among the M abnormal events whose abnormal event score is greater than a preset first scoring threshold; if there is an abnormal event among the M abnormal events whose abnormal event score is greater than the preset first scoring threshold, the abnormal event with the abnormal event score greater than the preset first scoring threshold is taken as the N abnormal events; and determining the abnormal event scores corresponding to the N abnormal events from the abnormal event scores corresponding to the M abnormal events.

[0009] Optionally, the method also includes: when there is no abnormal event among the M abnormal events whose abnormal event score is greater than the preset first scoring threshold, determining the weight values ​​corresponding to the M abnormal events respectively; determining a comprehensive scoring value based on the weight values ​​corresponding to the M abnormal events respectively and the abnormal event scores corresponding to the M abnormal events respectively; and issuing an alarm indication when the comprehensive scoring value is greater than the preset second scoring threshold.

[0010] Optionally, after determining the abnormal event scores corresponding to M abnormal events respectively based on the network traffic characteristics and using the network traffic anomaly detection model, the method further includes: storing the network traffic data and the abnormal event scores corresponding to the M abnormal events respectively in the historical traffic data set to obtain a new historical traffic data set; and optimizing and updating the network traffic anomaly detection model based on the new historical traffic data set.

[0011] Optionally, the feature extraction processing of the network traffic data to obtain network traffic features includes: performing data cleaning processing on the network traffic data to obtain the network traffic features, wherein the network traffic features include at least: source address, destination address, port number, protocol type, data packet size, data packet arrival time interval, and network traffic transmission direction.

[0012] Optionally, the N abnormal events include at least one of the following: port scanning, distributed denial of service, network congestion, network security anomaly, network configuration error, wherein the abnormality elimination method corresponding to the port scan is: isolating abnormal traffic, wherein the isolating abnormal traffic is used to indicate that data packets with source or destination addresses as abnormal traffic are discarded; the abnormality elimination method corresponding to the distributed denial of service is: redirecting normal traffic, wherein the redirecting normal traffic is used to indicate that data packets with source or destination addresses as normal traffic are forwarded to a backup network; the abnormality elimination method corresponding to the network congestion is: optimizing network resources, wherein the optimizing network resources is used to indicate determining a routing strategy based on the demand and priority of network traffic data, and the routing strategy is used to reallocate network bandwidth and routing paths for the network traffic data; the abnormality elimination method corresponding to the network security anomaly is: starting and strengthening the network firewall; the abnormality elimination method corresponding to the network configuration error is: updating and optimizing the network configuration, wherein the network configuration includes at least one of the following: IP address, subnet mask, gateway address, and domain name system DNS server in the network.

[0013] According to another aspect of an embodiment of the present invention, a network traffic processing device is also provided, including: a receiving module for receiving network traffic data forwarded by an intelligent gateway device; a feature extraction module for performing feature extraction processing on the network traffic data to obtain network traffic features; a first determination module for determining, based on the network traffic features, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein N is an integer greater than or equal to 1; a second determination module for determining an abnormality handling strategy for the network traffic data based on the abnormal event scores corresponding to the N abnormal events, wherein the abnormality handling strategy includes an abnormality exclusion order for the N abnormal events and an abnormality exclusion method corresponding to the N abnormal events; a sending module for sending the abnormality handling strategy to the intelligent gateway device, so that the intelligent gateway device performs abnormality exclusion on the network traffic data based on the abnormality handling strategy.

[0014] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executed by any one of the network traffic processing methods.

[0015] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the network traffic processing methods described. Beneficial effects

[0016] In an embodiment of the present invention, network traffic data forwarded by an intelligent gateway device is received; feature extraction processing is performed on the network traffic data to obtain network traffic features; based on the network traffic features, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events are determined, wherein N is an integer greater than or equal to 1; according to the abnormal event scores corresponding to the N abnormal events, an abnormality handling strategy for the network traffic data is determined, wherein the abnormality handling strategy includes an abnormality exclusion order of the N abnormal events and an abnormality exclusion method corresponding to the N abnormal events; the abnormality handling strategy is sent to the intelligent gateway device, so that the intelligent gateway device excludes abnormalities from the network traffic data based on the abnormality handling strategy, thereby achieving the purpose of accurately excluding abnormalities from the network traffic data in a specific order using the corresponding abnormality exclusion method by the intelligent gateway device, thereby achieving the technical effect of improving the efficiency and accuracy of network traffic abnormality exclusion, and thus solving the technical problem in the related art of relying on manual or preset rules and thresholds to detect and exclude abnormal events, which lacks flexibility and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] FIG1 is a schematic diagram of a network traffic processing method according to an embodiment of the present invention;

[0019] FIG2 is a schematic diagram of an optional network traffic processing method according to an embodiment of the present invention;

[0020] FIG3 is a schematic diagram of a network traffic processing device according to an embodiment of the present invention. Modes for Carrying Out the Invention

[0021] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] According to an embodiment of the present invention, an embodiment of a method for processing network traffic is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] FIG1 is a flow chart of a method for processing network traffic according to an embodiment of the present invention. As shown in FIG1 , the method includes the following steps:

[0025] Step S102: Receive network traffic data forwarded by the intelligent gateway device.

[0026] Optionally, smart gateway devices can be installed at the network edge to serve as the entry and exit points for network traffic data. These devices perform data collection, data transmission, data processing, and data control, enabling bidirectional communication with cloud-based data centers. These devices forward network traffic data from user terminals to the cloud-based data centers.

[0027] Step S104: performing feature extraction processing on the network traffic data to obtain network traffic features.

[0028] In an optional embodiment, feature extraction processing is performed on network traffic data to obtain network traffic features, including: data cleaning processing is performed on the network traffic data to obtain network traffic features, wherein the network traffic features include at least: source address, destination address, port number, protocol type, data packet size, data packet arrival time interval, and network traffic transmission direction.

[0029] Optionally, features such as source address, destination address, port number, protocol type, packet size, packet arrival time interval, and network traffic transmission direction are important indicators for measuring whether network traffic data is abnormal. The above features are extracted from network traffic data through data cleaning, and the extracted features are used as network traffic features for identifying abnormal events in subsequent network traffic data, thereby improving the accuracy and comprehensiveness of identifying abnormalities in network traffic data.

[0030] Step S106: Based on the network traffic characteristics, determine N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, where N is an integer greater than or equal to 1.

[0031] Optionally, model prediction can be used to identify abnormal events in network traffic data and assign a score to each identified abnormal event. This score indicates the degree of abnormality (i.e., the severity of the problem) of the corresponding abnormal event. The score is proportional to the degree of abnormality. A higher degree of abnormality indicates a more abnormal event and a more serious problem; a lower degree of abnormality indicates a less serious problem.

[0032] In an optional embodiment, based on network traffic characteristics, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events are determined, including: based on network traffic characteristics, using a network traffic anomaly detection model to determine N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein the network traffic anomaly detection model is based on a historical traffic data set and obtained through machine learning, and the historical traffic data set includes multiple historical traffic data, and abnormal events and abnormal time scores corresponding to the multiple historical traffic data.

[0033] Optionally, network traffic features such as source address, destination address, port number, protocol type, packet size, packet arrival time interval, and network traffic transmission direction are used as model inputs. A pre-trained network traffic anomaly detection model is used to detect abnormal events in network traffic data, identify abnormal events in the network traffic data, and score values ​​for the abnormal events. Through the above approach, the efficiency and accuracy of identifying abnormalities in network traffic data can be improved. Furthermore, the network traffic anomaly recognition model can not only identify abnormal events in network traffic data, but also obtain scores for the abnormal events. Based on the abnormal event scores, the severity of each abnormal event can be determined, providing a reference for determining the subsequent processing priority of each abnormal event.

[0034] Optionally, during the training of a network traffic anomaly detection model, multiple historical traffic data items are first cleaned to extract useful features, such as source address, destination address, protocol type, packet size, and timestamp, to obtain multiple historical traffic features. These multiple historical traffic features are then normalized to reduce the dimensionality and complexity of the data. Next, the normalized multiple traffic features, along with the abnormal events and abnormal event scores corresponding to these normalized multiple traffic features, are divided into training and test sets according to a certain ratio. A deep learning network model is further designed, comprising an input layer, a hidden layer, and an output layer. The input layer receives the normalized multiple traffic features, and the output layer outputs abnormal events and abnormal event scores, which indicate the degree of abnormality of the data. The hidden layer uses ELU as an activation function to retain more information and improve model stability. Dropout is selected as a regularization method to increase model diversity and generalization, accelerate model training, and improve model performance. During the model training phase, the deep neural network model is trained using the backpropagation algorithm and gradient descent method based on the data included in the training set. The network parameters are optimized to ensure that the anomaly score of the output layer is as close to 0 (indicating normality) as possible. Cross-validation and early stopping are used during training to prevent overfitting or underfitting of the model. After the model is trained on the training set data, the trained deep neural network is evaluated using the F1 score using the data included in the test set. Based on the evaluation results, the deep neural network hyperparameter configuration and the number of iterations are continuously adjusted to ultimately improve the model's performance and obtain a network traffic anomaly detection model.

[0035] In an optional embodiment, based on network traffic characteristics, a network traffic anomaly detection model is used to determine N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, including: based on network traffic characteristics, a network traffic anomaly detection model is used to determine the abnormal event scores corresponding to M abnormal events, where M is an integer greater than or equal to 2; judging whether there is an abnormal event with an abnormal event score greater than a preset first scoring threshold among the M abnormal events; if there is an abnormal event with an abnormal event score greater than the preset first scoring threshold among the M abnormal events, the abnormal event with an abnormal event score greater than the preset first scoring threshold is regarded as N abnormal events; and determining the abnormal event scores corresponding to the N abnormal events from the abnormal event scores corresponding to the M abnormal events.

[0036] Optionally, the types of abnormal events that can be output by the network traffic anomaly detection model are fixed. For example, in the model training phase, the model uses the abnormal event score values ​​corresponding to M abnormal events in the historical traffic data as the output for model training. In actual applications, the abnormal event scores corresponding to the M abnormal events included in the network traffic data can be predicted; further, the abnormalities are screened based on the abnormal event scores corresponding to the M abnormal events, and the abnormal events with higher abnormal event scores and higher than the preset first score threshold are screened out from the M abnormal events for abnormal exclusion. In this way, abnormal events with more serious problems can be accurately screened out, and such abnormal events can be promptly excluded.

[0037] In an optional embodiment, the method further includes: determining the weight values ​​corresponding to the M abnormal events respectively when there is no abnormal event with an abnormal event score greater than a preset first score threshold among the M abnormal events; determining a comprehensive score value based on the weight values ​​corresponding to the M abnormal events respectively and the abnormal event scores corresponding to the M abnormal events respectively; and issuing an alarm indication when the comprehensive score value is greater than a preset second score threshold.

[0038] Optionally, if there is no abnormal event with an abnormal event score greater than a preset first scoring threshold among the M abnormal events, it indicates that the problem of the current abnormal event detected by the abnormal event scoring is relatively minor. In this case, it is necessary to further perform a weighted calculation based on each abnormal event and the corresponding weight value to obtain a comprehensive score value for the network traffic data. If the comprehensive score value is high, it indicates that abnormality elimination is also required. In this case, an alarm indication is issued and abnormality elimination is carried out through manual intervention. In this way, comprehensive elimination of abnormal events is achieved.

[0039] In an optional embodiment, after determining the abnormal event scores corresponding to M abnormal events respectively using a network traffic anomaly detection model based on network traffic characteristics, the method further includes: storing the network traffic data and the abnormal event scores corresponding to the M abnormal events respectively into a historical traffic data set to obtain a new historical traffic data set; and optimizing and updating the network traffic anomaly detection model based on the new historical traffic data set.

[0040] Optionally, network traffic data is collected in real time through intelligent gateway devices, and the network traffic data is manually or semi-automatically labeled based on the prediction results and actual conditions (the annotation content is the abnormal events included in the network traffic data, and the corresponding abnormal event scores), which is used to optimize the existing network traffic anomaly detection model to improve the model accuracy and prediction performance.

[0041] Step S108, determining an abnormality handling strategy for the network traffic data based on the abnormal event scores corresponding to the N abnormal events;

[0042] Step S110: Send the exception handling strategy to the intelligent gateway device, so that the intelligent gateway device can eliminate exceptions from the network traffic data based on the exception handling strategy.

[0043] Optionally, the exception handling strategy includes an exception elimination order for N exception events, and exception elimination methods corresponding to the N exception events.

[0044] In an optional embodiment, the N abnormal events include at least one of the following: port scanning, distributed denial of service, network congestion, network security anomaly, network configuration error, specifically:

[0045] The abnormality elimination method corresponding to port scanning is: isolating abnormal traffic, where isolating abnormal traffic is used to indicate that data packets with abnormal source or destination addresses are discarded;

[0046] The corresponding exception elimination method for distributed denial of service is: redirecting normal traffic, where redirecting normal traffic is used to indicate that data packets with normal traffic source or destination addresses are forwarded to the backup network;

[0047] The method for eliminating network congestion anomalies is to optimize network resources. Optimizing network resources is used to determine routing strategies based on the needs and priorities of network traffic data. Routing strategies are used to reallocate network bandwidth and routing paths for network traffic data.

[0048] The corresponding exception elimination methods for network security anomalies are: starting and strengthening the network firewall;

[0049] The method for eliminating the abnormality corresponding to the network configuration error is: updating and optimizing the network configuration, wherein the network configuration includes at least one of the following: IP address, subnet mask, gateway address, and domain name system DNS server in the network.

[0050] Optionally, when the output of the network traffic anomaly detection model indicates a port scan, the corresponding adjustment and repair method is to isolate the abnormal traffic, namely, discarding packets with abnormal source or destination addresses to prevent port scans from posing a security threat to the network. When the output of the network traffic anomaly detection model indicates a distributed denial of service (DDos) attack, the corresponding adjustment and repair method is to redirect normal traffic, namely, forwarding packets with normal source or destination addresses to a backup network to ensure the quality of service for normal traffic. When the output of the network traffic anomaly detection model indicates network congestion, the corresponding adjustment and repair method is to optimize network resources, namely, dynamically allocate network bandwidth and routes based on the needs and priority of network traffic data to improve network efficiency and performance. When the output of the network traffic anomaly detection model indicates a network security anomaly, the corresponding adjustment and repair method is to activate and strengthen the network firewall, namely, use intelligent gateway devices to filter and intercept inbound and outbound traffic in the network, and strengthen it according to network security policies and rules to prevent the network from being attacked or infected by viruses. When the abnormal event in the output results of the network traffic anomaly detection model is a network configuration error, the corresponding adjustment and repair method is to update and optimize the network configuration, that is, through the intelligent gateway device, check and modify the IP address, subnet mask, gateway address, DNS server and other configurations in the network to improve network availability and security.

[0051] Through the above steps S102 to S110, the purpose of accurately eliminating anomalies in network traffic data can be achieved by using the intelligent gateway device in a specific order and using the corresponding anomaly elimination method, thereby achieving the technical effect of improving the efficiency and accuracy of network traffic anomaly elimination, and then solving the technical problem of relying on manual or preset rules and thresholds to detect and eliminate abnormal events in related technologies, which lacks flexibility and accuracy.

[0052] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation manner. FIG2 is a flowchart of an optional network traffic processing method according to an embodiment of the present invention. As shown in FIG2 , the method includes:

[0053] Step S1: training of network traffic anomaly detection model.

[0054] Step S11: Data preprocessing. Multiple historical traffic data items are cleaned to extract useful features, such as source address, destination address, protocol type, packet size, and timestamp, to obtain multiple historical traffic features. These features are normalized to reduce the data's dimensionality and complexity.

[0055] Step S12, data division: The normalized multiple traffic features, the abnormal events corresponding to the normalized multiple traffic features, and the abnormal event scores are divided into a training set and a test set according to a certain ratio.

[0056] Step S13: Model Construction. Design a deep learning network model, including an input layer, hidden layers, and an output layer. The input layer receives multiple normalized traffic features, and the output layer outputs abnormal events and an abnormality score, which indicates the degree of abnormality in the data. The hidden layer uses ELU as the activation function to retain more information and improve model stability. Dropout is selected as the regularization method to increase model diversity and generalization ability, accelerate model training, and improve model performance.

[0057] Step S14: Model training. Based on the data in the training set, the deep neural network model is trained using the backpropagation algorithm and gradient descent. The network parameters are optimized to ensure that the anomaly score of the output layer is as close to 0 (indicating normality) as possible. Cross-validation and early stopping are used during training to prevent overfitting or underfitting of the model.

[0058] Step S15: Model evaluation. Using the data in the test set, the trained deep neural network is evaluated using the F1 score. Based on the evaluation, the hyperparameter configuration and number of iterations of the deep neural network are continuously tuned to ultimately improve the model's performance and obtain a network traffic anomaly detection model.

[0059] Step S2: Deploy smart gateway devices. Install smart gateway devices at the edge of the network, serving as the entry and exit points for network traffic data. These devices have data collection, data transmission, data processing, and data control functions, enabling bidirectional communication with cloud-based data centers.

[0060] Step S3: Upload network traffic data. The intelligent gateway device uploads the user's requested network traffic data (such as source address, destination address, protocol type, packet size, timestamp, etc.) to the cloud data center through the network protocol. The data center has large-scale computing resources and storage space, which can efficiently store and process network traffic data.

[0061] Step S4: Analyze and evaluate network traffic and implement corresponding adjustment strategies. The data center analyzes and evaluates real-time network traffic data based on the established network traffic anomaly detection model, identifies anomalies, and implements corresponding anomaly execution strategies, including:

[0062] Step S41: Data preprocessing. The real-time network traffic data is cleaned and useful features, such as source address, destination address, protocol type, packet size, and timestamp, are extracted as network traffic features. The acquired network traffic features are normalized to reduce their dimensionality and complexity.

[0063] Step S42: Input the normalized network traffic features into the network traffic anomaly detection model to predict the abnormal events included in the network traffic features and the abnormal event scores corresponding to the abnormal events.

[0064] 1) When the output of the network traffic anomaly detection model is a port scan, the corresponding adjustment and repair method is to isolate the abnormal traffic, that is, to discard data packets with abnormal source or destination addresses to prevent port scanning from posing a security threat to the network.

[0065] 2) When the abnormal event in the output result of the network traffic anomaly detection model is a distributed denial of service (DDos attack), the corresponding adjustment and repair method is to redirect normal traffic, that is, forwarding data packets with normal source or destination addresses to the backup network to ensure the service quality of normal traffic.

[0066] 3) When the abnormal event in the output result of the network traffic anomaly detection model is network congestion, the corresponding adjustment and repair method is to optimize network resources, that is, dynamically allocate network bandwidth and routing according to the needs and priority of network traffic data to improve network efficiency and performance.

[0067] 4) When the abnormal event in the output result of the network traffic anomaly detection model is a network security anomaly, the corresponding adjustment and repair method is to start and strengthen the network firewall, that is, through the intelligent gateway device, filter and intercept the inbound and outbound traffic in the network, and strengthen it according to the network security policies and rules to prevent the network from being attacked abnormally or infected by viruses.

[0068] 5) When the abnormal event in the output results of the network traffic anomaly detection model is a network configuration error, the corresponding adjustment and repair method is to update and optimize the network configuration, that is, through the intelligent gateway device, check and modify the IP address, subnet mask, gateway address, DNS server and other configurations in the network to improve network availability and security.

[0069] Step S5: Asynchronously update the network traffic anomaly detection model. Real-time network traffic data is collected through the intelligent gateway device. Based on the prediction results and actual conditions, the network traffic data is manually or semi-automatically annotated (with the anomaly events included in the network traffic data and the corresponding anomaly event scores) to optimize the existing network traffic anomaly detection model.

[0070] Through the above approach, at least one of the following effects can be achieved: 1) Real-time detection, analysis, diagnosis, and repair of network traffic data can be achieved, improving the automation and intelligence level of the network. 2) Gateway configuration and policies can be dynamically adjusted based on network changes and needs, improving the network's adaptability and self-optimization capabilities. 3) Cloud computing technology can be utilized to provide powerful computing resources and storage space, reducing the cost and complexity of network management.

[0071] This embodiment also provides a network traffic processing device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the terms "module" and "device" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0072] According to an embodiment of the present invention, an embodiment of a device for implementing the above-mentioned network traffic processing method is also provided. FIG3 is a schematic structural diagram of a network traffic processing device according to an embodiment of the present invention. As shown in FIG3 , the above-mentioned network traffic processing device includes: a receiving module 300, a feature extraction module 302, a first determination module 304, a second determination module 306, and a sending module 308, wherein:

[0073] Receiving module 300, used to receive network traffic data forwarded by the intelligent gateway device;

[0074] The feature extraction module 302 is connected to the receiving module 300 and is used to perform feature extraction processing on the network traffic data to obtain network traffic features;

[0075] A first determination module 304, connected to the feature extraction module 302, is configured to determine N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events based on the network traffic features, where N is an integer greater than or equal to 1;

[0076] A second determining module 306 is configured to determine an anomaly handling strategy for the network traffic data based on the anomaly event scores corresponding to the N anomaly events, wherein the anomaly handling strategy includes an anomaly elimination order for the N anomaly events and an anomaly elimination method corresponding to the N anomaly events;

[0077] The sending module 308 is connected to the second determining module 306 and is used to send the exception handling strategy to the intelligent gateway device, so that the intelligent gateway device can eliminate exceptions from the network traffic data based on the exception handling strategy.

[0078] In an embodiment of the present invention, a receiving module 300 is provided for receiving network traffic data forwarded by an intelligent gateway device; a feature extraction module 302 is connected to the receiving module 300 and is used to perform feature extraction processing on the network traffic data to obtain network traffic features; a first determination module 304 is connected to the feature extraction module 302 and is used to determine, based on the network traffic features, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein N is an integer greater than or equal to 1; a second determination module 306 is used to determine an abnormality handling strategy for the network traffic data according to the abnormality event scores corresponding to the N abnormal events, wherein the abnormality handling strategy The invention briefly includes the order of exception elimination of N abnormal events, and the exception elimination methods corresponding to the N abnormal events; the sending module 308 is connected to the second determination module 306, and is used to send the exception handling strategy to the intelligent gateway device, so that the intelligent gateway device can eliminate the exceptions of the network traffic data based on the exception handling strategy, thereby achieving the purpose of accurately eliminating the exceptions of the network traffic data in a specific order by the intelligent gateway device using the corresponding exception elimination method, thereby achieving the technical effect of improving the efficiency and accuracy of network traffic exception elimination, and thus solving the technical problem of relying on manual or preset rules and thresholds to detect and eliminate abnormal events in related technologies, and lacking flexibility and accuracy.

[0079] In an optional embodiment, the second determination module includes: a first determination submodule, which is used to determine N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events based on network traffic characteristics and using a network traffic anomaly detection model, wherein the network traffic anomaly detection model is based on a historical traffic data set and obtained through machine learning, and the historical traffic data set includes multiple historical traffic data, as well as abnormal events and abnormal time scores corresponding to the multiple historical traffic data.

[0080] In an optional embodiment, the first determination submodule includes: a second determination submodule, used to determine the abnormal event scores corresponding to M abnormal events based on network traffic characteristics and using a network traffic anomaly detection model, where M is an integer greater than or equal to 2; a first judgment submodule, used to determine whether there is an abnormal event with an abnormal event score greater than a preset first scoring threshold among the M abnormal events; a first acquisition submodule, used to, if there is an abnormal event with an abnormal event score greater than the preset first scoring threshold among the M abnormal events, treat the abnormal event with an abnormal event score greater than the preset first scoring threshold as N abnormal events; and a third determination submodule, used to determine the abnormal event scores corresponding to the N abnormal events from the abnormal event scores corresponding to the M abnormal events.

[0081] In an optional embodiment, the device also includes: a fourth determination submodule, used to determine the weight values ​​corresponding to the M abnormal events when there is no abnormal event with an abnormal event score greater than a preset first score threshold among the M abnormal events; a fifth determination submodule, used to determine a comprehensive score value based on the weight values ​​corresponding to the M abnormal events and the abnormal event scores corresponding to the M abnormal events; and an alarm submodule, used to issue an alarm indication when the comprehensive score value is greater than a preset second score threshold.

[0082] In an optional embodiment, the device also includes: a storage submodule, which is used to store network traffic data and the abnormal event scores corresponding to M abnormal events into a historical traffic data set to obtain a new historical traffic data set; and an update submodule, which is used to optimize and update the network traffic anomaly detection model based on the new historical traffic data set.

[0083] In an optional embodiment, the feature extraction module includes: a data cleaning sub-module, which is used to perform data cleaning processing on network traffic data to obtain network traffic features, wherein the network traffic features include at least: source address, destination address, port number, protocol type, data packet size, data packet arrival time interval, and network traffic transmission direction.

[0084] In an optional embodiment, the N abnormal events include at least one of the following: port scanning, distributed denial of service, network congestion, network security anomaly, and network configuration error, wherein the abnormality elimination method corresponding to port scanning is: isolating abnormal traffic, wherein isolating abnormal traffic is used to indicate that data packets with source or destination addresses as abnormal traffic are discarded; the abnormality elimination method corresponding to distributed denial of service is: redirecting normal traffic, wherein redirecting normal traffic is used to indicate that data packets with source or destination addresses as normal traffic are forwarded to a backup network; the abnormality elimination method corresponding to network congestion is: optimizing network resources, wherein optimizing network resources is used to indicate determining routing strategies based on the needs and priorities of network traffic data, and routing strategies are used to reallocate network bandwidth and routing paths for network traffic data; the abnormality elimination method corresponding to network security anomaly is: starting and strengthening the network firewall; the abnormality elimination method corresponding to network configuration error is: updating and optimizing the network configuration, wherein the network configuration includes at least one of the following: IP address, subnet mask, gateway address, and domain name system DNS server in the network.

[0085] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0086] It should be noted that the receiving module 300, feature extraction module 302, first determination module 304, second determination module 306, and sending module 308 correspond to steps S102 to S110 in the embodiment. The examples and application scenarios implemented by these modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules, as part of the device, can be run on a computer terminal.

[0087] It should be noted that the optional or preferred implementation of this embodiment can be found in the relevant description in the embodiment, which will not be repeated here.

[0088] The above-mentioned network traffic processing device may also include a processor and a memory. The above-mentioned receiving module 300, feature extraction module 302, first determination module 304, second determination module 306, sending module 308, etc. are all stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to realize corresponding functions.

[0089] The processor includes a core, which retrieves corresponding program modules from memory. There can be one or more cores. Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip.

[0090] According to an embodiment of the present application, an embodiment of a non-volatile storage medium is further provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein when the program is executed, the device containing the non-volatile storage medium is controlled to execute any of the above-mentioned network traffic processing methods.

[0091] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group, and the non-volatile storage medium includes a stored program.

[0092] Optionally, when the program is running, the device where the non-volatile storage medium is located is controlled to perform the following functions: receiving network traffic data forwarded by the intelligent gateway device; performing feature extraction processing on the network traffic data to obtain network traffic features; based on the network traffic features, determining N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, where N is an integer greater than or equal to 1; determining an abnormality handling strategy for the network traffic data based on the abnormal event scores corresponding to the N abnormal events, where the abnormality handling strategy includes an abnormality exclusion order for the N abnormal events, and an abnormality exclusion method corresponding to the N abnormal events; sending the abnormality handling strategy to the intelligent gateway device, for the intelligent gateway device to perform abnormality exclusion on the network traffic data based on the abnormality handling strategy.

[0093] According to an embodiment of the present application, a processor embodiment is further provided. Optionally, in this embodiment, the processor is used to run a program, wherein the program executes any of the above-mentioned network traffic processing methods when running.

[0094] According to an embodiment of the present application, an embodiment of a computer program product is also provided, which, when executed on a data processing device, is suitable for executing a program that initializes any one of the steps of the above-mentioned network traffic processing method.

[0095] Optionally, the above-mentioned computer program product, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: receiving network traffic data forwarded by an intelligent gateway device; performing feature extraction processing on the network traffic data to obtain network traffic features; based on the network traffic features, determining N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, wherein N is an integer greater than or equal to 1; determining an abnormality handling strategy for the network traffic data based on the abnormal event scores corresponding to the N abnormal events, wherein the abnormality handling strategy includes an abnormality elimination order for the N abnormal events, and an abnormality elimination method corresponding to the N abnormal events; and sending the abnormality handling strategy to the intelligent gateway device, for the intelligent gateway device to perform abnormality elimination on the network traffic data based on the abnormality handling strategy.

[0096] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: receiving network traffic data forwarded by an intelligent gateway device; performing feature extraction processing on the network traffic data to obtain network traffic features; determining, based on the network traffic features, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein N is an integer greater than or equal to 1; determining an abnormality handling strategy for the network traffic data based on the abnormal event scores corresponding to the N abnormal events, wherein the abnormality handling strategy includes an abnormality elimination order for the N abnormal events and an abnormality elimination method corresponding to the N abnormal events; and sending the abnormality handling strategy to the intelligent gateway device, so that the intelligent gateway device eliminates abnormalities in the network traffic data based on the abnormality handling strategy.

[0097] The above sequence of the embodiments of the present invention is for description only and does not represent the superiority or inferiority of the embodiments.

[0098] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the above modules can be a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, modules or indirect coupling or communication connection of modules, which can be electrical or other forms.

[0100] The modules described above as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0101] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0102] If the above-mentioned integrated modules are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned non-volatile storage medium includes various media that can store program code, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard drives, magnetic disks, or optical disks.

[0103] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A network traffic processing method, characterized in that: include: Receive network traffic data forwarded by the intelligent gateway device; Performing feature extraction processing on the network traffic data to obtain network traffic features; Based on the network traffic characteristics, determine N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, where N is an integer greater than or equal to 1; Determine an exception handling strategy for the network traffic data according to the exception event scores respectively corresponding to the N exception events, wherein the exception handling strategy includes an exception elimination order for the N exception events and an exception elimination method respectively corresponding to the N exception events; The exception handling strategy is sent to the intelligent gateway device, so that the intelligent gateway device can eliminate exceptions from the network traffic data based on the exception handling strategy.

2. The method according to claim 1, characterized in that: The determining, based on the network traffic characteristics, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events respectively includes: Based on the network traffic characteristics, a network traffic anomaly detection model is used to determine N abnormal events included in the network traffic data, and abnormal event scores corresponding to the N abnormal events, wherein the network traffic anomaly detection model is based on a historical traffic data set and is obtained through machine learning. The historical traffic data set includes multiple historical traffic data, and the abnormal events and abnormal time scores corresponding to the multiple historical traffic data.

3. The method according to claim 2, characterized in that The method of using a network traffic anomaly detection model based on the network traffic characteristics to determine N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events respectively includes: Based on the network traffic characteristics, the network traffic anomaly detection model is used to determine the abnormal event scores corresponding to the M abnormal events, where M is an integer greater than or equal to 2; Determine whether there is an abnormal event among the M abnormal events whose abnormal event score is greater than a preset first score threshold; In the case where there is an abnormal event whose abnormal event score is greater than the preset first scoring threshold among the M abnormal events, the abnormal event whose abnormal event score is greater than the preset first scoring threshold is taken as the N abnormal events; The abnormal event scores respectively corresponding to the N abnormal events are determined from the abnormal event scores respectively corresponding to the M abnormal events.

4. The method according to claim 3, characterized in that The method further comprises: If, among the M abnormal events, there is no abnormal event whose abnormal event score is greater than the preset first score threshold, determine the weight values ​​corresponding to the M abnormal events respectively; Determine a comprehensive score value based on the weight values ​​corresponding to the M abnormal events and the abnormal event scores corresponding to the M abnormal events; When the comprehensive score value is greater than a preset second score threshold, an alarm indication is issued.

5. The method according to claim 3, characterized in that: After determining the abnormal event scores corresponding to the M abnormal events respectively by using the network traffic anomaly detection model based on the network traffic characteristics, the method further includes: The network traffic data and the abnormal event scores corresponding to the M abnormal events are stored in the historical traffic data set to obtain a new historical traffic data set; Based on the new historical traffic data set, the network traffic anomaly detection model is optimized and updated.

6. The method according to any one of claims 1 to 5, characterized in that The performing feature extraction processing on the network traffic data to obtain network traffic features includes: The network traffic data is cleaned to obtain the network traffic characteristics, wherein the network traffic characteristics at least include: source address, destination address, port number, protocol type, data packet size, data packet arrival time interval, and network traffic transmission direction.

7. The method according to any one of claims 1 to 5, characterized in that The N abnormal events include at least one of the following: port scanning, distributed denial of service, network congestion, network security anomaly, network configuration error, wherein, The abnormality elimination method corresponding to the port scan is: isolating abnormal traffic, wherein the isolating abnormal traffic is used to indicate that a data packet whose source address or destination address is abnormal traffic is discarded; The abnormality elimination method corresponding to the distributed denial of service is: redirecting normal traffic, wherein the redirecting normal traffic is used to indicate that a data packet with a source address or a destination address as normal traffic is forwarded to a backup network; The abnormality elimination method corresponding to the network congestion is: optimizing network resources, wherein the optimizing network resources is used to indicate determining a routing strategy according to the demand and priority of the network traffic data, and the routing strategy is used to reallocate network bandwidth and routing paths for the network traffic data; The abnormality elimination method corresponding to the network security abnormality is: starting and strengthening the network firewall; The abnormality elimination method corresponding to the network configuration error is: updating and optimizing the network configuration, wherein the network configuration includes at least one of the following: an IP address in the network, a subnet mask, a gateway address, and a domain name system DNS server.

8. A network traffic processing device, characterized in that: include: A receiving module, used for receiving network traffic data forwarded by the intelligent gateway device; A feature extraction module is used to perform feature extraction processing on the network traffic data to obtain network traffic features; A first determination module, configured to determine, based on the network traffic characteristics, N abnormal events included in the network traffic data and abnormal event scores corresponding to the N abnormal events, wherein N is an integer greater than or equal to 1; A second determination module is used to determine an exception handling strategy for the network traffic data according to the exception event scores respectively corresponding to the N abnormal events, wherein the exception handling strategy includes an exception elimination order of the N abnormal events and an exception elimination method respectively corresponding to the N abnormal events; A sending module is used to send the exception handling strategy to the intelligent gateway device, so that the intelligent gateway device can eliminate exceptions from the network traffic data based on the exception handling strategy.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the network traffic processing method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the network traffic processing method described in any one of claims 1 to 7.

Images