Firewall dynamic defense system based on big data stream processing

Abstract

The invention discloses a firewall dynamic defense method and system based on big data stream processing, and relates to the technical field of network security. The method comprises the steps of dividing traffic into low-risk traffic, medium-risk traffic and high-risk traffic according to a score threshold; extracting high-dimensional features from the high-risk traffic to generate a first high-value sample set; firewall rule matching judgment is executed on the low-risk traffic and the medium-risk traffic, if matching succeeds, a corresponding action instruction is executed, otherwise, action decision is made according to the structured traffic metadata and the risk score, and a second high-value sample set is generated; the problems of training delay and rule base stiffness in a big data scene are effectively solved through the cooperation of hierarchical traffic processing and intelligent decision, a first high-value sample set is associated with a second high-value sample set, a third high-value sample set is screened, the model training sample precision is improved, the model training effect is further improved, and the training efficiency is improved. Global model parameters are pruned and quantified to generate a lightweight model, and model deployment in a resource-constrained environment is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a firewall dynamic defense method and system, a storage medium and an electronic device. BACKGROUND

[0002] In recent years, the rapid development of network technology makes the attack and defense technology related to network security increasingly complex, and the means of attackers become more diversified. In this case, the traditional rule-based attack detection method faces challenges in rule maintenance and countering new network attack technologies. In order to more effectively strengthen the defense against network attacks, it is imperative to use machine learning technology for network attack detection.

[0003] The existing firewall technology combined with machine learning has the problem of high response delay in real-time analysis in the face of massive traffic in a big data scenario. Especially in the face of new attacks, the static rule library is difficult to quickly adapt to the changes in threats, resulting in high false negative or false positive rates.

[0004] Therefore, it is urgent to solve the problems of training delay in a big data scenario and the difficulty of static rule library to adapt. SUMMARY

[0005] The purpose of the present application is to provide a firewall dynamic defense method and system, a storage medium and an electronic device to solve the problems raised in the background.

[0006] In a first aspect, an embodiment of the present application provides a firewall dynamic defense method, which comprises: extracting session data packets from original network traffic to generate structured traffic metadata; calculating a risk score based on the structured traffic metadata, and dividing the traffic into low-risk traffic, medium-risk traffic and high-risk traffic according to the score threshold; extracting high-dimensional features from the high-risk traffic to generate a first high-value sample set; performing firewall rule matching judgment on the low-risk traffic and the medium-risk traffic, if the matching is successful, executing the corresponding action instruction, otherwise, through the reinforcement learning model, making an action decision according to the structured traffic metadata and the risk score, and generating an action instruction; executing the action instruction and calculating the immediate reward to generate a second high-value sample set; associating the first high-value sample set with the second high-value sample set to generate a third high-value sample set; updating the global model parameters according to the third high-value sample set, and generating a lightweight model after pruning and quantization processing.

[0007] In combination with the first aspect, in some implementations of the first aspect, the first high-value sample set is generated, including: associating the high-dimensional features with the structured traffic metadata to generate a high-dimensional feature vector; for the high-dimensional feature vector, if the SimHash fingerprint matches the malicious signature library, and / or the TLS handshake behavior entropy is less than a set threshold, the sample is added to the first high-value sample set.

[0008] In some implementations of the first aspect, the high-risk traffic is extracted with high-dimensional features, including: performing fragmentation processing on a data packet payload of the high-risk traffic, and calculating hash values of the fragments; performing weighted accumulation on the hash values to generate a SimHash fingerprint; extracting a cipher suite list from a TLS handshake packet to calculate a TLS handshake behavior entropy; wherein the high-dimensional features are associated with structured traffic metadata to generate a high-dimensional feature vector, and the structured traffic metadata includes a five-tuple and basic features, including: associating the SimHash fingerprint and the TLS handshake behavior entropy with the five-tuple and the basic features to generate the high-dimensional feature vector.

[0009] In some implementations of the first aspect, the low-risk traffic and the medium-risk traffic are subjected to firewall rule matching judgment, and if the matching is successful, corresponding action instructions are executed, otherwise, action decisions are made based on the structured traffic metadata and the risk score through a reinforcement learning model to generate action instructions, including: traversing a preset firewall rule library, matching the structured traffic metadata with rule conditions, if the structured traffic metadata matches the firewall rule, executing the action instruction corresponding to the rule, and recording a rule execution log; wherein the rule execution log includes a rule ID, a traffic five-tuple, an action, and a timestamp; if the structured traffic metadata does not match the firewall rule, inputting the structured traffic metadata and the risk score into the reinforcement learning model, and the reinforcement learning model is used to output action probabilities of allowing, blocking, and isolating, and selecting the action with the highest probability as the final action instruction.

[0010] In some implementations of the first aspect, the first high-value sample set and the second high-value sample set are associated to generate a third high-value sample set, including: matching and associating the high-dimensional feature vectors in the first high-value sample set with the decision triplets in the second high-value sample set based on session identifiers to generate a plurality of complete samples; storing the plurality of complete samples to a global experience pool, and calculating a priority of each complete sample, the priority being determined based on a time difference error and a preset offset; based on a sorting result of the priority of each complete sample, selecting a preset proportion of the complete samples as the third high-value sample set.

[0011] In some implementations of the first aspect, the action instructions are executed and an instant reward is calculated to generate a second high-value sample set, including: executing a corresponding network traffic control operation according to the action instructions, and calculating an instant reward value based on an execution result; the network traffic control operation includes: an allowing operation of releasing a data stream and recording a session log, a blocking operation of discarding a data packet and returning a connection rejection response, and an isolation operation of redirecting traffic to a sandbox detection environment; storing decision triplet data containing state features, executed actions, and instant rewards into a local experience pool; and selecting decision data groups satisfying a reward threshold condition from the local experience pool to form the second high-value sample set.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the global model parameters are updated according to the third highest value sample set, and a lightweight model is generated after pruning and quantization processing, including: batch processing and randomization processing of the third highest value sample set, and based on the samples after batch processing and randomization processing, distributed model training is performed using a parameter server architecture to update the global model parameters; neuron pruning processing is performed on the updated global model parameters to remove weights and their connecting nodes whose absolute values ​​are less than a preset threshold; the updated model parameters are converted into an 8-bit integer representation through linear quantization and a quantization table is generated, in which the sign bit occupies 1 bit and the value bit occupies 7 bits, and finally a lightweight model is generated.

[0013] In the second aspect, an embodiment of the present application provides a firewall dynamic defense system for implementing the firewall dynamic defense method mentioned in the first aspect, the system including: a first generation module for extracting session data packets from original network traffic and generating structured traffic metadata; a classification module for calculating risk scores based on structured traffic metadata, and dividing traffic into low-risk traffic, medium-risk traffic, and high-risk traffic according to a score threshold; a second generation module for extracting high-dimensional features for high-risk traffic and generating a first high-value sample set; a judgment module for performing firewall rule matching judgment on low-risk traffic and medium-risk traffic. If the match is successful, the corresponding action instruction is executed; otherwise, an action decision is made based on the structured traffic metadata and risk score through a reinforcement learning model to generate an action instruction; a third generation module for executing the action instruction and calculating the immediate reward to generate a second high-value sample set; a fourth generation module for associating the first high-value sample set with the second high-value sample set to generate a third high-value sample set; a fifth generation module for updating the global model parameters according to the third high-value sample set, and generating a lightweight model after pruning and quantization processing.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a processor; a memory for storing processor-executable instructions; and the processor is used to execute the firewall dynamic defense method mentioned in the first aspect above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is used to execute the firewall dynamic defense method mentioned in the first aspect above.

[0016] The firewall dynamic defense method provided by the embodiment of the application extracts basic features through lightweight calculation, extracts high-dimensional features from high-risk traffic, and performs hierarchical feature filtering, thereby reducing the load of the main CPU; the training delay and rule base rigidity problem in a big data scenario are effectively solved through hierarchical traffic processing and intelligent decision-making cooperation, the first high-value sample set and the second high-value sample set are associated, complete samples are generated, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved; the edge decision-making module is used for reasoning and decision-making, the global model parameters are updated, the global model parameters are quantized after pruning, and a lightweight model is generated, thereby solving the model deployment in a resource-constrained environment. BRIEF DESCRIPTION OF DRAWINGS

[0017] FIG. 1 Fig. 1 shows a flowchart of a firewall dynamic defense method provided by an embodiment of the application.

[0018] FIG. 2 Fig. 2 shows a flowchart of a firewall dynamic defense method provided by another exemplary embodiment of the application.

[0019] FIG. 3 Fig. 3 shows a flowchart of a firewall dynamic defense method provided by still another exemplary embodiment of the application.

[0020] FIG. 4 Fig. 4 shows a flowchart of a firewall dynamic defense method provided by still another exemplary embodiment of the application.

[0021] FIG. 5 Fig. 5 shows a flowchart of a firewall dynamic defense method provided by still another exemplary embodiment of the application.

[0022] FIG. 6 Fig. 6 shows a flowchart of a firewall dynamic defense method provided by still another exemplary embodiment of the application.

[0023] FIG. 7 Fig. 7 shows a structural diagram of a firewall dynamic defense system provided by an exemplary embodiment of the application.

[0024] FIG. 8 Fig. 8 shows a structural diagram of an electronic device provided by an exemplary embodiment of the application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0026] Exemplary method

[0027] FIG. 1 Fig. 1 shows a flowchart of a firewall dynamic defense method according to an embodiment of the present application. As shown in Fig. 1, the firewall dynamic defense method according to the embodiment of the present application comprises the following steps. FIG. 1

[0028] Step 100, extracting session data packets from original network traffic to generate structured traffic metadata.

[0029] Exemplarily, the structured traffic metadata comprises five-tuple and basic features.

[0030] Exemplarily, the five-tuple comprises source IP, destination IP, source port, destination port and protocol.

[0031] Exemplarily, the basic features refer to three types of structured metadata calculated from the network traffic, including packet rate, byte rate and session duration.

[0032] Exemplarily, the original network data packet comprises five-tuple, TCP / UDP header, TCP session state and data packet payload.

[0033] Specifically, first, bind the network card to the DataPlane Development Kit (DPDK) driver, configure the memory pool, poll the network card queue every 10 microseconds, and read the original network data packet from the original network traffic. If the original network data packet size is equal to 0, or the IP checksum is incorrect, or the TCP / UDP port number = 0, it is defined as an invalid packet, and all invalid packets are discarded. Then, according to the five-tuple and TCP session state, the data packet is grouped into different session data packets, to obtain clean session data packets and store them in a ring buffer. Set the sliding window size to 500 ms and the sliding step to 100 ms. For the clean session data packets, calculate three types of basic features: packet rate Packet_Rate = number of data packets of the session in the window / window size; byte rate Byte_Rate = total number of bytes of the session in the window / window size; session duration Session_Duration = current window end time-session establishment time. Among them, the packet rate is the number of data packets of the session per unit time, reflecting the traffic density; the byte rate is the total number of bytes of the session per unit time, reflecting the traffic size; the session duration is the time from the establishment of the session to the end of the current window, reflecting the session length; and the three types of basic features are associated with the five-tuple information, thereby generating structured traffic metadata.

[0034] ​Step 101 : Calculate risk scores based on structured traffic metadata, and classify traffic into low-risk traffic, medium-risk traffic, and high-risk traffic according to score thresholds.

[0035] It should be understood that based on structured traffic metadata, traffic stratification is performed by calculating risk scores in real time: low-risk traffic is sent to the edge decision module, medium-risk traffic is stored in the local cache, and high-risk traffic is sent to the Kafka queue module.

[0036] Specifically, it receives structured traffic metadata and obtains the IP blacklist ip_blacklist, IP whitelist ip_whitelist and port risk level port_risk from the external intelligence platform; among them, ip_blacklist, ip_whitelist and port_risk∈[0,1].

[0037] For each session packet's source IP address, the corresponding ip_blacklist and ip_whitelist are queried in the ring buffer. For each session packet's destination port, the corresponding port_risk is queried in the ring buffer. The ip_blacklist, ip_whitelist, and port_risk are then associated with the session packet's structured traffic metadata to generate threat intelligence-tagged structured traffic metadata. This threat intelligence-tagged structured traffic metadata includes a five-tuple, basic features, ip_blacklist, ip_whitelist, and port_risk.

[0038] For structured traffic metadata with threat intelligence labels, the packet rate and byte rate in the basic features are converted into normalized values ​​between 0 and 1: the normalized packet rate is calculated according to formula (1), and the normalized byte rate is calculated according to formula (2). Formulas (1) and (2) are as follows:

[0039] N_Packet_Rate=min(Packet_Rate / α, 1.0) (1)

[0040] N_Byte_Rate=min(Byte_Rate / β, 1.0) (2)

[0041] Among them, α is the upper threshold of the packet rate, which is obtained based on historical data statistics; β is the upper threshold of the byte rate, which can be obtained based on historical data statistics.

[0042] The risk score is calculated according to formula (3), which is as follows:

[0043] risk_score=w1×N_Packet_Rate+w2×N_Byte_Rate+w3×

[0044] (ip_blacklist + ip_whitelist) + w4 x port_risk(3)

[0045] wherein the weight coefficients w1, w2, w3, and w4 are set by security experts according to the risk influence degree, and w1 + w2 + w3 + w4 = 1.

[0046] The risk score risk_score is associated with the structured traffic metadata with threat intelligence labels to obtain structured traffic metadata with a risk score. If risk_score ∈ [0, 0.3), it belongs to low-risk traffic; if risk_score ∈ [0.3, 0.8), it belongs to medium-risk traffic; if risk_score ∈ [0.8, ∞), it belongs to high-risk traffic.

[0047] Step 102, extracting high-dimensional features from high-risk traffic to generate a first high-value sample set.

[0048] Specifically, for high-risk traffic, high-dimensional features are extracted, associated with five-tuples and basic features to generate high-dimensional feature vectors, which are sent to the central experience pool. The central experience pool selects a first high-value sample set and sends it to the central Critic module to train the reinforcement learning model.

[0049] Step 103, determining whether the low-risk traffic and the medium-risk traffic match the firewall rules. If yes, step 104 is performed, and if no, step 105 is performed.

[0050] Step 104, executing the corresponding action instruction.

[0051] Step 105, performing action decision according to the structured traffic metadata and the risk score through the reinforcement learning model to generate an action instruction.

[0052] Exemplarily, the action instruction includes allowing, blocking, and isolating.

[0053] Specifically, for low-risk traffic and medium-risk traffic, if the firewall preset rules are matched, the action instruction of the preset rules is executed; if the firewall preset rules are not matched, the reinforcement learning model is used to make action decisions according to the structured traffic metadata and the risk score, and the action instruction obtained by the decision is sent to the real-time inference module.

[0054] Step 106, executing the action instruction and calculating the immediate reward to generate a second high-value sample set.

[0055] Specifically, based on the action instruction derived from the decision, the real-time inference module executes the action instruction and calculates the immediate reward, and sends the decision triple to the local experience pool, screens the second high-value sample set, and sends it to the central Critic module. The central Critic module updates the model parameters of the reinforcement learning model, and synchronizes the updated reinforcement learning model to the edge decision module. The second high-value sample set is a sample whose result of meeting the action instruction is false blocking or correct blocking.

[0056] Illustratively, the decision triple includes a 6-dimensional feature vector, an action instruction, and an immediate reward.

[0057] Step 107, associate the first high-value sample set with the second high-value sample set to generate a third high-value sample set.

[0058] Specifically, the first high-value sample set and the second high-value sample set are associated to generate a complete sample, which is sent to the global experience pool, the priority of each complete sample is calculated, the third high-value sample set is screened, and sent to the central Critic module.

[0059] Step 108, update the global model parameters according to the third high-value sample set, and generate a lightweight model after pruning and quantization processing.

[0060] Specifically, according to the third high-value sample set, the global model parameters are calculated; the loss function and the gradient are calculated, the preset rule is fused, the global model parameters are updated; the global model parameters are pruned and quantized to obtain lightweight model parameters, the lightweight model parameters update the model parameters of the reinforcement learning model, and the updated reinforcement learning model is synchronized to the edge decision module.

[0061] The firewall dynamic defense method provided by the embodiment of the application reduces the main CPU load by extracting basic features through lightweight calculation and extracting high-dimensional features from high-risk traffic, and filtering features in layers. The training delay and rule base rigidity problem in the big data scenario is effectively solved through hierarchical traffic processing and intelligent decision-making collaboration. The first high-value sample set and the second high-value sample set are associated to generate complete samples, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved. Through the inference decision of the edge decision module, the global model parameters are updated, the global model parameters are pruned and quantized to generate a lightweight model, and the model deployment in a resource-constrained environment is solved.

[0062] FIG. 2 The firewall dynamic defense method provided by the embodiment of the application reduces the main CPU load by extracting basic features through lightweight calculation and extracting high-dimensional features from high-risk traffic, and filtering features in layers. The training delay and rule base rigidity problem in the big data scenario is effectively solved through hierarchical traffic processing and intelligent decision-making collaboration. The first high-value sample set and the second high-value sample set are associated to generate complete samples, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved. Through the inference decision of the edge decision module, the global model parameters are updated, the global model parameters are pruned and quantized to generate a lightweight model, and the model deployment in a resource-constrained environment is solved. FIG. 1 The firewall dynamic defense method provided by the embodiment of the application reduces the main CPU load by extracting basic features through lightweight calculation and extracting high-dimensional features from high-risk traffic, and filtering features in layers. The training delay and rule base rigidity problem in the big data scenario is effectively solved through hierarchical traffic processing and intelligent decision-making collaboration. The first high-value sample set and the second high-value sample set are associated to generate complete samples, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved. Through the inference decision of the edge decision module, the global model parameters are updated, the global model parameters are pruned and quantized to generate a lightweight model, and the model deployment in a resource-constrained environment is solved. FIG. 2 The firewall dynamic defense method provided by the embodiment of the application reduces the main CPU load by extracting basic features through lightweight calculation and extracting high-dimensional features from high-risk traffic, and filtering features in layers. The training delay and rule base rigidity problem in the big data scenario is effectively solved through hierarchical traffic processing and intelligent decision-making collaboration. The first high-value sample set and the second high-value sample set are associated to generate complete samples, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved. Through the inference decision of the edge decision module, the global model parameters are updated, the global model parameters are pruned and quantized to generate a lightweight model, and the model deployment in a resource-constrained environment is solved. FIG. 2 The firewall dynamic defense method provided by the embodiment of the application reduces the main CPU load by extracting basic features through lightweight calculation and extracting high-dimensional features from high-risk traffic, and filtering features in layers. The training delay and rule base rigidity problem in the big data scenario is effectively solved through hierarchical traffic processing and intelligent decision-making collaboration. The first high-value sample set and the second high-value sample set are associated to generate complete samples, the priority of each complete sample is calculated, the third high-value sample set is screened, the model training sample precision is improved, and the model training effect is improved. Through the inference decision of the edge decision module, the global model parameters are updated, the global model parameters are pruned and quantized to generate a lightweight model, and the model deployment in a resource-constrained environment is solved.FIG. 1 The differences between the embodiments are not described again.

[0063] As FIG. 2 shown, the firewall dynamic defense method provided by the embodiments of the present application generates a first high-value sample set, including the following steps.

[0064] Step 201, associate high-dimensional features with structured flow metadata to generate a high-dimensional feature vector.

[0065] Exemplarily, the high-dimensional features include SimHash fingerprints and TLS handshake behavior entropy.

[0066] For the high-dimensional feature vector, step 202 is performed.

[0067] Step 202, determine whether the SimHash fingerprint matches the malicious signature library and whether the TLS handshake behavior entropy is less than a set threshold value, and if so, perform step 203.

[0068] Step 203, the sample is added to the first high-value sample set.

[0069] Exemplarily, when the set threshold value is 1.0, the first high-value sample set is a sample set that satisfies the SimHash fingerprint matching the malicious signature library and the TLS handshake behavior entropy being less than 1.0.

[0070] It should be understood that in other embodiments, only one of the conditions that the SimHash fingerprint matches the malicious signature library and the TLS handshake behavior entropy is less than the set threshold value is satisfied, and the sample is added to the first high-value sample set.

[0071] The firewall dynamic defense method provided by the embodiments of the present application realizes accurate identification of encrypted traffic and covert attacks by extracting high-dimensional features such as SimHash fingerprints and TLS handshake behavior entropy of high-risk traffic, and associating them with structured metadata to form a feature vector; both double judgment conditions (SimHash matching malicious signature library and TLS entropy less than threshold 1.0) can be used to construct the first high-value sample set, which significantly improves the screening accuracy of malicious samples, while retaining the flexibility of single condition triggering, ensuring strict capture of high-risk threats, and also considering the adaptability requirements of different security scenarios, providing a high-quality training data basis for subsequent reinforcement learning models.

[0072] FIG. 3 The firewall dynamic defense method provided by the embodiments of the present application is shown in the flowchart. In the present application FIG. 2 The embodiments of the present application are extended on the basis of the embodiments of the present application FIG. 3 The embodiments of the present application are extended on the basis of the embodiments of the present application FIG. 3 The embodiments of the present application are extended on the basis of the embodiments of the present application FIG. 2The differences of the embodiments are described below, and the same parts are not described again.

[0073] As FIG. 3 shown, the firewall dynamic defense method provided by the embodiments of the application includes the following steps.

[0074] Step 300, the data packet payload of the high-risk traffic is processed by fragmentation, and the hash value of each fragment is calculated.

[0075] Step 301, the hash value is weighted and accumulated to generate a SimHash fingerprint.

[0076] Step 302, the cipher suite list is extracted from the TLS handshake packet, and the TLS handshake behavior entropy is calculated.

[0077] The structured traffic metadata includes five tuples and basic features.

[0078] Step 303, the SimHash fingerprint and the TLS handshake behavior entropy are associated with the five tuples and the basic features to generate a high-dimensional feature vector.

[0079] The Kafka queue module includes three FPGA acceleration cards, and three partitions are set, each partition corresponding to each FPGA acceleration card, and the partition rule adopts five tuple hash partitioning: hash (source IP + destination IP + source port + destination port + protocol) % 3; wherein %, remainder is 0, 1, 2 respectively corresponding to three partitions;

[0080] The structured traffic metadata with risk score is entered into different partitions according to the preset partition rule; the structured traffic metadata with risk score is divided into fixed-size fragments, and the insufficient is zero-filled.

[0081] Specifically, the data packet payload is all information in the structured traffic metadata with risk score, excluding the protocol header information; the fixed-size fragment can be 128, 256 bytes / fragment or other size fragments; MD5 hash is calculated for each fragment, and the first 64 bits are taken as the hash value hash_i of the fragment; wherein i is the fragment index; the hash value hash_i of each fragment is weighted and accumulated according to formula (4) to obtain the SimHash fingerprint, and the formula (4) is as follows:

[0082]

[0083] Wherein, bit_j is the value of the jth bit; 2 j is the weight of the jth bit; n is the number of fragments;

[0084] The accumulated SimHash value is taken as a sign bit by bit to obtain a 64-bit SimHash fingerprint; wherein the sign bit operation is as follows: if the accumulated value of a bit is greater than 0, 1 is taken, otherwise, 0 is taken.

[0085] The structured traffic metadata with risk scores is extracted from the cipher suite list in the TLS handshake packet in the session defined by the five-tuple; the occurrence frequency p(x) of each parameter x is counted: p(x) = the number of occurrences Nx of the parameter x / the total number of parameters m; the TLS handshake behavior entropy E is calculated by formula (5) as follows:

[0086]

[0087] The extracted SimHash fingerprint (64 bits), TLS behavior entropy, five-tuple, and basic features are associated to generate a high-dimensional feature vector, which is sent to the central experience pool; the central experience pool stores the high-dimensional feature vector using Redis cluster, the session ID composed of the five-tuple and the session start time is used as the key of the Redis cluster, the high-dimensional feature vector is used as the value of the Redis cluster, a 24-hour expiration time is set, and the high-dimensional feature vector exceeding 24 hours is automatically deleted; for the high-dimensional feature vector, if the SimHash matches the malicious signature and / or the TLS handshake behavior entropy is less than 1.0, the first high-value sample set is added; the first high-value sample set is sent to the central Critic module.

[0088] The firewall dynamic defense method provided by the embodiments of the application uses FPGA acceleration cards to process high-dimensional feature extraction with high computational complexity, thereby reducing the load of the main CPU; the accuracy of malicious traffic detection is improved and the feature extraction efficiency is improved by using the targeted high-dimensional features (SimHash fingerprint, TLS handshake behavior entropy); the Kafka queue is used to buffer high-risk traffic, thereby avoiding data loss and ensuring real-time performance; the high-dimensional features are used to train the reinforcement learning model, thereby reducing the dependence on artificial rules and avoiding conflicts with existing firewall rules.

[0089] An embodiment of the present application provides a firewall dynamic defense method, in which low-risk traffic and medium-risk traffic are subjected to firewall rule matching judgment, if the matching is successful, corresponding action instructions are executed, otherwise, an action decision is made according to structured traffic metadata and risk scores through a reinforcement learning model to generate action instructions, including: traversing a preset firewall rule library, matching the structured traffic metadata with rule conditions, if the structured traffic metadata matches the firewall rule, executing the action instruction corresponding to the rule and recording a rule execution log; wherein the rule execution log includes a rule ID, a traffic five-tuple, an action and a timestamp; if the structured traffic metadata does not match the firewall rule, inputting the structured traffic metadata and the risk scores into the reinforcement learning model, the reinforcement learning model being used to output action probabilities of allowing, blocking and isolating, and selecting the action with the highest probability as the final action instruction.

[0090] Exemplarily, the preset rules include but are not limited to:

[0091] Rule 1: Condition: source IP e ip_blacklist, Action: block;

[0092] Rule 2: Condition: destination port = 3389 (remote desktop), Action: isolate;

[0093] Rule 3: Condition: Session_Duration < 24 hours, Action: allow.

[0094] Exemplarily, the reinforcement learning model adopts a light-weight DQN (Deep Q-Network) model after knowledge distillation and quantization pruning, the parameter quantity of which is <1M, the input layer of which is a 6-dimensional feature vector: N_Packet_Rate, N_Byte_Rate, ip_blacklist, ip_whitelist, port_risk and risk_score, the hidden layer of which is a 2-layer convolution layer, and the output layer of which is a 3-dimensional action probability: the probabilities of allowing, blocking and isolating actions. The action with the highest probability is selected as the action instruction.

[0095] The firewall dynamic defense method provided by the embodiment of the present application realizes millisecond-level decision-making through a light-weight model, thereby guaranteeing real-time performance; and by preferentially traversing the rule library preset by the firewall, conflicts with existing firewall rules are avoided.

[0096] FIG. 4 Fig. 1 shows a flowchart of a firewall dynamic defense method provided by another exemplary embodiment of the present application. On the basis of the embodiment shown in Fig. 1, the present application extends to the embodiment shown in Fig. 2. FIG. 1 On the basis of the embodiment shown in Fig. 1, the present application extends to the embodiment shown in Fig. 2. FIG. 4 On the basis of the embodiment shown in Fig. 1, the present application extends to the embodiment shown in Fig. 2. FIG. 4 On the basis of the embodiment shown in Fig. 1, the present application extends to the embodiment shown in Fig. 2. FIG. 1The differences of the embodiments shown will not be described again.

[0097] As shown in FIG. 4 The first high-value sample set and the second high-value sample set are associated to generate a third high-value sample set, including the following steps.

[0098] Step 400, the high-dimensional feature vector in the first high-value sample set is matched and associated with the decision triplets in the second high-value sample set based on the session identifier, to generate a plurality of complete samples.

[0099] Specifically, the second high-value sample set of the local experience pool is obtained, and the first high-value sample set of the center experience pool is obtained; specifically, each sample in the second high-value sample set includes a decision triplet, and each sample in the first high-value sample set includes a high-dimensional feature vector; the second high-value sample set and the first high-value sample set are traversed, and the decision triplets and the high-dimensional feature vectors of the same session are matched according to the session ID; the 6-dimensional feature vector after splicing in the second high-value sample set, the action a, and the immediate reward r are associated with the SimHash fingerprint and the TLS handshake behavior entropy in the first high-value sample set to generate a complete sample, and the complete sample is transmitted to the global experience pool; the global experience pool stores the high-dimensional feature vector by using a Redis cluster, the session ID is used as a key of the Redis cluster, and the complete sample is used as a value of the Redis cluster.

[0100] Step 401, the plurality of complete samples are stored to the global experience pool, and the priority of each complete sample is calculated.

[0101] The priority is determined based on a time difference error and a preset offset.

[0102] Specifically, for each complete sample in the global experience pool, the TD error (time difference error) is calculated by using formula (6), and the formula (6) is as follows:

[0103] TD_error=r+γ×max(Q(s',a'))-Q(s,a) (6)

[0104] Wherein, s' is the next state; a' is the next action; γ is a discount factor, used to weigh the immediate reward and the future reward; max(Q(s',a')) is the maximum Q value of the next state s'; Q(s,a) is the Q value of the current state s and the action a; max(Q(s',a')) and Q(s,a) are predicted by the current reinforcement learning model of the center Critic module.

[0105] The priority Pri is calculated by using formula (7), and the formula (7) is as follows:

[0106] Pri=|TD_error|+ε (7)

[0107] wherein, ε = 0.01.

[0108] At step 402, based on the ranking result of the priority of each complete sample, a preset proportion of complete samples are selected as the third high-value sample set.

[0109] For example, the preset proportion can be 20%. For example, according to the priority from high to low, the first 20% of the complete samples are selected to join the third high-value sample set. Of course, the preset proportion can be adjusted according to the actual situation, and 10% or 30% can also be used.

[0110] The firewall dynamic defense method provided by the embodiment of the application improves the recognition accuracy of the model for malicious traffic by calculating the TD error, selecting samples with larger TD error, calculating the priority of each complete sample, screening the third high-value sample set, improving the information amount and training value of the sample, and improving the feature extraction efficiency.

[0111] FIG. 5 Fig. 4 shows a flowchart of a firewall dynamic defense method provided by another exemplary embodiment of the application. Based on the embodiment shown in Fig. 3, the embodiment shown in Fig. 4 extends the embodiment shown in Fig. 3. FIG. 1 Based on the embodiment shown in Fig. 3, the embodiment shown in Fig. 4 extends the embodiment shown in Fig. 3. FIG. 5 The differences between the embodiment shown in Fig. 4 and the embodiment shown in Fig. 3 will be described below. FIG. 5 The differences between the embodiment shown in Fig. 4 and the embodiment shown in Fig. 3 will be described below. FIG. 1 The differences between the embodiment shown in Fig. 4 and the embodiment shown in Fig. 3 will be described below.

[0112] As shown in Fig. 4, the firewall dynamic defense method provided by the embodiment of the application includes the following steps. FIG. 5 As shown in Fig. 4, the firewall dynamic defense method provided by the embodiment of the application includes the following steps.

[0113] At step 500, the corresponding network traffic control operation is performed according to the action instruction, and the instant reward value is calculated based on the execution result.

[0114] For example, the network traffic control operation includes: an allow operation of passing data flow and recording session log, a block operation of discarding data packets and returning a connection rejection response, and an isolation operation of redirecting traffic to a sandbox detection environment.

[0115] Specifically, the execution result of the firewall executor is fed back to the real-time inference module, and the instant reward r is calculated according to the action execution result by using formula (8), which is as follows:

[0116] r = (0.1 * Byte_Rate) * I (allow) - (5 * I (misblock)) + (10 * I (correct block)) (8)

[0117] Wherein, I (allow) is the instruction function of the allowed action, and the allowed is 1 and the unallowed is 0; I (mis-block) is the indication function of mis-blocking legal traffic, and the mis-block is 1 and the correct block is 0; I (correct block) is the indication function of correctly blocking malicious traffic, and the correct block is 1 and the mis-block is 0.

[0118] If the reinforcement learning model selects the blocking action, according to the execution result feedback, it is confirmed that the traffic is malicious, then r = 10; if it is legal traffic, then r = -5.

[0119] Step 501, store the decision triple data into the local experience pool.

[0120] The decision triple data includes state features, execution actions and immediate rewards.

[0121] Specifically, the local experience pool uses Redis cluster storage, adds a session ID to the newly recorded decision triple, the session ID is composed of a five-tuple and a session start time, the capacity of the local experience pool is 1000, the newly recorded decision triple covers the old sample, and the last 1000 are retained.

[0122] Step 502, filter the decision data that meets the reward threshold condition from the local experience pool to form a second high-value sample set.

[0123] Set the timing time, traverse the local experience pool, if the immediate reward in the decision triple |r|≥5, add this record to the second high-value sample set, when the timing time is up, send the second high-value sample set to the center experience pool.

[0124] Exemplarily, the timing time can be set at intervals of 5s, 10s, 15s, and different timing times represent the user's requirement for the training speed of the reinforcement learning model.

[0125] Exemplarily, |r|≥5 indicates the mis-blocked or correctly blocked sample.

[0126] In an embodiment, the center Critic module extracts the second high-value sample set from the center experience pool, trains the model parameters of the reinforcement learning model, and sends them to the edge decision module; the center Critic module only sends the changed model parameters, also called incremental parameters, such as the input layer, hidden layer and output layer parameters updated in the last 24 hours, to reduce the bandwidth occupied when synchronizing to the edge decision module; the edge decision module receives the model parameters, replaces the corresponding layer parameters of the old model, and sets the new model to the active state.

[0127] Exemplarily, the active state is the active state, and the model can normally perform inference jobs in this state.

[0128] The firewall dynamic defense method provided by the embodiment of the application quantifies the actual effect of network management and control operation into an instant reward value, realizes accurate evaluation of model decision quality, automatically filters high-value samples by setting a reward threshold, effectively focuses on key learning data, and greatly improves model training efficiency; the second high-value sample set is stored in the experience pool, the Critic module training model is fed back, and the feature extraction efficiency is improved.

[0129] FIG. 6 Fig. 1 shows a flowchart of a firewall dynamic defense method provided by another exemplary embodiment of the application. Based on the embodiment shown in Fig. 1, the embodiment shown in Fig. 2 is extended. FIG. 1 Based on the embodiment shown in Fig. 2, the embodiment shown in Fig. 3 is extended. FIG. 6 Based on the embodiment shown in Fig. 3, the embodiment shown in Fig. 4 is extended. FIG. 6 The embodiment shown in Fig. 4 is different from the embodiment shown in Fig. 3 in that FIG. 1 The same parts of the embodiment shown in Fig. 4 and the embodiment shown in Fig. 3 will not be described again.

[0130] As shown in Fig. 5, in the firewall dynamic defense method provided by the embodiment of the application, the global model parameters are updated according to the third high-value sample set, and a lightweight model is generated after pruning and quantization processing, including the following steps. FIG. 6 Step 600, batch processing and randomization processing are performed on the third high-value sample set.

[0131] The Critic module performs batch processing on the received third high-value sample set: the third high-value sample set is grouped according to batch size=64, and the order is shuffled.

[0132] Exemplarily, the batch size is the number of samples input to the reinforcement learning model for calculation at a time, the larger the batch size value, the faster the processing speed, and the larger the memory occupation.

[0133] Step 601, based on the samples after batch processing and randomization processing, distributed model training is performed using a parameter server architecture to update global model parameters.

[0134] Specifically, the Critic module adopts a parameter server (Parameter Server) architecture, including a parameter server node and a worker node, the parameter server node is responsible for storing global model parameters and updating parameters, the worker node obtains the current parameters from the parameter server node, processes batch samples, calculates the loss function Loss and gradient

[0135] and sends them back to the parameter server node.

[0136] ​It should be understood that the working node obtains the current global model parameters from the parameter server node. The current global model parameters include the convolutional layer weights and the fully connected layer bias items; the working node processes batch samples, calculates the gradients, and sends the gradients back to the parameter server node; after the parameter server node receives the gradients from all working nodes, it averages the gradients and updates the global model parameters.

[0137] The reinforcement learning model of the central critic module uses a deep Q network. The input layer is a concatenated 6-dimensional feature vector and a 2-dimensional high-dimensional feature (SimHash fingerprint, TLS handshake behavior entropy). The hidden layer is a 2-layer convolutional layer and a 1-layer fully connected layer. The output layer is a 3-dimensional Q value, corresponding to the Q value of the allow, block, and isolate actions. The working node uses the current reinforcement learning model to calculate the target Q value through formula (9), which is as follows:

[0138] r+γ×max(Q(s',a'))(9)

[0139] The working node calculates the loss function Loss through formula (10), and the working node converts the gradient Sent to the parameter server node, the parameter server node averages the gradients of all working nodes through formula (11), the formula (10) and formula (11) are as follows:

[0140] Loss=E[(r+γ×max(Q(s',a′))-Q(s,a)) 2 ] (10)

[0141]

[0142] Where n is the number of worker nodes.

[0143] The parameter server node updates the global model parameters: Among them, η is the learning rate, the default setting is η = 0.001; θ is the model parameter.

[0144] Step 602: Perform neuron pruning on the updated global model parameters, and remove weights and their connecting nodes whose absolute values ​​are less than a preset threshold.

[0145] Step 603: The updated model parameters are converted into 8-bit integer representation through linear quantization and a quantization table is generated, in which the sign bit occupies 1 bit and the value bit occupies 7 bits, and finally a lightweight model is generated.

[0146] All weights in the updated global model parameters are traversed, if the absolute value of the weight is less than 0.001, the neuron corresponding to the weight is marked; the marked neuron and its connection node are removed; the minimum value min_val and the maximum value max_val of the model parameters recorded by 32-bit floating point in the updated global model parameters are calculated; the 32-bit floating point value is mapped to the INT8 range: INT8_value = round ((FP32_value-min_val)*(255 / (max_val-min_val))-128); wherein, FP32_value represents the 32-bit floating point model parameter; the INT8 model parameter is stored, and a quantization table is generated, which is used for converting the INT8 value back to FP32 by the real-time inference module; the difference parameters are extracted by comparing the current model parameters with the old model parameters of the edge decision module, and the difference parameters are compressed and sent to the edge decision module.

[0147] The firewall dynamic defense method provided by the embodiment of the application can compress the model parameters sent to the edge decision module through pruning and quantization operation, reduce the bandwidth occupation, and guarantee the real-time performance; through the parameter server architecture, the computing load is shared, and the main CPU load is reduced.

[0148] Exemplary system

[0149] FIG. 7 As shown in the structural schematic diagram of the firewall dynamic defense system provided by an exemplary embodiment of the application. FIG. 7 As shown, the firewall dynamic defense system provided by the embodiment of the application comprises a first generation module 700, a classification module 701, a second generation module 702, a judgment module 703, a third generation module 704, a fourth generation module 705 and a fifth generation module 706.

[0150] The first generation module 700 is used for extracting session data packets from original network traffic and generating structured traffic metadata. The classification module 701 is used for calculating risk scores based on the structured traffic metadata, and classifying the traffic into low-risk traffic, medium-risk traffic and high-risk traffic according to the score threshold. The second generation module 702 is used for extracting high-dimensional features from the high-risk traffic and generating a first high-value sample set. The judgment module 703 is used for performing firewall rule matching judgment on the low-risk traffic and the medium-risk traffic, if the matching is successful, the corresponding action instruction is executed, otherwise, the action decision is made according to the structured traffic metadata and the risk score through the reinforcement learning model, and the action instruction is generated. The third generation module 704 is used for executing the action instruction and calculating the immediate reward, and generating a second high-value sample set. The fourth generation module 705 is used for associating the first high-value sample set and the second high-value sample set, and generating a third high-value sample set. The fifth generation module 706 is used for updating the global model parameters according to the third high-value sample set, and generating a lightweight model after pruning and quantization processing.

[0151] In some other embodiments, the second generation module 702 is further configured to associate the high-dimensional features with the structured traffic metadata to generate a high-dimensional feature vector; and add the sample into the first high-value sample set if the SimHash fingerprint matches the malicious signature library and / or the TLS handshake behavior entropy is less than a set threshold for the high-dimensional feature vector.

[0152] In some other embodiments, the second generation module 702 is further configured to perform fragmentation processing on the data packet payload of the high-risk traffic to calculate hash values of the fragments; and perform weighted accumulation on the hash values to generate a SimHash fingerprint; and extract a cipher suite list from the TLS handshake packet to calculate a TLS handshake behavior entropy; wherein the high-dimensional features are associated with the structured traffic metadata to generate a high-dimensional feature vector, and the structured traffic metadata includes a five-tuple and basic features, including: associating the SimHash fingerprint and the TLS handshake behavior entropy with the five-tuple and the basic features to generate the high-dimensional feature vector.

[0153] In some other embodiments, the judgment module 703 is further configured to traverse a preset firewall rule library, match the structured traffic metadata with rule conditions, execute a corresponding action instruction of the rule if the structured traffic metadata matches the firewall rule, and record a rule execution log; wherein the rule execution log includes a rule ID, a traffic five-tuple, an action, and a timestamp; and input the structured traffic metadata and the risk score into a reinforcement learning model if the structured traffic metadata does not match the firewall rule, and the reinforcement learning model is configured to output action probabilities of allowing, blocking, and isolating, and select the action with the highest probability as the final action instruction.

[0154] In some other embodiments, the fourth generation module 705 is further configured to match and associate the high-dimensional feature vectors in the first high-value sample set with the decision triplets in the second high-value sample set based on session identifiers to generate a plurality of complete samples; store the plurality of complete samples to a global experience pool, calculate a priority of each complete sample based on a time difference error and a preset offset; and select a preset proportion of the complete samples as a third high-value sample set based on a sorting result of the priority of each complete sample.

[0155] In some other embodiments, the third generation module 704 is further configured to perform a corresponding network traffic control operation according to the action instruction, calculate an immediate reward value based on an execution result; the network traffic control operation includes: an allowing operation of releasing a data stream and recording a session log, a blocking operation of discarding a data packet and returning a connection rejection response, and an isolation operation of redirecting traffic to a sandbox detection environment; store decision triplet data containing state features, execution actions, and immediate rewards into a local experience pool; and select decision data groups that meet a reward threshold condition from the local experience pool to form the second high-value sample set.

[0156] In some other embodiments, the fifth generation module 706 is also used to perform batch processing and randomization processing on the third high-value sample set, and based on the samples after batch processing and randomization processing, use the parameter server architecture to perform distributed model training and update the global model parameters; perform neuron pruning processing on the updated global model parameters to remove weights and their connecting nodes whose absolute values ​​are less than a preset threshold; convert the updated model parameters into an 8-bit integer representation through linear quantization and generate a quantization table, in which the sign bit occupies 1 bit and the value bit occupies 7 bits, and finally generate a lightweight model.

[0157] It should be understood that the operations and functions of the relevant modules mentioned in the firewall dynamic defense system can refer to the above FIG. 1 to FIG. 6 The provided dynamic firewall defense method will not be described here in order to avoid repetition.

[0158] Exemplary electronic device

[0159] Below, reference FIG. 8 To describe the electronic device according to the embodiment of the present application. FIG. 8 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0160] like FIG. 8 As shown, electronic device 800 includes one or more processors 801 and memory 802 .

[0161] The processor 801 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.

[0162] The memory 802 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 801 may execute the program instructions to implement the firewall dynamic defense method of the various embodiments of the present application described above and / or other desired functions. Various contents such as structured traffic metadata may also be stored in the computer-readable storage medium.

[0163] In one example, the electronic device 800 can further include an input device 803 and an output device 804, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0164] The input device 803 can include, for example, a keyboard, a mouse, and the like.

[0165] The output device 804 can output various information, including a lightweight model, etc., to the outside. The output device 804 can include, for example, a display, a communication network and a remote output device connected thereto, and the like.

[0166] Of course, in order to simplify, FIG. 8 Only some of the components of the electronic device 800 related to the present application are shown in the middle, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 800 can further include any other appropriate components according to specific application cases.

[0167] In addition to the above-mentioned methods and devices, the embodiments of the present application can also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the steps of the firewall dynamic defense method according to various embodiments of the present application described in the above "Exemplary Methods" section of the present specification.

[0168] The computer program product can be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, and the like, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0169] In addition, the embodiments of the present application can also be a computer readable storage medium, which stores computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the firewall dynamic defense method according to various embodiments of the present application described in the above "Exemplary Methods" section of the present specification.

[0170] The computer readable storage medium can be a combination of one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0171] The above description of the disclosed aspects is merely exemplary in nature and is not intended to limit the present disclosure, application, and uses. The description of the aspects together with the accompanying drawings are intended to explain aspects of the present disclosure and are not intended to limit the scope of the disclosure, its application, or uses. Not all of the depicted aspects are required, and some aspects can be utilized independently. The description herein is presented for the purpose of illustration and description and is not intended to limit the scope of the disclosure as set forth in the appended claims.

[0172] The block diagrams of the devices, apparatus, systems, and flowcharts of methods involved in the present disclosure are merely illustrative examples and are not intended to require or imply that connections, arrangements, configurations be made in the manner as shown in the block diagrams. As will be recognized by one of ordinary skill in the art, the devices, apparatus, systems, and flowcharts of methods can be connected, arranged, configured in any manner. The words comprising, including, having, containing, comprising, and the like are to be open ended. When the phrases "at least one of" or "one or more of" appear in this disclosure, the phrases are to be affixed to the list of elements and / or items which they precede, applying the term to each of the elements individually (not the list of elements as a whole), so that, for example, the phrasing "at least one of a, b, and c" indicates that the phrase applies to a individually, b individually, c individually, a and b in combination, a and c in combination, b and c in combination, and a, b, and c in combination.

[0173] It is also to be noted that the various components or steps can be divided into further components or steps, or combined into fewer components or steps. Such alternations and / or combinations should be considered as equivalent to the aspects of the present disclosure.

[0174] The above description of the disclosed aspects is merely exemplary in nature and is not intended to limit the present disclosure, application, and uses. The description of the aspects together with the accompanying drawings are intended to explain aspects of the present disclosure and are not intended to limit the scope of the disclosure, its application, or uses. Not all of the depicted aspects are required, and some aspects can be utilized independently. The description herein is presented for the purpose of illustration and description and is not intended to limit the scope of the disclosure as set forth in the appended claims.

[0175] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms invented herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A firewall dynamic defense system based on big data streaming processing, characterized in that: The firewall dynamic defense system is used to implement a firewall dynamic defense method, which includes: Extract session packets from raw network traffic and generate structured traffic metadata; Calculating a risk score based on the structured traffic metadata, and classifying the traffic into low-risk traffic, medium-risk traffic, and high-risk traffic according to a score threshold; Extracting high-dimensional features from the high-risk traffic to generate a first high-value sample set; Perform firewall rule matching on the low-risk traffic and the medium-risk traffic. If a match is successful, execute the corresponding action instruction. Otherwise, make an action decision based on the structured traffic metadata and the risk score through a reinforcement learning model to generate an action instruction. Executing the action instruction and calculating the immediate reward to generate the second highest value sample set; Associating the first high-value sample set with the second high-value sample set to generate a third high-value sample set; The global model parameters are updated according to the third highest value sample set, and a lightweight model is generated after pruning and quantization processing.

2. The firewall dynamic defense system according to claim 1, characterized in that: Generating the first high-value sample set includes: Associating the high-dimensional features with the structured traffic metadata to generate a high-dimensional feature vector, the high-dimensional features including SimHash fingerprint and TLS handshake behavior entropy; For the high-dimensional feature vector, if the SimHash fingerprint matches the malicious signature library, and / or the TLS handshake behavior entropy is less than a set threshold, the sample is added to the first high-value sample set.

3. The firewall dynamic defense system according to claim 2, characterized in that: The extracting high-dimensional features from the high-risk traffic includes: Slice the data packet payload of the high-risk traffic and calculate the hash value of each slice; Performing weighted accumulation on the hash values ​​to generate the SimHash fingerprint; Extract the cipher suite list from the TLS handshake packet and calculate the TLS handshake behavior entropy; The high-dimensional features are associated with the structured traffic metadata to generate a high-dimensional feature vector, and the structured traffic metadata includes a quintuple and basic features, including: The SimHash fingerprint, the TLS handshake behavior entropy, the quintuple, and the basic features are associated to generate the high-dimensional feature vector.

4. The firewall dynamic defense system according to any one of claims 1 to 3, characterized in that: The firewall rule matching judgment is performed on the low-risk traffic and the medium-risk traffic. If the match is successful, the corresponding action instruction is executed. Otherwise, an action decision is made based on the structured traffic metadata and the risk score through a reinforcement learning model to generate an action instruction, including: Traverse the preset firewall rule library and match the structured traffic metadata with the rule conditions. If the structured traffic metadata matches the firewall rule, execute the action instruction corresponding to the rule and record the rule execution log; wherein the rule execution log includes the rule ID, traffic quintuple, action and timestamp; If the structured traffic metadata does not match the firewall rule, the structured traffic metadata and the risk score are input into the reinforcement learning model, and the reinforcement learning model is used to output the action probabilities of allow, block, and isolate, and the action with the highest probability is selected as the final action instruction.

5. The firewall dynamic defense system according to any one of claims 1 to 3, characterized in that: The associating the first high-value sample set with the second high-value sample set to generate a third high-value sample set includes: Matching and associating the high-dimensional feature vectors in the first high-value sample set with the decision triples in the second high-value sample set based on the session identifier to generate multiple complete samples; Storing the multiple complete samples in a global experience pool, and calculating a priority of each complete sample, wherein the priority is determined based on a time difference error and a preset offset; Based on the ranking result of the priority of each complete sample, a preset proportion of complete samples is selected as the third highest value sample set.

6. The firewall dynamic defense system according to any one of claims 1 to 3, characterized in that: The executing the action instruction and calculating the immediate reward to generate the second highest value sample set includes: Executing corresponding network traffic control operations according to the action instructions and calculating an immediate reward value based on the execution results; the network traffic control operations include: an allow operation that releases data flow and records session logs, a block operation that discards data packets and returns a connection rejection response, and an isolation operation that redirects traffic to a sandbox detection environment; Storing the decision triplet data into the local experience pool, wherein the decision triplet data includes state features, execution actions, and immediate rewards; Decision data that meets the reward threshold condition is screened from the local experience pool to form the second high-value sample set.

7. The firewall dynamic defense system according to any one of claims 1 to 3, characterized in that: The updating of global model parameters according to the third highest value sample set and generating a lightweight model after pruning and quantization processing includes: The third highest value sample set is batch processed and randomized, Based on the samples after batch processing and randomization processing, a parameter server architecture is used to perform distributed model training and update global model parameters; Perform neuron pruning on the updated global model parameters, removing weights and their connected nodes whose absolute values ​​are less than a preset threshold; The updated model parameters are converted into 8-bit integer representation through linear quantization and a quantization table is generated, in which the sign bit occupies 1 bit and the value bit occupies 7 bits, and finally the lightweight model is generated.

8. A firewall dynamic defense system, used to implement the firewall dynamic defense method according to any one of claims 1 to 7, characterized in that: include: A first generation module is used to extract session data packets from original network traffic and generate structured traffic metadata; a classification module, configured to calculate a risk score based on the structured traffic metadata and classify the traffic into low-risk traffic, medium-risk traffic, and high-risk traffic according to a score threshold; A second generating module is used to extract high-dimensional features from the high-risk traffic to generate a first high-value sample set; a judgment module, configured to perform firewall rule matching judgment on the low-risk traffic and the medium-risk traffic, and if a match is successful, execute a corresponding action instruction; otherwise, make an action decision based on the structured traffic metadata and the risk score through a reinforcement learning model to generate an action instruction; A third generation module is used to execute the action instruction and calculate the immediate reward to generate a second high-value sample set; a fourth generating module, configured to associate the first high-value sample set with the second high-value sample set to generate a third high-value sample set; The fifth generation module is used to update the global model parameters according to the third high-value sample set, and generate a lightweight model after pruning and quantization processing.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory storing a computer program; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images