Computer network security dynamic protection method and system based on fusion factors

Abstract

The invention discloses a fusion factor-based computer network security dynamic protection method and system, and relates to the field of computer network security, and the method comprises the steps: setting a security dynamic protection detection mode for different computer network application environments; setting a corresponding security dynamic protection scheme based on the detection content of the security dynamic protection mode; based on a deep reinforcement learning model, constructing a decision-making mode of computer network security dynamic protection; according to a decision result of the deep reinforcement learning model, adopting a corresponding protection scheme to solve the network risk; and establishing a network abnormal behavior feature library, and comparing with a decision result of the model to adjust parameters of the model and optimize the network abnormal behavior feature library. The method has the advantages that the deep reinforcement learning model is utilized, through early-stage computer network security dynamic protection model learning, the computer network security dynamic autonomous protection and early warning capabilities in different environments are effectively improved, and the network risk is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer network security, and in particular to a computer network security dynamic protection method and system based on fusion factors. Background Art

[0002] With the rapid development of the information age, computer network technology has been applied to all walks of life. Ensuring the security of computer networks is of great significance to protecting personal information, maintaining the normal operation of corporate operations and ensuring corporate stability. For complex and changeable computer network environments, it is necessary to consider the influence of multiple factors. Therefore, by integrating multiple factors and combining dynamic protection technologies and methods, we can improve the defense capabilities against unknown advanced network threats and the decision-making capabilities of automatically handling risks.

[0003] Existing computer network security protection strategies mainly rely on traditional static security measures, such as firewalls and intrusion detection systems. Faced with complex computer network environments, they lack the ability to respond to different environments and the ability to actively deal with abnormal behavior risks, resulting in low efficiency of computer network security protection and increased load on computer processors. Summary of the invention

[0004] In order to solve the above technical problems, a computer network security dynamic protection method and system based on fusion factors are provided. This technical solution solves the problem raised in the above background technology that the lack of response capabilities for different environments and the ability to actively handle abnormal behavior risks leads to low efficiency of computer network security protection and increased load on computer processors.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A computer network security dynamic protection method based on fusion factors, comprising:

[0007] Set up security dynamic protection detection mode for different computer network application environments;

[0008] Based on the detection content of the security dynamic protection mode, set the corresponding security dynamic protection plan;

[0009] Based on the deep reinforcement learning model, a decision-making method for dynamic protection of computer network security is constructed;

[0010] According to the decision results of the deep reinforcement learning model, corresponding protection solutions are adopted to solve network risks;

[0011] Establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0012] Preferably, the setting of the security dynamic protection mode for different computer network application environments specifically includes:

[0013] According to the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection, set the security dynamic protection detection mode according to different computer network application environments;

[0014] For application environments where computer network security requirements are not high and information needs to be kept confidential, the port number detection single mode is used;

[0015] For application environments with high requirements for computer network security and no need for accurate identification of traffic behavior, deep packet inspection single mode is used;

[0016] For computer network security application environments that require accurate identification of traffic behavior and fast processing speed, deep flow inspection single mode is used;

[0017] For other needs of computer network security, the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection modes are integrated to set a joint detection mode, and the detection results are adjusted by setting the weight of each detection mode;

[0018] Based on DS evidence theory, the proportion of port number detection, deep packet inspection, and deep flow inspection in the joint detection mode is calculated;

[0019] The weight expression for each detection is:

[0020]

[0021] Among them, y is the linear relationship between several detection methods, x1 is the use of port number detection, x2 is the use of deep packet inspection, x3 is the use of deep flow inspection, ε1 is the weight of port number detection, ε2 is the weight of deep packet inspection, and ε3 is the weight of deep flow inspection;

[0022] The DS evidence theory expression is:

[0023]

[0024] Among them, m(A) is the credibility of the A-th detection method, K is the normalization constant, and m i (A i ) is the probability of using the i-th feature item of the A-th detection method, and n is the number of feature items using the A-th detection method.

[0025] Preferably, the detection content based on the security dynamic protection mode and setting the corresponding security dynamic protection scheme specifically include:

[0026] Based on the historical data of abnormal network behavior, determine the detection content feature items of each detection mode;

[0027] For each detection mode's detection content feature items, set up a standardized processing plan for network abnormal behavior characteristics;

[0028] Establish a standardized processing list for abnormal behaviors based on the detection content characteristics and standardized processing plans.

[0029] Preferably, the decision-making method for building dynamic computer network security protection based on the deep reinforcement learning model specifically includes:

[0030] According to the standardized processing list of abnormal behavior, confirm the probability of selecting strategies between different processing options;

[0031] Based on the Markov chain algorithm, the state probability value at the future moment is confirmed according to the computer network's recognition of different detection content feature items;

[0032] Based on the standardized processing list of abnormal behaviors, calculate the cumulative reward feedback value corresponding to completing a decision;

[0033] Based on the deep reinforcement learning model and the Monte Carlo algorithm, we can determine the expected total reward for a decision made by the computer network in this state.

[0034] Based on the deep reinforcement learning model and the SARSA algorithm, confirm the expected total reward of the computer network choosing this action to complete a decision;

[0035] The probability expression of the selection strategy between different actions is as follows:

[0036] π(b|s)=P(B k =b|S k =s)

[0037] Among them, π(b|s) is the probability that the computer network selects the bth processing solution in state s, B k A collection of solutions for dealing with abnormal behavior characteristics of computer networks. k is the set of current states of the computer network;

[0038] The Markov chain algorithm is:

[0039] P[S k+1 |S k ]=P[S1|S2,S3,…,S i ,…,S k+1 ]

[0040] Among them, P[S k+1 |S k] is the probability of the computer network being in the k+1th state in the kth state, S k+1 is the k+1th state of the computer network in the future, S k is the current k-th state of the computer network;

[0041] The calculation of the cumulative reward feedback value corresponding to a decision is expressed as:

[0042]

[0043] Among them, G k is the cumulative reward feedback value corresponding to the kth state of a decision made by the computer network, γ is the discount factor of the future impact of the computer network on the current state, G k+1 The cumulative reward feedback value corresponding to the k+1th state of a decision completed by the computer network, r k The reward feedback value corresponding to the kth state of a decision completed by the computer network;

[0044] The computer network completes a decision in this state corresponding to G k The expected expression is:

[0045]

[0046] in, G is the computer network corresponding to the kth state of this strategy k The updated expected value, V(S k ) is the corresponding G of the computer network in the kth state of this strategy k The expected value of , α is the learning rate, V(S k+1 ) is the corresponding G of the computer network in the k+1th state of this strategy k Expected value;

[0047] The computer network selects this action to complete a decision corresponding to G k The expected expression is:

[0048]

[0049] in, Select this action for the computer network to complete a decision. The corresponding G in the kth state k The updated expected value, Q(B k ,S k ) is the computer network that selects this action to complete a decision. The corresponding G in the kth state k The expected value of, β is the learning rate, Q(B k+1 ,S k+1 ) is the computer network that selects this action to complete a decision. In the k+1th state, the corresponding G kexpected value.

[0050] Preferably, taking corresponding protection solutions to solve network risks according to the decision results of the deep reinforcement learning model specifically includes:

[0051] The methods for handling abnormal network behavior characteristics are divided according to the impact on network operation caused by abnormal network behavior characteristics and the simplicity of the handling methods;

[0052] Based on the learning and training results of the deep reinforcement learning model, automatic decision-making is performed on common, simple, and insignificant network abnormal behavior features;

[0053] Based on the learning and training results of the deep reinforcement learning model, early warning alarm methods are set up and abnormal network behavior characteristics that are complex, new, have a large impact and cause model overload are recorded.

[0054] Preferably, the establishing of the network abnormal behavior feature library, adjusting the parameters of the model and optimizing the network abnormal behavior feature library by comparing with the decision result of the model specifically comprises:

[0055] Establish a network abnormal behavior feature library based on the abnormal behavior standardized processing list and the decision-making processing results of the model;

[0056] By comparing the record processing results of the network abnormal behavior feature library with the decision results of the model, the most suitable processing method is selected, and the parameters of the model are adjusted and the network abnormal behavior feature library is optimized according to the comparison results;

[0057] By judging whether the decision result of the model is better than the processing result of the feature library, if so, the processing result recorded in the feature library is optimized; if not, the parameters of the model need to be adjusted.

[0058] Furthermore, the present invention proposes a computer network security dynamic protection system based on fusion factors, which is used to implement the above-mentioned computer network security dynamic protection method based on fusion factors, including:

[0059] A detection and solution module, which is used to set a security dynamic protection detection mode for different computer network application environments; and to set a corresponding security dynamic protection solution based on the detection content of the security dynamic protection mode;

[0060] The protection and feedback module is used to construct a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model; adopt corresponding protection plans to solve security risks according to the decision results of the deep reinforcement learning model; establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0061] Preferably, the detection and solution module includes:

[0062] A detection mode setting unit, wherein the detection mode setting unit is used to set a security dynamic protection detection mode for different computer network application environments;

[0063] A protection scheme setting unit is used to set a corresponding security dynamic protection scheme based on the detection content of the security dynamic protection mode.

[0064] Preferably, the protection and feedback module includes:

[0065] A protection decision unit, which is used to build a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model;

[0066] A protection scheme unit, which is used to adopt a corresponding protection scheme to solve the security risk according to the decision result of the deep reinforcement learning model;

[0067] The feature library feedback unit is used to establish a network abnormal behavior feature library, and adjust the parameters of the model and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] Different network security detection modes are set according to different computer network environments. According to the detection content of the security dynamic protection mode, a computer network security dynamic protection decision-making method is constructed based on the deep reinforcement learning model. Through model learning and training, the dynamic and autonomous security protection of computer networks can be achieved, effectively improving the computer network security protection capabilities and early warning capabilities in different environments, and reducing network risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A flowchart of a computer network security dynamic protection method based on fusion factors of the present invention;

[0071] Figure 2 A flow chart of setting a secure dynamic protection mode for different computer network application environments of the present invention;

[0072] Figure 3 A flowchart of a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model of the present invention;

[0073] Figure 4The invention establishes a network abnormal behavior feature library, compares it with the decision result of the model, adjusts the parameters of the model and optimizes the flow chart of the network abnormal behavior feature library. DETAILED DESCRIPTION

[0074] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0075] Reference Figure 1 As shown, a computer network security dynamic protection method based on fusion factors includes:

[0076] Set up security dynamic protection detection mode for different computer network application environments;

[0077] Based on the detection content of the security dynamic protection mode, set the corresponding security dynamic protection plan;

[0078] Based on the deep reinforcement learning model, a decision-making method for dynamic protection of computer network security is constructed;

[0079] According to the decision results of the deep reinforcement learning model, corresponding protection solutions are adopted to solve network risks;

[0080] Establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0081] It can be explained that for complex and changeable computer network application environments, by adopting different detection modes, the computer network can be accurately applied to various external environments, reducing the load on the computer processor; secondly, through the training and learning of deep reinforcement learning models, in the face of different abnormal behavior characteristics detected, through the reward value fed back by model training, the computer network’s dynamic and autonomous protection and autonomous handling of abnormal behavior network risks can be achieved, effectively improving the computer network’s dynamic security protection and early warning capabilities in different environments, and reducing network risks.

[0082] Reference Figure 2 As shown, the setting of the security dynamic protection mode for different computer network application environments specifically includes:

[0083] According to the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection, set the security dynamic protection detection mode according to different computer network application environments;

[0084] For application environments where computer network security requirements are not high and information needs to be kept confidential, the port number detection single mode is used;

[0085] For application environments with high requirements for computer network security and no need for accurate identification of traffic behavior, deep packet inspection single mode is used;

[0086] For computer network security application environments that require accurate identification of traffic behavior and fast processing speed, deep flow inspection single mode is used;

[0087] For other needs of computer network security, the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection modes are integrated to set a joint detection mode, and the detection results are adjusted by setting the weight of each detection mode;

[0088] Based on DS evidence theory, the proportion of port number detection, deep packet inspection, and deep flow inspection in the joint detection mode is calculated;

[0089] The weight expression for each detection is:

[0090]

[0091] Among them, y is the linear relationship between several detection methods, x1 is the use of port number detection, x2 is the use of deep packet inspection, x3 is the use of deep flow inspection, ε1 is the weight of port number detection, ε2 is the weight of deep packet inspection, and ε3 is the weight of deep flow inspection;

[0092] The DS evidence theory expression is:

[0093]

[0094] Among them, m(A) is the credibility of the A-th detection method, K is the normalization constant, and m i (A i ) is the probability of using the i-th feature item of the A-th detection method, and n is the number of feature items using the A-th detection method.

[0095] It can be explained that common computer network security detection includes three modes: port number detection, deep packet inspection, and deep flow inspection. Among them, port number detection is used to detect whether the computer network port is being used or to query which servers or programs are using the port; deep packet inspection is used to analyze and detect network traffic data packets based on the three aspects of network signatures, application gateways, and behavior patterns. Deep flow inspection is mainly used to detect the total number of data packets in the flow, total size, duration, average speed, and start and end time of the flow. By analyzing the characteristic items corresponding to the three detection modes, we can effectively determine the detection mode of computer network security protection, improve the processing speed of the computer, and make it more suitable for different computer network application environments.

[0096] Reference Figure 3 As shown, the decision-making method for building dynamic computer network security protection based on the deep reinforcement learning model specifically includes:

[0097] According to the standardized processing list of abnormal behavior, confirm the probability of selecting strategies between different processing options;

[0098] Based on the Markov chain algorithm, the state probability value at the future moment is confirmed according to the computer network's recognition of different detection content feature items;

[0099] Based on the standardized processing list of abnormal behaviors, calculate the cumulative reward feedback value corresponding to completing a decision;

[0100] Based on the deep reinforcement learning model, according to the Monte Carlo algorithm, it is confirmed that the computer network completes a decision corresponding to G in this state. k expectations;

[0101] Based on the deep reinforcement learning model, according to the SARSA algorithm, confirm that the computer network selects this action to complete a decision corresponding to G k expectations;

[0102] The probability expression of the selection strategy between different actions is as follows:

[0103] π(b|s)=P(B k =b|S k =s)

[0104] Among them, π(b|s) is the probability that the computer network selects the bth processing solution in state s, B k A collection of solutions for dealing with abnormal behavior characteristics of computer networks. k is the set of current states of the computer network;

[0105] The Markov chain algorithm is:

[0106] P[S k+1 |S k ]=P[S1|S2,S3,…,S i ,…,S k+1 ]

[0107] Among them, P[S k+1 |S k ] is the probability of the computer network being in the k+1th state in the kth state, S k+1 is the k+1th state of the computer network in the future, S k is the current k-th state of the computer network;

[0108] The calculation of the cumulative reward feedback value corresponding to a decision is expressed as:

[0109]

[0110] Among them, G k is the cumulative reward feedback value corresponding to the kth state of a decision made by the computer network, γ is the discount factor of the future impact of the computer network on the current state, G k+1 The cumulative reward feedback value corresponding to the k+1th state of a decision completed by the computer network, r k The reward feedback value corresponding to the kth state of a decision completed by the computer network;

[0111] The computer network completes a decision in this state corresponding to G k The expected expression is:

[0112]

[0113] in, G is the computer network corresponding to the kth state of this strategy k The updated expected value, V(S k ) is the corresponding G of the computer network in the kth state of this strategy k The expected value of , α is the learning rate, V(S k+1 ) is the corresponding G of the computer network in the k+1th state of this strategy k Expected value;

[0114] The computer network selects this action to complete a decision corresponding to G k The expected expression is:

[0115]

[0116] in, Select this action for the computer network to complete a decision. The corresponding G in the kth state k The updated expected value, Q(B k ,S k ) is the computer network that selects this action to complete a decision. The corresponding G in the kth state k The expected value of, β is the learning rate, Q(B k+1 ,S k+1 ) is the computer network that selects this action to complete a decision. In the k+1th state, the corresponding G k expected value.

[0117] It can be explained that the deep reinforcement learning model is a reward value obtained by an intelligent machine when choosing different processing methods under different states, and by judging the size of the reward value, it chooses the method with a higher reward value as the optimal method. Through the training and learning of the model, the purpose of dynamic and autonomous protection of computer network security can be effectively achieved, and the ability to actively deal with abnormal behavior risks can be improved. Among them, the reward value for selecting different methods is determined by judging the matching degree of solutions corresponding to different feature items, and the reward value can be optimized through the results of multiple training and learning of the model.

[0118] Reference Figure 4 As shown, the establishment of a network abnormal behavior feature library, by comparing with the decision results of the model, adjusting the parameters of the model and optimizing the network abnormal behavior feature library specifically includes:

[0119] Establish a network abnormal behavior feature library based on the abnormal behavior standardized processing list and the decision-making processing results of the model;

[0120] By comparing the record processing results of the network abnormal behavior feature library with the decision results of the model, the most suitable processing method is selected, and the parameters of the model are adjusted and the network abnormal behavior feature library is optimized according to the comparison results;

[0121] By judging whether the decision result of the model is better than the processing result of the feature library, if so, the processing result recorded in the feature library is optimized; if not, the parameters of the model need to be adjusted.

[0122] It can be explained that the training and learning of deep reinforcement learning models require certain network abnormal behavior feature samples. For complex, new, and highly influential network abnormal behavior features that cause model overload, it is necessary to establish a new list of corresponding feature samples, and then update the deep reinforcement learning model through model training and learning. By comparing the processing results recorded in the network abnormal behavior feature library with the decision results of the model, the most suitable processing method can be selected, and then the model parameters can be adjusted and the network abnormal behavior feature library can be optimized according to the comparison results. A dynamic protection mechanism for computer network security can be established in real time.

[0123] Further, based on the same inventive concept as the above-mentioned computer network security dynamic protection method based on fusion factors, this solution proposes a computer network security dynamic protection system based on fusion factors, including:

[0124] A detection and solution module, which is used to set a security dynamic protection detection mode for different computer network application environments; and to set a corresponding security dynamic protection solution based on the detection content of the security dynamic protection mode;

[0125] The protection and feedback module is used to construct a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model; adopt corresponding protection plans to solve security risks according to the decision results of the deep reinforcement learning model; establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0126] The detection and solution module includes:

[0127] A detection mode setting unit, wherein the detection mode setting unit is used to set a security dynamic protection detection mode for different computer network application environments;

[0128] A protection scheme setting unit is used to set a corresponding security dynamic protection scheme based on the detection content of the security dynamic protection mode.

[0129] The protection and feedback module includes:

[0130] A protection decision unit, which is used to build a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model;

[0131] A protection scheme unit, which is used to adopt a corresponding protection scheme to solve the security risk according to the decision result of the deep reinforcement learning model;

[0132] The feature library feedback unit is used to establish a network abnormal behavior feature library, and adjust the parameters of the model and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

[0133] To sum up, the advantages of the present invention are: different network security detection modes are set according to different computer network environments, and according to the detection content of the security dynamic protection mode, a computer network security dynamic protection decision-making method is constructed based on the deep reinforcement learning model. Through the learning and training of the model, the computer network's dynamic and autonomous security protection is realized, which effectively improves the computer network security protection capability and early warning capability in different environments, and reduces network risks.

[0134] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A computer network security dynamic protection method based on fusion factors, characterized in that: include: Set up security dynamic protection detection mode for different computer network application environments; Based on the detection content of the security dynamic protection mode, set the corresponding security dynamic protection plan; Based on the deep reinforcement learning model, a decision-making method for dynamic protection of computer network security is constructed; According to the decision results of the deep reinforcement learning model, corresponding protection solutions are adopted to solve network risks; Establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

2. According to claim 1, a computer network security dynamic protection method based on fusion factors is characterized in that: The setting of the security dynamic protection mode for different computer network application environments specifically includes: According to the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection, set the security dynamic protection detection mode according to different computer network application environments; For application environments where computer network security requirements are not high and information needs to be kept confidential, the port number detection single mode is used; For application environments with high requirements for computer network security and no need for accurate identification of traffic behavior, deep packet inspection single mode is used; For application environments that require accurate identification of traffic behavior and increased computer network processing speed, deep flow inspection single mode is used; For other needs of computer network security, the advantages and disadvantages of port number detection, deep packet detection, and deep flow detection modes are integrated to set a joint detection mode, and the detection results are adjusted by setting the weight of each detection mode; Based on DS evidence theory, the proportion of port number detection, deep packet inspection, and deep flow inspection in the joint detection mode is calculated; The weight expression for each detection is: Among them, y is the linear relationship between several detection methods, x1 is the use of port number detection, x2 is the use of deep packet inspection, x3 is the use of deep flow inspection, ε1 is the weight of port number detection, ε2 is the weight of deep packet inspection, and ε3 is the weight of deep flow inspection; The DS evidence theory expression is: Among them, m(A) is the credibility of the A-th detection method, K is the normalization constant, and m i (A i ) is the probability of using the i-th feature item of the A-th detection method, and n is the number of feature items using the A-th detection method.

3. A computer network security dynamic protection method based on fusion factors according to claim 2, characterized in that: The detection content based on the security dynamic protection mode and setting the corresponding security dynamic protection scheme specifically include: Based on the historical data of abnormal network behavior, determine the detection content feature items of each detection mode; For each detection mode's detection content feature items, set up a standardized processing plan for network abnormal behavior characteristics; Establish a standardized processing list for abnormal behaviors based on the detection content characteristics and standardized processing plans.

4. A computer network security dynamic protection method based on fusion factors according to claim 3, characterized in that: The decision-making method for building dynamic computer network security protection based on the deep reinforcement learning model specifically includes: According to the standardized processing list of abnormal behavior, confirm the probability of selecting strategies between different processing options; Based on the Markov chain algorithm, the state probability value at the future moment is confirmed according to the computer network's recognition of different detection content feature items; Based on the standardized processing list of abnormal behaviors, calculate the cumulative reward feedback value corresponding to completing a decision; Based on the deep reinforcement learning model, according to the Monte Carlo algorithm, it is confirmed that the computer network completes a decision corresponding to G in this state. k expectations; Based on the deep reinforcement learning model, according to the SARSA algorithm, confirm that the computer network selects this action to complete a decision corresponding to G k expectations; The probability expression of the selection strategy between different actions is as follows: π(b|s)=P(B k =b|S k (s) Among them, π(b|s) is the probability that the computer network selects the bth processing solution in state s, B k A collection of solutions for dealing with abnormal behavior characteristics of computer networks. k is the set of current states of the computer network; The Markov chain algorithm is: P[S k+1 |S k ]=P[S1|S2,S3,…,S i ,…,S k+1 ] Among them, P[S k+1 |S k ] is the probability of the computer network being in the k+1th state in the kth state, S k+1 is the k+1th state of the computer network in the future, S k is the current k-th state of the computer network; The calculation of the cumulative reward feedback value corresponding to a decision is expressed as: Among them, G k is the cumulative reward feedback value corresponding to the kth state of a decision made by the computer network, γ is the discount factor of the future impact of the computer network on the current state, G k+1 The cumulative reward feedback value corresponding to the k+1th state of a decision completed by the computer network, r k The reward feedback value corresponding to the kth state of a decision completed by the computer network; The computer network completes a decision in this state corresponding to G k The expected expression is: in, G is the computer network corresponding to the kth state of this strategy k The updated expected value, V(S k ) is the corresponding G of the computer network in the kth state of this strategy k The expected value of , α is the learning rate, V(S k+1 ) is the corresponding G of the computer network in the k+1th state of this strategy k Expected value; The computer network selects this action to complete a decision corresponding to G k The expected expression is: in, Select this action for the computer network to complete a decision. The corresponding G in the kth state k The updated expected value, Q(B k ,S k ) is the computer network that selects this action to complete a decision. The corresponding G in the kth state k The expected value of, β is the learning rate, Q(B k+1 ,S k+1 ) is the computer network that selects this action to complete a decision. In the k+1th state, the corresponding G k expected value.

5. A computer network security dynamic protection method based on fusion factors according to claim 4, characterized in that: According to the decision results of the deep reinforcement learning model, the corresponding protection solutions are adopted to solve the network risks, including: The methods for handling abnormal network behavior characteristics are divided according to the impact on network operation caused by abnormal network behavior characteristics and the simplicity of the handling methods; Based on the learning and training results of the deep reinforcement learning model, automatic decision-making is performed on common, simple, and insignificant network abnormal behavior features; Based on the learning and training results of the deep reinforcement learning model, early warning alarm methods are set up and abnormal network behavior characteristics that are complex, new, have a large impact and cause model overload are recorded.

6. A computer network security dynamic protection method based on fusion factors according to claim 5, characterized in that: The establishment of the network abnormal behavior feature library, adjusting the parameters of the model and optimizing the network abnormal behavior feature library by comparing with the decision result of the model specifically includes: Establish a network abnormal behavior feature library based on the abnormal behavior standardized processing list and the decision-making processing results of the model; By comparing the record processing results of the network abnormal behavior feature library with the decision results of the model, the most suitable processing method is selected, and the parameters of the model are adjusted and the network abnormal behavior feature library is optimized according to the comparison results; By judging whether the decision result of the model is better than the processing result of the feature library, if so, the processing result recorded in the feature library is optimized; if not, the parameters of the model need to be adjusted.

7. A computer network security dynamic protection system based on fusion factors, used to implement the computer network security dynamic protection method based on fusion factors as described in any one of claims 1 to 6, characterized in that: include: A detection and solution module, which is used to set a security dynamic protection detection mode for different computer network application environments; Based on the detection content of the security dynamic protection mode, set the corresponding security dynamic protection plan; The protection and feedback module is used to construct a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model; adopt corresponding protection plans to solve security risks according to the decision results of the deep reinforcement learning model; establish a network abnormal behavior feature library, and adjust the model parameters and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

8. A computer network security dynamic protection system based on fusion factors according to claim 7, characterized in that: The detection and solution module specifically includes: A detection mode setting unit, wherein the detection mode setting unit is used to set a security dynamic protection detection mode for different computer network application environments; A protection scheme setting unit is used to set a corresponding security dynamic protection scheme based on the detection content of the security dynamic protection mode.

9. A computer network security dynamic protection system based on fusion factors according to claim 8, characterized in that: The protection and feedback module specifically includes: A protection decision unit, which is used to build a decision-making method for dynamic protection of computer network security based on a deep reinforcement learning model; A protection scheme unit, which is used to adopt a corresponding protection scheme to solve the security risk according to the decision result of the deep reinforcement learning model; The feature library feedback unit is used to establish a network abnormal behavior feature library, and adjust the parameters of the model and optimize the network abnormal behavior feature library by comparing it with the decision results of the model.

Images