Network intrusion prevention method and device, equipment and storage medium

Abstract

The invention discloses a network intrusion prevention method, apparatus and device, and a storage medium. The method comprises the steps of extracting feature information representing network behaviors from collected network flow data; inputting the feature information into a reinforcement learning model, and performing reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion prevention strategy; and acquiring environment feedback corresponding to execution of the intrusion prevention strategy, and adjusting the reinforcement learning model according to the environment feedback. Through the reinforcement learning model, automatic learning is carried out from a large amount of network flow data and adjustment is carried out to cope with continuously changing network threats, so that the accuracy and efficiency of network intrusion prevention are improved, and through automatic optimization of the model, the requirement of manual intervention is reduced, and the operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a network intrusion defense method, device, equipment and storage medium. Background Art

[0002] In the field of network security, as network attack methods become increasingly complex and diverse, traditional intrusion prevention systems face huge challenges. In order to effectively defend network security, rule-based intrusion prevention systems and machine learning-based intrusion prevention systems are currently commonly used.

[0003] Existing rule-based intrusion prevention systems rely on fixed rule sets to identify and block attacks. These systems cannot adapt to changes in new attack patterns, resulting in limited defense effectiveness; while machine learning-based intrusion prevention systems, although they can improve detection capabilities through learning, often rely on a large amount of labeled data and lack sufficient generalization capabilities for unseen attack types. The methods corresponding to existing intrusion prevention systems all have certain limitations. They often cannot dynamically adapt to the ever-changing network environment and attack methods, resulting in poor defense results. Summary of the invention

[0004] The invention provides a network intrusion defense method to achieve effective defense of network security.

[0005] According to a first aspect of the present invention, there is provided a network intrusion defense method, comprising: extracting characteristic information characterizing network behavior from collected network traffic data;

[0006] Inputting the feature information into a reinforcement learning model, and performing reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion defense strategy;

[0007] Obtain environmental feedback corresponding to the execution of the intrusion defense strategy, and adjust the reinforcement learning model according to the environmental feedback.

[0008] Optionally, feature information representing network behavior is extracted from the collected network traffic data, including:

[0009] Extracting user behavior patterns, traffic size and access frequency from the collected network traffic data;

[0010] The user behavior model, the traffic size and the access frequency are used as characteristic information to characterize network behavior.

[0011] Among them, the user behavior pattern includes user access rules, user operation behavior and user network activity, the traffic size includes data packet size, traffic rate and traffic direction, and the access frequency includes the number of visits, access interval and access pattern.

[0012] Optionally, performing reinforcement learning on the feature information by using the reinforcement learning model to obtain an intrusion defense strategy includes:

[0013] Initializing the parameters and environment state of the reinforcement learning model;

[0014] Analyze the feature vector through the initialized reinforcement learning model to obtain the current network state;

[0015] The intrusion prevention strategy is determined according to the current network status and matching the existing rules, wherein the intrusion prevention strategy includes allowing traffic or blocking traffic.

[0016] Optionally, obtaining environmental feedback corresponding to the execution of the intrusion defense strategy, and adjusting the reinforcement learning model according to the environmental feedback;

[0017] Obtaining environmental feedback corresponding to the execution of the intrusion prevention strategy, wherein the environmental feedback includes the occurrence or non-occurrence of a security incident;

[0018] The parameters of the reinforcement learning model are adjusted according to the environmental feedback, and the existing rules are adjusted according to the environmental feedback.

[0019] Optionally, adjusting the parameters of the reinforcement learning model according to the environmental feedback includes:

[0020] When it is determined that the reinforcement learning model is a neural network model, adjusting the structure and weight of the neural network model according to the environmental feedback;

[0021] When it is determined that the reinforcement learning model is a policy gradient algorithm, the parameters of the policy function in the policy gradient algorithm are adjusted according to the environmental feedback.

[0022] Optionally, adjusting the existing rules according to the environmental feedback includes:

[0023] When the environment feedback indicates that no security incident has occurred, the existing firewall rules are enhanced and modified, the intrusion detection rules are upgraded and updated, or new security policy rules are issued;

[0024] When the environment feedback indicates that a security incident has occurred, the existing firewall rules are weakened and modified, the intrusion detection rules are downgraded and updated, or the existing security policy rules are deleted.

[0025] According to another aspect of the present invention, there is provided a network intrusion prevention device, comprising: a feature information extraction module, for extracting feature information representing network behavior from collected network traffic data;

[0026] An intrusion defense strategy acquisition module, used to input the feature information into a reinforcement learning model, and perform reinforcement learning on the feature information through the reinforcement learning model to acquire an intrusion defense strategy;

[0027] The reinforcement learning model adjustment module is used to obtain environmental feedback corresponding to the execution of the intrusion defense strategy and adjust the reinforcement learning model according to the environmental feedback.

[0028] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0029] at least one processor; and

[0030] a memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the method described in any embodiment of the present invention.

[0032] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method described in any embodiment of the present invention when executed.

[0033] The technical solution of the embodiment of the present invention automatically learns and adjusts from a large amount of network traffic data through a reinforcement learning model to cope with ever-changing network threats, thereby improving the accuracy and efficiency of network intrusion defense, and reduces the need for manual intervention and operating costs through automatic model optimization.

[0034] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a flow chart of a network intrusion defense method provided according to Embodiment 1 of the present invention;

[0037] Figure 2is a flow chart of a network intrusion defense method provided according to Embodiment 2 of the present invention;

[0038] Figure 3 is a schematic diagram of the structure of a network intrusion prevention device provided according to Embodiment 3 of the present invention;

[0039] Figure 4 It is a schematic diagram of the structure of an electronic device provided by Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Embodiment 1

[0043] Figure 1 A flowchart of a network intrusion defense method is provided for the first embodiment of the present invention. This embodiment is applicable to the situation where network security is defended. The method can be executed by a network intrusion defense device, which can be implemented in the form of hardware and / or software. Figure 1 As shown, the method includes:

[0044] Step S101, extracting characteristic information representing network behavior from collected network traffic data.

[0045] Optionally, characteristic information characterizing network behavior is extracted from the collected network traffic data, including: extracting user behavior patterns, traffic size and access frequency from the collected network traffic data; using user behavior models, traffic size and access frequency as characteristic information characterizing network behavior, wherein user behavior patterns include user access rules, user operation behaviors and user network activities, traffic size includes data packet size, traffic rate and traffic direction, and access frequency includes number of visits, access interval and access pattern.

[0046] Among them, in this embodiment, after collecting the network flow data in the current network, the data will be cleaned, and the invalid and erroneous data will be removed during the cleaning. Since the data includes various types of parameters, and the units of the parameters are different, the cleaned data will be standardized in this embodiment, that is, each parameter is converted into data with a unified format, and features are selected from the network flow data with a unified format. When performing feature selection, feature information that is meaningful for intrusion detection and characterizes network behavior is specifically selected from a large number of features of the data. For example, the extracted feature information specifically includes user behavior patterns, traffic size, and access frequency. Of course, this embodiment is only an example, and does not limit the specific content of the extracted feature information.

[0047] Specifically, user behavior patterns mainly include user access rules, user operation behaviors and user network activities. Among them, user access rules are to analyze the frequency, time, and pages visited by users when they visit websites or applications, which can be used to identify abnormal access patterns, such as frequent visits to specific pages, long-term visits, and abnormal access times. User operation behaviors are to analyze user operation behaviors in the system, such as logging in, accessing files, and executing commands, which can be used to identify abnormal operations, such as frequent login failures, access to sensitive files, and execution of dangerous commands. User network activities are to analyze user network activities, such as the connection protocol, accessed ports, and the amount of data transmitted, which can be used to identify abnormal network activities, such as using uncommon protocols, accessing unauthorized ports, and transmitting large amounts of data. Traffic size mainly includes packet size, traffic rate, and traffic direction. Among them, packet size is to analyze the size of each packet and identify abnormal packets, such as packets that are too large or too small. Traffic rate is to analyze the amount of data transmitted per unit time and identify abnormal traffic, such as burst traffic, continuous high traffic, etc. Traffic direction is to analyze the source and destination addresses of packets and identify abnormal traffic directions, such as traffic from unknown addresses and traffic to sensitive addresses. The access frequency mainly includes the number of accesses, the access interval and the access pattern, wherein the number of accesses is to analyze the number of times a resource is accessed within a unit time, and to identify abnormal accesses, such as frequent accesses to a website or a file, etc.; the access interval is to analyze the time interval between two accesses, and to identify abnormal accesses, such as multiple accesses in a short period of time, sudden access after a long period of non-access, etc.; the access pattern is to analyze the order and regularity of accesses, and to identify abnormal access patterns, such as accessing multiple resources in a specific order, accessing unrelated resources, etc. Of course, this embodiment is only an example, and does not limit the specific content of the feature information extracted from the collected network traffic data.

[0048] Step S102: input the feature information into a reinforcement learning model, and perform reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion defense strategy.

[0049] Optionally, inputting the feature information into the reinforcement learning model includes: converting the feature information into a feature vector; and inputting the feature vector into the reinforcement learning model.

[0050] Optionally, reinforcement learning is performed on feature information through a reinforcement learning model to obtain an intrusion defense strategy, including: initializing parameters and environmental states of the reinforcement learning model; analyzing feature vectors through the initialized reinforcement learning model to obtain the current network state; and determining an intrusion defense strategy based on matching existing rules according to the current network state, wherein the intrusion defense strategy includes allowing traffic or intercepting traffic.

[0051] Specifically, in this embodiment, after acquiring the feature information, a feature vector is constructed according to the feature information to convert it into a data form that can be recognized by the reinforcement learning model, and the feature vector is input into the reinforcement learning model for subsequent use by the reinforcement learning model. Among them, after acquiring the feature vector, the reinforcement learning model will perform reinforcement learning on the feature vector to obtain the intrusion defense strategy. When performing reinforcement learning, the reinforcement learning model first initializes the parameters and environmental state of the model, and the model parameters mainly depend on the reinforcement learning algorithm used. For example, when a neural network model is used, the parameters include the structure and weight of the neural network model; when a policy gradient algorithm is used, the parameters include the parameters of the policy function. Of course, this embodiment is only an example, and the specific content of the parameters of the model is not limited. In addition, the environmental state is the perception of the current network by reinforcement learning, which usually includes historical traffic data, current network status and security policies, wherein the historical traffic data includes traffic characteristics and security events, etc., the current network status includes network traffic, connection status and resource usage, etc., and the security policy includes the currently effective firewall rules and access control policies, etc. Of course, this embodiment is only an example, and the specific content of the parameters and environmental status is not limited.

[0052] Among them, in this embodiment, the parameters and environmental state of the reinforcement learning model are initialized to prepare for the subsequent reinforcement learning process. The feature vector is used as the input of the reinforcement learning model to provide the reinforcement learning model with key information about network traffic. The reinforcement learning model will analyze the input feature vector to obtain the current network state, and determine an action according to the current network state matching the existing rules, that is, determine the intrusion defense strategy, such as allowing traffic or intercepting traffic, etc. Of course, this embodiment is only an example, and does not limit the specific type of the determined intrusion defense strategy.

[0053] Step S103, obtaining environmental feedback corresponding to the execution of the intrusion defense strategy, and adjusting the reinforcement learning model according to the environmental feedback.

[0054] Optionally, obtain environmental feedback corresponding to the execution of the intrusion defense strategy, and adjust the reinforcement learning model based on the environmental feedback; obtain environmental feedback corresponding to the execution of the intrusion defense strategy, wherein the environmental feedback includes the occurrence of a security incident or the non-occurrence of a security incident; adjust the parameters of the reinforcement learning model based on the environmental feedback, and adjust the existing rules based on the environmental feedback.

[0055] Optionally, the parameters of the reinforcement learning model are adjusted according to environmental feedback, including: when the reinforcement learning model is determined to be a neural network model, the structure and weights of the neural network model are adjusted according to the environmental feedback; when the reinforcement learning model is determined to be a policy gradient algorithm, the parameters of the policy function in the policy gradient algorithm are adjusted according to the environmental feedback.

[0056] Optionally, existing rules are adjusted based on environmental feedback, including: when environmental feedback is that a security incident has occurred, existing firewall rules are enhanced, intrusion detection rules are upgraded, or new security policy rules are issued; when environmental feedback is that no security incident has occurred, existing firewall rules are weakened, intrusion detection rules are downgraded, or existing security policy rules are deleted.

[0057] Specifically, in this embodiment, the above-obtained intrusion defense strategy will be executed by the execution unit, and the environmental changes will be observed to obtain the environmental feedback corresponding to the execution of the intrusion defense strategy, such as whether a security incident occurs or not. The reinforcement learning model in this embodiment can continuously optimize the intrusion defense strategy by interacting with the environment, so the model in this embodiment is dynamically learned. The traditional intrusion defense system relies on preset rules or models, but the reinforcement learning algorithm used in this application can continuously learn and extract new attack patterns from network traffic data. And it can monitor the changes in the network environment in real time, including network traffic, connection status, resource usage, etc. In addition, this application can integrate threat intelligence data from different sources, such as malware databases, vulnerability information libraries, etc., and can automatically adjust the defense strategy according to the learned attack mode and threat intelligence, such as modifying firewall rules, updating intrusion detection rules, etc.

[0058] Among them, when the reinforcement learning model is adjusted according to the environmental feedback, on the one hand, the parameters of the model itself are adjusted, and on the other hand, the existing rules configured in the model are adjusted. For example, when the reinforcement learning model is a neural network, the structure and weight of the model are adjusted specifically, and when the policy gradient algorithm is used, the parameters of the policy function in the policy gradient algorithm are adjusted specifically. When the rules are adjusted, specifically when the environmental feedback is determined to be a security incident, it means that the intrusion defense strategy generated for the first time is sufficient to resist external intrusion, so the existing rules can be weakened or downgraded to reduce the storage pressure and data processing pressure of the device, and even the security policy rules that are not involved can be deleted; and when the environmental feedback is determined to be no security incident, it means that the intrusion defense strategy generated for the first time is not enough to resist external intrusion, so the existing rules can be enhanced or upgraded to improve the model's ability to resist external intrusion, and even new security policy rules can be issued according to the resistance requirements. Of course, this embodiment is only an example, and does not limit the specific adjustment method of the reinforcement learning model.

[0059] In a specific implementation, when a new type of network attack occurs, the traditional intrusion prevention system may not be able to identify this attack because it does not have corresponding rules or models. However, the present application can learn the characteristics of this new type of attack from the network traffic data, and automatically add rules corresponding to the characteristics, and automatically update the intrusion defense strategy based on the newly generated rules, so that when the new type of attack is encountered again in the future, an effective intrusion defense strategy can be generated based on the newly generated rules, and the new type of attack can be effectively defended based on the newly generated intrusion defense strategy, thereby ensuring the security of the network. Alternatively, when a new vulnerability appears, an attacker can exploit this vulnerability to attack. The traditional intrusion prevention system may not be able to update the rules or models in time, but the present application can obtain information about this vulnerability from the threat intelligence, and automatically update or add rules corresponding to the vulnerability, and automatically update the intrusion defense strategy based on the newly generated rules, thereby effectively preventing attackers from exploiting this vulnerability to carry out destructive attacks.

[0060] It is worth mentioning that the intrusion defense strategy optimization system based on artificial intelligence reinforcement learning in this embodiment has higher adaptability. It can monitor the changes of the network environment in real time and automatically adjust the defense strategy according to the latest attack mode and threat intelligence, thereby significantly improving the accuracy and timeliness of the defense system; Unlike the system based on static rules, the present invention adopts a reinforcement learning algorithm, which can continuously learn and optimize the defense strategy in the interaction with the network environment. This dynamic learning process enables the system to better adapt to complex and changeable network attack methods; it can effectively integrate threat intelligence data from different sources, and conduct comprehensive analysis and evaluation of intelligence through deep learning and data analysis technology, so as to more comprehensively identify and defend potential network security threats; due to the use of advanced reinforcement learning technology, the present application can more accurately identify attack behaviors and effectively reduce false positives and false negatives. This enables the defense system to reduce unnecessary interference and waste of resources while ensuring network security; the design takes into account real-time and high efficiency, and can extract features and make decisions as soon as network traffic is received, quickly respond to potential security threats, and ensure the stability and security of the network environment; through intelligent learning and decision-making processes, the present invention reduces the reliance on manual intervention when fighting network attacks, reduces operating costs, and improves the automation level of the system. In summary, the present invention has obvious advantages over the existing technologies in terms of adaptability, threat intelligence integration, warning accuracy, system efficiency and degree of automation.

[0061] In the implementation mode of the present application, a reinforcement learning model is used to automatically learn and adjust from a large amount of network traffic data to cope with ever-changing network threats, thereby improving the accuracy and efficiency of network intrusion defense, and reducing the need for manual intervention and operating costs through automatic model optimization.

[0062] Embodiment 2

[0063] Figure 2 This is a flow chart of a network intrusion defense method provided in Embodiment 2 of the present invention. This embodiment is based on the above embodiment and, after adjusting the reinforcement learning model according to environmental feedback, further includes: verifying the adjusted reinforcement learning model.

[0064] Step S201: extracting characteristic information representing network behavior from collected network traffic data.

[0065] Optionally, characteristic information characterizing network behavior is extracted from the collected network traffic data, including: extracting user behavior patterns, traffic size and access frequency from the collected network traffic data; using user behavior models, traffic size and access frequency as characteristic information characterizing network behavior, wherein user behavior patterns include user access rules, user operation behaviors and user network activities, traffic size includes data packet size, traffic rate and traffic direction, and access frequency includes number of visits, access interval and access pattern.

[0066] Step S202: input the feature information into a reinforcement learning model, and perform reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion defense strategy.

[0067] Optionally, inputting the feature information into the reinforcement learning model includes: converting the feature information into a feature vector; and inputting the feature vector into the reinforcement learning model.

[0068] Optionally, reinforcement learning is performed on feature information through a reinforcement learning model to obtain an intrusion defense strategy, including: initializing parameters and environmental states of the reinforcement learning model; analyzing feature vectors through the initialized reinforcement learning model to obtain the current network state; and determining an intrusion defense strategy based on matching existing rules according to the current network state, wherein the intrusion defense strategy includes allowing traffic or intercepting traffic.

[0069] Step S203: Obtain environmental feedback corresponding to the execution of the intrusion defense strategy, and adjust the reinforcement learning model according to the environmental feedback.

[0070] Optionally, obtain environmental feedback corresponding to the execution of the intrusion defense strategy, and adjust the reinforcement learning model based on the environmental feedback; obtain environmental feedback corresponding to the execution of the intrusion defense strategy, wherein the environmental feedback includes the occurrence of a security incident or the non-occurrence of a security incident; adjust the parameters of the reinforcement learning model based on the environmental feedback, and adjust the existing rules based on the environmental feedback.

[0071] Optionally, the parameters of the reinforcement learning model are adjusted according to environmental feedback, including: when the reinforcement learning model is determined to be a neural network model, the structure and weights of the neural network model are adjusted according to the environmental feedback; when the reinforcement learning model is determined to be a policy gradient algorithm, the parameters of the policy function in the policy gradient algorithm are adjusted according to the environmental feedback.

[0072] Optionally, existing rules are adjusted based on environmental feedback, including: when environmental feedback is that a security incident has occurred, existing firewall rules are enhanced, intrusion detection rules are upgraded, or new security policy rules are issued; when environmental feedback is that no security incident has occurred, existing firewall rules are weakened, intrusion detection rules are downgraded, or existing security policy rules are deleted.

[0073] Step S204, verifying the adjusted reinforcement learning model.

[0074] Specifically, in this implementation, after the reinforcement learning model is adjusted, the adjusted reinforcement learning model will be verified. During the verification, it is specifically to check whether the adjusted model can perform effective intrusion defense. For example, after obtaining the intrusion defense strategies before and after the adjustment for the same network traffic data, it is observed whether the two output intrusion defense strategies have changed. If the intrusion defense strategies remain unchanged, it means that the model adjustment failed to be verified. On the other hand, the environmental feedback corresponding to the two intrusion defense strategies will be checked. If the corresponding environmental feedback after the adjustment still shows that no security incident has occurred, it means that the model adjustment failed to be verified.

[0075] In this embodiment, when the model verification is completed, corresponding prompt information will be generated. For example, when the verification is passed, "Model adjustment is successfully verified" will be generated, and when the verification is not passed, "Model adjustment failed verification" will be generated, and the generated prompt information will be displayed on the human-computer interaction interface. In addition, a verification report can be generated based on the verification results on the interactive interface, and the verification report includes prompt information, verification failure items, estimated reasons, and recommended maintenance methods, etc. Of course, this embodiment is only an example, and does not limit the specific content of the verification report.

[0076] It is worth mentioning that in the case of verification failure, the present embodiment will also send a prompt message to the maintenance personnel's mobile terminal, so that the maintenance personnel can obtain the prompt message in time when they are not on duty. The reason for the failure of the verification may be a software failure, for example, incorrect setting parameters, incorrect upgrade process, etc., or it may be a hardware failure of the device. Of course, the present embodiment is only an example for illustration and does not limit the cause of the failure. By sending the prompt message to the mobile terminal, it is convenient to maintain the model in time.

[0077] In the implementation mode of the present application, a reinforcement learning model is used to automatically learn and adjust from a large amount of network traffic data to cope with ever-changing network threats, thereby improving the accuracy and efficiency of network intrusion defense, and reducing the need for manual intervention and operating costs through automatic model optimization.

[0078] Embodiment 3

[0079] Figure 3 This is a schematic diagram of the structure of a network intrusion prevention device provided by Embodiment 3 of the present invention. Figure 3As shown, the device includes: a feature information extraction module 310, an intrusion defense strategy acquisition module 320 and a reinforcement learning model adjustment module 330.

[0080] The feature information extraction module 310 is used to extract feature information representing network behavior from the collected network traffic data;

[0081] The intrusion defense strategy acquisition module 320 is used to input the feature information into the reinforcement learning model, and perform reinforcement learning on the feature information through the reinforcement learning model to obtain the intrusion defense strategy;

[0082] The reinforcement learning model adjustment module 330 is used to obtain environmental feedback corresponding to the execution of the intrusion defense strategy and adjust the reinforcement learning model according to the environmental feedback.

[0083] Optionally, a feature information extraction module is used to extract user behavior patterns, traffic size and access frequency from the collected network traffic data;

[0084] User behavior model, traffic size and access frequency are used as characteristic information to characterize network behavior.

[0085] Among them, user behavior patterns include user access rules, user operation behaviors and user network activities; traffic size includes data packet size, traffic rate and traffic direction; access frequency includes number of visits, access interval and access pattern.

[0086] Optionally, the intrusion defense strategy acquisition module includes a feature information conversion unit, which is used to convert the feature information into a feature vector;

[0087] Feed the feature vector into the reinforcement learning model.

[0088] Optionally, the intrusion defense strategy acquisition module includes an intrusion defense strategy acquisition unit, which is used to initialize the parameters and environment state of the reinforcement learning model;

[0089] The feature vector is analyzed through the initialized reinforcement learning model to obtain the current network state;

[0090] An intrusion prevention strategy is determined according to the current network status and matching existing rules, wherein the intrusion prevention strategy includes allowing traffic or blocking traffic.

[0091] Optionally, the reinforcement learning model adjustment module includes: an execution unit, used to obtain environmental feedback corresponding to the execution of the intrusion defense strategy, wherein the environmental feedback includes the occurrence of a security incident or the absence of a security incident;

[0092] The adjustment unit is used to adjust the parameters of the reinforcement learning model according to the environmental feedback, and to adjust the existing rules according to the environmental feedback.

[0093] Optionally, an adjustment unit, for adjusting the structure and weights of the neural network model according to environmental feedback when the reinforcement learning model is determined to be a neural network model;

[0094] When the reinforcement learning model is determined to be a policy gradient algorithm, the parameters of the policy function in the policy gradient algorithm are adjusted according to the environmental feedback.

[0095] Optionally, an adjustment unit is used to enhance and modify existing firewall rules, upgrade and update intrusion detection rules, or issue new security policy rules when the environment feedback indicates that no security incident has occurred;

[0096] When the environment feedback indicates that a security incident has occurred, the existing firewall rules are weakened or the intrusion detection rules are downgraded or updated, or the existing security policy rules are deleted.

[0097] The network intrusion prevention device provided in the embodiment of the present invention can execute the network intrusion prevention method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] Embodiment 4

[0099] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a network intrusion prevention method.

[0103] In some embodiments, the network intrusion prevention method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the network intrusion prevention method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the network intrusion prevention method in any other appropriate manner (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A network intrusion defense method, characterized in that: include: Extract characteristic information representing network behavior from collected network traffic data; Inputting the feature information into a reinforcement learning model, and performing reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion defense strategy; Obtain environmental feedback corresponding to the execution of the intrusion defense strategy, and adjust the reinforcement learning model according to the environmental feedback.

2. The method according to claim 1, characterized in that The extracting characteristic information representing network behavior from the collected network traffic data includes: Extracting user behavior patterns, traffic size and access frequency from the collected network traffic data; The user behavior model, the traffic size and the access frequency are used as characteristic information to characterize network behavior. Among them, the user behavior pattern includes user access rules, user operation behavior and user network activity, the traffic size includes data packet size, traffic rate and traffic direction, and the access frequency includes the number of visits, access interval and access pattern.

3. The method according to claim 1, characterized in that The step of inputting the feature information into a reinforcement learning model comprises: Converting the feature information into a feature vector; The feature vector is input into the reinforcement learning model.

4. The method according to claim 2, characterized in that: The step of performing reinforcement learning on the feature information through the reinforcement learning model to obtain an intrusion defense strategy includes: Initializing the parameters and environment state of the reinforcement learning model; Analyze the feature vector through the initialized reinforcement learning model to obtain the current network state; The intrusion prevention strategy is determined according to the current network status and matching the existing rules, wherein the intrusion prevention strategy includes allowing traffic or blocking traffic.

5. The method according to claim 4, characterized in that The step of obtaining environmental feedback corresponding to executing the intrusion defense strategy, and adjusting the reinforcement learning model according to the environmental feedback; Obtaining environmental feedback corresponding to the execution of the intrusion prevention strategy, wherein the environmental feedback includes the occurrence or non-occurrence of a security incident; The parameters of the reinforcement learning model are adjusted according to the environmental feedback, and the existing rules are adjusted according to the environmental feedback.

6. The method according to claim 5, characterized in that The adjusting the parameters of the reinforcement learning model according to the environmental feedback includes: When it is determined that the reinforcement learning model is a neural network model, adjusting the structure and weight of the neural network model according to the environmental feedback; When it is determined that the reinforcement learning model is a policy gradient algorithm, the parameters of the policy function in the policy gradient algorithm are adjusted according to the environmental feedback.

7. The method according to claim 5, characterized in that The adjusting the existing rules according to the environmental feedback includes: When the environment feedback indicates that no security incident has occurred, the existing firewall rules are enhanced and modified, the intrusion detection rules are upgraded and updated, or new security policy rules are issued; When the environment feedback indicates that a security incident has occurred, the existing firewall rules are weakened and modified, the intrusion detection rules are downgraded and updated, or the existing security policy rules are deleted.

8. A network intrusion prevention device, characterized in that: include: A feature information extraction module is used to extract feature information representing network behavior from the collected network traffic data; An intrusion defense strategy acquisition module, used to input the feature information into a reinforcement learning model, and perform reinforcement learning on the feature information through the reinforcement learning model to acquire an intrusion defense strategy; The reinforcement learning model adjustment module is used to obtain environmental feedback corresponding to the execution of the intrusion defense strategy and adjust the reinforcement learning model according to the environmental feedback.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

Images