Immune-like safe antibody generation method and system based on parallel associated network reinforcement learning

Abstract

The invention discloses an immune-like safe antibody generation method and system based on parallel associated network reinforcement learning, and belongs to the technical field of network security. The method comprises the following steps: constructing antigen data according to network threat data in combination with a coding technology, and splicing the antigen data to construct an antigen sequence; a generative network # imgabs0 # and a discriminant network # imgabs1 # of adversarial learning are loaded in a parallel associated network, the input of the generative network is an antigen sequence, the output of the generative network is a candidate antibody sequence, the generative network # imgabs2 # and the discriminant network # imgabs3 # are trained under an adversarial learning framework, the generative network and the discriminant network are alternately updated, and the candidate antibody sequence is obtained. The loss of the generation network and the loss of the discrimination network are stable; evaluating affinity scores of the candidate antibody sequences and the corresponding antigen sequences, and selecting the candidate antibody sequence with the highest score as a specific antibody; the problems that in the prior art, network threat detection is difficult, and defense system type immunocompetence is deficient are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network security, and relates to a method for generating quasi-immune security antibodies based on parallel companion network reinforcement learning, which can be applied to unknown threat immunity and dynamic defense in the computer field. Background Art

[0002] Quasi-immune security is a network security concept inspired by the biological immune system. It aims to enhance the network's defense capabilities to combat evolving cyber threats by simulating the recognition and response mechanisms of the immune system in organisms. The immune system is a complex biological system in organisms that is responsible for identifying and responding to various pathogens, such as bacteria, viruses, and other external invading substances. Quasi-immune security attempts to learn from the working principles of the immune system and apply it to the field of network security to enhance the network's defense and recovery capabilities to cope with evolving and diverse cyber threats. In this article, 'antibodies' refer to a set of network threat features generated through reinforcement learning and adversarial deduction. These 'antibodies' are designed to detect, block, or respond to cyber threats, such as viruses or malware, and can achieve highly targeted, fast-response, and highly accurate cyber threat perception and defense. Quasi-immune security attempts to generate specific antibodies that have high specificity and affinity to identify and respond to specific cyber threats.

[0003] The field of quasi-immune security faces a series of challenges and demands. Cyber ​​threats continue to evolve and diversify, and attackers adopt new attack strategies and tools to make networks more vulnerable. Therefore, quasi-immune security needs to constantly adapt and evolve to detect and combat new threats. Quasi-immune security requires real-time monitoring of network traffic and log data, as well as rapid identification of potential threats and attacks. Therefore, rapid response is key to preventing the spread of attacks. Quasi-immune security requires the generation of highly specific antibodies to distinguish between normal network activities and potential attacks. Antibody generation methods must be highly accurate to avoid false positives and misjudgments. However, traditional antibody generation methods usually rely on laboratory techniques, which are complex and time-consuming and may be limited by antibody diversity. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and propose a method for generating quasi-immune safety antibodies based on parallel companion network reinforcement learning, which is used to solve the problems of difficulty in detecting network threats and lack of quasi-immune capabilities of the defense system in the prior art.

[0005] The present invention is achieved through the following technical solutions: A method for generating immune-safe antibodies based on parallel companion network reinforcement learning, comprising the following steps: Step 1: construct antigen data based on network threat data and combined with encoding technology, and splice the antigen data to construct an antigen sequence; Step 2: Load the adversarial learning generative network in the parallel companion network and the discriminant network The input of the generative network is the antigen sequence, and the output is the candidate antibody sequence. In the adversarial learning framework, the generative network and the discriminant network Perform training, alternately updating the generator network and the discriminator network until the loss of the generator network and the loss of the discriminator network are stable; Step 3: Evaluate the affinity scores of the candidate antibody sequences and the corresponding antigen sequences, and select the candidate antibody sequence with the highest score as the specific antibody.

[0006] Preferably, the network threat data includes network traffic data, log files, malicious code features, antivirus engine output and threat intelligence data.

[0007] Preferably, the method for constructing the antigen sequence is as follows: The network threat data is used as the initial antigen, the initial antigen is encoded to obtain the antigen data, and the first part of the antigen data is spliced ​​to obtain the antigen sequence.

[0008] Preferably, the initial antigen is binary-encoded using One-Hot encoding to obtain binary antigen data, and the first bits of the binary antigen data are concatenated to obtain the antigen sequence.

[0009] Preferably, in step 2, the antigen sequence is constructed as an antigen, a random noise vector is generated using a Gaussian distribution, and the antigen and the random noise vector are concatenated as input to the generation network.

[0010] Preferably, the generation network uses a loss function and a regularization term to optimize the generated candidate antibody sequences; Loss function for generating the network is the adversarial loss, and the formula is as follows:

[0011] in, True antigen the antigen vector sampled from the distribution; represents a random noise vector sampled from a noise distribution , Used to determine whether the input sequence is a real antibody sequence; Using regularization term To prevent overfitting, while ensuring the diversity of generated antibodies, control the weight of the regularization term in the total loss, and limit the excessive increase of the generated network parameters.

[0012] Preferably, the loss function formula of the discriminant network is as follows:

[0013] in, True antigen The samples drawn from the distribution, Represents candidate antibody sequences The samples drawn from the distribution, Represents the probability that the input sample is identified as a true antibody.

[0014] Preferably, the training method of the generator network and the discriminator network is as follows: First, the generation network parameters are fixed to generate candidate antibody sequences, and the discriminant network parameters are updated; then the discriminant network parameters are fixed to minimize the loss of the generation network, and the generation network parameters are updated. The above steps are repeated until the loss of the generation network and the loss of the discriminant network are stable and the generation network successfully outputs the candidate antibody sequence.

[0015] Preferably, in step 3, SVM is used to score the affinity between the selected antibody sequence and the antigen.

[0016] A system for generating immune-safe antibodies based on parallel companion network reinforcement learning, comprising: The antigen module is used to construct antigen data based on network threat data and combined with encoding technology, and to splice the antigen data to construct an antigen sequence; Antibody module, used to load the generative network of adversarial learning in the parallel companion network and the discriminant network The input of the generative network is the antigen sequence, and the output is the candidate antibody sequence. In the adversarial learning framework, the generative network and the discriminant network Perform training, alternately updating the generator network and the discriminator network until the loss of the generator network and the loss of the discriminator network are stable; The selection module is used to evaluate the affinity scores of candidate antibody sequences and corresponding antigen sequences, and select the candidate antibody sequence with the highest score as the specific antibody.

[0017] Compared with the prior art, the present invention has the following beneficial technical effects: The method for generating immune-like safety antibodies based on parallel companion network reinforcement learning provided by this application, combined with the reinforcement learning algorithm, enables the antibody library to autonomously learn and adjust according to the real-time threat situation to achieve an adaptive defense strategy. This adaptive capability can respond to new attacks and unknown threats in a timely manner, improve the security and adaptability of the system, and can cover threats of different types and variants, providing comprehensive defense capabilities. Finally, the method of parallel companion network reinforcement learning can improve the efficiency and real-time performance of network security defense, reduce response time, and quickly adapt to changing threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 The flowchart of the method for generating immune-safe antibodies based on parallel companion network reinforcement learning of the present invention; Figure 2 Schematic diagram of the SVM neural network of the present invention; Figure 3 A graph for determining the affinity threshold of the antibody of the present invention; Figure 4 The figure is a sequence diagram of the antibody of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0022] The present invention uses a method of loading adversarial learning in a parallel companion network and generating specific immune antibodies using reinforcement learning to adapt to the continuous evolution and diversification of network threats. The parallel companion network is a simulated network environment that combines cloud virtualization technology and network slicing technology. It is used to simulate various scenarios in actual networks, help study and test network security mechanisms, and provide a specialized "antibody" breeding ground for protected networks. By loading adversarial learning AI attacks in parallel companion networks, new attack modes are changed in advance in network attack and defense confrontation, and machine learning methods are used in adversarial learning to find and learn possible attack methods and their defense methods in advance. The generative network and the discriminative network are two key components in adversarial learning. The generative network is responsible for generating candidate antibody sequences, while the discriminative network evaluates these sequences to determine their effectiveness. The generative network is used to generate candidate antibody sequences, and it can generate multiple sequences from random noise or other inputs. These generated sequences may contain a series of potential antibodies, but they do not necessarily meet the requirements of specificity and affinity. The task of the discriminative network is to evaluate these generated antibody sequences to determine which sequences have the required specificity and affinity. The discriminant network compares the generated antibodies with known antibodies or antigens to identify the antibodies that meet the requirements. Reinforcement learning algorithms are used to optimize the strategy of the generation network so that it can generate better antibody sequences. Vaccines obtained from different parallel adversarial networks can form a vaccine library, which will gain the initiative for network defense.

[0023] See also Figure 1-4 , a method for generating immune-safe antibodies based on parallel companion network reinforcement learning, comprising the following steps: Step 1: According to the network threat data antigen, the antigen is converted into antigen data using encoding technology, and the antigen data is spliced ​​to construct an antigen sequence.

[0024] (1.1) Real-time perception and monitoring of the network environment, collection of network traffic data (including source and destination addresses, port numbers, protocol types, size of transmitted data, etc.), log files (such as operating system logs, network device logs, application logs, etc., used to record network activities and events), malicious code features (including feature codes or behavior pattern descriptions of viruses, Trojans, and malware), threat intelligence data (including known attack signatures, vulnerability information, and hacker behavior features), etc. These feature data are the initial antigens.

[0025] (1.2) The collected initial antigens are converted into binary antigen data using binary coding. The first bit of the binary antigen data is concatenated, and the concatenated binary antigen data is the antigen sequence, where the antigen sequence contains M initial antigens and has a total length of N.

[0026] Use One-Hot encoding to convert the antigen sequence into an N×M matrix, where each row corresponds to the position of an antigen in the sequence, and each column corresponds to the type of antigen. The value at the corresponding position is 1, indicating the antigen type at that position, and the other positions are 0. Feature extraction is performed on the encoded N×M matrix. For example, the time of each antigen type, the length of the sequence, the relationship between adjacent antigen positions, etc. can be calculated. Assuming that K features are extracted from the encoded N×M matrix, these features can be represented as a K-dimensional feature vector. This k-dimensional feature vector is the antigen. Through the above steps, the antigen sequence can be constructed into an antigen , used for subsequent model training and prediction.

[0027] Step 2: Load the adversarial learning generative network in the parallel companion network and the discriminant network The input of the generative network is the antigen , the output is the candidate antibody sequence .

[0028] (2.1) Replicate the topology of the real network environment, including nodes, communication links, configuration policies, traffic logs, and threat data, to generate a parallel companion network. The parallel companion network is a replica of the real network environment, simulating the operation of the real network. The parallel companion network collects data from the real network in real time and synchronizes the same state as the real network.

[0029] (2.2) Loading the Generative Network of Adversarial Learning into the Parallel Companion Network The generated network consists of an input layer, a hidden layer, and an output layer. The hidden layer consists of three layers, with 512, 256, and 128 neurons in each layer, respectively. The first two hidden layers are fully connected layers. Use Gaussian distribution to generate random noise vectors , with antigen Concatenate to form the input vector of the generated network , . The concatenated input vector Passed to the input layer of the generative network, the candidate antibody sequence is output by the output layer through the hidden layer .

[0030] The generated candidate antibody sequences are optimized using a loss function and a regularization term. The loss function of the generated network is the adversarial loss, which means that the antibody sequence generated by the generative network can deceive the discriminative network. The formula is as follows:

[0031] in, True antigen the antigen vector sampled from the distribution; represents a random noise vector sampled from a noise distribution , Used to determine whether the input sequence is a real antibody sequence.

[0032] Adding regularization terms To prevent overfitting and ensure the diversity of generated antibodies, use The regularization formula to suppress excessive growth of model parameters is as follows:

[0033] in, Represents the generation of network parameters The square of the norm, which is the sum of the squares of each parameter. is the regularization coefficient, which controls the weight of the regularization term in the total loss. By reducing the sum of squares of the parameters, the excessive increase of the generated network parameters is limited to prevent the model from overfitting. Combining the loss function of the generated network with the regularization term, the total loss function is defined as follows:

[0034] Through the above steps, antigens can be used to generate candidate antibody sequences. , antibodies are generated by subsequent steps.

[0035] (2.3) Loading the discriminant network of adversarial learning into the parallel companion network The discriminant network consists of an input layer, a hidden layer, and an output layer. The hidden layer consists of two layers, with 256 and 128 neurons in each layer, respectively. The first hidden layer is a fully connected layer. and candidate antibody sequences generated by the generative network It is passed to the first layer of the discriminant network, and a binary classification output is output by the output layer through the hidden layer, indicating whether the input candidate antibody sequence is real or generated. The loss function formula of the discriminant network is as follows:

[0036] in, True antigen The samples drawn from the distribution, Represents candidate antibody sequences The samples drawn from the distribution, Represents the probability that the input sample is identified as a true antibody.

[0037] In the adversarial learning framework, the generator network and the discriminator network are updated alternately.

[0038] The training steps are as follows: First, fix the parameters of the generative network to generate candidate antibody sequences and update the parameters of the discriminative network. Then fix the parameters of the discriminative network, minimize the loss of the generative network, and update the parameters of the generative network. Repeat the above steps until the loss of the generative network and the loss of the discriminative network are stable, which means that the generative network successfully generates a high-quality candidate antibody sequence Y.

[0039] Step 3: Use SVM to evaluate the affinity and effectiveness of the generated candidate antibody sequences. Input candidate antibody sequence Y and antigen , output a score that measures the affinity of the candidate antibody sequence to the antigen, and select the candidate antibody sequence with the highest score as the specific antibody.

[0040] (3.1) Reference Figure 2 , input candidate antibody sequence Y and antigen Train the SVM model to learn the relationship between antibody and antigen affinity. The goal of SVM is to find an optimal hyperplane that maximizes the projection distance of samples of different categories on the hyperplane. The formula is as follows:

[0041] Constraints: .in, : normal vector of the hyperplane; : bias term; : Regularization parameter, used to balance the complexity of the model and the training error; : A slack variable used to allow samples to be within the hyperplane error range. The training goal of the SVM model is to minimize the objective function while satisfying the constraints and finding the optimal hyperplane.

[0042] Reference Figure 3 , use the test set data to evaluate the affinity and effectiveness of the antibody sequence to the antigen for the trained SVM model. The formula is:

[0043] in, : Parameters of the companion network; : The number of training samples; : The predicted value of the companion network; : The real affinity label; : Loss function, used to measure the difference between the predicted value and the true label; the training goal of the companion network is to minimize the loss function so that the predicted value is as close to the true label as possible. Through the above steps, the affinity of the generated candidate antibody sequences is evaluated, and the candidate antibody sequence with the highest score is selected as the specific antibody.

[0044] (3.2) Add the selected specific antibodies to the antibody library and manage and update the antibody library. The antibody library should contain diverse antibodies to cover different types of threats.

[0045] The present invention proposes a method for generating quasi-immune safety antibodies based on parallel companion network reinforcement learning, which is used for threat detection and defense in the field of network security. This method realizes the dynamic generation of specific antibodies to cope with evolving and diverse network threats by utilizing parallel companion networks and reinforcement learning algorithms. The specific steps include environmental perception and threat identification, antibody generation, antibody selection, and antibody library construction. Compared with the prior art, the present invention has the advantages of adaptive dynamic defense capability, diversified antibody library covering multiple threats, and efficient network security defense.

[0046] Based on the above-mentioned method for generating immune-like safe antibodies by parallel companion network reinforcement learning, the present application also provides a system for generating immune-like safe antibodies based on parallel companion network reinforcement learning, which may include: The antigen module is used to construct antigen data based on network threat data and combined with encoding technology, and to splice the antigen data to construct an antigen sequence; Antibody module, used to load the generative network of adversarial learning in the parallel companion network and the discriminant network The input of the generative network is the antigen sequence, and the output is the candidate antibody sequence. In the adversarial learning framework, the generative network and the discriminant network Perform training, alternately updating the generator network and the discriminator network until the loss of the generator network and the loss of the discriminator network are stable; The selection module is used to evaluate the affinity scores of candidate antibody sequences and corresponding antigen sequences, and select the candidate antibody sequence with the highest score as the specific antibody.

[0047] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0048] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0049] An electronic device provided in an embodiment of the present application includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method for generating immune-like safe antibodies based on parallel companion network reinforcement learning as described in any of the above embodiments are implemented.

[0050] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize the communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).

[0051] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the method for generating immune-like safe antibodies based on parallel companion network reinforcement learning as described in any of the above embodiments are implemented.

[0052] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0053] For the description of the relevant parts of the immune-safe antibody generation system based on parallel companion network reinforcement learning, electronic device and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the immune-safe antibody generation method based on parallel companion network reinforcement learning provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0054] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for generating immune-safe antibodies based on parallel companion network reinforcement learning, characterized in that: The following steps are involved: Step 1: construct antigen data based on network threat data and combined with encoding technology, and splice the antigen data to construct an antigen sequence; Step 2: Load the adversarial learning generative network in the parallel companion network and the discriminant network The input of the generative network is the antigen sequence, and the output is the candidate antibody sequence. In the adversarial learning framework, the generative network and the discriminant network Perform training, alternately updating the generator network and the discriminator network until the loss of the generator network and the loss of the discriminator network are stable; Step 3: Evaluate the affinity scores of the candidate antibody sequences and the corresponding antigen sequences, and select the candidate antibody sequence with the highest score as the specific antibody.

2. According to claim 1, a method for generating immune-safe antibodies based on parallel companion network reinforcement learning is characterized in that: The network threat data includes network traffic data, log files, malicious code characteristics, antivirus engine output and threat intelligence data.

3. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: The method for constructing the antigen sequence is as follows: The network threat data is used as the initial antigen, the initial antigen is encoded to obtain the antigen data, and the first part of the antigen data is spliced ​​to obtain the antigen sequence.

4. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 3, characterized in that: One-Hot encoding is used to binary encode the initial antigen to obtain binary antigen data, and the first bit of the binary antigen data is concatenated to obtain the antigen sequence.

5. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: In step 2, the antigen sequence is constructed as an antigen, a random noise vector is generated using a Gaussian distribution, and the antigen and the random noise vector are concatenated as the input of the generation network.

6. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: The generation network optimizes the generated candidate antibody sequences using a loss function and a regularization term; Loss function for generating the network is the adversarial loss, and the formula is as follows: in, True antigen the antigen vector sampled from the distribution; represents a random noise vector sampled from a noise distribution , Used to determine whether the input sequence is a real antibody sequence; Using regularization term To prevent overfitting, while ensuring the diversity of generated antibodies, control the weight of the regularization term in the total loss, and limit the excessive increase of the generated network parameters.

7. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: The loss function formula of the discriminant network is as follows: in, True antigen The samples drawn from the distribution, Represents candidate antibody sequences The samples drawn from the distribution, Represents the probability that the input sample is identified as a true antibody.

8. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: The training methods of the generating network and the discriminating network are as follows: First, the generation network parameters are fixed to generate candidate antibody sequences, and the discriminant network parameters are updated; then the discriminant network parameters are fixed to minimize the loss of the generation network, and the generation network parameters are updated. The above steps are repeated until the loss of the generation network and the loss of the discriminant network are stable and the generation network successfully outputs the candidate antibody sequence.

9. The method for generating immune-safe antibodies based on parallel companion network reinforcement learning according to claim 1, characterized in that: In step 3, SVM is used to score the affinity between the selected antibody sequence and the antigen.

10. A system for generating immune-safe antibodies based on parallel companion network reinforcement learning, characterized in that: include: The antigen module is used to construct antigen data based on network threat data and combined with encoding technology, and to splice the antigen data to construct an antigen sequence; Antibody module, used to load the generative network of adversarial learning in the parallel companion network and the discriminant network The input of the generative network is the antigen sequence, and the output is the candidate antibody sequence. In the adversarial learning framework, the generative network and the discriminant network Perform training, alternately updating the generator network and the discriminator network until the loss of the generator network and the loss of the discriminator network are stable; The selection module is used to evaluate the affinity scores of candidate antibody sequences and corresponding antigen sequences, and select the candidate antibody sequence with the highest score as the specific antibody.

Images