A method for making bait based on unsupervised pre-training model

Abstract

The present invention relates to the field of network security technology application technology, and specifically discloses a production of bait based on an unsupervised pre-training model, including the following steps: S1, preparing relevant corpus; S2, data preprocessing; S3, loading pre-training generation model GPT; S4, downstream specific task training; S5, generating content according to keywords and real environment description; S6, bait file production; S7, generating bait file program binding word, pdf, web; S8, generating bait intelligently and sending it to the terminal. The present invention relies on the bait content generated by the powerful unsupervised pre-training model GPT-2, and combines it with further transformation, and implants a hidden tracking and tracing program in the document; unsupervised pre-training + supervised downstream task fine-tuning, while in NLP tasks, the upstream unsupervised pre-training tasks are mostly implemented using language models, and the downstream supervision tasks directly perform simple transformations on the upstream models.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and more specifically to a method for making a bait based on an unsupervised pre-training model. Background Art

[0002] Deception trapping is a honeynet technology composed of honeypots and bait technologies. It is an important means of active defense and can be a good supplement to the existing security protection system. According to the topological structure of the user's core business assets, it generates highly similar business scenarios to lure hackers to attack and confuse the information collected before the hacker attack, so as to detect hacker attacks in the first place.

[0003] Honey bait is generally a file, and its working principle is similar to that of a honeypot, which also tricks attackers into opening or downloading it. When hackers see files such as "XX second half work plan.docx", "XX environment operation and maintenance manual.pdf", and "employee salary list-20210630.xslx", they often find it difficult to resist the urge to download them, and thus fall into the trap of the defender. When the defender finds that the files here have traces of being opened or the attacker performs some operations based on the content of the honey bait file, they can trace the source and find the compromised device.

[0004] The existing problem is that these baits can only manually customize relevant bait content generation parameters through the terminal environment, and cannot adapt to the complex and changing network, equipment and production asset environment.

[0005] Therefore, an intelligent solution is needed to automatically generate baits that are related to the real environment or even similar to real files. Summary of the invention

[0006] In view of this, the present invention provides a method for making bait based on an unsupervised pre-training model, and a method for generating bait using a bait strategy for an active defense strategy to capture the attacker's behavioral intentions, introduce a honeynet, and perform tracking and tracing.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for making bait based on an unsupervised pre-training model, comprising the following steps:

[0008] S1. Prepare relevant corpus;

[0009] S2, data preprocessing;

[0010] S3, load the pre-trained generative model GPT;

[0011] S4, downstream task-specific training;

[0012] S5, generate content based on keywords and real environment description;

[0013] S6. Production of decoy documents;

[0014] S7, generate a decoy file program to bind word, pdf, web;

[0015] S8. Generate bait and send it to the terminal intelligently.

[0016] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, in S1 and S2, files accessed to the real network environment are used as training samples, data information is collected, and the data is encoded and segmented.

[0017] Preferably, in the above-mentioned method for making bait based on an unsupervised pre-training model, the data information includes corporate domain name, address book, and Internet assets.

[0018] Preferably, in the above-mentioned method for making bait based on an unsupervised pre-training model, in the training model of S4, a title or prompt related to the generated content is input to automatically generate bait content.

[0019] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, S8 also includes the following steps:

[0020] a. Process detection, rule conversion, and redirection;

[0021] b. Record the traces of adding, deleting and modifying the bait;

[0022] c. The generated bait content is not limited to word, pdf files or embedded web services, html.

[0023] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, in S3, for corpus u=(u1,.....un), GPT constructs a standard language model:

[0024]

[0025] Use the Transformer Decoder module and the feedforward neural network at the same time, and finally use softmax to output the distribution of the target word;

[0026] h0=UWe+Wp

[0027]

[0028]

[0029] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, the language model obtained through the pre-training in the first stage is fine-tuned for a specific task;

[0030] For a supervised dataset C, the data is a sequence χ1, ..., χm, and a label y; the sequence is input into the pre-trained model to obtain the output vector hlm, and then a linear layer is used to predict the label;

[0031]

[0032] The likelihood function to be maximized is:

[0033]

[0034] Preferably, in the above-mentioned method for making bait based on an unsupervised pre-trained model, the pre-trained generation model uses a GPT-2 model, which is an upgraded version of GPT, wherein GPT-2 constructs a data set WebText with a total of eight million documents and 40GB.

[0035] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, the GPT-2 model uses a huge network architecture, with a maximum of 48 layers of Transformer, 1542M parameters, and 1600 dimensions.

[0036] It can be seen from the above technical solutions that compared with the prior art, the present invention relies on the bait content generated by the powerful unsupervised pre-training model GPT-2, and combines it with further transformation, such as implanting a hidden tracking and tracing program in Word documents and PDF documents; GPT (Generative Pre-Training) is a typical two-stage model: unsupervised pre-training + supervised fine-tuning of downstream tasks. In NLP tasks, the upstream unsupervised pre-training tasks are mostly implemented using language models, and the downstream supervised tasks can be used by simply modifying the upstream models; in order to effectively capture the semantic information in the text, GPT uses a unidirectional Transformer Decoder module to build a standard language model, and then uses the network architecture and parameters obtained by pre-training to fine-tune the downstream supervised tasks, achieving good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 The accompanying drawing is a schematic structural diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Please see attached Figure 1 , is a method for making bait based on an unsupervised pre-training model disclosed in the present invention, which specifically includes the following steps:

[0041] S1. Prepare relevant corpus;

[0042] S2, data preprocessing;

[0043] S3, load the pre-trained generative model GPT;

[0044] S4, downstream task-specific training;

[0045] S5, generate content based on keywords and real environment description;

[0046] S6. Production of decoy documents;

[0047] S7, generate a decoy file program to bind word, pdf, web;

[0048] S8. Generate bait and send it to the terminal intelligently.

[0049] In S1 and S2, files accessed in a real network environment are used as training samples, data information is collected, and the data is encoded and segmented;

[0050] The data information includes enterprise domain name, address book, and Internet assets;

[0051] In the training model of S4, a title or prompt related to the generated content is input to automatically generate bait content;

[0052] S8 also includes the following steps:

[0053] a. Process detection, rule conversion, and redirection;

[0054] b. Record the traces of adding, deleting and modifying the bait;

[0055] c. The generated bait content is not limited to word, pdf files or embedded web services, html.

[0056] In S3, the training model (PT2 has 1.5 billion parameters, and it takes about 4 hours to face 8M corpus, which is still a feasible solution; for corpus u=(u1,.....un), GPT builds a standard language model:

[0057]

[0058] Use the Transformer Decoder module and the feedforward neural network at the same time, and finally use softmax to output the distribution of the target word;

[0059] h0=UWe+Wp

[0060]

[0061]

[0062] Fine-tuning is performed on the language model obtained through the first stage of pre-training for specific tasks;

[0063] For a supervised dataset C, the data is a sequence χ1, ..., χm, and a label y; the sequence is input into the pre-trained model to obtain the output vector hlm, and then a linear layer is used to predict the label;

[0064]

[0065] The likelihood function to be maximized is:

[0066]

[0067] Preferably, in the above-mentioned method for making bait based on unsupervised pre-training model, the GPT-2 model is used in the present invention, which is an upgraded version of GPT. In comparison, GPT-2 has the following improvements:

[0068] 1. We built a larger dataset, WebText, which covers a wide range of fields, with a total of 8 million documents and 40GB;

[0069] 2. Use a more massive network architecture: up to 48 layers of Transformer, 1542M parameters, 1600 dimensions;

[0070] 3. GPT-2 proposes that the model obtained by such training can also perform well in zero-shot situations, thus proving the significance of language model pre-training.

[0071] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0072] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for making bait based on an unsupervised pre-training model, characterized in that: The steps include: S1. Prepare relevant corpus; S2, data preprocessing; S3, load the pre-trained generative model GPT; S4, downstream task-specific training; S5, generate content based on keywords and real environment description; S6. Production of decoy documents; S7, generate a decoy file program to bind word, pdf, web; S8, generate bait and send it to the terminal intelligently; Among them, in S3, for the corpus u=(u1 ,.....un), GPT constructs a standard language model: ； Use the Transformer Decoder module and the feedforward neural network at the same time, and finally use softmax to output the distribution of the target word; h0=UWe+Wp ； Fine-tuning is performed on the language model obtained through the first stage of pre-training for specific tasks; For a supervised dataset C, the data is a sequence χ1, ..., χm, and a label y; the sequence is input into the pre-trained model to obtain the output vector hlm, and then a linear layer is used to predict the label; ； The likelihood function to be maximized is: 。 2. The method for making bait based on an unsupervised pre-training model according to claim 1, characterized in that: In S1 and S2, files accessed to a real network environment are used as training samples, data information is collected, and the data is encoded and segmented.

3. A method for making bait based on an unsupervised pre-training model according to claim 2, characterized in that: The data information includes enterprise domain name, address book, and Internet assets.

4. The method for making bait based on an unsupervised pre-training model according to claim 1, characterized in that: In the training model of S4, a title or prompt related to the generated content is input to automatically generate bait content.

5. The method for making bait based on an unsupervised pre-training model according to claim 1, characterized in that: S8 also includes the following steps: a. Process detection, rule conversion, and redirection; b. Record the traces of adding, deleting and modifying the bait; c. The generated bait content is not limited to word, pdf files or embedded web services, html.

6. The method for making bait based on an unsupervised pre-training model according to claim 1, characterized in that: The GPT-2 model used in the pre-trained generation model is an upgraded version of GPT. GPT-2 constructs a data set WebText with a total of eight million documents and 40GB.

7. The method for making bait based on an unsupervised pre-training model according to claim 6, characterized in that: The GPT-2 model uses a huge network architecture, with a maximum of 48 Transformer layers, 1542M parameters, and 1600 dimensions.

Images