Password guessing model compression method based on generative adversarial network

Abstract

The invention discloses a password guessing model compression method based on a generative adversarial network, and the method comprises the steps: constructing a model compression scheme for a PassGAN, namely, employing a knowledge distillation method, firstly determining a distillation mechanism and type, constructing a simplified student model, and training and testing the PassGAN through employing a preprocessed password set; and taking the trained PassGAN as a teacher model, distilling the learned password space distribution into the student model, and enabling the student model to simulate the output or characteristics of the teacher model so as to learn the generalization ability of the teacher model. According to the method, while the password guessing capability is ensured, the storage and computing resource requirements of the model are effectively reduced, the reasoning speed and efficiency of the model are improved, and the PassGAN is promoted to be more efficient and wider in application.

Description

Technical Field

[0001] This invention relates to a technology in the field of information security, specifically a compression method for a password guessing model based on generative adversarial networks. Background Technology

[0002] Text-based password cracking primarily utilizes dictionary attacks. PassGAN, a password guessing model trained to generate a password dictionary, can achieve high-quality password guessing. However, its high complexity makes it difficult to run on lightweight terminals, thus requiring the development of appropriate model compression schemes to reduce model complexity. Summary of the Invention

[0003] This invention addresses the problems of high computational redundancy, large space consumption, high complexity, and limited application scenarios in existing technologies. It proposes a password guessing model compression method based on generative adversarial networks, which effectively reduces the storage and computing resource requirements of the model while ensuring password guessing capabilities, improves the inference speed and efficiency of the model, and promotes more efficient and wider application of PassGAN.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a compression method for a password guessing model based on generative adversarial networks, comprising:

[0006] Step 1) To obtain a lightweight model with strong generalization ability, a model compression scheme for PassGAN is constructed, that is, the knowledge distillation method is adopted. First, the distillation mechanism and category are determined, and then a simplified student model is constructed.

[0007] The distillation mechanism described above employs offline distillation that does not require additional training of the teacher model. The teacher model has already been pre-trained before the student model is trained, which is a one-way knowledge transfer.

[0008] The distillation category adopts response-based knowledge, that is, learning the probability distribution of the last layer of the teacher generator output by the softmax function, and using this as the knowledge to optimize the parameters of the student generator, so that the student generator directly simulates the output of the teacher generator.

[0009] The student model includes a student generator and a student discriminator. The student generator is a simplified version of the teacher generator and includes a fully connected layer, three residual blocks, a one-dimensional convolutional layer, and a softmax function layer. The fully connected layer initializes the input random floating-point number, performs three residual operations on the three residual blocks, and then performs a one-dimensional convolution operation on the one-dimensional convolutional layer. The normalized output is mapped to the probability distribution of the cryptographic space.

[0010] The student discriminator described herein employs, but is not limited to, the discriminator in the PassGAN model.

[0011] The depth of each layer in the student generator has been changed from 128 to 128*scale, where scale is a depth scaling factor that is set manually and is less than 1, thereby further reducing model parameters and simplifying model complexity.

[0012] Step 2) Train and test PassGAN using the preprocessed cipher set.

[0013] The aforementioned cipher set is a publicly disclosed plaintext cipher database.

[0014] The PassGAN described herein is implemented using, but is not limited to, the methods described in "PassGAN: A Deep Learning Approach for Password Guessing".

[0015] Step 3) Using the PassGAN trained in Step 2 as the teacher model, the learned cryptographic space distribution is distilled into the student model, so that the student model can simulate the output or features of the teacher model and thus learn the generalization ability of the teacher model. Specifically, graph g1 and session sess1 are created for the teacher model, and graph g2 and session sess2 are created for the student model. The student model is trained iteratively in the second session sess2. During each iteration, the values ​​from the first session sess1 are called. The model saved after training is the compressed student model.

[0016] The first session sess1 includes: fake_pass is the output of the teacher generator, and running the run() function can obtain the password sample generated by the teacher model, which is also the knowledge that the student model needs to learn. Technical effect

[0017] This invention employs a response-based offline knowledge distillation algorithm to compress the generator of the model. The pre-trained original model is used as the teacher model, and a new small model is constructed as the student model. The output of the teacher generator is used as knowledge to optimize and train the student model, and the loss of the student generator is constructed as a weighted sum of the original loss and the mean squared error loss. This allows the student model to both optimize its parameters through its own training and learn the generalization ability of the teacher model.

[0018] Compared with existing technologies, the compressed model reduces the model size by more than 40%, increases the password generation speed by about 35%, and the password coverage difference is no more than 5%, achieving lossless compression of PassGAN and supporting the application of PassGAN models in a wider range of scenarios. Attached Figure Description

[0019] Figure 1 This is a flowchart of the present invention;

[0020] Figure 2 This is a diagram of the student generator structure in the embodiment;

[0021] Figure 3 This is a knowledge distillation framework used in the embodiments;

[0022] Figure 4 This is a flowchart of the knowledge distillation process in the embodiment;

[0023] Figure 5 (a)-(c) are comparison diagrams of teacher-student models in the embodiments. Detailed Implementation

[0024] like Figure 1 As shown, this embodiment relates to a password guessing model compression method based on generative adversarial networks, implemented on the Linux operating system platform. It employs a response-based offline knowledge distillation approach to construct, as shown... Figure 2 The simplified and less deep student model shown uses publicly available plaintext cipher sets leaked from different websites for preprocessing. It trains and tests the existing cipher guessing model PassGAN. The trained original model is then used as the teacher model, and a new loss function is constructed for distillation training to obtain the student model. The changes in complexity between the student model and the original model are compared, as well as the differences in the coverage of the generated cipher sets. Specifically, this includes:

[0025] Step 1: Construct a model compression scheme. Based on the characteristics of PassGAN generative adversarial network, select a compression method for knowledge distillation, and further determine to adopt the response-based offline knowledge distillation method to construct a low-complexity model as the student model for knowledge distillation.

[0026] like Figure 2 As shown, the generator of the student model takes a randomly generated 128-bit floating-point number as input, initializes it through a fully connected layer, changes the dimension of the tensor by the reshape function, and then performs one-dimensional convolution and normalization after three residual operations to output the result. The discriminator network is consistent with PassGAN, with input samples from real password data or the generator's output. It first performs one-dimensional convolution, then performs five residual operations, changes the dimension of the output tensor by the reshape function, and finally outputs the discriminator result through a fully connected layer.

[0027] The depth of each layer in the student generator has been changed from 128 to 128*scale, where scale is set to 0.4 in this embodiment. This means that the layer depth is reduced to 0.4 times the original depth, which significantly reduces the number of parameters in each layer.

[0028] like Figure 3 and Figure 4As shown, the distillation process is implemented by calling the default session and default graph, i.e., with sess1.as_default() and with g1.as_default(), each time the student generator calls the teacher model in each iteration. When the context manager exits, it does not automatically close the session and release resources. A default session and default graph are created for both the teacher model and the student model, which can ensure that the two sessions do not interfere with each other and can be used normally. If only one graph is created, it will contain the entire teacher network and student network, and the compressed student model cannot be obtained.

[0029] The student network described above introduces a knowledge distillation process. The distillation coefficient alpha controls the proportion of knowledge in the model. In practice, this is achieved by constructing a new generator loss function G_loss to replace the loss from normal generator training. Variables G_t and G_s are defined as the output of the teacher generator and the student generator, respectively. The student generator loss G_loss, after distillation, is controlled by the coefficient alpha. A higher alpha indicates a greater focus on the difference between the teacher and student generator outputs, while a lower alpha indicates a greater focus on normal student model training. Typically, the distillation coefficient should not exceed 0.5. The distillation process is as follows: G_t = GeneartorT(noise), G_s = GeneartorS(noise), D_s = Discriminator(G_s), G_s_loss = -mean(D_s), G_loss = (1-alpha)*G_s_loss + alpha*L MSE (G_t, G_s), where: G_t and G_s represent the outputs of the teacher-student model generator, fake_pass_t and fake_pass_s, respectively; D_s represents the discrimination result output by the student discriminator after fake_pass_s is processed; G_s_loss represents the loss of the student generator without distillation, calculated using the mean function; L MSE (G_t, G_s) is the mean squared error loss (MSE loss) of G_t and G_s, used to measure the difference in output between the teacher and student model generators. The calculation of G_loss is adjusted by alpha weighting, which corresponds to the knowledge distillation process and is the true loss of the student generator.

[0030] The student network calculates the discriminator loss in the same way as the teacher network. During iteration, the ADAM optimizer is used to adjust the parameters of the generator and discriminator based on the losses of the generator and discriminator, so that the student generator can simulate the output of the teacher generator, continuously optimize the performance of the model, and improve the password guessing ability of the student model.

[0031] Step 2: Select publicly available password databases from the websites RockYou and Gmail as the sample sets for training and testing PassGAN.

[0032] In this embodiment, the processing of password databases from two websites is exactly the same. First, passwords with a length not exceeding 10 are preprocessed. 80% of these passwords are used as the training set, encoded in ISO-8859-1, backward compatible with ASCII, and containing more literals. The remaining 20% ​​is used as the test set. Duplicate passwords are removed to obtain unique password data in both the training and test sets. Then, passwords from the test set that appear in the training set are removed to obtain the final test data. During model training, the training set containing duplicate passwords is used to simulate real-world password distribution. During testing, the proportion of passwords generated by the model that cover the test data is statistically analyzed to measure the model's password guessing ability.

[0033] In this embodiment, the RockYou password set contains a total of 32,584,852 passwords. After preprocessing, there are 29,605,747 passwords (90.8%). Among them, 80% of the passwords, totaling 23,685,133, constitute the training set, with 9,912,024 unique passwords. 20% of the passwords, totaling 5,920,614, have 3,089,832 unique passwords. Finally, the RockYou password set yields 1,974,741 passwords for testing.

[0034] In this embodiment, the Gmail password set contains a total of 4,663,677 passwords. After preprocessing, there are 4,327,187 passwords (92.8%). Among them, 80% of the passwords, totaling 3,461,914, constitute the training set, with 2,231,881 unique passwords. The remaining 20% ​​of the passwords, totaling 865,273, have 641,343 unique passwords. Finally, the test data obtained from the Gmail password set contains 486,388 passwords.

[0035] In this embodiment, the model is trained iteratively 200,000 times using the RockYou training set. By observing the JS divergence between the n-gram distributions of the real and generated data, we can roughly determine that the model training has reached its optimal state. The latest model is saved to the checkpoints folder, occupying 31.2MB of memory, including 10.2MB for the META file and 20.9MB for the DATA file. Considering system resources and processing time, the generator generates 10... 6 10 7 and 10 8 The sample passwords, statistically covering the test data (1,974,741 passwords), accounted for 0.358%, 2.029%, and 6.793% of the total, respectively, with password generation times of 57.02s, 535.65s, and 5217.90s, respectively.

[0036] In this embodiment, considering the small number of passwords in the Gmail training set, the model is trained iteratively 100,000 times, resulting in a model size of 29.0 MB. The generator then generates 10... 6 and 10 7 The percentages of passwords that were statistically covered in the test data (486,388 passwords) were 0.553% and 2.743% respectively, and the password generation times were 52.74s and 515.29s respectively.

[0037] Step 3: Construct the loss function of the student model during distillation. Use the trained PassGAN model as the teacher model to distill and train the student model. During training, in each iteration, first load the recovered teacher network, and then calculate the loss of the generated network and the discriminator network after distillation. The ADAM optimizer optimizes the model parameters, and saves the latest model every 5000 iterations.

[0038] In this embodiment, the deep learning framework used for both the teacher and student models is TensorFlow, specifically version 1.13.1 of tensorflow-gpu.

[0039] like Figure 3 As shown, this is the knowledge distillation framework used in this embodiment. The technical information it reveals is that the student model generator is affected by the mean squared error loss between the outputs of the teacher and student model generators during training. The specific distillation process is as follows: Figure 4 .

[0040] like Figure 4 As shown, the knowledge distillation specifically involves: the generators of the teacher and student networks output their respective guessed password samples, and the mean squared error loss (MSE loss) of both is calculated. Meanwhile, the output of the student generator network is processed by the discriminant network, and the negative of its mean is used as the loss G_s_loss of the student generator before distillation. The two losses are multiplied by the knowledge distillation coefficients α and (1-α) respectively and then added together to obtain the loss G_loss of the student generator after knowledge distillation. The loss value is calculated at each iteration, and the parameters of the student generator are changed using the ADAM optimizer to gradually improve its password guessing ability.

[0041] In this embodiment, the knowledge distillation coefficient α is set to 0.3.

[0042] In this embodiment, the knowledge of distillation is the output of the teacher generator. When the student model is trained, the teacher model only outputs and does not continue training. Calculating the mean squared error loss allows the student generator to directly simulate the output of the teacher generator.

[0043] The student model training is specifically as follows: graphs g1 and g2, and sessions sess1 and sess2 are created for the teacher model and the student model, respectively. The student model is trained iteratively in the second session sess2, and values ​​from the first session sess1 are called in each iteration.

[0044] In this embodiment, the training set for the student model is also the preprocessed public cryptographic databases RockYou and Gmail. Similar to the teacher model, the student model is trained iteratively 200,000 times with the RockYou training set and 100,000 times with the Gmail training set, resulting in one student model for each.

[0045] Step 4: Compare and test the differences between the teacher and student models in terms of complexity and password coverage.

[0046] like Figure 5 As shown in (a), the student models trained by RockYou and Gmail in step 3 are 17.8MB and 15.8MB respectively, which are 42.9% and 45.5% smaller than the teacher models of 31.2MB and 29.0MB in step 2. The training time is similar for both models, decreasing from approximately 0.257s per iteration to approximately 0.243s under the same computing resources. However, the student model significantly improves the password generation speed, decreasing from approximately 3.30s to approximately 2.05s per 64,000 passwords. Figure 5 As shown in (b), the overall password generation speed is improved by approximately 35% or more; Figure 5 As shown in (c), statistics were generated for 10... 6 10 7 and 10 8 The password coverage loss of the password samples on the RockYou test data compared to the teacher model was 0.84%, 3.25%, and 2.68%, respectively. The results on Gmail were similar. It can be seen that the loss of coverage of the compressed model is very small. The model compression in this embodiment did not affect the password guessing ability of the original model.

[0047] Based on the comprehensive experimental results, this method constructs and implements a knowledge distillation-based model compression scheme for the PassGAN password guessing model according to its model structure. In the test, the size of the compressed model is reduced by more than 40%, the password generation speed is increased by about 35%, and the password coverage is not directly affected. The password guessing performance is not changed while reducing the model complexity.

[0048] In summary, this invention provides a model compression method based on generative adversarial networks, achieving effective and lossless compression of the PassGAN text password guessing model. This solves the problem of application scenarios being limited by the high complexity of the model, and enables high-quality password guessing on lightweight clients.

[0049] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A method for compressing a password guessing model based on generative adversarial networks, characterized in that, include: Step 1) Construct a model compression scheme for PassGAN, that is, use the knowledge distillation method to first determine the distillation mechanism and category, and then construct a simplified student model. Step 2) Train and test PassGAN using the preprocessed cryptographic set; Step 3) Using the PassGAN trained in Step 2 as the teacher model, the learned cryptographic space distribution is distilled into the student model, so that the student model can simulate the output or features of the teacher model and thus learn the generalization ability of the teacher model.

2. The method for compressing a password guessing model based on a generative adversarial network according to claim 1, characterized in that, The distillation mechanism and categories mentioned refer to: offline distillation that does not require additional training of the teacher model, where the teacher model has been pre-trained before the student model is trained, which is a one-way knowledge transfer; and response-based knowledge, which learns the probability distribution of the last layer of the teacher generator output by the softmax function, and uses this as the parameter to optimize the student generator, so that the student generator directly simulates the output of the teacher generator.

3. The method for compressing a password guessing model based on a generative adversarial network according to claim 1, characterized in that, The student model includes a student generator and a student discriminator. The student generator is a simplified version of the teacher generator and includes a fully connected layer, three residual blocks, a one-dimensional convolutional layer, and a softmax function layer. The fully connected layer initializes the input random floating-point number, performs three residual operations on the three residual blocks, and then performs a one-dimensional convolution operation on the one-dimensional convolutional layer. The normalized output is mapped to the probability distribution of the cryptographic space.

4. The method for compressing a password guessing model based on a generative adversarial network according to claim 1, characterized in that, Step 3) specifically involves: creating graph g1 and session sess1 for the teacher model, creating graph g2 and session sess2 for the student model, iteratively training the student model in the second session sess2, calling values ​​from the first session sess1 during each iteration, and saving the model after training is the compressed student model. The first session sess1 includes: fake_pass is the output of the teacher generator, and running the run() function can obtain the password sample generated by the teacher model, which is also the knowledge that the student model needs to learn.

5. The method for compressing a cryptographic guessing model based on generative adversarial networks according to any one of claims 1-4, characterized in that, The distillation process described above is implemented by calling the default session and default graph, i.e., with sess1.as_default() and with g1.as_default(), each time the student generator calls the teacher model in each iteration. When the context manager exits, it does not automatically close the session and release resources. A default session and default graph are created for both the teacher model and the student model, which ensures that the two sessions do not interfere with each other and can be used normally. If only one graph is created, it will contain the entire teacher network and student network, and the compressed student model cannot be obtained.

6. The method for compressing a password guessing model based on a generative adversarial network according to claim 5, characterized in that, The knowledge distillation coefficient alpha controls the proportion of knowledge. In practice, this is achieved by constructing a new generator loss function G_loss to replace the loss from normal generator training. We define the variable G_t as the output of the teacher generator and G_s as the output of the student generator. The student generator's loss G_loss, after distillation, is controlled by the coefficient alpha. A higher alpha indicates a greater focus on the difference between the teacher and student generator outputs, while a lower alpha indicates a greater focus on normal student model training. Typically, the distillation coefficient should not exceed 0.

5. The distillation process is as follows: G_t = GeneartorT(noise), G_s = GeneartorS(noise), D_s = Discriminator(G_s), G_s_loss = -mean(D_s), G_loss = (1-alpha)*G_s_loss + alpha*L MSE (G_t, G_s), where: G_t and G_s represent the outputs of the teacher-student model generator, fake_pass_t and fake_pass_s, respectively; D_s represents the discrimination result output by the student discriminator after fake_pass_s is processed; G_s_loss represents the loss of the student generator without distillation, calculated using the mean function; L MSE (G_t, G_s) is the mean squared error loss (MSEloss) of G_t and G_s, used to measure the difference between the outputs of the teacher and student model generators. The calculation of G_loss is adjusted by alpha weighting, which corresponds to the knowledge distillation process and is the true loss of the student generator.

7. The method for compressing a password guessing model based on a generative adversarial network according to claim 6, characterized in that, The student network calculates the discriminator loss in the same way as the teacher network. During iteration, the ADAM optimizer is used to adjust the parameters of the generator and discriminator based on the losses of the generator and discriminator, so that the student generator can simulate the output of the teacher generator, continuously optimize the performance of the model, and improve the password guessing ability of the student model.

Images