Password strength measuring method based on reinforcement learning

Through the password intensity measurement method based on reinforcement learning, the problem of insufficient data dependence and attack resistance of password intensity evaluation in the prior art is solved, and more accurate and robust password intensity evaluation is achieved, which can generate diverse password patterns and detect them. User personal information significantly improves the accuracy and security of password strength measurement.

CN119312302AActive Publication Date: 2025-01-14NANKAI UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411359686.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-27
Publication Date
2025-01-14
Estimated Expiration
2044-09-27

AI Technical Summary

Technical Problem

The existing password strength evaluation methods rely on large-scale data sets, making it difficult to generate non-popular password patterns, and cannot effectively resist online targeting and offline stroll guessing attacks, and lack detection of user personal information usage.

Method used

Using the password intensity measurement method based on reinforcement learning, the password data set is constructed, and the reinforcement learning model and scoring model are established. The model is trained to maximize the reward value and KL divergence, generate a password probability dictionary, calculate the guess number of passwords, and evaluate the password strength. , able to resist online directional and offline stroll guessing attacks.

Benefits of technology

It realizes a more accurate and robust password strength evaluation, can generate diverse password patterns, detect user personal information usage, and significantly improves the accuracy and security of password strength measurement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119312302A_ABST
    Figure CN119312302A_ABST
Patent Text Reader

Abstract

The invention discloses a reinforcement learning-based password strength measurement method, which comprises the following steps of: firstly, constructing a password data set, then establishing a reinforcement learning model and a scoring model, and training the reinforcement learning model and the scoring model to obtain an optimal reinforcement learning model; after deployment is completed, the user inputs a password into the optimal reinforcement learning model and outputs a probability; comparing the probability with a password probability dictionary through a Monte Carlo calculation algorithm to obtain a guess number of the password; and obtaining the password strength of the password according to the corresponding relationship between the guess number of the password and the password strength. According to the password modeling mode based on reinforcement learning, the password generation process of the user is regarded as the Markov decision process, the reward brought by each action is calculated, and a password sequence can be better modeled; the method can resist online directional guessing attack and offline walking guessing attack at the same time, and is more accurate and more robust.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of information security technology, relates to the field of password security authentication, and specifically is a password strength measurement method based on reinforcement learning. Background Art

[0002] Today, user password authentication is still the main means of identity authentication. With the current surge in online services, the number of website accounts that each user needs to manage is increasing, but users can usually only remember 5 to 7 different passwords. At the same time, it is inevitable that users usually tend to choose popular strings, use personally identifiable information (PII), and are vulnerable to side channel attacks (such as shoulder surfing attacks, key tone feedback, etc.). Therefore, it becomes very important to measure the security of user passwords in real environments.

[0003] In practice, people have found that the number of guesses is a good indicator for evaluating the strength of user passwords. Passwords that can be easily guessed by an adversary are considered weak passwords, while passwords that require a large number of guesses to be guessed are considered secure passwords. In order to ensure the security of the system and users, mainstream Internet service providers will use password strength meters (PSMs) to provide users with feedback on the strength of their passwords when they register or update their passwords. Only PSMs that can accurately feedback password strength can effectively improve the security of user passwords and thus protect user account security. Mistakenly identifying a strong password as a weak password may cause usability problems, while mistakenly identifying a weak password as a strong password may bring security risks and fail to ensure security. In addition, current mainstream PSMs often ignore the use of user personal information when setting passwords, while in fact, the proportion of users introducing user personal information when creating passwords is quite high (12.8% to 51.4%). This phenomenon suggests that websites and service providers need to take effective measures to detect the use of users' personal information and guide users to avoid using personal information and passwords that are easy to guess when creating passwords, in order to reduce potential security risks.

[0004] The paper "Weir, M., Aggarwal, S., De Medeiros, B., & Glodek, B. Password cracking using probabilistic context-free grammars. In Proc. IEEE Symposium on Security and Privacy, pages 391-405." uses unsupervised learning represented by PCFG, and the paper "W. Melicher, B. Ur, S. Komanduri, L. Bauer, N. Christin, and L F Cranor. Fast, lean and accurate: Modeling password guessability using neural networks. In Proc. USENIX SEC 2016, pages 175-191." uses supervised learning models represented by FLA. Both methods rely on large-scale password datasets. The problems are: (1) PCFG requires a large-scale training set to fully learn the password partition template and template content, while the existing supervised learning needs to learn password features from large-scale data labels. (2) Supervised learning models can only learn from password data with popular labels, and unpopular password labels (such as password followed by D instead of the common d) are usually ignored by the learning model. As a result, existing guessing models can usually only imitate the existing password generation patterns in the training set. For unsupervised learning represented by PCFG, it can only generate password templates that appear in the training set and related context-independent content. For supervised learning, the training method is character-level autoregression, which can only generate existing paths and lack diversity. For example, if the training data does not contain or only contains a small amount of a certain specific password pattern (such as passwordD), the existing guessing model based on autoregressive learning may not be able to generate a password with the corresponding pattern, and thus cannot model the password strength well. Summary of the invention

[0005] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to provide a password strength measurement method based on reinforcement learning.

[0006] The technical solution of the present invention to solve the technical problem is to provide a password strength measurement method based on reinforcement learning, characterized in that the method comprises the following steps:

[0007] Step 1, data construction phase: construct a password data set, and then extract the user personal information of the passwords in the password data set;

[0008] Step 2, training phase: according to the password data set obtained in step 1, a reinforcement learning model and a scoring model are established respectively, and then the reinforcement learning model and the scoring model are trained simultaneously; during the training process, the reinforcement learning model needs to explore the path that maximizes the total reward value and needs to continuously generate a guess list, and the scoring model continuously scores the guess list output by the reinforcement learning model, and the scoring results are used to feedback and update the reinforcement learning model, and the scoring model is updated according to the sorting order of the total reward value of the guess list and the KL divergence between the proportion of mixed real user passwords; the training process is repeated until the optimal reinforcement learning model is obtained;

[0009] Step 3, deployment phase: Use the optimal reinforcement learning model obtained in step 2 to generate a password probability dictionary Θ, and then deploy the optimal reinforcement learning model, Monte Carlo calculation algorithm and password probability dictionary Θ in step 2;

[0010] Step 4, guess number calculation phase: After deployment, the user inputs the password α into the optimal reinforcement learning model, and the optimal reinforcement learning model outputs a probability; then compares this probability with the password probability dictionary Θ through the Monte Carlo calculation algorithm to obtain the number of guesses for the password; then, based on the corresponding relationship between the number of guesses for the password and the password strength, the password strength of the password is obtained.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] (1) Previous password strength assessment methods were either based on traditional statistical methods or on information entropy estimation. No new password modeling methods were tried, and it was difficult to capture more user password generation rules. The password modeling method based on reinforcement learning proposed in the present invention regards the process of user password generation as a Markov decision process, calculates the reward brought by each action, and thus models the user password generation process. It can better model the password sequence and solve the problem of dependence on the data set and inability to generate patterns that are not in the data set.

[0013] (2) The present invention can resist both online directional guessing attacks and offline wandering guessing attacks. Previous password strength evaluations were either based only on offline password guessing and could only resist offline password guessing attacks; or they could simply detect whether the user used personal identification information, but could not determine whether the user's password could resist online directional guessing attacks. Both offline wandering guessing attacks and online directional guessing attacks are threats that may be encountered during the user identity authentication process. Therefore, the present invention can detect and resist both types of attacks at the same time, thereby protecting the user's password security.

[0014] (3) The present invention designs a reinforcement learning model and a scoring model. The training steps of the two models are carried out simultaneously. The reinforcement learning model is used to measure the strength of the password, and the scoring model is used to assist the training of the reinforcement learning model and evaluate the effect of the reinforcement learning model to improve its effect.

[0015] (4) More accurate and robust password strength assessment. By comparing the evaluation effects of different PSMs on the strength of real user passwords, the results show that the accuracy of the password strength evaluation results output by the present invention is significantly better than that of other algorithms. At the same time, the robustness of the present method, that is, the password strength evaluation in extreme cases will not have too much deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall flow chart of the present invention;

[0017] Figure 2 A schematic diagram of matching a user password and user personal information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The specific embodiments of the present invention are given below. The specific embodiments are only used to further illustrate the present invention in detail and do not limit the protection scope of the present invention.

[0019] The present invention provides a password strength measurement method based on reinforcement learning (hereinafter referred to as the method), characterized in that the method comprises the following steps:

[0020] Step 1, data construction phase: construct a password dataset for training, and then extract user personal information of the passwords in the password dataset;

[0021] Preferably, in step 1, the specific method of constructing the password data set is:

[0022] The password generation process is regarded as a Markov decision process τ:

[0023] τ={s0,a0,s1,a1,…,s t ,a t ,…,s T ,a T} (1)

[0025] In formula (1), action a t It is from the password character set (i.e. the bag of words of the password dataset) A character selected in state s t is the previous action a t-1 The obtained password PW 1:t The sequence is (w0,w1,…,w t-1 ), where all characters t∈[0,T], represents the time step; the probability of the Markov decision process τ is:

[0026]

[0027] In formula (2), β(a t ∣s t ) represents the strategy of the reinforcement learning model;

[0028] Preferably, in step 1, when the password data set contains only passwords, For all printable characters; when the password data set contains passwords and user personal information, It is a collection of all printable characters and user personal information labels.

[0029] Preferably, in step 1, the specific method of extracting user personal information is: using a pre-defined user personal information tag to match the passwords in the password data set to determine whether the password contains user personal information; if the user personal information is matched, the user personal information is transformed into a field recognizable by the reinforcement learning model in step 2 according to the user personal information tag.

[0030] Preferably, in step 1, the user's personal information includes name, username, telephone number, ID number, birthday and email address.

[0031] Preferably, in step 1, the predefined user personal information tag is:

[0032] (1) In the name, N1 to N7 correspond to the full name, the initials of the full name, the surname, the given name, the initials of the given name + the surname, the given name + the initials of the surname, and the initials of the surname in capital letters;

[0033] (2) In the user name, U1~U2 correspond to the user name and the first letter of the user name is capitalized respectively;

[0034] (3) In the telephone number, T1 to T3 correspond to the telephone number, the first three digits of the telephone number, and the last four digits of the telephone number, respectively;

[0035] (4) In the ID number, I1 to I3 correspond to the first three digits, the first six digits, and the last four digits of the ID number, respectively;

[0036] (5) Birthday, B1~B 13 They correspond to month, day, and year, year, month, day, month, year, year, month, year, the last two digits of month, year, month, day, day, month, day, year;

[0037] (6) In the mailbox, E1 to E3 correspond to the mailbox name, second-level domain name, and top-level domain name respectively.

[0038] like Figure 2For example, the password daniel1027 is shown in the figure. It can be represented in different ways as {N3B4}, {N31,0,2,7}, {N3B 10 ,2,7},{N31,0,B 11}、{N3B 10 B 11}, {E3B4} and {d,a,n,i,e,l,1,0,2,7} and other 17 forms of representation.

[0039] Step 2, training phase: according to the password data set obtained in step 1, a reinforcement learning model and a scoring model are respectively established, and then the reinforcement learning model and the scoring model are trained simultaneously; during the training process, the reinforcement learning model needs to explore the path that maximizes the total reward value and needs to continuously generate a guess list, and the scoring model continuously scores the guess list output by the reinforcement learning model, and the scoring results are used to feedback and update the reinforcement learning model. The scoring model is updated according to the KL (Kullback-Leibler) divergence between the sorting order of the total reward value of the guess list and the proportion of mixed real user passwords; the training process is repeated until the optimal reinforcement learning model is obtained;

[0040] Preferably, in step 2, the optimal reinforcement learning model is saved as a binary format file with a compressed size of 1.34 MB.

[0041] Preferably, in step 2, both the reinforcement learning model and the scoring model need to be pre-trained before training, and the pre-training method is to perform training by using the maximum likelihood function on the training set.

[0042] Preferably, in step 2, the training process of the reinforcement learning model is as follows: the reinforcement learning model needs to be trained according to the current strategy π θ (a t ∣s t ) and the state s of the currently generated password sequence t , from the password character set Select the next character in action a t ; Each state s during command training t Obtained through Monte Carlo tree search; the reward for a single password is calculated using a scoring model Then take the average of the rewards of n passwords as the current state s t Rewards t ; The goal of reinforcement learning model training is to optimize the strategy π of the reinforcement learning model θ (a t ∣s t ) to maximize the expected total reward:

[0043]

[0044] In formula (3), is the objective function of the training task, For the Markov process τ = PW 1:T The total reward, that is, the sum of the rewards for each step, is used to find the path suitable for the reinforcement learning model to explore and obtain the maximum reward, so as to generate a guess list; the policy gradient trained by the reinforcement learning model is used to update the parameters θ of the reinforcement learning model.

[0045] Preferably, in step 2, the Monte Carlo tree search is specifically: for a certain state s t Search backwards for n paths until the exploration is complete.

[0046] Preferably, in step 2, the reward for a single password is By y PW The correlation score of the feature vector extracted from the Top-1w passwords in the feature space is:

[0047]

[0048] In formula (4), y e is the feature vector of the Top-1w passwords, is the real password sampled from the training set for comparison; γ is the temperature hyperparameter of the Softmax function, which is used to control the model training process.

[0049] For example, the password PW 1:T is {P,a,s,s,w,@,r,d}, so set s0 to B s (starting character), and then select action a0 to obtain the maximum reward 0.62 of the state of character w0=P, thus obtaining s1=B s P, then do the next action a1 to get s2 = B s Pa, and so on, until the complete password Passw@rd is generated.

[0050] Preferably, in step 2, the scoring model training process is specifically: using passwords of real users and passwords of a guess list generated by a reinforcement learning model mixed in different proportions, using the scoring model to extract features for calculating the KL divergence between the sorting order of the total reward value of the guess list and the mixing ratio as the objective function; the goal of the scoring model training is to minimize the KL divergence objective function.

[0051] Step 3, deployment phase: Use the optimal reinforcement learning model obtained in step 2 to generate a password probability dictionary Θ, and then deploy the optimal reinforcement learning model, Monte Carlo calculation algorithm and password probability dictionary Θ in step 2 for users to use;

[0052] Preferably, in step 3, deployment is performed to a web page front end or a browser plug-in.

[0053] Preferably, in step 3, the reference of the Monte Carlo calculation algorithm is "Dell'Amico, M., & Filippone, M.. Monte Carlo strength evaluation: Fast and reliable password checking. In Proc. ACM SIGSAC CCS 2015, pages 158-169.".

[0054] Step 4, guess number calculation phase: After deployment, the user inputs the password α into the optimal reinforcement learning model, and the optimal reinforcement learning model outputs a probability; then compares this probability with the password probability dictionary Θ through the Monte Carlo calculation algorithm to obtain the number of guesses for the password, which is the number of times the attacker guesses the password; then, based on the corresponding relationship between the number of guesses and the password strength, the password strength of the password is obtained.

[0055] Preferably, in step 4, after the deployment is completed, the user enters the password, and first determines whether the password α is in the blacklist password. If it is in the blacklist password, it directly returns that registration is not allowed, and reminds the user that a stronger password needs to be modified; when it is determined that the password α is not in the blacklist password, the password α is input into the optimal reinforcement learning model, and the subsequent steps are performed.

[0056] Preferably, in step 4, the number of guesses S of the password α is calculated p The process of (α) is considered as: calculating the number of passwords weaker than password α in the entire password space:

[0057]

[0058] From formula (5), we can get the number of guesses S of password α: p (α) is to calculate the number of passwords in the entire password space whose probability p(β) is greater than the probability p(α) of the password α; since β follows the distribution p(β), the number of guesses for password α is S p Estimate of (α)C Δ yes:

[0059]

[0060] In formula (6), n is the sample size of the password probability dictionary Θ. In this embodiment, the sample size of the password probability dictionary Θ is generally 100,000, which can control the detection error within 1%, and it decreases with the 1 / 2 power of the sample size. Therefore, the password probability dictionary is about 800KB.

[0061] Preferably, in step 4, the high-frequency password list used in the process of training the reinforcement learning model needs to be collected in advance as the website's blacklist password, which is about 100KB. The storage space required for the entire password guessing number calculation application is about 3MB, which can be well deployed to the web front end or browser plug-in.

[0062] Preferably, in step 4, the number of guesses is divided into an online directional guess number GN1 and an offline walking guess number GN2; if user personal information is extracted, this part of information can come from the user's publicly available user name and email address and other automatically filled information, or it can come from the field actively entered by the user, then the calculated number of guesses is the online directional guess number GN1; if user personal information is not extracted, then the calculated number of guesses is the offline walking guess number GN2.

[0063] Preferably, in step 4, according to actual business and scenario requirements, the corresponding relationship between the number of password guesses and the password strength is as follows:

[0064] (1) Online directional guess number GN1∈(0,10 3 ] or offline walking guess number GN2∈(0,10 6 ]: Extremely unsafe; considering that the network servers of modern major service providers are equipped with malicious traffic detection mechanisms and often limit the number of daily logins by IP, the upper limit of the number of guesses that an attacker can make for large-scale online targeted guessing is 10 3 ; Offline hashing can be done in 10 6 It is also extremely unsafe to guess the password within the specified number of guesses, and this type of password should be prohibited.

[0065] (2) Online directional guess number GN1∈(10 3 ,10 5 ] or offline walking guess number GN2∈(10 6 ,10 9 ]: relatively unsafe; passwords in this range are still relatively unsafe, but for websites with low security requirements, these passwords can be used, but they are still prohibited for use on websites with higher security (such as finance).

[0066] (3) Online directional guess number GN1∈(10 5 ,10 7 ] or offline walking guess number GN2∈(10 9 ,10 15]: Generally usable; for passwords in this range, the attacker's benefits will not increase significantly with the increase in the number of guesses. Therefore, passwords with a strength of PW3 can basically meet the requirements of most websites for password length and character type, and can also resist offline wandering guessing attacks and online targeted guessing attacks by most ordinary attackers.

[0067] (4) Online directional guess number GN1>10 7 Or offline walking guess number GN2>10 15 Password: safe; it is considered impossible for attackers to guess these passwords through various means in a short time, so they are safe passwords.

[0068] Example 1

[0069] The present invention and FLA-PSM are actually run, Markov-PSM from "C. Castelluccia, M. Dürmuth, and D. Perito. Adaptive password-strength meters from Markov models. In Proc. NDSS 2012.", Zxcvbn from "Wheeler, Daniel Lowe. zxcvbn: Low-Budget Password Strength Estimation. In Proc. USENIX SEC 2016, pages 157-173.", and Zxcvbn from "B. Ur, S. M. Segreti, L. Bauer, N. Christin, L. F. Cranor, S. Komanduri, D. Kurilova, M. L. Mazurek, W. Melicher, and R. Shay. Measuring real-world accuracies and biases in modeling password guessability. In Proc. USENIX SEC 2015, pages 157-173.". 463-481.》Min_auto Four password strength measurement methods with better performance.

[0070] LinkedIn, CSDN and Dodonew are used as the accuracy test sets, and a set of 100,000 passwords sampled from CSDN is used as the robustness test set.

[0071] Accuracy test: calculate the Spearman coefficient between the strength ranking result output by the algorithm and the strength result of the ideal strength ranking. The experimental results show that the accuracy of the password strength evaluation result output by the present invention is significantly better than that of other algorithms. Specifically, on the English data set LinkedIn, the similarity of the password strength measurement method proposed by the present invention is 0.652, while FLA-PSM, Zxcvbn, Markov-PSM, and Min_auto are 0.367, 0.579, 0.201, and 0.558, respectively. On the Chinese data set CSDN, the similarity of the password strength measurement method proposed by the present invention is 0.728, while the other four methods are 0.643, 0.514, 0.268, and 0.575, respectively. On the Chinese data set Dodonew, the similarity of the password strength measurement method proposed by the present invention is 0.535, while the other four methods are 0.322, 0.475, 0.297, and 0.423, respectively. This shows that the present invention has better accuracy than the previous strength measurement methods.

[0072] Robustness test: Mistakenly identifying a strong password as a weak password may lead to usability problems, while misidentifying a weak password as a strong password may bring security risks and fail to ensure the claimed security. Through the 1V1 confusion matrix test of the CSDN dataset, it can be found that Zxcvbn will have 24691 passwords that will be misidentified by the present invention in 10 4 ~10 8 guessed out, and Zxcvbn will think that this part of the password needs 10 8 It takes more than 100 times to guess it. This shows that Zxcvbn will regard a large number of passwords as strong passwords. The other algorithms all have similar phenomena, which shows that the present invention has better stability than the compared algorithms, that is, it is more robust and can avoid some extreme situations.

[0073] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A password strength measurement method based on reinforcement learning, characterized in that: The method comprises the following steps: Step 1, data construction phase: construct a password data set, and then extract the user personal information of the passwords in the password data set; Step 2, training phase: according to the password data set obtained in step 1, a reinforcement learning model and a scoring model are established respectively, and then the reinforcement learning model and the scoring model are trained simultaneously; during the training process, the reinforcement learning model needs to explore the path that maximizes the total reward value and needs to continuously generate a guess list, and the scoring model continuously scores the guess list output by the reinforcement learning model, and the scoring results are used to feedback and update the reinforcement learning model, and the scoring model is updated according to the sorting order of the total reward value of the guess list and the KL divergence between the proportion of mixed real user passwords; the training process is repeated until the optimal reinforcement learning model is obtained; Step 3, deployment phase: Use the optimal reinforcement learning model obtained in step 2 to generate a password probability dictionary Θ, and then deploy the optimal reinforcement learning model, Monte Carlo calculation algorithm and password probability dictionary Θ in step 2; Step 4, guess number calculation phase: After deployment, the user inputs the password α into the optimal reinforcement learning model, and the optimal reinforcement learning model outputs a probability; then compares this probability with the password probability dictionary Θ through the Monte Carlo calculation algorithm to obtain the number of guesses for the password; then, based on the corresponding relationship between the number of guesses for the password and the password strength, the password strength of the password is obtained.

2. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 1, the specific method of constructing the password dataset is: The password generation process is regarded as a Markov decision process τ: τ={s0,a0,s1,a1,…,s t ,a t ,…,s T ,a T } (1) In formula (1), action a t From the password character set A character selected in state s t is the previous action a t-1 The obtained password PW1:t sequence; t∈[0,T], represents the time step; the probability of the Markov decision process τ is: In formula (2), π θ (a t ∣s t ) represents the policy of the reinforcement learning model.

3. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 1, the specific method of extracting user personal information is: use the pre-defined user personal information label to match the passwords in the password data set to determine whether the password contains user personal information; if the user personal information is matched, the user personal information is transformed into a field that can be recognized by the reinforcement learning model in step 2 according to the user personal information label.

4. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 2, the training process of the reinforcement learning model is as follows: the reinforcement learning model needs to be trained according to the current strategy π θ (a t ∣s t ) and the state s of the currently generated password sequence t , from the password character set Select the next character in action a t ; Each state s during command training t Obtained through Monte Carlo tree search; Use the scoring model to calculate the reward for a single password Then take the average of the rewards of n passwords as the current state s t Rewards t ; The goal of reinforcement learning model training is to optimize the strategy π of the reinforcement learning model θ (a t ∣s t ) to maximize the expected total reward: In formula (3), is the objective function of the training task, For a Markov process τ = PW 1:T Total rewards; Update the parameters θ of the reinforcement learning model using the policy gradient trained by the reinforcement learning model.

5. The password strength measurement method based on reinforcement learning according to claim 4 is characterized in that: In step 2, the reward for a single password By y PW The correlation score of the feature vector extracted from the Top-1w passwords in the feature space is: In formula (4), y e is the feature vector of the Top-1w passwords, is the real password sampled from the training set for comparison; γ is the temperature hyperparameter of the Softmax function, which is used to control the model training process.

6. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 2, the scoring model training process is specifically as follows: using passwords of real users and passwords of a guess list generated by a reinforcement learning model mixed in different proportions, using the scoring model to extract features for calculating the KL divergence between the sorting order of the total reward value of the guess list and the mixing ratio as the objective function; the goal of scoring model training is to minimize the KL divergence.

7. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 4, after the deployment is completed, the user enters the password, and first determines whether the password α is in the blacklist password. If it is in the blacklist password, registration is not allowed, and the user is reminded to change the password; When it is determined that the password α is not in the blacklist password, the password α is input into the optimal reinforcement learning model to perform subsequent steps.

8. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 4, the number of guesses S for password α is calculated p The process of (α) is considered as: calculating the number of passwords weaker than password α in the entire password space: From formula (5), we can get the number of guesses S of password α: p (α) is to calculate the number of passwords in the entire password space whose probability p(β) is greater than the probability p(α) of the password α; Since β follows the distribution p(β), the number of guesses for password α is S p Estimate of (α)C Δ yes: In formula (6), n is the sample size of the password probability dictionary Θ.

9. The password strength measurement method based on reinforcement learning according to claim 1, characterized in that: In step 4, the guess number is divided into online directional guess number GN1 and offline walking guess number GN2.

10. The password strength measurement method based on reinforcement learning according to claim 9, characterized in that: In step 4, the corresponding relationship between the number of password guesses and password strength is as follows: (1) Online directional guess number GN1∈(0,10 3 ] or offline walking guess number GN2∈(0,10 6 ] password: extremely unsafe; (2) Online directional guess number GN1∈(10 3 ,10 5 ] or offline walking guess number GN2∈(10 6 ,10 9 ] password: relatively unsafe; (3) Online directional guess number GN1∈(10 5 ,10 7 ] or offline walking guess number GN2∈(10 9 ,10 15 ] password: generally available; (4) Online directional guess number GN1>10 7 Or offline walking guess number GN2>10 15 Password: safe.

Citation Information

Patent Citations

  • Password strength evaluation model based on ensemble learning

    CN108763920A

  • Password guessing system and method based on Transform model and deep reinforcement learning

    CN114462018A

  • System and methods for analyzing and modifying passwords

    US20140373088A1