Website sensitive information detection method and device based on deep reinforcement learning

Through deep reinforcement learning methods and DQN networks, combined with evolutionary algorithms, the problem of low accuracy of sensitive information detection in the existing technology is solved, and fast and effective sensitive information detection is achieved, which reduces labor costs and improves detection accuracy.

CN118890179APending Publication Date: 2024-11-01ZHEJIANG YUAN INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410917769.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-10
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The existing sensitive information detection system cannot be customized to identify according to user needs, resulting in low detection accuracy and inability to effectively discover sensitive information in the website.

Method used

Using a deep reinforcement learning method, through DQN network and evolutionary algorithm, sensitive information classification tasks are regarded as sequential decision-making process, and the best classification strategy is learned by using the agent to interact with the environment to generate feature vectors and determine whether they contain sensitive information. The evolving algorithm accelerates the training process.

Benefits of technology

It realizes rapid and efficient discovery of sensitive information in the website, reduces labor costs, improves detection accuracy, and greatly reduces the training time of deep reinforcement learning models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118890179A_ABST
    Figure CN118890179A_ABST
Patent Text Reader

Abstract

The invention discloses a website sensitive information detection method and device based on deep reinforcement learning. The method comprises the steps of 1, obtaining real data of a to-be-detected website; 2, extracting to-be-detected data in the real website text data to obtain a candidate information set; 3, performing feature extraction on the candidate information set to generate a feature vector of the candidate information set to be detected; and 4, inputting the feature vector of the candidate information set to be detected into the deep reinforcement learning model, and judging whether the candidate information set to be detected contains sensitive information or not. The method has the beneficial effect of accurately predicting the sensitive information classification label to which the website information belongs. The invention further discloses a website sensitive information detection device based on machine learning. The device comprises a website data acquisition module, a feature extraction module and a sensitive information detection module. According to the method, a method based on deep reinforcement learning is used for solving the classification problem of sensitive information and conventional information in a network, and whether the sensitive information exists in a website or not can be quickly and effectively found, so that relevant workers can take countermeasures in time, the sensitive information is prevented from being leaked, and the data security is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer machine learning. More specifically, the present invention relates to a method and device for detecting sensitive information on a website based on deep reinforcement learning. Background Art

[0002] Sensitive information detection is an important part of security. When corporate personnel communicate on the Internet, sensitive information is often used. If it is attacked or maliciously stolen, it is easy to leak sensitive information.

[0003] Among them, sensitive information includes: passwords, keys, certificates, session identifiers, licenses, privacy data (such as the content of short messages), authorization credentials and personal data (such as name, address, telephone number, etc.). Sensitive data may be contained in program files, configuration files, log files, backup files and databases.

[0004] The prior art generally uses a sensitive information identification system to detect sensitive information. However, most of the current sensitive information identification systems have been commercialized and cannot be customized to identify sensitive information according to user needs, resulting in low accuracy in sensitive information detection. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a method and device for detecting sensitive information on a website based on deep reinforcement learning, so as to overcome the current problems of low accuracy and non-customization of sensitive information detection.

[0006] When the present invention uses deep reinforcement learning to solve the problem of sensitive information classification, all sensitive information classification task processes are regarded as a sequential decision-making process in which the agent interacts with the environment to learn the best classification strategy.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A method for detecting sensitive information on a website based on deep reinforcement learning, comprising the following steps:

[0009] Get the real data of the website to be tested;

[0010] Extract the data to be tested from the real website text data to obtain a candidate information set;

[0011] Extract features from the candidate information set to generate a feature vector of the candidate information set to be detected;

[0012] The feature vector of the candidate information set to be detected is input into the deep reinforcement learning model to determine whether the candidate information set to be detected contains sensitive information.

[0013] Furthermore, the deep reinforcement learning model is implemented using a DQN network.

[0014] Furthermore, the convergence strategy during the DQN network training adopts an evolutionary algorithm.

[0015] The evolutionary algorithm adopts a multi-agent strategy to explore the optimal weight set of the neural network, which can effectively speed up the training of the deep reinforcement learning model.

[0016] The evolutionary algorithm adopts a centralized strategy, consisting of a coordinator process and multiple worker processes. The worker process is only responsible for interacting with the environment and evaluating. In addition to being responsible for the tasks of the worker process, the coordinator process also collects evaluation results and updates weights.

[0017] A website sensitive information detection device based on deep reinforcement learning using the above method comprises:

[0018] A website data acquisition module is used to obtain the real data of the website to be detected;

[0019] A feature extraction module is used to extract features from the information to be detected and generate a feature vector of the information to be detected;

[0020] The sensitive information detection module is used to input the feature vector of the information to be detected into the deep reinforcement learning model to determine whether the information to be detected is sensitive information.

[0021] The present invention has the following advantages:

[0022] 1. By training the deep reinforcement learning model and replacing manual detection with model decision-making, it is possible to quickly and effectively discover whether there is sensitive information on the website, so that relevant staff can take timely response measures to avoid leaking sensitive information, ensure data security, and greatly reduce labor costs.

[0023] 2. The trained deep reinforcement learning model can use the experience gained from historical learning to make the classification results of website information more in line with the actual situation.

[0024] 3. Using the improved centralized evolutionary algorithm can greatly reduce the training time of deep reinforcement learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In the attached picture:

[0026] Figure 1 It is the work flow chart of the present invention.

[0027] Figure 2 For training deep reinforcement learning algorithm model and model workflow diagram.

[0028] Figure 3Schematic diagram of evolutionary algorithm. DETAILED DESCRIPTION

[0029] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0030] The present invention provides a method for determining sensitive information, which first obtains real data of a website to be detected, extracts the data to be detected from the real website text data, obtains a candidate information set, then performs feature extraction on the candidate information set to generate a feature vector of the candidate information set to be detected, and then inputs the feature vector of the candidate information set to be detected into a deep reinforcement learning model to determine whether the candidate information set to be detected contains sensitive information.

[0031] Figure 1 A schematic diagram of the process flow of the sensitive information detection method provided by the present invention.

[0032] 1) Obtain real data from the website to be tested: Collect real data from the website to be tested through crawlers or manual collection.

[0033] 2) Extract the data to be tested from the real website text data: The crawled data is often in HTML format. The data is parsed through BeautifulSoup and filtered to obtain text information. The program is used to collect the obtained text information to obtain the candidate information set to be tested.

[0034] 3) Feature extraction: After obtaining the candidate information set to be detected, perform feature extraction on each candidate information to be detected. After the extraction is completed, for each candidate information to be detected d i , there will be a eigenvector v i ={f i1 , f i2 , …, f im}.

[0035] 4) Constructing a deep reinforcement learning environment: The reinforcement learning framework of the present invention is based on the DQN network. The DQN algorithm is described as follows:

[0036] Algorithm input: training data set D = {(d1, l1), (d2, l2), ..., (d T , l T )}, the number of iterations is K.

[0037] Algorithm output: parameter θ of the Q-value network.

[0038] a) Initialize the experience replay buffer M; Initialize the estimated network parameters θ; Initialize the mini-batch size batch_size;

[0039] b) Initialize the target network parameters θ' (θ' = θ); Initialize the target network parameter update frequency G;

[0040] c) Initialize episode = 1;

[0041] d) Shuffle D

[0042] e) Initialize s1 = d1, t = 1;

[0043] f) Use the ε-greedy algorithm to select an action a according to the state s t t ;

[0044] g) If t < T, then s t+1 = d t+1 and done = false, otherwise s t+1 = None and done = true;

[0045] h) Obtain the reward value r t = NEXT(s t , a t , l t );

[0046] i) Put the five-tuple (s t , a t , r t , s t+1 , d t ) into the experience replay buffer M;

[0047] j) Randomly sample batch_size samples from M and calculate y j for each sample:

[0048]

[0049] where γ represents the discount rate for future rewards;

[0050] k) Perform gradient descent on L(θ), i.e., L(θ) = (y j - Q(s i , a i ; θ)) 2 ;

[0051] l) t = t + 1;

[0052] m) If t ≤ T, go back to step f);

[0053] n) If episode%G=0, update parameter θ'=θ;

[0054] o)episode=episode+1;

[0055] p) If episode ≤ K, return to step d), otherwise end and output the parameters θ of the Q value network.

[0056] In the definition of reinforcement learning, i State execution action a i Will reach the next state s with a certain probability i+1 In the present invention, according to the order of the training data set, the state s of the i-th sample i After executing any action, the state s of the i+1th sample will be reached. i+1 Since there is no strict state transition relationship, we need to reorder the samples before each round of training.

[0057] Figure 2 For training deep reinforcement learning algorithm model and model workflow diagram.

[0058] In order to solve the problem of slow convergence of deep reinforcement learning agents in high-dimensional space, the present invention uses a centralized evolutionary algorithm as the convergence strategy of DQN. Specifically, the evolutionary algorithm adopts a centralized strategy, which consists of a coordinator process and multiple worker processes. The worker process is only responsible for interacting with the environment and evaluating it. In addition to being responsible for the tasks of the worker process, the coordinator process also collects the evaluation results and updates the weights. The more processes that operate simultaneously, the faster the training speed.

[0059] Figure 3 Schematic diagram of evolutionary algorithm.

[0060] In order to guide the DQN agent to learn the optimal classification strategy in the imbalanced data, the absolute reward value of minority class samples is set higher than that of majority class in this invention. When the agent correctly or incorrectly identifies the minority class samples, the environment feedback agent will receive a greater reward or penalty. In this invention, the reward of sensitive information samples is set to 1.0, and the reward of regular information samples is set to λ (0≤λ≤1). The specific value of λ is usually set to the ratio of the number of sensitive information to the number of regular information, as shown below:

[0061]

[0062] Among them, the state s t represents the feature vector of the tth candidate information to be detected in D, a t Indicates t The classification action performed, l trepresents the label of the tth candidate information to be detected, R(s t ,a t ,l t ) represents the return value, D M Represents a sensitive information set, D B Represents a general information set.

[0063] 5) Determine whether the candidate information to be detected contains sensitive information: Input the feature vector of the candidate information to be detected in 3) into the deep reinforcement learning model in 4) to determine whether each candidate information to be detected contains sensitive information.

[0064] Based on the above-mentioned scheme design, the present invention can solve the classification problem of sensitive information and conventional information in the network, and can quickly and effectively discover whether there is sensitive information in the website, so that relevant staff can take timely response measures to avoid leakage of sensitive information and ensure data security.

[0065] The present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0066] The present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, each step of the method of the present invention is implemented.

[0067] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for detecting sensitive information on a website based on deep reinforcement learning, characterized in that: The following steps are involved: Step 1: Obtain the real data of the website to be tested; Step 2: extract the data to be tested from the real website text data to obtain a candidate information set; Step 3: Extract features from the candidate information set to generate feature vectors of the candidate information set to be detected; Step 4: Input the feature vector of the candidate information set to be detected into the deep reinforcement learning model to determine whether the candidate information set to be detected contains sensitive information. When training the deep reinforcement learning model, the absolute reward value of minority class samples is set higher than that of majority class samples. When the agent correctly or incorrectly identifies minority class samples, the environment feedback agent will receive a larger reward or penalty. In the present invention, the reward of sensitive information samples is set to 1.0, and the reward of regular information samples is set to λ (0≤λ≤1). The specific value of λ is usually set to the ratio of the number of sensitive information to the number of regular information, as shown below: Among them, the state s t Represents the feature vector of the tth candidate information to be detected, a t Indicates t The classification action performed, l t represents the label of the tth candidate information to be detected, R(s t ,a t ,l t ) represents the return value, D M Represents a sensitive information set, D B Represents a general information set. The deep reinforcement learning model adjusts the imbalance rate during training, that is, after obtaining the feature vector, the ratio of the number of sensitive information samples to the number of regular information samples is adjusted to 1:λ by randomly discarding certain samples, and different λ values ​​are selected to train the models separately to simulate the imbalance ratio that may occur in real DNS traffic.

2. The method according to claim 1, characterized in that The deep reinforcement learning model is implemented using the DQN network.

3. The method according to claim 2, characterized in that The deep reinforcement learning model uses a centralized evolutionary algorithm as a convergence strategy.

4. A sensitive information detection device based on deep reinforcement learning using the method described in any one of claims 1 to 3, characterized in that: include: A website data acquisition module is used to obtain the real data of the website to be detected; A feature extraction module is used to extract features from the information to be detected and generate a feature vector of the information to be detected; The sensitive information detection module is used to input the feature vector of the information to be detected into the deep reinforcement learning model to determine whether the information to be detected is sensitive information.

5. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 5.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 3 is implemented.