A Sensitive Data Detection Method, System, Computer Terminal and Storage Medium

Abstract

The present invention discloses a sensitive data detection method, system, computer terminal and storage medium, relating to the technical field of information security. The technical key points are as follows: The present invention creates a dictionary of domain terms by extracting domain terms. First, it detects the data structure of the text data to be detected. When it is structured data, it uses the word list to obtain the tokens of each character of the text data to be detected, and then queries the set of words of all possible domain terms in the dictionary according to the tokens, ensuring the coverage of terms. It uses the embedding layer of the BERT model to convert the tokens into word vector encoding sequences, and splices the word vector encoding sequences with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains the output vector based on the input vector, and then conducts convolutional learning in the convolutional neural network based on the output vector, thereby enhancing the semantic representation of sensitive terms. In this way, the detection accuracy of sensitive information in the text data to be detected is improved.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and more specifically, it relates to a sensitive data detection method, system, computer terminal and storage medium. Background Art

[0002] With the rapid development of information technology, all walks of life highly rely on information systems. How to ensure the security of information systems, especially how to ensure the security of data reflecting the core value of enterprises, has become the most concerned thing for enterprises. Enterprise data contains a lot of user personal privacy information, business sensitive data, etc. Once leaked, it will bring huge economic losses to enterprises and they need to bear relevant legal responsibilities and huge fines for violations. Therefore, how to ensure the security of enterprise user personal privacy information, business sensitive data, etc. has become the top priority of enterprise information security work. The first thing to do is to find a detection method that can comprehensively, quickly and as much as possible without manual intervention to discover the distribution of enterprise user personal privacy information, business sensitive data, etc.

[0003] There are some sensitive fields stored in the data warehouse, such as user mobile phone numbers and communication addresses. These information will be directly transmitted over the network, which is very likely to cause the problem of password data leakage of users or enterprises. And the data warehouse needs to be as convenient as possible for users to use, while protecting the data of sensitive fields from being accessed randomly. On the one hand, it is necessary to know which fields are sensitive, and on the other hand, it is to perform a higher level of permission control on the data of sensitive fields.

[0004] Therefore, how to solve the detection accuracy of sensitive data in the process of sharing user or enterprise data in the database of the information system is an urgent problem to be solved. Summary of the Invention

[0005] In order to solve the problem of the detection accuracy of sensitive data in the process of sharing user or enterprise data in the database of the information system, the object of the present invention is to provide a sensitive data detection method, system, computer terminal and storage medium. The present invention creates a dictionary of domain terms by extracting domain terms. First, it detects the data structure of the text data to be detected. When it is structured data, it uses the word list to obtain the tokens of each character of the text data to be detected, so as to query the set of words of all possible domain terms in the dictionary according to the tokens, ensuring the coverage of terms. It uses the embedding layer of the BERT model to convert the tokens into a word vector encoding sequence, and splices the word vector encoding sequence with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains the output vector based on the input vector, and then performs convolutional learning in the convolutional neural network based on the output vector, thereby enhancing the semantic representation of sensitive terms, and thus improving the detection accuracy of sensitive information in the text data to be detected.

[0006] The above technical object of the present invention is achieved by the following technical solutions:

[0007] In the first aspect of the present application, a sensitive data detection method is provided, and the method includes:

[0008] Extract domain terms to create a periodically updated dictionary;

[0009] Obtain the text data to be detected;

[0010] When it is detected that the text data to be detected is not structured data, convert the characters in the text data to be detected into first tokens according to the word list, query the dictionary according to the first tokens to obtain a set of words of all possible domain terms, obtain the second tokens corresponding to each word in the set of words through the word list, and use the first tokens and the second tokens as the input sequence;

[0011] Generate absolute head position encoding and absolute tail position encoding according to the position of each character in the text data to be detected;

[0012] Input the input sequence into the embedding layer of the BERT model to obtain a first word vector encoding sequence;

[0013] Concatenate the first word vector encoding sequence, the absolute head position encoding and the absolute tail position encoding in the BERT model to obtain a first output vector;

[0014] Input the first output vector into a convolutional neural network to obtain a first vector including the word vector dimension and the sensitive level;

[0015] Input the first vector into the softmax function to predict the first probability distribution of the sensitive level, and select the sensitive level with the highest probability in the first probability distribution as the detection result of the text data to be detected.

[0016] In an implementation, after obtaining the text data to be detected whose sensitive level classification has been completed and before extracting the field information corresponding to the keyword fields from the data to be detected according to the character library, it further includes: preprocessing the data to be detected by using a word segmentation tool; wherein, the word segmentation tool includes the jieba word segmentation tool, the ltp word segmentation tool and the ir word segmentation tool; the preprocessing includes removing dirt, word segmentation and filtering invalid characters.

[0017] In an implementation, perform sensitive classification on the text data to be detected to obtain the sensitive category of the text data to be detected, where the sensitive category includes the first category, the second category, the third category and the fourth category, and the sensitivity degrees of the first category, the second category, the third category and the fourth category increase in sequence.

[0018] In an implementation, the method further includes:

[0019] When it is detected that the text data to be detected is structured data, a character library with keywords is created, and the field information corresponding to the keywords in the text data to be detected is extracted according to the character library;

[0020] Convert the field information into marker symbols, and convert the marker symbols into corresponding third tokens;

[0021] Input the third token into the embedding layer of the BERT model to obtain a second word vector encoding sequence;

[0022] Concatenate the second word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain a second output vector;

[0023] Input the second output vector into a convolutional neural network to obtain a second vector including the word vector dimension and the sensitivity level;

[0024] Input the second vector into the softmax function to predict the second probability distribution of the sensitivity level, and select the sensitivity level with the highest probability in the second probability distribution as the detection result of the text data to be detected.

[0025] In one implementation, the generating of the absolute head position encoding and the absolute tail position encoding according to the position of each character in the text data to be detected specifically includes:

[0026] When the character is a word, generate the absolute head position encoding and the absolute tail position encoding according to the position of each word in the text data to be detected; or,

[0027] When the character is a word, generate the absolute head and tail position encodings according to the start and end positions of the characters included in the word in the text data to be detected.

[0028] In one implementation, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer, where the convolutional layer, the pooling layer, and the fully connected layer are connected in sequence.

[0029] In one implementation, extracting domain terms to create a periodically updated dictionary includes:

[0030] Collect the terms in the authorized information system that have been labeled and in the domain term manual;

[0031] Set an update period. Each time of update, detect whether the collected terms are repeated in the dictionary. If the terms are repeated, do not add the terms to the dictionary. If the terms are not repeated, add the non-repeated terms to the dictionary.

[0032] In the second aspect of the present application, a sensitive data detection system is provided, and the system includes:

[0033] A dictionary creation module for extracting domain terms to create a periodically updated dictionary;

[0034] A data acquisition module for acquiring text data to be detected;

[0035] An input sequence determination module, when it detects that the text data to be detected is not structured data, converts the characters in the text data to be detected into first tokens according to a vocabulary, queries the dictionary based on the first tokens to obtain a set of words of all possible domain terms, obtains the second tokens corresponding to each word in the set of words through the vocabulary, and uses the first tokens and the second tokens as the input sequence;

[0036] A position encoding module for generating an absolute head position encoding and an absolute tail position encoding according to the position of each character in the text data to be detected;

[0037] A word vector conversion module for inputting the input sequence into the embedding layer of the BERT model to obtain a first word vector encoding sequence;

[0038] An splicing module for splicing the first word vector encoding sequence, the absolute head position encoding and the absolute tail position encoding in the BERT model to obtain a first output vector;

[0039] An output calculation module for inputting the first output vector into a convolutional neural network to obtain a first vector including the word vector dimension and the sensitivity level;

[0040] A sensitive category detection module for inputting the first vector into the softmax function to predict the first probability distribution of the sensitivity level, and selecting the sensitivity level with the largest probability in the first probability distribution as the detection result of the text data to be detected.

[0041] In a third aspect of the present application, a computer terminal is provided, including: a memory and a processor, where a computer program is stored on the memory, and the computer program can be executed by the processor so that the processor implements a sensitive data detection method as described in any one of the first aspects of the present application.

[0042] In a fourth aspect of the present application, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of a sensitive data detection method as described in any one of the first aspects of the present application are implemented.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. A sensitive data detection method provided by the present invention creates a dictionary of domain terms by extracting domain terms. First, it detects the data structure of the text data to be detected. When the data is structured data, it uses a word list to obtain the tokens of each character in the text data to be detected, and then queries the set of words of all possible domain terms in the dictionary based on the tokens, ensuring the coverage of terms. It uses the embedding layer of the BERT model to convert the tokens into a word vector encoding sequence, and splices the word vector encoding sequence with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains an output vector based on the input vector, and then performs convolutional learning in a convolutional neural network based on the output vector to enhance the semantic representation of sensitive terms. In this way, the detection accuracy of sensitive information in the text data to be detected is improved.

[0045] 2. A sensitive data detection method provided by the present invention, when detecting that the text data to be detected is structured data, creates a character library with keywords, and uses the field information extracted from the character library as an input sequence to be input into the embedding layer of the BERT model to be converted into a word vector encoding sequence. The word vector encoding sequence is spliced with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains an output vector based on the input vector, and then performs convolutional learning in a convolutional neural network based on the output vector to obtain a vector containing the word vector dimension and the sensitive level. This vector enhances the semantic representation of sensitive terms, and finally predicts the probability distribution of the sensitive term level based on the softmax function, and selects the sensitive level with the highest probability in the probability distribution as the detection result of the text data to be detected, thereby improving the detection accuracy and detection efficiency of sensitive information in the text data to be detected.

[0046] In addition, the present application also provides a sensitive data detection system, a computer terminal, and a storage medium, which have the same technical effects as the above-mentioned sensitive data detection method and will not be described in detail. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and do not limit the embodiments of the present invention. In the drawings:

[0048] Figure 1 is a schematic flowchart of a sensitive data detection method provided by an embodiment of the present application;

[0049] Figure 2 is a principle block diagram of a sensitive data detection system provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] To make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to embodiments and the accompanying drawings. The illustrative embodiments and descriptions thereof of the present invention are only used to explain the present invention and shall not be construed as limiting the present invention.

[0051] It should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0052] First, some proprietary technical terms in this embodiment are explained as follows:

[0053] An information system is a man-machine integrated system composed of computer hardware, networks and communication devices, computer software, information resources, information users and rules and regulations for the purpose of processing information flows. It mainly has five basic functions, namely input, storage, processing, output and control of information.

[0054] The BERT model is a general natural language processing model, which is a language processing model in the prior art and will not be elaborated in this embodiment.

[0055] Convolutional Neural Networks (CNN) are a class of feedforward neural networks (Feedforward Neural Networks) that contain convolutional computations and have a deep structure, and are one of the representative algorithms of deep learning. Convolutional neural networks have the ability of representation learning and can perform shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also known as "Shift-Invariant Artificial Neural Networks (SIANN)".

[0056] There are some sensitive fields stored in the data warehouse, such as user mobile phone numbers and communication addresses. These information will be directly transmitted over the network, which is likely to cause the problem of password data leakage of users or enterprises. While the data warehouse needs to be as convenient as possible for users to use, and at the same time, the data of sensitive fields should be protected from being accessed randomly. On the one hand, it is necessary to know which fields are sensitive, and on the other hand, it is to perform a higher level of permission control on the data of sensitive fields. This application aims to solve the problem that the detection accuracy of sensitive data during the sharing process of user or enterprise data in the database of the information system is an urgent problem to be solved. A sensitive data detection method is provided. By extracting domain terms to create a dictionary of domain terms, first, the data structure of the text data to be detected is detected. When it is not structured data, the tokens of each character of the text data to be detected are obtained using the word list, so as to query the set of words of all possible domain terms in the dictionary based on the tokens, ensuring the coverage of terms. Then, the tokens are used as the input sequence and input into the embedding layer of the BERT model to be transformed into a word vector encoding sequence. The word vector encoding sequence is concatenated with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains the output vector based on the input vector, and then based on the output vector, convolutional learning is performed in the convolutional neural network to further enhance the semantic representation of sensitive terms, thereby improving the detection accuracy of sensitive information in the text data to be detected.

[0057] The following will explain the detection method in detail in combination with specific embodiments. Please refer to Figure 1 , which shows a schematic flowchart of a sensitive data detection method. The method includes the following steps:

[0058] S110, extract domain terms to create a periodically updated dictionary.

[0059] Specifically, collect the terms in the authorized information system that have been labeled and in the domain term manual; set an update period. Each time an update is performed, check whether the collected terms are repeated in the dictionary. If the terms are repeated, do not add the terms to the dictionary. If the terms are not repeated, add the non-repeated terms to the dictionary.

[0060] It can be understood that the extraction means can be collected and extracted through methods such as expert annotation and domain term manuals.

[0061] S120, obtain the text data to be detected.

[0062] In this embodiment, the text data to be detected can be obtained through the interface of the information system. In addition to obtaining text data, image data, audio data, video data, etc. can also be obtained. Of course, since this application mainly involves the detection of sensitive data of user or enterprise information data, therefore, only text data is obtained.

[0063] It should be understood that there is no sequence precedence between the above-mentioned step S110 and step S120.

[0064] S130. When it is detected that the text data to be detected is not structured data, convert the characters in the text data to be detected into first tokens according to a word list, query a dictionary based on the first tokens to obtain a set of words of all possible domain terms, obtain the second tokens corresponding to each word in the set of words through the word list, and use the first tokens and the second tokens as the input sequence.

[0065] In this embodiment, since the information data of users or enterprises involves structured data and unstructured data, and structured data is very easy to perform sensitive detection. For example, sensitive detection can be performed according to the feature codes of preset text data, and the feature codes are used to represent the feature information of the text data. Different types of text data correspond to different types of feature codes. Among them, structured text data may include: document type, ID number type, birthday type, address type, mobile phone number type, etc. It can be seen from the names of the above structured data that it is highly organized and neatly formatted data. It is the type of data that can be put into tables and spreadsheets. It may not be the type of data that people are most likely to find, but compared with unstructured data, it is undoubtedly the type of data that people are easier to use. On the other hand, terminal devices such as computers, laptops, and mobile phones can easily search it. Structured data is also called quantitative data, which is information that can be represented by data or a unified structure, such as numbers and symbols.

[0066] Therefore, in this embodiment, for unstructured data, convert the characters in the text data to be detected into first tokens according to a word list. It can be understood that the word list is an existing technology for realizing keyword retrieval. Therefore, using the word list to convert the characters in the text data to be detected into first tokens, it can be understood that a token is a serial number. Therefore, the first token represents the identifier corresponding to each character. Then, query a dictionary based on the first tokens to obtain a set of words of all possible domain terms, and obtain the second tokens corresponding to each word in the set of words through the word list, ensuring the coverage range of terms. Similar to the first token, the second token also represents the identifier of each word of each character in the created domain term dictionary; use the first tokens and the second tokens as the input sequence, that is, mark them as the input sequence of the subsequent model in the order of the serial numbers of the first tokens and the second tokens.

[0067] S140. Generate absolute head position encoding and absolute tail position encoding according to the position of each character in the text data to be detected.

[0068] In this embodiment, generating absolute head position encoding and absolute tail position encoding based on the position of characters in the text data to be detected is an existing technology, which will not be elaborated in this embodiment.

[0069] S150, input the input sequence into the embedding layer of the BERT model to obtain the first word vector encoding sequence.

[0070] In this embodiment, encoding the sequence based on the input sequence input into the embedding layer of the BERT model, which is an existing technology and will not be elaborated further.

[0071] S160, concatenate the first word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain the first output vector.

[0072] In this embodiment, by concatenating the first word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding, the input vector [E i , H i , T i is obtained, where E i represents the first word vector encoding sequence, H i represents the absolute head position encoding, and T i represents the absolute tail position encoding, ensuring the accuracy of the text address. The concatenated encoding is used as the input vector of the BERT model, so that the output vector {O1, O2, …, O N} can be obtained.

[0073] S170, input the first output vector into a convolutional neural network to obtain the first vector including the word vector dimension and the sensitivity level.

[0074] In this embodiment, input the output vector into a two-layer convolutional neural network to obtain the vector Y ∈ R d×M , where d is the dimension of the word vector and M is the number of sensitive recognition categories, enhancing the semantic representation of sensitive terms.

[0075] S180, input the first vector into the softmax function to predict the first probability distribution of the sensitivity level, and select the sensitivity level with the highest probability in the first probability distribution as the detection result of the text data to be detected.

[0076] In this embodiment, Softmax is an activation function that can normalize a numerical vector into a probability distribution vector, and the sum of all probabilities is 1. The Softmax function can be used as the last layer of a convolutional neural network for the output of multi-classification problems. The Softmax layer is often used in combination with the cross-entropy loss function. Therefore, in this embodiment, the first probability distribution of the sensitivity level of the data to be detected is predicted based on the prediction probability of the softmax function, and the sensitivity level with the highest probability in the first probability distribution is selected as the detection result of the text data to be detected.

[0077] In the above-described embodiments, a sensitive data detection method is provided. The present invention creates a dictionary of domain terms by extracting domain terms. First, it detects the data structure of the text data to be detected. When it is structured data, it uses the word list to obtain the tokens of each character in the text data to be detected, and then queries the set of words of all possible domain terms in the dictionary based on the tokens, ensuring the coverage of terms. It uses the embedding layer of the BERT model to convert the tokens into a word vector encoding sequence, and splices the word vector encoding sequence with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains an output vector based on the input vector, and then performs convolutional learning in the convolutional neural network based on the output vector, thereby enhancing the semantic representation of sensitive terms. In this way, the detection accuracy of sensitive information in the text data to be detected is improved.

[0078] In one embodiment, after obtaining the text data to be detected that has been classified according to the sensitive level, before extracting the field information corresponding to the keyword fields from the data to be detected according to the character library, it further includes: preprocessing the data to be detected using a word segmentation tool; wherein, the word segmentation tools include the jieba word segmentation tool, the ltp word segmentation tool, and the ir word segmentation tool; the preprocessing includes removing dirty data, word segmentation, and filtering invalid characters.

[0079] Specifically, using a word segmentation tool to preprocess the text data to be detected, the preprocessing includes removing dirty data, word segmentation, and filtering invalid characters, and the word segmentation tools include the jieba word segmentation tool, the ltp word segmentation tool, and the ir word segmentation tool. This is prior art and no redundant explanation is given in this embodiment.

[0080] In one embodiment, the text data to be detected is classified according to the sensitive level to obtain the sensitive category of the text data to be detected, where the sensitive category includes the first category, the second category, the third category, and the fourth category, and the sensitivity levels of the first category, the second category, the third category, and the fourth category increase in sequence.

[0081] In this embodiment, for the specific characteristics of the data itself, a sensitive level classification model is provided, and the text data to be detected is classified according to the first category, the second category, the third category, and the fourth category.

[0082] Among them, the fourth category represents that the sensitive level of the text data to be detected is extremely sensitive;

[0083] Its text data includes: user entity identification, user private information, user password and associated information; enterprise internal core management data, market core business data, network device and IT system password and associated information, core network device and IT system resource data

[0084] The third category represents that the sensitive level of the text data to be detected is sensitive;

[0085] Its text data includes: natural person identity identifiers, network identity identifiers, user basic information, service content data, contact information, service records and logs, location data; important internal enterprise management data, important market operation data, enterprise reported information, important business operation service data, important network equipment and IT system resource data, channel basic data, CP / SP basic data

[0086] The second category indicates that the sensitivity level of the text data to be detected is a relatively sensitive level;

[0087] Its text data includes: consumption information and bills, terminal device identifiers, terminal device information; general internal enterprise management data, general market operation data, general business operation service data, general network equipment and IT system resource data, network equipment and IT system support data

[0088] The second category indicates that the sensitivity level of the text data to be detected is a low sensitive level;

[0089] Its text data includes: service subscription relationships, violation record data; enterprise publicly disclosed information, business operation service data, digital content business operation data, publicly available network equipment and IT system resource data.

[0090] In one embodiment, the method further includes:

[0091] When it is detected that the text data to be detected is structured data, a character library with keywords is created, and the field information corresponding to the keywords in the text data to be detected is extracted according to the character library;

[0092] The field information is converted into marker symbols, and the marker symbols are converted into corresponding third tokens;

[0093] The third tokens are input into the embedding layer of the BERT model to obtain a second word vector encoding sequence;

[0094] The second word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding are concatenated in the BERT model to obtain a second output vector;

[0095] The second output vector is input into a convolutional neural network to obtain a second vector including the word vector dimension and the sensitivity level;

[0096] The second vector is input into the softmax function to predict the second probability distribution of the sensitivity level, and the sensitivity level with the largest probability in the second probability distribution is selected as the detection result of the text data to be detected.

[0097] In this embodiment, sensitive data detection of structured data is considered. By creating a word library with keywords, the field information extracted from the word library is used as an input sequence and input into the embedding layer of the BERT model to be transformed into a word vector encoding sequence. The word vector encoding sequence is concatenated with the head and tail position encodings of the characters to obtain the input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains an output vector based on the input vector, and then through convolutional learning in the convolutional neural network, a vector containing the word vector dimension and the sensitive level is obtained. This vector enhances the semantic representation of sensitive terms. Finally, based on the softmax function, the probability distribution of the sensitive term level is predicted, and the sensitive level with the highest probability in the probability distribution is selected as the detection result of the text data to be detected, thereby improving the detection accuracy and efficiency of sensitive information in the text data to be detected.

[0098] In one embodiment, generating the absolute head position encoding and the absolute tail position encoding according to the position of each character in the text data to be detected specifically includes:

[0099] When the character is a word, generating the absolute head position encoding and the absolute tail position encoding according to the position of each word in the text data to be detected; or,

[0100] When the character is a term, generating the absolute head and tail position encodings according to the head and tail positions of the words included in the term in the text data to be detected.

[0101] In this embodiment, for the encoding method of the head and tail positions of words or terms, it is the prior art and no redundant explanation is made.

[0102] In one embodiment, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer, where the convolutional layer, the pooling layer, and the fully connected layer are connected in sequence. This is the prior art and no redundant explanation is made.

[0103] Based on the same inventive concept, this embodiment provides a sensitive data detection system. Since the principles of these systems for solving problems are similar to Figure 1 a sensitive data detection method shown, the implementation of these systems can refer to Figure 1 the embodiments of the method shown, and the repeated parts will not be elaborated. As Figure 2 shown, the system includes:

[0104] A dictionary creation module 210, configured to extract domain terms to create a periodically updated dictionary;

[0105] A data acquisition module 220, configured to acquire the text data to be detected;

[0106] An input sequence determination module 230, configured to, when detecting that the text data to be detected is not structured data, convert the characters in the text data to be detected into first tokens according to a vocabulary, query a dictionary based on the first tokens to obtain a set of words of all possible domain terms, obtain second tokens corresponding to each word in the set of words through the vocabulary, and use the first tokens and the second tokens as an input sequence;

[0107] A position encoding module 240, configured to generate an absolute head position encoding and an absolute tail position encoding according to the position of each character in the text data to be detected;

[0108] A word vector conversion module 250, configured to input the input sequence into the embedding layer of a BERT model to obtain a first word vector encoding sequence;

[0109] A splicing module 260, configured to splice the first word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain a first output vector;

[0110] An output calculation module 270, configured to input the first output vector into a convolutional neural network to obtain a first vector including a word vector dimension and a sensitivity level;

[0111] A sensitive category detection module 280, configured to input the first vector into a softmax function to predict a first probability distribution of the sensitivity level, and select the sensitivity level with the largest probability in the first probability distribution as the detection result of the text data to be detected.

[0112] It can be seen that for a sensitive data detection system provided in the foregoing embodiment, the present invention creates a dictionary of domain terms by extracting domain terms, first detects the data structure of the text data to be detected, and when it is structured data, uses a vocabulary to obtain tokens for each character of the text data to be detected, so as to query a set of words of all possible domain terms in the dictionary based on the tokens, ensuring the coverage of terms. The embedding layer of the BERT model is used to convert the tokens into a word vector encoding sequence, and the word vector encoding sequence and the head and tail position encodings of the characters are spliced to obtain an input vector of the BERT model, ensuring the accuracy of the text address. The BERT model obtains an output vector based on the input vector, and then performs convolutional learning in the convolutional neural network based on the output vector, thereby enhancing the semantic representation of sensitive terms. In this way, the detection accuracy of sensitive information in the text data to be detected is improved.

[0113] Based on the same inventive concept, an embodiment of the present application provides an electronic device, including a processor and a memory storing a computer program. When the processor executes the computer program, the steps of the sensitive data detection method described in the above embodiment are implemented. Among them, the electronic device can be an intelligent terminal such as a computer or a tablet computer. The processor can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program. The communication interface is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The memory can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), or other types of dynamic storage devices that can store information and instructions. It can also be an electrically erasable programmable read-only memory, a read-only optical disc or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but not limited thereto. The memory can exist independently and be connected to the processor through a bus. The memory can also be integrated with the processor. Among them, the memory is used to store the application program code for executing the above solution and is controlled by the processor to execute. The processor is used to execute the application program code stored in the memory. The code stored in the memory can execute the steps of the sensitive data detection method executed by the above-provided terminal device.

[0114] In another embodiment of the present invention, a readable storage medium is further provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in this storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the sensitive data detection method in the above embodiment. Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0115] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only the specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A sensitive data detection method, characterized in that, The method includes: Extracting domain terms to create a periodically updated dictionary; Obtaining the text data to be detected; When it is detected that the text data to be detected is not structured data, converting the characters in the text data to be detected into first tokens according to the vocabulary, querying the dictionary based on the first tokens to obtain a set of words of all possible domain terms, obtaining the second tokens corresponding to each word in the set of words through the vocabulary, and using the first tokens and the second tokens as the input sequence; Generating absolute head position encoding and absolute tail position encoding according to the position of each character in the text data to be detected; Inputting the input sequence into the embedding layer of the BERT model to obtain a first word vector encoding sequence; Concatenating the first word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain a first output vector; Inputting the first output vector into a convolutional neural network to obtain a first vector including the word vector dimension and the sensitivity level; Inputting the first vector into the softmax function to predict the first probability distribution of the sensitivity level, and selecting the sensitivity level with the highest probability in the first probability distribution as the detection result of the text data to be detected.

2. The sensitive data detection method according to claim 1, characterized in that, After obtaining the text data to be detected that has completed sensitivity level classification and before extracting the field information corresponding to the keyword fields from the text data to be detected according to the character library, it further includes: preprocessing the text data to be detected using a word segmentation tool; wherein, the word segmentation tool includes the jieba word segmentation tool, the ltp word segmentation tool, and the ir word segmentation tool; the preprocessing includes removing dirt, word segmentation, and filtering invalid characters.

3. A sensitive data detection method according to claim 1, characterized in that Performing sensitivity grading on the text data to be detected to obtain the sensitivity category of the text data to be detected, where the sensitivity category includes the first category, the second category, the third category, and the fourth category, and the sensitivity levels of the first category, the second category, the third category, and the fourth category increase in sequence.

4. A sensitive data detection method according to claim 1, characterized in that The method further includes: When it is detected that the text data to be detected is structured data, creating a character library with keywords, and extracting the field information corresponding to the keywords from the text data to be detected according to the character library; Converting the field information into marker symbols and converting the marker symbols into corresponding third tokens; Inputting the third tokens into the embedding layer of the BERT model to obtain a second word vector encoding sequence; Concatenating the second word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain a second output vector; Inputting the second output vector into a convolutional neural network to obtain a second vector including the word vector dimension and the sensitivity level; Inputting the second vector into the softmax function to predict the second probability distribution of the sensitivity level, and selecting the sensitivity level with the highest probability in the second probability distribution as the detection result of the text data to be detected.

5. A sensitive data detection method according to claim 4, wherein, The generating of the absolute head position encoding and the absolute tail position encoding according to the position of each character in the text data to be detected specifically includes: When the character is a Chinese character, generating the absolute head position encoding and the absolute tail position encoding according to the position of each Chinese character in the text data to be detected; or, When the character is a word, generating the absolute head and tail position encoding according to the start and end positions of the Chinese characters included in the word in the text data to be detected.

6. A sensitive data detection method according to claim 1, characterized in that, The convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer, where the convolutional layer, the pooling layer, and the fully connected layer are connected in sequence.

7. A sensitive data detection method according to claim 1, characterized in that Extract domain terms to create a periodically updated dictionary, including: Collect the terms in the labeled information system that has been authorized and in the domain term manual; Set an update period. Each time of update, detect whether the collected terms are repeated in the dictionary. If the terms are repeated, do not add the terms to the dictionary. If the terms are not repeated, add the non-repeated terms to the dictionary.

8. A sensitive data detection system, characterized in that the system Include: A dictionary creation module for extracting domain terms to create a periodically updated dictionary; A data acquisition module for acquiring the text data to be detected; An input sequence determination module for, when detecting that the text data to be detected is not structured data, converting the characters in the text data to be detected into first tokens according to the word list, querying the dictionary based on the first tokens to obtain a set of words of all possible domain terms, obtaining the second tokens corresponding to each word in the set of words through the word list, and using the first tokens and the second tokens as the input sequence; A position encoding module for generating an absolute head position encoding and an absolute tail position encoding according to the position of each character in the text data to be detected; A word vector conversion module for inputting the input sequence into the embedding layer of the BERT model to obtain a first word vector encoding sequence; A splicing module for splicing the first word vector encoding sequence, the absolute head position encoding, and the absolute tail position encoding in the BERT model to obtain a first output vector; An output calculation module for inputting the first output vector into a convolutional neural network to obtain a first vector including the word vector dimension and the sensitivity level; A sensitive category detection module for inputting the first vector into the softmax function to predict the first probability distribution of the sensitivity level, and selecting the sensitivity level with the largest probability in the first probability distribution as the detection result of the text data to be detected.

9. A computer terminal, comprising: A memory and a processor, wherein a computer program is stored on the memory, and characterized in that the computer program can be executed by the processor so that the processor implements a sensitive data detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, wherein when the computer program is executed by the processor, the steps of a sensitive data detection method as described in any one of claims 1 to 7 are implemented.

Images