Sensitive information detection method and device, computer equipment and readable storage medium

Abstract

The invention relates to a sensitive information detection method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: acquiring to-be-retrieved data; preprocessing the to-be-retrieved data to obtain target data; converting the target data into a target vector; performing rule matching on the target vector; if the rule matching is successful, performing similarity retrieval in a preset vector database according to the target vector; a plurality of sensitive information vectors are stored in the preset vector database; if the sensitive information vector meeting the similarity requirement is retrieved from the preset vector database, inputting the target vector into a preset classification model; and determining a sensitive information detection result of the to-be-retrieved data according to a classification result of the preset classification model. By adopting the method, missing report and false report can be avoided, and the accuracy of sensitive information detection is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sensitive information detection method, apparatus, computer equipment and computer-readable storage medium. Background Art

[0002] In the daily operation of an enterprise, it is often necessary to send notification messages to users through SMS, in-app messages, etc., for users to view or confirm. Since the enterprise's application system manages a large amount of user data and enterprise data, abnormal users may use the application system to leak user personal information, spread sensitive information, etc. For example, abnormal users may send refund notices in the name of a bankrupt financial platform. Enterprises need to search and screen the message records sent by each user.

[0003] Enterprises generally use keyword search solutions to search for messages containing suspicious keywords in message records. However, since keywords may have different meanings in different contexts, it is easy to misidentify sensitive information. In addition, keyword search also has problems such as low recall, poor precision, and poor multi-word search results, which makes keyword search unable to achieve the expected search results in actual application scenarios. Summary of the invention

[0004] Based on this, it is necessary to provide a sensitive information detection method, device, computer equipment and computer-readable storage medium to address the above-mentioned technical problems, which can improve the accuracy of sensitive information detection.

[0005] In a first aspect, the present application provides a sensitive information detection method, comprising:

[0006] Get the data to be retrieved;

[0007] Preprocess the data to be retrieved to obtain target data;

[0008] Convert target data into target vector;

[0009] Perform rule matching on the target vector;

[0010] If the rule is matched successfully, a similarity search is performed in a preset vector database according to the target vector; the preset vector database stores several sensitive information vectors;

[0011] If a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into the preset classification model;

[0012] Based on the classification results of the preset classification model, the sensitive information detection results of the data to be retrieved are determined.

[0013] In one embodiment, the method further comprises:

[0014] Receive a sample dataset;

[0015] Preprocess each piece of data in the sample data set to obtain a number of sample data;

[0016] Convert sensitive information sample data in the sample data into a sample vector;

[0017] Perform rule matching on sample vectors;

[0018] The sample vectors that successfully match the rules are taken as sensitive information vectors and stored in the preset vector database.

[0019] In one embodiment, the method further comprises:

[0020] Receive sensitive data sent by operation and maintenance terminals;

[0021] Convert sensitive data into sensitive information vectors and update them into a preset vector database;

[0022] Generate extended data based on sensitive data;

[0023] Convert the extended data into an extended vector;

[0024] Perform rule matching on the extended vector;

[0025] The extended vector that successfully matches the rule is used as a sensitive information vector and updated to the preset vector database.

[0026] In one embodiment, the preset classification model includes an input layer, a word embedding layer, an attention layer, a weighted sum layer, a fully connected layer and an output layer; wherein:

[0027] The input layer is used to receive the target vector;

[0028] The word embedding layer is used to map the vocabulary included in the target vector into a low-dimensional dense vector;

[0029] The attention layer is used to calculate the importance weight of each word;

[0030] The weighted summation layer is used to perform weighted summation on low-dimensional dense vectors using importance weights to obtain text representation;

[0031] The fully connected layer is used to classify according to the text representation and obtain the classification result;

[0032] The output layer is used to output the classification results.

[0033] In one embodiment, the sample data also includes non-sensitive information sample data, and the method further includes:

[0034] By setting labels, sensitive information sample data is set as positive sample data, and non-sensitive information sample data is set as negative sample data;

[0035] The preset classification model is trained using positive sample data and negative sample data.

[0036] In one embodiment, after determining the sensitive information detection result of the data to be retrieved according to the classification result of the preset classification model, the method further includes:

[0037] If the sensitive information detection result indicates that the data to be retrieved contains sensitive information, sensitive text is generated based on the target data;

[0038] Writing sensitive text into a relational database;

[0039] The relational database is scanned regularly through a preset program, and when new sensitive text is scanned, a notification message is sent to the operation and maintenance terminal.

[0040] In one embodiment, the method further comprises:

[0041] Predefine vector templates of sensitive information and store them in a vector template library;

[0042] According to different categories of sensitive information, generate reference category vectors corresponding to each category;

[0043] Establish regular boundaries in the vector dimension space and generate vector restriction ranges;

[0044] Perform rule matching on the target vector, including:

[0045] Calculate the similarity between the target vector and each vector template in the vector template library respectively, and determine the first rule matching result based on the calculation results;

[0046] Calculate the similarity between the target vector and the reference category vector corresponding to each category respectively, and determine the second rule matching result based on the calculation result;

[0047] Determine whether the target vector matches the vector restriction range, and obtain the third rule matching result;

[0048] A rule matching result of the target vector is determined according to the first rule matching result, the second rule matching result, and the third rule matching result.

[0049] In a second aspect, the present application also provides a sensitive information detection device, comprising:

[0050] A preprocessing module is used to obtain the data to be retrieved; preprocess the data to be retrieved to obtain the target data;

[0051] A rule matching module is used to convert target data into a target vector; and perform rule matching on the target vector;

[0052] A retrieval module is used to perform similarity retrieval in a preset vector database according to the target vector if the rule is successfully matched; the preset vector database stores a number of sensitive information vectors;

[0053] The classification module is used to input the target vector into the preset classification model if a sensitive information vector that meets the similarity requirements is retrieved from the preset vector database; and determine the sensitive information detection result of the data to be retrieved based on the classification result of the preset classification model.

[0054] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0055] Get the data to be retrieved;

[0056] Preprocess the data to be retrieved to obtain target data;

[0057] Convert target data into target vector;

[0058] Perform rule matching on the target vector;

[0059] If the rule is matched successfully, a similarity search is performed in a preset vector database according to the target vector; the preset vector database stores several sensitive information vectors;

[0060] If a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into the preset classification model;

[0061] Based on the classification results of the preset classification model, the sensitive information detection results of the data to be retrieved are determined.

[0062] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0063] Get the data to be retrieved;

[0064] Preprocess the data to be retrieved to obtain target data;

[0065] Convert target data into target vector;

[0066] Perform rule matching on the target vector;

[0067] If the rule is matched successfully, a similarity search is performed in a preset vector database according to the target vector; the preset vector database stores several sensitive information vectors;

[0068] If a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into the preset classification model;

[0069] Based on the classification results of the preset classification model, the sensitive information detection results of the data to be retrieved are determined.

[0070] The above-mentioned sensitive information detection method, device, computer equipment and computer-readable storage medium obtain the data to be retrieved; pre-process the data to be retrieved to obtain the target data; convert the target data into a target vector; perform rule matching on the target vector; if the rule matching is successful, perform similarity search in a preset vector database based on the target vector; the preset vector database stores several sensitive information vectors; if a sensitive information vector that meets the similarity requirements is retrieved from the preset vector database, the target vector is input into a preset classification model; according to the classification result of the preset classification model, the sensitive information detection result of the data to be retrieved is determined. Through the above-mentioned method, through vectorized representation, the semantic features of the data to be retrieved can be accurately characterized, thereby improving the accuracy of sensitive information detection; on the basis of the conversion vector, combined with rule matching, the accuracy and recall rate of sensitive information detection can be improved; through the vector database, sensitive information similarity search can be performed, and all sensitive information vectors that meet the similarity requirements can be simultaneously retrieved, which is suitable for the retrieval scenario of multiple sensitive words; the category of the data is determined again through the preset classification model to avoid underreporting and false alarms, and further improve the accuracy of sensitive information detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0072] FIG1 is a diagram of an application environment of a sensitive information detection method according to an embodiment;

[0073] FIG2 is a schematic diagram of a process of a sensitive information detection method in one embodiment;

[0074] FIG3 is a schematic diagram of a flow chart of a sensitive information detection method in another embodiment;

[0075] FIG4 is a schematic diagram of the structure of a preset classification model in one embodiment;

[0076] FIG5 is a schematic diagram of a data processing flow in one embodiment;

[0077] FIG6 is a schematic diagram of a data processing flow in another embodiment;

[0078] FIG7 is a structural block diagram of a sensitive information detection device in one embodiment;

[0079] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0081] It is understandable that in the daily operation of an enterprise, it is often necessary to send notification messages to users through SMS, in-app messages, etc., for users to view or confirm. Since the enterprise's application system manages a large amount of user data and enterprise data, abnormal users may use the application system to leak user personal information, spread sensitive information, etc. For example, abnormal users send refund notices in the name of a bankrupt financial platform. Enterprises need to retrieve and screen the message records sent by each user.

[0082] Enterprises generally use keyword search solutions to search for messages containing suspicious keywords in message records. The specific method is to search for text containing these keywords in the document collection based on one or more keywords entered by the user. The usage scenarios of keyword search include, for example, searching for relevant web pages on search engines based on keywords entered by users; searching for products to be purchased on e-commerce platforms based on keywords such as product names entered by users; using desktop search or file search tools to quickly find required files; and searching for relevant files or materials in databases based on keywords. The main advantages of keyword search solutions are simplicity, intuitiveness, and ease of operation, and no need to build complex indexes and retrieval algorithms. However, it also has some obvious disadvantages: 1. Since keywords may have different meanings in different contexts, it is easy to cause misidentification of sensitive information. 2. The principle of keyword matching is complete matching, that is, the document must contain all the input keywords to be retrieved; this makes the recall rate low, and many relevant documents that do not directly use these keywords cannot be recalled. 3. As long as the document contains the input keywords, it will be retrieved, even if the subject content of the document does not match the user's query intent, which leads to poor precision. 4. Keyword search cannot discover the semantic information contained in the document and can only achieve accurate matching at the word level. 5. When users enter multiple keywords to search, the search results are often poor because it is difficult to determine the relationship between these keywords. These shortcomings lead to the inability of keyword search to achieve the expected search results in actual application scenarios.

[0083] Based on this, the present application provides a sensitive information detection method, apparatus, computer device and computer-readable storage medium. Compared with the keyword retrieval scheme, the method provided by the present application can accurately characterize the semantic features of the data to be retrieved through vectorized representation, thereby improving the accuracy of sensitive information detection; on the basis of the conversion vector, combined with rule matching, the accuracy and recall rate of sensitive information detection are improved; through the vector database, sensitive information similarity retrieval is performed, all sensitive information vectors that meet the similarity requirements can be simultaneously retrieved, which is suitable for retrieval scenarios of multiple sensitive words; the data category is determined again through a preset classification model to avoid underreporting and false alarms, further improving the accuracy of sensitive information detection.

[0084] The sensitive information detection method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 sends messages to other devices through the server 104; the server 104 records the messages sent by each terminal 102, and uses the sensitive information detection method provided in the present application to detect the messages sent by the terminal 102 or the message records stored in the data storage system to obtain sensitive information detection results. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0085] In an exemplary embodiment, as shown in FIG. 2 , a sensitive information detection method is provided, which is described by taking the method applied to the server 104 in FIG. 1 as an example, including:

[0086] Step 202, obtaining the data to be retrieved.

[0087] The data to be retrieved may be a message sent by the terminal through the server, or a message record stored in a database managed by the server, which is not limited in this embodiment. The data to be retrieved may be text data or audio data, and the subsequent processing steps are performed after the audio data is converted into text data.

[0088] Step 204, pre-process the data to be retrieved to obtain target data.

[0089] The target data is obtained by removing unnecessary information in the data to be retrieved through preprocessing, for example, removing unnecessary information such as spaces, special symbols, and URLs (Uniform Resource Locations).

[0090] Step 206, converting the target data into a target vector.

[0091] The target data is vectorized and converted by the semantic vector model to obtain the target vector. Optionally, the present embodiment uses the BGE-M3 semantic vector model for vector conversion, without consuming a large amount of computing resources and training time to train the semantic vector model.

[0092] It is understandable that although deep learning network models (such as convolutional neural networks (CNN), long short-term memory networks (LSTM), etc.) can extract high-level semantic features of text, they require a large amount of labeled data for training, and have high requirements on the quality and quantity of training data. The semantic vector model (Semantic Vector Model), as a model used to represent the semantic information of words or documents in text, can map text into a continuous vector space, enabling it to be operated and analyzed in the vector space. Common semantic vector models include Word2Vec, GloVe, BERT, etc. By calculating the distance between semantic vectors, the semantic similarity between texts can be measured. BGE-M3 is the first semantic embedding model that integrates the three major technical features of multi-linguality, multi-granularity, and multi-functionality, which greatly improves the usability of semantic embedding models in the real world.

[0093] Step 208: performing rule matching on the target vector.

[0094] Among them, based on the use of semantic vector models, combined with specific rules to form a hybrid model can further improve the accuracy and recall rate of sensitive information identification. Among them, the semantic vector model (BGE-M3) is used to generate deep semantic representations of text, and rule matching is used to match possible sensitive information. Rule matching has high accuracy, while the semantic vector model can identify more complex semantic information. Combining the advantages of both, the hybrid model can detect sensitive information more comprehensively.

[0095] Optionally, rule matching specifically includes: detection based on specific patterns, regular expressions or keywords, used to identify certain explicit or structured sensitive information (such as ID card numbers, credit card numbers, etc.).

[0096] Step 210: If the rule is matched successfully, a similarity search is performed in a preset vector database according to the target vector; the preset vector database stores a number of sensitive information vectors.

[0097] If the rule matching for the target vector meets certain requirements, the rule matching is successful, indicating that sensitive information may exist in the data to be retrieved, and similarity retrieval is continued to further detect sensitive information in the data to be retrieved. Alternatively, if the rule matching fails, it indicates that there is no sensitive information in the data to be retrieved.

[0098] Optionally, vector elements that successfully match the rule are intercepted from the target vector to form a new, shorter vector, and similarity search is performed in a preset vector database based on the formed vector.

[0099] Optionally, the target vector is segmented into a plurality of vector segments, and a similarity search is performed in a preset vector database for each vector segment.

[0100] In an optional implementation, the preset vector database uses the Elasticsearch vector database, which is a powerful distributed search and analysis engine that supports vector storage and search and can handle applications such as semantic search, similarity retrieval, and vector-based machine learning models. The similarity between the qualified vector and the sensitive information vector stored in the Elasticsearch vector database is calculated to determine whether a sensitive information vector with a similarity higher than a set threshold is retrieved.

[0101] Step 212: If a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into the preset classification model.

[0102] The similarity requirement is a similarity metric defined in advance based on actual needs. Optionally, the similarity requirement corresponds to a set threshold; if the similarity between a certain sensitive information vector retrieved and the target vector is higher than the set threshold, it indicates that the target vector and the retrieved sensitive information vector have a high degree of similarity, and it is determined that a sensitive information vector that meets the similarity requirement is retrieved. Optionally, if a sensitive information vector that meets the similarity requirement is not retrieved from the preset vector database, it indicates that there is no sensitive information in the data to be retrieved.

[0103] It is understandable that after similarity retrieval, a preset classification model is introduced to perform category identification and reconfirm the sensitive information in the data to be retrieved.

[0104] The preset classification model can be a decision tree model, a support vector machine model, a neural network model, etc. Optionally, the preset classification model is used for binary classification, that is, the preset classification model is used to determine whether the target vector belongs to a positive class or a negative class, the positive class indicates that the data to be retrieved contains sensitive information, and the negative class indicates that the data to be retrieved does not contain sensitive information. Optionally, the preset classification model is used to determine whether the data to be retrieved contains sensitive information, and is also used to determine the category of sensitive information in the data to be retrieved.

[0105] Step 214, determining the sensitive information detection result of the data to be retrieved according to the classification result of the preset classification model.

[0106] Among them, according to the classification result of the preset classification model, it is determined whether the data to be retrieved contains sensitive information, and the sensitive information detection result of the data to be retrieved is obtained.

[0107] In the above sensitive information detection method, the data to be retrieved is obtained; the data to be retrieved is preprocessed to obtain the target data; the target data is converted into a target vector; the target vector is matched by rules; if the rule matching is successful, a similarity search is performed in a preset vector database according to the target vector; a number of sensitive information vectors are stored in the preset vector database; if a sensitive information vector that meets the similarity requirements is retrieved from the preset vector database, the target vector is input into a preset classification model; according to the classification result of the preset classification model, the sensitive information detection result of the data to be retrieved is determined. In the above manner, through vectorized representation, the semantic features of the data to be retrieved can be accurately characterized, thereby improving the accuracy of sensitive information detection; on the basis of the conversion vector, combined with rule matching, the accuracy and recall rate of sensitive information detection can be improved; sensitive information similarity retrieval is performed through the vector database, and all sensitive information vectors that meet the similarity requirements can be simultaneously retrieved, which is suitable for retrieval scenarios of multiple sensitive words; the category of the data is determined again through the preset classification model to avoid underreporting and false positives, thereby further improving the accuracy of sensitive information detection.

[0108] In an exemplary embodiment, as shown in FIG3 , the method further includes:

[0109] Step 302: Receive a sample data set.

[0110] The server collects sample data sets in advance and can store the sample data sets in the form of EXCEL documents; the sample data sets include sensitive information samples and non-sensitive information samples. Optionally, operation and maintenance personnel and security personnel collect sensitive information samples and upload them to the server during daily inspections; customer service, success, sales and other business-related departments collect sensitive information samples and upload them to the server when handling complaints or feedback.

[0111] Step 304 , pre-process each piece of data in the sample data set to obtain a plurality of pieces of sample data.

[0112] For each piece of data in the sample data set, unnecessary information in each piece of data is removed through preprocessing to obtain sample data, for example, unnecessary information such as spaces, special symbols, and URLs (Uniform Resource Locations) are removed.

[0113] Optionally, after preprocessing, two queues are set to store sensitive information sample data and non-sensitive information sample data respectively.

[0114] Step 306: Convert the sensitive information sample data in the sample data into a sample vector.

[0115] For each record in the sensitive information sample data queue, the sensitive information sample data is vectorized and converted through the semantic vector model to obtain a sample vector. The description of step 306 can refer to the explanation of step 206, which will not be repeated here.

[0116] Step 308: Perform rule matching on the sample vector.

[0117] The description of step 308 may refer to the explanation of step 208 and will not be repeated here.

[0118] Step 310: The sample vector that successfully matches the rule is used as a sensitive information vector and stored in a preset vector database.

[0119] The sample vectors that successfully match the rules are stored in an index of the Elasticsearch vector database to facilitate subsequent similarity retrieval.

[0120] In this embodiment, a centralized method for storing and managing sensitive information vectors is provided through a preset vector database, providing data support for subsequent similarity retrieval; sensitive information sample data is verified through rule matching, and sensitive information vectors that successfully match the rules are stored, which can improve the quality of data stored in the preset vector database, thereby improving the accuracy of subsequent similarity retrieval.

[0121] In an exemplary embodiment, the method also includes: receiving sensitive data sent by the operation and maintenance terminal; converting the sensitive data into a sensitive information vector and updating it into a preset vector database; generating extended data based on the sensitive data; converting the extended data into an extended vector; performing rule matching on the extended vector; and using the extended vector that successfully matches the rules as a sensitive information vector and updating it into the preset vector database.

[0122] Among them, the operation and maintenance terminal refers to the terminal device that logs in to an account with operation and maintenance permissions. The sensitive data sent by the operation and maintenance terminal is new sensitive data or unreported sensitive data determined by manual investigation. The server vectorizes the sensitive data received through a semantic vector model to obtain a sensitive information vector, and expands the internal data of the preset vector database. In addition, the server expands the sensitive data uploaded by the operation and maintenance terminal, generates extended data, and vectorizes it through a semantic vector model to generate an extended vector; after the rule is successfully matched, it is updated to the preset vector database to achieve data expansion of the preset vector database. The method of generating extended data can be: based on semantic expansion of synonyms or related words of sensitive data; it can also be: generating new extended data through deformation (such as pinyin form, abbreviation form) and combination (such as combination between sensitive words).

[0123] In an optional implementation, the server optimizes the matching rules according to the sensitive data uploaded by the operation and maintenance terminal, so that the sensitive information vector corresponding to the sensitive data matches the optimized rules.

[0124] In an exemplary embodiment, the preset classification model includes an input layer, a word embedding layer, an attention layer, a weighted sum layer, a fully connected layer and an output layer; wherein: the input layer is used to receive a target vector; the word embedding layer is used to map the vocabulary included in the target vector into a low-dimensional dense vector; the attention layer is used to calculate the importance weight of each vocabulary; the weighted sum layer is used to perform weighted summation of the low-dimensional dense vector using the importance weight to obtain a text representation; the fully connected layer is used to classify according to the text representation to obtain a classification result; and the output layer is used to output the classification result.

[0125] Wherein, referring to FIG. 4, FIG. 4 shows a structure of a preset classification model, and the preset classification model includes an input layer, a word embedding layer, an attention layer, a weighted sum layer, a fully connected layer and an output layer. Optionally, the preset classification model is a classification model generated after FASTTEXT training, which can classify text data into different categories. FASTTEXT is a powerful text classification tool. It is based on the Skip-gram model of the bag-of-words model and the N-gram feature, and can quickly and efficiently process a large amount of text data. FASTTEXT is based on the Skip-gram model and attempts to predict the context of a word. During the training process, it represents each word as an N-gram vector, where N represents the length of the N-gram. Each word is represented by the N-1 words in front of it, thereby capturing the dependency between words. FASTTEXT also maps words to dense space through the Embedding layer to form word vectors. Compared with the traditional Word2Vec method, the word vector of FASTTEXT not only considers the context information of the word, but also considers the N-gram information of the word, so it can better capture the semantic information of the word. When it performs text classification, it represents the input text as the average value of word vectors. Therefore, by default, it lacks the ability to weight the importance of different words. In this embodiment, an attention mechanism is introduced in FASTTEXT to achieve data weighting. This not only improves the model's ability to identify sensitive information, but also retains the advantages of the FASTTEXT model, which is fast and lightweight.

[0126] The main features of FASTTEXT include: (1) High efficiency: FASTTEXT uses a shallow neural network, which is very fast to train and can quickly complete training and prediction even on large-scale data sets. (2) Word vector representation: FASTTEXT can represent words as vectors, which can capture the semantic information of words. Through these word vectors, FASTTEXT can better understand the semantics of the text. (3) Character-level features: FASTTEXT supports character-level N-gram features, which enables it to handle some spelling errors or variant words and improve classification results in some cases. (4) Multi-language support: FASTTEXT supports text classification in multiple languages, and texts in different languages ​​can also be well processed.

[0127] In the specific implementation, the word embedding layer maps the vocabulary into a low-dimensional dense vector, a linear layer calculates the importance weight of each word, and the softmax is used to obtain the normalized weight. The word vectors are weighted and summed using the attention weight to obtain the text representation. The final text representation is input to the fully connected layer for classification. By introducing the attention mechanism, the model can weight the vocabulary according to its importance in different texts, which helps the model capture key information.

[0128] For example, before calculating the average value of the word embedding vector, an attention mechanism is introduced to weight different word embedding vectors. The weight reflects the importance of each word to the current classification task. The specific implementation method is as follows:

[0129] 1. Calculate the attention score:

[0130] The embedding vector v for each word i , and calculates its attention score through a trainable attention layer. This attention layer can be a simple feedforward neural network (such as a fully connected layer). The calculation formula is as follows:

[0131] ;

[0132] Among them, u, w and b are trainable weight parameters, tanh is the activation function, is the attention score of the i-th word.

[0133] 2. Weighted Embedding Vector:

[0134] Weight the embedding vector of each word to get the weighted embedding vector: .

[0135] 3. Calculate the weighted average vector and take the weighted average of all words’ weighted embedding vectors as the representation of the sentence:

[0136] .

[0137] 4. Classification: The weighted average sentence is represented as Input to the original classification layer of FASTTEXT (such as linear classifier) ​​for prediction.

[0138] It is understandable that the FASTTEXT model used and the introduction of the attention mechanism only requires training on sample data, consumes less computing resources and training time, and achieves controllable costs; the semantic vector model is combined with specific rules to form a hybrid model, which is combined with the machine learning model that introduces the attention mechanism for cross-validation, greatly improving the recall and precision of sensitive information detection.

[0139] In an exemplary embodiment, the sample data also includes non-sensitive information sample data, and the method also includes: setting the sensitive information sample data as positive sample data and setting the non-sensitive information sample data as negative sample data by setting labels; and training a preset classification model using the positive sample data and the negative sample data.

[0140] Among them, the pre-processed sensitive information sample data and non-sensitive information sample data are stored in two queues, and the sample data are obtained from the queues and set with labels, such as "sensitive" and "non-sensitive". Select a classification model, use positive sample data (including sensitive information sample data and labels) and negative sample data (including non-sensitive information sample data and labels) to train the preset classification model, and when the number of iterative training reaches the set value or the calculated loss function value is less than a certain value, obtain the trained preset classification model and store it as a classification model file for subsequent use.

[0141] In an optional implementation, referring to FIG. 5 , FIG. 5 shows a data processing flow; specifically including: collecting a sample data set, preprocessing the data of the sample data set to obtain preprocessed sensitive information sample data and non-sensitive information sample data; based on the sensitive information sample data, vectorizing the data through a semantic vector model, and performing rule matching; after the rule matching is successful, storing the data in an Elasticsearch vector database; based on the sensitive information sample data and the non-sensitive information sample data, training a FASTTEXT model that introduces an attention mechanism to generate a classification model file.

[0142] In an exemplary embodiment, after step 214, the method further includes: if the sensitive information detection result indicates that the data to be retrieved contains sensitive information, generating sensitive text based on the target data; writing the sensitive text into a relational database; regularly scanning the relational database through a preset program, and sending a notification message to the operation and maintenance terminal when newly added sensitive text is scanned.

[0143] Among them, the data to be retrieved is determined to be sensitive data through the method provided in the embodiment of the present application, and the corresponding sensitive text is written into the relational database. The relational database is scanned regularly through a preset program, and the operation and maintenance personnel are notified in time for management and control when there is new sensitive data.

[0144] In an optional implementation, referring to FIG. 6 , FIG. 5 shows a data processing flow; specifically including: preprocessing the data to be retrieved; vectorizing the preprocessed data to be retrieved through a semantic vector model, and performing rule matching; after the rule matching is successful, performing similarity retrieval based on the Elasticsearch vector database; further performing category comparison for data that meets the similarity requirements through the FASTTEXT model that introduces an attention mechanism; if the data is determined to be sensitive through the above steps, the text is written into a relational database; a notification program is sent to scan the database regularly, and the operation and maintenance personnel are notified of new sensitive data.

[0145] In an exemplary embodiment, the method further includes: predefining vector templates of sensitive information and storing them in a vector template library; generating reference category vectors corresponding to different categories of sensitive information; establishing rule boundaries in the vector dimension space and generating a vector restriction range;

[0146] Step 208 includes: calculating the similarity between the target vector and each vector template in the vector template library, and determining the first rule matching result based on the calculation results; calculating the similarity between the target vector and the reference category vector corresponding to each category, and determining the second rule matching result based on the calculation results; judging whether the target vector matches the vector restriction range to obtain the third rule matching result; determining the rule matching result of the target vector according to the first rule matching result, the second rule matching result and the third rule matching result.

[0147] Among them, predefined vector templates can generate vector representations of known sensitive information (such as sensitive words of a specific category, identity information, etc.) in advance to form a vector template library. For the vector to be matched, the similarity between it and each vector representation in the vector template library is calculated to determine whether it conforms to a specific pattern. This method can be understood as converting keywords and regular expressions from text mode to vector mode.

[0148] Different categories of sensitive information (such as bank account numbers, emails, etc.) are converted into category vectors in advance. For the vector to be matched, the similarity between it and different category vectors is calculated, the most suitable category is found, and rule matching is performed. In this way, sensitive information is classified through vector matching.

[0149] Since some vectors may have significant sensitive features in specific dimensions, for example, for a certain type of sensitive information, the model may embed it in a relatively concentrated value in a specific dimension. In this case, upper and lower limit rules can be set for these dimensions, and only vectors that meet the specific dimensional range can be matched successfully. This method can be understood as establishing a "rule boundary" in the dimensional space of the vector to restrict the sensitive information vector.

[0150] It can be understood that if the similarity between the target vector and any vector template in the vector template library is higher than a certain value (adjustable parameter, which can be set according to actual conditions), the first rule matching result is determined to be a successful match. If the similarity between the target vector and any reference category vector is higher than a threshold value (adjustable parameter, which can be set according to actual conditions), the second rule matching result is determined to be a successful match. If the target vector matches the vector restriction range, the third rule matching result is determined to be a successful match. If any one of the first rule matching result, the second rule matching result and the third rule matching result is a successful match, the rule matching result of the target vector is determined to be a successful match.

[0151] In an optional implementation, all collected sensitive sample data is tested through the same code in the test environment to determine whether the code can normally identify sensitive data. If it is missed or misidentified, the specific reasons are analyzed and the code is optimized. Code optimization includes adjusting rules or generating new rules, adjusting parameters of preset classification models, adjusting similarity retrieval mechanisms, etc. Optionally, in the test environment, A / B testing is performed to compare the effects of different optimization solutions or new rules to ensure that the adjustments can bring actual improvements.

[0152] It can be understood that in the sensitive information detection method provided by the embodiment of the present application, the semantic features of sensitive information can be accurately characterized through vectorized representation, thereby improving the accuracy of sensitive information detection. The open source semantic vector model is directly used, and there is no need to consume a lot of computing resources and training time to train the semantic vector model. On the basis of using a semantic vector model (such as BGE-M3), a hybrid model is formed in combination with specific rules to further improve the accuracy and recall of sensitive information identification. With the help of the vector storage capability of Elasticsearch, the entire sample data can be effectively maintained, which is convenient for viewing and updating operations. Based on the preset classification model, the category of the data is reconfirmed to avoid omissions and false positives. Using the open source FASTTEXT model as the preset classification model, only the sample data needs to be trained, and the computing resources and training time consumed are less, so that the cost can be controlled. The introduction of the attention mechanism can generate a more fine-grained and context-related sentence representation by weighted summation of different words, which can capture words with stronger semantic influence in the sentence, and then generate a more accurate overall text representation. The modular decoupled design is flexible and convenient for subsequent model switching and system expansion. The automated notification mechanism improves response efficiency and helps to quickly deal with suspicious behavior. It has low requirements for computing resources and training time, providing a low-cost, fast-to-online solution for identifying and preventing sensitive information.

[0153] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0154] Based on the same inventive concept, the embodiment of the present application also provides a sensitive information detection device for implementing the sensitive information detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more sensitive information detection device embodiments provided below can refer to the limitations on the sensitive information detection method above, and will not be repeated here.

[0155] In an exemplary embodiment, as shown in FIG7 , a sensitive information detection device is provided, comprising:

[0156] The preprocessing module 702 is used to obtain the data to be retrieved; preprocess the data to be retrieved to obtain the target data.

[0157] The rule matching module 704 is used to convert the target data into a target vector and perform rule matching on the target vector.

[0158] The search module 706 is used to perform similarity search in a preset vector database according to the target vector if the rule matching is successful; the preset vector database stores a number of sensitive information vectors.

[0159] The classification module 708 is used to input the target vector into the preset classification model if a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database; and determine the sensitive information detection result of the data to be retrieved according to the classification result of the preset classification model.

[0160] In the above-mentioned sensitive information detection device, the data to be retrieved is obtained; the data to be retrieved is preprocessed to obtain the target data; the target data is converted into a target vector; the target vector is matched by rules; if the rule matching is successful, a similarity search is performed in a preset vector database according to the target vector; a number of sensitive information vectors are stored in the preset vector database; if a sensitive information vector that meets the similarity requirements is retrieved from the preset vector database, the target vector is input into a preset classification model; according to the classification result of the preset classification model, the sensitive information detection result of the data to be retrieved is determined. Through the above-mentioned method, the semantic features of the data to be retrieved can be accurately characterized through vectorized representation, thereby improving the accuracy of sensitive information detection; on the basis of the conversion vector, combined with rule matching, the accuracy and recall rate of sensitive information detection can be improved; sensitive information similarity retrieval is performed through the vector database, and all sensitive information vectors that meet the similarity requirements can be simultaneously retrieved, which is suitable for retrieval scenarios of multiple sensitive words; the category of the data is determined again through the preset classification model to avoid underreporting and false alarms, and further improve the accuracy of sensitive information detection.

[0161] In an exemplary embodiment, the sensitive information detection device also includes a database processing module; the database processing module is used to receive a sample data set; pre-process each data in the sample data set to obtain a plurality of sample data; convert the sensitive information sample data in the sample data into a sample vector; perform rule matching on the sample vector; and store the sample vector that successfully matches the rule as a sensitive information vector in a preset vector database.

[0162] In an exemplary embodiment, the sensitive information detection device also includes a database update module; the database update module is used to receive sensitive data sent by the operation and maintenance terminal; convert the sensitive data into a sensitive information vector, and update it into a preset vector database; generate extended data based on the sensitive data; convert the extended data into an extended vector; perform rule matching on the extended vector; and use the extended vector that successfully matches the rules as a sensitive information vector, and update it into the preset vector database.

[0163] In an exemplary embodiment, the preset classification model includes an input layer, a word embedding layer, an attention layer, a weighted sum layer, a fully connected layer and an output layer; wherein: the input layer is used to receive a target vector; the word embedding layer is used to map the vocabulary included in the target vector into a low-dimensional dense vector; the attention layer is used to calculate the importance weight of each vocabulary; the weighted sum layer is used to perform weighted summation of the low-dimensional dense vector using the importance weight to obtain a text representation; the fully connected layer is used to classify according to the text representation to obtain a classification result; and the output layer is used to output the classification result.

[0164] In an exemplary embodiment, the sample data also includes non-sensitive information sample data, and the sensitive information detection device also includes a model training module; the model training module is used to set the sensitive information sample data as positive sample data and the non-sensitive information sample data as negative sample data by setting labels; and use the positive sample data and the negative sample data to train the preset classification model.

[0165] In an exemplary embodiment, the sensitive information detection device also includes a notification module; the notification module is used to generate sensitive text based on the target data if the sensitive information detection result indicates that the data to be retrieved contains sensitive information; write the sensitive text into a relational database; regularly scan the relational database through a preset program, and send a notification message to the operation and maintenance terminal when new sensitive text is scanned.

[0166] In an exemplary embodiment, the sensitive information detection device further includes a pre-definition module; the notification module is used to pre-define vector templates of sensitive information and store them in a vector template library; generate reference category vectors corresponding to different categories of sensitive information; establish rule boundaries in the vector dimension space and generate a vector restriction range;

[0167] The rule matching module 704 is also used to respectively calculate the similarity between the target vector and each vector template in the vector template library, and determine the first rule matching result based on the calculation results; respectively calculate the similarity between the target vector and the reference category vector corresponding to each category, and determine the second rule matching result based on the calculation results; determine whether the target vector matches the vector restriction range to obtain the third rule matching result; determine the rule matching result of the target vector according to the first rule matching result, the second rule matching result and the third rule matching result.

[0168] Each module in the above-mentioned sensitive information detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0169] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sensitive information vectors, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sensitive information detection method is implemented.

[0170] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining data to be retrieved; preprocessing the data to be retrieved to obtain target data; converting the target data into a target vector; performing rule matching on the target vector; if the rule matching is successful, performing a similarity search in a preset vector database based on the target vector; a plurality of sensitive information vectors are stored in the preset vector database; if a sensitive information vector that meets the similarity requirements is retrieved from the preset vector database, the target vector is input into a preset classification model; and determining a sensitive information detection result of the data to be retrieved based on the classification result of the preset classification model.

[0172] In one embodiment, when the processor executes the computer program, the following steps are also implemented: receiving a sample data set; preprocessing each piece of data in the sample data set to obtain a plurality of sample data; converting sensitive information sample data in the sample data into sample vectors; performing rule matching on the sample vectors; and storing the sample vectors that successfully match the rules as sensitive information vectors in a preset vector database.

[0173] In one embodiment, when the processor executes the computer program, the following steps are also implemented: receiving sensitive data sent by the operation and maintenance terminal; converting the sensitive data into a sensitive information vector, and updating it into a preset vector database; generating extended data based on the sensitive data; converting the extended data into an extended vector; performing rule matching on the extended vector; and using the extended vector that successfully matches the rules as a sensitive information vector, and updating it into the preset vector database.

[0174] In one embodiment, when the processor executes the computer program, the following steps are also implemented: by setting labels, the sensitive information sample data is set as positive sample data, and the non-sensitive information sample data is set as negative sample data; and the preset classification model is trained using the positive sample data and the negative sample data.

[0175] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if the sensitive information detection result indicates that the data to be retrieved contains sensitive information, sensitive text is generated based on the target data; the sensitive text is written into a relational database; the relational database is regularly scanned through a preset program, and when new sensitive text is scanned, a notification message is sent to the operation and maintenance terminal.

[0176] In one embodiment, when the processor executes the computer program, the following steps are also implemented: pre-define vector templates of sensitive information and store them in a vector template library; generate reference category vectors corresponding to different categories of sensitive information; establish rule boundaries in the vector dimensional space to generate a vector restriction range; respectively calculate the similarity between the target vector and each vector template in the vector template library, and determine a first rule matching result based on the calculation results; respectively calculate the similarity between the target vector and the reference category vector corresponding to each category, and determine a second rule matching result based on the calculation results; determine whether the target vector matches the vector restriction range to obtain a third rule matching result; and determine the rule matching result of the target vector based on the first rule matching result, the second rule matching result and the third rule matching result.

[0177] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining data to be retrieved; preprocessing the data to be retrieved to obtain target data; converting the target data into a target vector; performing rule matching on the target vector; if the rule matching is successful, performing a similarity search in a preset vector database based on the target vector; the preset vector database stores a number of sensitive information vectors; if a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into a preset classification model; and determining a sensitive information detection result of the data to be retrieved based on the classification result of the preset classification model.

[0178] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: receiving a sample data set; preprocessing each piece of data in the sample data set to obtain a plurality of sample data; converting sensitive information sample data in the sample data into sample vectors; performing rule matching on the sample vectors; and storing the sample vectors that successfully match the rules as sensitive information vectors in a preset vector database.

[0179] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: receiving sensitive data sent by the operation and maintenance terminal; converting the sensitive data into a sensitive information vector, and updating it into a preset vector database; generating extended data based on the sensitive data; converting the extended data into an extended vector; performing rule matching on the extended vector; and using the extended vector that successfully matches the rules as a sensitive information vector, and updating it into the preset vector database.

[0180] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: by setting labels, sensitive information sample data is set as positive sample data, and non-sensitive information sample data is set as negative sample data; and a preset classification model is trained using the positive sample data and the negative sample data.

[0181] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: if the sensitive information detection result indicates that the data to be retrieved contains sensitive information, sensitive text is generated based on the target data; the sensitive text is written into a relational database; the relational database is regularly scanned through a preset program, and when new sensitive text is scanned, a notification message is sent to the operation and maintenance terminal.

[0182] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: pre-define vector templates of sensitive information and store them in a vector template library; generate reference category vectors corresponding to different categories of sensitive information; establish rule boundaries in the vector dimensional space to generate a vector restriction range; calculate the similarity between the target vector and each vector template in the vector template library, and determine a first rule matching result based on the calculation results; calculate the similarity between the target vector and the reference category vector corresponding to each category, and determine a second rule matching result based on the calculation results; determine whether the target vector matches the vector restriction range to obtain a third rule matching result; determine the rule matching result of the target vector based on the first rule matching result, the second rule matching result and the third rule matching result.

[0183] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0184] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0185] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0186] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A sensitive information detection method, characterized in that: The method comprises: Get the data to be retrieved; Preprocessing the data to be retrieved to obtain target data; Converting the target data into a target vector; Performing rule matching on the target vector; If the rule is successfully matched, a similarity search is performed in a preset vector database according to the target vector; the preset vector database stores a number of sensitive information vectors; If a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database, the target vector is input into a preset classification model; According to the classification result of the preset classification model, the sensitive information detection result of the data to be retrieved is determined.

2. The method according to claim 1, characterized in that The method further comprises: Receive a sample dataset; Preprocessing each piece of data in the sample data set to obtain a plurality of pieces of sample data; Converting sensitive information sample data in the sample data into a sample vector; Performing rule matching on the sample vector; The sample vectors that successfully match the rules are taken as sensitive information vectors and stored in the preset vector database.

3. The method according to claim 2, characterized in that The method further comprises: Receive sensitive data sent by operation and maintenance terminals; Convert the sensitive data into a sensitive information vector and update the vector into the preset vector database; Generate extended data according to the sensitive data; Converting the extended data into an extended vector; Performing rule matching on the extended vector; The extended vector that successfully matches the rule is used as a sensitive information vector and updated into the preset vector database.

4. The method according to claim 1, characterized in that: The preset classification model includes an input layer, a word embedding layer, an attention layer, a weighted sum layer, a fully connected layer and an output layer; wherein: The input layer is used to receive the target vector; The word embedding layer is used to map the vocabulary included in the target vector into a low-dimensional dense vector; The attention layer is used to calculate the importance weight of each word; The weighted summation layer is used to perform weighted summation on the low-dimensional dense vector using the importance weight to obtain a text representation; The fully connected layer is used to perform classification according to the text representation to obtain a classification result; The output layer is used to output the classification result.

5. The method according to claim 2, characterized in that: The sample data also includes non-sensitive information sample data, and the method further includes: By setting labels, the sensitive information sample data is set as positive sample data, and the non-sensitive information sample data is set as negative sample data; The positive sample data and the negative sample data are used to train a preset classification model.

6. The method according to any one of claims 1 to 5, characterized in that After determining the sensitive information detection result of the data to be retrieved according to the classification result of the preset classification model, the method further includes: If the sensitive information detection result indicates that the data to be retrieved contains sensitive information, generating sensitive text based on the target data; Writing the sensitive text into a relational database; The relational database is scanned regularly through a preset program, and when new sensitive text is scanned, a notification message is sent to the operation and maintenance terminal.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Predefine vector templates of sensitive information and store them in a vector template library; According to different categories of sensitive information, generate reference category vectors corresponding to each category; Establish regular boundaries in the vector dimension space and generate vector restriction ranges; The performing rule matching on the target vector includes: Calculating the similarity between the target vector and each vector template in the vector template library respectively, and determining a first rule matching result based on the calculation results; Calculating the similarity between the target vector and the reference category vector corresponding to each category respectively, and determining the second rule matching result based on the calculation result; Determine whether the target vector matches the vector restriction range, and obtain a third rule matching result; A rule matching result of the target vector is determined according to the first rule matching result, the second rule matching result, and the third rule matching result.

8. A sensitive information detection device, characterized in that: The device comprises: A preprocessing module is used to obtain the data to be retrieved; preprocess the data to be retrieved to obtain target data; A rule matching module, used for converting the target data into a target vector; and performing rule matching on the target vector; A retrieval module, configured to perform a similarity search in a preset vector database according to the target vector if the rule is successfully matched; the preset vector database stores a plurality of sensitive information vectors; A classification module is used to input the target vector into a preset classification model if a sensitive information vector that meets the similarity requirement is retrieved from the preset vector database; and determine the sensitive information detection result of the data to be retrieved based on the classification result of the preset classification model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images