Sensitive data detection method, related device and medium

Abstract

The embodiment of the invention provides a sensitive data detection method, a related device and a medium. The method comprises the following steps: acquiring to-be-detected data; a double-encoder rule generation model is called to convert the user query instruction into first vector data, text information of a knowledge source library is converted into second vector data, target knowledge text information is determined according to the first vector data and the second vector data, and the knowledge source library at least comprises one of a law and regulation library, an industry word library and an enterprise database; calling a dual-encoder rule generation model to generate a first detection rule according to the templated instruction and the target knowledge text information; calling a large language model to carry out adaptive optimization on the first detection rule to obtain an optimized second detection rule; and calling the multi-mode joint detection model to perform sensitive data detection on the to-be-detected data according to a second detection rule to obtain a sensitive data detection report. The embodiment of the invention aims to improve the accuracy and timeliness of sensitive data detection.

Description

Technical Field

[0001] This disclosure relates to the field of data security technology, and in particular to a sensitive data detection method, related apparatus and medium. Background Art

[0002] Sensitive data detection technology is a fundamental part of data governance and model training, and a core means of ensuring data security and compliance. Its core objective is to identify and monitor sensitive content such as personal identification information, financial data, and medical records to prevent leakage or misuse.

[0003] In related technologies, reliance on static rule bases or shallow models suffers from problems such as rule lag, insufficient semantic understanding, and weak multimodal support, leading to high false positive rates and difficulty in handling new sensitive data and dynamically updated regulations. How to improve the accuracy and timeliness of sensitive data detection is an urgent issue that needs to be discussed. Summary of the Invention

[0004] This disclosure provides a sensitive data detection method, related apparatus, and medium, aiming to improve the accuracy and timeliness of sensitive data detection.

[0005] In a first aspect, embodiments of this disclosure provide a sensitive data detection method, the method comprising:

[0006] Acquire the data to be tested;

[0007] The dual encoder rule generation model is invoked to convert user query instructions into first vector data and text information from the knowledge source database into second vector data. The target knowledge text information is determined based on the first vector data and the second vector data. The knowledge source database includes at least one of the following: a regulatory database, an industry thesaurus, and an enterprise database.

[0008] The dual encoder rule generation model is invoked to generate a first detection rule based on the templated instructions and the target knowledge text information;

[0009] The first detection rule is adapted and optimized by calling a large language model to obtain the optimized second detection rule;

[0010] The multimodal joint detection model is invoked to perform sensitive data detection on the data to be detected according to the second detection rule, and a sensitive data detection report is obtained.

[0011] Secondly, embodiments of this disclosure provide a sensitive data detection device, comprising:

[0012] The data acquisition unit is used to acquire the data to be detected;

[0013] The knowledge retrieval unit is used to call the dual encoder rule generation model to convert the user query command into first vector data, convert the text information of the knowledge source database into second vector data, and determine the target knowledge text information based on the first vector data and the second vector data. The knowledge source database includes at least one of the following: a legal database, an industry thesaurus, and an enterprise database.

[0014] The rule generation unit is used to call the dual encoder rule generation model to generate a first detection rule based on the templated instructions and the target knowledge text information;

[0015] The rule optimization unit is used to call the large language model to adapt and optimize the first detection rule to obtain the optimized second detection rule.

[0016] The data detection unit is used to call the multimodal joint detection model to perform sensitive data detection on the data to be detected according to the second detection rule, and obtain a sensitive data detection report.

[0017] Optionally, the knowledge retrieval unit is specifically used for:

[0018] The text in the legal database is segmented to obtain multiple segments of legal text.

[0019] Obtain the metadata tags for each segment of the regulatory text, wherein the metadata tags include the clause number, effective date, and scope of application;

[0020] The legal texts in each segment are converted into vector representations, and the vector representations corresponding to the legal texts in each segment are associated with the metadata tags to obtain the second vector data corresponding to the legal database.

[0021] Optionally, the dual-encoder rule generation model is trained through the following steps:

[0022] Obtain multiple first sample data;

[0023] Determine the query vector, positive sample embedding vector, and negative sample embedding vector corresponding to each of the first sample data;

[0024] The dual encoder rule generation model is modified based on the similarity between each query vector and the corresponding positive sample embedding vector, and the similarity between each query vector and the corresponding negative sample embedding vector.

[0025] Optionally, the dual-encoder rule generation model is further trained through the following steps:

[0026] Obtain multiple second sample data;

[0027] Multiple second sample data are respectively input into the dual encoder rule generation model and the pre-set strong supervision model to obtain the first output result of the dual encoder rule generation model and the second output result of the strong supervision model;

[0028] The dual encoder rule generation model is corrected based on the difference between the first output and the second output.

[0029] Optionally, the rule generation unit is specifically used for:

[0030] The target knowledge text information is vector-converted with the corresponding vertical domain vocabulary, and semantic association between the target knowledge text information and the vertical domain vocabulary is performed through word vector space alignment to obtain fused information.

[0031] The rule generation conditions are determined based on the templated instructions;

[0032] Based on the rule generation conditions, the first detection rule is generated according to the target knowledge text information, the information of the vertical domain vocabulary library, and the fusion information.

[0033] Optionally, the multimodal joint detection model includes a text detection module and an unstructured data detection module; the data detection unit is specifically used for:

[0034] The text detection module performs detection on named entities of the text data in the data to be detected according to the second detection rule to obtain the sensitive data detection report;

[0035] And / or,

[0036] The unstructured data detection module performs multi-module feature extraction on the unstructured data to be detected to obtain a feature vector; the feature text information is determined based on the feature vector; and the named entities of the feature text information are detected according to the second detection rule to obtain the sensitive data detection report.

[0037] Optionally, the multimodal joint detection model further includes a parameter steganalysis detection module; the data detection unit is specifically used for:

[0038] Obtain the historical model parameters of the text detection module and the unstructured data detection module;

[0039] The historical model parameters are processed in layers, and the distribution characteristics of the historical model parameters in each layer are statistically analyzed to generate a baseline distribution model.

[0040] The current model parameters of the text detection module and the unstructured data detection module are divided into the same layer, and the KL divergence value between the current model parameters of each layer and the baseline distribution corresponding to the baseline distribution model is calculated.

[0041] If the KL divergence value is greater than a preset threshold, it is determined that the current model parameters of the corresponding layer are abnormal.

[0042] Thirdly, embodiments of this disclosure provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the sensitive data detection method as described in the first aspect.

[0043] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, wherein when a computer executes the computer program, it implements the sensitive data detection method as described in the first aspect.

[0044] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program or computer instructions, the computer program or computer instructions being stored in a computer-readable storage medium, a processor of a computer device reading the computer program or computer instructions from the computer-readable storage medium, and the processor executing the computer program or computer instructions to cause the computer device to perform the sensitive data detection method as described in the first aspect.

[0045] This disclosure employs a dual-encoder rule generation model combined with user query commands to retrieve the required target knowledge text information from a knowledge source library containing legal databases, industry thesaurus, and enterprise databases. A first detection rule is then generated using templated commands. By retrieving the knowledge source library and dynamically generating detection rules based on current legal and industry thesaurus information, the detection rules are better adapted to current industry conditions and regulatory requirements, avoiding the lag of existing static rules. This detection rule generation mechanism allows for timely responses to changes in regulations and industries, ensuring the accuracy and timeliness of sensitive data detection. Furthermore, this disclosure optimizes the first detection rule using a large language model to determine an optimized second detection rule. Sensitive data detection is then performed using this optimized rule, leveraging the semantic parsing capabilities of the large language model. This allows the optimized second detection rule to better adapt to sensitive word detection in complex scenarios, thereby improving the detection accuracy and reducing false positive rates in subsequent sensitive word detection using the second detection rule.

[0046] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0047] Figure 1 A system architecture diagram of the sensitive data detection method used in embodiments of this disclosure;

[0048] Figure 2 This is a flowchart of a sensitive data detection method according to an embodiment of the present disclosure;

[0049] Figure 3 for Figure 2 A flowchart of step 220, which converts the text information of the knowledge source base into second vector data;

[0050] Figure 4 for Figure 2 A flowchart of step 230 for generating the first detection rule;

[0051] Figure 5 for Figure 2 A flowchart for step 250 of the process of detecting sensitive data;

[0052] Figure 6 A flowchart for performing sensitive data detection is provided as an example of this disclosure;

[0053] Figure 7 A schematic diagram illustrating the performance of sensitive data detection as an example of this disclosure;

[0054] Figure 8 This is a schematic diagram of the structure of the sensitive data detection device provided in the embodiments of this disclosure;

[0055] Figure 9 Based on embodiments of this disclosure Figure 2 The diagram shows the terminal structure of the sensitive data detection method.

[0056] Figure 10 Based on embodiments of this disclosure Figure 2 The server architecture diagram for the sensitive data detection method is shown. Detailed Implementation

[0057] To make the objectives, technical methods, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0058] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0059] In the description of the embodiments of this disclosure, unless otherwise expressly limited, terms such as "setting," "installing," and "connecting" should be interpreted broadly. Those skilled in the art can reasonably determine the specific meaning of the above terms in the embodiments of this disclosure in conjunction with the specific content of the technical solution. In the embodiments of this disclosure, terms such as "furthermore," "exemplarily," or "optionally" are used to indicate examples, illustrations, or explanations, and should not be construed as being more preferred or more advantageous than other embodiments or design solutions. The use of terms such as "furthermore," "exemplarily," or "optionally" is intended to present the relevant concepts in a specific manner.

[0060] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0061] RAG framework: Retrieval-Augmented Generation. This framework is a technology system that deeply integrates external knowledge bases with language models. By dynamically retrieving and supplementing real-time or domain knowledge, it significantly improves the accuracy and timeliness of generated content.

[0062] Large language models are a class of artificial intelligence models based on deep learning architectures and pre-trained on massive amounts of unlabeled text data. Their core characteristics lie in their ability to handle hundreds of billions of parameters and their powerful contextual semantic understanding capabilities. These models learn the statistical patterns and semantic structures of human language from large-scale corpora such as internet text, books, and code through self-supervised learning methods (such as masked language models and causal language modeling). They can perform various natural language processing tasks such as text generation, question answering, reasoning, and translation.

[0063] Strongly supervised models are machine learning models that rely on high-quality, precisely labeled supervisory signals (such as explicit class labels, bounding boxes, sequence labels, etc.) for learning during training. Their core characteristic is that each input sample in the training data corresponds to a clear and accurate target output. The model directly fits the mapping relationship between input and output by optimizing the loss function (such as cross-entropy or mean squared error), making it the most typical paradigm in supervised learning.

[0064] The Prompt framework refers to a structured approach in Large Language Modeling (LLM) applications that uses prompts to systematically guide the model in generating expected outputs. Its core principle is to break down task requirements into reusable components, improving the efficiency, controllability, and generalization ability of the prompts through standardized structures.

[0065] Reinforcement Learning from Human Feedback (RLHF) is a technological paradigm that uses human preference data to build reward models and drive reinforcement learning to optimize AI behavior. It aims to solve the problem of difficulty in quantifying objectives in complex tasks and to align model output with human expectations.

[0066] Named Entity Recognition (NRE) models are fundamental tasks in Natural Language Processing (NLP), aiming to identify entities with specific meanings from text and classify them into predefined categories (such as names of people, places, and organizations). NRE is a crucial preliminary step for downstream tasks such as information extraction, question answering systems, and machine translation, and its accuracy directly affects the effectiveness of subsequent processing.

[0067] CLIP (Contrastive Language-Image Pre-training) model: This is a multimodal pre-trained model based on contrastive learning. Its core is to train the model using large-scale image-text data pairs, aligning image and text features in a shared semantic space to achieve cross-modal understanding.

[0068] Optical Character Recognition (OCR) aims to convert text information in images and videos into editable and searchable text. It is an interdisciplinary application of computer vision and natural language processing.

[0069] The Transformer-Decoder architecture is one of the core components of the Transformer model, specifically responsible for sequence generation tasks (such as text generation, machine translation, code completion, etc.). Its design revolves around the logic of "producing the output sequence step by step based on input information," and achieves the modeling of temporal dependencies through a combination of multi-layer attention mechanisms and feedforward networks.

[0070] The T5 (Text-to-Text Transfer Transformer) model is a general-purpose NLP pre-trained model whose core idea is to unify all natural language processing tasks (such as translation, summarization, question answering, etc.) into a "text-to-text" conversion problem.

[0071] BART (Bidirectional and Auto-Regressive Transformers) model: A pre-trained language model, essentially a denoising autoencoder. It combines bidirectional contextual understanding with autoregressive generation capabilities, achieving a unity of text understanding and generation by reconstructing the original text corrupted by noise.

[0072] Hard Negative Mining is a technique in machine learning that optimizes the training process. Its core objective is to improve the model's discriminative ability by actively selecting negative samples that the model has difficulty distinguishing (i.e., "hard negative samples").

[0073] Among related technologies, sensitive data detection technology is a fundamental step in data governance and model training, and a core means of ensuring data security and compliance. Its core objective is to identify and monitor sensitive content such as personal identification information, financial data, and medical records to prevent leakage or misuse.

[0074] In related technologies, reliance on static rule bases or shallow models suffers from problems such as rule lag, insufficient semantic understanding, and weak multimodal support, leading to high false positive rates and difficulty in handling new sensitive data and dynamically updated regulations. Differences in industry scenarios further exacerbate the difficulty of rule adaptation, while traditional manual maintenance methods are costly and prone to policy conflicts. How to improve the accuracy and timeliness of sensitive data detection is a pressing issue that needs to be addressed.

[0075] Based on this, embodiments of this disclosure provide a sensitive data detection method, related apparatus, and medium. By combining a dual-encoder rule generation model with user query commands, the required target knowledge text information is retrieved from a knowledge source library containing a regulatory database, industry thesaurus, and enterprise database. Then, a first detection rule is generated using templated commands. By retrieving the knowledge source library and dynamically generating detection rules based on current regulatory and industry thesaurus information, the detection rules can be better adapted to current industry conditions and regulatory requirements, avoiding the lag of existing static rules. Through the above-mentioned detection rule generation mechanism, timely responses to changes in regulations and industries can be made, ensuring the accuracy and timeliness of sensitive data detection. This disclosure also optimizes the first detection rule using a large language model to determine an optimized second detection rule. Sensitive data detection is then performed using the optimized detection rule, integrating the semantic parsing capabilities of the large language model. This allows the optimized second detection rule to better adapt to sensitive word detection in complex scenarios, thereby improving the detection accuracy and reducing the false positive rate in subsequent sensitive word detection using the second detection rule.

[0076] System architecture and scenario description of the embodiments disclosed herein

[0077] Figure 1This is a system architecture diagram of the sensitive data detection method applied according to embodiments of the present disclosure. It includes a target terminal 130, an Internet connection 120, a server 110, etc.

[0078] Server 110 refers to a computer system capable of providing sensitive data detection capabilities to target terminal 130. Compared to target terminal 130, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can also be a database capable of providing data acquisition services to target terminal 130. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). Server 110 can also communicate with target terminal 130 and exchange data via wired or wireless means, or through the Internet 120.

[0079] The object terminal 130 is a device used to display the results of sensitive data detection, the user query command input interface, etc., allowing the object to view them, or providing the object with an interface for uploading and configuring sensitive data detection-related task information. The object can input sensitive data detection commands, model training datasets, and / or various task information into the object terminal 130. The object terminal 130 can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, dedicated terminals, and graphics tablets. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple devices can be connected via a local area network, sharing a single display device to work collaboratively, forming a single terminal. The terminal can also communicate with the server 110 via wired or wireless means, or via the Internet 120, to exchange data.

[0080] General Description of Embodiments in this Disclosure

[0081] It should be noted that the embodiments disclosed herein can be applied to fields such as finance and healthcare, specifically in scenarios such as payment transactions, account management, clinical data management, and electronic medical record management. In related technologies, relying on static rule bases or shallow models for sensitive data identification and detection suffers from problems such as rule lag, insufficient semantic understanding, and weak multimodal support. Some embodiments of this disclosure provide a sensitive data detection method, related apparatus, and medium, aiming to improve the accuracy and timeliness of sensitive data detection.

[0082] The sensitive data detection method disclosed herein is a method for detecting sensitive data by retrieving external knowledge from a knowledge source base to dynamically generate detection rules with higher effectiveness and accuracy, and optimizing and adapting the detection rules to detect sensitive data based on the optimized dynamic detection rules.

[0083] The sensitive data detection method of this disclosure can be executed on a server, on a terminal, or partially on a server and partially on a terminal.

[0084] like Figure 2 As shown, according to one embodiment of this disclosure, a sensitive data detection method includes:

[0085] Step 210: Obtain the data to be detected;

[0086] Step 220: Call the dual encoder rule generation model to convert the user query command into first vector data, convert the text information of the knowledge source base into second vector data, and determine the target knowledge text information based on the first vector data and the second vector data. The knowledge source base includes at least one of the following: a legal database, an industry thesaurus, or an enterprise database.

[0087] Step 230: Call the dual encoder rule generation model to generate the first detection rule based on the templated instructions and target knowledge text information;

[0088] Step 240: Use the large language model to adapt and optimize the first detection rule to obtain the optimized second detection rule;

[0089] Step 250: Call the multimodal joint detection model to perform sensitive data detection on the data to be detected according to the second detection rule, and obtain a sensitive data detection report.

[0090] Steps 210 to 250 are briefly described below.

[0091] In step 210, the data to be detected refers to the data for which sensitive word detection needs to be performed. For example, the data to be detected can be text data such as user-inputted text or documents, or unstructured data such as images or videos, or a database for which sensitive data detection needs to be performed.

[0092] In step 220, the dual encoder rule generation model refers to a model based on the RAG framework that includes a retrieval module for retrieving the required knowledge from the knowledge source base and a generation module for generating detection rules based on the retrieved knowledge.

[0093] For example, the retrieval module in this embodiment specifically employs Dense Passage Retrieval (DPR), a core technology for open-domain question answering. It uses a dual-encoder architecture to transform questions and passages into high-dimensional dense vectors, and then achieves accurate semantic-level retrieval based on vector similarity. The DPR model in this disclosure trains two independent encoders: a query encoder and a passage encoder, embedding the query and passage text into the same semantic space. The query encoder embeds the user's query command, while the passage encoder embeds information from a knowledge source base, such as regulations and thesaurus documents. For example, a regulatory database or industry-specific data can be used to label the retrieval task, training and optimizing the DPR module. This embodiment utilizes the DPR model to quickly and accurately locate the relevant passages from massive amounts of text data, providing high-quality knowledge information input for the subsequent generation module.

[0094] For example, the generation module adopts a Transformer-Decoder architecture model, such as T5 or BART. Based on relevant paragraphs provided by the retrieval module, the generation module performs semantic analysis and text generation algorithms to generate dynamic rules or natural and fluent text content that meets user needs.

[0095] For example, a knowledge source base can be a data system consisting of at least several databases, including a regulatory database, an industry thesaurus, and an enterprise database. Alternatively, it can be a database storing regulatory texts, industry-specific terms, and enterprise data. Specifically, the regulatory database is a digital system that centrally stores and manages normative texts such as laws, regulations, policy documents, and judicial interpretations; the industry thesaurus is a collection of professional terms for specific fields (such as finance, healthcare, and law); and the enterprise database is a system that stores enterprise operational data, encompassing internal business data, external market data, and third-party data.

[0096] For example, to ensure the accuracy and effectiveness of the retrieval and generation of the first detection rule, a high-quality knowledge source base needs to be prepared, including a regulatory base, an industry thesaurus, and enterprise data. Specifically, the regulatory base includes industry regulations and standard documents (such as GDPR, HIPAA, and ISO standards), providing authoritative legal basis for the generated content; the industry thesaurus collects industry terms and sensitive data markers, and expands the thesaurus coverage using domain expert review or data mining techniques; and the enterprise database contains internal historical data, such as historical rule bases and detection logs, providing rich practical cases and experience references for the generated content. HIPAA stands for Health Insurance Portability and Accountability Act, and ISO standards stand for International Organization for Standardization.

[0097] User query instructions refer to questions that users input in real time or pre-set. User query instructions input the user's search needs into the model. For example, the user inputs "the data minimization principle in GDPR" (GDPR, or General Data Protection Regulation).

[0098] The first vector data refers to the model-understandable vector obtained by transforming the user's input question through the Query Encoder of the DPR model.

[0099] The second vector data refers to the model-understandable vectors obtained by converting text in the knowledge source base through the Passage Encoder of the DPR model.

[0100] Target knowledge text information refers to data related to user queries retrieved from the knowledge source database by the retrieval module of the dual encoder rule generation model.

[0101] In step 230, calling the dual encoder rule generation model to generate the first detection rule means generating the first detection rule through the generation module in the dual encoder rule generation model. The specific meaning of the generation module has been described in detail above and will not be repeated here.

[0102] Templated instructions refer to instructions that are standardized through a predefined structured framework (template). For example, a three-part prompt such as "Generate testing rules based on [regulatory clauses] and [industry terminology]" can be used. Specific templated instructions can be set according to requirements and are not limited here.

[0103] The first detection rule refers to the detection rule generated by the generation module according to the templated instructions and the target knowledge text retrieved by the generation module, used to determine sensitive words. In this embodiment, the first detection rule is generated based on the real-time retrieved knowledge text, that is, the first detection rule is dynamically generated, and it will change as the data in the knowledge source base is updated, or as the user's query instructions change, or as the search results change.

[0104] In step 240, the second detection rule refers to the detection rule obtained after adapting and optimizing the first detection rule. For example, the large language model can employ the DeepSeek model. For complex scenarios in sensitive data detection, during the dynamic detection rule execution adaptation stage, the deep semantic parsing capabilities of the DeepSeek large model are integrated to overcome the limitations of shallow models in understanding contextual relationships (such as multi-field co-occurrence patterns), thus reducing the false positive rate. When using DeepSeek for contextual semantic parsing, it can not only adapt to multi-field data, especially complex relational data such as multi-field co-occurrence patterns, but also optimize contextual understanding capabilities, accurately capturing the semantic relationships of cross-paragraph citations. To better align the model with enterprise business scenarios, it is adapted by fine-tuning the model using domain-specific text relation annotation data and designing dedicated training tasks, such as multi-field condition generation or inference, to enhance the model's understanding of multi-field co-occurrence patterns.

[0105] For example, by annotating data (specifically sensitive data such as ID card numbers, bank card numbers, and medical records), the co-occurrence relationships between fields are marked (e.g., the sensitivity increases when the combination of "name + ID card number" appears), and sensitive information scattered across multiple paragraphs (e.g., the association between "confidentiality clauses" and "liability for breach of contract" in a contract) is marked for cross-paragraph association. DeepSeek is then fine-tuned using a dedicated training task on these annotated datasets to enable it to understand the specialized terminology and complex relationships within the data security field.

[0106] For example, patent training tasks include, but are not limited to: multi-field condition generation: given some security-related fields (such as "name + mobile phone number"), the model generates missing fields (such as completing the ID number or associated email) or infers potential association rules; co-occurrence risk inference: given a list of fields (such as "name, birthday, department"), the model is required to determine whether the combination of fields constitutes a risk of sensitive data leakage; cross-paragraph semantic association task: capture the association of sensitive information scattered across multiple paragraphs (such as "confidentiality clause" on page 2 and "liability for breach of contract" on page 5 in a contract).

[0107] For example, in optimizing the false positive rate, the DeepSeek model generates a semantic dependency graph between fields based on deep semantic layer associations to identify abnormal co-occurrence patterns. Simultaneously, it designs a dynamic confidence threshold to filter low-confidence rule detection results and adjusts the detection criteria based on semantic similarity scores. Furthermore, for semantic relationships across fields and paragraphs, DeepSeek can generate complete context chains, effectively reducing false positives. To ensure continuous model optimization, the performance of the DeepSeek model is regularly evaluated using false positive rate analysis and recall as metrics. In conjunction with self-supervised learning methods, semantic variants are generated using unlabeled data augmentation to enrich the training samples and continuously improve model performance.

[0108] In step 250, the multimodal joint detection model refers to a large model obtained by combining detection modules targeting multiple different modalities of data. For example, the multimodal joint detection model can detect structured data, such as text, documents, and tables; it can also detect unstructured data, such as images and videos; and it can also mine sensitive information hidden in the model parameters.

[0109] A sensitive data detection report refers to data generated based on the detection results after the data to be detected has been processed. For example, it can be a report that identifies sensitive data based on the original data, or it can be a report that statistically analyzes the detected sensitive data to generate relevant display data.

[0110] A sensitive data detection report can also be data obtained by identifying and processing sensitive words in the data to be detected according to dynamically generated detection rules. For example, sensitive words are identified from the data to be detected according to dynamically generated and optimized second detection rules, and the sensitive words are then processed such as masking and deletion to obtain the processed data.

[0111] In the embodiments of steps 210 to 250 above, the dual-encoder rule generation model, combined with the user's query instructions, retrieves the required target knowledge text information from a knowledge source library containing a legal database, an industry thesaurus, and an enterprise database. Then, a first detection rule is generated using templated instructions. By retrieving the knowledge source library, detection rules are dynamically generated based on the current legal database, industry thesaurus, and other information in the library. This ensures that the detection rules are better adapted to the current industry situation and regulatory requirements, avoiding the lag of existing static rules. Through the above-mentioned detection rule generation mechanism, detection rules that comply with regulatory requirements can be generated in real time, responding promptly to changes in regulations and industries, ensuring the accuracy and timeliness of sensitive data detection, and eliminating the need for manual maintenance of the rule library. This embodiment also optimizes the first detection rule using a large language model to determine an optimized second detection rule. Sensitive data detection is then performed using the optimized detection rule, integrating the semantic parsing capabilities of the large language model. This allows the optimized second detection rule to better adapt to sensitive word detection in complex scenarios, thereby improving the detection accuracy of sensitive word detection using the second detection rule and reducing the false positive rate.

[0112] The above is a general description of steps 210 to 250. The following is a detailed description of the specific implementation process of steps 210 to 250.

[0113] In one embodiment, reference is made to Figure 3 Step 220 includes:

[0114] Step 310: Segment the text in the legal database to obtain multiple segments of legal text;

[0115] Step 320: Obtain the metadata tags for each section of the regulation text, whereby the metadata tags include the clause number, effective date, and scope of application;

[0116] Step 330: Convert each segment of the legal text into a vector representation, associate the vector representation of each segment of the legal text with the metadata tags, and obtain the second vector data corresponding to the legal database.

[0117] In step 310, the text in the legal database refers to the various legal provisions, standards, and normative text data stored in the legal database. Legal text refers to segments of text data obtained by segmenting documents / texts in the legal database.

[0118] In step 320, metadata tags refer to tags constructed by combining structured information describing data attributes. This combines the structured descriptive capabilities of metadata with the semantic abstraction capabilities of tags, forming tags that possess both "attribute description" and "semantic classification." For example, in this embodiment, metadata tags including clause number, effective date, and scope of application are constructed for each segment of regulatory text based on its corresponding regulatory clause, effective date, and applicable scope. This enables rapid retrieval and matching of regulatory texts based on these tags.

[0119] In step 330, each segment of the legal text is first converted into a vector form that the model can recognize. Then, the vector representation of each segment of the legal text is associated with its corresponding metadata tag to form the second vector data of each segment of the legal text. This allows for quick filtering of the second vector data that meets the requirements based on the recognized metadata tag, thereby improving the retrieval speed.

[0120] For example, after obtaining the knowledge source base, the data in the knowledge source base can be further preprocessed. For instance, the legal database and industry thesaurus can be segmented, and the Passage Encoder can be used to convert each segment of legal text into an embedding vector so that the model can retrieve it quickly. For the legal text, metadata tags such as clause number, effective date, and scope of application can be added to further enhance the model's ability to understand the text context, thereby improving the performance of the entire dual-encoder rule generation model.

[0121] In one embodiment, the dual-encoder rule generation model in step 220 is trained through the following steps:

[0122] Obtain multiple first sample data;

[0123] Determine the query vector, positive sample embedding vector, and negative sample embedding vector corresponding to each first sample data;

[0124] The dual encoder rule generation model is modified based on the similarity between each query vector and its corresponding positive sample embedding vector, and the similarity between each query vector and its corresponding negative sample embedding vector.

[0125] In this embodiment, "training the dual encoder rule generation model" specifically refers to training the retrieval module within the dual encoder rule generation model. Correspondingly, this training process involves correcting the model parameters of the retrieval module.

[0126] The first sample data refers to the sample data used to train the dual-encoder rule generation model. The sample data includes the query vector, the corresponding positive sample embedding vector, and the corresponding negative sample embedding vector. The positive sample embedding vector is a vector obtained by transforming documents / text related to the query in the knowledge source base, while the negative sample embedding vector is a vector obtained by transforming documents / text randomly selected from the entire document set that are irrelevant to the query. For example, negative samples can be hard negative samples obtained by filtering paragraphs similar to positive samples but irrelevant to the query using Hard Negative Mining techniques. Training the retrieval module using hard negative samples helps the model better distinguish fine-grained text differences, improving the model's retrieval accuracy and precision.

[0127] For example, the model is trained with the objective of maximizing the similarity between the query and the correct text while minimizing the similarity with irrelevant documents. That is, incorporating this objective, a loss function is constructed based on the similarity between the query vector and its corresponding positive sample embedding vector, and the similarity between each query vector and its corresponding negative sample embedding vector. Specifically, a negative log-likelihood loss function is used with a dual-tower architecture from contrastive learning to maximize the similarity of the correct document / text. The loss function in this example is shown in the following formula:

[0128]

[0129] Where, q i Let i be the query vector for the i-th sample. For the corresponding positive sample embedding vector, This is the embedding vector for the negative sample.

[0130] The above embodiments utilize the similarity between the query vector and the positive and negative sample embedding vectors to construct a loss function to train the dual encoder rule generation model, thereby improving the accuracy of the dual encoder rule generation model for retrieving query-related knowledge text.

[0131] In one embodiment, the dual-encoder rule generation model in step 220 is further trained through the following steps:

[0132] Obtain multiple second sample data;

[0133] Multiple second sample data are input into the dual encoder rule generation model and the pre-set strong supervision model respectively to obtain the first output result of the dual encoder rule generation model and the second output result of the strong supervision model.

[0134] The dual encoder rule generation model is corrected based on the difference between the first and second output results.

[0135] In this embodiment, the retrieval module in the dual-encoder rule generation model is also trained. Correspondingly, the first output result is the vector result output by the retrieval module of the dual-encoder rule generation model when the second sample data is input; the second output result is the vector result output by the strongly supervised model when the second sample data is input.

[0136] A loss function is constructed based on the difference between the first and second output results to optimize the dual-encoder rule generation model.

[0137] For example, the loss function in this embodiment can be adopted using the following formula:

[0138]

[0139] Among them, f DPR The vector generated for the retrieval module (DPR model), i.e., the first output result; f teacher The vector generated by the strongly supervised model is the second output.

[0140] In the above embodiments, a loss function is constructed using the difference between the first output of the retrieval module and the second output of the strongly supervised model to correct the model parameters of the retrieval module of the dual encoder rule generation model. Through knowledge distillation, the representation capabilities of the strongly supervised model are transferred to the retrieval module to further improve retrieval accuracy and model performance.

[0141] In one embodiment, reference is made to Figure 4 Step 230 includes:

[0142] Step 410: Perform vector transformation on the target knowledge text information and the corresponding vertical domain vocabulary, and perform semantic association between the target knowledge text information and the vertical domain vocabulary through word vector space alignment to obtain fused information;

[0143] Step 420: Determine the rule generation conditions based on the template instructions;

[0144] Step 430: Based on the rule generation conditions, generate the first detection rule according to the target knowledge text information, the vertical domain vocabulary information and the fusion information.

[0145] In step 410, the vertical domain lexicon refers to a collection of professional terms built for a specific industry or professional field (such as healthcare, finance, law, technology, etc.). Integrating the vertical domain lexicon can accurately capture the semantic relationships, contextual differences, and professional concepts of words within the domain, solving the problem of insufficient semantic coverage of general-purpose lexicons in vertical scenarios.

[0146] The target knowledge text information and the corresponding vertical domain vocabulary are both input into the generation module. The generation module converts the target knowledge text information and the words in the vertical domain vocabulary into vector form data. Then, semantic association is achieved through word vector space alignment, providing multi-dimensional data support for rule generation. The fusion information is the data after semantically associating the vectorized vertical domain vocabulary and knowledge text.

[0147] In step 420, the rule generation conditions refer to the conditions obtained by parsing the templated instructions to define the boundaries of rule generation. These conditions ensure that the generation module prioritizes calling associated nodes in the domain knowledge graph when decoding and generating detection rules, thereby improving the domain adaptability of the generated detection rules. The associated nodes in the domain knowledge graph refer to entities, concepts, relationships, and other nodes in the domain knowledge graph that are relevant to the current task.

[0148] For example, an RLHF model adjustment generation module can also be introduced. By collecting feedback data on the quality of manually labeled rules, the reinforcement learning model can be trained to dynamically adjust the generation strategy, thereby improving the accuracy of detection rule generation and the industry adaptability of detection rules, and further realizing efficient and accurate dynamic detection rule generation.

[0149] The embodiments of steps 410 to 430 above perform semantic association by aligning the target knowledge text information and the vertical domain vocabulary with word vector space, and integrate the vertical domain vocabulary for detection generation, so that the domain adaptability of the first detection rule is better, thereby improving the accuracy of subsequent sensitive data detection.

[0150] In one embodiment, the multimodal joint detection model includes a text detection module and an unstructured data detection module. (Refer to...) Figure 5 Step 250 includes:

[0151] Step 510: The text detection module detects named entities in the text data to be detected according to the second detection rule, and obtains a sensitive data detection report.

[0152] And / or,

[0153] Step 520: Using the unstructured data detection module, perform multi-module feature extraction on the unstructured data in the data to be detected to obtain feature vectors; determine the feature text information based on the feature vectors; and detect the named entities of the feature text information according to the second detection rule to obtain a sensitive data detection report.

[0154] In step 510, for example, the text detection module can use a pre-trained NER model as a foundation, training the NER model with a broad corpus to give it strong generalization capabilities. To better adapt to specific enterprise scenarios, the model needs to be fine-tuned, expanding entity categories to cover specific industry terms, sensitive information, etc. Combined with a dynamic rule base consisting of detection rules generated using the methods provided in the above embodiments, named entities in the text are accurately matched, significantly improving the ability to identify sensitive data (such as names, addresses, and bank card numbers).

[0155] In step 520, for example, the unstructured data detection module may include CLIP, OCR, and NER. For unstructured data such as images, audio, and video, the CLIP model is used for feature extraction. The CLIP model can efficiently extract multimodal feature vectors from images and audio, providing a rich information foundation for subsequent analysis. For images or videos, after extracting image feature vectors using the CLIP model, OCR technology is used to extract text information. Based on the extracted text information, the NER model is used to detect sensitive entities using dynamically generated detection rules. For audio, the audio data is processed into speech-to-text and matched with a dynamic rule base to ensure that no sensitive data is missed during detection.

[0156] For example, to improve the consistency of cross-modal detection, a joint training task can be designed to train the model by fusing feature embeddings from text, images, and audio, thereby further improving the accuracy of the detection results.

[0157] It should be noted that the multimodal joint detection model of this disclosure can be used to detect data by using only the text detection module when the data to be detected is plain text; it can also be used to detect data by using only the unstructured data detection module when the data to be detected is pure unstructured data; and it can also be used to detect data of different modalities by using both the text detection module and the unstructured data detection module to detect the corresponding types of data.

[0158] The embodiments of steps 510 and 520 above establish a multimodal joint detection engine that covers the detection of text, unstructured data (images, audio), and other data. Each module works collaboratively to achieve secure detection of cross-modal data, compensating for the deficiencies in multimodal detection. This improves the adaptability to multi-scenario data and the accuracy of detection.

[0159] In one embodiment, the multimodal joint detection model further includes a parameter steganalysis detection module. Step 250 further includes:

[0160] Obtain historical model parameters from the text detection module and the unstructured data detection module;

[0161] The historical model parameters are processed in layers, and the distribution characteristics of the historical model parameters in each layer are statistically analyzed to generate a baseline distribution model.

[0162] The current model parameters of the text detection module and the unstructured data detection module are divided into the same layer, and the KL divergence value between the current model parameters of each layer and the baseline distribution corresponding to the baseline distribution model is calculated.

[0163] If the KL divergence value is greater than a preset threshold, it is determined that the current model parameters of the corresponding layer are abnormal.

[0164] For example, a parameter steganography detection module is used to deeply mine potentially sensitive information hidden in model parameters. First, a baseline model of parameter distribution is constructed, and the distribution characteristics of normal parameters are statistically analyzed. Then, KL divergence is used to measure the deviation of the current parameter distribution from the baseline distribution, thereby identifying anomalous parameters. Layer-by-layer analysis of the model parameters is performed to accurately locate possible steganographic information. When a specified KL divergence threshold is exceeded, the system automatically triggers an anomaly alarm and quickly locates the anomalous parameter layer, providing the security team with timely and effective risk warnings.

[0165] For example, to further improve detection performance, distributed computing technology can be used to accelerate the processing speed of large-scale detection tasks. Simultaneously, edge computing devices can be introduced to achieve real-time detection capabilities, ensuring comprehensive data security protection.

[0166] In one embodiment, a sandbox verification module is also deployed to run the entire detection system in an isolated environment, verifying the security and accuracy of rule execution and monitoring whether the output leaks retrieved sensitive information. The results detected by the sandbox verification module can also be used to provide feedback for the dynamic detection rule generation in steps 220 and 230, the detection rule adaptation and optimization in step 240, and the multimodal detection engine in step 250, forming a closed-loop adjustment. The sandbox verification module enables a comprehensive evaluation of rule generation, execution, and detection results.

[0167] The following example provides a comprehensive and detailed description of the sensitive data detection method of this disclosure. It is understood that the following embodiments are merely illustrative examples to better illustrate the sensitive data detection method of this disclosure and are not intended to limit its application.

[0168] For example, such as Figure 6 and Figure 7As shown, firstly, dynamic knowledge injection is achieved through the Retrieval Enhancement Generation (RAG) framework. A dual-encoder DPR model is used to retrieve external knowledge sources such as legal databases and industry thesaurus in real time, automatically generating dynamic detection rules and solving the problem of traditional rule base updates being highly dependent on manual intervention. Secondly, the deep semantic parsing capabilities of the DeepSeek large model are integrated to overcome the limitations of shallow models in understanding contextual relationships (such as multi-field co-occurrence patterns), reducing the false positive rate. Finally, a multimodal joint detection engine (i.e., a multimodal joint detection model) is constructed. A named entity recognition model is introduced at the text detection level. For unstructured data such as images and audio, the CLIP model is used for feature extraction and cross-validation with text rules. A model parameter steganography detection module is also added, using KL divergence statistics to identify residual sensitive information in the parameters. Finally, a sandbox validation module is used to run the above model in an isolated environment to monitor whether the retrieved sensitive information is leaked.

[0169] The above example, through a technical closed loop of "knowledge retrieval - semantic parsing - multimodal fusion", not only realizes the dynamic generation of detection rules and cross-modal collaborative detection, but also builds an intelligent detection system with self-learning capabilities through large model driving, which can provide dynamic protection solutions for sensitive data-intensive industries such as finance and healthcare throughout their entire lifecycle.

[0170] In combination with the above embodiments, the embodiments of this disclosure have the following effects:

[0171] 1) A dynamic rule adaptive generation mechanism was established: Utilizing a dual-encoder rule generation model based on the RAG framework, and leveraging the powerful semantic understanding capabilities of the dual-encoder DPR model, information is retrieved in real-time from external knowledge sources such as regulatory databases to generate sensitive data detection rules that are adapted to current business scenarios and regulatory requirements, thus solving the lag problem of using static rules in related technologies. Through this dynamic detection rule generation mechanism, it is possible to quickly respond to regulatory changes and business developments, ensuring the accuracy and timeliness of sensitive data detection.

[0172] 2) Optimization Method for Precise Adaptation of Compliance Semantics: Addressing the inefficiency of compliance adaptation in related technologies, this disclosure proposes an optimization method based on the fusion of RAG and DeepSeek. RAG continuously tracks regulatory updates, injecting the latest regulatory information into the system. DeepSeek performs precise analysis of regulatory semantics, automatically adjusting detection strategies and parameters based on enterprise business characteristics to achieve precise adaptation of compliance requirements. This avoids the inefficiency and subjectivity of manual adaptation, significantly improving the response speed and adaptation accuracy of the sensitive data detection system to compliance requirements.

[0173] 3) Construct a multimodal semantic collaborative detection system: A multimodal joint detection module is constructed, covering text, unstructured data (images, audio), and parameter steganography detection modules. These modules work collaboratively to achieve secure detection of cross-modal data, compensating for gaps in multimodal detection capabilities.

[0174] Description of apparatus and devices according to embodiments of this disclosure

[0175] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0176] Figure 8 This is a schematic diagram of a sensitive data detection device provided in an embodiment of the present disclosure. The sensitive data detection device 800 includes:

[0177] Data acquisition unit 810 is used to acquire the data to be detected;

[0178] The knowledge retrieval unit 820 is used to call the dual encoder rule generation model to convert the user query command into first vector data, convert the text information of the knowledge source database into second vector data, and determine the target knowledge text information based on the first vector data and the second vector data. The knowledge source database includes at least one of the following: a legal database, an industry thesaurus, and an enterprise database.

[0179] The rule generation unit 830 is used to call the dual encoder rule generation model to generate the first detection rule based on the templated instructions and target knowledge text information.

[0180] The rule optimization unit 840 is used to call the large language model to adapt and optimize the first detection rule to obtain the optimized second detection rule.

[0181] The data detection unit 850 is used to call the multimodal joint detection model to perform sensitive data detection on the data to be detected according to the second detection rule, and obtain a sensitive data detection report.

[0182] Optionally, the knowledge retrieval unit 820 is specifically used for:

[0183] The text in the legal database is segmented to obtain multiple segments of legal text.

[0184] Obtain the metadata tags for each section of the regulation text, including the clause number, effective date, and scope of application.

[0185] Each segment of the legal text is converted into a vector representation, and the vector representation corresponding to each segment of the legal text is associated with the metadata tags to obtain the second vector data corresponding to the legal database.

[0186] Optionally, the dual-encoder rule generation model is trained through the following steps:

[0187] Obtain multiple first sample data;

[0188] Determine the query vector, positive sample embedding vector, and negative sample embedding vector corresponding to each first sample data;

[0189] The dual encoder rule generation model is modified based on the similarity between each query vector and its corresponding positive sample embedding vector, and the similarity between each query vector and its corresponding negative sample embedding vector.

[0190] Optionally, the dual-encoder rule generation model is also trained through the following steps:

[0191] Obtain multiple second sample data;

[0192] Multiple second sample data are input into the dual encoder rule generation model and the pre-set strong supervision model respectively to obtain the first output result of the dual encoder rule generation model and the second output result of the strong supervision model.

[0193] The dual encoder rule generation model is corrected based on the difference between the first and second output results.

[0194] Optionally, the rule generation unit 830 is specifically used for:

[0195] The target knowledge text information is vectorized with the corresponding vertical domain vocabulary, and semantic association between the target knowledge text information and the vertical domain vocabulary is performed through word vector space alignment to obtain fused information.

[0196] The rule generation conditions are determined based on the template instructions;

[0197] Based on the rule generation conditions, the first detection rule is generated according to the target knowledge text information, the information of the vertical domain vocabulary, and the fusion information.

[0198] Optionally, the multimodal joint detection model includes a text detection module and an unstructured data detection module; the data detection unit 850 is specifically used for:

[0199] The text detection module detects named entities in the text data in the data to be detected according to the second detection rule, and obtains a sensitive data detection report.

[0200] And / or,

[0201] The unstructured data detection module performs multi-module feature extraction on the unstructured data to be detected to obtain feature vectors; feature text information is determined based on feature vectors; and named entities of the feature text information are detected according to the second detection rule to obtain a sensitive data detection report.

[0202] Optionally, the multimodal joint detection model also includes a parameter steganalysis detection module; the data detection unit 850 is specifically used for:

[0203] Obtain historical model parameters from the text detection module and the unstructured data detection module;

[0204] The historical model parameters are processed in layers, and the distribution characteristics of the historical model parameters in each layer are statistically analyzed to generate a baseline distribution model.

[0205] The current model parameters of the text detection module and the unstructured data detection module are divided into the same layer, and the KL divergence value between the current model parameters of each layer and the baseline distribution corresponding to the baseline distribution model is calculated.

[0206] If the KL divergence value is greater than a preset threshold, it is determined that the current model parameters of the corresponding layer are abnormal.

[0207] Reference Figure 9 , Figure 9 To implement the structural block diagram of the terminal portion of the sensitive data detection method according to the embodiments of this disclosure, the terminal includes: a radio frequency (RF) circuit 910, a memory 915, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990, etc. Those skilled in the art will understand that... Figure 9 The terminal structure shown does not constitute a limitation on mobile phones or computers and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0208] The RF circuit 910 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 980; in addition, it transmits uplink data to the base station.

[0209] The memory 915 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing of the content terminal by running the software programs and modules stored in the memory 915.

[0210] The input unit 930 can be used to receive input numeric or character information, and to generate key signal inputs related to the settings and function control of the content terminal. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932.

[0211] The display unit 940 can be used to display input or provided information, as well as various menus of the content terminal. The display unit 940 may include a display panel 941.

[0212] Audio circuitry 960, speaker 961, and microphone 962 provide an audio interface.

[0213] In this embodiment, the processor 980 included in the terminal can execute the sensitive data detection method of the previous embodiment.

[0214] The terminals disclosed in this embodiment include, but are not limited to, mobile phones, computers, etc. This embodiment can be applied to various scenarios, including but not limited to drug discovery, materials science, and molecular virtual screening.

[0215] Figure 10 This is a structural block diagram of a server portion for implementing the sensitive data detection method of this disclosure embodiment. The server can vary significantly due to differences in configuration or performance, and may include one or more Central Processing Units (CPUs) 1022 (e.g., one or more processors) and a memory 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044. The memory 1032 and storage media 1030 may be temporary or persistent storage. The program stored in the storage media 1030 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1022 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server.

[0216] The server may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0217] The central processing unit 1022 in the server can be used to execute the sensitive data detection method of the present disclosure embodiments.

[0218] This disclosure also provides a computer-readable storage medium for storing program code for executing the sensitive data detection methods of the foregoing embodiments.

[0219] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the aforementioned sensitive data detection method.

[0220] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar terms and are not necessarily used to describe a particular order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0221] It should be understood that in this disclosure, "at least one item" refers to one or more items, and "more than one item" refers to two or more items. "And / or" is used to describe the relationship between related content, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related content are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0222] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0223] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0224] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0225] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0226] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0227] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0228] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for detecting sensitive data, characterized in that, The method includes: Acquire the data to be tested; The dual encoder rule generation model is invoked to convert user query instructions into first vector data and text information from the knowledge source database into second vector data. The target knowledge text information is determined based on the first vector data and the second vector data. The knowledge source database includes at least one of the following: a regulatory database, an industry thesaurus, and an enterprise database. The dual encoder rule generation model is invoked to generate a first detection rule based on the templated instructions and the target knowledge text information; The first detection rule is adapted and optimized by calling a large language model to obtain the optimized second detection rule; The multimodal joint detection model is invoked to perform sensitive data detection on the data to be detected according to the second detection rule, and a sensitive data detection report is obtained.

2. The method according to claim 1, characterized in that, The process of converting text information from the knowledge source database into second vector data includes: The text in the legal database is segmented to obtain multiple segments of legal text. Obtain the metadata tags for each segment of the regulatory text, wherein the metadata tags include the clause number, effective date, and scope of application; The legal texts in each segment are converted into vector representations, and the vector representations corresponding to the legal texts in each segment are associated with the metadata tags to obtain the second vector data corresponding to the legal database.

3. The method according to claim 1, characterized in that, The dual-encoder rule generation model is trained through the following steps: Obtain multiple first sample data; Determine the query vector, positive sample embedding vector, and negative sample embedding vector corresponding to each of the first sample data; The dual encoder rule generation model is modified based on the similarity between each query vector and the corresponding positive sample embedding vector, and the similarity between each query vector and the corresponding negative sample embedding vector.

4. The method according to claim 1, characterized in that, The dual-encoder rule generation model is further trained through the following steps: Obtain multiple second sample data; Multiple second sample data are respectively input into the dual encoder rule generation model and the pre-set strong supervision model to obtain the first output result of the dual encoder rule generation model and the second output result of the strong supervision model; The dual encoder rule generation model is corrected based on the difference between the first output and the second output.

5. The method according to claim 1, characterized in that, The step of calling the dual encoder rule generation model to generate a first detection rule based on the templated instructions and the target knowledge text information includes: The target knowledge text information is vector-converted with the corresponding vertical domain vocabulary, and semantic association between the target knowledge text information and the vertical domain vocabulary is performed through word vector space alignment to obtain fused information. The rule generation conditions are determined based on the templated instructions; Based on the rule generation conditions, the first detection rule is generated according to the target knowledge text information, the information of the vertical domain vocabulary library, and the fusion information.

6. The method according to claim 1, characterized in that, The multimodal joint detection model includes a text detection module and an unstructured data detection module; The process of calling the multimodal joint detection model to perform sensitive data detection on the data to be detected according to the second detection rule, and obtaining a sensitive data detection report, includes: The text detection module performs detection on named entities of the text data in the data to be detected according to the second detection rule to obtain the sensitive data detection report; And / or, The unstructured data detection module performs multi-module feature extraction on the unstructured data to be detected to obtain a feature vector; the feature text information is determined based on the feature vector; and the named entities of the feature text information are detected according to the second detection rule to obtain the sensitive data detection report.

7. The method according to claim 6, characterized in that, The multimodal joint detection model also includes a parameter steganalysis detection module; The method further includes: Obtain the historical model parameters of the text detection module and the unstructured data detection module; The historical model parameters are processed in layers, and the distribution characteristics of the historical model parameters in each layer are statistically analyzed to generate a baseline distribution model. The current model parameters of the text detection module and the unstructured data detection module are divided into the same layer, and the KL divergence value between the current model parameters of each layer and the baseline distribution corresponding to the baseline distribution model is calculated. If the KL divergence value is greater than a preset threshold, it is determined that the current model parameters of the corresponding layer are abnormal.

8. A sensitive data detection device, characterized in that, include: The data acquisition unit is used to acquire the data to be detected; The knowledge retrieval unit is used to call the dual encoder rule generation model to convert the user query command into first vector data, convert the text information of the knowledge source database into second vector data, and determine the target knowledge text information based on the first vector data and the second vector data. The knowledge source database includes at least one of the following: a legal database, an industry thesaurus, and an enterprise database. The rule generation unit is used to call the dual encoder rule generation model to generate a first detection rule based on the templated instructions and the target knowledge text information; The rule optimization unit is used to call the large language model to adapt and optimize the first detection rule to obtain the optimized second detection rule. The data detection unit is used to call the multimodal joint detection model to perform sensitive data detection on the data to be detected according to the second detection rule, and obtain a sensitive data detection report.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the sensitive data detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the computer, implement the sensitive data detection method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program or computer instructions, characterized in that, The computer program or the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer program or the computer instructions from the computer-readable storage medium and executes the computer program or the computer instructions, causing the computer device to perform the sensitive data detection method as described in any one of claims 1 to 7.

Images