Sensitive data detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a sensitive data detection method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining text data; performing word segmentation processing on the text data to obtain target text data; performing feature extraction on the target text data by using a first model to obtain text features; performing semantic analysis on the text features to obtain target text features; and inputting the target text features into a second model and a third model, and outputting sensitive data. According to the technical scheme, sensitive data features can be deeply mined, the method has higher adaptability to complex and changeable data forms, the recognition accuracy is greatly improved, and the probability of misjudgment and missed judgment is reduced. And deep fusion of natural language processing and text mining enhances semantic comprehension and further guarantees recognition precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data detection, and in particular to a sensitive data detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid advancement of digitalization, data-intensive enterprises have accumulated massive amounts of data, of which sensitive data occupies an important position. Sensitive data covers personal information, financial information, and business secrets, etc. Once these sensitive data are leaked, the enterprise will face serious consequences such as legal lawsuits, reputation damage, and economic compensation. At the same time, globally, data security and privacy protection regulations are becoming increasingly stringent, and enterprises must ensure that sensitive data management complies with relevant requirements.

[0003] Exponential growth in data volume and continuous evolution of sensitive data forms have brought unprecedented challenges to traditional sensitive data management methods. Traditional methods that rely on manual rule setting and keyword matching are difficult to cope with complex and changing data environments.

[0004] Therefore, there is an urgent need for a new intelligent and automated sensitive data management solution to address the pain points of existing technology. SUMMARY

[0005] The present application provides a sensitive data detection method, device, electronic equipment and storage medium, which can realize automatic and accurate identification of sensitive data.

[0006] According to an aspect of the present application, a sensitive data detection method is provided, which comprises:

[0007] obtaining text data;

[0008] performing word segmentation processing on the text data to obtain target text data;

[0009] extracting features from the target text data using a first model to obtain text features; wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional context information of words;

[0010] performing semantic analysis on the text features to obtain target text features;

[0011] inputting the target text features into a second model and a third model to output sensitive data; wherein the second model learns basic features of sensitive data through sample training; the third model learns deep features of sensitive data through sample training; the deep features include semantic features, context association features, or cross-field association features.

[0012] According to another aspect of the present application, there is provided a sensitive data detection device, the device comprising:

[0013] a text data obtaining module configured to obtain text data;

[0014] a target text data obtaining module configured to perform word segmentation processing on the text data to obtain target text data;

[0015] a text feature obtaining module configured to extract features from the target text data using a first model to obtain text features, wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional contextual information of words;

[0016] a target text feature obtaining module configured to perform semantic analysis on the text features to obtain target text features;

[0017] a sensitive information output module configured to input the target text features into a second model and a third model to output sensitive data, wherein the second model learns basic features of sensitive data through sample training, and the third model learns deep features of sensitive data through sample training, and the deep features include semantic features, contextual correlation features or cross-field correlation features.

[0018] According to another aspect of the present application, there is provided an electronic device, the electronic device comprising:

[0019] at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform a sensitive data detection method according to any one of the embodiments of the present application.

[0020] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to perform a sensitive data detection method according to any one of the embodiments of the present application when executed by the processor.

[0021] The technical solution of the embodiments of the present application comprises the following steps: obtaining text data; performing word segmentation processing on the text data to obtain target text data; extracting features from the target text data using a first model to obtain text features; performing semantic analysis on the text features to obtain target text features; and inputting the target text features into a second model and a third model to output sensitive data. The technical solution can automatically and accurately identify sensitive data, and solves the problem that traditional methods relying on manual rule setting and keyword matching are difficult to cope with complex and variable data environments.

[0022] It should be understood that the matters described herein are intended to be illustrative and are not intended to limit or restrict the scope of the application to the specific forms set forth. The present application is therefore not limited to the embodiments described and shown herein but can be carried out in other ways by persons skilled in the art without departing from the scope of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0024] Figure 1 is a flow chart of a sensitive data detection method according to the first embodiment of the present application;

[0025] Figure 2 is a flow chart of a sensitive data detection process according to the second embodiment of the present application;

[0026] Figure 3 is a sensitive data intelligent detection flow chart according to the second embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a sensitive data detection process according to the third embodiment of the present application;

[0028] Figure 5 is a data desensitization flow chart according to the third embodiment of the present application;

[0029] Figure 6 is a sensitive data compliance display flow chart according to the third embodiment of the present application;

[0030] Figure 7 is a structural schematic diagram of a sensitive data detection device according to the fourth embodiment of the present application;

[0031] Figure 8 is a structural schematic diagram of an electronic device implementing a sensitive data detection method according to the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should be within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment one

[0035] Figure 1 is a flowchart of a sensitive data detection method according to an embodiment one of the present application. The present embodiment can be applicable to the case of detecting sensitive data in text data. The method can be executed by a sensitive data detection device, which can be realized in the form of hardware and / or software, and can be configured in a device. For example, the device can be a background server or other device with communication and computing capabilities. As shown in the figure, the method comprises: Figure 1

[0036] S110, obtaining text data.

[0037] In the present scheme, text data refers to a collection of information in the form of characters that can be recognized and processed by a computer. The text data can be obtained from the database of a target enterprise.

[0038] S120, performing word segmentation processing on the text data to obtain target text data.

[0039] In the present embodiment, the text data can be processed by natural language processing technology (NLP) to obtain target text data. Natural language processing is a technology that enables computers to understand, process and generate human language. It can help the system to analyze the semantic information of sensitive information in the text data, for example, to identify sensitive words or phrases implied in the text data.

[0040] Optionally, the word segmentation processing on the text data to obtain target text data comprises:

[0041] performing at least one operation of cleaning, word segmentation, removing stop words, word form reduction and part-of-speech tagging on the text data to obtain target text data.

[0042] In the present embodiment, cleaning refers to the process of removing irrelevant or interfering information in the text and retaining the core content.​

[0043] In the embodiment, word segmentation refers to the process of splitting continuous text into the smallest semantic units ("words" or "subwords") according to rules.

[0044] In the scheme, stop word removal refers to removing words in the text that have high frequency of occurrence but small semantic contribution (such as function words, common words).

[0045] In the embodiment, lemmatization refers to the process of reducing the inflected forms of words (such as past tense, plural) to their roots (basic forms). Lemmatization can unify the forms of words, avoid misjudging different forms of the same word as different words, and facilitate the model to learn the core semantics of the vocabulary. For example, for the word "running", lemmatization is "run".

[0046] In the embodiment, part-of-speech tagging refers to the process of tagging each word with its grammatical category (such as noun, verb, adjective). Through part-of-speech tagging, the model can better understand the grammatical role and semantic relationship of the word in the sentence. For example, "hit" is a verb in "hit the game" and a preposition in "hit the sauce".

[0047] Specifically, at least one of cleaning, word segmentation, stop word removal, lemmatization, and part-of-speech tagging is performed on the text data in the word segmentation processing of the text data.

[0048] Further, a model-based noise filtering technique can be used to effectively remove false sensitive information and noise data in the text and improve data quality.

[0049] By performing word segmentation on the text data, the model can more accurately grasp the meaning of the vocabulary when processing the text.

[0050] S130, using a first model to extract features from the target text data, and obtaining text features; wherein the first model extracts features associated with sensitive data from the target text data by considering the context bidirectional information of the word.

[0051] In the scheme, the first model extracts features associated with sensitive data from the target text data by considering the context bidirectional information of the word.

[0052] Specifically, the target text data can be input into the first model, and the first model is used to extract features from the target text data to obtain text features.

[0053] Optionally, the first model is a BERT model, and the first model is used to extract features from the target text data to obtain text features, including:

[0054] The target text data is input into the BERT model.

[0055] The target text data is encoded by using the multi-layer Transformer structure of the BERT model, and text features are extracted from the output of the multi-layer Transformer structure.

[0056] In this scheme, BERT can simultaneously consider the context bidirectional information of words to generate more accurate and rich semantic representations. By fine-tuning the BERT model to adapt to sensitive data identification tasks. At the same time, combined with the association rule mining technology in text mining, the association rules between sensitive data and other related words are mined from a large amount of text data. For example, strong association relationships between "financial statements" and "revenue" and "profit" are found, further enriching the feature library of sensitive data identification.

[0057] Specifically, after inputting the target text data into the BERT model, the target text data is encoded by using the multi-layer Transformer structure built in the model, and text features are extracted from the output of the multi-layer Transformer structure.

[0058] Using the BERT model to extract features from the target text data can make the text feature expression more accurate and comprehensive, and the extracted features have strong generalization ability and can adapt to various text scenarios.

[0059] S140, performing semantic analysis on the text features to obtain target text features.

[0060] In this scheme, to solve the ambiguity problem in natural language, natural language processing techniques (such as word vectors, pre-trained language models, etc.) can be used to analyze the semantic association, sentiment orientation, and theme connotation of words, sentences, and chapters in the text, and extract target text features that can represent the core meaning of the text or meet specific needs.

[0061] Optionally, the semantic analysis of the text features to obtain target text features comprises:

[0062] The semantic analysis of the text features is performed by using a pre-constructed semantic model to obtain target text features; wherein the semantic model is used to represent the words and their semantic relationships in the text.

[0063] In this solution, first, the target text is processed for lexical clarification, accurately extracting and defining core entities and concepts (such as terms, objects, abstract concepts, etc.), ensuring clear semantic boundaries. Second, by analyzing the semantic logical relationships between concepts (such as association: "financial statements" and "revenue"), the concepts are converted into nodes in the semantic graph, and the relationships are converted into edges connecting the nodes, forming a structured initial association network. Finally, based on a deep understanding of semantic logic, the connection between nodes and edges is dynamically optimized, such as supplementing implicit associations, correcting logical biases, and refining relationship levels, ultimately presenting the complete semantic association between knowledge and completing the construction of the semantic model.

[0064] Further, the text features are input into the semantic model, and the target text features are obtained based on semantic analysis of the semantic model. For example, for the word "apple", through semantic model analysis, it can be determined that it has different meanings in the two sentences "I like eating apples" and "Apple Inc. releases new products".

[0065] In this embodiment, multi-modal information fusion technology is also introduced to combine text data with image, audio and other modal data, further eliminating ambiguity and improving the accuracy of sensitive data recognition.

[0066] By constructing a semantic-based analysis model. This model extracts words from the text and organizes the semantic associations between words to form a structured semantic network; through deep analysis of this semantic network, it can accurately identify the specific meaning of words in different contexts.

[0067] S150, input the target text features into the second model and the third model, and output the sensitive data; wherein the second model learns the basic features of the sensitive data through sample training; the third model learns the deep features of the sensitive data through sample training; the deep features include semantic features, context association features or cross-field association features.

[0068] Among them, sensitive data refers to data that may cause harm to individuals or organizations if it is leaked, misused, tampered with, or accessed without authorization.

[0069] In this solution, the second model learns the basic features of the sensitive data through sample training. Basic features refer to core attributes or identifiers that can be used to identify and distinguish sensitive data. For example, financial data involves fund security, and basic features can be account, amount, and transaction.

[0070] In this embodiment, the second model is a lightweight shallow neural network model, which is trained based on a large number of common sensitive data features and can filter data in a short time.

[0071] In the present scheme, the third model learns deep features of sensitive data through sample training. The deep features include semantic features, context correlation features or cross-field correlation features. The semantic features are the abstraction of the essence of data, rather than the simple extraction of surface words or symbols. The context correlation features refer to the fact that the meaning or sensitivity of data depends on the context before and after it. A piece of data may not be meaningful alone, but it may exhibit sensitive properties when combined with the context. The cross-field correlation features refer to the fact that multiple different fields or dimensions of data may not be sensitive alone, but may form sensitive information when combined.

[0072] In the present embodiment, the third model deeply integrates a convolutional neural network (CNN) and a long short-term memory network (LSTM). The CNN is used to accurately capture local sensitive features in the data. The LSTM is used to process the long-term dependencies of the data. For example, when processing a long text containing sensitive information, the CNN can quickly identify local sensitive word features such as "bank card number" and "password" in the text, and the LSTM can accurately determine the sensitivity and mutual relationship of these words in the entire text in combination with the context, thereby comprehensively determining whether the text contains sensitive data.

[0073] Further, in the training data preparation stage, an active learning and semi-supervised learning method is used. The former selects the most valuable data from a large amount of unlabeled data for manual labeling to reduce the labeling workload. The latter trains the model with a small amount of labeled data and a large amount of unlabeled data to improve the generalization ability, and at the same time, performs fine processing on the labeled data. In addition to labeling the type and range of sensitive data, it also labels the sensitive degree level, related business scenarios and other multi-dimensional information. These rich labeling information help the model to learn the sensitive data features more comprehensively and deeply.

[0074] In the present embodiment, during the training process, the Adam algorithm is combined with a learning rate dynamic adjustment strategy (the former adjusts the learning rate of the parameters to speed up convergence, and the latter automatically adjusts the learning rate according to the performance of the model on the training set and the validation set, such as automatically reducing the learning rate when the accuracy of the model on the validation set does not significantly improve for several training rounds to help stable convergence to the optimal solution), and regularization techniques such as Dropout and batch normalization are introduced to prevent model overfitting and improve model robustness.

[0075] Specifically, the target text features are input into the second model, the second model predicts the target text features, outputs the prediction results, and then inputs the prediction results into the third model, and the third model predicts the prediction results to output the sensitive data.

[0076] The technical scheme of the embodiment of the present application comprises the following steps: obtaining text data; performing word segmentation processing on the text data to obtain target text data; extracting features from the target text data by using a first model to obtain text features; performing semantic analysis on the text features to obtain target text features; and inputting the target text features into a second model and a third model to output sensitive data. By executing the technical scheme, the features of sensitive data can be deeply mined, the data forms are more adaptable to complex changes, the recognition accuracy is greatly improved, and the probability of misjudgment and omission is reduced. The deep fusion of natural language processing and text mining enhances semantic understanding and further guarantees the recognition accuracy.

[0077] Embodiment two

[0078] Figure 2 A flowchart of a sensitive data detection process provided by the second embodiment of the present application is shown in FIG. 2. The relationship between the present embodiment and the above-mentioned embodiment is a detailed description of the sensitive data prediction process based on the second model and the third model. As shown in FIG. 2, the method comprises the following steps. Figure 2

[0079] S210, obtaining text data.

[0080] S220, performing word segmentation processing on the text data to obtain target text data.

[0081] S230, extracting features from the target text data by using a first model to obtain text features; wherein the first model extracts features associated with sensitive data from the target text data by considering the bidirectional information of the context of the words.

[0082] S240, performing semantic analysis on the text features to obtain target text features.

[0083] S250, processing the target text features according to the data types of the target text features to obtain processed target text features.

[0084] The data types include structured data, unstructured data and semi-structured data. The structured data is data with fixed format, clear structure and easy to represent in a table. The unstructured data is data without fixed format and cannot be represented in a two-dimensional table. The semi-structured data is data between structured and unstructured: without strict two-dimensional table structure, but containing self-description information (i.e. the data itself carries structure identification), which can be parsed to obtain part of the rules.

[0085] ​In the present solution, the target text feature is processed according to the data type to which the target text feature belongs, so as to obtain a processed target text feature. The specific logic is as follows: structured data: the discrete text field is converted into a calculable numerical value (for example, by encoding, standardization, etc.), while the classification attribute and internal logical relationship of the field itself are completely retained. Unstructured data: effective information (such as keywords, semantics, etc.) is extracted from the free text and converted into structured or quantifiable features. Semi-structured data: the structured part is extracted first, and the remaining text is processed according to the processing method of unstructured data.

[0086] Optionally, the target text feature is processed according to the data type of the target text feature to obtain a processed target text feature, including:

[0087] determining the data type of the target text feature, wherein the data type includes structured data, unstructured data and semi-structured data;

[0088] if the target text feature is structured data, the target text feature is determined as the processed target text feature;

[0089] if the target text feature is unstructured data, the target text feature is processed using natural language processing technology to obtain a processed target text feature;

[0090] if the target text feature is semi-structured data, the target text feature is format-converted, and the format-converted target text feature is determined as the processed target text feature; or the format-converted target text feature is processed using natural language processing technology to obtain a processed target text feature.

[0091] In the present embodiment, corresponding information can be extracted from the target text feature according to the inherent characteristics of different data types, and the data type to which the target text feature belongs can be determined.

[0092] In the present solution, the data format from different data sources (database, file system, network transmission, etc.) can be automatically recognized, and dynamic shunting can be performed according to the data type (structured, semi-structured, unstructured). Figure 3 The sensitive data intelligent detection flowchart provided for the second embodiment of the present application is as follows: Figure 3As shown, the data type of the target text feature is determined, if the target text feature is structured data (such as table data in a database), the structured data detection module is directly entered, and the processed target text feature is output; if the target text feature is unstructured data, the natural language processing and text mining module is first entered, the natural language processing technology is used to complete the processing, and the processed target text feature is obtained; if the target text feature is semi-structured data (such as XML, JSON format data), format conversion is first performed, and then the corresponding detection module is transferred according to the content characteristics, and finally the processed target text feature is generated.

[0093] By processing the target text feature according to the data type, the accuracy and speed of model prediction are improved.

[0094] S260, input the processed target text feature into the second model, and output the first data set.

[0095] The first data set is a data set containing sensitive information.

[0096] In this scheme, in the sensitive data detection process, a multi-level preliminary detection and intelligent filtering strategy is designed. Figure 3 As shown, first, the second model is used to quickly scan the processed target text feature, and preliminarily judge whether the processed target text feature may contain sensitive information. This model is trained based on a large number of common sensitive data characteristics, and can filter sensitive information in a short time. For example, for a piece of text, if the model judges that it does not contain any key features of common sensitive information, such as no keywords related to personal information and financial information, it directly skips the subsequent complex detection process, greatly improving the detection efficiency.

[0097] S270, if the first data set meets the preset data condition, the first data set is input into the third model, and the sensitive data is output; wherein the data condition is used to filter and limit the data related to sensitive data in the first data set.

[0098] In this scheme, if there is data related to sensitive data in the first data set, the first data set is input into the third model, and the sensitive data is output.

[0099] Further, as shown in Figure 3As shown, for the data that has passed the preliminary detection, the third model is used for deep detection and accurate identification. The third model can not only accurately determine whether the first data set is sensitive data, but also accurately identify the type, range, and sensitivity level of the sensitive data, and other detailed information. For example, for a contract text containing complex sensitive information, the model can accurately identify the business secret clauses, personal information parts, and corresponding sensitivity levels involved, providing accurate basis for subsequent data desensitization and processing. At the same time, by establishing a real-time feedback mechanism, the detection results are compared and verified with the actual business scene, the model is continuously optimized, and the accuracy and reliability of the detection are improved.

[0100] The technical scheme of the embodiment of the application comprises the following steps: obtaining text data; performing word segmentation processing on the text data to obtain target text data; extracting features from the target text data by using a first model to obtain text features; performing semantic analysis on the text features to obtain target text features; and inputting the target text features into a second model and a third model to output sensitive data. By executing the technical scheme, the features of sensitive data can be deeply mined, the adaptability to complex and variable data forms is stronger, the identification accuracy is greatly improved, and the probability of misjudgment and omission is reduced. The deep fusion of natural language processing and text mining enhances semantic understanding and further guarantees the identification accuracy.

[0101] Embodiment three

[0102] Figure 4 A schematic diagram of a sensitive data detection process provided by the third embodiment of the application is shown. The relationship between the present embodiment and the above-mentioned embodiments is a detailed description of the sensitive data desensitization process. As shown, Figure 4 The method comprises the following steps:

[0103] S410, obtaining text data.

[0104] S420, performing word segmentation processing on the text data to obtain target text data.

[0105] S430, extracting features from the target text data by using a first model to obtain text features; wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional context information of words.

[0106] S440, performing semantic analysis on the text features to obtain target text features.

[0107] S450, inputting the target text features into a second model and a third model to output sensitive data; wherein the second model learns the basic features of sensitive data through sample training; the third model learns the deep features of sensitive data through sample training; and the deep features include semantic features, context association features, or cross-field association features.

[0108] S460, determine a desensitization rule matched with the sensitive data according to the data content of the sensitive data.

[0109] The desensitization rule refers to a specification and method for processing sensitive information by a specific technical means to prevent sensitive data leakage without affecting the usability of the data.

[0110] In this scheme, Figure 5 The data desensitization flowchart provided by Embodiment Three of the present application is shown in Figure 5 According to the type, sensitivity, business requirements and related requirements of the sensitive data, different desensitization rules are set.

[0111] In this embodiment, in the personal information desensitization process, the combination strategy of "partial replacement + randomization" is adopted for name information. For example, the original name "Zhang San" can be processed as "Zhang X", and "X" is randomly generated from the common surname library, which not only hides the key information, but also reduces the risk of being identified in reverse through randomness.

[0112] For the ID number, not only the middle core digits are masked (such as replaced by "*"), but also the tail check code is specially processed for adaptability - a new check code is calculated and generated according to the ID number coding rules, to ensure that the desensitized number still conforms to the specification in the overall format, avoiding the influence of abnormal format on the flow of data in the system or basic verification.

[0113] In this scheme, for the amount data in financial information, the method of "value transformation + interval mapping" is used for desensitization. Specifically, while maintaining the relative size relationship of the original data (such as the logic of "A amount > B amount" unchanged) and the business analysis value (such as the core indicators of proportion, trend, etc. can be normally extracted), the amount value is adjusted proportionally or randomly mapped within the interval through a preset algorithm, which hides the real amount and does not affect the use of data in statistical analysis, business decision-making and other scenarios.

[0114] Further, in the implementation process of the desensitization rule, a variety of advanced desensitization algorithms and data transformation techniques are comprehensively used, and these techniques are optimized and upgraded for actual application scenarios. Taking the hash desensitization algorithm as an example, a "salt mechanism" is specially introduced, which generates a random salt value for each piece of sensitive data and combines it with the hash operation after the combination. This method not only effectively resists common cracking methods such as rainbow table, improves the security of the hash result, but also avoids the problem of the same original data producing the same hash value, further ensuring the uniqueness of the desensitized data.

[0115] In this embodiment, in terms of technical optimization, for the partial masking algorithm, the core processing efficiency is significantly improved through fine optimization of string operation functions; the synchronously developed parallel desensitization rules can efficiently schedule the computing resources of multi-core processors to realize fast desensitization processing of large-scale data and greatly shorten the processing time. To ensure the stability of the desensitization process, a perfect fault-tolerant mechanism is specially designed. When encountering sudden conditions such as data format abnormalities and algorithm execution errors, the system can automatically trigger error identification, processing and recovery processes to effectively avoid overall interruption caused by local problems and ensure the continuous and reliable operation of desensitization from a technical level.

[0116] Specifically, first, the matching relationship between data content and desensitization rules is constructed, and then the sensitive data content is searched from the pre-stored matching relationship between data content and desensitization rules to determine the desensitization rules matched with the sensitive data.

[0117] S470, desensitizing the sensitive data by using the desensitization rules to obtain target sensitive data.

[0118] Specifically, according to the processing requirements of sensitive data in the desensitization rules, corresponding operations are performed on the sensitive data, and the target sensitive data is obtained.

[0119] In this scheme, as shown in Figure 5 To ensure that the desensitized data effectively protects sensitive information and does not affect its usability in subsequent business processes and data analysis, a comprehensive desensitization effect verification and evaluation system is established. In the verification process, the data security, business usability, compliance and other dimensions are evaluated.

[0120] Further, in terms of data security, through cryptographic analysis and vulnerability scanning, it is detected whether the desensitized data has sensitive information leakage risk. In terms of business usability, the use of desensitized data in various business scenarios is simulated and tested, such as data analysis, report generation, business process approval, etc., to ensure that the data can still meet business requirements after desensitization. In terms of compliance, according to industry standards, it is checked whether the desensitization strategy and result meet the data protection requirements. Through continuous verification and evaluation, the desensitization strategy and algorithm are adjusted and optimized in time to ensure the high quality of desensitization effect.

[0121] S480, visualizing the target sensitive data based on the flow relationship, distribution and hierarchical structure of the target sensitive data.

[0122] In this solution, in order to meet the needs of different target sensitive data types and business analysis, in addition to basic visualization charts such as bar charts, line charts, and pie charts, targeted charts such as relationship diagrams, heat maps, and tree diagrams can also be selected according to data characteristics to present data logic and core information more accurately.

[0123] Furthermore, for illustrating the flow of sensitive data between different departments, relationship diagrams can clearly present the data's source, destination, and interaction. For the distribution of sensitive data across different time and geographical dimensions, heatmaps can visually display the data density and trends. Tree diagrams are suitable for displaying the hierarchical structure of sensitive data, such as the classification and proportion of sensitive data in each department within an organization's organizational structure. These visualization templates help business personnel gain a more comprehensive and in-depth understanding of the characteristics and internal relationships of sensitive data.

[0124] In this embodiment, Figure 6 The flowchart for sensitive data compliance demonstration provided in Embodiment 3 of this application is as follows: Figure 6 As shown, a precise data mapping algorithm and interactive display technology were used in the process of mapping the de-identified sensitive data to the visualization template. Based on the attributes and business metrics of the target sensitive data, the data was accurately mapped to various dimensions of the visualization elements, such as the height of the bar chart, the coordinates of the line chart, and the color intensity of the heatmap.

[0125] Among them, such as Figure 6 As shown, the visualization module features a wealth of interactive functions, covering core scenarios such as displaying details on mouse hover, triggering data drill-down analysis by clicking charts, and dynamically demonstrating time-series data by sliding sliders. Taking a bar chart as an example: when the mouse hovers over a bar, the system not only instantly displays the specific value and type of the sensitive data corresponding to that bar, but also simultaneously shows the associated business background explanation and risk level assessment results. This type of interactive design allows business personnel to break free from the limitations of fixed views and independently explore data relationships based on actual needs. For example, they can quickly locate abnormal branches from the overall data or trace related factors from a single indicator, ultimately supporting decision-making through more accurate data analysis and effectively improving decision-making efficiency and reliability.

[0126] Furthermore, such as Figure 6 As shown, during the display of sensitive data, we strictly adhere to relevant laws and regulations and the company's internal data security policies to ensure the compliance of the displayed content. Sensitive data is displayed in a tiered and categorized manner, with different visual labels and access control measures applied to data of varying sensitivity levels.

[0127] In this embodiment, as Figure 6As shown, a dynamic updating mechanism is established, which can track the changes of sensitive data in real time and automatically update the visual display content. For example, when the enterprise adds a batch of sensitive data or updates the existing sensitive data, the system can timely reflect these changes in the visual chart, ensuring that business personnel obtain the latest and most accurate sensitive data information. In addition, data export and printing functions are provided to facilitate data sharing and archiving by business personnel, but strict security control is also implemented during the export and printing process to prevent sensitive data leakage.

[0128] The technical scheme of the embodiment of the application comprises the following steps: obtaining text data; performing word segmentation processing on the text data to obtain target text data; extracting features from the target text data by using a first model to obtain text features; performing semantic analysis on the text features to obtain target text features; inputting the target text features into a second model and a third model to output sensitive data. According to the data content of the sensitive data, a desensitization rule matched with the sensitive data is determined; the sensitive data is desensitized by using the desensitization rule to obtain target sensitive data; and the target data is visually displayed based on the flow relationship, distribution and hierarchical structure of the target sensitive data. By executing the technical scheme, the advanced data desensitization technology maximizes the retention of data value on the basis of strict protection of sensitive information and effectively prevents data leakage risks. The compliance and intuitive visual display, through innovative templates and interactive functions, helps business personnel quickly understand data, improves decision-making efficiency, and promotes efficient circulation and deep value mining of data within the enterprise. The good expansibility and adaptability of the system can keep up with changes in the enterprise data environment and business needs, continuously optimize strategies, and ensure long-term stable data security and compliance.

[0129] Embodiment Four

[0130] Figure 7 A structural schematic diagram of a sensitive data detection device provided by Embodiment Four of the application.

[0131] As Figure 7 shown, the device comprises:

[0132] The text data acquisition module 710 is configured to acquire text data.

[0133] The target text data obtaining module 720 is configured to perform word segmentation processing on the text data to obtain target text data.

[0134] The text feature obtaining module 730 is configured to extract features from the target text data by using a first model to obtain text features; wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional context information of words.

[0135] The target text feature obtaining module 740 is configured to perform semantic analysis on the text feature to obtain a target text feature.

[0136] The sensitive information output module 750 is configured to input the target text feature into a second model and a third model to output sensitive data. The second model learns basic features of the sensitive data through sample training. The third model learns deep features of the sensitive data through sample training. The deep features include semantic features, context correlation features, or cross-field correlation features.

[0137] Optionally, the sensitive information output module 750 includes:

[0138] The target text feature processing unit is configured to process the target text feature according to a data type of the target text feature to obtain a processed target text feature.

[0139] The first data set output unit is configured to input the processed target text feature into the second model to output a first data set.

[0140] The sensitive data output unit is configured to input the first data set into the third model to output sensitive data if the first data set meets a preset data condition. The data condition is used to filter and limit data related to the sensitive data in the first data set.

[0141] Optionally, the target text feature processing unit is specifically configured to:

[0142] determine a data type of the target text feature. The data type includes structured data, unstructured data, and semi-structured data.

[0143] if the target text feature is structured data, the target text feature is determined as the processed target text feature.

[0144] if the target text feature is unstructured data, the target text feature is processed using a natural language processing technology to obtain the processed target text feature.

[0145] if the target text feature is semi-structured data, the target text feature is format-converted, and the format-converted target text feature is determined as the processed target text feature. Alternatively, the format-converted target text feature is processed using a natural language processing technology to obtain the processed target text feature.

[0146] Optionally, the target text data obtaining module 720 is specifically configured to:

[0147] The text data is subjected to at least one operation of cleaning, word segmentation, stop word removal, word reduction and part-of-speech tagging to obtain target text data.

[0148] Optionally, the first model is a BERT model, and the text feature obtaining module 730 is specifically configured to:

[0149] input the target text data into the BERT model;

[0150] perform encoding processing on the target text data by using a multi-layer Transformer structure of the BERT model, and extract text features from an output of the multi-layer Transformer structure.

[0151] Optionally, the target text feature obtaining module 740 is specifically configured to:

[0152] perform semantic analysis on the text features by using a pre-constructed semantic model to obtain target text features, wherein the semantic model is used to represent words and semantic relationships thereof in the text.

[0153] Optionally, the apparatus further includes:

[0154] a desensitization rule determining module configured to determine a desensitization rule matched with the sensitive data according to data content of the sensitive data;

[0155] a target sensitive information obtaining module configured to perform desensitization on the sensitive data by using the desensitization rule to obtain target sensitive data;

[0156] a visual display module configured to visually display the target sensitive data based on a flow relationship, a distribution and a hierarchical structure of the target sensitive data.

[0157] The sensitive data detection apparatus provided in the embodiments of the present application can execute the sensitive data detection method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0158] Embodiment five

[0159] Figure 8A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0160] As shown in Figure 8 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0161] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0162] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a sensitive data detection method.

[0163] In some embodiments, a sensitive data detection method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of a sensitive data detection method as described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a sensitive data detection method by other means, e.g., with the aid of firmware.

[0164] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0165] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0166] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0167] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0168] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0169] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0170] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0171] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method of sensitive data detection, the method comprising: The method comprises: obtaining text data; performing word segmentation processing on the text data to obtain target text data; extracting features from the target text data by using a first model to obtain text features, wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional context information of words; performing semantic analysis on the text features to obtain target text features; inputting the target text features into a second model and a third model to output sensitive data, wherein the second model learns basic features of sensitive data through sample training, and the third model learns deep features of sensitive data through sample training, and the deep features include semantic features, context association features, or cross-field association features.

2. The method of claim 1, wherein, Inputting the target text features into the second model and the third model to output sensitive data comprises: processing the target text features according to a data type of the target text features to obtain processed target text features; inputting the processed target text features into the second model to output a first data set; if the first data set meets a preset data condition, inputting the first data set into the third model to output sensitive data, wherein the data condition is used to filter and limit data related to sensitive data in the first data set.

3. The method of claim 2, wherein, Processing the target text features according to a data type of the target text features to obtain processed target text features comprises: determining the data type of the target text features, wherein the data type includes structured data, unstructured data, and semi-structured data; if the target text features are structured data, the target text features are determined as the processed target text features; if the target text features are unstructured data, the target text features are processed by using natural language processing technology to obtain the processed target text features; if the target text features are semi-structured data, the target text features are format-converted, and the format-converted target text features are determined as the processed target text features; or the format-converted target text features are processed by using natural language processing technology to obtain the processed target text features.

4. The method of claim 1, wherein, Performing word segmentation processing on the text data to obtain target text data comprises: performing at least one operation of cleaning, word segmentation, stop word removal, morphological restoration, and part-of-speech tagging on the text data to obtain target text data.

5. The method of claim 1, wherein, The first model is a BERT model, and extracting features from the target text data by using the first model to obtain text features comprises: inputting the target text data into the BERT model; encoding the target text data by using a multi-layer Transformer structure of the BERT model, and extracting text features from an output of the multi-layer Transformer structure.

6. The method of claim 1, wherein, Performing semantic analysis on the text features to obtain target text features comprises: The semantic model is used for representing words in the text and semantic relationship thereof.

7. The method of claim 1, wherein, After the sensitive data is output, the method further includes: According to the data content of the sensitive data, a desensitization rule matched with the sensitive data is determined; The sensitive data is desensitized by using the desensitization rule to obtain target sensitive data; The target sensitive data is visualized based on flow relationship, distribution, and hierarchical structure of the target sensitive data.

8. A sensitive data detection apparatus characterized by comprising: Comprise: A text data acquisition module is configured to acquire text data; A target text data obtaining module is configured to perform word segmentation on the text data to obtain target text data; A text feature obtaining module is configured to extract features from the target text data by using a first model to obtain text features, wherein the first model extracts features associated with sensitive data from the target text data by considering bidirectional context information of words; A target text feature obtaining module is configured to perform semantic analysis on the text features to obtain target text features; A sensitive information output module is configured to input the target text features into a second model and a third model to output sensitive data, wherein the second model learns basic features of the sensitive data through sample training, and the third model learns deep features of the sensitive data through sample training, and the deep features include semantic features, context correlation features, or cross-field correlation features.

9. An electronic device, comprising: The electronic device comprises: At least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the sensitive data detection method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the sensitive data detection method of any one of claims 1-7 when executed.

Images