Method, apparatus, apparatus and computer storage medium for identifying sensitive data

Abstract

The application discloses a sensitive data identification method and device, equipment and a computer storage medium, adopts a text mining technology to mine a plurality of sensitive data rules from a data security specification file of a target industry to form a sensitive data rule library, and continuously expands the rule library by using NLP, NER and other technologies, and after obtaining to-be-identified data of the target industry, the sensitive classification and sensitive level of the to-be-identified data can be identified by matching the sensitive data rules in the sensitive data rule library corresponding to the target industry with the to-be-identified data. According to the embodiment of the application, the sensitive data rule library corresponding to the industry obtained by text mining on the data security specification file in the industry is suitable for all enterprises and individuals in the industry, compared with the enterprise-level data product prepared by human, has a wider application range, has universality and a wider application prospect, and has higher compliance, so that the industry standard can be technically executed.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a sensitive data identification method, apparatus, device, and computer storage medium. Background Technology

[0002] Currently, data security oversight is increasing across many industries, and relevant industry-specific data security standards are being introduced. How to technically ensure that sensitive data is legally and compliantly identified and processed is a key research area in data security.

[0003] Current methods for identifying sensitive data primarily employ customization. Experts develop specific sensitive word lists, identification rules, and identification models tailored to the needs of individual enterprises, based on their expertise and historical data. However, this customization approach results in data products that are not universally applicable and lack general applicability. Different companies require different data products, leading to an immense workload for experts. Summary of the Invention

[0004] This application provides a sensitive data identification method, apparatus, device, and computer storage medium, which can automatically mine a sensitive data rule base applicable to the entire industry from industry data security specification documents. Based on the sensitive data rule base, it can identify sensitive data across the entire industry and has extremely high universality.

[0005] In a first aspect, embodiments of this application provide a method for identifying sensitive data, including:

[0006] Acquire the target industry's data to be identified;

[0007] Find the sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified. The sensitive data rule base includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology.

[0008] Based on the sensitive data rules in the sensitive data rule base that match the data to be identified, the sensitive information of the data to be identified is determined.

[0009] As one possible implementation, before searching for sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, the method also includes:

[0010] Obtain data security specification documents for the target industry;

[0011] Text mining techniques are used to extract multiple sensitive data rules from the obtained data security specification documents. Each sensitive data rule includes the following parameters: rule name, sensitivity category, sensitivity level, feature item, sensitive words, and keywords. Each feature item specifically includes the following parameters: feature item name, regular expression for identifying the feature item, dictionary, and checksum.

[0012] Multiple sensitive data rules are combined to form a sensitive data rule library corresponding to the target industry;

[0013] The values ​​of the parameter items included in multiple feature items in the sensitive data rule base are used to form the feature item library corresponding to the sensitive data rule base.

[0014] As one possible implementation, before searching for sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, the method also includes:

[0015] For each sensitive data rule in the sensitive data rule base, the parameter values ​​of keywords and sensitive words in the sensitive data rule are augmented to obtain synonyms for those parameter values, and these synonyms are added to the keywords and sensitive words of the sensitive data rule; and / or,

[0016] For each sensitive data rule in the sensitive data rule base, the parameter values ​​in the feature terms of the sensitive data rule are augmented to obtain synonyms and / or similar words for that parameter value. The synonyms and / or similar words are then used to form a dictionary of feature terms.

[0017] As one possible approach, this involves searching the sensitive data rule base corresponding to the target industry for rules that match the data to be identified, including:

[0018] Obtain structured data from the data to be identified;

[0019] Extract the field names to be identified and the corresponding data from the structured data;

[0020] Search the sensitive data rule base for sensitive data rules that match the keywords and field names, and use them as the first sensitive data rule;

[0021] In response to finding a first sensitive data rule from the sensitive data rule base, determine whether the first feature item matches the data corresponding to the field name based on the preset regular expression and / or dictionary of the first feature item, wherein the first feature item is the feature item of the first sensitive data rule;

[0022] In response to the first feature item matching the data corresponding to the field name, the first sensitive data rule is determined to be a sensitive data rule that matches the structured data.

[0023] As one possible implementation, searching the sensitive data rule base corresponding to the target industry for sensitive data rules that match the data to be identified also includes:

[0024] In response to the failure to find the first sensitive data rule in the sensitive data rule base, or the mismatch between the first feature item and the data corresponding to the field name, the data corresponding to the field name is divided into data to be detected and data to be verified.

[0025] The regular expressions and / or dictionaries of the feature terms of all sensitive data rules in the sensitive data rule base are matched with the data to be detected to determine whether there are any feature terms that match the data items to be detected.

[0026] In response to the existence of a feature term that matches the data to be detected, the feature term that matches the data to be detected is used as the second feature term;

[0027] The regular expression and / or dictionary of the second feature term are matched against the validation data to determine whether the second feature term matches the validation data.

[0028] In response to the second feature matching the validation data, the sensitive data rule to which the second feature belongs is determined to be a sensitive data rule that matches the structured data.

[0029] As one possible implementation, sensitive information of the data to be identified is determined based on sensitive data rules in a sensitive data rule base that match the data to be identified, including:

[0030] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches the structured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches the structured data are used as the sensitive classification and sensitivity level of the structured data.

[0031] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match structured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match structured data is used as the sensitive data category and sensitivity level of the structured data.

[0032] As one possible approach, this involves searching the sensitive data rule base corresponding to the target industry for rules that match the data to be identified, including:

[0033] Obtain unstructured data from the data to be identified;

[0034] Unstructured data is segmented into words to obtain the segmentation results.

[0035] The full-text matching of the segmented unstructured data is performed using all the sensitive words in the sensitive data rule base to identify the sensitive words in the sensitive data rule base that match the unstructured data.

[0036] In response to sensitive words that match unstructured data in the sensitive data rule base, the sensitive words that match the unstructured data are taken as target sensitive words, and the sensitive data rules to which the target sensitive words belong are taken as target sensitive data rules.

[0037] Record the location of target sensitive words in unstructured data;

[0038] Extract the N characters before and after a given position from unstructured data, where N is a positive integer;

[0039] The regular expression and / or dictionary of the feature terms of the target sensitive data rule are matched against N characters to determine whether the feature terms of the target sensitive data rule match N characters.

[0040] In response to a feature item of a target sensitive data rule matching N characters, the target sensitive data rule is determined to be a sensitive data rule that matches unstructured data. One possible implementation is to search the sensitive data rule library corresponding to the target industry for sensitive data rules that match the data to be identified, including:

[0041] The dictionary of all feature terms in the sensitive data rule base and / or the trained named entity recognition model are used to identify feature terms in unstructured data as the third feature term;

[0042] Search the sensitive data rule base for sensitive data rules that contain the third feature term, and use them as the third sensitive data rule;

[0043] In response to finding a third sensitive data rule in the sensitive data rule base, a sensitive data rule matching the unstructured data is determined from the third sensitive data rule.

[0044] As one possible implementation, sensitive data rules that match unstructured data are determined from the third set of sensitive data rules, including:

[0045] In response to the existence of multiple third-party sensitive data rules in the sensitive data rule base, semantic classification processing is performed on unstructured data to obtain the sensitive category to which the unstructured data belongs; among the multiple third-party sensitive data rules, the sensitive data rule whose sensitive category is consistent with the sensitive category to which the unstructured data belongs is identified as the sensitive data rule that matches the data to be identified.

[0046] As one possible implementation, sensitive information of the data to be identified is determined based on sensitive data rules in a sensitive data rule base that match the data to be identified, including:

[0047] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches unstructured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches unstructured data are taken as the sensitive classification and sensitivity level of unstructured data.

[0048] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match unstructured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match unstructured data is used as the sensitive data category and sensitivity level of the unstructured data.

[0049] Secondly, embodiments of this application also provide a sensitive data identification device, comprising:

[0050] The data acquisition module is used to acquire the data to be identified for the target industry;

[0051] The rule lookup module is used to find sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified. The sensitive data rule base includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology.

[0052] The sensitive information determination module is used to determine the sensitive information in the data to be identified based on the sensitive data rules existing in the sensitive data rule bank that match the data to be identified.

[0053] Thirdly, embodiments of this application also provide an electronic device, the device comprising: a processor and a memory storing computer program instructions;

[0054] When the processor executes the computer program instructions, it implements the sensitive data identification method as described in the first aspect.

[0055] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the sensitive data identification method as described in the first aspect.

[0056] Fifthly, embodiments of this application also provide a computer program product, wherein instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the sensitive data identification method as described in the first aspect.

[0057] This application provides a sensitive data identification method, apparatus, device, and computer storage medium. It employs text mining technology to extract multiple sensitive data rules from data security specification documents of a target industry, forming a sensitive data rule library. After obtaining the data to be identified from the target industry, it searches the sensitive data rule library corresponding to the target industry for sensitive data rules that match the data to be identified. Based on the matching sensitive data rules, the sensitive information of the data to be identified is determined. According to this application, the sensitive data rule library corresponding to an industry, obtained through text mining of data security specification documents, is applicable to all enterprises and individuals within the industry. Compared to manually formulated enterprise-level data products, it has a wider scope of application, greater universality and broader application prospects, and higher compliance, making industry standards technically enforceable. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating a sensitive data identification method provided in one embodiment of this application;

[0060] Figure 2 This is a schematic diagram of a word2vec model provided in one embodiment of this application;

[0061] Figure 3 This is a flowchart illustrating a dictionary augmentation scheme based on bootstrapping, provided in one embodiment of this application.

[0062] Figure 4 This is a flowchart illustrating a specific implementation of step S12 according to an embodiment of this application;

[0063] Figure 5 This is a flowchart illustrating a specific implementation of step S12 provided in another embodiment of this application;

[0064] Figure 6 This is a flowchart illustrating a specific implementation of step S12 provided in another embodiment of this application;

[0065] Figure 7 This is a flowchart illustrating a specific implementation of step S12 provided in another embodiment of this application;

[0066] Figure 8 This is a schematic diagram of a named entity recognition model provided in one embodiment of this application;

[0067] Figure 9 This is a flowchart of a sensitive data identification method provided in one embodiment of this application;

[0068] Figure 10 This is a schematic diagram of a sensitive data identification system provided in one embodiment of this application;

[0069] Figure 11 This is a schematic diagram of the structure of a sensitive data identification device provided in one embodiment of this application;

[0070] Figure 12 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0071] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0072] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0073] Taking the financial industry as an example, with the deepening of open banking practices, data security has become a crucial aspect of the open banking ecosystem that urgently needs strengthening. Ensuring the legal and compliant identification and processing of sensitive data during its sharing with all parties in the ecosystem (banks and application scenarios) is a vital part of open banking scenario security. Research in the financial industry reveals that most small and medium-sized banks, as well as financial institutions operating within open banking scenarios, currently lack robust data standards and management capabilities. Their technical ability to effectively identify and process sensitive data is relatively weak, posing a significant data security risk in the open ecosystem. Furthermore, current market offerings for sensitive data detection primarily consist of customized data products tailored to specific enterprise needs; there are no universal, industry-level technical tools to help enterprises automatically identify and process sensitive data under industry standards.

[0074] Existing solutions for sensitive data identification primarily involve: 1) Pre-setting sensitive word lists and identification rules, then using regular expressions, scripting languages, etc., to determine whether sensitive data exists in structured data records. 2) Defining sensitive data by pre-defining security goals and influencing factors, constructing sensitive datasets under different categories using classification learning models, and then using deep learning to train a sensitive data identification model to identify unstructured sensitive data with semantic features. 3) Based on pre-set sensitive words, analyzing the associations between words in a corpus to mine related words and association rules, updating the security policies corresponding to the sensitive words; and simultaneously using a full-text search engine framework to improve the efficiency of data matching.

[0075] Overall, existing technical solutions have two main limitations: First, they require manual pre-setting of sensitive words and rules. Due to limitations in expert knowledge and historical data, the number of sensitive rules is limited and their scalability is poor. Furthermore, the lack of a more universal sensitive data rule library makes it impossible to guarantee full compliance with industry standards. Second, existing technical methods are mainly used to determine the existence of sensitive data. On the one hand, they cannot handle scenarios where data fields contain multiple categories and levels of sensitive words. On the other hand, when the amount of data to be identified is large, the system overhead is high, and the identification efficiency and accuracy will decrease significantly.

[0076] In view of this, this application proposes a universal, industry-grade sensitive data identification method and apparatus that can help enterprises automatically identify and process sensitive data under industry standard requirements.

[0077] The sensitive data identification method provided in this application can be applied to various industries that need to identify sensitive information, such as the financial industry, to identify sensitive data therein.

[0078] See Figure 1 The above is a flowchart illustrating a sensitive data identification method provided in an embodiment of this application. Figure 1 As shown, the sensitive data identification method provided in this embodiment may include the following steps:

[0079] S11. Obtain the data to be identified for the target industry.

[0080] The target industry can be any industry that requires sensitive data identification, such as the financial industry. The data to be identified within the target industry is the data that needs to be identified as sensitive data within that industry.

[0081] The sensitive data identification method provided in this embodiment can be executed by a sensitive data identification system.

[0082] In one example, the target industry's data to be identified can be stored in a sensitive data identification system, allowing the data to be retrieved directly from the system. This data can be entered into the sensitive data identification system by a user or automatically retrieved and stored by the system from a server or in the cloud.

[0083] S12. Locate the sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified. The sensitive data rule base includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology.

[0084] In this embodiment, multiple sensitive data rules can be pre-mined from the data security specification documents of the target industry using text mining technology, forming a sensitive data rule library corresponding to the target industry. Thus, after obtaining the data to be identified, sensitive data identification can be performed directly based on the sensitive data rule library corresponding to the target industry. The data security specification documents of the target industry refer to the unified technical requirements for data security established within the target industry, which are industry standards that enterprises or individuals within the target industry must comply with. Sensitive data rules are rules used to identify sensitive data; for any data, if the data matches a sensitive data rule, it indicates that the data contains sensitive data.

[0085] In one example, when searching for sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, each sensitive data rule in the sensitive data rule base can be matched with the data to be identified separately, thereby finding the sensitive data rules that match the data to be identified.

[0086] S13. Determine the sensitive information of the data to be identified based on the sensitive data rules in the sensitive data rule base that match the data to be identified. If a sensitive data rule matching the data to be identified is found in the sensitive data rule base, it indicates that sensitive data exists in the data to be identified. At this point, the sensitive information of the data to be identified can be further determined based on the found sensitive data rules. Sensitive information refers to information related to the sensitive data, such as sensitive classification, sensitivity level, etc. Here, sensitive classification refers to the category to which the sensitive data belongs, and sensitivity level refers to the sensitivity level of the sensitive data.

[0087] This embodiment provides a sensitive data identification method that uses text mining technology to extract multiple sensitive data rules from data security specification documents of a target industry, forming a sensitive data rule library. After obtaining the data to be identified in the target industry, the method searches for sensitive data rules in the corresponding sensitive data rule library that match the data to be identified, and determines the sensitive information of the data to be identified based on the matching sensitive data rules. According to this embodiment, the sensitive data rule library corresponding to an industry, obtained by text mining data security specification documents within the industry, is applicable to all enterprises and individuals within the industry. Compared with manually formulated enterprise-level data products, it has a wider scope of application, universality, and broader application prospects, and also has higher compliance, making industry standards technically enforceable.

[0088] In some embodiments, before performing step S12 above, the sensitive data rule base corresponding to the target industry can be set up in the following manner:

[0089] Obtain data security specification documents for the target industry;

[0090] Text mining techniques are used to extract multiple sensitive data rules from the obtained data security specification documents. Each sensitive data rule includes the following parameters: rule name, sensitivity category, sensitivity level, feature item, sensitive words, and keywords. Each feature item specifically includes the following parameters: feature item name, regular expression for identifying the feature item, dictionary, and checksum.

[0091] The discovered sensitive data rules are used to create a sensitive data rule library corresponding to the target industry.

[0092] Each feature item specifically includes the following parameter values: feature item name, regular expression for identifying the feature item, dictionary, and checksum.

[0093] In one example, when retrieving data security specifications for a target industry, the relevant data security specifications can be uploaded manually through a human-computer interaction interface, or the relevant data security specifications can be selected manually from the local machine or server. Alternatively, data security specifications can be automatically retrieved from the network using search technology.

[0094] After obtaining the data security specification document for the target industry, during text mining, the parameters to be mined are set. Then, based on the set parameters, existing mature text mining algorithms are used to extract the corresponding sensitive data rules from the data security specification document. Thus, each mined sensitive data rule contains values ​​for each parameter. The set parameters include: rule name, sensitivity category, sensitivity level, feature items, sensitive words, and keywords. In addition to the above parameters, other parameters such as rule number can also be included, depending on actual needs. The values ​​of each parameter can include text (such as words, characters, sentences, etc.), numbers, characters, etc.

[0095] In one example, a database table as shown in Table 1 below can be set up in advance, and multiple sensitive data rules mined can be filled into this database table to form a sensitive rule database.

[0096] Table 1:

[0097] Rule Number Rule Name Sensitive Classification Sensitivity Level Feature Term Sensitive words Keywords

[0098] As shown in Table 1, the sensitive data rule base table can contain the following parameter items: "Rule Number", "Rule Name", "Sensitive Category", "Sensitivity Level", "Feature Item", "Sensitive Word", and "Keyword". When filling sensitive data rules into this database table, each parameter value contained in the sensitive data rule can be added to the corresponding parameter item. All parameter values ​​in the same sensitive data rule should be filled in the same row, and parameter values ​​in different sensitive data rules should be filled in different rows. The database table containing multiple sensitive data rules is called the sensitive rule database.

[0099] By setting up a sensitive data rule database table, all sensitive data rules can be stored in a unified format, making them easier to manage and use.

[0100] In one example, taking the financial industry as the target sector, the corresponding data security specifications can include the "Guidelines for Financial Data Security Classification" (JR / T 0197-2020) and the "Technical Specifications for the Protection of Personal Financial Information" issued by the People's Bank of China. When mining sensitive data rules based on the above data security specifications, the parameter values ​​of "rule name," "sensitive category," and "sensitive level" can be obtained from Appendix A of the "Guidelines for Financial Data Security Classification" using text mining techniques. Taking the rule name as an example, the rule name can be extracted through text parsing, word segmentation, and based on sentence structure and part-of-speech, using the "fourth-level subclass" in Appendix A as the "sensitive category." "Feature item" refers to the characteristics of the rule. For example, the rules "admission date," "graduation date," and "certificate validity date" share the common feature item "date." "Keywords" and "sensitive words" are used to locate sensitive rules in structured and unstructured data, respectively. For example, the feature item of the rule "home address" is "address," the keyword is "home address\nhome address," and the sensitive word is "home\naddress\naddress\nresidence."

[0101] The features in a sensitive data rule base can be described using regular expressions, dictionaries, and checksums. Each feature specifically includes the values ​​of the following parameters: feature name, the regular expression used to identify the feature, the dictionary, and the checksum. For ease of use, a feature base corresponding to the sensitive data rule base can be constructed based on the values ​​of the parameters included in multiple features within the sensitive data rule base.

[0102] In one example, a feature library table as shown in Table 2 can be constructed. The values ​​of the parameter items included in multiple feature items are filled into this database table to form a feature item library.

[0103] Table 2:

[0104] Feature number Feature name Regular expressions Feature dictionary Checksum

[0105] As shown in Table 2, the feature database table may include the following parameters: "Feature ID", "Feature Name", "Regular Expression", "Feature Dictionary", and "Checksum". The "Regular Expression" describes the format pattern of the feature; the "Feature Dictionary" includes dictionaries obtained through augmented and named entity recognition models, such as dictionaries of institution names and bank names; and the "Checksum" is a Boolean value indicating whether the feature has a publicly available checksum function, such as for ID card verification.

[0106] By setting up a feature database table, the content of all feature items can be stored in a unified format, making it easier to manage and use.

[0107] In this embodiment, each sensitive data rule mined using text mining technology includes a rule name, sensitivity category, sensitivity level, feature items, sensitive words, and keywords. Each feature item includes a corresponding regular expression, feature dictionary, and checksum. Based on these sensitive data rules, sensitive data, its sensitivity category, and sensitivity level are identified. This enables the effective identification of sensitive data within the data to be identified and its classification according to industry standards.

[0108] In some embodiments, considering that some parameter values ​​in sensitive data rules may have different names in different data, resulting in non-standard naming, for example, fields with the same meaning may have different naming methods in the data security specification text and the data to be identified. Therefore, if parameter values ​​mined from the data security specification text are directly used for sensitive data identification, some sensitive data may not be accurately identified, leading to a low identification accuracy. Therefore, to solve this problem, this application embodiment also proposes to use automatic augmentation technology to augment the parameter values ​​in sensitive data, so that the sensitive data rule base can be continuously and dynamically optimized to improve the identification accuracy. Correspondingly, before performing the above step S12, the following steps can also be performed first:

[0109] For each sensitive data rule in the sensitive data rule base, the parameter values ​​of keywords and sensitive words in the sensitive data rule are augmented to obtain synonyms for those parameter values, and these synonyms are added to the keywords and sensitive words of the sensitive data rule; and / or,

[0110] For each sensitive data rule in the sensitive data rule base, the parameter values ​​in the feature terms of the sensitive data rule are augmented to obtain synonyms and / or similar words for that parameter value. The synonyms and / or similar words are then used to form a dictionary of feature terms.

[0111] The parameter values ​​for both keywords and feature terms can include words.

[0112] Natural Language Processing (NLP) techniques can be used to augment the parameter values ​​of keywords, sensitive words, and / or feature items. For example, various augmentation schemes such as thesaurus, word2vec, and / or bootstrapping can be used to address issues such as the non-standard naming of synonyms like "address" and "residential address," similar words like "Bank of China" and "Industrial Bank," and the same word like "Agricultural Bank of China," "China Agricultural Bank," and "abc," thereby improving the rule base's expansion capabilities and the accuracy of sensitive information identification.

[0113] In one example, if the same meaning has different expressions, it will bring certain difficulties to the accurate matching of sensitive data identification. To solve this problem, a thesaurus-based augmentation scheme can be used to augment the parameter values ​​of "keywords" and / or "sensitive words". Taking the financial industry as an example, the thesaurus based on augmentation can include: (1) publicly available financial datasets, such as field names in statistical data provided by the China Banking and Insurance Regulatory Commission and publicly available interface information of banks; (2) "Thesaurus Extended Edition", which uses a tree structure to include nearly 70,000 entries and a 5-layer coding mode, and uses a hierarchical algorithm and information content to calculate similarity and realize synonym mining; (3) HowNet, also known as CNKI, describes the characteristics of words by constructing a tree-like semantic original graph and a network-like semantic original term graph, and uses the similarity of terms to calculate word similarity and realize synonym mining. The above thesaurus is just an example. In addition to the above thesaurus, other thesaurus can also be used. The specific selection can be based on the actual situation, and no examples will be given here. The above augmentation method can augment the parameter values ​​mined from the text to their corresponding synonyms. For example, if the parameter value of the mined "keyword" is "salary," it can be augmented to obtain synonyms such as "salary," "compensation," and "wages." By augmenting keywords and sensitive words, expressions with the same meaning can be added to the corresponding parameter items, thus improving the matching accuracy of keywords and sensitive words.

[0114] In one example, the lexicon-based augmentation scheme is mainly used for synonym augmentation, but it ignores the contextual information of the corpus in which the words are located. Therefore, a context-based word2vec (word to vector) augmentation scheme can also be used to expand "keywords," "sensitive words," and / or "features." When using the context-based word2vec augmentation scheme, a word vector library for the target industry can be trained, and the similarity of word vectors can be used to augment "keywords," "sensitive words," and / or "features." The word2vec model can use two training modes, CBOW (Continuous Bag-of-Words Model) and Skip-gram (Continuous Skip-gram Model), to measure contextual information. CBOW predicts the current word based on its context; Skip-gram predicts its context based on the current word. Taking the financial industry as an example, this augmentation scheme can identify various forms of bank names, app names, addresses, etc., and form a dictionary, improving the recognition efficiency of sensitive data rule bases. Figure 2 As shown, the CBOW mode uses X1, X2, X c The context is taken as input, and it passes through the input layer P in sequence. jAfter processing by the hidden layer and the output layer, X1, X2, and X are output. c The middle word P in the context j Skip-gram pattern uses the middle word P. j As input, it passes sequentially through the input layer and P. j After processing the hidden layer and the output layer, the corresponding contexts X1, X2, and X3 are obtained. c etc., among which, W V×N W represents the input weight matrix. V ′ ×N This represents the output weight matrix, where V-dim and N-dim represent the vector dimensions, and h... j This represents the output value of the hidden layer.

[0115] In one example, certain specialized "features" in sensitive data rules may have aliases. For instance, in the financial industry, the same bank might have a full name, an abbreviation, and an English name. To improve recognition accuracy, a corresponding specialized dictionary needs to be built for these types of "features." A dictionary augmentation scheme based on bootstrapping can be used to address this. Figure 3 As shown, when using this augmentation scheme, a seed word set is first constructed. Then, based on the seed word set, patterns of occurrence of the original name and its alias in the online corpus are mined to generate a pattern set. Finally, new pairs of original names and aliases are mined from the corpus using this pattern set, thereby achieving the augmentation purpose. The online corpus can be a corpus constructed from sources such as Wikipedia or Baidu Encyclopedia.

[0116] In this embodiment, an automatic augmentation technique is used to augment the parameter values ​​in the sensitive data rules, which consumes less manpower and time compared to manual augmentation. In some embodiments, data can generally be divided into structured data and unstructured data. Structured data, also known as row data, is data logically expressed and implemented using a two-dimensional table structure, strictly adhering to data format and length specifications, and is mainly stored and managed through relational databases. In contrast to structured data, unstructured data is unsuitable for representation by a two-dimensional database table, including all formats of office documents, XML, HTML, various reports, images, and audio / video information. Based on this, in this embodiment, different methods are used to find matching sensitive data rules for both structured and unstructured data in the data to be identified.

[0117] Below, we will first introduce how to find sensitive data rules that match structured data.

[0118] like Figure 4 As shown, the specific implementation of step S12 above may include the following steps:

[0119] S41. Obtain the structured data from the data to be identified.

[0120] In one example, data logically expressed and implemented using a two-dimensional table structure is extracted from the data to be identified and treated as structured data.

[0121] S42. Extract the field names to be identified and the corresponding data from the structured data.

[0122] In this context, the field name refers to the identifier of each column in the structured data, also known as the column name.

[0123] In one example, the sensitive data identification system can extract, based on at least one pre-defined field name, the field name matching the pre-defined field name and the corresponding data from structured data as the field name to be identified and the corresponding data. The pre-defined field name can be set according to actual needs.

[0124] S43. Search the sensitive data rule library for sensitive data rules that match the keywords and field names, and use them as the first sensitive data rule.

[0125] When searching for the first sensitive data rule, you can match all values ​​contained in the keywords of each sensitive data rule in the sensitive data rule library against the field name to determine if there is a value that matches that field name. The sensitive data rule whose keywords contain a value that matches the field name is then taken as the first sensitive data rule that matches that field name.

[0126] S44. In response to finding a first sensitive data rule in the sensitive data rule base, determine whether the first feature item matches the data corresponding to the field name based on the preset regular expression and / or dictionary of the first feature item.

[0127] Among them, the first feature term is the feature term of the first sensitive data rule.

[0128] In one example, the regular expression and / or dictionary of the first feature can be obtained from a pre-defined feature library.

[0129] In one example, when determining whether the first feature item matches the data corresponding to the field name based on the regular expression and / or dictionary of the first feature item, the regular expression or dictionary of the first feature item can be obtained, the data corresponding to the field name can be stratified and sampled, the obtained regular expression or dictionary can be matched with the sampled data, the matching rate between the regular expression or dictionary and the data corresponding to the field name can be obtained, and then the obtained matching rate can be compared with a preset matching rate threshold. If the matching rate is greater than the matching rate threshold, it is determined that the first feature item matches the data corresponding to the field name; otherwise, it is determined that the first feature item does not match the data corresponding to the field name.

[0130] In another example, when determining whether the data corresponding to the first feature term matches the data corresponding to the field name based on the regular expression and / or dictionary of the first feature term, the regular expression of the first feature term can be obtained first. With the regular expression obtained, stratified sampling is performed on the data corresponding to the field name. The obtained regular expression is then matched against the sampled data to obtain the matching rate between the regular expression and the data corresponding to the field name. This matching rate is then compared with a preset matching rate threshold. If the matching rate is greater than the threshold, the data corresponding to the first feature term is determined to match the data corresponding to the field name; otherwise, the data corresponding to the first feature term does not match the data corresponding to the field name. If the regular expression is not obtained, the dictionary of the first feature term is then obtained. Stratified sampling is performed on the data corresponding to the field name. The obtained dictionary is then matched against the sampled data to obtain the matching rate between the dictionary and the data corresponding to the field name. This matching rate is then compared with a preset matching rate threshold. If the matching rate is greater than the threshold, the data corresponding to the first feature term is determined to match the data corresponding to the field name; otherwise, the data corresponding to the first feature term does not match the data corresponding to the field name.

[0131] The matching rate between the regular expression and the data corresponding to the field name refers to the ratio of the number of data points in the sampled data that match the regular expression to the total sampled data. For example, if 10 words are sampled from the data corresponding to the field name, and 8 of them match the regular expression, the matching rate is 0.8. The matching rate threshold can be set according to actual needs.

[0132] S45. In response to the first feature item matching the data corresponding to the field name, the first sensitive data rule is determined to be a sensitive data rule that matches the structured data. In this embodiment, when identifying sensitive data in structured data, the method of first identifying sensitive fields and then verifying sampled data under those fields can reduce the global retrieval of real data in the structured data, thereby reducing data processing pressure and improving the speed of sensitive data identification.

[0133] In some embodiments, Figure 4 Based on what is shown, as Figure 5 As shown, the specific implementation of step S12 above may also include the following steps:

[0134] S51. In response to the failure to find the first sensitive data rule in the sensitive data rule base, or the mismatch between the first feature item and the data corresponding to the field name, the data corresponding to the field name is divided into data to be detected and data to be verified.

[0135] In one example, when dividing the data corresponding to a field name into test data and validation data, a sampling threshold can be set. If the number of rows corresponding to a field name is less than the sampling threshold, the data is divided into two parts: one part is used as test data, and the other part is used as validation data. If the number of rows corresponding to a field name is greater than the sampling threshold, stratified sampling is performed on the data corresponding to the field name, and the stratified sampled data is used as test data. Random sampling is then performed on the data corresponding to the field name, and the randomly sampled data is used as validation data.

[0136] S52. Match the regular expressions and / or dictionaries of the feature terms of all sensitive data rules in the sensitive data rule base with the data to be detected to determine whether there are any feature terms that match the data item to be detected.

[0137] The regular expressions and / or dictionaries for each feature can be obtained from the feature library.

[0138] For each feature term, the method of matching its regular expression and / or dictionary with the data to be detected is the same as the matching method used in step S44 above, and will not be repeated here. The matching process yields the matching rate between the regular expression or dictionary of each feature term and the data to be detected. Feature terms with a matching rate greater than the matching rate threshold are considered to be matched with the data to be detected.

[0139] S53. In response to the existence of a feature term that matches the data to be detected, the feature term that matches the data to be detected is taken as the second feature term.

[0140] S54. Match the regular expression and / or dictionary of the second feature with the validation data to determine whether the second feature matches the validation data.

[0141] Similarly, the method of matching the regular expression and / or dictionary of the second feature with the verification data is the same as that in step S44 above, and will not be repeated here. The matching rate between the regular expression and dictionary of the second feature and the verification data will be obtained through matching. If the matching rate is greater than the matching rate threshold, it is determined that the second feature matches the verification data; otherwise, it is determined that the second feature does not match the verification data.

[0142] S55. In response to the second feature matching the validation data, determine that the sensitive data rule to which the second feature belongs is a sensitive data rule that matches the structured data.

[0143] In this embodiment, if it is impossible to find a sensitive data rule that matches the structured data based on the field name, instead of directly determining that there is no sensitive data in the structured data, the system further searches for a matching sensitive data rule through the data corresponding to the field name. This can improve the accuracy of identification.

[0144] In some embodiments, to further improve identification accuracy, before determining whether the sensitive data rule to which the second feature belongs matches the structured data if the second feature matches the verification data, the following steps may also be performed:

[0145] Obtain the checksum of the second feature term, and use the checksum to further verify the data to be identified. If the verification passes, determine the sensitive data rule to which the second feature term belongs as a sensitive data rule that matches the structured data.

[0146] In some embodiments, considering that the extracted field name format may not be standardized and may contain some descriptive text, the above-mentioned identification method for structured data may fail to identify the sensitive data rules that match it from the sensitive data rule base. In this case, in order to further improve the identification accuracy, a trained named entity recognition model can be used to extract the feature items in the structured data, and then the sensitive data identification can be performed by manual verification based on the extracted feature items.

[0147] The above describes the method for finding sensitive data rules from a sensitive data rule base that match the structured data in the data to be identified. After determining the sensitive data rules that match the structured data using the above method, the sensitive information of the structured data can be determined based on the determined sensitive data rules. Accordingly, the specific implementation of step S13 above may include:

[0148] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches the structured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches the structured data are used as the sensitive classification and sensitivity level of the structured data.

[0149] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match structured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match structured data is used as the sensitive data category and sensitivity level of the structured data.

[0150] In one example, when searching for sensitive data rules that match structured data from the sensitive data rule base, a sequential search is performed. When a sensitive data rule that matches the structured data is found, the sensitive data rule, its sensitive category, and sensitivity level are recorded as sensitive information of the structured data. When other sensitive data rules that match the structured data are subsequently found, the sensitivity levels of the other sensitive data rules are compared with the previously recorded sensitivity levels. If the sensitivity level of the subsequently found sensitive data rule is higher than the recorded sensitivity level, the sensitive information of the structured data is updated to include the other sensitive data rule found, its sensitive category, and sensitivity level. Otherwise, the recorded sensitive information remains unchanged.

[0151] In this embodiment, when there are multiple sensitive data rules that match the structured data, the sensitive information of the structured data is determined according to the sensitive data rule with the highest sensitivity level, thus ensuring the accuracy of the final determined sensitive information.

[0152] The following section introduces methods for finding sensitive data rules that match unstructured data.

[0153] In some embodiments, such as Figure 6 As shown, the specific implementation of step S12 above may also include the following steps:

[0154] S61. Obtain unstructured data from the data to be identified.

[0155] In one example, data other than structured data is extracted from the data to be identified and treated as unstructured data.

[0156] S62. Perform word segmentation on unstructured data to obtain the word segmentation results of unstructured data.

[0157] Existing mature text segmentation technologies can be used to segment unstructured data to obtain the corresponding segmentation results.

[0158] S63. Use all sensitive words in the sensitive data rule base to perform full-text matching on the segmented unstructured data to determine the sensitive words in the sensitive data rule base that match the unstructured data.

[0159] Each sensitive word is matched against the unstructured data in its entirety. If a word matching the sensitive word exists in the unstructured data, then the sensitive word is considered to have matched the unstructured data.

[0160] S64. In response to the existence of sensitive words in the sensitive data rule base that match unstructured data, the sensitive words that match the unstructured data are taken as target sensitive words, and the sensitive data rule to which the target sensitive words belong is taken as the target sensitive data rule.

[0161] S65. Record the location of the target sensitive words in the unstructured data.

[0162] S66. Extract N characters before and after the given position from the unstructured data, where N is a positive integer.

[0163] The value of N can be set according to the actual situation.

[0164] In one example, characters can be taken from before and after the position corresponding to the target sensitive word, for a total of N characters. These N characters are then used as the content to be identified and further identified using the target sensitive data rules.

[0165] S67. Match the regular expression and / or dictionary of the feature terms of the target sensitive data rule with N characters to determine whether the feature terms of the target sensitive data rule match the N characters.

[0166] S68. In response to the feature terms of the target sensitive data rule matching N characters, determine the target sensitive data rule as a sensitive data rule that matches unstructured data.

[0167] In this embodiment, the main focus is on identifying sensitive data from unstructured data within the data to be identified. During identification, sensitive words are first used to locate the content to be identified, and then only the content to be identified is matched with the target sensitive data rules. This method eliminates the need to match all unstructured data with sensitive data rules, reducing the workload required for identification and thus improving identification efficiency.

[0168] In some embodiments, considering that sensitive words may not exist in unstructured data, such as Figure 7 As shown, the specific implementation of step S12 above may also include the following steps:

[0169] S71. Identify the feature terms in unstructured data using a dictionary of all feature terms in the sensitive data rule base and / or a trained named entity recognition model, and use them as the third feature term.

[0170] In one example, a dictionary of all features can be obtained from a feature term library. After obtaining the dictionary, each feature in the dictionary can be matched against the unstructured data in full text. The features that match the unstructured data are then used as the features of the unstructured data. In this way, features of unstructured data can be quickly identified with only simple text matching.

[0171] In one example, the Named Entity Recognition (NER) model is a named entity recognition model built by combining a Bidirectional Long Short-Term Memory (Bi-LSTM) network and a Conditional Random Field (CRF), such as... Figure 8 The diagram shown is a schematic of a named entity recognition model provided in an embodiment of this application. Bi-LSTM relies on the nonlinear fitting capability of neural networks to extract features of the function from samples to labels by subjecting samples to complex nonlinear changes in high-dimensional space during the training of the named entity recognition model, thus constructing a named entity library and a corresponding label knowledge base. CRF can express dependencies between observations, as well as complex dependencies between the current observation and multiple preceding and following states. CRF uses the output of Bi-LSTM, text labels, and matrices in the neural network as input parameters. It utilizes the state transition matrix to constrain labels with obvious errors, such as "O cannot be directly followed by I" and "B cannot be followed by O," where I, O, and B all represent labels. After multiple iterations of training, the model parameters are optimized to obtain a well-trained named entity recognition model.

[0172] By inputting unstructured data into a trained named entity recognition model, we can obtain the sensitive features with entity properties in the unstructured data output by the model; these are the feature terms of the unstructured data. This method allows for the identification of more accurate and comprehensive feature terms.

[0173] The features can include name, brand name, product name, organization name, address, date, etc.

[0174] S72. Search for sensitive data rules containing the third feature term from the sensitive data rule base, and use them as the third sensitive data rules.

[0175] S73. In response to finding a third sensitive data rule from the sensitive data rule base, determine the sensitive data rule that matches the unstructured data from the third sensitive data rule.

[0176] In this embodiment, a sensitive data rule base and a named entity recognition model are combined to identify sensitive features with entity properties in unstructured data, making up for the shortcomings of the sensitive data rule base in identifying unstructured data, thereby realizing sensitive data identification based on the sensitive data rule base for unstructured data.

[0177] In some embodiments, considering that when identifying unstructured data, multiple sensitive data rules containing the third feature may be identified based on the third feature, such as the third feature of unstructured data containing "date," and there may be many sensitive data rules involving dates in the sensitive data rule base, in order to improve identification accuracy, it is necessary to further clarify which sensitive data rule the third feature specifically belongs to. Therefore, determining the sensitive data rule matching the unstructured data based on the third sensitive data rule may include:

[0178] In response to the existence of multiple third-party sensitive data rules in the sensitive data rule base, semantic classification processing is performed on unstructured data to obtain the sensitive category to which the unstructured data belongs. For example, in the financial industry, the multiple rules corresponding to the feature item "date" can belong to different categories such as personal basic information and account basic information.

[0179] Multiple sensitive data rules that are consistent with the sensitive classification of unstructured data in the third sensitive data rules are used as sensitive data rules that match unstructured data.

[0180] Correspondingly, if there is only one third sensitive data rule in the sensitive data rule base, it can be directly used as the sensitive data rule that matches the unstructured data. Alternatively, the third sensitive data rule can be verified based on the sensitive category to which the unstructured data belongs. If it is determined that the sensitive category of the third sensitive data rule is consistent with the sensitive category to which the unstructured data belongs, then the third sensitive data rule can be used as the sensitive data rule that matches the unstructured data.

[0181] In the process of semantic classification of unstructured data, mature text classification technology can be used to classify unstructured data based on the data security specifications of the target industry.

[0182] In one example, taking the financial industry as the target sector, unstructured data can be classified using RNN+ALBERT according to the second-level subcategories of the "Financial Data Security Classification Guidelines," yielding classification results. Since sensitive data rules are also based on data security specifications, the method for determining sensitive categories within them is consistent with this classification method. Therefore, the classification results can correspond to the sensitive categories in the sensitive data rules. This allows for the filtering of sensitive data rules based on the classification results. In this embodiment, filtering the third sensitive data rule based on the sensitive category to which the unstructured data belongs to determines the sensitive data rule that matches the unstructured data, improving the accuracy of sensitive data identification.

[0183] In some embodiments, to further improve the accuracy of identification, the data to be identified can be manually verified after step S12. The manual verification can employ existing, mature verification methods.

[0184] The above describes the method for finding sensitive data rules from a sensitive data rule base that match the unstructured data in the data to be identified. After determining the sensitive data rules that match the unstructured data using the above method, the sensitive information of the unstructured data can be determined based on the determined sensitive data rules. Accordingly, the specific implementation of step S13 above may include:

[0185] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches unstructured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches unstructured data are taken as the sensitive classification and sensitivity level of unstructured data.

[0186] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match unstructured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match unstructured data is used as the sensitive data category and sensitivity level of the unstructured data.

[0187] When searching for sensitive data rules that match unstructured data in the sensitive data rule base, a sequential search is conducted. When a sensitive data rule that matches unstructured data is found, the sensitive data rule, its sensitive category, and sensitivity level are recorded as sensitive information of the unstructured data. When other sensitive data rules that match unstructured data are subsequently found, the sensitivity levels of the other sensitive data rules are compared with the previously recorded sensitivity levels. If the sensitivity level of the subsequently found sensitive data rule is higher than the recorded sensitivity level, the sensitive information of the unstructured data is updated to include the other sensitive data rule found, its sensitive category, and sensitivity level. Otherwise, the recorded sensitive information remains unchanged.

[0188] In this embodiment, when there are multiple sensitive data rules that match unstructured data, the sensitive information of the unstructured data is determined according to the sensitive data rule with the highest sensitivity level, thus ensuring the accuracy of the final determined sensitive information.

[0189] See Figure 9 This is a schematic diagram illustrating the overall process of sensitive data identification provided in an embodiment of this application. Figure 9 As shown, the following steps can be performed when identifying sensitive data:

[0190] S901. Load the preset sensitive data rule library for the target industry.

[0191] S902. Classify the data to be identified into structured data and unstructured data.

[0192] For structured data, perform the following steps:

[0193] S911. Data partitioning.

[0194] S912. Extraction of metadata information.

[0195] Metadata extraction involves extracting the field names to be identified and the data corresponding to those field names.

[0196] S913. Keyword determination: If the determination is successful, execute S914; if the determination fails, execute S916.

[0197] Keyword determination involves searching the sensitive data rule base for sensitive data rules that match the keyword with the field name. A successful determination means a sensitive data rule matching the field name has been found; a failed determination means no sensitive data rule matching the field name has been found.

[0198] S914. Perform partial regular expression / dictionary validation, and execute S915 if the validation is successful.

[0199] In other words, only the regular expressions / dictionaries corresponding to the sensitive data rules that match the field names are used to match and validate the sampled data corresponding to the field names.

[0200] S915. Record sensitive information.

[0201] S916. Regular expression / dictionary check. If the check is successful, execute S917; if the check fails, execute S918.

[0202] In other words, the data corresponding to the field name is matched using regular expressions / dictionaries of all feature items in the sensitive data rule base. A sensitive data rule matching the data corresponding to that field name is found upon successful verification.

[0203] S917. Perform a check and verification, and if the verification is successful, execute S919.

[0204] That is, the data corresponding to the field name is further verified by using the checksum of the feature items in the found sensitive data rules.

[0205] S918.NER model marking.

[0206] That is, using a trained NER model to determine the feature terms of structured data.

[0207] S903. Manual verification.

[0208] For unstructured data, perform the following steps:

[0209] S921. Semantic analysis.

[0210] That is, performing word segmentation on unstructured data to obtain the word segmentation results.

[0211] S922. Text Classification.

[0212] That is, classifying unstructured data to obtain classification results.

[0213] S923.NER model marking.

[0214] That is, to extract feature terms from unstructured data using a trained NER model.

[0215] S924. Rule detection.

[0216] That is, based on the feature terms extracted from the unstructured data, sensitive data rules that match the feature terms are searched from the sensitive data rule base.

[0217] After identifying sensitive data in unstructured data through S923 and S924, manual verification can be further performed through S903.

[0218] In addition to the steps mentioned above, such as Figure 9 As shown, after S912 and S922, the NER model can also be trained online to improve model accuracy.

[0219] It should be noted that sensitive data identification can be performed in parallel for both structured and unstructured data.

[0220] See Figure 10 This is a schematic diagram of a sensitive data identification system provided in an embodiment of this application, such as... Figure 10 As shown, the system may include a local computer 1001 and a server 1002.

[0221] The computer 1001 is used to identify sensitive data from the data to be identified. The computer 1001 only needs to store the sensitive data rule base and the trained NER model. The maintenance (including the maintenance of regular expressions), augmentation, and optimization of the NER model can be completed by the remote server 1002, thereby reducing the space occupied by the system and ensuring the operating efficiency of the data system.

[0222] This embodiment provides a sensitive data identification method and system that comprehensively utilizes the sensitive data rule base and NER model of the target industry to effectively identify sensitive information in structured and unstructured data, solving problems such as limited sensitive data rules, inconsistent naming of sensitive words, and low identification accuracy.

[0223] Based on the sensitive data identification method provided in the above embodiments, this application also provides specific implementation methods of the sensitive data identification device.

[0224] See Figure 11 This is a schematic diagram of the structure of a sensitive data identification device provided in an embodiment of this application, as shown below. Figure 11 As shown, the sensitive data identification device provided in this embodiment may include the following modules:

[0225] The data acquisition module 1101 is used to acquire the data to be identified in the target industry.

[0226] The rule lookup module 1102 is used to find sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified. The sensitive data rule base includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology.

[0227] The sensitive information determination module 1103 is used to determine the sensitive information of the data to be identified based on the sensitive data rules in the sensitive data rule base that match the data to be identified.

[0228] This application provides a sensitive data identification device that uses text mining technology to extract multiple sensitive data rules from data security specification documents of a target industry, forming a sensitive data rule library. After obtaining the data to be identified from the target industry, it searches the sensitive data rule library corresponding to the target industry for sensitive data rules that match the data to be identified, and determines the sensitive information of the data to be identified based on the matching sensitive data rules. According to this application embodiment, the sensitive data rule library corresponding to the industry obtained by text mining data security specification documents is applicable to all enterprises and individuals within the industry. Compared with manually formulated enterprise-level data products, it has a wider scope of application, universality, and broader application prospects, and also has higher compliance, making industry standards technically enforceable.

[0229] In some embodiments, the above apparatus may further include: a rule base construction module, specifically used for:

[0230] Before checking whether there are sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, obtain the data security specification document of the target industry;

[0231] Text mining techniques are used to extract multiple sensitive data rules from the obtained data security specification documents. Each sensitive data rule includes the following parameters: rule name, sensitivity category, sensitivity level, feature item, sensitive words, and keywords. Each feature item specifically includes the following parameters: feature item name, regular expression for identifying the feature item, dictionary, and checksum.

[0232] Multiple sensitive data rules are combined to form a sensitive data rule library corresponding to the target industry;

[0233] The values ​​of the parameter items contained in multiple feature items in the sensitive data rule base are combined to form the feature item library corresponding to the sensitive data rule base.

[0234] In some embodiments, the above-described apparatus may further include: an augmentation module, specifically used for:

[0235] Before checking whether there are sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, for each sensitive data rule in the sensitive data rule base, the parameter values ​​of keywords and sensitive words in the sensitive data rule are augmented to obtain synonyms for those parameter values, and these synonyms are added to the keywords and sensitive words of the sensitive data rule; and / or,

[0236] For each sensitive data rule in the sensitive data rule base, the parameter values ​​in the feature terms of the sensitive data rule are augmented to obtain synonyms and / or similar words for that parameter value. The synonyms and / or similar words are then used to form a dictionary of feature terms.

[0237] In some embodiments, the rule lookup module 1102 is specifically used for:

[0238] Obtain structured data from the data to be identified;

[0239] Extract the field names to be identified and the corresponding data from the structured data;

[0240] Search the sensitive data rule base for sensitive data rules that match the keywords and field names, and use them as the first sensitive data rule;

[0241] In response to finding a first sensitive data rule from the sensitive data rule base, determine whether the first feature item matches the data corresponding to the field name based on the preset regular expression and / or dictionary of the first feature item, wherein the first feature item is the feature item of the first sensitive data rule;

[0242] In response to the first feature item matching the data corresponding to the field name, the first sensitive data rule is determined to be a sensitive data rule that matches the structured data.

[0243] In some embodiments, the rule lookup module 1102 can also be used for:

[0244] In response to the failure to find the first sensitive data rule in the sensitive data rule base, or the mismatch between the first feature item and the data corresponding to the field name, the data corresponding to the field name is divided into data to be detected and data to be verified.

[0245] The regular expressions and / or dictionaries of the feature terms of all sensitive data rules in the sensitive data rule base are matched with the data to be detected to determine whether there are any feature terms that match the data items to be detected.

[0246] In response to the existence of a feature term that matches the data to be detected, the feature term that matches the data to be detected is used as the second feature term;

[0247] The regular expression and / or dictionary of the second feature term are matched against the validation data to determine whether the second feature term matches the validation data.

[0248] In response to the second feature matching the validation data, the sensitive data rule to which the second feature belongs is determined to be a sensitive data rule that matches the structured data.

[0249] In some embodiments, the sensitive information determination module 1103 is specifically used for:

[0250] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches the structured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches the structured data are used as the sensitive classification and sensitivity level of the structured data.

[0251] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match structured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match structured data is used as the sensitive data category and sensitivity level of the structured data.

[0252] In some embodiments, the rule lookup module 1102 can also be used for:

[0253] Obtain unstructured data from the data to be identified;

[0254] Unstructured data is segmented into words to obtain the segmentation results.

[0255] The full-text matching of the segmented unstructured data is performed using all the sensitive words in the sensitive data rule base to determine whether there are any sensitive words in the sensitive data rule base that match the unstructured data.

[0256] In response to the existence of sensitive words in the sensitive data rule base that match unstructured data, the sensitive words that match the unstructured data are taken as target sensitive words, and the sensitive data rules to which the target sensitive words belong are taken as target sensitive data rules.

[0257] Record the location of target sensitive words in unstructured data;

[0258] Extract the N characters before and after a given position from unstructured data, where N is a positive integer;

[0259] The regular expression and / or dictionary of the feature terms of the target sensitive data rule are matched against N characters to determine whether the feature terms of the target sensitive data rule match N characters.

[0260] In response to the feature terms of the target sensitive data rule matching N characters, the target sensitive data rule is determined to be a sensitive data rule that matches unstructured data.

[0261] In some embodiments, the rule lookup module 1102 can also be used for:

[0262] The dictionary of all feature terms in the sensitive data rule base and / or the trained named entity recognition model are used to identify feature terms in unstructured data as the third feature term;

[0263] Search the sensitive data rule base for sensitive data rules that contain the third feature term, and use them as the third sensitive data rule;

[0264] In response to finding a third sensitive data rule in the sensitive data rule base, a sensitive data rule matching the unstructured data is determined from the third sensitive data rule base. In some embodiments, determining the sensitive data rule matching the unstructured data from the third sensitive data rule base includes:

[0265] In response to the existence of multiple third-party sensitive data rules in the sensitive data rule base, semantic classification processing is performed on unstructured data to obtain the sensitive category to which the unstructured data belongs;

[0266] Sensitive data rules whose sensitive classification matches the sensitive classification of unstructured data are considered as sensitive data rules that match unstructured data.

[0267] In some embodiments, the sensitive information determination module 1103 is specifically used for:

[0268] In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches unstructured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches unstructured data are taken as the sensitive classification and sensitivity level of unstructured data.

[0269] In response to the existence of multiple sensitive data rules in the sensitive data rule base that match unstructured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match unstructured data is used as the sensitive data category and sensitivity level of the unstructured data.

[0270] The sensitive data identification device provided in this application can implement all the processes implemented in any of the above-described sensitive data identification method embodiments. To avoid repetition, it will not be described again here.

[0271] Figure 12 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0272] The electronic device may include a processor 1201 and a memory 1202 storing computer program instructions.

[0273] Specifically, the processor 1201 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0274] Memory 1202 may include mass storage for data or instructions. For example, and not limitingly, memory 1202 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1202 may include removable or non-removable (or fixed) media. Where appropriate, memory 1202 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1202 is non-volatile solid-state memory.

[0275] Memory 1202 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 1202 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the sensitive data identification methods in the above embodiments.

[0276] The processor 1201 implements any of the sensitive data identification methods in the above embodiments by reading and executing computer program instructions stored in the memory 1202.

[0277] In one example, the electronic device may also include a communication interface 1203 and a bus 1210. For example, Figure 12 As shown, the processor 1201, memory 1202, and communication interface 1203 are connected through bus 1210 and complete communication with each other.

[0278] The communication interface 1203 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0279] Bus 1210 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1210 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0280] Furthermore, in conjunction with the sensitive data identification methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the sensitive data identification methods in the above embodiments.

[0281] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0282] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0283] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0284] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0285] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for identifying sensitive data, characterized in that, include: Acquire the target industry's data to be identified; Find the sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified. The sensitive data rule base includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology. Based on the sensitive data rules in the sensitive data rule base that match the data to be identified, the sensitive information of the data to be identified is determined; Before searching for sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, the method further includes: Obtain the data security specification documents for the target industry; Text mining techniques are used to extract multiple sensitive data rules from the obtained data security specification file. Each sensitive data rule includes the following parameter values: rule name, sensitivity category, sensitivity level, feature item, sensitive word, and keyword; each feature item specifically includes the following parameter values: feature item name, regular expression for identifying the feature item, dictionary, and checksum. The multiple sensitive data rules are combined to form a sensitive data rule library corresponding to the target industry; The values ​​of the parameter items included in multiple feature items in the sensitive data rule base are used to form the feature item library corresponding to the sensitive data rule base; The process of finding sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified includes: Obtain unstructured data from the data to be identified; The unstructured data is segmented into words to obtain the segmentation results of the unstructured data; The unstructured data after word segmentation is matched with all the sensitive words in the sensitive data rule library to determine the sensitive words in the sensitive data rule library that match the unstructured data. In response to a sensitive word in the sensitive data rule base that matches the unstructured data, the sensitive word that matches the unstructured data is taken as the target sensitive word, and the sensitive data rule to which the target sensitive word belongs is taken as the target sensitive data rule; Record the position of the target sensitive words in the unstructured data; Extract the N characters before and after the specified position from the unstructured data, where N is a positive integer; The regular expression and / or dictionary of the feature terms of the target sensitive data rule are matched with the N characters to determine whether the feature terms of the target sensitive data rule match the N characters; In response to the feature terms of the target sensitive data rule matching the N characters, the target sensitive data rule is determined to be a sensitive data rule that matches the unstructured data.

2. The method according to claim 1, characterized in that, Before searching for sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified, the method further includes: For each sensitive data rule in the sensitive data rule base, the parameter values ​​of keywords and sensitive words in the sensitive data rule are augmented to obtain synonyms for those parameter values, and these synonyms are added to the keywords and sensitive words of the sensitive data rule; and / or, For each sensitive data rule in the sensitive data rule base, the parameter value in the feature item of the sensitive data rule is augmented to obtain synonyms and / or similar words of the parameter value, and the synonyms and / or similar words are used to form a dictionary of the feature item.

3. The method according to claim 1, characterized in that, The process of finding sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified includes: Obtain the structured data from the data to be identified; Extract the field names to be identified and the data corresponding to the field names from the structured data; Search the sensitive data rule library for sensitive data rules whose keywords match the field name, and use them as the first sensitive data rule; In response to finding the first sensitive data rule from the sensitive data rule base, the system determines whether the first feature item matches the data corresponding to the field name based on the preset regular expression and / or dictionary of the first feature item, wherein the first feature item is the feature item of the first sensitive data rule; In response to the first feature item matching the data corresponding to the field name, the first sensitive data rule is determined to be a sensitive data rule that matches the structured data.

4. The method according to claim 3, characterized in that, The step of finding sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified also includes: In response to the failure to find the first sensitive data rule in the sensitive data rule base, or the mismatch between the first feature item and the data corresponding to the field name, the data corresponding to the field name is divided into data to be detected and data to be verified. The regular expressions and / or dictionaries of the feature terms of all sensitive data rules in the sensitive data rule base are matched with the data to be detected to determine whether there are any feature terms that match the data to be detected. In response to the existence of a feature term that matches the data to be detected, the feature term that matches the data to be detected is taken as the second feature term; The regular expression and / or dictionary of the second feature term are matched with the verification data to determine whether the second feature term matches the verification data. In response to the second feature matching the verification data, the sensitive data rule to which the second feature belongs is determined to be a sensitive data rule that matches the structured data.

5. The method according to any one of claims 3-4, characterized in that, The step of determining the sensitive information of the data to be identified based on the sensitive data rules in the sensitive data rule base that match the data to be identified includes: In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches the structured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches the structured data are taken as the sensitive classification and sensitivity level of the structured data. In response to the existence of multiple sensitive data rules in the sensitive data rule base that match the structured data, the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match the structured data is taken as the sensitive data category and sensitivity level of the structured data.

6. The method according to claim 1, characterized in that, The process of finding sensitive data rules in the sensitive data rule base corresponding to the target industry that match the data to be identified includes: The features in the unstructured data are identified using a dictionary of all features in the sensitive data rule base and / or a trained named entity recognition model, and are used as the third feature. Search the sensitive data rule base for the sensitive data rule containing the third feature term, and use it as the third sensitive data rule; In response to finding the third sensitive data rule in the sensitive data rule base, a sensitive data rule matching the unstructured data is determined from the third sensitive data rule.

7. The method according to claim 6, characterized in that, The step of determining the sensitive data rules that match the unstructured data from the third sensitive data rules includes: In response to the existence of multiple third sensitive data rules in the sensitive data rule base, semantic classification processing is performed on the unstructured data to obtain the sensitive category to which the unstructured data belongs; among the multiple third sensitive data rules, the sensitive data rule whose sensitive category is consistent with the sensitive category to which the unstructured data belongs is determined to be the sensitive data rule that matches the unstructured data.

8. The method according to any one of claims 1-4, characterized in that, The step of determining the sensitive information of the data to be identified based on the sensitive data rules in the sensitive data rule base that match the data to be identified includes: In response to the fact that there is only one sensitive data rule in the sensitive data rule base that matches the unstructured data, the sensitive classification and sensitivity level contained in the sensitive data rule that matches the unstructured data are taken as the sensitive classification and sensitivity level of the unstructured data. In response to the existence of multiple sensitive data rules in the sensitive data rule base that match the unstructured data, the sensitive classification and sensitivity level contained in the sensitive data rule with the highest sensitivity level among the multiple sensitive data rules that match the unstructured data are taken as the sensitive classification and sensitivity level of the unstructured data.

9. A sensitive data identification device, characterized in that, include: The data acquisition module is used to acquire the data to be identified for the target industry; The rule lookup module is used to find sensitive data rules in the sensitive data rule library corresponding to the target industry that match the data to be identified. The sensitive data rule library includes multiple sensitive data rules mined from the data security specification documents of the target industry using text mining technology. The sensitive information determination module is used to determine the sensitive information of the data to be identified based on the sensitive data rules in the sensitive data rule base that match the data to be identified. The rule base building module is used for: Before searching for sensitive data rules in the sensitive data rule library corresponding to the target industry that match the data to be identified, obtain the data security specification document of the target industry; Text mining techniques are used to extract multiple sensitive data rules from the obtained data security specification file. Each sensitive data rule includes the following parameter values: rule name, sensitivity category, sensitivity level, feature item, sensitive word, and keyword; each feature item specifically includes the following parameter values: feature item name, regular expression for identifying the feature item, dictionary, and checksum. The multiple sensitive data rules are combined to form a sensitive data rule library corresponding to the target industry; The values ​​of the parameter items included in multiple feature items in the sensitive data rule base are used to form the feature item library corresponding to the sensitive data rule base; The rule lookup module is used for: Obtain unstructured data from the data to be identified; The unstructured data is segmented into words to obtain the segmentation results of the unstructured data; The unstructured data after word segmentation is matched with all the sensitive words in the sensitive data rule library to determine the sensitive words in the sensitive data rule library that match the unstructured data. In response to a sensitive word in the sensitive data rule base that matches the unstructured data, the sensitive word that matches the unstructured data is taken as the target sensitive word, and the sensitive data rule to which the target sensitive word belongs is taken as the target sensitive data rule; Record the position of the target sensitive words in the unstructured data; Extract the N characters before and after the specified position from the unstructured data, where N is a positive integer; The regular expression and / or dictionary of the feature terms of the target sensitive data rule are matched with the N characters to determine whether the feature terms of the target sensitive data rule match the N characters; In response to the feature terms of the target sensitive data rule matching the N characters, the target sensitive data rule is determined to be a sensitive data rule that matches the unstructured data.

10. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the sensitive data identification method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the sensitive data identification method as described in any one of claims 1-8.

12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the sensitive data identification method as described in any one of claims 1-8.

Images