Data desensitization method based on semantic analysis, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data desensitization method based on semantic analysis, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring to-be-desensitized data; preprocessing the to-be-desensitized data to obtain text data corresponding to the to-be-desensitized data; according to a pre-stored desensitization recognition algorithm, performing recognition processing on the text data to obtain data needing desensitization in the text data; wherein the pre-stored desensitization recognition algorithm at least comprises a first recognition mode and a second recognition mode, the second recognition mode is obtained by combining a semantic analysis algorithm on the basis of the first recognition mode, and desensitization processing is carried out on data needing desensitization in the text data according to the preset desensitization algorithm to obtain a desensitization result; and according to the data type of the to-be-desensitized data and the desensitization result, the target desensitized data corresponding to the to-be-desensitized data is generated, so that the accuracy of identifying the sensitive data is improved, the range of identifying the sensitive data is expanded, and the workload of manual participation is reduced.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a data desensitization method, electronic device, storage medium and program product based on semantic analysis. Background Art

[0002] Data desensitization refers to the deformation of certain sensitive information through desensitization rules to achieve reliable protection of sensitive privacy data. In this way, desensitized real data sets can be safely used in development, testing and other non-production environments and outsourcing environments. In cases involving customer security data or some commercial sensitive data, real data can be transformed and provided for testing without violating system rules. For example, personal information such as ID card number, mobile phone number, card number, customer number, etc. need to be desensitized.

[0003] The current data desensitization mainly uses pattern matching to identify the input to be desensitized. When performing desensitization, keywords and regular expressions are first defined, and then the system scans the keywords and regular expressions contained in the data to be desensitized. After the corresponding keywords and regular expressions are identified, the configured desensitization algorithm is used to rewrite the data to achieve the purpose of desensitization. However, this pattern matching is essentially a string search technology. Since it does not care about the context of the text, if a word is part of another word, it will cause misjudgment, resulting in incorrect desensitization, or in the actual use of the desensitization system, data with similar meanings will be encountered, and this type of data also contains sensitive information. Pattern matching cannot capture this type of information, resulting in some data not being desensitized. Therefore, how to avoid the problem of incorrect desensitization and missed desensitization and improve the accuracy of data desensitization is an urgent problem to be solved. Summary of the invention

[0004] The purpose of some embodiments of the present application is to provide a data desensitization method, electronic device, storage medium and program product based on semantic analysis. By adopting a semantic analysis method to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, an enhanced pattern recognition method is obtained. Through the enhanced pattern recognition method, data that needs to be desensitized in the desensitized data is identified, that is, the same, similar or similar words can be identified, avoiding the problems of erroneous desensitization and missed desensitization, improving the accuracy of identifying sensitive data, increasing the scope of identifying sensitive data, and reducing the workload of manual participation.

[0005] In a first aspect, some embodiments of the present application provide a data desensitization method based on semantic analysis, including:

[0006] Obtain the data to be desensitized;

[0007] Preprocessing the data to be desensitized to obtain text data corresponding to the data to be desensitized;

[0008] According to a pre-stored desensitization recognition algorithm, the text data is recognized and processed to obtain data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained on the basis of the first recognition method in combination with a semantic analysis algorithm;

[0009] According to a preset desensitization algorithm, desensitize the data that needs to be desensitized in the text data to obtain a desensitization result;

[0010] According to the data type of the data to be desensitized and the desensitization result, target desensitized data corresponding to the data to be desensitized is generated.

[0011] Some embodiments of the present application adopt a semantic analysis method to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, to obtain an enhanced pattern recognition method. Through the enhanced pattern recognition method, data that needs to be desensitized in the desensitized data is identified, thereby improving the accuracy of identifying sensitive data, increasing the scope of identifying sensitive data, and reducing the workload of manual participation.

[0012] Optionally, the second identification method at least includes:

[0013] Acquire source input data, where the source input data at least includes plain text data and non-plain text data;

[0014] Normalizing the source input data to obtain plain text data corresponding to the source input data;

[0015] Using a preset word segmentation method, the plain text data is segmented to obtain a word segmentation result corresponding to the plain text data, wherein the word segmentation result includes a word segmentation and a sentence;

[0016] For each segmentation in the segmentation results, a semantic analysis method is used to determine synonyms or similar words associated with the segmentation, and a semantic description of similar sentences associated with the sentence;

[0017] The word segmentation results, synonyms or similar words associated with the word segmentation, and semantic descriptions of similar sentences associated with the sentence are stored in a corpus.

[0018] The second recognition method is determined according to the corpus.

[0019] Some embodiments of the present application obtain an enhanced pattern recognition method by adopting a semantic analysis method to generate synonyms or similar words corresponding to each word segmentation, as well as semantic descriptions of similar sentences. The word segmentation method based on semantic analysis improves the accuracy of word segmentation and reduces the false alarm rate.

[0020] Optionally, the non-pure text data is database data or word, pdf, ppt type file data in office documents.

[0021] Optionally, the normalizing the source input data to obtain plain text data corresponding to the source input data includes:

[0022] Extract text from office documents and generate plain text files;

[0023] Read the data in the database, separate the column data in each row with identifiers, and generate plain text data.

[0024] In some embodiments of the present application, due to different data sources, there are large differences in data formats, which is not conducive to data recognition, and the plain text format does not need to be processed; the text in the office document is extracted to generate a plain text file; the data in the database is read, and the column data in each row is separated by commas to improve the accuracy and efficiency of data recognition.

[0025] Optionally, the identifying and processing the text data according to a pre-stored desensitization identification algorithm to obtain data in the text data that needs to be desensitized includes:

[0026] Segmenting the text data using a preset segmentation method to obtain a segmentation result corresponding to the text data;

[0027] Using a pre-stored desensitization recognition algorithm, the word segmentation result is compared with the data in the corpus;

[0028] If the segmented word matches the data in the corpus, the segmented word is determined as a segmented word to be desensitized;

[0029] If the semantic description of the sentence matches that of a similar sentence in the corpus, the sentence is determined as a sentence to be desensitized;

[0030] According to the to-be-desensitized word segment and the to-be-desensitized sentence, data in the text data that needs to be desensitized is determined.

[0031] Some embodiments of the present application utilize semantic analysis to obtain synonyms of a configured pattern, and add the synonyms to a new recognition pattern, thereby enhancing the desensitization of words in the same field; utilize semantic analysis to obtain sentence patterns with the same semantics as a configured pattern, and add the semantic description to a new recognition pattern, thereby enhancing the desensitization of similar sentence patterns.

[0032] Optionally, the pre-set desensitization algorithm includes at least one or more of: complete desensitization, partial desensitization, data encryption or data masking.

[0033] Some embodiments of the present application set up multiple desensitization algorithms to desensitize the data that needs to be desensitized, making the desensitization more diverse.

[0034] Optionally, the method further comprises:

[0035] After desensitizing the data to be desensitized, a desensitization report is generated, and the desensitization report at least includes selecting the first identification method for desensitization, or selecting the second identification method for desensitization.

[0036] Some embodiments of the present application classify the recognition modes in desensitization, and the classification includes: desensitization using the original mode and desensitization using the enhanced mode, so as to facilitate the understanding of the causes of desensitization.

[0037] In a second aspect, some embodiments of the present application provide a data desensitization device based on semantic analysis, including:

[0038] The acquisition module is used to obtain the data to be desensitized;

[0039] A preprocessing module, used for preprocessing the data to be desensitized to obtain text data corresponding to the data to be desensitized;

[0040] A recognition module, used to perform recognition processing on the text data according to a pre-stored desensitization recognition algorithm to obtain data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained on the basis of the first recognition method in combination with a semantic analysis algorithm;

[0041] A desensitization module is used to perform desensitization processing on the data that needs to be desensitized in the text data according to a preset desensitization algorithm to obtain a desensitization result;

[0042] A generation module is used to generate target desensitized data corresponding to the data to be desensitized according to the data type of the data to be desensitized and the desensitization result.

[0043] Some embodiments of the present application adopt a semantic analysis method to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, to obtain an enhanced pattern recognition method. Through the enhanced pattern recognition method, data that needs to be desensitized in the desensitized data is identified, thereby improving the accuracy of identifying sensitive data, increasing the scope of identifying sensitive data, and reducing the workload of manual participation.

[0044] Optionally, the second identification method at least includes:

[0045] Acquire source input data, where the source input data at least includes plain text data and non-plain text data;

[0046] Normalizing the source input data to obtain plain text data corresponding to the source input data;

[0047] Using a preset word segmentation method, the plain text data is segmented to obtain a word segmentation result corresponding to the plain text data, wherein the word segmentation result includes a word segmentation and a sentence;

[0048] For each segmentation in the segmentation results, a semantic analysis method is used to determine synonyms or similar words associated with the segmentation, and a semantic description of similar sentences associated with the sentence;

[0049] The word segmentation results, synonyms or similar words associated with the word segmentation, and semantic descriptions of similar sentences associated with the sentence are stored in a corpus.

[0050] The second recognition method is determined according to the corpus.

[0051] Some embodiments of the present application obtain an enhanced pattern recognition method by adopting a semantic analysis method to generate synonyms or similar words corresponding to each word segmentation, as well as semantic descriptions of similar sentences. The word segmentation method based on semantic analysis improves the accuracy of word segmentation and reduces the false alarm rate.

[0052] Optionally, the non-pure text data is database data or word, pdf, ppt type file data in office documents.

[0053] Optionally, the preprocessing module is used to:

[0054] Extract text from office documents and generate plain text files;

[0055] Read the data in the database, separate the column data in each row with identifiers, and generate plain text data.

[0056] In some embodiments of the present application, due to different data sources, there are large differences in data formats, which is not conducive to data recognition, and the plain text format does not need to be processed; the text in the office document is extracted to generate a plain text file; the data in the database is read, and the column data in each row is separated by commas to improve the accuracy and efficiency of data recognition.

[0057] Optionally, the identification module is used to:

[0058] Segmenting the text data using a preset segmentation method to obtain a segmentation result corresponding to the text data;

[0059] Using a pre-stored desensitization recognition algorithm, the word segmentation result is compared with the data in the corpus;

[0060] If the segmented word matches the data in the corpus, the segmented word is determined as a segmented word to be desensitized;

[0061] If the semantic description of the sentence matches that of a similar sentence in the corpus, the sentence is determined as a sentence to be desensitized;

[0062] According to the to-be-desensitized word segment and the to-be-desensitized sentence, data in the text data that needs to be desensitized is determined.

[0063] Some embodiments of the present application utilize semantic analysis to obtain synonyms of a configured pattern, and add the synonyms to a new recognition pattern, thereby enhancing the desensitization of words in the same field; utilize semantic analysis to obtain sentence patterns with the same semantics as a configured pattern, and add the semantic description to a new recognition pattern, thereby enhancing the desensitization of similar sentence patterns.

[0064] Optionally, the pre-set desensitization algorithm includes at least one or more of: complete desensitization, partial desensitization, data encryption or data masking.

[0065] Some embodiments of the present application set up multiple desensitization algorithms to desensitize the data that needs to be desensitized, making the desensitization more diverse.

[0066] Optionally, the generating module is used to:

[0067] After desensitizing the data to be desensitized, a desensitization report is generated, and the desensitization report at least includes selecting the first identification method for desensitization, or selecting the second identification method for desensitization.

[0068] Some embodiments of the present application classify the recognition modes in desensitization, and the classification includes: desensitization using the original mode and desensitization using the enhanced mode, so as to facilitate the understanding of the causes of desensitization.

[0069] In a third aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the data desensitization method based on semantic analysis as described in any embodiment of the first aspect can be implemented.

[0070] In a fourth aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the data desensitization method based on semantic analysis as described in any embodiment of the first aspect.

[0071] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, can implement the data desensitization method based on semantic analysis as described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 A flowchart of a data desensitization method based on semantic analysis provided in an embodiment of the present application;

[0074] Figure 2 A flowchart of another data desensitization method based on semantic analysis provided in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of the structure of a data desensitization device based on semantic analysis provided in an embodiment of the present application;

[0076] Figure 4 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0078] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0079] Data desensitization refers to the deformation of certain sensitive information through desensitization rules to achieve reliable protection of sensitive privacy data. In this way, desensitized real data sets can be safely used in development, testing and other non-production environments and outsourcing environments. In cases involving customer security data or some commercial sensitive data, real data can be transformed and provided for testing without violating system rules. For example, personal information such as ID card number, mobile phone number, card number, customer number, etc. need to be desensitized.

[0080] Current data desensitization mainly uses pattern matching to identify the input to be desensitized. When desensitizing, keywords and regular expressions are first defined, and then the system scans the keywords and regular expressions contained in the data to be desensitized. After the corresponding keywords and regular expressions are identified, the configured desensitization algorithm is used to rewrite the data to achieve the purpose of desensitization. However, this pattern matching is essentially a string search technology. Since it does not care about the context of the text, if a word is part of another word, it will cause misjudgment, resulting in incorrect desensitization, or in the actual use of the desensitization system, data with similar meanings will be encountered, and this type of data also contains sensitive information. Pattern matching cannot capture this type of information, resulting in some data not being desensitized. Therefore, how to avoid the problem of erroneous desensitization and missed desensitization and improve the accuracy of data desensitization is an urgent problem to be solved. In view of this, some embodiments of the present application provide a data desensitization method based on semantic analysis, which includes obtaining the data to be desensitized; preprocessing the data to be desensitized to obtain text data corresponding to the data to be desensitized; according to a pre-stored desensitization recognition algorithm, identifying and processing the text data to obtain the data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is based on the first recognition method and combined with semantic analysis. The method is obtained by using a semantic analysis method; according to a pre-set desensitization algorithm, the data to be desensitized in the text data is desensitized to obtain a desensitization result; according to the data type and desensitization result of the data to be desensitized, target desensitized data corresponding to the data to be desensitized is generated. In the embodiment of the present application, a semantic analysis method is adopted to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, to obtain an enhanced pattern recognition method. Through the enhanced pattern recognition method, the data to be desensitized in the desensitized data is identified, thereby improving the accuracy of identifying sensitive data, improving the scope of identifying sensitive data, and reducing the workload of manual participation.

[0081] like Figure 1 As shown, an embodiment of the present application provides a data desensitization method based on semantic analysis, the method comprising:

[0082] S101, obtaining data to be desensitized;

[0083] The terminal device obtains the data to be desensitized, and the data type of the data to be desensitized can be plain text, database file or office document.

[0084] S102, preprocessing the data to be desensitized to obtain text data corresponding to the data to be desensitized;

[0085] Since the types of data to be desensitized are diverse, the data to be desensitized is preprocessed to obtain text data corresponding to the data to be desensitized.

[0086] S103, according to the pre-stored desensitization recognition algorithm, the text data is recognized and processed to obtain the data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained on the basis of the first recognition method in combination with a semantic analysis algorithm;

[0087] Specifically, a desensitizing recognition algorithm is pre-stored on the terminal device, which is used to identify sensitive words in text data, that is, to identify words that need to be desensitized. The desensitizing recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained by a geometric semantic analysis algorithm based on the first recognition method. Each recognition method includes keywords and regular expressions, which are used to define a sensitive data recognition method.

[0088] After acquiring the text data, the terminal device uses a pre-stored desensitization recognition algorithm to recognize the text data, for example, matching the words or sentences in the text data with the data in the desensitization recognition algorithm, and then identifying the data in the text data that needs to be desensitized, and the data that needs to be desensitized includes at least one or two of the words or sentences.

[0089] S104, performing desensitization processing on the data that needs to be desensitized in the text data according to a pre-set desensitization algorithm to obtain a desensitization result;

[0090] Specifically, a desensitizing algorithm is pre-set on the terminal device. For example, the data to be desensitized is deleted, encrypted, or masked. The desensitizing algorithm is used to desensitize the data to be desensitized identified in the text data to obtain a desensitization result.

[0091] S105. Generate target desensitized data corresponding to the data to be desensitized according to the data type and desensitization result of the data to be desensitized.

[0092] Specifically, the terminal device processes the data type and the desensitization result of the data to be desensitized, and obtains target desensitized data corresponding to the data type of the data to be desensitized.

[0093] Exemplarily, the desensitization result obtained after the desensitization process is text-type data. Therefore, it is necessary to combine the text-type data with the original data type of the data to be desensitized, so as to obtain target desensitized data corresponding to the data type of the data to be desensitized.

[0094] Some embodiments of the present application adopt a semantic analysis method to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, to obtain an enhanced pattern recognition method. Through the enhanced pattern recognition method, data that needs to be desensitized in the desensitized data is identified, thereby improving the accuracy of identifying sensitive data, increasing the scope of identifying sensitive data, and reducing the workload of manual participation.

[0095] Another embodiment of the present application further supplements the data desensitization method based on semantic analysis provided in the above embodiment.

[0096] like Figure 2 As shown, optionally, the second identification method at least includes:

[0097] Acquire source input data, where the source input data at least includes plain text data and non-plain text data;

[0098] Normalizing the source input data to obtain plain text data corresponding to the source input data;

[0099] Using a preset word segmentation method, the plain text data is segmented to obtain a word segmentation result corresponding to the plain text data, wherein the word segmentation result includes words and sentences;

[0100] For each segmentation result, a semantic analysis method is used to determine synonyms or similar words associated with the segmentation, and a semantic description of similar sentences associated with the sentence is determined;

[0101] The word segmentation results, synonyms or similar words associated with the word segmentation, and semantic descriptions of similar sentences associated with the sentence are stored in the corpus.

[0102] According to the corpus, a second recognition method is determined.

[0103] Specifically, a corpus is pre-stored on the terminal device. In order to achieve more accurate word segmentation, terms and expressions in various scenarios are added to the corpus, and the corpus is continuously updated to ensure that professional terms and expressions can be correctly identified. For example, the names of drugs in the medical industry are added to the corpus, and the parameter information of the drug names is adjusted to improve the recognition rate.

[0104] The dedicated corpus is built on the basis of the general corpus to facilitate more accurate word segmentation. According to the usage scenario, first collect the specialized terms and expressions; after the collection is completed, use the general tools to update the corpus to ensure that the professional terms and expressions can be correctly identified. For example, add the names of medicines in the medical industry to the corpus, adjust the parameter information of the medicine names, and improve the recognition rate.

[0105] In this application, open source tools are used to normalize the source input data to obtain word segmentation in the text. During the processing, the semantics of words, phrases and sentences need to be generated for each file separately to ensure that the recognition accuracy of a single file is above 95%.

[0106] The automated enhancement mode includes: as an optional implementation mode, in the embodiment of the present application, a semantic analysis method is used to obtain synonyms of the configured mode, i.e., the first recognition mode, that is, for each participle, a synonym or similar word corresponding to the participle is obtained, and the synonym or similar word is added to the new recognition mode to obtain the second recognition mode, thereby enhancing the desensitization of words in the same field;

[0107] As another optional implementation method, a semantic analysis method is used to obtain sentences with the same semantics as the configured mode, i.e., the first recognition method, and the semantic description of the sentence is added to the new recognition mode, i.e., the second recognition method is obtained, thereby enhancing the desensitization of similar sentences.

[0108] Among them, the method of semantic differential is also called semantic differentiation method, semantic difference method, and bipolar adjective analysis method.

[0109] Some embodiments of the present application obtain an enhanced pattern recognition method by adopting a semantic analysis method to generate synonyms or similar words corresponding to each word segmentation, as well as semantic descriptions of similar sentences. The word segmentation method based on semantic analysis improves the accuracy of word segmentation and reduces the false alarm rate.

[0110] Optionally, the non-plain text data is database data or word, pdf, or ppt type file data in an office document.

[0111] Optionally, the source input data is normalized to obtain plain text data corresponding to the source input data, including:

[0112] Extract text from office documents and generate plain text files;

[0113] Read the data in the database, separate the column data in each row with identifiers, and generate plain text data.

[0114] Specifically, the types of desensitized source data include plain text files, databases, and word, pdf, and ppt type files in office documents.

[0115] In some embodiments of the present application, due to different data sources, there are large differences in data formats, which is not conducive to data recognition, and the plain text format does not need to be processed; the text in the office document is extracted to generate a plain text file; the data in the database is read, and the column data in each row is separated by commas to improve the accuracy and efficiency of data recognition.

[0116] Optionally, the text data is identified and processed according to a pre-stored desensitization identification algorithm to obtain data in the text data that needs to be desensitized, including:

[0117] Segment the text data using a preset segmentation method to obtain a segmentation result corresponding to the text data;

[0118] Use the pre-stored desensitization recognition algorithm to compare the word segmentation results with the data in the corpus;

[0119] If the segmented word matches the data in the corpus, the segmented word is determined as the segmented word to be desensitized;

[0120] If the semantic description of a sentence matches that of similar sentences in the corpus, the sentence is determined to be a sentence to be desensitized;

[0121] According to the to-be-desensitized word segmentations and to-be-desensitized sentences, the data in the text data that needs to be desensitized is determined.

[0122] Specifically, the terminal device will identify the acquired words, phrases and semantics using the enhanced recognition mode, i.e., the second recognition method, and desensitize the identified data according to the configured algorithm; after desensitization, the data will be regenerated into plain text, office documents or database data depending on the input source.

[0123] Some embodiments of the present application utilize semantic analysis to obtain synonyms of a configured pattern, and add the synonyms to a new recognition pattern, thereby enhancing the desensitization of words in the same field; utilize semantic analysis to obtain sentence patterns with the same semantics as a configured pattern, and add the semantic description to a new recognition pattern, thereby enhancing the desensitization of similar sentence patterns.

[0124] Optionally, the pre-set desensitization algorithm includes at least one or more of: complete desensitization, partial desensitization, data encryption or data masking.

[0125] Specifically, the complete desensitization rule: delete or replace all information in sensitive data to make it unrecognizable;

[0126] Partial desensitization rule: only some information in sensitive data is desensitized, and other information is retained, ensuring privacy while retaining the identification and usage functions of the data;

[0127] Data encryption rules: Encrypt sensitive data so that only those who hold the key can decrypt and obtain the original information. The encryption algorithms include but are not limited to symmetric encryption and asymmetric encryption.

[0128] Data masking rules: replace the original information in sensitive data with specific characters or symbols to protect the privacy of the data;

[0129] For example, masking, setting certain characters to special characters *, retaining the format, such as converting the official ID number into a simulated ID number and other types of algorithms.

[0130] Some embodiments of the present application set up multiple desensitization algorithms to desensitize the data that needs to be desensitized, making the desensitization more diverse.

[0131] Optionally, the method further comprises:

[0132] After desensitizing the data to be desensitized, a desensitization report is generated, and the desensitization report at least includes selecting the first identification method for desensitization, or selecting the second identification method for desensitization.

[0133] After desensitization is completed, a desensitization report is generated. The main content of the desensitization report is to classify the identification methods used in desensitization. The classification methods include: desensitization using the original mode and desensitization using the enhanced mode, which is convenient for understanding the reasons for desensitization.

[0134] Some embodiments of the present application classify the recognition patterns in desensitization, such as using original pattern desensitization and using enhanced pattern desensitization, so as to facilitate understanding the causes of desensitization.

[0135] In the embodiment of the present application, sensitive data that may be contained in the pattern scan is compared with the recognized words to prevent missing words in semantic recognition on the one hand, and to correct the sensitive data recognized by the pattern on the other hand, determine part of the data to be desensitized, automatically enhance the recognition pattern based on synonyms, recognize synonyms in the data, and mark them as data to be desensitized, automatically enhance the recognition pattern based on similar semantics, recognize similar semantics in sensitive data, and mark them as data to be desensitized, and improve the accuracy of word segmentation and reduce the false alarm rate through the word segmentation method based on semantic analysis. The automated enhancement mode reduces the workload of manual participation; the automated enhancement mode increases the scope of identifying sensitive data; the automated enhancement mode increases the accuracy of identifying sensitive data; and the accuracy and practicality of the desensitization system are improved.

[0136] The embodiments of the present application are used to solve the problems of erroneous desensitization and missed desensitization caused by pattern recognition. First, the original text data is read from plain text, office documents, and databases; then, basic data such as word segmentation, synonyms, named entities, and semantics are obtained through semantic analysis, and on this basis, combined with the configured pattern recognition, a method for automatically enhancing recognition patterns is provided; pattern recognition is performed on the above basic data, thereby reducing misrecognition of pattern recognition directly from the original text data, and then the recognized data replaces the basic data according to the defined desensitization algorithm; finally, depending on the input source, the data is regenerated into plain text, office documents, or databases.

[0137] It should be noted that each implementable method in this embodiment may be implemented separately, or may be implemented in combination in any combination without conflict, and this application is not limited thereto.

[0138] Another embodiment of the present application provides a data desensitization device based on semantic analysis, which is used to execute the data desensitization method based on semantic analysis provided in the above embodiment.

[0139] like Figure 3 As shown, it is a structural schematic diagram of a data desensitization device based on semantic analysis provided in an embodiment of the present application. The data desensitization device based on semantic analysis includes an acquisition module 301, a preprocessing module 302, an identification module 303, a desensitization module 304 and a generation module 305, wherein:

[0140] The acquisition module 301 is used to obtain the data to be desensitized;

[0141] The preprocessing module 302 is used to preprocess the data to be desensitized to obtain text data corresponding to the data to be desensitized;

[0142] The recognition module 303 is used to perform recognition processing on the text data according to a pre-stored desensitization recognition algorithm to obtain data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained on the basis of the first recognition method in combination with a semantic analysis algorithm;

[0143] The desensitization module 304 is used to perform desensitization processing on the data that needs to be desensitized in the text data according to a preset desensitization algorithm to obtain a desensitization result;

[0144] The generation module 305 is used to generate target desensitized data corresponding to the data to be desensitized according to the data type and desensitization result of the data to be desensitized.

[0145] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0146] Some embodiments of the present application adopt a semantic analysis method to generate synonyms or similar words corresponding to each participle, as well as semantic descriptions of similar sentences, to obtain an enhanced pattern recognition method. Through the enhanced pattern recognition method, data that needs to be desensitized in the desensitized data is identified, thereby improving the accuracy of identifying sensitive data, increasing the scope of identifying sensitive data, and reducing the workload of manual participation.

[0147] Another embodiment of the present application further supplements the data desensitization device based on semantic analysis provided in the above embodiment.

[0148] Optionally, the second identification method at least includes:

[0149] Acquire source input data, where the source input data at least includes plain text data and non-plain text data;

[0150] Normalizing the source input data to obtain plain text data corresponding to the source input data;

[0151] Using a preset word segmentation method, the plain text data is segmented to obtain a word segmentation result corresponding to the plain text data, wherein the word segmentation result includes words and sentences;

[0152] For each segmentation result, a semantic analysis method is used to determine synonyms or similar words associated with the segmentation, and a semantic description of similar sentences associated with the sentence is determined;

[0153] The word segmentation results, synonyms or similar words associated with the word segmentation, and semantic descriptions of similar sentences associated with the sentence are stored in the corpus.

[0154] According to the corpus, a second recognition method is determined.

[0155] Some embodiments of the present application obtain an enhanced pattern recognition method by adopting a semantic analysis method to generate synonyms or similar words corresponding to each word segmentation, as well as semantic descriptions of similar sentences. The word segmentation method based on semantic analysis improves the accuracy of word segmentation and reduces the false alarm rate.

[0156] Optionally, the non-plain text data is database data or word, pdf, or ppt type file data in an office document.

[0157] Optionally, a preprocessing module is used to:

[0158] Extract text from office documents and generate plain text files;

[0159] Read the data in the database, separate the column data in each row with identifiers, and generate plain text data.

[0160] In some embodiments of the present application, due to different data sources, there are large differences in data formats, which is not conducive to data recognition, and the plain text format does not need to be processed; the text in the office document is extracted to generate a plain text file; the data in the database is read, and the column data in each row is separated by commas to improve the accuracy and efficiency of data recognition.

[0161] Optionally, an identification module is used to:

[0162] Segment the text data using a preset segmentation method to obtain a segmentation result corresponding to the text data;

[0163] Use the pre-stored desensitization recognition algorithm to compare the word segmentation results with the data in the corpus;

[0164] If the segmented word matches the data in the corpus, the segmented word is determined as the segmented word to be desensitized;

[0165] If the semantic description of a sentence matches that of similar sentences in the corpus, the sentence is determined to be a sentence to be desensitized;

[0166] According to the to-be-desensitized word segmentations and to-be-desensitized sentences, the data in the text data that needs to be desensitized is determined.

[0167] Some embodiments of the present application utilize semantic analysis to obtain synonyms of a configured pattern, and add the synonyms to a new recognition pattern, thereby enhancing the desensitization of words in the same field; utilize semantic analysis to obtain sentence patterns with the same semantics as a configured pattern, and add the semantic description to a new recognition pattern, thereby enhancing the desensitization of similar sentence patterns.

[0168] Optionally, the pre-set desensitization algorithm includes at least one or more of: complete desensitization, partial desensitization, data encryption or data masking.

[0169] Some embodiments of the present application set up multiple desensitization algorithms to desensitize the data that needs to be desensitized, making the desensitization more diverse.

[0170] Optionally, generate a module for:

[0171] After desensitizing the data to be desensitized, a desensitization report is generated, and the desensitization report at least includes selecting the first identification method for desensitization, or selecting the second identification method for desensitization.

[0172] Some embodiments of the present application classify the recognition modes in desensitization, and the classification includes: desensitization using the original mode and desensitization using the enhanced mode, so as to facilitate the understanding of the causes of desensitization.

[0173] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0174] It should be noted that each implementable method in this embodiment may be implemented separately, or may be implemented in combination in any manner without conflict, and this application is not limited thereto.

[0175] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the operation of the method corresponding to any embodiment of the data desensitization method based on semantic analysis provided in the above embodiments can be implemented.

[0176] An embodiment of the present application also provides a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any embodiment of the data desensitization method based on semantic analysis provided in the above embodiments.

[0177] like Figure 4 As shown, some embodiments of the present application provide an electronic device 400, which includes: a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420, wherein the processor 420 can implement a method of any embodiment of the data desensitization method based on semantic analysis as described above when reading the program from the memory 410 through a bus 430 and executing the program.

[0178] Processor 420 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 420 can be a microprocessor.

[0179] The memory 410 may be used to store instructions executed by the processor 420 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 420 of the disclosed embodiment may be used to execute instructions in the memory 410 to implement the method shown above. The memory 410 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0180] The above are only embodiments of the present application and are not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0181] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0182] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A data desensitization method based on semantic analysis, characterized in that: The method comprises: Obtain the data to be desensitized; Preprocessing the data to be desensitized to obtain text data corresponding to the data to be desensitized; According to a pre-stored desensitization recognition algorithm, the text data is recognized and processed to obtain data that needs to be desensitized in the text data; wherein the pre-stored desensitization recognition algorithm includes at least a first recognition method and a second recognition method, and the second recognition method is obtained on the basis of the first recognition method in combination with a semantic analysis algorithm; According to a preset desensitization algorithm, desensitize the data that needs to be desensitized in the text data to obtain a desensitization result; According to the data type of the data to be desensitized and the desensitization result, target desensitized data corresponding to the data to be desensitized is generated.

2. The data desensitization method based on semantic analysis according to claim 1 is characterized in that: The second identification method at least includes: Acquire source input data, where the source input data at least includes plain text data and non-plain text data; Normalizing the source input data to obtain plain text data corresponding to the source input data; Using a preset word segmentation method, the plain text data is segmented to obtain a word segmentation result corresponding to the plain text data, wherein the word segmentation result includes a word segmentation and a sentence; For each segmentation in the segmentation results, a semantic analysis method is used to determine synonyms or similar words associated with the segmentation, and a semantic description of similar sentences associated with the sentence; storing the word segmentation results, synonyms or similar words associated with the word segmentation, and semantic descriptions of similar sentences associated with the sentence in a corpus; The second recognition method is determined according to the corpus.

3. The data desensitization method based on semantic analysis according to claim 2 is characterized in that: The non-pure text data is database data or word, pdf, ppt type file data in office documents.

4. The data desensitization method based on semantic analysis according to claim 2 is characterized in that: The normalizing the source input data to obtain plain text data corresponding to the source input data includes: Extract text from office documents and generate plain text files; Read the data in the database, separate the column data in each row with identifiers, and generate plain text data.

5. The data desensitization method based on semantic analysis according to claim 2 is characterized in that: The step of performing identification processing on the text data according to a pre-stored desensitization identification algorithm to obtain data in the text data that needs to be desensitized includes: Segmenting the text data using a preset segmentation method to obtain a segmentation result corresponding to the text data; Using a pre-stored desensitization recognition algorithm, the word segmentation result is compared with the data in the corpus; If the segmented word matches the data in the corpus, the segmented word is determined as a segmented word to be desensitized; If the semantic description of the sentence matches that of a similar sentence in the corpus, the sentence is determined as a sentence to be desensitized; According to the to-be-desensitized word segment and the to-be-desensitized sentence, data in the text data that needs to be desensitized is determined.

6. The data desensitization method based on semantic analysis according to claim 1 is characterized in that: The preset desensitization algorithm includes at least one or more of: complete desensitization, partial desensitization, data encryption or data masking.

7. The data desensitization method based on semantic analysis according to claim 1 is characterized in that: The method further comprises: After desensitizing the data to be desensitized, a desensitization report is generated, and the desensitization report at least includes selecting the first identification method for desensitization, or selecting the second identification method for desensitization.

8. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data desensitization method based on semantic analysis as described in any one of claims 1 to 7 can be implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the program is executed by a processor, the data desensitization method based on semantic analysis as described in any one of claims 1 to 7 can be implemented.

10. A computer program product, comprising a computer program, wherein: When the computer program is executed by a processor, it can implement the data desensitization method based on semantic analysis as described in any one of claims 1 to 7.

Images