Intelligent document desensitization tool and method based on deep learning

Abstract

The invention discloses an intelligent document desensitization tool and method based on deep learning. The method comprises the following steps: converting a Word document to be processed into structured data; recording the type, the position and the context environment of the sensitive information through a deep learning model and a natural language processing technology, and analyzing and judging the function and the importance of the recognized sensitive information in a document by utilizing the trained deep learning model; selecting matched alternative words or phrases from the virtual element library to carry out unified replacement on the semantic elements needing to keep the context consistency, so as to ensure the continuity and logicality of the text; comparing the structured data before and after desensitization to ensure that the related sensitive information is correctly processed; and meanwhile, the readability and semantic coherence of the desensitized text are evaluated by utilizing a natural language understanding technology. According to the method, increasingly complicated privacy protection challenges can be effectively handled, and the potential value of data can be fully mined and utilized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an intelligent document desensitization tool and method based on deep learning. Background Art

[0002] In the current context of digital transformation, data has become a core resource for business operations and decision-making. However, with the surge in data volume and the diversification of application scenarios, data security and privacy protection face unprecedented challenges. In particular, sensitive information contained in high-value documents such as corporate contracts and customer records, once leaked, will cause serious legal, economic and reputation losses to individuals and organizations. Traditional desensitization methods, such as simple replacement and shielding, often fail to balance data privacy protection and business value retention. They may undermine the integrity and statistical properties of the data, limiting the possibility of subsequent analysis and utilization. When processing documents that need to maintain semantic coherence, such as contract terms, technical reports, etc., existing technologies often ignore the context, resulting in confusing logic of the desensitized text, affecting reading comprehension and practical application. Summary of the invention

[0003] The purpose of this invention is to provide an intelligent document desensitization tool and method based on deep learning, which can not only effectively cope with the increasingly complex privacy protection challenges, but also fully explore and utilize the potential value of data.

[0004] The technical solution adopted by the present invention is: The intelligent document desensitization method based on deep learning includes the following steps: Step 1: Import the Word document to be processed into the system. The system reads and parses the document and extracts the structured data of the document after parsing. Step 2: Use deep learning models and natural language processing technology to identify structured data and mark sensitive information; Step 3: Select an appropriate desensitization strategy based on the predefined desensitization rule base and the type and context of sensitive information; Step 4: For semantic elements that need to maintain contextual consistency, select matching alternative words or phrases from the virtual element library for unified replacement to ensure the coherence and logic of the text; Step 5: Compare the replaced data and use data dimensionality reduction technology to reduce the complexity of sensitive information to obtain desensitized data; Specifically, data dimension reduction technology is used to reduce the complexity of sensitive information, while special algorithms are used to keep the statistical characteristics and business logic of the data unchanged, ensuring the analytical value of the desensitized data; Step 6: Automatically compare the documents before and after desensitization to confirm that the sensitive information involved has been properly processed and the non-sensitive content has not been affected; Step 7: Use natural language understanding technology to evaluate the readability and semantic coherence of the desensitized text to ensure that no sentences with unclear semantics or logical errors are generated due to the desensitization operation; Step 8: Convert the de-encrypted document back to the original Word format, keeping the same appearance and layout as the input document; Furthermore, step 1 cleans, converts and stores the extracted structural data.

[0005] Furthermore, step 2 records the type, location, and context of the sensitive information.

[0006] Furthermore, the desensitization strategy of step 3 includes shielding and replacement.

[0007] Furthermore, in step 7, a detailed desensitization log is synchronously generated for easy tracking.

[0008] The intelligent document desensitization tool based on deep learning adopts the intelligent document desensitization method based on deep learning, which includes: Data pre-processing unit: import the Word documents to be processed into the system and convert them into structured data; Sensitive information identification and labeling unit: Through deep learning models and natural language processing technology, the type, location and context of sensitive information are recorded. The trained deep learning model is used to further analyze the identified sensitive information to determine the function and importance of the sensitive information in the document. For semantic elements that need to maintain contextual consistency, matching alternative words or phrases are selected from the virtual element library for unified replacement to ensure the coherence and logic of the text. Desensitization verification unit: compares structured data before and after desensitization to ensure that sensitive information involved has been correctly processed; at the same time, natural language understanding technology is used to evaluate the readability and semantic coherence of the desensitized text; Desensitized document export: Provides the function of downloading desensitized Word documents to facilitate user management and use.

[0009] Furthermore, it also includes a log recording unit: providing a detailed desensitized log.

[0010] The present invention adopts the above technical solution and has the following technical features: 1. Dynamically configured desensitization elements. The system allows users to customize desensitization rules, including but not limited to sensitive categories such as personal identity information and contract data. 2. Comprehensive document-level desensitization capabilities, supporting automated desensitization processing of the entire Word document, covering all pages and paragraphs. Whether it is text content or sensitive information in embedded tables and charts, it can be accurately identified and desensitized, ensuring the overall security and compliance of the document. 3. Deep learning-driven intelligent desensitization, introducing a deep learning network. By learning a large number of samples, the system can intelligently distinguish and process sensitive information in complex contexts, improve the accuracy and efficiency of desensitization, and maintain the consistency and readability of the data. 4. Unified management of virtual element libraries, using virtual element libraries to centrally manage and uniformly replace semantic elements that need to maintain contextual consistency. For example, for specific positions, place names or professional terms, predefined virtual aliases are used to ensure consistency in the document and avoid semantic confusion or information distortion caused by desensitization operations.

[0011] The present invention improves the automation level and accuracy of the data desensitization process by combining the intelligent analysis capabilities of the deep learning network and the unified management mechanism of the virtual factor library. It can effectively ensure the statistical characteristics, business logic and contextual consistency of the document while removing sensitive content, thereby retaining the analytical value of the desensitized data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 This is a schematic diagram of the principle architecture of the intelligent document desensitization tool based on deep learning of the present invention. Implementation

[0013] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0014] like Figure 1 As shown, the present invention discloses an intelligent document desensitization method based on deep learning, which includes the following steps: Step 1: Import the Word document to be processed into the system. The system reads and parses the document and extracts the structured data of the document after parsing. Step 2: Use deep learning models and natural language processing technology to identify structured data and mark sensitive information; Step 3: Select an appropriate desensitization strategy based on the predefined desensitization rule base and the type and context of sensitive information; Step 4: For semantic elements that need to maintain contextual consistency, select matching alternative words or phrases from the virtual element library for unified replacement to ensure the coherence and logic of the text; Step 5: Compare the replaced data and use data dimensionality reduction technology to reduce the complexity of sensitive information to obtain desensitized data; Specifically, data dimension reduction technology is used to reduce the complexity of sensitive information, while special algorithms are used to keep the statistical characteristics and business logic of the data unchanged, ensuring the analytical value of the desensitized data; Step 6: Automatically compare the documents before and after desensitization to confirm that the sensitive information involved has been properly processed and the non-sensitive content has not been affected; Step 7: Use natural language understanding technology to evaluate the readability and semantic coherence of the desensitized text to ensure that no sentences with unclear semantics or logical errors are generated due to the desensitization operation; Step 8: Convert the de-encrypted document back to the original Word format, keeping the same appearance and layout as the input document; Furthermore, step 1 cleans, converts and stores the extracted structural data.

[0015] Furthermore, the desensitization strategy of step 3 includes shielding and replacement.

[0016] Furthermore, in step 7, a detailed desensitization log is synchronously generated for easy tracking.

[0017] The intelligent document desensitization tool based on deep learning adopts the intelligent document desensitization method based on deep learning, which includes: Data pre-processing unit: import the Word documents to be processed into the system and convert them into structured data; Sensitive information identification and labeling unit: Through deep learning models and natural language processing technology, the type, location and context of sensitive information are recorded, and the trained deep learning model is used to further analyze the identified sensitive information to determine its function and importance in the document. For semantic elements that need to maintain contextual consistency, the system will select matching alternative words or phrases from the virtual element library for unified replacement to ensure the coherence and logic of the text; Desensitization verification unit: compares structured data before and after desensitization to ensure that sensitive information involved has been correctly processed; at the same time, natural language understanding technology is used to evaluate the readability and semantic coherence of the desensitized text; Desensitized document export: Provides the function of downloading desensitized Word documents to facilitate user management and use.

[0018] Furthermore, it also includes a log recording unit: providing a detailed desensitized log.

[0019] The present invention adopts the above technical solution and has the following technical features: 1. Dynamically configured desensitization elements. The system allows users to customize desensitization rules, including but not limited to sensitive categories such as personal identity information and contract data. 2. Comprehensive document-level desensitization capabilities, supporting automated desensitization processing of the entire Word document, covering all pages and paragraphs. Whether it is text content or sensitive information in embedded tables and charts, it can be accurately identified and desensitized, ensuring the overall security and compliance of the document. 3. Deep learning-driven intelligent desensitization, introducing a deep learning network. By learning a large number of samples, the system can intelligently distinguish and process sensitive information in complex contexts, improve the accuracy and efficiency of desensitization, and maintain the consistency and readability of the data. 4. Unified management of virtual element libraries, using virtual element libraries to centrally manage and uniformly replace semantic elements that need to maintain contextual consistency. For example, for specific positions, place names or professional terms, predefined virtual aliases are used to ensure consistency in the document and avoid semantic confusion or information distortion caused by desensitization operations.

[0020] The present invention improves the automation level and accuracy of the data desensitization process by combining the intelligent analysis capabilities of the deep learning network and the unified management mechanism of the virtual factor library. It can effectively ensure the statistical characteristics, business logic and contextual consistency of the document while removing sensitive content, thereby retaining the analytical value of the desensitized data.

[0021] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. Intelligent document desensitization tool based on deep learning, characterized by: It includes: Data pre-processing unit: import the Word documents to be processed into the system and convert them into structured data; Sensitive information identification and labeling unit: Through deep learning models and natural language processing technology, the type, location and context of sensitive information are recorded, and the trained deep learning model is used to further analyze the identified sensitive information to determine the function and importance of the sensitive information in the document; for semantic elements that need to maintain contextual consistency, matching alternative words or phrases are selected from the virtual element library for unified replacement to ensure the coherence and logic of the text; Desensitization verification unit: compares structured data before and after desensitization to ensure that sensitive information involved has been correctly processed; at the same time, natural language understanding technology is used to evaluate the readability and semantic coherence of the desensitized text; Desensitized document export: Provides the function of downloading desensitized Word documents to facilitate user management and use.

2. The deep learning-based intelligent document desensitization tool according to claim 1, characterized in that: It also includes a logging unit: providing detailed anonymized logs.

3. An intelligent document desensitization method based on deep learning, using an intelligent document desensitization tool based on deep learning according to any one of claims 1 or 2, characterized in that: The method comprises the following steps: Step 1: Import the Word document to be processed into the system. The system reads and parses the document and extracts the structured data of the document after parsing. Step 2: Use deep learning models and natural language processing technology to identify structured data and mark sensitive information; Step 3: Select an appropriate desensitization strategy based on the predefined desensitization rule base and the type and context of sensitive information; Step 4: For semantic elements that need to maintain contextual consistency, select matching alternative words or phrases from the virtual element library for unified replacement to ensure the coherence and logic of the text; Step 5: Compare the replaced data and use data dimensionality reduction technology to reduce the complexity of sensitive information to obtain desensitized data; Step 6: Automatically compare the documents before and after desensitization to confirm that the sensitive information involved has been properly processed and the non-sensitive content has not been affected; Step 7: Use natural language understanding technology to evaluate the readability and semantic coherence of the desensitized text to ensure that no sentences with unclear semantics or logical errors are generated due to the desensitization operation; Step 8: Convert the de-encrypted document back to the original Word format, maintaining the same appearance and layout as the input document.

4. The intelligent document desensitization method based on deep learning according to claim 3 is characterized in that: Step 1: Clean, transform and store the extracted structural data.

5. The intelligent document desensitization method based on deep learning according to claim 3 is characterized in that: The desensitization strategies in step 3 include shielding and replacement.

6. The intelligent document desensitization method based on deep learning according to claim 3 is characterized in that: In step 7, a detailed anonymization log is generated synchronously for easy tracking.

Images