Learning corpus making method and system for text sensitive information protection and computer readable storage medium

Abstract

The invention discloses a learning corpus making method and system for text sensitive information protection and a computer readable storage medium, and relates to the technical field of data security and privacy protection. Through a multi-level and self-adaptive intelligent processing flow, while sensitive information is thoroughly removed, according to different requirements of downstream AI applications, a physical format structure, a logic semantic structure, entity type information and even a field language normal form of an original document are selectively reserved and reconstructed, and a high-quality and high-fidelity desensitization learning corpus is generated. The corpus generated by the method can be safely delivered to a third party for efficient and low-cost model training, so that the training cost and period of the AI model in a specific field are greatly reduced on the premise of ensuring data security. The model covers an information extraction model and a semantic understanding model for solving specific tasks, and also comprises domain enhancement of a large language model for assisting content creation as a technical base thereof.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security and privacy protection, and particularly relates to a learning corpus production method and system for text sensitive information protection and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, especially the wide application of deep learning in the field of natural language processing, the model performance is highly dependent on large-scale and high-quality training corpus. In government, finance, law, medical care and high-end manufacturing industries, a large amount of business documents containing rich knowledge have been accumulated, such as government documents, business contracts, financial reports, research and development documents and medical archives. These documents are valuable data resources for training high-performance and professional industry AI models.

[0003] However, these documents often contain a large amount of sensitive information such as personal privacy, business passwords and even state secrets. At present, when using these data for model training, there is a serious contradiction, and the existing technical solutions have obvious drawbacks: Traditional data desensitization technology. This kind of technology usually uses random replacement, hash disturbance or complete deletion method to process sensitive information. Although this method protects the original data to a certain extent, its processing method is too rough, which seriously damages the semantic integrity, syntactic structure and logical association of the original text. For example, after replacing all the amounts, dates and names in the contract with meaningless characters or uniform labels, the text is "safe", but the internal language pattern and business logic no longer exist. The model trained using such data is difficult to learn the "grammar" and "paradigm" of the specific field documents, resulting in a significant reduction in the accuracy and practicality of the final model, which cannot meet the high requirements of industry applications.

[0004] On-site development mode. In order to avoid data offshoring risks, data holders may require AI service providers to send engineers to carry out "on-site development" in an internal closed environment. Although this mode alleviates the concern of data leakage to a certain extent, it brings new problems such as high labor and time cost, long project cycle and difficulty in scaling. Each project needs to repeat the environment deployment, personnel scheduling and security audit, which is very inefficient and cannot meet the needs of enterprises to quickly iterate and scale AI technology, seriously hindering the popularization and deepening of AI technology in the field of secrecy.

[0005] Dependence on general model or public corpus. Some institutions choose not to use internal data and instead seek training based on general large models or public corpus. However, the model lacks deep understanding of specific industry terminology, document structure and business logic, and its performance is often poor when dealing with professional and format-specific industry documents, and the accuracy is difficult to meet business requirements.

[0006] Therefore, there is an urgent need for innovative technical solutions in this field that can preserve and transmit the structural features and semantic paradigms that are crucial for model training in the original data to the greatest extent possible without disclosing the original de-identified information, thereby truly achieving a high-efficiency unity between data security and model performance and promoting the large-scale deployment of AI technology in sensitive data scenarios. Summary of the Invention

[0007] The purpose of this invention is to address the aforementioned problems by providing a method, system, and computer-readable storage medium for creating learning corpora for protecting sensitive textual information. Through a multi-layered, adaptive intelligent processing flow, sensitive information is thoroughly removed while selectively retaining and reconstructing the physical layout structure, logical semantic structure, entity type information, and even domain-specific language paradigms of the original document according to the different needs of downstream AI applications, generating high-quality, high-fidelity desensitized learning corpora. The corpora generated by this method can be securely delivered to third parties for efficient and low-cost model training, thereby significantly reducing the training cost and cycle of AI models in specific domains while ensuring data security. The models encompass information extraction models and semantic understanding models for solving specific tasks, as well as domain-specific enhancements of large language models used as their technological foundation to assist content creation.

[0008] The technical solution of the present invention is as follows: A method for creating a learning corpus for protecting sensitive textual information includes the following steps: Document Acquisition and Structured Parsing: The original document is processed through a multimodal document understanding model, which identifies and segments the logical regions in the document, and labels each unit with physical layout attributes and logical function categories to form a structured document object model. Semantic Intent and Sensitive Entity Recognition: Perform natural language processing on logical region units to identify their macro-intent categories and locate sensitive information entities within them; Content Reconstruction: Based on the pre-established entity processing strategy library, a replacement operation or a generative rewrite operation is applied to each sensitive information entity, and the content of the logical area unit is reconstructed to generate desensitized text blocks; wherein, the entity processing strategy library defines the mapping relationship between sensitive entity types and processing strategies, and the processing strategies include replacement operation and generative rewrite operation. Corpus generation: Reorganize all the anonymized text blocks according to the original document structure to generate the final anonymized learning corpus.

[0009] Furthermore, in the document acquisition and structured parsing process, the identification, segmentation, and annotation of logical region units are performed in parallel by calling a multimodal large language model.

[0010] Furthermore, the semantic intent and the identification criteria in sensitive entity identification are established or enhanced through one or a combination of the following methods: Built-in general entity recognition model; User-defined regular expressions or keyword dictionaries; An adaptive learning mechanism that fine-tunes the recognition model using user-labeled samples.

[0011] Furthermore, the strategies defined in the entity processing strategy library include probabilistic reconstruction strategies, which configure a probability distribution for at least one sensitive entity type, and the content reconstruction step applies a replacement operation or a generative rewrite operation to the entity decision based on the probability distribution.

[0012] Furthermore, in the content reconstruction, when the decision applies a generative rewrite operation to at least one sensitive information entity, the following sub-steps are performed: Construct and generate instructions that contain at least macro-level intent category information for logical region units; Invoke the generative language model to execute generation instructions to generate generalized text; Replace the original logical region units with generalized text.

[0013] Furthermore, when constructing the generation instructions, relevant knowledge fragments obtained from external knowledge bases through retrieval enhancement technology are also incorporated, and / or text templates directly injected by the user through the instructions.

[0014] Furthermore, after the corpus is generated, it also includes: For the same batch of original documents, multiple versions of de-identified learning corpora are generated by executing processing flows configured with different entity processing strategy libraries; Multiple versions of anonymized learning corpora and their generation configuration metadata are stored together in a corpus for management.

[0015] This application also includes a learning corpus creation system for protecting text-based sensitive information, comprising: The multimodal document parsing module is used to process the original document through the multimodal document understanding model, identify and segment the logical region units in the document, and label each unit with physical layout attributes and logical function categories to form a structured document object model. The semantic recognition module is used to perform natural language processing on logical region units, identify their macro-intention categories, and locate sensitive information entities within them; The content reconstruction module is used to apply replacement or generative rewriting operations to each sensitive information entity based on a pre-established entity processing strategy library, and to reconstruct the content of logical area units to generate desensitized text blocks. The entity processing strategy library defines the mapping relationship between sensitive entity types and processing strategies, and the processing strategies include replacement operations and generative rewriting operations. The corpus generation module is used to reorganize all the anonymized text blocks according to the original document structure to generate the final anonymized learning corpus.

[0016] Furthermore, the core functions of the multimodal document parsing module, semantic recognition module, and content reconstruction module can be implemented by calling the basic large language model.

[0017] This application also includes a computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of a method for creating a learning corpus for protecting text-based sensitive information.

[0018] Compared with existing technologies, the advantages of this invention are: 1. Significantly enhanced document processing scope and structural fidelity: This application adopts multimodal document understanding technology, which can directly process unstructured documents with complex layouts, including scanned documents, significantly expanding the applicability of automated de-identification methods; this technology not only extracts text, but also accurately preserves the physical layout and logical structure of documents, providing downstream models with rich structured features that traditional methods cannot provide. 2. High level of intelligence and adaptability, significantly reducing labor costs: This application introduces an adaptive semantic recognition and user sample learning mechanism to replace the heavy and rigid manual rule configuration in traditional desensitization solutions; the system can automatically understand the semantic intent of different industries and continuously learn new patterns, achieving a high degree of adaptability to diverse business scenarios, which greatly reduces the labor costs of system deployment and maintenance. 3. Corpus generation and performance improvement for training objectives: This application can generate corpus that is highly matched with the training objectives of downstream models, thereby improving model performance; it can provide "semantic corpus" that retains the complete semantic paradigm for language understanding tasks; more importantly, it can also fuse the features of both to generate "hybrid corpus" that is rich in both structural and semantic information, providing accurate and efficient data support for the training of advanced models such as multi-task learning, and maximizing the training value of the original data. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the method described in this application.

[0020] Figure 2 This is a schematic diagram of the functional modules of the system in this application.

[0021] Figure 3 This is a flowchart illustrating one embodiment.

[0022] Figure 4 This is a flowchart illustrating another embodiment.

[0023] Figure 5 This is a schematic diagram of the internal logic flow of another embodiment.

[0024] Figure 6 This is a schematic diagram illustrating the effect of data conversion on an exemplary text in the final embodiment. Detailed Implementation

[0025] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0027] Please see Figures 1-6 A method for creating learning corpora for protecting sensitive textual information, such as Figure 1 As shown, it includes the following steps: S1: Document Acquisition and Multimodal Structured Parsing: This step is performed by the multimodal document parsing module (210) and includes the following sub-steps: S1.1: Batch acquisition of original document data through file interface or optical character recognition (OCR) technology, the data including text content and layout information (such as coordinates, font, size). S1.2: The module analyzes each document in the dataset by calling its internally integrated multimodal document understanding model (e.g., a pre-trained model that integrates text, images and layout information), identifies and segments the logical region units in the document in parallel, and labels each unit with its physical layout attributes and logical function category (such as title, body text, list, table, header and footer, etc.), forming a structured document object model.

[0028] S2: Two-tiered identification of semantic intent and sensitive entity types: This step is performed by the semantic recognition module (220). The core functionality of this module is built upon configurable recognition criteria and a two-tiered execution process.

[0029] S2.1: Establishing and Configuring Criteria for Sensitive Entity Identification. Before performing the identification task, the module establishes and enhances the criteria for sensitive entity identification through a hybrid mechanism that integrates general knowledge and user customization. This mechanism can combine one or more of the following methods: a) Built-in general entity recognition capability: The pre-trained model built into the module has basic recognition capabilities for general entity types such as names and amounts.

[0030] b) User-defined rules and thesaurus: The system can provide a configuration interface that allows users to define domain-specific and explicit sensitive entities based on the characteristics of their own business domain, using regular expressions or keyword dictionaries (such as internal project names, core technology codes, etc.).

[0031] c) Model adaptive learning capability: As an advanced customization method, the system can provide model fine-tuning function, allowing users to enhance the module's adaptive learning capability for new and complex sensitive entities by labeling a small number of samples online.

[0032] S2.2: Execute a two-level semantic recognition process. After the recognition criteria are configured, when a text block is received, the semantic recognition module (220) will use the configured capabilities to execute a two-level analysis process from macro to micro: 1) Macro-level intent classification. The module first analyzes the overall content of the text block through its internal semantic classification capabilities, and identifies and labels its macro-level intent (such as "contract definition clauses" and "risk warning section").

[0033] 2) Micro-entity localization. The module comprehensively utilizes built-in capabilities, custom rules, custom dictionaries, and adaptively learned capabilities to scan the text block to comprehensively locate and label all specific sensitive information entities (such as "Party A Company" [organization name], "trade secret" [proper term]).

[0034] S3: Fine-grained, adaptive content reconstruction based on a configurable strategy library: This step is performed by the content refactoring module (230). The core functionality of this module is built upon a configurable strategy library and generated resources, as well as a fine-grained execution flow.

[0035] S3.1: Establish and configure the resources required for content reconstruction: Before performing the refactoring task, the capabilities and resources of the content refactoring module (230) can be configured and enhanced in one or more of the following ways: a) Establish an entity processing strategy library. The system provides a configuration interface that allows users to create an "entity processing strategy library" containing various basic strategies. Users can specify the processing method for each sensitive entity type. The strategy library may include one or more of the following: Basic Strategy 1: Structured Placeholder Replacement. Its core operation is to retain all non-sensitive text within a text block except for sensitive entities, and replace the sensitive entity with its corresponding type placeholder (e.g., replacing "1 million yuan" with "[amount]"). This strategy aims to generate a "structured extraction corpus" that preserves the complete syntactic structure.

[0036] Basic Strategy Two: Incorporating Generative Rewriting. Its core operation is to use the information of the sensitive entity as context, and then generalize it as a whole using a generative language model in a natural language generation manner. This strategy aims to generate a "semantic generative corpus" that is linguistically natural and highly faithful in its statistical features.

[0037] Basic Strategy 3: Probabilistic Reconstruction. This allows users to specify a probability distribution for a given entity type, such as "40% probability of applying Basic Strategy 1, 60% probability of applying Basic Strategy 2." The system will then randomly select based on this probability to generate diverse training corpora to enhance model robustness.

[0038] Basic Strategy Four: Default Strategy. The system has a built-in default processing strategy (for example, using Basic Strategy One by default) to handle any sensitive entity types that have not been explicitly configured by the user, ensuring the robustness and ease of use of the system.

[0039] b) Prepare a generative language model. For tasks requiring generative rewriting (basic strategy two), the language model used can be prepared. For example, a general large model can be adapted to a more specialized "legal expert base model" through domain-specific fine-tuning.

[0040] c) Configuring Generated Knowledge Sources: To further improve the quality of generated content, the system allows users to configure additional knowledge sources. This can include injecting templates at the instruction level in a single task, or pre-building an external domain knowledge base for use in Retrieval Enhanced Generation (RAG).

[0041] S3.2: Execute a fine-grained differential reconstruction process. After the required resources are configured, when the labeled text block output by S2 is received, the content reconstruction module (230) will independently execute a "query-decision-application" reconstruction process for each sensitive entity within the text block: 1) Query and Decision. The module iterates through all sensitive entities within the text block and, based on the strategy library established in S3.1, independently determines the basic strategy to be applied to each entity. Regardless of whether the user-configured strategy is specifically specified, probabilistically refactored, or adopted by the system using the default strategy, for a single processing of a single entity, the final decision result will converge to one of the following two basic operations: "Basic Strategy One" (replacement) or "Basic Strategy Two" (rewrite).

[0042] 2) Application and Execution. After determining the respective operation strategies for all entities within the text block, the specified strategy operations are performed on each entity.

[0043] a) If the decision outcome is basic strategy one, then perform placeholder replacement on the entity.

[0044] b) If the decision outcome is basic strategy two, the system initiates the generative rewrite process. This process includes: ① Constructing a fusion instruction: The system constructs a generation instruction based on information such as the macro-intention and length of the text block.

[0045] ②Integration of multi-source knowledge: During generation, the model's built-in knowledge, user-injected instruction-level knowledge via prompts, or professional knowledge retrieved from external knowledge bases via RAG can be flexibly integrated.

[0046] ③ Model generation: Invoke the language model to execute instructions and generate a replacement text that does not contain the original sensitive entities but is guaranteed in terms of professionalism and context.

[0047] ④ Replacement and Output: Replace the original text block with the generated text.

[0048] S4: Corpus Reorganization and Corpus Construction: This step can be performed by the corpus generation module (240). This step is responsible for recombining all logical region units (text blocks) processed by S3 into a structurally complete document-level corpus and managing it systematically.

[0049] S4.1: Parallel Reassembly and Generation. The corpus generation module (240) receives the complete set of text blocks that have undergone replacement processing. This module can perform parallel reassembly of text blocks from different documents based on the original document identifier and structural information associated with each text block. Finally, one or more complete de-identified corpus files are generated in batches.

[0050] S4.2 (Optional): Multi-version and Metadata Management. Due to the high configurability of step S3 of this invention (e.g., users can change entity processing strategies, enable or disable probabilistic reconstruction), the system allows for the generation of multiple versions of anonymized corpora (e.g., a purely structured version, a purely semantic version, and a hybrid version) from the same batch of original documents by executing processing flows with different configurations multiple times. The system can store these different versions of corpora, along with their generation configurations (such as the strategies used, timestamps, etc.), as metadata in a centralized corpus. In this way, a multi-layered, multi-purpose, version-traceable secure corpus asset library is constructed, providing enterprises with unprecedented data management flexibility.

[0051] This application also provides a system for creating learning corpora for protecting text-based sensitive information, such as... Figure 2 As shown, this system can be represented as an AI core engine (200), whose functional module structure may include: Multimodal document parsing module (210): Responsible for executing step S1 in the method. This module can use a multimodal AI model to parse structured logical region units (text blocks) from batches of input raw documents.

[0052] Semantic recognition module (220): Responsible for performing the core recognition task in step S2 of the method. This module can use natural language processing models to perform macro-intention classification and micro-sensitive entity localization on text blocks.

[0053] Content Restructuring Module (230): Responsible for executing step S3 in the method. This module is the core scheduling engine of the system, which can perform fine-grained and differentiated content reconstruction for each sensitive entity in the text block according to the "entity processing strategy library" pre-built by the user.

[0054] Corpus generation module (240): Responsible for executing step S4 in the method. This module can reassemble all the reconstructed text blocks in parallel and generate the final de-identified corpus files in batches.

[0055] Model Adaptive Learning Module (250): Responsible for implementing the "Model Adaptive Learning" function described in step S2. As an optional enhancement module, it can receive user-provided labeled data, fine-tune the semantic recognition module (220), and enhance its capabilities through update streams.

[0056] Regarding the internal implementation architecture of this AI core engine (200), various flexible approaches can be adopted. The core principle lies in the logical decoupling of functional modules, rather than specific physical deployment. For the core AI-intensive modules (i.e., 210, 220, 230, 250) in this engine, typical implementation methods include, but are not limited to, the following two: Unified Large Model Architecture: In this highly integrated implementation, multiple core AI functions of the system can be implemented by calling the same powerful underlying large platform model (e.g., a multimodal large language model). a) Document parsing (210): This is achieved by leveraging the multimodal understanding capabilities of the large model.

[0057] b) Semantic recognition (220): This is accomplished by providing the large model with specific cue words for entity extraction and classification.

[0058] c) Content Restructuring (230): This is accomplished by providing the large model with instructions (prompt words) for text generation.

[0059] d) Adaptive learning (250): Enhance the capabilities of this unified large model on specific tasks by fine-tuning it itself.

[0060] Collaborative Multi-Model Architecture: In this more targeted implementation, different functional modules of the system can be handled by specialized models optimal for their respective domains, forming an efficient "expert system". a) Document parsing (210): A specialized multimodal layout analysis model (such as LayoutLM or DiT model) can be used.

[0061] b) Semantic recognition (220): A small to medium-sized model optimized for natural language understanding tasks (such as NER and classification models of the BERT series) can be used.

[0062] c) Content Restructuring (230): A dedicated generative model (such as T5 or a lightweight GPT model) can be used.

[0063] d) Adaptive learning (250): Its fine-tuning algorithm will be applied to one or more of the above specialized models respectively, for example, incremental training is performed only on the NER model in the semantic recognition module (220).

[0064] The core of this application lies in the logical division of the above-mentioned functional modules (210 to 250) and their collaborative work process, rather than being limited to adopting any specific model deployment architecture.

[0065] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps described above.

[0066] In one embodiment, the creation of a "structured extraction corpus" for a contract key information extraction model is as follows: This embodiment aims to illustrate a typical application scenario of the present invention in the field of financial contract processing, namely, the creation of a "structured extraction corpus" for training an information extraction model. Specifically, this embodiment will demonstrate how to use thousands of "Personal Consumer Loan Contract" documents to achieve the above objective.

[0067] 1) Preparation Phase: Establishing and Enhancing Financial Sector Identification Capabilities Before formally processing the "Personal Consumer Loan Contract," the administrator needs to configure the system for the financial sector to ensure that the system can accurately identify all unique sensitive data points in the business. This process is the foundation for all subsequent anonymization strategies. Typical configurations for consumer loan contracts may include: a) Define sensitive information through rules and dictionaries: Use the configuration interface provided by the system to define highly structured sensitive information within the organization.

[0068] Define the contract number format: A unique number format for the bank's "Personal Consumer Loan Contracts" is defined using regular expressions, such as XFD-CL-

[2024] -\d{6}. This ensures that all internal contract numbers can be accurately captured.

[0069] Define the format of personally identifiable information: Add a regular expression to match customer ID numbers (18 digits, or 17 digits plus X), which is one of the highest priority sensitive information in the financial field.

[0070] Importing the list of partner institutions: Using the keyword dictionary function, hundreds of "partner car dealers" and "designated insurance companies" that have business dealings with the bank were imported in batches and categorized into a new custom entity type [partner institutions].

[0071] (b) (Optional) Identification of unstructured risk clauses through sample learning: For sensitive information with complex patterns, such as “description of special collateral”, the system can provide a model adaptive learning function. This function allows users to enhance the semantic recognition module (220)’s ability to identify such ambiguous concepts by annotating online in sample documents and triggering model fine-tuning.

[0072] 2) When generating structured corpora, the system initiates and automatically executes the following processing flow, such as... Figure 3 As shown.

[0073] S301: Task Startup and Configuration: Once the system is ready, the user initiates a corpus creation task and configures it through a task configuration interface. Typical configuration items include: a) Input data source specification. Allows users to specify one or more locations containing the documents to be processed (e.g., the Personal Consumer Loan Contract), such as a local folder or a server address.

[0074] b) Confirm the target corpus: Select the "Information Extraction Task (Strategy 1)" mode.

[0075] S302: Document Acquisition and Structured Parsing (corresponding technical solution S1): The data acquisition and parsing module (210) reads thousands of scanned copies of "Personal Consumer Loan Contracts" in batches based on the input source specified by the user. This module processes all documents in parallel and parses their content into a unified set of structured text blocks. For each contract, the module utilizes its multimodal understanding capabilities to not only extract text through OCR but also analyze the document's layout, accurately parsing unstructured page content into a series of structured text blocks, such as "Party A Information Block," "Party B Information Block," and "Loan Basic Information Clause Block," etc.

[0076] S303: Semantic Intent Recognition and Sensitive Entity Localization The semantic recognition module (220) receives the complete set of text blocks output from the previous step. To achieve efficient processing, this module can divide these text blocks into multiple batches and then feed them into its internal AI model for parallel inference all at once. After the model has finished processing, it returns the recognition and annotation results for all text blocks.

[0077] Taking the processing of one of the text blocks parsed from the "Personal Consumption Loan Contract" as an example, let's assume that its original content is: "Article 1 Basic Loan Information. The total loan amount under this contract is RMB 500,000.00 (￥500,000.00), the loan term is thirty-six months, and it will be paid by the Beijing Branch of the Industrial and Commercial Bank of China to Zhang San's personal account from January 1, 2024." This module leverages its enhanced capabilities through the "preparation phase" to perform two-level identification: S303.1: Macro-level Intent Identification: Label the overall intent of this text block as "Core Loan Terms". This label helps with chapter-level understanding and navigation of the contract.

[0078] S303.2: Micro-entity Location: Inside the text, accurately locate and mark multiple sensitive entities such as "Five hundred thousand yuan in full" (type: capital amount), "￥500,000.00" (type: numerical amount), "Thirty-six months" (type: term), etc.

[0079] S304: Perform entity-level structured replacement: The content reconstruction module (230) receives all the marked results output in step S303 and can perform text replacement operations in batches, retaining all non-sensitive "skeleton words", and only replacing all the sensitive entities identified in step S303 with their corresponding type placeholders. Taking the sample in S303 above as an example, after being processed by this step, its content can become: "Article 1 Basic Information of the Loan. The total amount of the loan under this contract is RMB [capital amount] ([numerical amount]), the loan term is [term], starting from [date], and will be paid by [institution name] to the personal account of [person name]." S305: Corpus Generation and Storage: The corpus generation module (240) receives all the text blocks that have been replaced, reorganizes them in parallel according to their original document structures, and finally generates thousands of complete "structured extraction corpus" files and stores them in the corpus library. Finally, the system outputs the same number of complete "structured extraction corpus" files as the input, which do not contain any specific sensitive information but retain the complete syntactic structure.

[0080] These generated corpora, because they completely retain the syntactic structure and entity type context of the original contract text, can be used as ideal training data for efficiently training and optimizing information extraction models (such as named entity recognition models). The purpose is to enable downstream models to accurately learn the context patterns of various key information (such as amount, date, contract number, etc.) in financial contracts. Such a specialized model trained in this way can provide core AI capability support for achieving high-precision and automated information extraction functions when applied to real business processes (such as automated contract review, key information verification, etc.) later.

[0081] In another embodiment, the production of "semantic generation corpus" for pre-training large language models in the legal field: This embodiment aims to illustrate a typical application scenario of the present invention in the legal field, that is, to produce "semantic generation corpus" for domain enhancement of large language models. Specifically, this embodiment will demonstrate how to process thousands of "Technology R & D Cooperation Agreements".

[0082] 1) Preparation Phase 1: Establish the basis for identifying sensitive information in the legal field Before processing documents, the general counsel and intellectual property expert (hereinafter referred to as "legal expert") of a high-tech company configured the system in a domain-specific manner to ensure accurate identification of unique and sensitive information in technology contracts. Typical configurations may include: a) Rule definition: Define the format of the internal "project security level and code" (such as [TOP-SECRET]-PROJ-ALPHA-\d{3}) using regular expressions.

[0083] b) Dictionary definition: Batch import keyword dictionaries containing "core technology patent numbers", "list of competitor companies" and "list of key R&D personnel".

[0084] 2) Preparation Phase Two: Configuring the Knowledge Sources for Generated Content: a) Basic Model Preparation. For professional scenarios that prioritize the highest quality, AI engineers can fine-tune a general-purpose basic language model using anonymized or publicly available legal documents within the company. This step evolves the model from a "generalist" to a "legal expert basic model" that better understands legal terminology and writing style, providing higher-quality underlying capabilities for subsequent generation tasks.

[0085] b) Generate knowledge source configuration: Employs RAG professional-grade enhancements: To ensure the legal professionalism of the generated corpus, legal experts further utilized the system's knowledge base function to construct a RAG knowledge source called the "Corporate Legal Professional Knowledge Base." This knowledge base can include publicly available texts of the Patent Law, standard corporate contract templates, and non-sensitive legal opinions provided by external law firms.

[0086] ① Preparation Phase: Building a Legal Knowledge Base (RAG). Utilizing the system's "Knowledge Base Management" module, the company's accumulated documents over the years, including publicly available or anonymized contract templates, legal training materials, and professional opinions from external lawyers, are constructed into a systematic "Corporate Legal Knowledge Base." The system automatically slices, vectorizes, and indexes these documents.

[0087] ② Execution phase: When starting the batch processing task, the "Corporate Legal Professional Knowledge Base" was selected and specified as the RAG knowledge source for this task in the configuration options.

[0088] ③ Generation process: For each clause of each contract, before generating the rewritten content, the system can first retrieve 3-5 of the most relevant knowledge fragments from the knowledge base, and then use instructions to ask the large language model to generate content based on these retrieved, highly reliable internal knowledge.

[0089] With RAG professional-grade enhancement, the corpora produced by the system in batches have reached the highest standards in terms of professionalism, accuracy and internal terminology consistency, ensuring the highest quality of downstream model training.

[0090] 3) When generating semantic corpus, the system initiates and automatically executes the following processing flow (refer to...). Figure 4 ): S401: Task Startup and Configuration: To conduct quarterly iterations of the large-scale legal model, the company's AI algorithm team initiated a task to mass-produce corpora and made the following settings: a) Input data source: Specifies the folder path where thousands of "Technology Research and Development Cooperation Agreements" are stored.

[0091] b) Confirm the corpus target: Select the "Understanding and Generation Task (Strategy 2)" mode.

[0092] c) Knowledge Enhancement Configuration: In the "Knowledge Source" configuration, "Corporate Legal Professional Knowledge Base" was selected and specified as the RAG knowledge source for this task.

[0093] S402: Document Acquisition and Structured Parsing (corresponding technical solution S1): The data acquisition and parsing module (210) reads thousands of “Technology Research and Development Cooperation Agreement” documents in batches and parses each document into a structured set of “clauses” text blocks in parallel.

[0094] S403: Semantic recognition and entity localization (corresponding technical solution S2): The semantic recognition module (corresponding to S2) (S303) analyzes one of the clauses regarding "ownership of intellectual property rights". The original text is: "During the term of this agreement, the patent application right for the commercially valuable technological achievement A123 jointly developed by both parties belongs to Party A. Party B has the priority right to use the technological achievement, and the specific licensing fee shall be negotiated separately by both parties." The semantic recognition module labeled the intent category of this clause as "Intellectual Property Ownership - Joint Research and Development". At the same time, the NER model identified "Technological Achievement A123", "Party A Company", and "Party B Company" as sensitive entities.

[0095] S404: Batch execution of knowledge base retrieval (corresponding to the multi-source knowledge fusion part of technical solution S3): Based on the "knowledge enhancement" mode enabled by the user in the task configuration, the content reconstruction module (230) first performs a batch knowledge retrieval step for all clauses that have detected sensitive entities.

[0096] Taking the aforementioned "Intellectual Property Ownership" clause as an example, the module will, based on its macro-intention of "jointly developing intellectual property ownership clauses," retrieve the most relevant legal provisions or contract templates from the user-specified "corporate legal expertise base" as contextual references.

[0097] S405: Batch execution of generative semantic rewriting (corresponding technical solution S3): The content reconstruction module (230) receives all the annotated clause text blocks. For clauses that are detected to contain sensitive entities, the module will initiate a generative rewriting process in batches and in parallel.

[0098] Taking the aforementioned "ownership of intellectual property" clause as an example, the module execution process is as follows: ① Enhanced Search: First, based on the macro intent of the clause, "Joint R&D Intellectual Property Ownership Clause", the most relevant legal provisions or contract templates are retrieved from the "Corporate Legal Professional Knowledge Base".

[0099] ② Build the generation instruction: Then build the Prompt, which requires the language model to refer to the content retrieved from RAG and generate a professional, general text without any specific project or company information around the theme of "jointly developed intellectual property".

[0100] ③ Model Invocation and Generation: The system invokes the language model (whether a general model or a finely tuned expert model) that has been adapted during the preparation phase, executes the generation instruction, and generates a high-quality, synthetic text free of sensitive information. For example, for a "property ownership" clause, the following content might be generated: "Any technologically innovative results generated by the cooperation between the two parties under this agreement... shall be specified in a supplementary agreement signed by both parties according to their respective contributions."

[0101] ④ Replace the original text: Use the generated composite text to completely replace the original text content.

[0102] S406: Batch corpus generation and storage (corresponding technical solution S4): The system replaces the original text with this generated text, which contains no sensitive information but has a strong legal flavor, and reassembles it with other processed clauses to generate a "semantic generation corpus." Through the above process, the system can batch convert thousands of original "Technology Research and Development Cooperation Agreements" into the same amount of high-quality and secure "semantic generation corpus."

[0103] Because these generated corpora retain the professional writing style and language paradigm of the legal field and contain no sensitive information, they can be directly and safely used to perform domain-specific enhancement training on a general-purpose foundational model. The aim is to adapt and optimize the general-purpose model into a "legal domain expert model" that is better at understanding and processing professional legal texts. This will provide a higher-performance, more domain-knowledge-rich foundational model to support the subsequent deployment of advanced AI applications such as text classification, risk identification, or intelligent question answering within enterprise intranets.

[0104] In another embodiment, hybrid corpus generation for advanced risk modeling: This embodiment aims to illustrate a typical application scenario of the present invention in the field of fintech. Quantitative risk analysts need to create a "hybrid corpus" to train a next-generation credit risk assessment model. This model needs to learn both "information extraction" and "deep semantic understanding" capabilities simultaneously, thereby achieving accurate extraction of key quantitative indicators (such as amount, interest rate, and term) from contracts and a deep understanding of the natural language context describing unstructured risks in the text.

[0105] 1) Preparation Phase One: Establishing a Hybrid Processing Strategy: Analysts utilize the system's "Entity Processing Strategy Library" configuration feature to create a hybrid processing strategy called "Advanced Risk Modeling Dedicated," which incorporates probabilistic selection. This strategy aims to generate training corpora with richer features and enhanced model generalization capabilities. A typical configuration is as follows: a) Mandatory structuring: For the most sensitive data such as Personal Identification Information (PII) and core quantitative indicators, the "Basic Strategy 1" (structured placeholder replacement) is mandatory to achieve the maximum degree of information desensitization.

[0106] Configuration examples: [Name], [ID Number], [Amount], [Date], [Interest Rate], etc.

[0107] b) Forced semanticization: For unstructured concepts that are mainly used to enrich contextual understanding and whose replacement with placeholders would seriously damage the readability of the text, “Basic Strategy 2” (integrating generative rewriting) can be adopted.

[0108] Configuration example: [Non-standard risk clauses], [technical assessment opinions], etc., learned from samples.

[0109] c) Probabilistic Hybridization: For entities that need to be located and can be semantically generalized to a certain extent, “Basic Strategy Three” (probabilistic reconstruction) can be used to balance the naturalness of structured information and context.

[0110] Configuration example: For the entity types [Organization Name] and [Partner], set "70% probability to apply Strategy 1, 30% probability to apply Strategy 2".

[0111] The purpose of probabilistic mixture is to allow the downstream model to learn to accurately locate the concept of "organization" most of the time, while also having the opportunity to learn to understand the natural language expression of sentences containing "organization", thereby enhancing the model's overall capabilities.

[0112] 2) Preparation Phase Two: Configuring the Knowledge Sources for Generated Content: a) Preparation of the underlying financial technology model: Similar to the preparation of the underlying model in another embodiment.

[0113] b) Knowledge source configuration: Employing instruction-level knowledge injection. A junior legal professional needed to quickly work on a confidentiality clause in an agreement. He didn't have a readily available knowledge base, but he found a standard, publicly available template of a confidentiality clause from a company.

[0114] ①Preparation phase: No system-level configuration required.

[0115] ② Execution Phase: In the content reconstruction module, he chose the "generative semantic rewriting" strategy. When constructing the prompt words, he not only entered the length, category, and other information of the original clause, but also pasted the publicly available confidentiality clause template into the "style reference" field of the prompt words.

[0116] ③ Generation process: The system sends instructions to the large language model, explicitly requiring that the content be rewritten by strictly imitating the legal wording and sentence structure of the following template. Based on this, the model generates a desensitized text with a highly consistent style and rigorous wording.

[0117] This instruction-level knowledge injection method enables rapid and accurate quality control of a single task without prior investment.

[0118] 3) Task startup and configuration: The analyst initiated a corpus-building task targeting "Personal Consumer Loan Contracts" and other types of credit agreements, with the following settings: a) Input data source: Specify a document library containing various credit agreements.

[0119] b) Output Corpus Target: In the "Corpus Generation Mode", the "Advanced Risk Modeling Dedicated" hybrid processing strategy that he just configured was explicitly selected.

[0120] c) Knowledge Enhancement Configuration: Because the contracts processed in this task involved numerous new clauses related to "supply chain finance," and the company's existing RAG knowledge base lacked sufficient coverage for such emerging businesses, the analyst pasted a non-sensitive standard description template of "supply chain finance" risk, recently written by the risk control department, into the "Instruction-Level Knowledge Injection" text box in the task configuration.

[0121] After receiving the user's task configuration, the system will execute an automated processing flow similar to that of the aforementioned embodiments, including basic steps such as document acquisition and parsing (S1) and semantic recognition and entity localization (S2). To avoid unnecessary repetition and to clearly demonstrate the unique features of this embodiment, the following will focus on explaining the execution details of the most crucial step, S3 (strategy-based content reconstruction).

[0122] 4) Detailed internal logic flow (refer to...) Figure 5 As shown, taking a single text block as an example): Taking a text block identified by S2 as a [non-standard risk clause] and whose content is related to "supply chain finance" as an example, the content reconstruction module (230) will execute the following process that integrates entity-level differential processing and instruction-level knowledge injection when processing the text block: S501: Making strategic decisions: The module first iterates through all sensitive entities within the text block (e.g., it may contain [amount], [organization name], etc.). Based on the "Advanced Risk Modeling Dedicated" strategy library, it decides whether to use a "replace" or "rewrite" strategy for each entity.

[0123] The module then processes the entities in parallel: S502: Path A: Replacement Processing: For entities that are decided to be "replaced", the system performs the processing of path A: collect the "replacement" entities (S502a) and generate a "original text-placeholder" mapping list for them (S502b).

[0124] S503: Path B: Rewrite Processing: For entities and their contexts that are determined to be "rewritten," the system executes path B, an enhanced generation process that incorporates real-time user commands: Collect “rewritten” entities (S503a): The module collects the entities that need to be generalized and the complete sentences or paragraphs in which they are located as the base context.

[0125] Constructing "Enhanced" Generation Instructions (S503b): When constructing prompt instructions, the module performs a crucial knowledge injection step. It integrates the supply chain finance risk description template provided by the user during the "Task Startup and Configuration" phase as a powerful style and content reference into the instruction.

[0126] Model generation (S503c): Invokes the language model, executes this complex instruction that has been "knowledge-enhanced", and generates a "generalized base text" that does not contain original sensitive entities but is guaranteed in terms of professionalism and context.

[0127] S504: Results Integration Receive the "original text-placeholder" mapping list produced by path A and the "generalized base text" produced by path B, replace the sensitive entities in the original text, and generate a "hybrid" text block that contains both natural language generalized content and structured placeholders.

[0128] This embodiment clearly demonstrates that the present invention, through its unique fine-grained and configurable reconstruction strategy, can meet the complex training data requirements of advanced AI tasks (such as multi-task risk modeling). It not only achieves entity-level fine-grained control over desensitization methods but also generates complex corpora containing both structured and semantic features, providing unprecedented, high-quality secure data support for training cutting-edge AI models.

[0129] In the final embodiment, a comparison of the data transformation effects under different strategies is provided: This embodiment aims to visually demonstrate and compare the core effects of the present invention under different content restructuring strategies through a specific example. For example... Figure 6 As shown.

[0130] 1) Example of raw sensitive text (601): Take, for example, a typical original text (601) that might appear in a financial contract. The original text reads: "Article 1 Basic Loan Information. The loan amount is RMB 500,000.00 (￥500,000.00), and the loan term is thirty-six months, starting from October 27, 2023." This original text contains various sensitive information that needs to be anonymized, such as the amount, term, and date.

[0131] 2) Structured parsing and recognition results (602): After receiving the original text, the semantic recognition module (220) performs step S2 on it. For example... Figure 6 As shown in (602), this module not only identifies the macro-level intent of this text block as "core loan terms," ​​but also precisely locates and labels various micro-level sensitive entities and their types within it, for example: "Fifty Hundred Thousand Yuan Only" was identified as the amount in capital letters; "￥500,000.00" was identified as a digital amount; "Thirty-six months" was identified as the time limit; "October 27, 2023" was identified as the date; This step provides a precise "target" for subsequent content reconstruction.

[0132] 3) Strategy 1 output (603): When the user selects the "Strategy 1" (for information extraction tasks) mode, the system will generate a structured extraction corpus (603). As can be seen, in this corpus, all non-sensitive syntactic structures and "skeleton vocabulary" are completely preserved, while sensitive entities are replaced by their corresponding type placeholders (such as [amount in capital letters], [date]).

[0133] 4) Strategy 2 output (604): When the user selects "Strategy Two" (for understanding and generation tasks), the system generates semantically generated corpus (604). In this corpus, the original sensitive content is replaced by a fluent, professionally written, but completely generic synthetic text. For example: "First, basic loan information. The specific amount and term of the loan under this contract shall be separately agreed upon by both parties. The loan interest rate and interest accrual date shall be subject to the relevant loan disbursement voucher." While retaining the core theme of "basic loan information," all specific sensitive values ​​are completely eliminated.

[0134] This comparative embodiment clearly demonstrates that the present invention provides two content reconstruction strategies with distinctly different features and effects: one is the "structured placeholder replacement" strategy, which produces text that retains syntactic structure to the maximum extent, suitable for optimizing the training of information extraction models; the other is the "integrated generative rewriting" strategy, which produces text that maintains semantic themes and natural fluency, suitable for optimizing the training of language understanding models. This clearly demonstrates the diversity and specificity of the content reconstruction method of the present invention, and its core value in providing customized data solutions for different AI training objectives.

[0135] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for creating a learning corpus for protecting sensitive textual information, characterized in that, Includes the following steps: Document Acquisition and Structured Parsing: The original document is processed through a multimodal document understanding model, which identifies and segments the logical regions in the document, and labels each unit with physical layout attributes and logical function categories to form a structured document object model. Semantic Intent and Sensitive Entity Recognition: Perform natural language processing on logical region units to identify their macro-intent categories and locate sensitive information entities within them; Content Reconstruction: Based on the pre-established entity processing strategy library, a replacement operation or a generative rewrite operation is applied to each sensitive information entity, and the content of the logical area unit is reconstructed to generate desensitized text blocks; wherein, the entity processing strategy library defines the mapping relationship between sensitive entity types and processing strategies, and the processing strategies include replacement operation and generative rewrite operation. Corpus generation: Reorganize all the anonymized text blocks according to the original document structure to generate the final anonymized learning corpus.

2. The method for creating a learning corpus for protecting text-based sensitive information according to claim 1, characterized in that, In the document acquisition and structured parsing process, the identification, segmentation, and annotation of logical region units are performed in parallel by calling a multimodal large language model.

3. The method for creating a learning corpus for protecting sensitive textual information according to claim 1, characterized in that, The identification criteria in the semantic intent and sensitive entity identification are established or enhanced through one or a combination of the following methods: Built-in general entity recognition model; User-defined regular expressions or keyword dictionaries; An adaptive learning mechanism that fine-tunes the recognition model using user-labeled samples.

4. The method for creating a learning corpus for protecting sensitive textual information according to claim 1, characterized in that, The strategies defined in the entity processing strategy library include probabilistic reconstruction strategies, which configure a probability distribution for at least one sensitive entity type. The content reconstruction step applies a replacement operation or a generative rewrite operation to the entity decision based on the probability distribution.

5. The method for creating a learning corpus for protecting sensitive textual information according to claim 1, characterized in that, In the content reconstruction, when a decision applies a generative rewrite operation to at least one sensitive information entity, the following sub-steps are executed: Construct and generate instructions that contain at least macro-level intent category information for logical region units; Invoke the generative language model to execute generation instructions to generate generalized text; Replace the original logical region units with generalized text.

6. The method for creating a learning corpus for protecting sensitive textual information according to claim 1, characterized in that, When constructing the generation instructions, relevant knowledge fragments obtained from external knowledge bases through retrieval enhancement technology are also incorporated, and / or text templates directly injected by the user through the instructions.

7. The method for creating a learning corpus for protecting sensitive textual information according to claim 1, characterized in that, After the corpus is generated, it also includes: For the same batch of original documents, multiple versions of de-identified learning corpora are generated by executing processing flows configured with different entity processing strategy libraries; Multiple versions of anonymized learning corpora and their generation configuration metadata are stored together in a corpus for management.

8. A system for creating learning corpora for protecting sensitive textual information, characterized in that, include: The multimodal document parsing module is used to process the original document through the multimodal document understanding model, identify and segment the logical region units in the document, and label each unit with physical layout attributes and logical function categories to form a structured document object model. The semantic recognition module is used to perform natural language processing on logical region units, identify their macro-intention categories, and locate sensitive information entities within them; The content reconstruction module is used to apply replacement or generative rewriting operations to each sensitive information entity based on a pre-established entity processing strategy library, and to reconstruct the content of logical area units to generate desensitized text blocks. The entity processing strategy library defines the mapping relationship between sensitive entity types and processing strategies, and the processing strategies include replacement operations and generative rewriting operations. The corpus generation module is used to reorganize all the anonymized text blocks according to the original document structure to generate the final anonymized learning corpus.

9. A learning corpus production system for protecting text-based sensitive information according to claim 8, characterized in that, The core functions of the multimodal document parsing module, semantic recognition module, and content reconstruction module can be implemented by calling the basic large language model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of a method for creating a learning corpus for protecting text-based sensitive information as described in any one of claims 1-7.

Images