Chinese medical text and document-oriented end-side PHI de-identification method and system

Abstract

The invention discloses a Chinese medical text and document-oriented end-side PHI (Physical Healthcare Information) de-identification method and system, the method is executed on user equipment or an edge node, and the method comprises the following steps: S1, receiving a Chinese medical text or document containing a protected PHI (Physical Healthcare Information); s2, PHI entity recognition is conducted on the text or the document on the end side, and recognition based on rules and named entity recognition based on a neural network are comprehensively applied in the recognition process; s3, based on the identified PHI entity type, performing hierarchical desensitization processing on the PHI entity by adopting a corresponding desensitization strategy; and S4, outputting the desensitized text or document. According to the method, privacy protection can be realized at a data source, high accuracy is ensured through collaborative recognition of rules and NERs, and medical semantic value is reserved to the maximum extent while security is ensured by means of a hierarchical desensitization strategy.

Description

Technical Field

[0001] This invention relates to the field of computer data security and privacy computing, specifically to a method and system for removing PHI (Personal Hierarchical Identification) from Chinese medical texts and documents. Background Technology

[0002] With the development of artificial intelligence and intelligent consultation technology, users are increasingly using large-scale models or online human consultations via the internet. The symptom descriptions, medical history, test results, prescriptions, or electronic medical records that users input on the platform typically contain sensitive personal information (PHIs) such as name, ID number, contact information, address, and precise dates. In accordance with laws, regulations, and industry standards, platforms must minimize the risk of re-identification at every stage of data collection, transmission, processing, and storage to prevent the leakage of sensitive information during the process.

[0003] Currently, the industry's common engineering practice is the "upload first, process later" cloud link model. That is, the raw data input by the user is collected at the front end and directly transmitted to the back-end server or cloud, and then the centralized service performs de-identification or de-identification processing. This model has two inherent defects: (1) Increased privacy exposure: The raw plaintext data that has not been de-identified needs to be transmitted and temporarily stored between the network and multiple service nodes, which greatly increases the risk of data leakage; (2) Limited latency and throughput: In high-concurrency scenarios, the centralized de-identification processing and the back-end large model inference and other services will queue up with each other, superimposed on the latency, affecting the system throughput and user experience.

[0004] In terms of privacy technology research, existing solutions also have many shortcomings: Differential privacy (DP) achieves privacy protection by adding random noise to the data. However, for free-form medical texts, noise can disrupt key medical semantics such as disease progression time, medication dosage, lesion location, and positive / negative test results, severely impairing the readability and usability of the text. In diagnostic scenarios that rely on accurate original text, this can lead to fatal biases. Furthermore, there is an irreconcilable conflict between privacy budgets and data usability.

[0005] Fully Homomorphic Encryption (FHE): Supports computation in ciphertext. However, it requires inference to be performed in the ciphertext domain or necessitates edge-cloud collaborative encryption and decryption, making its implementation complex. Currently, mainstream large language models cannot directly process ciphertext, and modifying these models involves extremely high engineering difficulty and computational overhead, resulting in an order-of-magnitude increase in inference latency, which cannot meet the real-time requirements of diagnostic dialogues. Furthermore, FHE's support for long text semantic processing is still immature.

[0006] Other non-endpoint-side data masking solutions: failed to fundamentally solve the "privacy exposure window" problem during data uplink, and the latency problem still exists due to the centralized operation of the data masking service and subsequent applications.

[0007] Furthermore, existing edge-side anonymization solutions typically employ only a single technical approach, such as those based on predefined rules (e.g., Presidio) or named entity recognition (NER) models (e.g., Hugging Face Transformers pipeline, LTP). These approaches have significant limitations when dealing with complex Chinese medical texts. Single methods lack robustness: Relying solely on rule matching is prone to missed detections due to colloquial expressions, entity variations, and OCR recognition noise, especially performing poorly on free text entities such as addresses and institution names; while relying solely on NER models is susceptible to interference from factors such as differences in hospital templates, professional abbreviations, and mixed Chinese and English spellings, resulting in boundary recognition errors or false detections. A single path cannot simultaneously achieve high accuracy and high recall on complex text.

[0008] Chinese clinical text recognition is challenging: Chinese lacks clear capitalization and word boundary markers, and entities such as names, drug names, and institution names are highly ambiguous. Furthermore, the prevalence of mixed Chinese and English writing, abbreviations, and colloquialisms makes methods directly transferred to English ineffective. At the same time, high-quality Chinese medical annotation corpora are scarce, making model training difficult, and the overall difficulty of de-identification is significantly higher than that of English.

[0009] Therefore, there is an urgent need in this field for a PHI de-identification technology solution that can overcome the above-mentioned defects, achieve high efficiency, accuracy, and low latency on the device side, and is specifically optimized for the characteristics of Chinese medical texts. Summary of the Invention

[0010] To overcome the shortcomings of existing technologies, one of the objectives of this invention is to provide a client-side PHI de-identification method for Chinese medical texts and documents. This method can achieve privacy protection at the data source, ensure high accuracy through the collaborative identification of rules and NER, and maximize the preservation of medical semantic value while ensuring security through a hierarchical de-identification strategy.

[0011] The second objective of this invention is to provide a client-side PHI de-identification system for Chinese medical texts and documents. This system can achieve privacy protection at the data source, ensure high accuracy through the collaborative identification of rules and NER, and maximize the preservation of medical semantic value while ensuring security through a hierarchical de-identification strategy.

[0012] To achieve one of the objectives of this invention, the following solution is adopted: A terminal-side PHI de-identification method for Chinese medical texts and documents, executed on a user device or edge node, includes the following steps: S1. Receive Chinese medical text or document containing Protected Health Information (PHI); S2. Perform PHI entity recognition on the text or document at the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition. S3. Based on the identified PHI entity type, the corresponding desensitization strategy is used to perform hierarchical desensitization processing on the PHI entity. S4. Output the anonymized text or document.

[0013] Furthermore, the comprehensive application of rule-based recognition and neural network-based named entity recognition includes: Identify PHI entities with strong formatting characteristics using predefined rules; Based on gating conditions, the neural network-based named entity recognition is activated within a local area of ​​the text or document to identify PHI entities with weak formatting features.

[0014] Furthermore, the gating condition includes at least one of the following: Predefined trigger words appear in the text; Heuristic anchors related to weakly formatted entities appear in the text; A character window of a certain size is formed centered on the position of the trigger word or anchor point, and named entity recognition based on neural networks is performed only within the window.

[0015] Furthermore, the rule-based recognition includes: performing pattern matching using regular expressions, and performing structural and semantic verification on the matching results, outputting the recognition result only after the verification passes.

[0016] Furthermore, the hierarchical desensitization strategy varies depending on the PHI entity type, and includes at least one of the following: masking, pseudo-nameing, generalization, and date offsetting.

[0017] Furthermore, a stable pseudo-random offset is performed on date-type entities to ensure that the same date value obtains the same offset result in multiple processing steps.

[0018] To achieve the second objective of this invention, the following solution is adopted: A PHI de-identification system for Chinese medical texts and documents, deployed on user devices or edge nodes, the system comprising: The input module is used to receive Chinese medical text or documents containing Protected Health Information (PHI); The recognition module is used to perform PHI entity recognition on the text or document on the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition. The desensitization module is used to perform hierarchical desensitization processing on the PHI entities based on the identified PHI entity types and the corresponding desensitization strategies. The output module is used to output the anonymized text or document.

[0019] Furthermore, the identification module includes: The rule recognition module is used to identify PHI entities with strong formatting features based on predefined regular expression patterns and semantic verification rules. The NER inference module is used to identify soft entities in free text based on a lightweight neural network model. The gating fusion module is used to schedule the NER inference module to run within a local text window based on the gating conditions triggered by the output of the rule recognition module, and to resolve and fuse the recognition results of the two modules.

[0020] Furthermore, the edge-side PHI de-identification system for Chinese medical texts and documents also includes a data augmentation module, which enhances the training data through at least one of template synthesis, text perturbation, and noise simulation to train a neural network model for neural network-based named entity recognition.

[0021] Furthermore, the neural network model in the recognition module is a lightweight model and is exported in ONNX format to achieve millisecond-level inference on the edge.

[0022] Compared to existing technologies, the beneficial effects of this invention are as follows: 1. Compared to differential privacy technology, this invention has the significant advantage of overcoming the defect of adding noise to the overall text, which leads to distortion of key medical semantics. By adopting a hierarchical fuzzing strategy, this invention can accurately identify different types of PHI entities and apply targeted desensitization processing (such as masking, pseudo-naming, generalization, date offset, etc.), thereby preserving the non-privacy medical semantics in the original text to the maximum extent while ensuring privacy and security, and ensuring the usability and readability of the desensitized text in the diagnostic process. 2. Compared to fully homomorphic encryption technology, this invention has outstanding advantages in terms of deployment convenience and cost. Specifically, this invention can complete the entire de-identification process on the client side, and can be directly connected without any modification to the backend server and large model, resulting in stronger system compatibility and engineering feasibility. At the same time, its rule base and lightweight neural network model support hot updates, significantly reducing operation and maintenance costs. 3. This invention has achieved a breakthrough in the computational efficiency bottleneck faced by fully homomorphic encryption technology, realizing efficient real-time processing. By adopting a lightweight edge-side model and optimized inference mechanism, this invention successfully overcomes the inherent bottlenecks of huge computational overhead and prolonged inference time in fully homomorphic encryption, achieving millisecond-level processing latency and fully meeting the real-time requirements of highly interactive scenarios such as online medical consultations. 4. Compared to non-edge-side de-identification schemes, this invention fundamentally improves privacy and security at the system architecture level. By placing the de-identification process locally on the user device or edge node, this invention completely avoids the flow of un-de-identified raw PHI data during network transmission and server processing, thereby significantly reducing the privacy exposure surface from the source and eliminating the risk of data leakage during the uplink process. 5. This invention exhibits higher recognition accuracy and robustness compared to other edge-side single-route technologies. Its core lies in the comprehensive application of rule-based recognition and neural network-based named entity recognition, achieving complementary advantages of the two technologies. The rule-based method can accurately capture entities with strong format features, while the NER method can effectively identify soft entities in free text. This collaborative mechanism overcomes the shortcomings of single-route methods in complex Chinese medical texts, which are prone to missed or false detections, comprehensively improving the accuracy and recall rate of recognition. 6. This invention is deeply optimized for Chinese medical scenarios, solving technical challenges in specific contexts. By designing models and strategies specifically tailored to the linguistic characteristics of Chinese medical texts (such as lack of word boundaries, high entity ambiguity, and mixed Chinese and English writing), this invention achieves superior recognition accuracy while maintaining low latency, particularly meeting the actual needs of Chinese online consultation platforms. 7. This invention ultimately possesses excellent practicality and reliability, adapting to complex application environments. The entire system supports offline operation, still providing de-identification services normally in weak or no network environments, ensuring the continuity and reliability of the medical consultation process. Attached Figure Description

[0023] Figure 1 This is a block diagram of the terminal-side PHI de-identification system for Chinese medical texts and documents in an embodiment of the present invention. Figure 2 This is a comparison chart of the F1 scores of the embodiments of the present invention and LTP on various entity types; Figure 3 This is a comparison chart of the F1 scores of the embodiments of the present invention and Presidio on various entity types; Figure 4 This is a comparison chart of the end-side processing latency of the embodiments of the present invention and the prior art; Figure 5 This is a performance comparison chart of different de-identification modes in the embodiments of the present invention; Figure 6 This is a comparison chart of latency for different de-identification modes in an embodiment of the present invention; Figure 7 This is a comparison chart of F1 scores for various entity types in the embodiments of the present invention; Figure 8 This is a flowchart of the end-side PHI de-identification method for Chinese medical texts and documents in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0025] Example 1 like Figure 1-7 As shown in the figure, this embodiment of the invention provides an edge-side PHI de-identification system for Chinese medical texts and documents, deployed on user devices or edge nodes. The system includes an input module, a recognition module, a de-identification module, and an output module, which are described in detail below: The input module is used to receive Chinese medical text or documents containing Protected Health Information (PHI).

[0026] Specifically, the system first performs local de-identification on the received user input (including free text, electronic medical record documents, forms or scans) on the data generating end or user end device, and then uploads the de-identified minimum necessary information to the server or large model.

[0027] The recognition module is used to perform PHI entity recognition on the text or document on the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition.

[0028] Furthermore, the identification module includes: The rule recognition module is used to identify PHI entities with strong formatting features based on predefined regular expression patterns and semantic verification rules. The NER inference module is used to identify soft entities in free text based on a lightweight neural network model. The gating fusion module is used to schedule the NER inference module to run within a local text window based on the gating conditions triggered by the output of the rule recognition module, and to resolve and fuse the recognition results of the two modules.

[0029] The execution flow of the rule recognition module is as follows: This module receives text or documents as input and outputs recognition results including entity type, character start and end positions, original text fragments, confidence level and optional meta-information (such as normalized number strings, whether it contains delimiters, source, etc.).

[0030] Specifically, the following processing steps are included: First, through the pattern matching stage, regular expression patterns designed for ID cards, mobile phone numbers, email addresses, dates, postal codes, bank cards, etc., are used to cover common writing variations in the Chinese context. During this process, normalization and error tolerance are performed, that is, character normalization (unifying full / half-width characters, removing visible / invisible separators such as spaces, hyphens, narrow non-breaking spaces, etc.) and structural pre-validation are performed before matching to enhance tolerance to noise and OCR variations.

[0031] Furthermore, semantic validation is performed, such as checking the check digits of ID cards and performing Luhn validation on bank cards. A hit result is only output when both structural and semantic validations pass, significantly reducing false positives. Simultaneously, a boundary robustness strategy is adopted, employing assertion strategies adapted to Chinese to avoid missed detections or overlaps caused by the failure of English "word boundaries" in the Chinese context.

[0032] In addition, the module also performs weak tagging of trigger words, that is, it projects low-weight labels onto the trigger word list such as "name / address / phone / ID card / patient / contact person", providing a local activation window for the subsequent Gated NER mechanism, thereby improving the coverage of difficult cases while ensuring system throughput.

[0033] Based on this, the module grades the recognition results by confidence level, classifying them according to "verifiable strength / boundary reliability" to provide a basis for subsequent overlap and conflict resolution. Ultimately, at the system collaboration level, the rule side focuses on the task of "identifying who," while the engine side performs operations such as masking, pseudonymization, generalization, and date offset according to the strategy, and implements the "Gated NER" mechanism based on trigger words / anchor points, thereby taking into account both high-precision hard entity and soft entity recall.

[0034] The execution flow of the NER inference module is as follows: Within the "rule-first" framework, this module is positioned as a supplementary recognition component for soft entities (such as names, addresses, etc.). Its function is to output character-level spans and confidence scores for subsequent engine fusion. This module accepts any Chinese text or document as input and outputs recognition results including entity type, character start and end positions, original text fragments, and confidence scores.

[0035] Regarding the tagging and annotation system, the O / B-XXX / I-XXX tag set is adopted, covering entity types such as NAME, ADDRESS, ORG, ID_NUMBER, PHONE, EMAIL, and DATE, and maintaining alignment with the entities on the rule side. The annotation scope is based on character-level BIO annotations, and then aligned to the token-level supervision of the tokenizer.

[0036] The training process includes the following steps: First, the open-source MultiPriv dataset is converted to PHI-based training format. Then, data cleaning is performed to remove information that does not conform to common sense or norms. Next, the dataset is split into test and training sets at a 1:4 ratio. Finally, data augmentation is applied to the training set, resulting in 4000 training data points. During the encoding phase, character-level BIO tags are mapped to token sequences according to the tokenizer's word_ids, and invalid bits are marked as -100. Training uses the standard cross-entropy loss function, setting the batch size to 16, the learning rate to approximately (2~3)e-5, and the number of training epochs to 2~5. The model is evaluated and saved step-by-step or per epoch. In the model export phase, after training, the PyTorch weights and tokenizer are saved, converted to ONNX format using an export script, and the tag mapping is copied to support variable sequence lengths. Optimization measures include adversarial training, early stopping, learning rate warm-up and linear decay, mixed-precision training, gradient accumulation, and step-by-step evaluation and saving of the optimal model. The evaluation metrics used are weighted / macro-micro average precision / recall / F1 score, which are used to select the optimal weights and tracking quality.

[0037] In the inference pipeline (ONNX side), session initialization begins by using CPUExecutionProvider to load the tokenizer and id2label, with configurable window size, stride, sentence segmentation options, and max_length parameter. Next, text segmentation is performed, dividing sentences by Chinese punctuation or line breaks. Within each sentence, inference is performed in blocks according to a "character window + sliding stride" approach to balance long text processing and latency stability. During tensor construction, offset_mapping is enabled to preserve the start and end offsets from tokens to characters, facilitating the reconstruction of character-level spans. Label decoding then occurs, aggregating consecutive B / I labels according to BIO rules, calculating the average confidence of the span (the average of the maximum probabilities of the corresponding B / I values) to obtain character-level entities. Finally, cross-window and overlap merging are performed. When similar entities overlap, those with higher confidence are retained; when confidence is similar, those with longer coverage are retained, achieving deduplication and boundary stabilization. The entire process also supports cache reuse, caching the absolute offset and token probability of the most recent inference for faster subsequent scoring, avoiding repeated tokenization and inference processes.

[0038] The core control logic of the gating fusion module is the GatedNER mechanism, which is a rule and NER fusion mechanism. The execution logic of this mechanism is as follows: The general principle is to first use rules to accurately cover "strongly constrained" entities (including ID, PHONEMAIL, DATE, ZIP, BANK_CARD), and then enable NER in local areas according to gating conditions to supplement the identification of "soft entities" (such as NAME, ADDRESS).

[0039] Specifically, this is achieved through the following gating conditions: First, set up a tag whitelist, and only run NER on tags in the whitelist (such as NAME, ADDRESS).

[0040] Secondly, establish trigger and anchor mechanisms, including: Triggered when keywords such as "name / address / phone number / ID card / patient / contact person" appear in the text; Even without a specific trigger word, it can be triggered by heuristic regular expression anchors such as "province / city / street / number, Mr / Ms / name".

[0041] After determining the trigger location, a windowed processing method is adopted: with the trigger location as the center, N characters are expanded to the left and right to form an analysis window, which supports merging overlapping windows and limits the maximum number of windows to K; when there are no trigger conditions, the system can degenerate into processing the full text.

[0042] Ultimately, this mechanism enables NER inference to be performed only on whitelisted tags within the trigger window, thereby significantly reducing processing latency and false alarm risk while ensuring recognition effectiveness.

[0043] The desensitization module is used to perform hierarchical desensitization processing on the PHI entities based on the identified PHI entity types and the corresponding desensitization strategies.

[0044] Specifically, the system implements the de-identification operation through a predefined policy file, which contains the following components: The entity action area defines the desensitization processing methods for different entity types. The specific configuration is as follows: For entities of type NAME, perform pseudonymization (e.g., replace with "patient_{seq}").

[0045] For entities of type ID_NUMBER, a mask is used while the last digit is retained.

[0046] For PHONE type entities, implement a pattern mask (e.g., displayed as "--####").

[0047] For EMAIL type entities, only the username portion is retained (keep_user).

[0048] For entities of type ADDRESS, perform geographic level generalization and retain them to the district or city level.

[0049] For DATE type entities, a date shifting strategy is adopted, which uses a stable pseudo-random offset to ensure that the same date value obtains the same offset result.

[0050] The runtime area records are used to configure various parameters for system runtime, mainly including: The `ner_enabled_labels` parameter limits the range of labels for which NER processing is enabled (e.g., NAME / ADDRESS).

[0051] The `ner_require_triggers` parameter controls whether NER (Network Request Execution) is only executed if trigger words or anchor conditions are met. It also allows configuration of runtime parameters such as the specific size of the trigger window, the maximum number of windows, and whether to enable heuristic anchors (such as keywords related to address or name).

[0052] The output module is used to output the anonymized text or document.

[0053] Specifically, its output includes: de-identified text, along with hit audit information (covering type, location, original / replacement content, confidence level, and source), and a risk score. For input documents (such as PDFs), the document masking module handles the process, visually masking the text by drawing black borders, and ultimately outputting a document with privacy information masked.

[0054] This document outlines the complete process of PHI de-identification for edge-side Chinese medical text, including test environment setup, data preparation, model training and deployment, and the overall runtime workflow. The test environment was configured with x86-64 / ARM edge devices and Windows 11 operating system and Python 3.12. During data preparation and annotation, the MultiPriv corpus was converted to PHI sequence labeling format and cleaned using rules. The data was then divided into test and training sets at approximately a 1:4 ratio. Finally, data augmentation was performed on the training set to obtain approximately 4000 training data entries. The annotation system adopted a character-level BIO format (O / B-XXX / I-XXX), covering entity types such as NAME, ADDRESS, ID_NUMBER, PHONE, EMAIL, and DATE.

[0055] In the NER training and export phase, a lightweight Chinese BERT / RoBERTa TokenClassification model was used for training. The batch size was set to approximately 16, the learning rate to approximately (2-3)e-5, and the number of training epochs was 2-5. Optimization strategies such as adversarial training, early stopping, learning rate warm-up and linear decay, mixed precision, gradient accumulation, and step-by-step evaluation were enabled. After training, the PyTorch weights and tokenizer were exported in ONNX format, and the corresponding labels.json file was generated.

[0056] The specific operation process includes the following steps: First, read the input data, including free text or text converted from PDF / OCR.

[0057] Next, rule recognition is performed, including unifying full-width and half-width characters, stripping delimiters, matching entities such as ID_NUMBER, PHONE, EMAIL, DATE, ZIP, and BANK_CARD using regular expressions, performing check digit verification on ID cards, performing Luhn verification on bank cards, and adapting Chinese prefixes and suffixes and boundary processing for postal codes, while generating a weak label "TRIGGER".

[0058] Then, GatedNER processing is performed, and ONNX inference is performed only on whitelisted tags (NAME, ADDRESS) within the trigger word or anchor window. BIO decoding is performed through sentence segmentation and sliding window strategies to obtain character-level spans and their average confidence, and overlapping entities are merged, retaining entities with high confidence or longer length.

[0059] During the integration and conflict resolution phase, the principle of prioritizing hard entities (rule hits) is followed, while entities of the same level are selected based on confidence level, length, and boundary reliability.

[0060] Then, the strategy is executed, and corresponding desensitization operations are performed according to the entity type, including masking (such as retaining the last digit or template masking), pseudonymization (using serial numbers), address generalization (to the district / city level), and date shifting (stable pseudo-random offset). Finally, the de-identified text and audit entries containing information such as type, location, original text, replacement content, confidence level, and source are generated.

[0061] If the input is in document format, the document masking process begins. The identified spans are mapped to document layout coordinates, and a black frame is drawn to achieve visual masking. The output is a version of the document with the privacy information masked.

[0062] Evaluation process and results: Data: The test set obtained earlier is processed by adding noise and shuffling word order.

[0063] Evaluation script: Output Micro / Macro P / R / F1, text-level metrics, and latency quantiles.

[0064] result: Rules-only mode (NER off): Overall:P=1.000,R=1.000,F1=1.000;EMAIL / PHONE / ID_NUMBERF1=1.00; Latency: Average ≈ 0.404ms / text; Gated NER mode (NER and rule fusion): Overall:P≈0.975,R=0.975,F1≈0.975;EMAIL / PHONE / ID_NUMBER; F1=1.00;NAME F1=0.847;ADDRESS F1=0.952; Latency: Average ≈ 26.50ms / text; NER+ rule mode (uses NER+ rules for all text): Overall: P≈0.991, R=1.00, F1≈0.995; EMAIL / PHONE / ID_NUMBER F1=1.00; NAMEF1=1.0; ADDRESS F1=0.978; Latency: Average ≈ 54.41ms / text.

[0065] Experimental results show that the edge-side rule-first strategy maintains an accuracy of 1.0 and millisecond-level latency in "hard entity" recognition; enabling GatedNER can significantly improve the recall rate of NAME and ADDRESS, improve the overall F1 score, and still meet the real-time requirements of interaction; with both NER and rules enabled, the system can fully cover "hard entities" and "soft entities" such as names and addresses, maintaining an accuracy close to 1.0. Although the latency increases slightly, it still meets the real-time requirements of interaction.

[0066] This embodiment fully demonstrates the following key advantages: by implementing end-to-end processing to prevent un-anonymized text from being uploaded, the risk of privacy exposure is significantly reduced; by adopting a collaborative mechanism of rules and gated NER, both the accurate identification of hard entities and the high recall rate of soft entities are ensured, while maintaining overall low latency; flexible policy configuration is supported, and both the rule base and the lightweight model support hot updates, which greatly reduces the cost of engineering implementation and maintenance.

[0067] This invention constructs an edge-side de-identification closed loop, completing the entire chain of processing from preprocessing, identification, fusion to hierarchical de-identification and uplink locally on user devices or edge nodes within the hospital. This ensures that the original data that has not been de-identified does not need to be uploaded; only the de-identified results and non-sensitive statistical information are uploaded, thus minimizing the privacy exposure from the source.

[0068] In terms of technical implementation, this invention adopts a strategy of rule-first approach combined with gated NER. First, a rule engine is used to deterministically identify and match strongly formatted entities such as ID card numbers and mobile phone numbers. Then, through a gated link mechanism, neural network named entity recognition is initiated only when specific trigger words or heuristic anchors are detected, targeting whitelisted tags such as names and addresses within a local text window. This balances recognition accuracy with inference overhead. Simultaneously, by comparing a tuple composed of entity type priority and recognition confidence, efficient and reliable conflict resolution of recognition results is achieved.

[0069] For Chinese text, this invention designs a robust Chinese strong-format entity recognition scheme. By constructing a multi-pattern regular expression family covering full-width / half-width characters and delimiter variations, combined with character normalization and delimiter stripping before matching, and a boundary assertion strategy adapted to Chinese, the robustness of pattern matching is effectively improved. After hitting a potential entity, a concatenated verification mechanism at the structural and semantic levels is introduced to ensure that output is only performed after double verification passes, significantly reducing the false positive rate.

[0070] To achieve real-time response on the edge, this invention integrates a lightweight NER model and inference optimization techniques. Employing a distilled lightweight Transformer model, combined with optimization strategies such as INT8 / INT4 quantization, sliding window inference, and cache reuse, millisecond-level processing latency and stable throughput performance can be achieved on a general-purpose computing module, fully meeting the real-time requirements of interactive consultations.

[0071] To overcome the challenge of scarce Chinese medical annotation data, this invention proposes an integrated method for data augmentation and small model generalization in Chinese medical PHI (Paraphrased Hierarchical Inference) models. This process encompasses a series of data augmentation techniques, from template-slot synthesis, text perturbation, and OCR noise simulation to the construction of non-PHI approximate adversarial examples. Through carefully orchestrated training and sampling strategies, the augmented data is effectively used for model training. Finally, a lightweight inference format is derived and deployed collaboratively with a rule engine, significantly improving the generalization ability and recall rate of small models in complex real-world scenarios.

[0072] This invention implements a tiered desensitization and stable offset mechanism, applying differentiated processing strategies such as deletion, masking, generalization, pseudo-nameization, or date offset based on the different categories of PHI entities. In particular, for date-related information, a stable pseudo-random offset algorithm is used to ensure that the same original value obtains a consistent offset result in multiple processing steps. This approach thoroughly protects privacy and security while maximizing the integrity of non-sensitive medical semantics and the subsequent usability of the data.

[0073] Example 2 like Figure 8 As shown, this embodiment of the invention provides an edge-side PHI de-identification method for Chinese medical texts and documents. The method is executed on a user device or edge node and includes the following steps: S1. Receive Chinese medical text or documents containing Protected Health Information (PHI).

[0074] S2. Perform PHI entity recognition on the text or document on the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition.

[0075] S3. Based on the identified PHI entity type, the corresponding desensitization strategy is used to perform hierarchical desensitization processing on the PHI entity.

[0076] S4. Output the anonymized text or document.

[0077] Furthermore, the comprehensive application of rule-based recognition and neural network-based named entity recognition includes: Identify PHI entities with strong formatting characteristics using predefined rules; Based on gating conditions, the neural network-based named entity recognition is activated within a local area of ​​the text or document to identify PHI entities with weak formatting features.

[0078] Furthermore, the gating condition includes at least one of the following: Predefined trigger words appear in the text; Heuristic anchors related to weakly formatted entities appear in the text; A character window of a certain size is formed centered on the position of the trigger word or anchor point, and named entity recognition based on neural networks is performed only within the window.

[0079] Furthermore, the rule-based recognition includes: performing pattern matching using regular expressions, and performing structural and semantic verification on the matching results, outputting the recognition result only after the verification passes.

[0080] Furthermore, the hierarchical desensitization strategy varies depending on the PHI entity type, and includes at least one of the following: masking, pseudo-nameing, generalization, and date offsetting.

[0081] Furthermore, a stable pseudo-random offset is performed on date-type entities to ensure that the same date value obtains the same offset result in multiple processing steps.

[0082] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A client-side PHI de-identification method for Chinese medical texts and documents, characterized in that, Performing the method on a user device or edge node includes the following steps: S1. Receive Chinese medical text or document containing Protected Health Information (PHI); S2. Perform PHI entity recognition on the text or document at the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition. S3. Based on the identified PHI entity type, the corresponding desensitization strategy is used to perform hierarchical desensitization processing on the PHI entity. S4. Output the anonymized text or document.

2. The endpoint PHI de-identification method for Chinese medical texts and documents according to claim 1, characterized in that, The integrated application of rule-based recognition and neural network-based named entity recognition includes: Identify PHI entities with strong formatting characteristics using predefined rules; Based on gating conditions, the neural network-based named entity recognition is activated within a local area of ​​the text or document to identify PHI entities with weak formatting features.

3. The endpoint PHI de-identification method for Chinese medical texts and documents according to claim 2, characterized in that, The gating condition includes at least one of the following: Predefined trigger words appear in the text; Heuristic anchors related to weakly formatted entities appear in the text; A character window of a certain size is formed centered on the position of the trigger word or anchor point, and named entity recognition based on neural networks is performed only within the window.

4. The endpoint PHI de-identification method for Chinese medical texts and documents according to claim 2, characterized in that, The rule-based recognition includes: using regular expressions for pattern matching, and performing structural and semantic verification on the matching results, outputting the recognition result only after the verification passes.

5. The endpoint PHI de-identification method for Chinese medical texts and documents according to claim 1, characterized in that, The hierarchical desensitization strategy varies depending on the PHI entity type and includes at least one of the following: masking, pseudo-nameing, generalization, and date offsetting.

6. The endpoint PHI de-identification method for Chinese medical texts and documents according to claim 5, characterized in that, Perform a stable pseudo-random offset on date-type entities to ensure that the same date value yields the same offset result in multiple processing steps.

7. A client-side PHI de-identification system for Chinese medical texts and documents, characterized in that, Deployed on user devices or edge nodes, the system includes: The input module is used to receive Chinese medical text or documents containing Protected Health Information (PHI); The recognition module is used to perform PHI entity recognition on the text or document on the terminal side. The recognition process combines rule-based recognition and neural network-based named entity recognition. The desensitization module is used to perform hierarchical desensitization processing on the PHI entities based on the identified PHI entity types and the corresponding desensitization strategies. The output module is used to output the anonymized text or document.

8. The edge-side PHI de-identification system for Chinese medical texts and documents according to claim 7, characterized in that, The identification module includes: The rule recognition module is used to identify PHI entities with strong formatting features based on predefined regular expression patterns and semantic verification rules. The NER inference module is used to identify soft entities in free text based on a lightweight neural network model. The gating fusion module is used to schedule the NER inference module to run within a local text window based on the gating conditions triggered by the output of the rule recognition module, and to resolve and fuse the recognition results of the two modules.

9. The edge-side PHI de-identification system for Chinese medical texts and documents according to claim 7, characterized in that, It also includes a data augmentation module for augmenting training data through at least one of template synthesis, text perturbation, and noise simulation to train a neural network model for neural network-based named entity recognition.

10. The edge-side PHI de-identification system for Chinese medical texts and documents according to claim 7, characterized in that, The neural network model in the recognition module is a lightweight model and is exported in ONNX format to achieve millisecond-level inference on the edge.

Images