A localized intelligent desensitization method and system for multi-modal medical data and a storage medium

Abstract

The application relates to the technical field of medical data security and artificial intelligence, in particular to a localized intelligent desensitization method and system for multi-modal medical data and a storage medium, which coordinates an OCR and a local instruction fine-tuning large language model, first positions an image Chinese text coordinate by the OCR, and then accurately identifies and classifies sensitive entities by a domain fine-tuning large model; the desensitization granularity is refined from a text line level to a character level by adopting a vertical projection method and other technologies, pixel-level accurate shielding of sensitive text in an image is realized, a cross-modal correlation mechanism is constructed, sensitive entities in different modal data (such as DICOM images, PDF reports and electronic medical record texts) are correlated to the same patient node, a layered desensitization strategy based on a quantitative risk score is uniformly applied, the application supports localized offline deployment, patient privacy is completely protected, the medical diagnosis and scientific research value of data are maximally reserved, and the strict data security compliance requirements of the medical industry are met.

Description

Technical Field

[0001] This invention relates to the fields of medical data security and artificial intelligence technology, specifically to a localized intelligent desensitization method, system, and storage medium for multimodal medical data. Background Technology

[0002] With the accelerated pace of healthcare digitalization, medical data containing patient privacy (such as DICOM images like CT / MRI scans, PDF / Word examination reports, and structured electronic medical records) is being used increasingly frequently in clinical practice, research, and data sharing. This data contains direct identifiers (such as names and ID numbers) and quasi-identifiers (such as hospitals and dates), posing an extremely high risk of leakage. If used directly for data sharing, model training, or research analysis without effective anonymization, it can easily lead to privacy breaches and violate relevant laws and regulations such as the Personal Information Protection Law and the Medical Data Security Guidelines.

[0003] Existing medical data anonymization technologies have the following drawbacks: 1. Modal fragmentation and isolated processing: Existing solutions mostly target a single data modality (plain text or plain image). For example, text desensitization tools cannot handle text embedded in images (such as scanned reports or image watermarks); while image desensitization tools (such as overall blurring) can destroy the medical diagnostic value of images. There is a lack of collaborative desensitization capabilities for mixed text and image data. 2. Coarse precision, damaging data utility: Traditional image desensitization often uses global or regional blurring / mosaic, which indiscriminately damages non-sensitive areas (such as lesion features, key charts). For text in images, existing OCR technology can locate "text lines," but cannot recognize their content, let alone accurately mask only specific sensitive words within the same text line; 3. Lack of tiered sensitivity classification for data anonymization: A single entry like "Age: 45 years old" is unlikely to identify an individual, but when combined with "Hospital visited: XX Community Hospital" and "Diagnosed rare disease: Disease A," it may be possible to pinpoint an individual. This indirectly leads to the exposure of patient information, and the existing tiered classification rules are vague and logically unclear. 4. Reliance on the cloud poses security and latency risks: Many intelligent solutions based on large models need to call cloud APIs, causing sensitive raw medical data to leave the local environment, violating data security compliance requirements, and being affected by network latency; 5. Lack of multimodal association: No association mechanism for sensitive entities between different modal data has been established, which may lead to some modalities being missed in the desensitization process; core algorithms such as vertical projection lack key parameters and anomaly handling logic, resulting in insufficient technical reproducibility.

[0004] Therefore, there is a need for a technical solution that can run in a local environment, collaboratively process mixed text and image data, implement algorithm details, and achieve pixel-level accurate desensitization, so as to protect privacy while maximizing the preservation of the clinical and research value of the data. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a localized intelligent de-identification method, system, and storage medium for multimodal medical data.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a localized intelligent de-identification method for multimodal medical data, comprising the following steps: S1: Multimodal data multi-branch unified preprocessing, select the corresponding image or text preprocessing branch according to the input data type, and output a standardized image array or a normalized text block list; S2: Perform optical character recognition (OCR) on image-type modal data to obtain the text content and its coarse-grained coordinates in the image; S3: Input the OCR text identified in step S2 and / or the normalized text preprocessed in step S1 into a large language model LLM that has been fine-tuned by instructions and deployed locally, identify and output a structured sensitive entity list in a hierarchical manner, and the LLM outputs the corresponding desensitization level based on the cumulative risk scoring algorithm; S4: Perform string matching and alignment between the sensitive entities identified by LLM in step S3 and the text lines identified by OCR in step S2 or the text blocks in step S1 to establish a mapping relationship between sensitive entities and location information. S5: For the text line region containing sensitive entities matched in step S4, the vertical projection method is used to refine the character-level coordinates and obtain the pixel-level bounding box coordinates of the sensitive characters in the original image. S6: Based on the patient's core identifier, aggregate sensitive entities in different modal data to the same patient node, recalculate the cross-modal global risk score and determine a unified desensitization level, and perform cross-modal consistency verification; S7: Based on the desensitization level determined in step S3 and the precise coordinates obtained in step S5, perform layered precise desensitization processing on the corresponding pixel areas in the image, and perform replacement or blurring processing on the corresponding fields in the text data. S8: Output the de-identified multimodal data file and the corresponding de-identification log.

[0007] In some embodiments, in step S1, the image preprocessing branch includes: removing window width and window level interference from DICOM images and extracting superimposed text; converting scanned PDFs into image sequences; and performing resolution normalization, denoising, and binarization on all images; the text preprocessing branch includes: unified encoding, removing special characters, and converting to structured JSON text blocks.

[0008] In some embodiments, the cumulative risk scoring algorithm in step S3 is as follows: Patient record cumulative risk score = Σ (preset risk weight for each sensitive entity × number of times the entity appears); And based on the cumulative risk score, the desensitization level is mapped, including: When the risk score is greater than or equal to the first threshold, full data anonymization is performed. When the risk score is between the second threshold and the first threshold, only highly sensitive and moderately sensitive entities are desensitized. When the risk score is less than the second threshold, desensitization is only applied to highly sensitive entities.

[0009] In some embodiments, the vertical projection method in step S5 specifically includes the following sub-steps: S5.1: Perform grayscale and binarization processing on the cropped image of text lines containing sensitive entities; S5.2: Project the binarized image column by column, calculate and normalize the projection value of each column; S5.3: Use a sliding window to smooth column projection values; S5.4: Calculate the adaptive segmentation threshold based on the mean of column projection values; S5.5: Identify consecutive columns with projection values ​​lower than the segmentation threshold as character segmentation gaps; S5.6: Divide the character sub-regions according to the segmentation gaps, and map the sub-regions to the character order in the OCR text; S5.7: Based on the start and end indices of the sensitive entity in the OCR text, locate the corresponding character sub-region and transform its coordinates back to the original image coordinate system to obtain pixel-level bounding boxes.

[0010] In some embodiments, in step S6, the core identifier includes at least one of medical record number and patient ID, and the auxiliary identifier includes patient name; sensitive entities identified from DICOM images, document images, and structured text are associated with the same patient node through primary key matching and auxiliary key similarity matching.

[0011] In some embodiments, the layered precise desensitization processing in step S7 performs different operations on sensitive regions in the image according to their sensitivity level: rectangular filling is performed on regions corresponding to highly sensitive entities, Gaussian blur with a first parameter is performed on regions corresponding to medium-sensitive entities, and Gaussian blur with a second parameter is performed on regions corresponding to weakly sensitive entities; the blur intensity of the first parameter is higher than that of the second parameter.

[0012] In some embodiments, the rectangular fill is white and supports rounded rectangle fill with a corner radius of 1 to 10 pixels.

[0013] To achieve the above objectives, the present invention also provides the following technical solution: a localized intelligent de-identification system for multimodal medical data, employing the aforementioned method, comprising the following modules deployed in a local hardware environment: The input and management module is used to receive multimodal medical data input and implement desensitization strategies. A multimodal preprocessing module, connected to the input and management module, is used to perform unified preprocessing across multiple branches; The OCR recognition module is connected to the multimodal preprocessing module and is used to perform optical character recognition on image data and output text and coordinates. The sensitive entity identification and classification module connects the multimodal preprocessing module and the OCR recognition module, and is used to call the local large language model to identify and classify sensitive entities. The OCR text and large model output alignment module connects the OCR recognition module and the sensitive entity recognition and classification module, and is used to perform entity-position mapping alignment. A coordinate refinement module, connecting the alignment module and the OCR recognition module, is used to obtain character-level precise coordinates through the vertical projection method; The cross-modal association and consistency verification module is connected to the sensitive entity identification and classification module, and is used to construct patient nodes and unify desensitization strategies; The desensitization execution module, connected to the coordinate refinement module and the cross-modal association and consistency verification module, is used to perform hierarchical and precise desensitization. The result export module is connected to the de-identification execution module and is used to output the de-identified data and logs.

[0014] In some embodiments, the system is containerized using Docker and provides a RESTful API interface that receives an input request containing file paths and policy identifiers and returns the anonymized file paths and log file paths.

[0015] To achieve the above objectives, the present invention also provides the following technical solution: a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the localized intelligent desensitization method for multimodal medical data.

[0016] Compared with existing technologies, the beneficial effects of this invention are: it provides a high-precision, high-security, highly available, and practical multimodal medical data anonymization solution. Compared with existing technologies, it not only significantly leads in technical indicators (such as character-level accuracy and multimodal association accuracy), but more importantly, it solves the core pain points of medical data sharing, circulation, and research—maximizing the preservation of the clinical and research value of medical data itself while strictly protecting patient privacy, thus possessing significant practical application value. Its creativity is mainly reflected in the following aspects: 1. Innovation in the Technical Collaboration Framework: This framework creatively combines OCR (Optical Character Recognition), Large Language Model (LLM) for instruction fine-tuning, and vertical projection image processing algorithms to construct a closed-loop pipeline: "Coordinate localization (OCR) → Semantic recognition and classification (LLM) → Pixel-level coordinate refinement (vertical projection)." This framework systematically solves the complex challenge of "finding text location, understanding text semantics, and achieving pixel-level precise operation." 2. Breakthrough in desensitization granularity from "line-level" to "character-level": For embedded text in images, existing technologies mostly stop at blurring the entire text area. This invention, through a detailed engineering scheme of vertical projection method (including normalized column projection, adaptive thresholding, moving average, anomaly handling, etc.), successfully achieves pixel-level coordinate extraction and masking of specific sensitive characters or phrases within the same text line, greatly improving the desensitization accuracy and reducing the damage to non-sensitive information; 3. Cross-modal association and unified strategy mechanism: A method for constructing "patient nodes" based on core patient identifiers (such as medical record number, ID) is proposed to associate sensitive entities from different modalities such as DICOM images, report images, and structured text. Based on this, unified risk scoring and de-identification strategy decisions are made, and cross-modal consistency checks are performed to ensure that all data of the same patient is synchronously and without omissions, solving the problem of missed de-identification caused by "information silos" in multimodal data. 4. A balanced design between localized deployment and layered anonymization: Localization: The entire system (including high-performance OCR and a large language model that has been lightweighted and fine-tuned) is designed to be fully deployed on local servers, eliminating the security and compliance risks of uploading raw medical data to the cloud; Layered anonymization: A dynamic anonymization strategy based on quantitative risk scoring is introduced. A cumulative risk score is calculated based on the type, weight, and combination of sensitive entities, dynamically determining the processing intensity for strongly, moderately, and weakly sensitive entities. This design achieves "on-demand anonymization," maximizing privacy protection while preserving the research and clinical value of the data as much as possible.

[0017] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. The embodiments of this application will provide a detailed description and understanding of the application. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the overall workflow of the present invention; Figure 2 This is a schematic diagram of OCR coordinate extraction in the embodiment; Figure 3 A schematic diagram illustrating the principle of refining the coordinates of characters within a text line using the vertical projection method; Figure 4 This is a diagram showing the effect of desensitization in the example; Figure 5 This is a flowchart for multimodal association and consistency verification. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention aims to overcome the shortcomings of existing technologies and provide a multimodal medical data desensitization method and system deployed in a local hardware environment, based on the collaborative operation of OCR and a large language model with instruction fine-tuning. This method can automatically identify sensitive information in images and text, achieve pixel-level precise masking through standardized algorithmic logic, and establish a multimodal data association mechanism, ensuring the integrity and usability of non-sensitive medical information while thoroughly desensitizing it. Example

[0021] Please see Figures 1 to 5 This invention provides a technical solution: a localized intelligent de-identification method for multimodal medical data, the specific steps of which are as follows: S1: Unified preprocessing of multimodal data across multiple branches The purpose of this step is to transform input data from different sources and in different formats into a standardized intermediate format that can be efficiently and accurately processed in subsequent steps (OCR, large-scale model recognition). The preprocessing module executes the corresponding branch process based on the essential form of the input data (image or text).

[0022] 1.1 Image-based modality preprocessing branch Input data includes: DICOM medical image files, scanned PDF documents, JPG / PNG image files, and scanned Word documents that are essentially image content. The processing steps are as follows: a) Format conversion and information extraction: DICOM files: The pydicom library is used to read the files. This involves extracting the pixel array for subsequent image processing, as well as handling any text that may be superimposed on the image (such as patient names, IDs, and other burned information). These superimposed text layers are extracted by reading specific DICOM tags (such as proprietary multimedia tags of (60xx, 3000)) or by performing preliminary text region detection on the pixel array.

[0023] Scanned PDFs / Scanned Word documents: The pdf2image library (based on Poppler) is used to convert each page of the PDF file into an independent RGB image (PIL.Image object). For scanned Word documents, the embedded images are first identified using libraries such as python-docx and extracted as an image sequence.

[0024] b) Image normalization: Resolution standardization: To ensure consistency in subsequent OCR and projection algorithms, all images are uniformly scaled to a standard resolution of 1920×1080. The LANCZOS resampling algorithm is used to maintain text clarity.

[0025] Denoising: To reduce the impact of scanning noise and compression artifacts on OCR and projection segmentation, a 5×5 Gaussian kernel is used to smooth the image. The standard deviation σ of the Gaussian kernel is usually set to 1.0.

[0026] Binarization: Converting a color or grayscale image into a black-and-white binary image to enhance the contrast between text and background. Different strategies are used depending on the image type. For document-type images (reports, tables): the Otsu adaptive thresholding method is used to automatically calculate the optimal segmentation threshold.

[0027] For imaging images (CT, MRI): Due to the complex background, a fixed threshold (such as 127) or a locally adaptive threshold is used to ensure that the superimposed text is effectively extracted, while minimizing interference with the original medical image content.

[0028] c) Output: After the above processing, a series of standardized, binary image arrays are obtained, denoted as List[Image_Array], and information such as the original file and page number corresponding to each image is recorded.

[0029] 1.2 Text-based modal preprocessing branch Input data includes: native Word / Excel files, CSV / JSON files exported from a Hospital Information System (HIS) or Electronic Medical Record (EMR), and structured electronic medical records in plain text format. The processing steps are as follows: a) Text extraction and normalization: Use the appropriate libraries (such as python-docx, pandas, json) to extract the original text content.

[0030] Unified encoding: Convert all text content to UTF-8 encoding.

[0031] Cleaning and noise reduction: Remove invisible control characters, extra spaces, and line breaks (preserve meaningful paragraph breaks based on context).

[0032] b) Structured splicing: To facilitate the large language model's understanding of context, the extracted fields are semantically concatenated according to a pre-defined medical document template. For example, the "Name," "Age," and "Diagnosis" fields exported from the database are concatenated into a natural language sentence: "Name: {NAME} Age: {AGE} Diagnosis: {DIAGNOSIS}". Preserving contextual information is crucial for the large model to accurately identify and classify sensitive entities.

[0033] The concatenated text blocks, along with their metadata (source file, field types, line numbers, etc.), are encapsulated into a structured list of JSON objects. For example: [ { "source_id": "emr_record_001", "text_block": "Patient Name: Zhang San; Age: 45; Date of Visit: 2024-05-01", "type": "demographic_info } ] c) Output: The final output is a normalized, structured list of text blocks, denoted as List[Text_Block].

[0034] S2: OCR Recognition and Coarse-grained Coordinate Extraction This step is specifically designed to process image-based modal data, aiming to answer the fundamental question of "where is the text located in the image," thus providing a spatial positioning basis for subsequent precise desensitization.

[0035] a) OCR Engine Selection and Invocation: This invention preferably uses PaddleOCR's PP-OCRv4 model. This model has the advantages of high accuracy and high speed in Chinese scenarios, and supports localized deployment. It is encapsulated as a service, taking the standardized image array output by S1 as input.

[0036] b) Text detection and recognition: Text detection: The model first detects the position of all text lines in the image and outputs the coordinates of the bounding polygon (poly) of each text line. The poly is usually represented by a list of four points [x1,y1, x2,y2, x3,y3, x4,y4], which can accurately define slanted or curved text lines.

[0037] Text recognition: For each detected text line region, the model identifies its text content.

[0038] c) Result Structuring: The detection and recognition results are correlated to generate a structured record for each line of text. The record format is shown in the example below. The confidence field represents the confidence level of the OCR recognition, which is used for subsequent matching error tolerance.

[0039] { "id": "line_001_page_01", / / Unique identifier "page": 1, / / Image page number "text": "Patient Name: Li Si", / / Recognized text content "poly": [[100,200], [500,200], [500,230], [100,230]], / / Text line coordinates (four points) "confidence": 0.98 / / Identify confidence level } d) Output: Output a list of all OCR text items for all image pages, denoted as List[OCR_Item]. This list contains the text content and its coarse-grained (line-level) location information within the image.

[0040] S3: Sensitive Entity Identification, Classification, and Risk Scoring This step acts as the "brain" for desensitization, aiming to answer the core question of "which texts are sensitive, and to what extent." It receives normalized text from S1 and OCR-recognized text from S2, and uses a large language model that has been fine-tuned and deployed locally for semantic understanding and intelligent judgment.

[0041] 3.1 Definition of Sensitive Entity Types and Risk Weights This invention predefines a dictionary of sensitive entity types in the medical field and assigns a quantified risk weight to each type. This definition forms the basis for all subsequent automated decision-making. An example is shown below:

[0042] 3.2 Construction of the Instruction-Based Fine-Tuning Large Language Model (LLM) Base model selection: Large open-source models that support local deployment, such as Qwen-7B and DeepSeek, were chosen. Qwen-7B performs well in Chinese language understanding and command following, has a moderate number of parameters, and is suitable for deployment on hospital-grade hardware.

[0043] Fine-tuning the data construction: Collect or manually annotate a corpus containing medical text (approximately 1000-5000 entries). Each sample includes "medical text" and "standardized sensitive entity annotation JSON". The annotations must cover all predefined entity types and include combined contexts (e.g., name + hospital + rare disease).

[0044] Fine-tuning method: Highly efficient parameter fine-tuning techniques such as LoRA are employed to perform instruction fine-tuning on the base model. The training prompt template is designed as follows, aiming to enable the model to learn to extract structured information from free text and consider risks: Please identify sensitive entities in the following medical text and output them in the specified JSON format.

[0045] The entity type must be one of the following: PATIENT_NAME, ID_NUMBER, MEDICAL_RECORD_NUMBER, HOSPITAL_NAME, RARE_DISEASE_NAME, DEPARTMENT, DATE, TIME, AGE.

[0046] Please also estimate the cumulative privacy risk score for this patient record (calculated based on entity risk weights).

[0047] Text content: [Enter the text to be recognized here] Fine-tuning objectives: to enable the model to: (1) accurately identify various sensitive entities in medical texts; (2) understand the context and make reasonable judgments (for example, "Director Li" in the main text of the report may be a doctor's title rather than a patient's name); and (3) initially develop an awareness of quantitative risk assessment. 3.3 Identification, Classification, and Risk Scoring Process a) Input text preparation: Merge or batch the text fields in List[Text_Block] obtained from S1 and List[OCR_Item] obtained from S2 to form text paragraphs suitable for LLM processing.

[0048] b) Invoking the LLM: Input the prepared text and fixed instruction prompt into the locally deployed, fine-tuned LLM.

[0049] c) Parsing the output: The LLM should return a structured JSON object, as shown in the example below: { "entities": [ { "entity": "Li Si", "type": "PATIENT_NAME", "source_page": 1, "context": "Patient Name: Li Si, Age: 45 years old", "confidence": 0.99 }, { "entity": "45 years old", "type": "AGE", "source_page": 1, "context": "Patient Name: Li Si, Age: 45 years old", "confidence": 0.98 } ], "risk_score": 11.5 } Among them, entities is a list of all identified sensitive entities; source_page records the entity source (image page number or text block ID); risk_score is the key output, representing the patient's cumulative risk score.

[0050] d) Cumulative risk score algorithm: For all data related to a patient (potentially spanning multiple pages and modalities), the cumulative risk score is calculated as follows: Risk score = Σ (Entity risk weight × Number of times the entity appears in the current dataset) For example, if a set of data contains "name" once (weight 10), "age" twice (weight 1.5), and "a certain community hospital" once (weight 4), then the risk score = 101 + 1.52 + 4*1 = 17.

[0051] e) Desensitization level mapping (dynamic decision-making strategy): Based on the calculated risk score, the strictness of desensitization is dynamically determined, achieving a "tiered and differentiated approach": Level 1 (Full Desensitization): When the risk score is ≥ 15, it indicates a high risk of privacy leakage, and desensitization is required for all entities at the strong, medium, and weak sensitivity levels.

[0052] Level 2 (Partial Desensitization): When 8 ≤ Risk Score < 15, the risk is medium. Only highly sensitive and moderately sensitive entities are desensitized, while weakly sensitive entities (such as individual dates) are retained.

[0053] Level 3 (Minimum Desensitization): When the risk score is < 8, the risk is low. Only highly sensitive entities are desensitized, while medium and weakly sensitive information is retained to maximize data availability.

[0054] (Thresholds 15 and 8 are examples and can be configured in the system.) Output: This step ultimately outputs a List[Sensitive_Entity] (containing type, level, and source context) and an integrated risk_score and the mapped desensitization_level.

[0055] S4: Alignment of OCR text with large model output This step serves as a bridge connecting "semantics" (S3) and "location" (S2), aiming to establish a precise mapping relationship between sensitive entities and their physical locations in the original data, thus solving the problem of "which text line (or text block) in the image corresponds to the found sensitive word".

[0056] a) Input: List[OCR_Item] from S2 and List[Sensitive_Entity] from S3.

[0057] b) Exact string matching: Preferred algorithm. Traverse the Sensitive_Entity list, and exactly match its entity field (such as "Li Si") with the text field of each OCR_Item. If the match is successful, establish a mapping: Sensitive_Entity[i] -> OCR_Item[j], and record the start and end character indices of the entity in the text.

[0058] c) Fault-tolerant matching (edit distance): When exact matching fails (mainly due to OCR recognition errors, such as "Li Si" being recognized as "Li Si" or "Li Yue"), use the Levenshtein edit distance to calculate the similarity.

[0059] Set a similarity threshold (such as ≥ 0.9).

[0060] If the similarity = 1 - (edit distance / max(len(entity), len(OCR text))) ≥ threshold, it is determined as a successful match.

[0061] This mechanism effectively improves the robustness of the system to OCR recognition noise.

[0062] d) Multi-position processing: The same sensitive entity (such as patient name) may appear multiple times in the document. The alignment algorithm needs to find all matching positions to ensure no omission.

[0063] Output: An aligned list, where each entry associates the Sensitive_Entity information with the OCR_Item it matches (including its poly coordinates and text content), and records the start and end indices [start_idx, end_idx) of the entity in the text string.

[0064] S5: Character-level coordinate refinement based on vertical projection method This step is the core technical link for the present invention to achieve "pixel-level precise desensitization". Its input is a single text line area (bounded by poly) containing sensitive words obtained after alignment in S4. The goal is to accurately calculate the pixel-level bounding box (bbox) of the two characters "Li Si" (instead of the entire line) in the original image for a text line such as "Patient Name: Li Si". Figure 3 (It can be inferred from the text of the specification) illustrates this principle.

[0065] The specific sub-steps are as follows: S5.1: Cropping and Preprocessing: Based on the poly coordinates of OCR_Item, crop the text line region from the standardized binary image obtained in S1 to obtain crop_img. The image is already a binary image (background is 0 / black, text is 255 / white).

[0066] S5.2: Column projection calculation: Let the height of crop_img be H and the width be W.

[0067] Calculate the projection value of each column j (0 ≤ j < W): count the number of pixels with a pixel value of 255 (text part) in that column, count_j.

[0068] Normalize: v_j = count_j / H. The range of v_j is [0, 1], reflecting the proportion of text in the column.

[0069] S5.3: Projection Curve Smoothing: Due to noise or stroke adhesion, the projection curve may not be smooth. A moving average filter with a window size of 3 is used to smooth the sequence v_j, eliminating small spikes.

[0070] S5.4: Adaptive segmentation threshold determination: Calculate the mean of the smoothed projected values: mean_v = average(v_j).

[0071] Set an adaptive threshold: T = k * mean_v. The coefficient k is a key parameter, and empirically it is usually set between 0.15 and 0.25, such as k=0.2. The threshold T is used to distinguish between the "character body" and the "spaces between characters".

[0072] S5.5: Character Segmentation Gap Recognition: Scan the projection value v_j from left to right.

[0073] When there are n consecutive columns (n ​​≥ 2) where v_j is less than the threshold T, these n columns are considered a valid character segmentation gap. The requirement of n ≥ 2 can prevent missegmentation caused by noise in a single pixel.

[0074] S5.6: Subregion Division and Character Mapping: Based on all the identified segmentation gaps, crop_img is divided into M consecutive "candidate character sub-regions" in the horizontal direction.

[0075] From step S4, we know the start and end indices of the sensitive entity (such as "Li Si") in the OCR text string (let's assume it's [5, 7], corresponding to two characters).

[0076] These M physical sub-regions need to be mapped from left to right to the N characters (logical characters) recognized by the OCR. For monospaced fonts, a simple one-to-one mapping can be used. To handle non-monospaced or contiguous cases, more complex heuristic rules can be used, such as allocating them according to the width ratio of the sub-regions.

[0077] Finally, the two specific sub-regions corresponding to index [5, 7] were determined (let's assume they are the 3rd and 4th sub-regions).

[0078] S5.7: Coordinate Transformation: Get the relative coordinates [x_left, x_right] of these two sub-regions within crop_img.

[0079] By combining the poly coordinates of OCR_Item (i.e., the position of crop_img in the original image), the coordinates of the sub-region are accurately mapped back to the pixel coordinate system of the original normalized image through affine transformation or poly-based quadrilateral correction (if the text line is skewed), resulting in the final pixel-level bounding box: bbox = [x1_global, y1_global, x2_global, y2_global].

[0080] S5.8: Exception handling mechanism (enhanced robustness): Character overlap: If the number of sub-regions M is significantly less than the number of logical characters N, it indicates that overlap has occurred. In this case, you can try lowering the threshold T (e.g., k=0.1) and re-segmenting, or divide the width evenly according to the number of characters N.

[0081] Text line skewing: Before cropping in S5.1, you can use Hough transform or the minimum bounding rectangle of poly to detect and correct the skewing angle of the text lines to ensure that the projection is in the horizontal direction.

[0082] Insufficient whitespace: For compact fonts, insufficient whitespace between characters may lead to segmentation failure. This can be addressed by reducing T or introducing morphological processing (erosion first).

[0083] Output: For each sensitive entity requiring desensitization, output a list of one or more pixel-level bounding boxes in the original image. This coordinate precision has been improved from "text line level" to "character level," which is a prerequisite for achieving precise occlusion without destroying surrounding valid information.

[0084] S6: Cross-modal entity association and consistency verification This step aims to address the issues of "unified strategy" and "consistency" in collaborative desensitization of multimodal data, ensuring that all information of the same patient, regardless of the file format, is desensitized according to the same set of standards, leaving no blind spots.

[0085] a) Patient node construction: Core identifier (primary key): Typically, the medical record number (MEDICAL_RECORD_NUMBER) or patient ID (ID_NUMBER) is chosen. Ideally, these two identifiers are globally unique and stable.

[0086] Auxiliary identifier: Patient name (PATIENT_NAME). This is used as an auxiliary verification because there may be duplicate names or OCR misidentification.

[0087] Association Process: Iterate through the list of Sensitive Entities identified in step S3 from all modalities, including DICOM images, PDF reports, and structured text. Aggregate entities with the same core identifier under the same "Patient Node". If no core identifier is available, attempt to perform fuzzy association using auxiliary identifiers combined with Levenshtein similarity (threshold, e.g., ≥0.95).

[0088] b) Unified desensitization strategy: For all sensitive entities aggregated under the same patient node, recalculate the global cumulative risk score (across all modalities).

[0089] Based on this global risk score, a unified, highest-level desensitization strategy is determined according to the mapping rules defined in S3. For example, if the risk score in the DICOM image is 12 (strong + medium desensitization), and the risk score in the associated electronic medical record is 20 (full desensitization), then all data for that patient node will ultimately be subject to the "full desensitization" strategy.

[0090] c) Cross-modal consistency verification: Verification 1 (Gap Filling): Check if a highly sensitive entity (such as a name) that should have been identified in a certain modality was actually not identified. For example, the DICOM header information was successfully desensitized, but the overlaid name in the image was not recognized due to OCR failure. In this case, the system will check the identified names in other modalities (such as report text). If the name is highly similar to the blurred OCR result of the suspected unrecognized area in the current modality, then forced desensitization will be performed at that position in the current modality.

[0091] Verification 2 (Synchronization): Ensure that the desensitization treatment (whether to cover, blur, or retain) of the same core identifier (such as medical record number) is consistent across all associated modalities to avoid contradictions.

[0092] Output: Output a list of desensitization tasks centered around the patient node. Each task specifies the target data, the sensitive entities to be processed (and their precise coordinates or text locations from S5), and the unified desensitization_level.

[0093] S7: Layered Precise Desensitization Execution This step is the stage for making the final physical modifications based on all the aforementioned analysis and decisions. It receives precise pixel coordinates from S5, entity sensitivity levels and desensitization level strategies from S3 / S6, and performs differentiated operations.

[0094] For image-based modalities (including DICOM pixel data and PDF image layers): Highly sensitive entities: Perform rectangular fill coverage within the corresponding bounding box area in the image. The preferred fill method is white (RGB: 255,255,255) to simulate the effect of paper covering. To improve visual appeal, rounded rectangle fill is supported, with a configurable corner radius, typically between 1 and 10 pixels, for example, set to 5 pixels.

[0095] Medium-sensitive entities: Apply Gaussian blur within the bounding box region. Use strong parameters, such as kernel size (15, 15), standard deviation sigma=5, and perform multiple rounds (e.g., 3 rounds) of blurring to achieve sufficient information concealment while still retaining traces of the field's existence.

[0096] Weakly sensitive entities: Within the bounding box region, perform a Gaussian blur. Use parameters with low intensity, such as kernel size (9, 9), standard deviation sigma=3, and perform a single round of blurring. At the "Partial Desensitization" level, weakly sensitive entities may be preserved without any action.

[0097] Optional advanced processing: In addition to overlay and blur, the system can be configured with other processing methods, such as: Pixelation (mosaic): Divide the area into 8×8 or 16×16 blocks and fill them with the average color within each block.

[0098] Generative inpainting: Using image inpainting models such as GANs, non-sensitive content that conforms to the context is generated based on the surrounding texture and then filled in, resulting in a more natural visual effect (but it requires higher computing power).

[0099] For the header information (metadata) of DICOM images: Use the pydicom library to directly read and modify DICOM tags.

[0100] Highly sensitive tags: such as (0010, 0010) PatientName, (0010, 0020) PatientID, directly replace their values ​​with general placeholders, such as "ANONYMIZED" or "XXXX".

[0101] Medium / weakly sensitive tags: such as (0008, 0080) InstitutionName, can be obfuscated (e.g., replaced with "a certain hospital") or retained, depending on the strategy.

[0102] Synchronization verification: Ensure that the information desensitized in the DICOM header is consistent with the overlay text content that has been identified and desensitized by OCR from the pixels of the DICOM image.

[0103] For text-based modalities: Highly sensitive fields: directly replace with predefined placeholders, such as "[Name]" or "[ID Number]".

[0104] Medium / weakly sensitive fields: Processed according to the global desensitization level strategy. For example, under "full desensitization", the date "2024-05-01" may be obfuscated to "2024-05-XX"; the hospital name "XX City First People's Hospital" may be obfuscated to "XX City A Certain Hospital". Under "partial desensitization", these fields may be retained.

[0105] Output: An anonymized image array and modified structured text data in memory.

[0106] S8: Results Export and Log Recording This step is the end of the process, responsible for persisting the processing results and providing a complete audit trail.

[0107] a) Data restoration and export: The anonymized image array is re-encoded and packaged into a file of the appropriate format according to the original input format.

[0108] Image sequence -> Multi-page PDF file.

[0109] DICOM pixel data + modified header information -> new DICOM file.

[0110] Modified JSON / text data -> Restore to CSV, Word, or original format.

[0111] All de-identified files should be named in a clear manner (e.g., original filename_de-identified.suffix) and output to the specified directory.

[0112] b) Audit log generation: Generate a structured log file (such as JSON format).

[0113] The log records in detail: the original file processed, a list of all identified sensitive entities, the source (page number / location) of each entity, the sensitivity level, the processing method adopted (such as "white rectangle filling"), and key parameters of the processing (such as risk_score, desensitization_level).

[0114] c) Output: The final de-identified data file, and a complete de-identification audit log file.

[0115] Through the detailed implementation of the above eight steps, this invention realizes the entire process of privacy desensitization, from multimodal medical data input to localization, intelligence, precision, and consistency.

[0116] This technical solution achieves the following: 1. End-to-end automated and precise data masking: It provides a complete and automated solution from multimodal data input to masking result output, significantly improving the efficiency and standardization of medical data masking processing. 2. Overcoming modal fragmentation: The unified workflow framework can process different modal data such as images and text in parallel or serially, and achieve correlation through subsequent steps (S6), overcoming the limitations of traditional single-modal masking tools. 2. Enhanced System Compatibility and Robustness: Adaptable to various common medical data formats such as DICOM, scanned / native PDF, Word, and structured text. Preprocessing unifies the data format, providing stable and standardized input for subsequent steps. 2. Retention of Key Information: Preprocessing of DICOM images specifically considers removing display parameter interference and extracting overlaid text, ensuring that no diagnostic information is lost. 3. Achieve intelligent and differentiated privacy protection: It changes the traditional "one-size-fits-all" de-identification model, dynamically adjusting the de-identification intensity based on the actual privacy risk level of the data, making it more scientific and reasonable. 2. Optimize the balance between data utility and privacy: It avoids the damage to data usability (such as research and statistical analysis value) caused by excessive de-identification, maximizing the preservation of data's reuse value while meeting compliance requirements; 4. Revolutionary Improvement in Desensitization Precision: Achieves a leap in desensitization granularity from "line-level" to "character-level." It can precisely mask "Zhang San" in "Name: Zhang San," while completely preserving non-sensitive fields such as "Name:," greatly reducing damage to the original layout information. 2. Strong Technical Applicability: Provides complete engineering details including anomaly handling (character concatenation, line skewing), making the algorithm stable, reliable, and adaptable to the complex layout situations in real medical documents; 5. Ensure the consistency and thoroughness of data anonymization: Effectively prevent sensitive information from being missed due to data being scattered across different files, significantly reducing the risk of privacy leaks; 6. Layered Visual Processing: Different visual processing methods are used for information with different levels of sensitivity (e.g., strong sensitivity - complete coverage, weak sensitivity - slight blurring), which provides data users with intuitive prompts about the sensitivity of information while protecting privacy; 7. Improve the appearance and readability of desensitized documents: Use rounded rectangles and other processing methods to make the obscured areas blend more naturally with the document background, reduce the visual abruptness caused by rough mosaics or black blocks, and improve the reading experience of desensitized documents. Example

[0117] Based on the above embodiment 1, the system architecture, specific deployment method, and recommended hardware configuration for implementing the above method are as follows: 1. Modular System Architecture like Figure 1 As shown, the system of this invention can be deployed on a local server and includes the following core functional modules, which interact with each other through internal APIs or message queues: Input and Management Module: Provides a web interface or API to receive user-uploaded or specified file paths and manage the de-identification task queue, priority, and status.

[0118] Multimodal preprocessing module: Implements the S1 function and contains two sub-processors for images and text.

[0119] OCR recognition module: Implements S2 functionality and encapsulates PP-OCRv4 model services.

[0120] Sensitive Entity Recognition and Classification Module: Implements S3 functions, hosts and calls locally modified LLMs (such as Qwen-7B-LoRA).

[0121] OCR and Large Model Output Alignment Module: Implements S4 functionality.

[0122] Coordinate Refinement Module: Implements S5 functionality, with the core being the vertical projection algorithm engine.

[0123] Cross-modal association and consistency verification module: Implements S6 functionality.

[0124] Desensitization Execution Module: Implements S7 functions and integrates image processing library (OpenCV) and DICOM processing library.

[0125] Configuration and Policy Management Module: Provides a graphical interface that allows administrators to customize sensitive entity dictionaries, risk weights, de-identification thresholds (first threshold, second threshold), and de-identification styles (fill color, blur intensity, corner radius, etc.). This is crucial for the system to adapt to different hospital security strategies.

[0126] Results export module: Implements S8 functionality.

[0127] 2. Deployment Interface Definition To facilitate integration into existing hospital IT systems (such as PACS and research platforms), the system provides standardized RESTful API interfaces and can be packaged into Docker containers.

[0128] Docker deployment: The entire system and all its dependencies (Python environment, model files, libraries) are packaged into a single Docker image. The hospital's IT department only needs to install the Docker engine on a local server to start the data masking service with a single command, greatly simplifying deployment and maintenance.

[0129] RESTful API: Request endpoint: POST / api / v1 / desensitize Input request body: { "file_path": " / data / uploads / patient_001 / CT_and_Report.zip", / / Input file or directory path "strategy_id": "hospital_default", / / The ID of the de-identification strategy used, read from the configuration module. "output_dir": " / data / desensitized / " / / Output directory } Output response body: { "status": "success", "task_id": "task_20240501123456", "desensitized_files": [" / data / desensitized / CT_anon.dcm", " / data / desensitized / Report_anon.pdf"], "log_file": " / data / desensitized / desensitize_log_task_20240501123456.json" } 3. Recommended Hardware Configuration The following configuration ensures efficient system operation when processing high-resolution images and complex documents (refer to the implementation effect data in the handover document): CPU: Intel Core i9-13900K or AMD Ryzen 9 7950X and above, used for data preprocessing, workflow scheduling and OCR pre- and post-processing.

[0130] GPU: NVIDIA GeForce RTX 4090 (24GB VRAM) or NVIDIA RTX 3090 (24GB). Large VRAM is crucial for loading large language models (approximately 14GB for the Qwen-7B) and processing high-resolution images simultaneously. This is the core hardware for local deployment.

[0131] Memory: 64GB DDR5. When processing large amounts of data or complex documents, peak system memory usage may reach 30-40GB.

[0132] Storage: 1TB NVMe SSD for fast read and write of model files, temporary data and results.

[0133] Operating system: Ubuntu 22.04 LTS or Rocky Linux 8.x.

[0134] Optional expansion: For large-scale batch processing, a cluster mode of multiple servers or a single server with multiple GPUs can be used for parallel processing to improve throughput.

[0135] This technical solution achieves the following: 1. Meets the stringent compliance requirements of the healthcare industry: A complete localized deployment solution ensures that sensitive medical data remains within the institution's internal network throughout the entire process, complying with the data localization security requirements of regulations such as the Personal Information Protection Law. 2. Facilitates system integration and maintenance: Modular design reduces system complexity, facilitating maintenance and upgrades; Docker containerization and RESTful API interfaces enable easy integration into existing hospital PACS, RIS, or data platform systems, offering flexible and convenient deployment. Example

[0136] As a preferred embodiment, the system further includes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the localized intelligent de-identification method for multimodal medical data as described in the above embodiments.

[0137] It can be understood that if the methods in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this application, in essence, or the parts that contribute to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0138] In this embodiment, a computer program is stored on a computer-readable storage medium. When the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented. Embodiment

[0139] The following demonstrates how this method is actually applied through two typical scenarios.

[0140] Scenario 1: Desensitization of a single scanned PDF medical examination report (focusing on document image processing) Input: A scanned "Medical Examination Report.pdf", with 3 pages, containing patient personal information, examination conclusions, and doctor's handwritten signatures (all in image form).

[0141] Process: S1: The system identifies it as an image modality and converts it into 3 standardized binary images of 1080p.

[0142] S2: Perform OCR on the 3 images to identify all text lines and their coordinates. For example, identify: "Name: Geng Hongzhen", "Age: 45 years old", "Company: XX Technology Co., Ltd.".

[0143] S3: Input the OCR text into the local LLM. The LLM identifies: {"entity": "Geng Hongzhen", "type": "PATIENT_NAME",...} (weight 10), {"entity": "45 years old", "type": "AGE",...} (weight 1.5). After calculation, the risk score of this report is 10 + 1.5 = 11.5.

[0144] S4 / S5: The system precisely matches "Geng Hongzhen" to the OCR text line and locates the exact pixel boxes of the three characters "Geng", "Hong", and "Zhen" in the second-page image through the vertical projection method.

[0145] S6: Since it is a single file, no complex cross-modal association is required.

[0146] S7: According to the risk score of 11.5 (belonging to the level of 8 ≤ risk score < 15), execute the "strong desensitization + medium sensitivity" strategy. Perform filling with a white rounded rectangle (radius 5px) on the three character boxes of "Geng Hongzhen" (strong sensitivity). Decide whether to retain or blur "45 years old" (weak sensitivity) and "XX Technology Co., Ltd." (if defined as medium sensitivity) according to the strategy. Since the risk score does not reach 15, "45 years old" is temporarily retained here, and Gaussian blur (kernel 15x15, σ = 5) is performed on "XX Technology Co., Ltd.".

[0147] S8: Generate "physical_examination_report_desensitized.pdf" and a JSON log, recording the covering operation on "Geng Hongzhen".

[0148] Effect: The patient's name in the report is accurately covered, and the unit and age information are differentially processed according to the risk assessment results, which not only protects privacy but also retains some valuable anonymized information for scientific research.

[0149] Scenario 2: Collaborative desensitization of DICOM images and structured electronic medical records (focusing on multi-modal association) Input: A DICOM file of a chest CT with the words "Name: Zhang San" and "Medical Record Number: 202405001" burned on its pixel data.

[0150] An electronic medical record text record associated with this CT examination, with the content: Name: Zhang San, Medical Record Number: 202405001, Diagnosis: Pulmonary nodule, Suggested follow-up, Hospital visited: Peking Union Medical College Hospital.

[0151] Process: S1: The DICOM file enters the image branch, extracts the pixel matrix and normalizes it; the electronic medical record enters the text branch, normalizes and splices it.

[0152] S2: Perform OCR on the DICOM image to identify the superimposed name, medical record number text and coordinates.

[0153] S3: The LLM analyzes the OCR text and the electronic medical record text simultaneously and identifies: Zhang San (PATIENT_NAME), 202405001 (MEDICAL_RECORD_NUMBER), Peking Union Medical College Hospital (HOSPITAL_NAME). Cumulative risk score = 10 + 9 + 4 = 23.

[0154] S4 / S5: Match the entities and use the vertical projection method to locate the exact coordinates of characters such as "Zhang", "San", "2", "0"... on the DICOM image.

[0155] S6 (Key step): Using the medical record number (202405001) as the primary key, the system associates the DICOM images and the electronic medical record text to the same "patient node". Since the global risk score 23 ≥ 15, it is determined to execute the full-scale desensitization strategy on all data of this node. This definition is for all subsequent automatic decisions

[0156] S7: DICOM image part: Fill the character coordinates of "Zhang San" and "202405001" with white rectangles. There is no modification to the medical content of the image itself.

[0157] DICOM header: Use pydicom to replace the values of the PatientName and PatientID tags with "ANONYMIZED".

[0158] Electronic medical record text: Replace "Zhang San" with "[Name]", "202405001" with "[Medical record number]", and blur "Peking Union Medical College Hospital" to "[A top three hospital]" (under full-scale desensitization, moderately sensitive entities also need to be processed).

[0159] Consistency check: The system checks whether the desensitization information in the DICOM header is consistent with the covered text in the image (both are anonymized), and ensures that the information with the same name and number in the electronic medical record is also processed.

[0160] S8: Output the desensitized DICOM file, the desensitized electronic medical record text file, and a detailed cross-modal desensitization audit log.

[0161] Effect: Achieved synchronous, collaborative, and non-omissive desensitization of the image data and text data of the same patient. Even if the data is scattered in different modalities and different files, it can be uniformly discovered and processed through the association mechanism, greatly improving the reliability and security of the overall desensitization.

[0162] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

[0163] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A localized intelligent de-identification method for multimodal medical data, characterized in that: Includes the following steps: S1: Multimodal data multi-branch unified preprocessing, select the corresponding image or text preprocessing branch according to the input data type, and output a standardized image array or a normalized text block list; S2: Perform optical character recognition (OCR) on image-type modal data to obtain the text content and its coarse-grained coordinates in the image; S3: Input the OCR text identified in step S2 and / or the normalized text preprocessed in step S1 into a large language model LLM that has been fine-tuned by instructions and deployed locally, identify and output a structured sensitive entity list in a hierarchical manner, and the LLM outputs the corresponding desensitization level based on the cumulative risk scoring algorithm; S4: Perform string matching and alignment between the sensitive entities identified by LLM in step S3 and the text lines identified by OCR in step S2 or the text blocks in step S1 to establish a mapping relationship between sensitive entities and location information. S5: For the text line region containing sensitive entities matched in step S4, the vertical projection method is used to refine the character-level coordinates and obtain the pixel-level bounding box coordinates of the sensitive characters in the original image. S6: Based on the patient's core identifier, aggregate sensitive entities in different modal data to the same patient node, recalculate the cross-modal global risk score and determine a unified desensitization level, and perform cross-modal consistency verification; S7: Based on the desensitization level determined in step S3 and the precise coordinates obtained in step S5, perform layered precise desensitization processing on the corresponding pixel areas in the image, and perform replacement or blurring processing on the corresponding fields in the text data. S8: Output the de-identified multimodal data file and the corresponding de-identification log.

2. The localized intelligent de-identification method for multimodal medical data according to claim 1, characterized in that: In step S1, the image preprocessing branch includes: removing window width and window level interference from DICOM images and extracting superimposed text; converting scanned PDFs into image sequences; and performing resolution normalization, noise reduction, and binarization on all images; the text preprocessing branch includes: unified encoding, removal of special characters, and conversion into structured JSON text blocks.

3. The localized intelligent de-identification method for multimodal medical data according to claim 1, characterized in that: In step S3, the cumulative risk scoring algorithm is as follows: Patient record cumulative risk score = Σ (preset risk weight for each sensitive entity × number of times the entity appears); And based on the cumulative risk score, the desensitization level is mapped, including: When the risk score is greater than or equal to the first threshold, full data anonymization is performed. When the risk score is between the second threshold and the first threshold, only highly sensitive and moderately sensitive entities are desensitized. When the risk score is less than the second threshold, only highly sensitive entities are desensitized; Its first threshold is greater than the second threshold.

4. The localized intelligent de-identification method for multimodal medical data according to claim 1, characterized in that: The vertical projection method in step S5 specifically includes the following sub-steps: S5.1: Perform grayscale and binarization processing on the cropped image of text lines containing sensitive entities; S5.2: Project the binarized image column by column, calculate and normalize the projection value of each column; S5.3: Use a sliding window to smooth column projection values; S5.4: Calculate the adaptive segmentation threshold based on the mean of column projection values; S5.5: Identify consecutive columns with projection values ​​lower than the segmentation threshold as character segmentation gaps; S5.6: Divide the character sub-regions according to the segmentation gaps, and map the sub-regions to the character order in the OCR text; S5.7: Based on the start and end indices of the sensitive entity in the OCR text, locate the corresponding character sub-region and transform its coordinates back to the original image coordinate system to obtain pixel-level bounding boxes.

5. A localized intelligent de-identification method for multimodal medical data according to claim 1, characterized in that: In step S6, the core identifier includes at least one of medical record number and patient ID, and the auxiliary identifier includes patient name; by matching the primary key and the auxiliary key similarity, sensitive entities identified from DICOM images, document images, and structured text are associated with the same patient node.

6. The localized intelligent de-identification method for multimodal medical data according to claim 1, characterized in that: In step S7, the layered precise desensitization process performs different operations on sensitive areas in the image according to their sensitivity level: rectangular filling is performed on areas corresponding to highly sensitive entities, Gaussian blur with a first parameter is performed on areas corresponding to medium-sensitive entities, and Gaussian blur with a second parameter is performed on areas corresponding to weakly sensitive entities; the blur intensity of the first parameter is higher than that of the second parameter.

7. A localized intelligent de-identification method for multimodal medical data according to claim 6, characterized in that: The rectangle is filled with white and supports rounded rectangles with a radius of 1 to 10 pixels.

8. A localized intelligent de-identification system for multimodal medical data, employing the method described in any one of claims 1-7, characterized in that: This includes the following modules deployed on the local hardware environment: The input and management module is used to receive multimodal medical data input and implement desensitization strategies. A multimodal preprocessing module, connected to the input and management module, is used to perform unified preprocessing across multiple branches; The OCR recognition module is connected to the multimodal preprocessing module and is used to perform optical character recognition on image data and output text and coordinates. The sensitive entity identification and classification module connects the multimodal preprocessing module and the OCR recognition module, and is used to call the local large language model to identify and classify sensitive entities. The OCR text and large model output alignment module connects the OCR recognition module and the sensitive entity recognition and classification module, and is used to perform entity-position mapping alignment. A coordinate refinement module, connecting the alignment module and the OCR recognition module, is used to obtain character-level precise coordinates through the vertical projection method; The cross-modal association and consistency verification module is connected to the sensitive entity identification and classification module, and is used to construct patient nodes and unify desensitization strategies; The desensitization execution module, connected to the coordinate refinement module and the cross-modal association and consistency verification module, is used to perform hierarchical and precise desensitization. The result export module is connected to the de-identification execution module and is used to output the de-identified data and logs.

9. A localized intelligent de-identification system for multimodal medical data according to claim 8, characterized in that: The system is containerized using Docker and provides a RESTful API interface. The interface receives an input request containing file paths and policy identifiers and returns the anonymized file paths and log file paths.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the localized intelligent desensitization method for multimodal medical data as described in any one of claims 1 to 7.

Images