Medical big data query method and system based on dynamic desensitization, terminal and medium
By constructing an adaptive isomorphic index and using differentiated privacy protection technology, the problems of privacy leakage and data availability in medical big data caused by traditional static de-identification technology are solved. Dynamic adaptive privacy protection and efficient data access are achieved, improving the query performance and security of medical data.
Patent Information
- Application Number
- CN202511305397.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-12
- Publication Date
- 2026-01-13
AI Technical Summary
Traditional static desensitization techniques in medical big data suffer from excessive desensitization, which damages the statistical analysis value of the data and poses a risk of privacy leakage. They are also difficult to achieve granular and dynamically adaptive privacy protection, and traditional indexing mechanisms are unable to cope with the access needs of high-frequency and low-frequency data, resulting in response delays or wasted storage space.
A dynamic desensitization approach is adopted. By constructing an adaptive isomorphic index structure based on pathological semantics, and utilizing semantic hash keys and embedded vector indexes, combined with techniques such as Laplacian noise addition, deep learning desensitization, and k-anonymization, the index and desensitization strategies are dynamically adjusted according to access permissions to achieve differentiated privacy protection.
While ensuring data security, it improves the availability and access performance of medical data, ensures the integrity and statistical characteristics of data in diagnosis and treatment and scientific research scenarios, and achieves millisecond-level high-frequency data retrieval and reduces low-frequency data storage overhead.
Smart Images

Figure CN121326976A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data query, specifically to a method, system, terminal, and medium for querying medical big data based on dynamic desensitization. Background Technology
[0002] Electronic medical records (EMR), laboratory information management systems (LIS), and picture archiving and communication systems (PACS) generate massive amounts of multi-source heterogeneous medical data. This data contains a large amount of highly sensitive information such as patients' personal identity information, health status, and medical records. Traditional static desensitization techniques can lead to over-desensitization, which can seriously impair the statistical analysis value and clinical research utility of the data. On the other hand, insufficient desensitization can lead to huge privacy leaks and compliance risks. It is difficult to provide granular and dynamically adaptive privacy protection solutions in different application scenarios, and it is impossible to simultaneously meet the dual requirements of data security and data availability.
[0003] Furthermore, medical data is characterized by high heterogeneity and uneven access patterns. Most relevant retrieval solutions employ fixed-structure indexing mechanisms, which are ineffective against such mixed loads. For high-frequency access demands, fixed indexes may cause response delays due to excessive depth or improper node splitting; for massive amounts of low-frequency data, index redundancy may lead to significant waste of storage space. Moreover, the traditional approach of de-identifying data before indexing often results in a significant decrease in indexing efficiency and query accuracy because the de-identification process destroys the original semantic features of the data. Summary of the Invention
[0004] To address the aforementioned issues, this invention provides a method, system, terminal, and medium for querying medical big data based on dynamic anonymization, which improves the availability and access performance of medical data while ensuring data security and compliance.
[0005] In a first aspect, the technical solution of the present invention provides a method for querying medical big data based on dynamic desensitization, comprising the following steps: Collect multi-source heterogeneous medical data from multiple medical systems, preprocess the multi-source heterogeneous medical data, including semantic normalization based on coding system and medical terminology set, construct unique patient identifiers to associate scattered data, use cleaning rule base to remove duplicate and abnormal data, and generate standardized intermediate plaintext data. An adaptive isomorphic index structure based on pathological semantics is constructed based on standardized intermediate plaintext data. Specifically, a B+ tree index based on semantic hash keys is constructed for structured data, and features are extracted from unstructured data to generate embedding vectors. An index for content retrieval is then constructed based on the embedding vectors. The B+ tree index is dynamically layered according to the access frequency of data items. Receive data query requests, extract the access permission context of the query requests, use an adaptive isomorphic index structure to locate and read the corresponding intermediate plaintext data, and perform differentiated privacy protection on the read intermediate plaintext data according to the access permission context to generate de-identified results that match the permissions. The differentiated privacy protection includes: for research-level access, structured data is desensitized using a Laplacian noise addition mechanism, unstructured medical image data is desensitized using a deep learning-based sensitive region detection and blurring algorithm, and unstructured medical text data is desensitized by generalization or removal of quasi-identifiers; the noise parameters of the Laplacian noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm; for clinical-level access, structured data is desensitized using a combination of k-anonymization and direct identifier masking, unstructured medical image data is not desensitized, and unstructured medical text data is desensitized by masking direct identifiers.
[0006] In an optional implementation, a B+ tree index based on semantic hash keys is constructed on the structured data, specifically including: Extract disease classification codes, medical terminology concept vectors, and patient unique identifiers from standardized intermediate plaintext data; The disease classification code and the medical term concept vector are concatenated and input into the first hash function to generate the pathological topological hash value. The patient's unique identifier is concatenated with the data timestamp and input into the second hash function to generate the patient's time-series hash value; The pathological topological hash value is coupled with the patient temporal hash value to generate the final semantic hash key; A B+ tree index structure is constructed based on semantic hash keys, where the semantic hash keys serve as the keys of the B+ tree, and the physical address of the data corresponding to the key value in the storage system serves as the leaf node storage value of the B+ tree.
[0007] In one optional implementation, features are extracted from unstructured data to generate embedding vectors, and an index for content retrieval is constructed based on these embedding vectors, specifically including: High-dimensional feature vectors of unstructured medical image data are extracted using a pre-trained deep convolutional neural network. The high-dimensional feature vector is scalar quantized to obtain the embedding vector; The quantized embedding vector is used as the index value to construct a vector index based on approximate nearest neighbor search.
[0008] In an optional implementation, the method further includes: The B+ tree index is periodically collected within a time window. The access logs within the node provide statistics for each node. Frequency of access and average query response time ; Based on the access frequency, calculate the access popularity weight of each node in the B+ tree. :
[0009] in, As a weighting adjustment factor, For time window The highest access frequency among all nodes; If a node's access popularity weight The value is higher than the preset upper threshold for several consecutive time windows. If a node is identified as a hotspot, its tree depth is reduced by increasing the branch factor B of its child nodes. If a node's access popularity weight The value is below the preset lower threshold for several consecutive time windows. If it is a cold node, its branch factor B is reduced and a node merging operation is performed.
[0010] In an optional implementation, differential privacy protection is performed on the read intermediate plaintext data according to the access permission context to generate a de-identified result matching the permissions, specifically including: Parse the access permission context to obtain the role and purpose of the requesting entity; Based on the role and purpose of the requesting entity, and according to the preset privacy protection policy mapping rules, the corresponding de-identification execution algorithm is selected. The mapping rules are as follows: if the role is a researcher and the purpose of access is statistical analysis, then a research-level permission policy is matched; if the role is a clinician and the purpose of access is medical services, then a medical-level permission policy is matched. Specifically, when matching a research-level permission policy, Laplace noise is added to the numerical medical data fields in the structured data; the scale parameter of the Laplace noise... Privacy Budget Decide.
[0011] In an optional implementation, the noise parameters of the Laplace noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm, specifically including: using a differential privacy budget allocation mechanism based on multi-objective reinforcement learning, taking the AUC decline rate of the clinical efficacy prediction model as a constraint loss function, and dynamically adjusting the Laplace noise parameters.
[0012] In an optional implementation, the method further includes: Perform benchmark query tests periodically to calculate the current average retrieval accuracy of the vector index. Compared to the current average query time The average retrieval accuracy By calculating the previous results in the query results The overlap of the nearest real neighbors is used as a metric. Vector indexing employs a product quantization algorithm, whose index parameters include the codebook size. Number of subspaces Each benchmark query test dynamically adjusts index parameters by solving the following optimization problem.
[0013] in, For parameter combination Average query time under, This represents the average retrieval accuracy under the corresponding parameter combinations. This is the minimum required precision threshold; After receiving a data query request, the size of the candidate set to be scanned is determined using the following formula. ,
[0014] in, Based on the size of the candidate set, The target response time for the current query request. This represents the average query time under the latest index parameters.
[0015] Secondly, the technical solution of the present invention provides a medical big data query system based on dynamic desensitization, comprising: The medical data acquisition module is used to collect multi-source heterogeneous medical data from multiple medical systems, preprocess the multi-source heterogeneous medical data, including semantic normalization based on the coding system and medical terminology set, construct a unique patient identifier to associate scattered data, use a cleaning rule base to remove duplicate and abnormal data, and generate standardized intermediate plaintext data. The index structure construction module is used to construct an adaptive isomorphic index structure based on pathological semantics based on standardized intermediate plaintext data. Specifically, it constructs a B+ tree index based on semantic hash keys for structured data, extracts features from unstructured data to generate embedding vectors, and constructs an index for content retrieval based on the embedding vectors. The B+ tree index is dynamically layered according to the access frequency of data items. The data query execution module receives data query requests, extracts the access permission context of the query requests, locates and reads the corresponding intermediate plaintext data using an adaptive isomorphic index structure, and performs differentiated privacy protection on the read intermediate plaintext data according to the access permission context, generating desensitized results that match the permissions. The differentiated privacy protection includes: for research-level access permissions, a Laplace noise addition mechanism is used to desensitize structured data; a deep learning-based sensitive region detection and blurring algorithm is used to desensitize unstructured medical image data; and a generalization or removal of quasi-identifiers is used to desensitize unstructured medical text data. The noise parameters of the Laplace noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm. For clinical-level access permissions, a combination of k-anonymization and direct identifier masking is used to desensitize structured data; unstructured medical image data is not desensitized; and direct identifiers are masked for unstructured medical text data.
[0016] Thirdly, the technical solution of the present invention provides a terminal, comprising: Storage device for storing medical big data query programs based on dynamic desensitization; A processor is configured to implement the steps of the medical big data query method based on dynamic desensitization as described above when executing the medical big data query program based on dynamic desensitization.
[0017] Fourthly, the present invention provides a computer-readable storage medium storing a medical big data query program based on dynamic desensitization, wherein the medical big data query program based on dynamic desensitization, when executed by a processor, implements the steps of the medical big data query method based on dynamic desensitization as described in any of the above claims.
[0018] As can be seen from the above technical solutions, this application has the following advantages: By using a dynamic desensitization mechanism based on access permission context, it overcomes the drawbacks of traditional static desensitization. For clinical scenarios, k-anonymization and identifier masking are employed to protect the patient's direct identity information while ensuring the integrity of the data required for diagnosis and treatment. For research scenarios, differential privacy technology based on multi-objective reinforcement learning to optimize noise parameters is used to maximize the preservation of statistical characteristics and clinical research value of the data while meeting strict privacy budget constraints, thus achieving precise on-demand adaptation of security and usability. By constructing an adaptive isomorphic index structure based on pathological semantics, data content and index key values are integrated. Indexing semantic hash keys generated from structured data and feature embedding vectors extracted from unstructured data imbues the index itself with rich semantic information, improving query accuracy. Simultaneously, by periodically adjusting the B+ tree branching factor, node layout, and vector index parameters according to data access frequency, the index structure can adapt to changes in business load, while efficiently managing high-frequency and low-frequency data, achieving millisecond-level high-frequency data retrieval response and significantly reducing the storage overhead of low-frequency data. Attached Figure Description
[0019] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of a medical big data query method based on dynamic desensitization provided in an embodiment of the present invention.
[0021] Figure 2 This is a schematic block diagram of a medical big data query system based on dynamic desensitization, provided as an embodiment of the present invention.
[0022] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation
[0023] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
[0025] Figure 1 This is a schematic flowchart illustrating a medical big data query method based on dynamic desensitization, provided in an embodiment of the present invention. Wherein, Figure 1 The executing entity can be a medical big data query system based on dynamic anonymization. The medical big data query method based on dynamic anonymization provided in this embodiment of the invention is executed by a computer device; correspondingly, the medical big data query system based on dynamic anonymization runs on the computer device. Depending on different needs, the order of the steps in this flowchart can be changed, and some steps can be omitted.
[0026] like Figure 1 As shown, the method includes the following steps.
[0027] S1 collects multi-source heterogeneous medical data from multiple medical systems, preprocesses the multi-source heterogeneous medical data, including semantic normalization based on coding system and medical terminology set, constructs unique patient identifiers to associate scattered data, uses cleaning rule base to remove duplicate and abnormal data, and generates standardized intermediate plaintext data.
[0028] S2, constructing an adaptive isomorphic index structure based on pathological semantics based on standardized intermediate plaintext data; wherein, a B+ tree index based on semantic hash keys is constructed for structured data, features are extracted from unstructured data to generate embedding vectors, and an index for content retrieval is constructed based on the embedding vectors.
[0029] S3 receives a data query request, extracts the access permission context of the query request, uses an adaptive isomorphic index structure to locate and read the corresponding intermediate plaintext data, and performs differentiated privacy protection on the read intermediate plaintext data according to the access permission context, generating a de-identified result that matches the permissions.
[0030] The differentiated privacy protection includes: for research-level access, structured data is desensitized using a Laplacian noise addition mechanism, unstructured medical image data is desensitized using a deep learning-based sensitive region detection and blurring algorithm, and unstructured medical text data is desensitized by generalization or removal of quasi-identifiers; the noise parameters of the Laplacian noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm; for clinical-level access, structured data is desensitized using a combination of k-anonymization and direct identifier masking, unstructured medical image data is not desensitized, and unstructured medical text data is desensitized by masking direct identifiers.
[0031] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another medical big data query method based on dynamic desensitization is provided, which includes the following steps.
[0032] S101 collects multi-source heterogeneous medical data.
[0033] Specifically, multi-source heterogeneous medical data from multiple medical systems is collected, and the multi-source heterogeneous medical data is preprocessed, including semantic normalization based on the coding system and medical terminology set, and a unique patient identifier is constructed to associate the scattered data. Duplicate and abnormal data are removed using a cleaning rule base to generate standardized intermediate plaintext data.
[0034] By configuring various data interface adapters, data can be extracted in real time or in batches from different medical business systems. These systems typically include: Hospital Information System (HIS): Provides structured data such as patient demographic information, registration information, medical orders, and diagnostic records; Laboratory Information System (LIS): Provides structured data such as test requests and reports, result values and indicators; Picture Archiving and Communication System (PACS): Provides DICOM format medical image files and related structured metadata; Other systems: such as unstructured text records in electronic medical record (EMR) systems.
[0035] Based on a universal medical coding system, a mapping dictionary between localized terms and standard codes is established. For example, ICD-11 is used for disease diagnosis, SNOMED CT for clinical terminology, and LOINC for laboratory test items. Diagnostic names and test item names from different hospitals are automatically mapped to unique standardized codes. For instance, "acute myocardial infarction," "AMI," and "acute myocardial infarction" are uniformly mapped to the ICD-11 code "BA41.0."
[0036] For free text written by doctors, Named Entity Recognition (NER) technology based on the medical BERT model is used to extract medical terms and map them to the aforementioned standard coding system.
[0037] Based on the patient's core identifying information (such as name, gender, date of birth, ID number, and phone number), rule-based matching and fuzzy matching algorithms are used to perform patient identification and matching across systems. A globally unique and non-repeatable Patient Identifier (PUID) is generated for each unique patient. This PUID will then serve as the core key linking all of that patient's medical data.
[0038] To ensure data quality, a predefined cleansing rule base is invoked to verify and correct the data, outputting standardized intermediate plaintext data.
[0039] S102, Construct an adaptive isomorphic index structure.
[0040] Specifically, an adaptive isomorphic index structure based on pathological semantics is constructed based on standardized intermediate plaintext data. This includes building a B+ tree index based on semantic hash keys for structured data, extracting features from unstructured data to generate embedding vectors, and constructing an index for content retrieval based on these embedding vectors. The index structure is dynamically layered according to the access frequency of data items.
[0041] The process of constructing a B+ tree index based on semantic hash keys for structured data includes the following steps.
[0042] S102.11 Extract disease classification codes, medical terminology concept vectors, and patient unique identifiers from standardized intermediate plaintext data; S102.11, the disease classification code and the medical term concept vector are concatenated and input into the first hash function to generate the pathological topological hash value.
[0043] S102.12, the patient's unique identifier is concatenated with the data timestamp and input into the second hash function to generate the patient's time-series hash value.
[0044] S102.13, the pathological topological hash value is coupled with the patient temporal hash value to generate the final semantic hash key.
[0045] S102.14, Construct a B+ tree index structure based on semantic hash keys, wherein the semantic hash keys are used as the key values of the B+ tree, and the physical address of the data corresponding to the key values in the storage system is used as the leaf node storage value of the B+ tree.
[0046] In some alternative implementations, the B+ tree index is dynamically layered based on the access frequency of data items. Specifically, query performance is optimized by adjusting the branching factor and tree depth of its nodes, including the following steps.
[0047] Step 1: Periodically collect the B+ tree index within a time window. The access logs within the node provide statistics for each node. Frequency of access and average query response time .
[0048] Step 2: Based on the access frequency, calculate the access popularity weight of each node in the B+ tree. :
[0049] in, As a weighting adjustment factor, For time window The highest access frequency among all nodes.
[0050] This reflects the node's popularity. A higher access frequency indicates that the data handled by the node is more likely to be frequently accessed and should therefore receive more performance optimization resources. It is a normalization process that maps frequency values to the range of [0, 1], eliminating the influence of the absolute value's dimensions, and allowing for a fair comparison of the popularity between different nodes.
[0051] This indicates the performance health of a node. A longer average response time for a node may indicate that it is poorly positioned on the disk, carries too much data (potentially requiring splitting), or is too sparse (potentially requiring merging), thus becoming a performance bottleneck. It's a reciprocal transformation, and its physical meaning is "access rate," meaning the shorter the response time (the better the performance), the higher the rate value. This transformation allows nodes with "better performance" to receive a higher contribution value in this item, consistent with the logic of the frequency item.
[0052] This is used to dynamically balance the relative importance of the two factors, "access frequency" and "response performance," in the final decision. When As the value approaches 1, the formula degenerates into a decision primarily based on access frequency. This is suitable for environments with stable business scenarios and ample system I / O capacity, where the decision objective is to prioritize ensuring the access efficiency of the most frequently accessed data. As the value approaches 0, the formula degenerates into making decisions primarily based on response performance (access rate). This is suitable for environments with high system I / O pressure and prominent performance bottlenecks, where the decision objective is to prioritize "treating" those nodes that slow down overall performance, regardless of their access frequency.
[0053] Step 3, if a node has a high access popularity weight The value is higher than the preset upper threshold for several consecutive time windows. If a node is identified as a hotspot, its tree depth is reduced by increasing the branch factor B of its child nodes.
[0054] Step 4, if a node has a high access popularity weight The value is below the preset lower threshold for several consecutive time windows. If it is a cold node, its branch factor B is reduced and a node merging operation is performed.
[0055] For hot nodes, the tree depth of the branch is reduced by increasing the branch factor B of its child nodes (i.e., allowing the node to have more child nodes), thereby improving query efficiency; for cold nodes, storage space is freed up and memory usage is optimized by reducing its branch factor B and performing node merging operations.
[0056] The process of extracting features from unstructured data to generate embedding vectors and then building an index for content retrieval based on these embedding vectors includes the following steps.
[0057] S102.21 Extracts high-dimensional feature vectors from unstructured medical image data using a pre-trained deep convolutional neural network.
[0058] S102.22, scalar quantization is performed on the high-dimensional feature vector to obtain the embedding vector.
[0059] S102.23, construct a vector index based on approximate nearest neighbor search by using the quantized embedded vector as the index value.
[0060] The deep convolutional neural network uses ResNet-50 or DenseNet-121 architecture as the core feature extractor, removing its top classification layer and using the output of the last convolutional layer as the input to the global average pooling layer. Supervised pre-training is performed using a multi-source, multimodal medical image dataset containing chest X-rays (CheXpert), brain MRI (BraTS), and fundus images (OCT). The annotation information of the dataset includes at least one of disease labels and lesion segmentation masks. During the pre-training phase, the network's optimization objective is to minimize the weighted sum of the cross-entropy classification loss function and the Dice segmentation loss function, so that its output high-dimensional feature vector simultaneously encodes global semantic information and local lesion features. The feature tensor obtained after global average pooling is used as the high-dimensional feature vector of the unstructured medical image data, with its dimension fixed at 2048 or 1024.
[0061] The specific steps for scalar quantization of high-dimensional feature vectors are as follows: using the Product Quantization (PQ) algorithm, the 2048-dimensional or 1024-dimensional feature vector is divided into M continuous sub-vectors; using the K-Means clustering algorithm, a codebook containing K cluster centers is independently constructed in each sub-vector space; finally, each sub-vector is quantized into the index ID of its nearest neighbor cluster center, and the sequence of index IDs together constitutes the quantized compact embedding vector.
[0062] S103, execute data query.
[0063] Specifically, the system receives data query requests, extracts the access permission context of the query requests, uses an adaptive isomorphic index structure to locate and read the corresponding intermediate plaintext data, and performs differentiated privacy protection on the read intermediate plaintext data according to the access permission context to generate de-identified results that match the permissions.
[0064] The process of receiving a data query request, extracting the access permission context of the query request, locating and reading the corresponding intermediate plaintext data using an adaptive isomorphic index structure includes the following steps.
[0065] S103.1 receives data query requests initiated by external applications through a well-defined RESTful API or message queue interface.
[0066] The request should include at least the query subject, the query content, and key access permission context. The query subject can be a patient ID or imaging examination number. The query content may include retrieving all laboratory test records for a specific patient or finding cases with similar imaging features. The access permission context is typically transmitted in the form of an encrypted token, such as a JWT token, which embeds attributes such as the requester's role, affiliated institution, and purpose of access.
[0067] S103.2, perform security verification and parsing on the query request, decrypt and verify the legality and validity of the access token it carries, and extract the core permission attributes from the token's payload.
[0068] Permission attributes include: the role of the requesting entity (such as attending physician, researcher, insurance checker), the purpose of access (such as clinical diagnosis and treatment, scientific research), and the scope of data access (such as data of this institution, specific datasets of multi-center research).
[0069] The requesting subject role, access purpose, and data access scope triple together constitute the access permission context that determines the subsequent de-identification strategy.
[0070] S103.3, Based on the query conditions, convert the query request into a query key that can be recognized by the adaptive isomorphic index structure.
[0071] If the queried object is structured data, the system uses the same hash function and coupling rules as when building the index to convert the query conditions into corresponding semantic hash keys, and then uses this key to perform a fast lookup in the upper-level B+ tree index.
[0072] If the query object is unstructured data, the same pre-trained deep convolutional network is first used to extract the feature embedding vector of the query image, and the same quantization process is performed on it to generate a query vector. Then, this vector is input into the vector index of the lower layer for approximate nearest neighbor search.
[0073] Index lookup operations return one or more pointers to data locations in the storage system.
[0074] S103.4, based on the data pointer returned by the index, read the corresponding standardized intermediate plaintext data block from the specified secure storage area.
[0075] A data block is an authoritative copy of data that has been preprocessed and semantically normalized but has not yet undergone any de-identification operations. It has a regular format and contains complete field information, providing a unique and accurate data source for subsequent dynamic de-identification operations.
[0076] Based on the access permission context, differentiated privacy protection is performed on the corresponding intermediate plaintext data read, generating a de-identified result matching the permissions. Specifically, this includes: parsing the access permission context to obtain the role and purpose of the requesting subject; and selecting the corresponding de-identification execution algorithm based on the role and purpose of the requesting subject and according to the preset privacy protection policy mapping rules. The mapping rules are as follows: if the role is a researcher and the purpose of access is statistical analysis, then a research-level permission policy is matched; if the role is a clinician and the purpose of access is medical services, then a medical-level permission policy is matched.
[0077] When matching research-level permission policies, structured data is desensitized using a Laplacian noise addition mechanism, unstructured medical image data is desensitized using a deep learning-based sensitive region detection and blurring algorithm, and unstructured medical text data is desensitized by generalization or removal of quasi-identifiers.
[0078] When matching the diagnostic-level permission policy, structured data is anonymized using a combination of k-anonymization and direct identifier masking, unstructured medical image data is not anonymized, and unstructured medical text data is anonymized by masking direct identifiers.
[0079] k-anonymity-based data masking specifically includes: partially masking or hashing the name and ID number fields; generalizing the range of quasi-identifiers such as age and postal code; and ensuring that the processed data record is indistinguishable from at least k-1 other records in the dataset in terms of quasi-identifiers, thus satisfying k-anonymity.
[0080] It should be noted that when matching a research-level permission policy, Laplace noise is added to the numerical medical data fields in the structured data; the scale parameter of the Laplace noise is... Privacy Budget Decision. Privacy Budget It is a core parameter of differential privacy. The smaller the value, the greater the added noise, the stronger the privacy protection, but the lower the data availability.
[0081] A differential privacy budget allocation mechanism based on multi-objective reinforcement learning is adopted, using the AUC decline rate of the clinical efficacy prediction model as a constraint loss function to dynamically adjust the Laplace noise parameter, which is the privacy budget. Specifically, it includes the following steps.
[0082] Step 1, for each decision moment Collect state vectors The state vector This includes the data characteristics involved in the current data query (including disease type ICD-11 codes, data dimensions, and global sensitivity of numerical fields), historical efficacy evaluation feedback (the moving average of the AUC decline rate of clinical prediction models trained on desensitized data under recent similar queries), and the baseline of privacy protection level (recently allocated privacy budget). (historical records).
[0083] Step 2, convert the state vector The input is fed into a pre-trained policy network, which outputs the privacy budget allocation action for the current query at that moment. ,Right now ,in These are continuous action values, representing the privacy budget allocated in this instance.
[0084] Policy Network It is an approximate function, and its parameters are... Optimized through multi-objective reinforcement learning algorithms.
[0085] Step 3, Perform the action After obtaining the query results, a multi-objective reward signal is obtained. , is represented as ,
[0086] in, These are preset weight hyperparameters used to adjust the relative importance of the two objectives of privacy and utility.
[0087] Privacy protection rewards Negatively correlated with the allocated privacy budget, defined as .
[0088] Treatment maintenance reward The effectiveness of subsequent clinical applications of the anonymized data will determine whether the AUC value of the diagnostic model trained on this anonymized data decreases by less than a threshold compared to the baseline model trained on the original data. If the result is positive, a positive reward is given; otherwise, a negative reward is given, as indicated by:
[0089] In the formula, The AUC value of the diagnostic model trained based on this anonymized data. The threshold value is used.
[0090] Step 4: Use the proximal policy optimization algorithm based on the collected state-action-reward sequence. Update policy network parameters, optimization objective function of PPO algorithm The aim is to maximize the expected cumulative reward while constraining the magnitude of policy updates to ensure training stability; its expression is:
[0091] in, Expressing expected value based on experience; This represents the probability ratio between the old and new strategies. The advantage function estimate at time t is used to measure the action. Advantages and disadvantages relative to the average level; These are pruning hyperparameters used to limit the magnitude of policy updates.
[0092] In some alternative implementations, an optimization step for the vector index is also included, specifically comprising the following steps.
[0093] Step 1: Periodically execute benchmark query tests to calculate the current average retrieval accuracy of the vector index. Compared to the current average query time The average retrieval accuracy By calculating the previous results in the query results The degree of overlap with the nearest real neighbors is used as a metric.
[0094] The quality of the entire search process is judged by verifying the accuracy of the "Top-R results" returned by the system. This is a pre-set value representing the "top few" search results that the user is most interested in. True nearest neighbor refers to the closest neighbor found by comparing all the data one by one to accurately identify the most similar result to the query target. Each data item. The query result is the evaluated object, that is, the vector index, which quickly returns what it considers to be the most similar based on the user query. Each data item. Overlap rate is a measure of calculation precision. The specific method. Returning the index... The results (which may contain errors) are compared with the true results. Compare the two nearest neighbors (standard answer) and calculate the number of overlaps between them, which is represented as:
[0095] Step 2: The vector index uses a product quantization algorithm, whose index parameters include the codebook size. Number of subspaces Each benchmark query test dynamically adjusts index parameters by solving the following optimization problem.
[0096] in, For parameter combination Average query time under, This represents the average retrieval accuracy under the corresponding parameter combinations. This is the minimum required precision threshold.
[0097] Step 3: After receiving the data query request, determine the size of the candidate set to be scanned using the following formula. ,
[0098] in, Based on the size of the candidate set, The target response time for the current query request. This represents the average query time under the latest index parameters.
[0099] Retrieval performance evaluation is automatically initiated at fixed intervals (e.g., every 24 hours) or when a significant decline in retrieval performance is detected (e.g., multiple consecutive query timeouts). A pre-defined benchmark query set is executed, representing typical business scenarios. Then, macro-level performance metrics under the current index parameters are calculated: average retrieval accuracy. and average query time .
[0100] if This indicates that the current index quality is unacceptable and parameters must be adjusted immediately. If We will also try to find ways to further reduce the time consumption while meeting the accuracy requirements. The goal is to find a better codebook size by solving the aforementioned optimization problem. Number of subspaces The combination of [parameters]. Then use the latest parameters. Regenerate or incrementally update the quantized codebook of the vector index.
[0101] The following provides further details on differentiated privacy protection practices for unstructured data.
[0102] For medical imaging data (such as DICOM files like CT, MRI, and X-rays), when the access permission is clinical level, access to the original or lossless compressed images is granted without anonymization. When the access permission is research level: a deep learning-based sensitive region detection and blurring technique is employed. Specifically, a pre-trained object detection neural network is used to automatically identify directly personally identifiable regions (such as faces, ID cards, name tags, etc.) in the images, and these regions are pixelated, Gaussian blurred, or masked, while preserving all medical and pathological features for research and analysis.
[0103] For medical text data (such as unstructured text like clinical notes, diagnostic reports, and discharge summaries): Named Entity Recognition (NER) technology is used for dynamic masking. BERT or BiLSTM-CRF models pre-trained on medical corpora are used to automatically identify sensitive entities in the text, such as names, ID numbers, phone numbers, detailed addresses, and specific dates. The masking granularity is dynamically determined based on access permissions: for clinical-level permissions, only direct identifiers are masked, retaining all clinically relevant descriptions; for research-level permissions, in addition to direct identifiers, quasi-identifiers are further generalized or removed to provide stronger privacy protection.
[0104] The foregoing has described in detail an embodiment of a medical big data query method based on dynamic desensitization. Based on the medical big data query method based on dynamic desensitization described in the above embodiment, this invention also provides a medical big data query system based on dynamic desensitization corresponding to the method.
[0105] Figure 2 This is a schematic block diagram of a medical big data query system based on dynamic anonymization provided in an embodiment of the present invention. In this embodiment, the medical big data query system 200 based on dynamic anonymization can be divided into multiple functional modules according to its functions, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory.
[0106] The medical data acquisition module 210 is used to collect multi-source heterogeneous medical data from multiple medical systems, preprocess the multi-source heterogeneous medical data, including semantic normalization based on the coding system and medical terminology set, construct a unique patient identifier to associate scattered data, use a cleaning rule base to remove duplicate and abnormal data, and generate standardized intermediate plaintext data.
[0107] The index structure construction module 220 is used to construct an adaptive isomorphic index structure based on pathological semantics based on standardized intermediate plaintext data; wherein, a B+ tree index based on semantic hash keys is constructed for structured data, features are extracted from unstructured data to generate embedding vectors, and an index for content retrieval is constructed based on the embedding vectors; the B+ tree index is dynamically layered according to the access frequency of data items.
[0108] The data query execution module 230 is used to receive data query requests, extract the access permission context of the query request, locate and read the corresponding intermediate plaintext data using an adaptive isomorphic index structure, and perform differentiated privacy protection on the read intermediate plaintext data according to the access permission context, generating desensitized results that match the permissions. The differentiated privacy protection includes: for research-level access permissions, desensitizing structured data using a Laplace noise addition mechanism, desensitizing unstructured medical image data using a deep learning-based sensitive region detection and blurring algorithm, and desensitizing unstructured medical text data by generalizing or removing quasi-identifiers; the noise parameters of the Laplace noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm. For clinical-level access permissions, desensitizing structured data using a combination of k-anonymization and direct identifier masking, not desensitizing unstructured medical image data, and masking direct identifiers for unstructured medical text data.
[0109] The medical big data query system based on dynamic desensitization in this embodiment is used to implement the aforementioned medical big data query method based on dynamic desensitization. Therefore, the specific implementation of this system can be found in the embodiment section of the medical big data query method based on dynamic desensitization mentioned above. Thus, the specific implementation can be referred to the description of the corresponding embodiments, and will not be elaborated here.
[0110] Furthermore, since the medical big data query system based on dynamic desensitization in this embodiment is used to implement the aforementioned medical big data query method based on dynamic desensitization, its function corresponds to the function of the above method, and will not be repeated here.
[0111] Figure 3 This is a schematic diagram of a terminal 300 provided in an embodiment of the present invention, including a processor 310, a memory 320, and a communication unit 330. The processor 310 is used to implement the process steps of the above-described embodiment of the medical big data query method based on dynamic desensitization when implementing the medical big data query program based on dynamic desensitization stored in the memory 320.
[0112] The terminal 300 includes a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0113] The memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile memory terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 is able to perform some or all of the steps in the above method embodiments.
[0114] The processor 310 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.
[0115] The communication unit 330 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.
[0116] The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0117] The computer storage medium stores a medical big data query program based on dynamic desensitization. When the medical big data query program based on dynamic desensitization is executed by the processor, it implements the process steps of the above-described embodiment of the medical big data query method based on dynamic desensitization.
[0118] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.
[0119] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0121] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0122] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for querying medical big data based on dynamic anonymization, characterized in that, Includes the following steps: Collect multi-source heterogeneous medical data from multiple medical systems, preprocess the multi-source heterogeneous medical data, including semantic normalization based on coding system and medical terminology set, construct unique patient identifiers to associate scattered data, use cleaning rule base to remove duplicate and abnormal data, and generate standardized intermediate plaintext data. An adaptive isomorphic index structure based on pathological semantics is constructed based on standardized intermediate plaintext data. Specifically, a B+ tree index based on semantic hash keys is constructed for structured data, and features are extracted from unstructured data to generate embedding vectors. An index for content retrieval is then constructed based on the embedding vectors. The B+ tree index is dynamically layered according to the access frequency of data items. Receive data query requests, extract the access permission context of the query requests, use an adaptive isomorphic index structure to locate and read the corresponding intermediate plaintext data, and perform differentiated privacy protection on the read intermediate plaintext data according to the access permission context to generate de-identified results that match the permissions. The differentiated privacy protection includes: for research-level access, structured data is desensitized using a Laplacian noise addition mechanism, unstructured medical image data is desensitized using a deep learning-based sensitive region detection and blurring algorithm, and unstructured medical text data is desensitized by generalization or removal of quasi-identifiers; the noise parameters of the Laplacian noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm; for clinical-level access, structured data is desensitized using a combination of k-anonymization and direct identifier masking, unstructured medical image data is not desensitized, and unstructured medical text data is desensitized by masking direct identifiers.
2. The medical big data query method based on dynamic desensitization according to claim 1, characterized in that, Constructing a B+ tree index based on semantic hash keys for structured data, specifically including: Extract disease classification codes, medical terminology concept vectors, and patient unique identifiers from standardized intermediate plaintext data; The disease classification code and the medical term concept vector are concatenated and input into the first hash function to generate the pathological topological hash value. The patient's unique identifier is concatenated with the data timestamp and input into the second hash function to generate the patient's time-series hash value; The pathological topological hash value is coupled with the patient temporal hash value to generate the final semantic hash key; A B+ tree index structure is constructed based on semantic hash keys, where the semantic hash keys serve as the keys of the B+ tree, and the physical address of the data corresponding to the key value in the storage system serves as the leaf node storage value of the B+ tree.
3. The medical big data query method based on dynamic desensitization according to claim 2, characterized in that, Extracting features from unstructured data to generate embedding vectors, and constructing an index for content retrieval based on these embedding vectors, specifically including: High-dimensional feature vectors of unstructured medical image data are extracted using a pre-trained deep convolutional neural network. The high-dimensional feature vector is scalar quantized to obtain the embedding vector; The quantized embedding vector is used as the index value to construct a vector index based on approximate nearest neighbor search.
4. The medical big data query method based on dynamic desensitization according to claim 1, characterized in that, The method also includes: The B+ tree index is periodically collected within a time window. The access logs within the node provide statistics for each node. Frequency of access and average query response time ; Based on the access frequency, calculate the access popularity weight of each node in the B+ tree. : in, As a weighting adjustment factor, For time window The highest access frequency among all nodes; If a node's access popularity weight The value is higher than the preset upper threshold for several consecutive time windows. If a node is identified as a hotspot, its tree depth is reduced by increasing the branch factor B of its child nodes. If a node's access popularity weight The value is below the preset lower threshold for several consecutive time windows. If it is a cold node, its branch factor B is reduced and a node merging operation is performed.
5. The medical big data query method based on dynamic desensitization according to claim 1, characterized in that, Based on the access permission context, differentiated privacy protection is performed on the corresponding intermediate plaintext data read, generating an anonymized result that matches the permissions, specifically including: Parse the access permission context to obtain the role and purpose of the requesting entity; Based on the role and purpose of the requesting entity, and according to the preset privacy protection policy mapping rules, the corresponding de-identification execution algorithm is selected. The mapping rules are as follows: if the role is a researcher and the purpose of access is statistical analysis, then a research-level permission policy is matched; if the role is a clinician and the purpose of access is medical services, then a medical-level permission policy is matched. Specifically, when matching a research-level permission policy, Laplace noise is added to the numerical medical data fields in the structured data; the scale parameter of the Laplace noise... Privacy Budget Decide.
6. The medical big data query method based on dynamic desensitization according to claim 5, characterized in that, The noise parameters of the Laplace noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm. Specifically, it adopts a differential privacy budget allocation mechanism based on multi-objective reinforcement learning, uses the AUC decline rate of the clinical efficacy prediction model as a constraint loss function, and dynamically adjusts the Laplace noise parameters.
7. The medical big data query method based on dynamic desensitization according to claim 1, characterized in that, The method also includes: Perform benchmark query tests periodically to calculate the current average retrieval accuracy of the vector index. Compared to the current average query time The average retrieval accuracy By calculating the previous results in the query results The overlap of the nearest real neighbors is used as a metric. Vector indexing employs a product quantization algorithm, whose index parameters include the codebook size. Number of subspaces Each benchmark query test dynamically adjusts index parameters by solving the following optimization problem. in, For parameter combination Average query time under, This represents the average retrieval accuracy under the corresponding parameter combinations. This is the minimum required precision threshold; After receiving a data query request, the size of the candidate set to be scanned is determined using the following formula. , in, Based on the size of the candidate set, The target response time for the current query request. This represents the average query time under the latest index parameters.
8. A medical big data query system based on dynamic anonymization, characterized in that, include: The medical data acquisition module is used to collect multi-source heterogeneous medical data from multiple medical systems, preprocess the multi-source heterogeneous medical data, including semantic normalization based on the coding system and medical terminology set, construct a unique patient identifier to associate scattered data, use a cleaning rule base to remove duplicate and abnormal data, and generate standardized intermediate plaintext data. The index structure construction module is used to construct an adaptive isomorphic index structure based on pathological semantics based on standardized intermediate plaintext data. Specifically, it constructs a B+ tree index based on semantic hash keys for structured data, extracts features from unstructured data to generate embedding vectors, and constructs an index for content retrieval based on the embedding vectors. The B+ tree index is dynamically layered according to the access frequency of data items. The data query execution module receives data query requests, extracts the access permission context of the query requests, locates and reads the corresponding intermediate plaintext data using an adaptive isomorphic index structure, and performs differentiated privacy protection on the read intermediate plaintext data according to the access permission context, generating desensitized results that match the permissions. The differentiated privacy protection includes: for research-level access permissions, a Laplace noise addition mechanism is used to desensitize structured data; a deep learning-based sensitive region detection and blurring algorithm is used to desensitize unstructured medical image data; and a generalization or removal of quasi-identifiers is used to desensitize unstructured medical text data. The noise parameters of the Laplace noise addition mechanism are dynamically adjusted based on a multi-objective reinforcement algorithm. For clinical-level access permissions, a combination of k-anonymization and direct identifier masking is used to desensitize structured data; unstructured medical image data is not desensitized; and direct identifiers are masked for unstructured medical text data.
9. A terminal, characterized in that, include: Storage device for storing medical big data query programs based on dynamic desensitization; A processor is configured to implement the steps of the medical big data query method based on dynamic desensitization as described in any one of claims 1 to 7 when executing the medical big data query program based on dynamic desensitization.
10. A computer-readable storage medium, characterized in that, The readable storage medium stores a medical big data query program based on dynamic desensitization, which, when executed by a processor, implements the steps of the medical big data query method based on dynamic desensitization as described in any one of claims 1 to 7.
Citation Information
Cited By
Medical data security processing method and device based on dynamic strategy and storage medium
CN121545657A
A medical data security processing method and device based on a dynamic strategy and a storage medium
CN121545657B
Differential privacy-based medical data desensitization traceability auditing method
CN121637571A
Differential privacy-based medical data desensitization traceability auditing method
CN121637571B
