Intelligent medical big data information management system

Abstract

The invention discloses an intelligent medical big data information management system, and the system comprises a data collection layer which serves as a sensing neural network, employs an Internet of Things gateway to be compatible with a medical equipment protocol, and deploys an edge calculation node for physiological signal preprocessing; the intelligent preprocessing layer is used for cleaning and standardizing the original data transmitted by the data acquisition layer; the informatization service layer is used for realizing fine-grained authority control based on an RBAC (Role Based Access Control) model, realizing inspection process automation by integrating an LIS (Line Information System) interface through a Hyperdger Fabric block chain evidence storage key operation hash, and constructing a zero-footprint DICOM (Digital Imaging and Communications in Medicine) viewer; the intelligent analysis layer is used for carrying out data life cycle management by adopting a mixed storage architecture, carrying out feature engineering on data, carrying out label matching and classified storage by using efficient neighbor search and similarity / distance measurement, quantifying contribution of each feature or feature combination to a single prediction result through an SHAP value, generating an interpretation result, and outputting the interpretation result; and presenting an interpretation result to a clinician or related personnel through a query interface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical data information management, and particularly relates to a smart medical big data information management system. BACKGROUND

[0002] Under the background of the global wave of medical informatization and intelligentization, it has become an urgent need to build a smart medical big data information management system to break the data silos in the medical industry, improve the efficiency of diagnosis and treatment, and empower precision medicine. The system integrates Internet of Things gateways, edge computing, blockchain storage, multi-modal artificial intelligence and other cutting-edge technologies to break the shackles of incompatible device protocols, heterogeneous data formats and fragmented diagnosis and treatment information in traditional medical systems. Through the deep mining and intelligent application of medical big data, the system promotes the transformation of medical services from experience-driven to data-driven and provides intelligent infrastructure for scenarios such as hierarchical diagnosis and treatment, drug research and development, and public health prevention and control.

[0003] In the prior art, a pseudo-label and big data label comparison method is used to sort and retrieve medical data. However, the pseudo-label and big data label comparison method (bubble sort) is inefficient with a time complexity of O(n²) and is not feasible in the context of massive medical data. Therefore, a smart medical big data information management system is proposed. SUMMARY

[0004] The purpose of the present application is to solve the problems existing in the prior art and to provide a smart medical big data information management system.

[0005] To achieve the above purpose, the present application adopts the following technical solutions: A smart medical big data information management system comprises: A data acquisition layer: as a perception neural network, it adopts an Internet of Things gateway compatible with medical device protocols and deploys edge computing nodes for physiological signal preprocessing. It uses an unstructured data collector to process image and text data, obtains structured electronic medical records through an integrated hospital information system interface, and transmits raw data through a real-time stream pipeline constructed by Kafka+Flink. An intelligent preprocessing layer: it cleanses and standardizes the raw data transmitted from the data acquisition layer, including missing value processing, outlier detection, standardization and multi-modal fusion. The missing value processing generates five complete data sets and retains uncertainty parameters by constructing a random forest regression model. The outlier detection detects outliers through isolation forest and verifies abnormalities against a unified medical language system medical atlas. The standardization performs Z-Score conversion on test indicators and Min-Max normalization on image data. The multi-modal fusion associates medical events through a FHIR standard to construct a patient holographic timeline. The processed data is transmitted to the information service layer and the intelligent analysis layer. Information service layer: based on RBAC model to realize fine-grained permission control, store key operation hash through Hyperledger Fabric block chain, realize inspection process automation by integrating LIS interface, build zero footprint DICOM viewer, deploy SciBERT-NER model to extract medical entities, use time convolution network to analyze disease evolution trend; Intelligent analysis layer: adopt hybrid storage architecture for data life cycle management, perform feature engineering on data, use efficient nearest neighbor search and similarity / distance measurement for label matching and classified storage, quantify the contribution of each feature or feature combination to a single prediction result through SHAP value, generate explanation result, and present the explanation result to clinical doctors or relevant personnel through query interface.

[0006] The above technical solution further comprises: Further, the analysis and processing of unstructured data by the data acquisition layer includes DICOM image processing and text data OCR processing. The specific steps of the DICOM image processing are: Metadata extraction: read file header information using DICOM standard analysis library, extract structured fields, and match key labels through regular expression; Pixel data conversion: apply analog-digital conversion formula to image pixel array , store value Mapping to display gray value , Slope, Intercept, combined with VOI LUT for window width and window level adjustment, represented as , wherein is the mapped display gray value, is the center value, is the window width; Feature embedding: extract image feature vector using pre-trained DenseNet-121 model, and compress spatial dimension to feature representation through global average pooling; The specific steps of the text data OCR processing are: Image preprocessing: perform grayscale and binarization on scanned documents; Text region detection: locate text blocks using connected component analysis, filter noise regions, and retain connected components with text characteristics in aspect ratio; Character recognition and post-processing: recognize character sequences through Tesseract engine, and perform context correction combined with medical dictionary.

[0007] Further, the specific steps of the data acquisition layer for obtaining structured electronic medical records by integrating hospital information system interface are: Hospital information system interface protocol adaptation: build interface engine based on HL7 v3 standard and define standardized transmission format containing message header, structured medical record data body and digital signature; Patient master index matching: adopt third-order matching algorithm for real-time synchronization of patient master index, calculate field consistency score through weighted accurate matching, use Jaro-Winkler distance to measure text similarity, and combine biological feature verification to associate patient identity, the field consistency score calculation formula is represented as , wherein, is the field weight, is the matching function, is the value of the i-th feature field in the source system and the value of the i-th feature field in the target system, when ≥ 0.9 is determined as the same patient, for <0.9 record, calculate Jaro-Winkler distance, represented as , wherein, are two strings that need to be fuzzy matched, is the Jaro distance, is the prefix matching length, is the scaling factor, when the name similarity ≥ 0.85 and the gender / birth date are consistent, trigger manual verification; CDC change capture: deploy CDC component based on database transaction log analysis, parse INSERT / UPDATE / DELETE operation type by listening to log source, generate change record metadata containing timestamp, primary key value and data snapshot; Incremental data synchronization: adopt two-phase commit protocol to ensure data consistency, create temporary transaction table on the source side to record changes to be synchronized, after the distributed transaction coordinator Seata verifies the preparation state of all participating nodes, push the diagnosis and treatment record change event to the data lake through the Kafka message queue, trigger the intelligent preprocessing layer to update the patient holographic timeline and start the clinical rule engine.

[0008] Further, the missing value processing generates five complete data sets by constructing a random forest regression model and retaining uncertainty parameters through the following specific steps: Variable correlation analysis: select feature variables with significant correlation with family history by maximum mutual information coefficient, and construct a feature matrix; Random forest model training: train the random forest regression model on the complete data subset, adopt the integrated structure of k (500) decision trees, set the minimum sample number for node splitting to , the feature sampling ratio is , and p is the total number of features. The out-of-bag error is used to optimize the hyperparameters. Multiple imputation generation: for each missing sample, 5 different imputed values are generated using the trained random forest. Each time imputation is performed, random noise following normal distribution N is added to the prediction result of each tree in the forest to simulate the prediction uncertainty. The generated imputed values are limited within the medically reasonable range. Uncertainty parameter reservation: for each imputed dataset, the uncertainty parameters are saved. Downstream analysis fusion: Rubin's Rules are used to combine the results of the 5 imputed datasets: a weighted average is calculated for each cluster center coordinate; the imputed variance component is included when calculating the intra-class variance; and finally the clustering results including the uncertainty interval are generated.

[0009] Further, the information service layer builds a whole-process medical information service system covering permission management, operation audit, inspection early warning, image browsing, text analysis and trend prediction, including the following steps: Fine-grained permission control based on RBAC model: a three-level permission system is built, and access control matrix is used for permission inheritance and dynamic allocation: a role set is defined , wherein for patients, for doctors, for administrators, a permission-operation mapping table is established, which contains multiple basic operations, and permission delegation is performed through a role inheritance tree: ← indicates that the doctor role inherits the basic access permission of the patient, and the access decision function uses Boolean matrix operation, which is represented as , wherein Roles(u) is the user role set, is the resource-role allocation matrix; Hyperledger Fabric block chain storage: deploy a consortium chain network to realize key operation storage, and use an improved UTXO model to record the audit trail. When the doctor submits an electronic prescription, a transaction containing the operation type, timestamp and prescription hash is generated, which is stored in the block after the endorsement node consensus, forming an unalterable audit chain. LIS interface integration and critical value early warning engine: integrate the laboratory information system through the HL7 standard to build a real-time early warning pipeline. Zero-footprint DICOM viewer: develop a plug-in-free image browser based on WebGL, using block loading and progressive rendering.

[0010] Further, the intelligent analysis layer constructs a three-level storage system, distinguishes data storage levels by defining data temperature thresholds, and deploys an automatic migration strategy by monitoring access patterns through Hive metadata to trigger data migration tasks.

[0011] Further, the intelligent analysis layer performs feature engineering on data, including the following steps: Recursive feature elimination: automatically selects the most relevant feature subset for medical targets, calculates feature weights based on ensemble learning models, and the calculation formula is represented as wherein, is the model coefficient of the feature in the k-fold cross-validation, K is the number of cross-validation folds, and the features are sorted by importance. The least important set of features is removed each time, and the model performance of the remaining feature subset is evaluated on the validation set. The feature quantity corresponding to the performance inflection point is selected; Uniform manifold approximation and projection: non-linear dimensionality reduction and structure preservation are performed on high-dimensional medical data. The local similarity between data points is calculated, and the calculation formula is represented as wherein, is the local connectivity parameter determined by binary search, represents the probability that point j is a neighbor of point i, and the KL divergence loss is minimized by stochastic gradient descent; Multi-modal Transformer cross-attention modeling: fusion of text-image-numeric data, capture of cross-modal interaction patterns, setting of modality-specific encoders, including text branch, image branch and numerical branch. The text branch generates context embedding through SciBERT, represented as wherein, is the text, the image branch uses ResNet-50 to extract visual features, represented as wherein, is the feature map, and the numerical branch is represented as wherein, is the numerical value, the attention weights of the text branch, image branch and numerical branch are determined through the cross-attention mechanism, and the outputs of each modality are dynamically fused through the gating mechanism, represented as wherein, σ is the Sigmoid activation function, and is the element-wise multiplication.

[0012] Further, in the intelligent analysis layer, efficient neighbor search is completed through local sensitive hashing, and then the approximate nearest neighbor is quickly located in the medical data, including the following steps: Hash function family design: construct a random projection hash function to map high-dimensional data to low-dimensional hash buckets, represented as wherein, is a random unit vector, is the input data vector, b is the random translation amount, and w is the bucket width parameter; Multi-table overlay strategy: Combine multiple hash tables through logical "and" and "or" operations, expressed as ,in, is the number of hash tables for the “AND” operation, For the number of hash functions in each “and” group, the multi-table overlay strategy strikes a balance between recall and query latency; Dynamic index update: A sliding window mechanism is used to maintain index freshness, which is expressed as , where τ is the time window length, and only the data within the most recent τ is retained to participate in the hash mapping.

[0013] Furthermore, the hash candidate set is sorted and classified and stored using similarity / distance metrics, including the following steps: Cosine similarity calculation: Calculating cosine similarity is used to measure the directional consistency of text or high-dimensional sparse data, expressed as ,in, is the feature vector of text or high-dimensional sparse data, is a vector The module length, is a vector Length of the module; Mahalanobis distance correction: The dimension and correlation effects of structured data are eliminated by Mahalanobis distance correction, which is expressed as ,in, Represents two different data points, Represents a vector The transpose of Represents the inverse matrix of the feature covariance matrix Σ, Σ is the feature covariance matrix, estimated by sliding window, expressed as , where n represents the number of samples, that is, the number of data points contained in the data set. represents the i-th sample vector, representing the i-th data point in the data set, is the sample mean vector, which represents the average value of all data points in the data set; Hybrid sorting: Combine the number of hash collisions and the similarity score to perform weighted fusion, and sort based on the fused data.

[0014] Furthermore, the intelligent analysis layer quantifies the contribution of each feature or feature combination to a single prediction result through SHAP value, and generates specific steps for explaining the results: Cooperative game theory modeling: The prediction problem is modeled as a cooperative game, where the features are the participants and the model output is the payoff, expressed as , where F is the entire feature set and S is the feature subset. is the model prediction for the subset S only, is the Shapley value for feature i, representing its average marginal contribution to the prediction; Feature interaction effect decomposition: SHAP interaction values capture feature synergies, represented as , the total contribution is decomposed into main effects + and interaction effects ; Sample-level explanation generation: An explanation vector is generated for each prediction, represented as , containing first-order feature contributions and second-order interaction contributions.

[0015] The present application has the following beneficial effects: In the present application, the high-efficiency approximate search algorithm verified by the industry and academia and the mathematically reasonable similarity / distance metric are introduced for label matching and classified storage, ensuring the feasibility, speed and classification accuracy under massive data, while the SHAP value quantifies the feature contribution, generates the feature importance map, highlights the area and other explanation results, assists the clinical decision-making and provides deep analysis capability. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A system block diagram of a smart medical big data information management system is provided in the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0018] Please refer to Figure 1 Fig. 1, the present application is a smart medical big data information management system, comprising: Data acquisition layer: as a perception neural network, adopt Internet of Things gateway compatible medical device protocol and deploy edge computing node for physiological signal preprocessing, use unstructured data collector to integrate DICOM analysis middleware and Tesseract OCR engine to process image and text data, through integrated hospital information system (HIS) interface to obtain structured electronic medical record, realize patient main index real-time synchronization, through Kafka+Flink constructed real-time stream pipeline to transmit raw data; Intelligent preprocessing layer: clean and standardize the raw data transmitted from the data acquisition layer, including missing value processing, outlier detection, standardization, and multi-modal fusion. The missing value processing generates five complete data sets and retains uncertainty parameters by constructing a random forest regression model. The outlier detection detects outliers by isolation forest and verifies abnormalities by unified medical language system (UMLS) medical atlas. The standardization performs Z-Score conversion on test indicators and Min-Max normalization on image data. The multi-modal fusion associates medical events through FHIR standard to build a patient holographic timeline. The processed data is transmitted to the information service layer and the intelligent analysis layer. Information service layer: implement fine-grained permission control (patient / doctor / administrator three-level permission system) based on RBAC model, store key operation hash through Hyperledger Fabric blockchain, integrate LIS interface to realize laboratory process automation (develop critical value early warning engine such as blood potassium <3.0mmol / L to automatically trigger emergency channel), build zero footprint DICOM viewer, deploy SciBERT-NER model to extract medical entities, and use time convolution network (TCN) to analyze disease evolution trend. Intelligent analysis layer: use hybrid storage architecture (structured data stored in HBase, image data stored in HDFS, and text data stored in ElasticSearch) for data lifecycle management (hot and cold data hierarchical storage (hot data SSD, warm data SAS, and cold data object storage)), feature engineering (apply Recursive Feature Elimination (RFE) to automatically select the most relevant feature subset for target (such as disease prediction and efficacy evaluation), perform uniform manifold approximation and projection (UMAP) for nonlinear dimensionality reduction, and use Multi-modal Transformer to model text-image-numeric data cross-attention), use efficient neighbor search (local sensitive hashing LSH / approximate nearest neighbor ANN) and similarity / distance measurement (cosine / Mahalanobis distance) for label matching and classification storage, quantify the contribution of each feature or feature combination to a single prediction result using SHAP values, generate explanation results, and present the explanation results (such as feature importance chart, highlighted area, and key text segment) to clinicians or relevant personnel through query interface.

[0019] In one embodiment, the data acquisition layer includes DICOM image processing and text data OCR processing for unstructured data analysis and processing. The specific steps of the DICOM image processing are as follows: Metadata extraction: read file header information using DICOM standard parsing library, extract structured fields such as patient identification (PatientID), examination type (Modality), device parameters (such as kVp, mA), etc., and match key tags through regular expressions; Pixel data conversion: apply analog-digital conversion formula to image pixel array , map the storage value to the display gray value , , where is the slope and is the intercept, adjust the window width and window level with VOI LUT (value range look up table), represented as , where is the mapped display gray value, is the center value, is the window width, and an 8-bit PNG image conforming to DICOM PS3.3 standard is generated; Feature embedding: use a pre-trained DenseNet-121 model to extract image feature vectors, and compress the spatial dimension to 1024-dimensional feature representation through global average pooling (GAP); The specific steps of the text data OCR processing are: Image preprocessing: perform grayscale and binarization on the scanned document; Text region detection: use connected component analysis to locate text blocks, filter noise regions, and retain connected components with text characteristics in aspect ratio; Character recognition and post-processing: recognize character sequences through Tesseract engine and perform context correction with medical dictionary.

[0020] In one embodiment, the data acquisition layer obtains structured electronic medical records by integrating hospital information system (HIS) interface, and the specific steps are: Hospital information system (HIS) interface protocol adaptation: build an interface engine based on HL7 v3 standard and define a standardized transmission format containing message header, structured medical record data body and digital signature; Patient master index matching: use a third-order matching algorithm to synchronize the patient master index in real time, calculate the field consistency score through weighted exact matching, measure the text similarity using Jaro-Winkler distance, and associate the patient's identity by combining biometric verification, the field consistency score calculation formula is represented as , where is the field weight (ID number = 0.6, medical record number = 0.3, name = 0.1), is the matching function (1 when completely consistent, otherwise 0), For the value of the i-th feature field in the source system and the value of the i-th feature field in the target system, when ≥ 0.9 is determined to be the same patient, and for records with < 0.9, the Jaro-Winkler distance is calculated and represented as wherein are two strings that require fuzzy matching, is the Jaro distance, is the length of the prefix match, is a scaling factor (usually = 0.1), when the name similarity ≥ 0.85 and the gender / birth date is consistent, a manual verification is triggered; CDC change capture: a CDC component based on database transaction log analysis is deployed to parse INSERT / UPDATE / DELETE operation types by listening to log sources such as MySQL Binlog, and generate change record metadata containing timestamp, primary key value and data snapshot; Incremental data synchronization: a two-phase commit protocol is used to ensure data consistency, a temporary transaction table is created at the source end to record changes to be synchronized, and after the preparation state of all participating nodes is verified by the distributed transaction coordinator Seata, the diagnosis and treatment record change events are pushed to the data lake through the Kafka message queue, triggering the intelligent preprocessing layer to update the patient holographic timeline and start the clinical rule engine.

[0021] In one embodiment, the missing value processing generates five complete data sets and retains the uncertainty parameter by constructing a random forest regression model through the following specific steps: Variable correlation analysis: filter feature variables (such as diagnosis code, age segment, number of other test indicators abnormal, etc.) that have significant correlation with family history by using the maximum mutual information coefficient (MIC), and construct a feature matrix containing 15 prediction factors; Random forest model training: train a random forest regression model on the complete data subset, use an ensemble structure of k (500) decision trees, set the minimum sample size for node splitting to (10), the feature sampling ratio to , and p is the total number of features. The out-of-bag error (OOB Error) is used to optimize the hyperparameters; Multiple imputation generation: for each missing sample, generate five different predicted values using the trained random forest. Each time the data is imputed, add random noise following a normal distribution N (μ = predicted value, σ = OOB standard error) to the prediction results of each tree in the forest to simulate prediction uncertainty. The generated imputed values are limited within the medically reasonable range; Uncertainty parameter preservation: save for each imputed dataset (variance-covariance matrix: record the deviation of each imputed value from the mean of the complete data subset; imputation frequency weight: assign a weight coefficient of 0.8-1.2 to each imputed value according to the model prediction confidence (calculated by forest voting consistency); missing mode label: append variable labels "family history_imputed1" to "family history_imputed5" to retain missing source information); Downstream analysis fusion: combine the results of the five imputed datasets using Rubin's Rules: calculate the weighted average of each cluster center coordinate (weight is the imputation frequency weight); include the imputation variance component when calculating the intra-class variance (decomposed by variance-covariance matrix); finally generate a clustering result containing uncertainty intervals (e.g., the probability of a patient belonging to the "high-risk diabetes group" is 85% ± 7%).

[0022] In one embodiment, the information service layer builds a full-process medical information service system covering permission management, operation audit, inspection early warning, image browsing, text analysis and trend prediction, including the following steps: Fine-grained permission control based on RBAC model: build a three-level permission system (patient / doctor / administrator), use access control matrix (Access Control Matrix, ACM) for permission inheritance and dynamic allocation: define a role set , where is the patient, is the doctor, is the administrator, and a permission-operation mapping table is established, which contains multiple basic operations and performs permission delegation through a role inheritance tree: ← indicates that the doctor role inherits the basic access permissions of the patient, and the access decision function uses Boolean matrix operations, represented as , where Roles(u) is the user role set, is the resource-role allocation matrix; Hyperledger Fabric blockchain notarization: deploy a consortium chain network to implement key operation notarization, use an improved UTXO model to record audit trails, when a doctor submits an electronic prescription, generate a transaction containing operation type, timestamp, and prescription hash, after consensus by the endorsement node, store it in the block, forming an unalterable audit chain; LIS interface integration and critical value early warning engine: integrate the laboratory information system (LIS) through HL7 standards, and build a real-time early warning pipeline; Zero-footprint DICOM viewer: develop a plugin-free image viewer based on WebGL, using block loading and progressive rendering.

[0023] In one embodiment, the intelligent analysis layer constructs a three-level storage system, distinguishes data storage levels by defining data temperature thresholds, high-frequency access data within 30 days is hot data, 30-day-1-year access data is warm data, and super-1-year low-frequency access data is cold data, hot data SSD, warm data SAS, cold data object storage, deploys an automatic migration strategy, monitors access patterns through Hive metadata, and triggers data migration tasks (such as migrating structured data in HBase that has not been accessed for 90 days to S3 object storage).

[0024] In one embodiment, the intelligent analysis layer performs feature engineering on the data, including the following steps: Recursive feature elimination: automatically filters the most relevant feature subset for medical targets (such as disease prediction, efficacy evaluation), calculates feature weights based on ensemble learning models (such as XGBoost), and the calculation formula is represented as wherein, is the model coefficient of the feature in the k-fold cross-validation, K is the number of cross-validation folds (usually 5 or 10), the features are sorted by importance, the least important set of features is removed each time, and the model performance (such as AUC, F1-score) of the remaining feature subset is evaluated on the validation set, and the feature number corresponding to the performance inflection point is selected; Uniform manifold approximation and projection: nonlinear dimensionality reduction and structure preservation of high-dimensional medical data, calculation of local similarity between data points, calculation formula represented as wherein, is the local connectivity parameter determined by binary search, represents the probability that point j is a neighbor of point i, and the KL divergence loss is minimized by stochastic gradient descent; Multi-modal Transformer cross-attention modeling: fusion of text-image-numeric data, capture of cross-modal interaction patterns, setting of modality-specific encoders, including text branch, image branch, and numeric branch, the text branch generates context embedding through SciBERT represented as wherein, is the text, L is the length of the text, the image branch uses ResNet-50 to extract visual features represented as wherein, is the feature map, H, W is the spatial dimension of the feature map, the numeric branch is represented as wherein, is the numeric value, D is the dimension of the numeric feature, the attention weights of the text branch, image branch, and numeric branch are determined through the cross-attention mechanism, and the outputs of each modality are dynamically fused through the gating mechanism represented as wherein, σ is the Sigmoid activation function, and is the element-wise multiplication.

[0025] In one embodiment, in the intelligent analysis layer, efficient neighbor search is completed by local sensitive hashing (LSH), and then the approximate nearest neighbor is quickly located in the medical data, including the following steps: Hash function family design: a random projection hash function is constructed to map high-dimensional data to low-dimensional hash buckets, denoted as wherein, is a random unit vector, is an input data vector, b~U(0,w) is a random translation, and w is a bucket width parameter, w is usually set to 2 times the standard deviation of the data to balance the collision probability and search accuracy; Multi-table superposition strategy: multiple hash tables are combined by logical "and" and "or" operations, denoted as wherein, is the number of hash tables for "and" operation (usually 2-3), is the number of hash functions in each "and" group (usually 4-6), and the multi-table superposition strategy balances the recall (Recall@100) and query delay; Dynamic index update: a sliding window mechanism is used to maintain the freshness of the index, denoted as wherein τ is the length of the time window (such as 7 days), only the data in the last τ is retained to participate in the hash mapping, and the historical data is avoided to interfere with real-time search.

[0026] In one embodiment, the hash candidate set is sorted and stored by similarity / distance measurement, including the following steps: Cosine similarity calculation: cosine similarity is calculated to measure the directional consistency of text or high-dimensional sparse data, denoted as wherein, is the feature vector of the text or high-dimensional sparse data, is the modulus of the vector , that is, the square root of the sum of squares of all elements in the vector a, is the modulus of the vector ; Mahalanobis distance correction: the dimension and correlation influence of structured data are eliminated by Mahalanobis distance correction, denoted as wherein, denotes two different data points, denotes the transpose of the vector , denotes the inverse matrix of the feature covariance matrix Σ, and Σ is the feature covariance matrix, which is estimated by a sliding window, denoted as wherein n denotes the number of samples, that is, the number of data points in the data set, denotes the i-th sample vector, and denotes the i-th data point in the data set, is the sample mean vector, which represents the average value of all data points in the data set; Hybrid sorting: Combine the number of hash collisions and the similarity score to perform weighted fusion, and sort based on the fused data.

[0027] In one embodiment, the intelligent analysis layer quantifies the contribution of each feature or feature combination to a single prediction result through SHAP values, and generates the following specific steps for explaining the result: Cooperative game theory modeling: The prediction problem is modeled as a cooperative game, where the features are the participants and the model output is the payoff, expressed as , where F is the entire feature set and S is the feature subset. is the model prediction value when only subset S is used, is the Shapley value of feature i, indicating its average marginal contribution to the prediction result; Feature interaction effect decomposition: The SHAP interaction value is used to capture the synergy between features, expressed as , decomposing the total contribution into main effects + and interaction effects ; Sample-level explanation generation: Generate an explanation vector for each prediction result, expressed as , including first-order feature contribution and second-order interaction contribution.

[0028] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A smart medical big data information management system, characterized by: include: Data acquisition layer: As a perception neural network, it uses an IoT gateway compatible with medical device protocols and deploys edge computing nodes to preprocess physiological signals. It uses unstructured data collectors to process image and text data, obtains structured electronic medical records through integrated hospital information system interfaces, and transmits raw data through a real-time streaming pipeline built with Kafka and Flink. Intelligent preprocessing layer: Cleans and standardizes the raw data transmitted from the data acquisition layer, including missing value processing, outlier detection, standardization, and multimodal fusion. The missing value processing generates five complete data sets and retains uncertainty parameters by constructing a random forest regression model. The outlier detection detects outliers through isolation forest and connects to the unified medical language system medical atlas to verify anomalies. The standardization performs Z-Score conversion on test indicators and uses Min-Max normalization on image data. The multimodal fusion constructs a patient holographic timeline associated with medical events according to the FHIR standard. The processed data is transmitted to the information service layer and the intelligent analysis layer. Information service layer: Implement fine-grained permission control based on the RBAC model, store key operation hashes on the Hyperledger Fabric blockchain, integrate LIS interfaces to automate the inspection process, build a zero-footprint DICOM viewer, deploy the SciBERT-NER model to extract medical entities, and use a temporal convolutional network to analyze disease evolution trends. Intelligent analysis layer: A hybrid storage architecture is used for data lifecycle management, feature engineering is performed on the data, efficient nearest neighbor search and similarity / distance metrics are used for label matching and categorized storage, the contribution of each feature or feature combination to a single prediction result is quantified through SHAP values, interpretation results are generated, and the interpretation results are presented to clinicians or relevant personnel through a query interface.

2. A smart medical big data information management system according to claim 1, characterized in that: The data acquisition layer analyzes and processes unstructured data, including DICOM image processing and text data OCR processing; The specific steps of the DICOM image processing are: Metadata extraction: Use the DICOM standard parsing library to read the file header information, extract structured fields, and match key tags through regular expressions; Pixel Data Conversion: Apply analog-to-digital conversion formulas to image pixel arrays , will store the value Mapping to display grayscale values , is the slope, is the intercept, and the window width and window position are adjusted in combination with VOI LUT, which is expressed as ,in, is the displayed grayscale value after mapping, is the center value, is the window width; Feature Embedding: Use the pre-trained DenseNet-121 model to extract image feature vectors and compress the spatial dimensions into feature representations through global average pooling; The specific steps of the text data OCR processing are: Image preprocessing: grayscale and binarization of scanned documents; Text region detection: Use connected domain analysis to locate text blocks, filter out noisy areas, and retain connected domains whose aspect ratios match the text characteristics; Character recognition and post-processing: Character sequences are recognized using the Tesseract engine and contextually corrected using medical dictionaries.

3. The intelligent medical big data information management system according to claim 1, characterized in that: The specific steps for the data acquisition layer to obtain structured electronic medical records through the integrated hospital information system interface are: Hospital information system interface protocol adaptation: Build an interface engine based on the HL7 v3 standard and define a standardized transmission format including message headers, structured medical record data bodies, and digital signatures; Patient master index matching: A three-order matching algorithm is used to synchronize the patient master index in real time. The field consistency score is calculated through weighted exact matching, the Jaro-Winkler distance is used to measure text similarity, and the patient identity is associated with biometric verification. The field consistency score calculation formula is expressed as follows: ,in, is the field weight, is the matching function, is the value of the i-th feature field in the source system and the value of the i-th feature field in the target system. ≥0.9, the patients were considered to be the same. For records with a value less than 0.9, the Jaro-Winkler distance is calculated and expressed as ,in, For the two strings that need fuzzy matching, is the Jaro distance, is the prefix matching length, is the scaling factor, when the name similarity Manual verification is triggered when the score is ≥0.85 and the gender / date of birth are consistent; CDC change capture: Deploys a CDC component based on database transaction log analysis. By monitoring the log source, it parses the INSERT / UPDATE / DELETE operation types and generates change record metadata including timestamps, primary key values, and data snapshots. Incremental data synchronization: A two-phase commit protocol is used to ensure data consistency. A temporary transaction table is created at the source to record changes to be synchronized. After the distributed transaction coordinator Seata verifies the readiness of all participating nodes, the medical record change event is pushed to the data lake through the Kafka message queue, triggering the intelligent preprocessing layer to update the patient's holographic timeline and start the clinical rule engine.

4. The intelligent medical big data information management system according to claim 1, characterized in that: The missing value processing is carried out by constructing a random forest regression model, generating 5 complete data sets and retaining the specific steps of the uncertainty parameters: Variable correlation analysis: The characteristic variables that are significantly associated with family history are screened by the maximum mutual information coefficient, and a matrix containing the characteristics is constructed; Random forest model training: A random forest regression model was trained on the complete data subset, using an ensemble of k (500) decision trees and setting the minimum number of samples for node splitting to , the feature sampling ratio is , p is the total number of features, and the hyperparameters are optimized by out-of-bag error; Multiple imputation generation: For each missing sample, a trained random forest is used to generate five different predictions. During each imputation, random noise with a normal distribution N is added to the prediction results of each tree in the forest to simulate prediction uncertainty and limit the generated imputed values ​​to a medically reasonable range. Uncertainty parameter retention: saved for each imputed dataset; Downstream analysis fusion: Rubin's Rules are used to merge the results of the five imputed datasets: a weighted average is calculated for each cluster center coordinate; the imputed variance component is included in the calculation of the within-cluster variance; and finally a clustering result with an uncertainty interval is generated.

5. The intelligent medical big data information management system according to claim 3, characterized in that: The information service layer builds a full-process medical information service system covering authority management, operation auditing, inspection warning, image browsing, text analysis and trend prediction, including the following steps: Fine-grained permission control based on the RBAC model: building a three-level permission system, using the access control matrix for permission inheritance and dynamic allocation: defining role sets ,in For patients, For doctors, For administrators, create a permission-operation mapping table that includes multiple basic operations and delegate permissions through the role inheritance tree: ← Indicates that the doctor role inherits the patient's basic access rights. The access decision function uses Boolean matrix operations and is expressed as , where Roles(u) is the set of user roles, Assign a resource-role matrix; Hyperledger Fabric blockchain evidence storage: Deploys a consortium chain network to implement evidence storage for key operations and uses an improved UTXO model to record audit trails. When a doctor submits an electronic prescription, a transaction is generated containing the operation type, timestamp, and prescription hash. After consensus among endorsing nodes, it is stored in a block, forming an unalterable audit chain. LIS interface integration and critical value warning engine: Integrate laboratory information systems through HL7 standards to build a real-time warning pipeline; Zero-footprint DICOM viewer: A plug-in-free image viewer developed based on WebGL, using tile loading and progressive rendering.

6. The intelligent medical big data information management system according to claim 1, characterized in that: The intelligent analysis layer builds a three-level storage system, which distinguishes data storage levels by defining data temperature thresholds. Data with high frequency access in the past 30 days is considered hot data, data with access from 30 days to one year is considered warm data, and data with low frequency access over one year is considered cold data. An automated migration strategy is deployed, access patterns are monitored through Hive metadata, and data migration tasks are triggered.

7. The intelligent medical big data information management system according to claim 1, characterized in that: The intelligent analysis layer performs feature engineering on the data. The following steps are involved: Recursive feature elimination: Automatically screen the feature subset most relevant to the medical goal and calculate the feature weight based on the ensemble learning model. The calculation formula is expressed as ,in, is the model coefficient of the feature in the k-fold cross-validation, K is the number of cross-validation folds, sort by feature importance, remove the least important set of features each time, evaluate the model performance of the retained feature subset on the validation set, and select the number of features corresponding to the performance inflection point; Uniform manifold approximation and projection: Perform nonlinear dimensionality reduction and structure preservation on high-dimensional medical data, and calculate the local similarity between data points. The calculation formula is expressed as ,in, is the local connectivity parameter determined by binary search, represents the probability that point j is a neighbor of point i, minimizing the KL divergence loss through stochastic gradient descent; Multimodal Transformer Cross Attention Modeling: Fusing text, image, and numerical data, capturing cross-modal interaction patterns, setting modality-specific encoders, including text branch, image branch, and numerical branch. The text branch generates contextual embedding representation through SciBERT ,in, For text, the image branch uses ResNet-50 to extract visual features represented as ,in, is the feature map, and the numerical branch is represented as ,in, The attention weights of the text branch, image branch, and numerical branch are determined by the cross attention mechanism, and the output of each modality is dynamically fused through the gating mechanism. , where σ is the Sigmoid activation function and ⊙ is the element-wise multiplication.

8. The intelligent medical big data information management system according to claim 1, characterized in that: In the intelligent analysis layer, efficient nearest neighbor search is performed through locality-sensitive hashing, which allows rapid location of approximate nearest neighbors in medical data. This involves the following steps: Hash function family design: Construct a random projection hash function to map high-dimensional data to low-dimensional hash buckets, expressed as ,in, is a random unit vector, is the input data vector, b is the random translation amount, and w is the bucket width parameter; Multi-table overlay strategy: Combine multiple hash tables through logical "and" and "or" operations, expressed as ,in, is the number of hash tables for the "AND" operation, For the number of hash functions in each "AND" group, the multi-table overlay strategy strikes a balance between recall rate and query latency; Dynamic index update: A sliding window mechanism is used to maintain index freshness, which is expressed as , where τ is the time window length, and only the data within the most recent τ is retained to participate in the hash mapping.

9. The intelligent medical big data information management system according to claim 8, characterized in that: Using similarity / distance metrics to sort and classify hash candidate sets includes the following steps: Cosine similarity calculation: Calculating cosine similarity is used to measure the directional consistency of text or high-dimensional sparse data, expressed as ,in, is the feature vector of text or high-dimensional sparse data, is a vector The module length, is a vector Length of the module; Mahalanobis distance correction: The dimension and correlation effects of structured data are eliminated by Mahalanobis distance correction, which is expressed as ,in, Represents two different data points, Represents a vector The transpose of Represents the inverse matrix of the feature covariance matrix Σ, Σ is the feature covariance matrix, estimated by sliding window, expressed as , where n represents the number of samples, that is, the number of data points contained in the data set. represents the i-th sample vector, representing the i-th data point in the data set, is the sample mean vector, which represents the average value of all data points in the data set; Hybrid sorting: Combine the number of hash collisions and the similarity score to perform weighted fusion, and sort based on the fused data.

10. The intelligent medical big data information management system according to claim 6, characterized in that: The intelligent analysis layer quantifies the contribution of each feature or feature combination to a single prediction result through SHAP values, and generates the following specific steps to explain the results: Cooperative game theory modeling: The prediction problem is modeled as a cooperative game, where the features are the participants and the model output is the payoff, expressed as , where F is the entire feature set and S is the feature subset. is the model prediction value when only subset S is used, is the Shapley value of feature i, indicating its average marginal contribution to the prediction result; Feature interaction effect decomposition: The SHAP interaction value is used to capture the synergy between features, expressed as , decomposing the total contribution into main effects + and interaction effects ; Sample-level explanation generation: Generate an explanation vector for each prediction result, expressed as , including first-order feature contribution and second-order interaction contribution.

Images