A lymphoma special disease library system based on multi-dimensional data

Abstract

This invention discloses a lymphoma-specific database system based on multi-dimensional data, aiming to address the problems of heterogeneous lymphoma data, inefficient treatment, and limited application. It comprises a data acquisition layer, a treatment layer, and an application layer connected sequentially. The acquisition layer connects to HIS, EMR, LIS, and PACS systems via API, and utilizes ETL to achieve full and incremental acquisition of historical data, obtaining multi-dimensional data including clinical, pathological, and imaging data. The treatment layer de-identifies sensitive information, cleans it, matches it with the EMPI master index, and structures it using a BioBERT pre-trained model for NLP, outputting standardized data. The application layer builds a database with 662 fields (demographic, clinical, etc.) according to standards such as OMOP, providing research analysis functions. It also constructs an AI module using random forests / neural networks to predict disease recurrence, treatment efficacy, and toxicity, and supports federated learning for cross-institutional data sharing. The system improves data quality and contributes to precision medicine and research in lymphoma.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, specifically to a lymphoma-specific database system based on multi-dimensional data. Background Technology

[0002] Lymphoma is a highly heterogeneous malignant tumor with complex pathological types. Its incidence in China has been rising steadily in recent years, with an annual growth rate of 4%-6%, and more than 90,000 new cases in 2020. Clinical diagnosis and research rely on multi-dimensional data, including demographic, clinical characteristics, pathology, molecular testing, imaging, and follow-up information. However, existing lymphoma databases have significant shortcomings: First, the data sources are highly heterogeneous, involving multiple systems such as HIS, EMR, LIS, and PACS, with inconsistent formats, leading to integration difficulties; second, traditional data governance relies on manual entry, which is inefficient, has a high missing rate, and lacks automated cleaning and standardized processes; third, although existing systems partially adopt international standards such as OMOP, CDASH, and SNOMED, ​​the governance process is fragmented and lacks end-to-end automation mechanisms, making it difficult to guarantee data integrity, consistency, and timeliness; fourth, the application scope is limited, with most databases only supporting basic queries and statistics, lacking AI-driven predictive functions, and failing to meet the needs of real-world research for predicting disease relapse, efficacy, and toxicity. Furthermore, cross-institutional data sharing is restricted by privacy and security concerns, hindering multi-center collaboration.

[0003] To address the aforementioned issues, existing technologies have attempted data governance using ETL (Extract, Transform, Load) tools or NLP techniques. However, these technologies suffer from incomplete overall architectures. For example, the incremental update mechanism at the data acquisition layer is flawed, the EMPI matching accuracy at the governance layer is low, and the integration of AI models at the application layer is insufficient. The system architecture and method of this invention specifically address the core problems of data heterogeneity, low governance efficiency, and limited application scenarios. Through a layered design, it achieves automated processing and intelligent application of multi-dimensional data throughout the entire process. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a lymphoma-specific database system based on multi-dimensional data to solve the problems mentioned in the background.

[0005] To achieve the above objectives, a specific embodiment of the present invention provides a lymphoma-specific disease database system based on multi-dimensional data, comprising a data acquisition layer, a data governance layer, and a disease database application layer connected in sequence. The data acquisition layer is used to collect multi-dimensional data on the clinical characteristics, pathological / immunohistochemistry, molecular detection, imaging, treatment process, follow-up, and outcomes of lymphoma patients from multi-source heterogeneous systems such as Hospital Information System (HIS), Electronic Medical Record System (EMR), Laboratory Information System (LIS), and Picture Archiving System (PACS). The data governance layer is used to desensitize, clean, establish a unified identity identifier based on the Patient Master Index (EMPI), and perform structured processing using natural language processing technology on the multi-dimensional data collected by the data acquisition layer. The disease database application layer is used to store the standardized data processed by the data governance layer and includes a research analysis workbench and an AI prediction model module. The AI ​​prediction model module is used to predict disease recurrence, efficacy, and toxicity based on the multi-dimensional data.

[0006] According to an embodiment of this application, a lymphoma-specific disease database system based on multi-dimensional data achieves full-process integration from multi-source heterogeneous systems to intelligent applications through a layered architecture design. The system adopts automated data acquisition and processing technology, effectively solving the industry problems of inconsistent medical data formats and low processing efficiency, and transforming unstructured text into standardized and analyzable data. By constructing a disease database application layer that integrates a scientific research analysis platform and an AI prediction model, it provides clinicians and researchers with powerful data search, statistical analysis, and disease prediction functions.

[0007] In addition, the lymphoma disease database system based on multi-dimensional data proposed in this application may also have the following additional technical features: In one embodiment of this application, the data acquisition layer includes a multi-source data interface module and an ETL extraction module; the multi-source data interface module connects to the hospital information system (HIS), electronic medical record system (EMR), laboratory system (LIS), and imaging system (PACS) through API interfaces, supporting the full acquisition of historical data and the acquisition of incremental data during operation; the ETL extraction module realizes the automatic extraction, transformation, synchronization, and aggregation of data, and adopts an incremental update mechanism to ensure that the timeliness of data is less than 24 hours.

[0008] In one embodiment of this application, the data governance layer includes: a desensitization module for desensitizing sensitive personal information, including patient names and ID numbers; a cleaning module for standardizing the format of collected data, mapping it to a medical dictionary, and constraining its value range; an EMPI master index module for generating a globally unique identifier for each patient based on a matching algorithm for key information such as ID number, name, and gender; and a structuring module for converting unstructured text information in pathology reports and follow-up records into structured, analyzable data using natural language processing (NLP) technology.

[0009] In one embodiment of this application, the application layer of the disease database includes: a storage module, which constructs a lymphoma disease database based on the international standards OMOP, CDASH, and SNOMED, ​​and the database covers 662 standardized data fields including demographics, diagnosis and treatment, imaging, pathology, treatment and follow-up; a research analysis workbench module, which supports precise patient search based on keywords and logical combinations, and provides statistical functions including frequency analysis, chi-square test, and cluster analysis; and an AI prediction module, which is constructed based on machine learning algorithms and is used to predict the recurrence risk, treatment effect and drug toxicity of lymphoma.

[0010] In one embodiment of this application, the AI ​​prediction module includes a feature extraction submodule, a model training submodule, and a prediction output submodule; the model training submodule is configured to run a random forest algorithm and a neural network algorithm for model training; the prediction output submodule is used to output a risk score, decision tree visualization results, and clinical net benefit assessment.

[0011] In one embodiment of this application, the system also supports federated learning technology, which enables collaborative model training and optimization between disease database systems of different institutions, achieving cross-institutional data sharing and application without transmitting the original data.

[0012] The data governance method of the system includes the following steps: Multidimensional lymphoma data are collected from multiple heterogeneous systems, including Hospital Information System (HIS), Electronic Medical Record System (EMR), Laboratory System (LIS), and Picture Archiving System (PACS), through the data acquisition layer. The data governance layer sequentially performs data anonymization, cleaning, EMPI master index creation, and natural language processing structured processing on the collected data. The standardized data after treatment is stored in the application layer of the disease database, and users are provided with scientific research analysis and AI-based disease prediction functions.

[0013] In one embodiment of this application, the natural language processing structured processing step uses a BioBERT-based pre-trained model to parse unstructured text and extract key medical terms and clinical entities.

[0014] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method.

[0015] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0016] The advantages of this invention compared to existing technologies are: (1) The data is fully automated from collection to application through a layered architecture, improving data integrity (>95%), consistency (>98%), accuracy (>97%) and timeliness (<24 hours).

[0017] (2) Based on international standards, 662 fields are defined, supporting multi-center sharing and federated learning, which facilitates cross-institutional cooperation.

[0018] (3) The integrated AI prediction model can achieve an AUC of over 0.85, providing clinical net benefit assessment and assisting in precision medicine for lymphoma.

[0019] (4) By using NLP and EMPI technologies, the utilization rate of unstructured data can be increased by 30%, and labor costs can be reduced.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall architecture of a lymphoma disease database system based on multi-dimensional data in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall system architecture of a lymphoma disease database system based on multi-dimensional data, according to one embodiment of the present invention. Figure 3This is a detailed flowchart of data governance for a lymphoma disease database system based on multi-dimensional data, according to one embodiment of the present invention. Figure 4 This is a flowchart of an AI prediction module for a lymphoma disease database system based on multi-dimensional data, according to one embodiment of the present invention. Figure 5 This is a flowchart illustrating the data quality control and monitoring process of a lymphoma specialty database system based on multi-dimensional data, as described in one embodiment of the present invention. Figure 6 This is a flowchart of a federated learning inter-institutional collaboration process for a lymphoma specialty database system based on multi-dimensional data, as described in one embodiment of the present invention. Figure 7 This is a flowchart illustrating the EMPI master index generation process of a lymphoma disease database system based on multi-dimensional data, according to one embodiment of the present invention. Figure 8 This is a flowchart of NLP (Natural Language Processing) for a lymphoma disease database system based on multi-dimensional data, according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figures 1 to 8 As shown in the figure, a lymphoma disease database system based on multi-dimensional data according to an embodiment of the present invention may include a data acquisition layer, a data governance layer and a disease database application layer connected in sequence.

[0025] The lymphoma specialist database system in this embodiment is deployed within a medical institution's local area network environment. It adopts a distributed architecture design, supporting single-machine deployment and multi-node cluster expansion. The following environment configurations must be completed before system deployment: 1. Hardware environment: The server uses an Intel Xeon Gold series processor (≥8 cores), memory ≥32GB, and storage uses 1TB or more SSD hard drives (supporting RAID5 redundancy backup); the client computer must support Chrome 90.0 or above or Edge 88.0 or above browser, and the network bandwidth must be ≥100Mbps.

[0026] For small medical institutions (less than 1000 patients), a basic configuration with 16GB of memory is recommended; for large centers (more than 10,000 patients), a cluster deployment with 64GB of memory is recommended, supporting horizontal scaling. Network bandwidth needs to be dynamically adjusted according to data volume to ensure a transmission rate of ≥100Mbps.

[0027] 2. Software Environment: The server operating system is CentOS 7.9, the database is PostgreSQL 13 (supporting JSON data type and spatial index), the programming language is Python 3.9, the deep learning framework is TensorFlow 2.10, the ETL tool is Apache Airflow, NLP processing relies on the BioBERT pre-trained model (based on TensorFlow), and the federated learning framework is FedML 0.8.0.

[0028] 3. Network environment: Configure internal LAN HTTPS encrypted transmission (TLS1.3 protocol), set firewall policies to only open necessary ports (database port 5432, application service port 8080, API interface port 443), and support secure network access across departments and institutions.

[0029] I. Specific Implementation of Each Module in the System (I) Implementation of the data acquisition layer The data acquisition layer consists of a multi-source data interface module and an ETL extraction module. Its core function is to collect lymphoma-related multi-dimensional data from multi-source heterogeneous systems to ensure the comprehensiveness and timeliness of the data.

[0030] 1. Multi-source data interface module This module establishes connections with the hospital's existing HIS, EMR, LIS, and PACS systems through standardized API interfaces. The interfaces adopt RESTful design specifications and support HTTP / HTTPS protocol communication.

[0031] Interface adaptation methods: Dedicated adapter plugins are developed to address the interface differences between different source systems. For example, when interfacing with a HIS system, basic patient information, medical records, and cost information are obtained through the standard data access interface provided by the HIS vendor (such as HL7 FHIRAPI); when interfacing with an EMR system, the electronic medical record text interface is used to extract progress notes, medical orders, and examination reports; when interfacing with a LIS system, laboratory data such as complete blood count, biochemical indicators, and immunohistochemistry results are obtained through the laboratory data interface; and when interfacing with a PACS system, imaging data and image report texts such as CT, MRI, and PET-CT are obtained through the DICOM protocol interface.

[0032] Data Acquisition Modes: Supports two modes: full historical data acquisition and real-time incremental acquisition. Full historical data acquisition is achieved by triggering an interface call once to retrieve lymphoma patient-related data stored before system deployment in batches. During the acquisition process, a batch transmission method (1000 records per batch) is used to avoid data congestion. Incremental acquisition is implemented through two mechanisms: timed polling (once per hour by default) and event triggering. During timed polling, the interface only requests data added since the previous round of acquisition. Event triggering is achieved by listening for data addition / modification events in the source system (such as EMR medical record signing events and LIS test result review events) to trigger data acquisition requests in real time.

[0033] Data integrity assurance: A retry mechanism is added during the interface call process. When the network is interrupted or the interface response times out (the timeout threshold is set to 30 seconds), it will automatically retry 3 times, with retry intervals of 5 seconds, 10 seconds and 15 seconds respectively. After each collection is completed, a collection log is generated to record information such as the source system of the collected data, the amount of data, the collection time and whether it was successful, so as to facilitate subsequent verification.

[0034] If the retry fails, the system will automatically switch to degrade mode, use cached data or trigger a manual intervention process, and record the exception log.

[0035] 2. ETL Extraction Module This module is responsible for extracting, transforming, synchronizing, and aggregating data, and uses a visual process configuration tool (Apache Airflow) to define the ETL task process.

[0036] Data Extraction: Target data is extracted from the databases of various source systems using Structured Query Language (SQL). For unstructured data (such as image files and PDF reports), a file stream extraction method is used, and the data is stored on a temporary file server. During the extraction process, different extraction strategies are set according to the data type. For example, for structured data (such as patient age, gender, and test values), field values ​​are directly extracted; for semi-structured data (such as JSON format medical order data), key fields are extracted after parsing; and for unstructured data, the original file is retained and file metadata (such as file name, creation time, and file size) is extracted.

[0037] Data Conversion: Standardize and convert data based on differences in data formats from different source systems. For example, convert date formats (such as "2025-11-18", "2025 / 11 / 18", "18-11-2025") from different HIS systems to "YYYY-MM-DD" format; standardize the units of test indicators (such as converting "mg / dL" to "mmol / L"); mark missing key fields (such as patient gender and age) as "unknown", and retain blank values ​​for missing non-key fields.

[0038] Data synchronization and aggregation: An incremental update mechanism is adopted, which determines whether new or changed data is added or changed based on the data's timestamp field (such as data creation time and modification time). Only the changed parts are synchronized, ensuring that the data timeliness is less than 24 hours. After synchronization is completed, all data is aggregated to a temporary data storage area (mirror library). The mirror library uses the same data structure as the source system to facilitate data traceability.

[0039] (II) Implementation of the Data Governance Layer The data governance layer includes a de-identification module, a cleaning module, an EMPI master index module, and a structuring module. Through a series of automated processing steps, it transforms the collected raw data into standardized, structured, and usable data.

[0040] 1. Desensitization module This module de-identifies sensitive personal information such as patient names, ID numbers, phone numbers, and home addresses to ensure data privacy and security. The processing method is as follows: ID card number: Keep the first 6 digits (administrative division code) and the last 4 digits, and replace the middle part with "****", for example, "510105********1234"; Name: Retain the surname, replace the given name with “”, replace two-character names with one “”, and replace three-character names with two “”, for example, “Zhang” or “Li**”; Phone number: Keep the first 3 and last 4 digits, and replace the middle part with "", for example, "1385678"; Home address: Keep the province, city, and district (county) information. Replace the detailed address with “”, for example, “Qingyang District, Chengdu, Sichuan Province”.

[0041] During the de-identification process, a de-identification mapping table is generated (stored in an encrypted database) to record the correspondence between the original information and the de-identified information. Only authorized administrators can query the mapping relationship under the premise of compliance.

[0042] 2. Cleaning module This module further cleans the anonymized data, resolving issues such as inconsistent data formats, abnormal value ranges, and logical conflicts. The specific process is as follows: Standardized format conversion: Except for date format, numeric data will be uniformly converted to floating point (keeping 2 decimal places), and text data will have leading and trailing spaces and special characters (such as newline characters and tab characters) removed, and the character encoding will be uniformly set to UTF-8; Medical dictionary mapping: A data mapping table is established based on standard medical dictionaries such as the International Classification of Diseases, 10th Revision (ICD-10) and the Systematic Terminology Collection for Medicine (SNOMEDCT). For example, different expressions of "diffuse large B-cell lymphoma" in different source systems (such as "DLBCL", "diffuse large B-cell lymphoma", "diffuse large B-cell lymphoma") are uniformly mapped to the standard term "diffuse large B-cell lymphoma (ICD-10 code: C83.3)"; test item names are mapped to LOINC codes; Value range compliance constraints: Based on medical common sense and business rules, set the legal value range for each field. For example, the patient age value range is set to 0-120 years old, and data outside this range is marked as abnormal and automatically corrected to "unknown"; the body temperature value range is set to 35.0-42.0℃, and abnormal data triggers a manual review process; Logical conflict verification: Check the logical consistency between data. For example, a patient's consultation date must not be earlier than their birth date, and the treatment start date must not be later than the follow-up date. If a logical conflict exists, mark the conflict type and log it. The data administrator will then manually verify and correct it.

[0043] 3. EMPI Master Index Module This module generates a globally unique patient master index (EMPI) based on key patient information, ensuring consistent patient identity across different systems and medical records. The specific implementation is as follows: Key matching fields selection: ID number (before anonymization) is used as the core matching field, supplemented by name (before anonymization), gender, date of birth, and contact number (before anonymization) as auxiliary matching fields; Matching algorithm process: The first step is to preprocess the collected patient information, removing rare and variant characters from the names, standardizing gender representation ("male", "female", "unknown"), and standardizing the date of birth format; The second step is to prioritize precise matching based on the ID number. If the ID numbers of two records are exactly the same, they are determined to be the same patient and assigned the same EMPI. The third step involves using a fuzzy matching algorithm to calculate the similarity of the combination of name, gender, date of birth, and contact number if the ID number is missing or inconsistent. The similarity calculation uses a cosine similarity algorithm with a similarity threshold of 90%. When the combination similarity is ≥90%, the patient is considered to be the same and assigned the same EMPI. When the similarity is between 80% and 90%, a manual review process is triggered, and the administrator confirms whether the patient is the same. When the similarity is <80%, the patient is considered to be a different patient and assigned a new EMPI. EMPI Generation Rule: EMPI adopts the UUID v4 format, consisting of 32 hexadecimal characters (such as "550e8400-e29b-41d4-a716-446655440000"). After generation, it is associated and stored with the desensitized patient information as the unique identity identifier of the patient in the system.

[0044] For patients without ID numbers, a combination of name, gender, date of birth, and contact phone number is used for matching. The similarity calculation threshold is adjusted to 85%, and an artificial review process is triggered.

[0045] 4. Structured Module This module uses natural language processing (NLP) technology and is based on the BioBERT pre-trained model to convert unstructured text data such as pathological reports, follow-up records, and course records into structured data. The specific process is as follows: Text Preprocessing: Clean the input unstructured text, remove punctuation marks, special characters, redundant spaces, and perform word segmentation (using the jieba word segmentation tool, combined with a custom dictionary in the lymphoma field, including professional terms such as "lymphoma", "immunohistochemistry", "chemotherapy", "radiotherapy", etc.), and remove stop words (such as meaningless words like "de", "le", "zai", etc.); BioBERT Model Fine-tuning: Based on publicly available medical text corpora (such as PubMed abstracts, Chinese medical literature databases) and lymphoma field-specific corpora (including lymphoma pathological reports, follow-up records, etc. from Sichuan Provincial People's Hospital in the past 5 years), fine-tune the BioBERT pre-trained model to optimize the accuracy of the model in processing lymphoma field texts. During the fine-tuning process, use the stochastic gradient descent (SGD) optimizer, set the learning rate to 2e-5, the batch size to 32, and the number of training rounds to 10 rounds; Entity and Term Extraction: Use the fine-tuned BioBERT model to extract key medical entities and terms in the text, including disease names (such as "diffuse large B-cell lymphoma", "follicular lymphoma"), pathological features (such as "CD20 positive", "Ki-67 index 30%"), treatment regimens (such as "chemotherapy with R-CHOP regimen", "rituximab combined with radiotherapy"), follow-up results (such as "no recurrence", "disease progression"), etc.; Structured Mapping: According to the 662 standardized data fields defined by the system, establish the mapping relationship between the extracted entities and the target fields, and fill the extracted key information into the corresponding fields. For example, the "CD20 positive" extracted from the pathological report is mapped to the "Immunohistochemistry Index - CD20" field, and the "chemotherapy with R-CHOP regimen" extracted is mapped to the "Treatment Regimen" field to form a structured data record.

[0046] (III) Implementation of the Application Layer of the Special Disease Database The application layer of the disease database includes a storage module, a research analysis workbench module, and an AI prediction module, which are responsible for the standardized storage and intelligent application of data, providing support for clinical diagnosis and treatment and scientific research.

[0047] 1. Storage module This module, based on international standards such as OMOP (Observational Medical Outcomes Collaboration), CDASH (Harmonized Guidelines for Clinical Data Acquisition Standards), and SNOMED (Medical Systems Terminology), constructs a lymphoma-specific database, specifically implemented as follows: Data Model Design: Following international standards and considering the characteristics of lymphoma diagnosis and treatment, the database table structure was designed, encompassing 662 standardized data fields, divided into 6 main categories: (1) Demographic information (32 fields): including patient EMPI, gender, age, date of birth, ethnicity, place of origin, etc.; (2) Clinical diagnosis and treatment information (186 fields): including date of visit, chief complaint, present illness, past medical history, physical examination results, diagnosis conclusion, etc.; (3) Imaging information (98 fields): including imaging examination type, examination date, description of imaging features, lesion size, location, etc.; (4) Pathological information (125 fields): including pathological type, immunohistochemical indicators, molecular detection results (such as MYC gene rearrangement status), etc. (5) Treatment information (113 fields): including treatment plan, medication dosage, treatment cycle, treatment start time, treatment end time, etc.; (6) Follow-up information (108 fields): including follow-up date, follow-up method, survival status, recurrence, adverse reactions, etc.; Database optimization: A partitioned table design is adopted, with partitions based on the year of patient visit to improve data query efficiency; B-tree indexes are created for frequently used query fields (such as EMPI, diagnosis conclusion, and treatment plan); for large files such as image data, a storage method combining database and file system is adopted, with the database storing file paths and metadata, and the file system storing the original image files to support fast access; Data backup: The backup strategy adopts "daily incremental backup + weekly full backup". Incremental backup data is stored on the local server, while full backup data is stored in an off-site backup center (≥50 kilometers away from the main data center). Backup data is retained for 6 months and supports rapid recovery after accidental data deletion or system failure.

[0048] 2. Scientific Research Analysis Workbench Module This module provides researchers with precise patient search and statistical analysis functions, specifically implemented as follows: Precise search function: (1) Keyword indexing: Using the Elasticsearch search engine, a full-text index is built on the text fields in the database (such as diagnostic conclusions, pathology reports, and treatment plans) to support fast keyword retrieval; (2) Search logic implementation: Supports single keyword search and multi-keyword logical combination search (AND / OR / NOT). After the user enters keywords (such as "DLBCL", "relapse", "R-CHOP"), the system parses the meaning of the keywords and matches the corresponding database fields. Based on the logical relationship selected by the user (such as "DLBCLAND relapse AND NOT R-CHOP"), the system constructs a query statement and retrieves patient records that meet the conditions. (3) Search results display: Search results are displayed in a list format, including key information such as patient EMPI, gender, age, diagnosis, treatment plan, and follow-up results. Users can click on a single record to view detailed multi-dimensional data (including imaging reports, original pathology reports, immunohistochemistry results, etc.), and the results can be exported (Excel, CSV format). Statistical analysis functions: (1) Frequency analysis: Statistically analyze the distribution of values ​​of specified fields, such as the number and percentage of patients with different pathological types. The calculation method is the number of patients with a certain pathological type / the total number of patients × 100%. It supports the generation of visual charts such as bar charts and pie charts. (2) Chi-square test: used to analyze the association between two categorical variables, such as analyzing the association between “treatment plan” (R-CHOP plan, CHOP plan) and “treatment effect” (effective, ineffective). The system automatically calculates the chi-square value, degrees of freedom and p-value, and judges whether the association is significant based on the p-value (with a significance level of 0.05). (3) Cluster analysis: The K-means clustering algorithm is used to group patients according to their multi-dimensional characteristics (such as age, pathological type, immunohistochemical indicators, and treatment plan). Users can customize the number of clusters (K value). The system outputs the clustering results and principal component analysis diagram to help identify disease subtypes and treatment patterns.

[0049] 3. AI Prediction Module This module uses machine learning algorithms to build predictive models for lymphoma recurrence risk, treatment efficacy, and drug toxicity. The specific implementation is as follows: The feature extraction submodule extracts features required for the prediction model from the standardized data after treatment, including clinical features (age, gender, pathological type, disease stage), imaging features (lesion size, number, metabolic activity), molecular features (immunohistochemical indicators, gene detection results), and treatment features (treatment plan, drug dosage, treatment cycle), extracting a total of 128 core features. The extracted features undergo preprocessing, including missing value imputation (mean imputation for numerical features, mode imputation for categorical features), feature standardization (converting numerical features to values ​​in the [0,1] interval), and feature selection (using recursive feature elimination (RFE) to select the 64 features that contribute the most to the prediction results). Feature scaling employs the Min-Max normalization method, mapping numerical features to the [0,1] interval, as shown in the formula: For categorical features, one-hot encoding is used to ensure consistency of model input.

[0050] The model training submodule supports random forest and neural network algorithms to build prediction models. The specific process is as follows: (1) Data set partitioning: The patient data in the system is divided into a training set (for model training), a validation set (for model parameter adjustment) and a test set (for model performance evaluation) in a ratio of 7:2:1. Stratified sampling is used in the partitioning process to ensure that the disease types and treatment plans of each dataset are consistent. (2) Model parameter settings: Random Forest Model: The number of decision trees is set to 100, the maximum depth of each decision tree is set to 10, the minimum number of samples for node splitting is set to 2, and the minimum number of samples for leaf nodes is set to 1. Neural network model: A 3-layer fully connected neural network is used. The input layer has 64 neurons (corresponding to 64 selected features), the hidden layer has 2 layers (the first layer has 32 neurons and the second layer has 16 neurons), and the output layer has 1 neuron (outputs 0 or 1 for binary classification problems and continuous values ​​for regression problems). The activation functions used are ReLU (hidden layer) and Sigmoid (output layer). (3) Model training and optimization: The model is trained using the training set, and the model performance is monitored using the validation set. Training is stopped when the accuracy on the validation set no longer improves for three consecutive rounds to avoid overfitting. Early stopping and L2 regularization (penalty coefficient 0.01) are used to optimize the model and improve its generalization ability. The prediction output submodule receives multi-dimensional patient data uploaded by the user (supports importing Excel templates, which contain 662 standardized fields), calls the trained prediction model for calculation, and outputs the prediction results. (1) Risk score: The recurrence risk score ranges from 0 to 1. The higher the score, the higher the risk of recurrence. Among them, 0-0.3 is low risk, 0.3-0.7 is medium risk, and 0.7-1.0 is high risk; The treatment effect score ranges from 0 to 1. The higher the score, the higher the probability of effective treatment. (2) Decision tree visualization: For the random forest model, output decision tree visualization charts to show the key nodes of the model's judgment (such as the threshold of a certain immunohistochemical indicator, age segmentation) to help doctors understand the basis of prediction; (3) Clinical net benefit assessment: Based on the predicted results and the actual clinical situation, calculate the clinical net benefit value (considering the trade-off between treatment benefits and risks), output an assessment report, and provide a reference for the selection of treatment options.

[0051] (iv) Implementation of Federated Learning Technology This system supports federated learning technology, enabling cross-institutional data sharing and collaborative model training without transmitting raw data. The specific implementation is as follows: Architecture Design: The architecture adopts a distributed architecture of "Federated learning server + local systems of each institution". The federated learning server is deployed on a neutral third-party node (such as a regional medical data center), and each participating institution (such as Sichuan Provincial People's Hospital, Xinqiao Hospital, etc.) deploys a local system, which includes a local database and a local model training module. Data preprocessing: Each institution's local system performs data governance (de-identification, cleaning, and structuring) on ​​its own lymphoma patient data in a manner consistent with its own system, ensuring that the data format and standards are consistent, without the need to transfer raw data to other institutions or federated learning servers; Local model training: Each institution initializes a model (random forest or neural network) consistent with the AI ​​prediction module of its own system based on the preprocessed data, and performs local model training. During the training process, the model parameters (such as the decision tree structure of the random forest, the weights and biases of the neural network) are recorded. Model parameter transmission: Each institution transmits the trained local model parameters to the federated learning server through an encrypted channel (using the AES-256 encryption algorithm). Only the model parameters are sent during the transmission process, and no original patient data is included. Parameter aggregation and optimization: The federated learning server uses the federated averaging algorithm (FedAvg) to aggregate the model parameters uploaded by each institution and calculate the average value of the parameters as the global model parameters. If an institution has a large amount of data, a weight coefficient (positively correlated with the amount of data) can be set to increase the contribution of the institution's parameters in the aggregation. Global model distribution: The federated learning server distributes the aggregated and optimized global model parameters to each participating institution through an encrypted channel. After receiving the parameters, each institution updates its local model parameters and completes one round of federated learning training. Iterative training: Repeat the above steps of local training, parameter transfer, aggregation optimization, and model distribution until the model performance (such as AUC value) reaches a preset threshold (≥0.85), then stop the iteration. Finally, each institution obtains a global model with optimized performance, which can be used for predictive analysis of local patients.

[0052] Neutral third-party nodes can be deployed on cloud platforms (such as Alibaba Cloud or Tencent Cloud), using containerization technology (Docker) to ensure isolation. Server configuration must support HTTPS two-way authentication, and model parameter aggregation should be set to once every 24 hours to reduce network load.

[0053] II. Specific Implementation Process of Data Governance Methods The data governance method of the lymphoma disease database system based on multi-dimensional data of the present invention includes the following specific implementation steps: 1. Data Acquisition Steps: Activate the multi-source data interface module of the data acquisition layer, connect to multi-source heterogeneous systems such as HIS, EMR, LIS, and PACS through the API interface, select the full or incremental acquisition mode of historical data according to actual needs, and collect multi-dimensional data such as clinical characteristics, pathology / immunohistochemistry, molecular detection, imaging, treatment process, follow-up and outcome of lymphoma patients. The ETL extraction module completes the extraction, transformation, synchronization and aggregation of data and stores it in a temporary mirror library. 2. Data desensitization steps: Call the desensitization module of the data governance layer to desensitize sensitive information such as patients' names, ID numbers, phone numbers, and home addresses in the mirror database. After processing, generate a desensitization log to record the desensitized fields and processing methods. 3. Data cleaning steps: Start the cleaning module to perform format standardization conversion, medical dictionary mapping, value range compliance constraints and logical conflict verification on the de-identified data, remove invalid and abnormal data, mark abnormal data that cannot be automatically corrected and trigger manual review, and data that passes the review will proceed to the next step of processing; 4. Steps for establishing the EMPI master index: Through the EMPI master index module, based on key information such as ID number, name, gender, and date of birth, a combination algorithm of exact matching and fuzzy matching is used to generate a globally unique EMPI for each patient, establish a unified identifier for the patient's identity, and associate all relevant data of the patient; 5. Structured processing steps: Using the structured module, based on the fine-tuned BioBERT pre-trained model, unstructured text data such as pathology reports and follow-up records are pre-processed, entity extracted, and structured mapped, and converted into standardized structured data. 6. Data storage and application steps: Standardized data, after being desensitized, cleaned, indexed by the EMPI master index, and structured, is stored in the storage module of the application layer of the disease database (a disease database based on the OMOP / CDASH / SNOMED standard); users can perform precise patient searches and statistical analysis through the scientific research analysis workbench module, and upload patient data through the AI ​​prediction module to obtain prediction results of disease recurrence, efficacy, and toxicity.

[0054] III. Specific Implementation of Electronic Devices The electronic device of the present invention is used to implement the above-described data governance method, and is specifically configured as follows: Hardware components: including processor, memory, communication interface, input devices, and output devices, among which: (1) Processor: Select an Intel Core i7-12700H or equivalent processor that supports multi-threaded processing, has a clock speed of ≥2.7GHz, and a cache of ≥24MB, for executing computer programs; (2) Memory: including random access memory (RAM, capacity ≥16GB) and read-only memory (ROM), and also including at least one non-volatile storage medium (such as solid-state drive SSD, capacity ≥512GB) for storing computer programs, system configuration files, processed data and logs; (3) Communication interfaces: including Ethernet interface (RJ45), wireless LAN interface (Wi-Fi6) and USB interface (USB3.2), used for data communication with multi-source heterogeneous systems, federated learning servers and client devices; (4) Input devices: including keyboard, mouse and touch screen, used for users to input operation instructions, query conditions, etc.; (5) Output devices: including a monitor (resolution ≥ 1920×1080) and a printer, used to display the system interface, query results, statistical charts and forecast reports, and support print output; Software configuration: (1) Operating system: Install Windows 10 Professional or Linux Ubuntu 20.04 LTS operating system; (2) Computer program: stored in memory, the computer program includes data acquisition module, data governance module, data storage module, scientific research analysis module, AI prediction module and federated learning module, which correspond to the functional implementation of each layer of the system; Workflow: The processor reads and executes the computer program from the memory, receives data from the multi-source heterogeneous system through the communication interface, sequentially calls the data acquisition module and the data governance module to process the data, stores the processed data in the memory, responds to the query, analysis or prediction requests initiated by the user through the input device, calls the scientific research analysis module or the AI ​​prediction module for processing, displays the processing results through the output device, and records the operation log and processing results in the memory.

[0055] IV. Specific Implementation of Computer-Readable Storage Media The computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the above-described data governance method, specifically as follows: Storage media types include, but are not limited to, hard drives, USB flash drives, external hard drives, CD-ROMs, DVDs, Blu-ray discs, flash memory, and cloud storage media. Program storage format: Computer programs are stored in storage media in binary file format, containing program code and configuration files. The program code is written in Python language, and the configuration file is in JSON format, recording system parameters (such as database connection information, model training parameters, interface addresses, etc.). Data reading and execution: The storage medium is connected to the electronic device (such as a USB flash drive connected via a USB interface, or cloud storage connected via a network). The processor of the electronic device reads the computer program in the storage medium through the storage controller, loads it into the memory and executes it. During the execution process, the data collection, processing, storage, analysis and prediction functions are completed in accordance with the steps of the data governance method described above. Storage media protection: Supports encrypted data storage, using the AES-256 encryption algorithm to encrypt computer programs and stored patient data to prevent data leakage; supports data backup and recovery, allowing data in the storage media to be backed up to other storage devices, and data can be recovered from the backup device when the storage media is damaged or data is lost.

[0056] The technical solutions described in the above embodiments of this application achieve full-process integration from multi-source heterogeneous systems to intelligent applications through a layered architecture design. The system adopts automated data acquisition and governance technology, which effectively solves the industry problems of inconsistent medical data formats and low governance efficiency, and transforms unstructured text into standardized and analyzable data. By constructing a disease database application layer that integrates a scientific research analysis platform and an AI prediction model, it provides clinicians and researchers with powerful data search, statistical analysis, and disease prediction functions.

[0057] Obviously, the above-described embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A lymphoma-specific database system based on multi-dimensional data, characterized in that, It includes a data acquisition layer, a data governance layer, and a disease database application layer connected in sequence, among which, The data acquisition layer is used to collect multi-dimensional data on the clinical characteristics, pathology / immunohistochemistry, molecular detection, imaging, treatment process, follow-up and outcome of lymphoma patients from a multi-source heterogeneous system including Hospital Information System (HIS), Electronic Medical Record System (EMR), Laboratory System (LIS) and Picture Archiving System (PACS). The data governance layer is used to desensitize, clean, establish a unified identity based on the patient master index (EMPI), and perform structured processing using natural language processing technology on the multi-dimensional data collected by the data acquisition layer. The disease database application layer is used to store standardized data processed by the data governance layer, and it includes a scientific research analysis workbench and an AI prediction model module. The AI ​​prediction model module is used to predict the recurrence, efficacy, and toxicity of diseases based on the multi-dimensional data.

2. The lymphoma specialty database system based on multi-dimensional data according to claim 1, characterized in that, The data acquisition layer includes a multi-source data interface module and an ETL extraction module; The multi-source data interface module connects to the hospital information system (HIS), electronic medical record system (EMR), laboratory system (LIS), and imaging system (PACS) through API interfaces, supporting the full collection of historical data and the collection of incremental data during operation. The ETL extraction module automatically extracts, transforms, synchronizes, and aggregates data, and adopts an incremental update mechanism to ensure that the timeliness of the data is less than 24 hours.

3. The lymphoma specialty database system based on multi-dimensional data according to claim 1, characterized in that, The data governance layer includes: The desensitization module is used to desensitize sensitive personal information, including patient names and ID numbers. The cleaning module is used to perform format standardization conversion, medical dictionary mapping, and value range compliance constraints on the collected data; The EMPI master index module generates a globally unique identity identifier for each patient based on a matching algorithm that uses key information such as ID number, name, and gender. The structured module uses natural language processing (NLP) technology to convert unstructured text information in pathology reports and follow-up records into structured, analyzable data.

4. The lymphoma specialty database system based on multi-dimensional data according to claim 1, characterized in that, The specialized disease database application layer includes: The storage module is based on the international standards OMOP, CDASH, and SNOMED to build a lymphoma-specific database, which covers 662 standardized data fields including demographics, diagnosis and treatment, imaging, pathology, treatment and follow-up. The research analysis workbench module supports precise patient searches based on keywords and logical combinations, and provides statistical functions including frequency analysis, chi-square test, and cluster analysis. The AI ​​prediction module, built on machine learning algorithms, is used to predict the risk of lymphoma recurrence, treatment effectiveness, and drug toxicity.

5. A lymphoma specialty database system based on multi-dimensional data according to claim 4, characterized in that, The AI ​​prediction module includes a feature extraction submodule, a model training submodule, and a prediction output submodule; The model training submodule is configured to run the random forest algorithm and the neural network algorithm for model training; The prediction output submodule is used to output risk scores, decision tree visualization results, and clinical net benefit assessments.

6. A lymphoma-specific database system based on multi-dimensional data according to claim 1, characterized in that, The system also supports federated learning technology, enabling collaborative model training and optimization between disease database systems in different institutions, achieving cross-institutional data sharing and application without transmitting raw data.

7. A data governance method based on the system described in any one of claims 1 to 6, characterized in that, Includes the following steps: Multidimensional lymphoma data are collected from multiple heterogeneous systems, including Hospital Information System (HIS), Electronic Medical Record System (EMR), Laboratory System (LIS), and Picture Archiving System (PACS), through the data acquisition layer. The data governance layer sequentially performs data anonymization, cleaning, EMPI master index creation, and natural language processing structured processing on the collected data. The standardized data after treatment is stored in the application layer of the disease database, and users are provided with scientific research analysis and AI-based disease prediction functions.

8. The method according to claim 7, characterized in that, In the structured processing step of natural language processing, a BioBERT-based pre-trained model is used to parse unstructured text and extract key medical terms and clinical entities.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in claim 7 or 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in claim 7 or 8.

Images