A multi-modal precision medical record image modeling system with clinical verification

Abstract

The application belongs to the technical field of medical imaging, and particularly relates to a multi-modal precision medical record image modeling system with clinical verification, which comprises a multi-modal data access and standardization module, a cross-modal feature alignment module, an intelligent multi-modal fusion module, a lesion feature extraction and quantification module, an interpretable decision modeling module, a medical record modeling module, a clinical interactive verification module, a result output and archiving module; through a three-level collaborative system of feature quantization, interpretable decision and clinical interactive verification, the model decision can be traced, verified and adjusted, the clinical adoption rate is greatly improved, and the system is particularly suitable for diagnosis of difficult cases.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, specifically to a clinically validated multimodal precision medical record image modeling system. Background Technology

[0002] In modern medical diagnosis and treatment systems, medical imaging is the core basis for disease diagnosis, condition assessment, treatment plan formulation, and efficacy follow-up. Medical records containing these images are a crucial component of medical institution medical record management, carrying patient information, treatment processes, and imaging characteristics, and possessing significant value in clinical diagnosis and treatment, medical education and research, legal documentation, and public health research. With the rapid advancement of medical informatization, the number of medical imaging records accumulated by various medical institutions is growing exponentially. How to efficiently manage and accurately model these massive, heterogeneous, and multimodal medical record images, and achieve deep integration of imaging features and medical record information, has become a critical issue that urgently needs to be addressed in the field of medical informatization.

[0003] Currently, medical institutions mainly rely on traditional medical record management systems and basic image processing tools for processing and modeling medical image records. However, existing medical record image modeling systems still have the following technical problems:

[0004] Existing modeling systems are mostly based on deep learning (such as CNN, Transformer, U-Net), which makes it difficult to explain the correlation between lesion features and clinical diagnostic conclusions. Doctors cannot verify the rationality of model decisions, resulting in low clinical adoption rates. Especially in the diagnosis of difficult cases, model results can only be used as a reference and cannot replace human judgment.

[0005] Clinical cases often require comprehensive modeling by combining multimodal images. However, the physical meaning and data dimensions of different modal images vary greatly. For example, CT focuses on anatomical structure, MRI focuses on soft tissue resolution, and PET focuses on metabolic function. Existing fusion algorithms are unable to fully explore cross-modal correlation information, which is prone to information redundancy or missing information.

[0006] To address these issues, a clinically validated, multimodal precision medical record image modeling system is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a clinically validated multimodal precision medical record image modeling system to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a multimodal precision medical record image modeling system with clinical validation, comprising a multimodal data access and standardization module, a cross-modal feature alignment module, an intelligent multimodal fusion module, a lesion feature extraction and quantification module, an interpretable decision modeling module, a medical record modeling module, a clinical interaction validation module, and a result output and archiving module;

[0009] The multimodal data access and standardization module provides a unified data foundation for multimodal fusion, avoiding misalignment or distortion of fused information due to differences in format, parameters, and quality.

[0010] The cross-modal feature alignment module breaks down the fusion barriers caused by the different physical meanings and spatial misalignments of multimodal images, providing a prerequisite for subsequent accurate fusion;

[0011] The intelligent multimodal fusion module fully leverages the complementary value of multimodal images, eliminates information redundancy or missing key features, and achieves more accurate multimodal information integration than traditional fusion algorithms.

[0012] The lesion feature extraction and quantification module provides quantitative feature support for model interpretability, avoiding deep learning models that only output results without specific feature basis;

[0013] The interpretable decision modeling module breaks down the barriers of deep learning models, allowing doctors to verify and trace the rationality of model decisions, thereby improving clinical adoption rates, especially suitable for comprehensive judgment of difficult cases;

[0014] The medical record modeling module transforms the abstract information after multimodal fusion into an intuitive visual model, solving the problem of complex multimodal information and the difficulty for doctors to quickly integrate it, and assisting in accurate clinical decision-making.

[0015] The clinical interactive verification module addresses the problem that model results are difficult to replace human judgment. Through the interactive participation of doctors, the modeling results are made more consistent with clinical practice, thereby improving the accuracy of diagnosis of difficult cases.

[0016] The output and archiving module prevents the modeling results from becoming disconnected from the existing medical system, thereby improving the system's usability and the value of data reuse.

[0017] Preferably, the multimodal data access and standardization module includes a multi-format data compatibility unit, a medical system interface adaptation unit, an image preprocessing unit, and a privacy desensitization unit;

[0018] The multi-format data compatibility unit supports the parsing of mainstream medical image formats such as DICOM, NIfTI, and PNG, and automatically identifies CT, MRI, PET, and pathological slide image types, solving the problem of heterogeneous multimodal image formats and enabling one-stop data access.

[0019] The medical system interface adapter unit has built-in standardized interfaces for HIS, PACS, and EMR systems, enabling automatic import of images, electronic medical records, and pathology reports without manual input, thus opening up the clinical data flow channel.

[0020] The image preprocessing unit includes a parameter standardization subunit and an artifact correction subunit. The parameter standardization subunit unifies the image resolution, layer thickness, and pixel value range to eliminate differences in scanning parameters between different devices. The artifact correction subunit uses adaptive median filtering and metal artifact correction algorithms to remove motion artifacts and interference from metal implants, thereby improving image quality.

[0021] The privacy desensitization unit uses a reversible privacy shielding algorithm to automatically identify and shield patient name, medical record number, and ID card number privacy information, while retaining the image anatomical structure and lesion characteristics.

[0022] Preferably, the cross-modal feature alignment module includes a spatial registration unit, a feature dimension mapping unit, and a temporal synchronization unit;

[0023] The spatial registration unit, based on an automatic anatomical landmark recognition algorithm and combined with B-spline elastic registration technology, adjusts the local pixel positions of images from different modalities to ensure that the spatial coordinate error of the lesion is ≤1mm. The feature dimension mapping unit adopts a depth domain adaptive mapping algorithm to uniformly map the heterogeneous features of CT density values, MRI T2-weighted signals, and PET SUV metabolic values ​​to the same high-dimensional feature space, eliminating the fusion barrier caused by differences in data dimensions. The temporal synchronization unit automatically extracts the image acquisition timestamps for images at multiple time points before and after treatment, and synchronizes the acquisition time sequence of different modalities through a temporal calibration algorithm to ensure that the trend of lesion changes can be accurately compared in multiple modalities.

[0024] Preferably, the intelligent multimodal fusion module includes a disease-specific weight allocation unit, a redundant information filtering unit, and a two-stage fusion unit;

[0025] The disease-specific weight allocation unit, based on a deep learning sub-model with an attention mechanism, automatically learns the multimodal value weights of different diseases, avoiding the dilution of key information caused by average weighting. The redundant information filtering unit, through the mutual information entropy calculation sub-module, quantifies the information overlap between multimodal images, automatically removes duplicate information, and retains only complementary information, solving the problem of information redundancy in fusion. The two-stage fusion unit fuses the underlying lesion features, retains fine-grained information, and optimizes the fusion results by combining clinical diagnostic rules, avoiding information loss caused by a single fusion method and improving fusion accuracy.

[0026] Preferably, the lesion feature extraction and quantification module includes a full-dimensional feature extraction unit, a feature quantification and annotation unit, and a key feature screening unit;

[0027] The full-dimensional feature extraction unit is based on an improved U-Net+Transformer hybrid model, which automatically extracts the anatomical features, functional features, and texture features of lesions, covering the core dimensions of clinical diagnosis; the feature quantification and annotation unit transforms abstract features into quantifiable indicators and generates a standardized feature data table; the key feature screening unit has a built-in authoritative medical guideline rule base, which automatically filters feature combinations that are strongly related to diagnosis through a feature importance ranking algorithm.

[0028] Preferably, the interpretable decision modeling module includes a decision reasoning chain generation unit, a feature contribution visualization unit, and a clinical standard mapping unit;

[0029] The decision reasoning chain generation unit adopts a hybrid model of clinical rules and deep learning. It first generates preliminary reasoning logic based on the guideline rule base, and then optimizes the feature weights through deep learning to output a structured reasoning chain. The feature contribution visualization unit uses heatmaps, radar charts, and bar charts to intuitively show the role of each feature in the diagnostic results, helping doctors quickly understand the model's decision focus. The clinical standard mapping unit maps the features extracted by the model to authoritative medical guidelines, marking the diagnostic basis corresponding to the features.

[0030] Preferably, the medical record modeling module includes a three-dimensional fusion modeling unit, a four-dimensional dynamic modeling unit, and a quantitative analysis unit;

[0031] The three-dimensional fusion modeling unit, based on the feature data after multimodal fusion, uses a volume rendering algorithm to automatically reconstruct a three-dimensional model of the lesion and surrounding tissues. It supports rotation, sectioning, and scaling operations, intuitively displaying the spatial relationship between the lesion and blood vessels, nerves, and organs, and assisting in surgical path planning. The four-dimensional dynamic modeling unit integrates multimodal images from different time points to generate a 3D, time-dynamic model, which displays the trend of lesion changes through animation, and assists in efficacy evaluation. The quantitative analysis unit automatically calculates quantitative indicators such as lesion volume, surface area, shortest distance to surrounding important structures, density, and average metabolic value, and generates an analysis report, providing accurate data support for radiotherapy target delineation and prognostic assessment.

[0032] Preferably, the clinical interactive verification module includes a custom adjustment unit, a similar case matching unit, and an operation log recording unit;

[0033] The custom adjustment unit provides a visual operation interface, allowing doctors to manually correct lesion boundaries, adjust feature weights, and supplement clinical information, with the model updating decision results in real time. The similar case matching unit, based on a multimodal feature similarity algorithm, retrieves modeling results, clinical diagnostic conclusions, and treatment plans for historical similar cases, providing references for difficult cases and assisting doctors in validating model decisions. The operation log recording unit automatically records the model decision-making process, doctor adjustment traces, final diagnostic conclusions, and modeling time information, forming a traceable operation log.

[0034] Preferably, the result output and archiving module includes a multi-format result generation unit, a clinical system integration unit, and an intelligent archive retrieval unit;

[0035] The multi-format result generation unit supports output in multiple formats, including 3D models, diagnostic reports, and visualization charts, to meet clinical and research needs. The clinical system interface unit has a built-in EMR system data writing interface, which automatically synchronizes the modeling results to the electronic medical record without secondary processing, adapts to the doctor's existing workflow, and avoids the modeling results from becoming disconnected from the clinical system. The intelligent archiving and retrieval unit classifies and archives medical record data according to disease type, patient ID, modeling time, and lesion type, and supports multi-condition combination retrieval.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] 1) This application adopts a three-level collaborative system of feature quantification, interpretable decision-making, and clinical interactive verification to make model decisions traceable, verifiable, and adjustable, thereby significantly improving the clinical adoption rate. It is especially suitable for the diagnosis of difficult cases. The lesion feature extraction and quantification module and the interpretable decision-making modeling module solve the problem of model decisions lacking specific feature basis, providing doctors with quantitative references to verify the rationality of decisions. Through the interpretable decision-making modeling module and the case modeling module, doctors can clearly trace the basis of model decisions, verify the rationality of results, and increase their trust in the model. Through the interactive participation of doctors, the modeling results are more in line with clinical practice. Especially in difficult cases, the model becomes an auxiliary tool for doctors rather than an independent decision-maker, which greatly improves the clinical adoption rate.

[0038] 2) This application breaks down the barriers to multimodal fusion through a three-level collaborative system of data standardization, feature alignment, and intelligent fusion, fully explores complementary information, and avoids redundancy or missing information. The multimodal data access and standardization module eliminates the problems of heterogeneous multimodal data formats, inconsistent parameters, and varying quality, providing a clean and unified data foundation for subsequent fusion. The cross-modal feature alignment module breaks down the fusion barriers of spatial misalignment, dimensional heterogeneity, and temporal asynchrony of multimodal images, providing consistent alignment for accurate fusion. The feature data is used to explore the complementary value of multimodal images through the intelligent multimodal fusion module, preventing information redundancy from obscuring key features or insufficient fusion from causing feature loss, and achieving more accurate information integration than traditional fusion algorithms. Attached Figure Description

[0039] Figure 1 A diagram showing the module composition of a medical record image modeling system;

[0040] Figure 2 A diagram showing the unit composition of the multimodal data access and standardization module;

[0041] Figure 3 This is a diagram showing the unit composition of the cross-modal feature alignment module.

[0042] Figure 4 This is a diagram showing the unit composition of the intelligent multimodal fusion module.

[0043] Figure 5 This is a diagram showing the unit composition of the lesion feature extraction and quantification module;

[0044] Figure 6 A diagram showing the unit composition of the interpretable decision modeling module;

[0045] Figure 7 A diagram showing the unit composition of the medical record modeling module;

[0046] Figure 8 This is a diagram showing the unit composition of the clinical interactive verification module.

[0047] Figure 9 This is a diagram showing the unit composition of the output and archiving modules. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example:

[0050] Please see Figure 1-9 The present invention provides a technical solution:

[0051] A clinically validated multimodal precision medical record image modeling system includes a multimodal data access and standardization module, a cross-modal feature alignment module, an intelligent multimodal fusion module, a lesion feature extraction and quantification module, an interpretable decision modeling module, a medical record modeling module, a clinical interaction validation module, and a result output and archiving module.

[0052] The multimodal data access and standardization module provides a unified data foundation for multimodal fusion, avoiding misalignment or distortion of fused information due to differences in format, parameters, and quality.

[0053] The cross-modal feature alignment module breaks down the fusion barriers caused by the different physical meanings and spatial misalignments of multimodal images, providing a prerequisite for subsequent accurate fusion;

[0054] The intelligent multimodal fusion module fully leverages the complementary value of multimodal images, eliminates information redundancy or missing key features, and achieves more accurate multimodal information integration than traditional fusion algorithms.

[0055] The lesion feature extraction and quantification module provides quantitative feature support for model interpretability, avoiding deep learning models that only output results without specific feature basis;

[0056] The interpretable decision modeling module breaks down the barriers of deep learning models, allowing doctors to verify and trace the rationality of model decisions, thereby improving clinical adoption rates, especially suitable for comprehensive judgment of difficult cases;

[0057] The medical record modeling module transforms the abstract information after multimodal fusion into an intuitive visual model, solving the problem of complex multimodal information and the difficulty for doctors to quickly integrate it, and assisting in accurate clinical decision-making.

[0058] The clinical interactive verification module addresses the problem that model results are difficult to replace human judgment. Through the interactive participation of doctors, the modeling results are made more consistent with clinical practice, thereby improving the accuracy of diagnosis of difficult cases.

[0059] The output and archiving module prevents the modeling results from becoming disconnected from the existing medical system, thereby improving the system's usability and the value of data reuse.

[0060] The multimodal data access and standardization module includes a multi-format data compatibility unit, a medical system interface adaptation unit, an image preprocessing unit, and a privacy desensitization unit;

[0061] The multi-format data compatibility unit supports the parsing of mainstream medical image formats such as DICOM, NIfTI, and PNG, and automatically identifies CT, MRI, PET, and pathological slide image types, solving the problem of heterogeneous multimodal image formats and enabling one-stop data access.

[0062] The medical system interface adapter unit has built-in standardized interfaces for HIS, PACS, and EMR systems, enabling automatic import of images, electronic medical records, and pathology reports without manual input, thus opening up the clinical data flow channel.

[0063] The image preprocessing unit includes a parameter standardization subunit and an artifact correction subunit. The parameter standardization subunit unifies the image resolution, layer thickness, and pixel value range to eliminate differences in scanning parameters between different devices. The artifact correction subunit uses adaptive median filtering and metal artifact correction algorithms to remove motion artifacts and interference from metal implants, thereby improving image quality.

[0064] The privacy desensitization unit uses a reversible privacy shielding algorithm to automatically identify and shield patient name, medical record number, and ID card number privacy information, while retaining the image anatomical structure and lesion characteristics.

[0065] The multimodal data access and standardization module is the data entry point for the entire system. First, it enables one-stop access and association of multi-format images such as DICOM and NIfTI, as well as HIS, PACS, and EMR clinical data, through the multi-format data compatibility unit and medical system interface adaptation unit. Then, it unifies parameters such as resolution and slice thickness through the image preprocessing unit and corrects artifacts. Finally, it shields privacy information through the privacy desensitization unit and outputs high-quality, privacy-free multimodal raw data with unified format and parameters.

[0066] The cross-modal feature alignment module includes a spatial registration unit, a feature dimension mapping unit, and a temporal synchronization unit;

[0067] The spatial registration unit, based on an automatic anatomical landmark recognition algorithm and combined with B-spline elastic registration technology, adjusts the local pixel positions of images from different modalities to ensure that the spatial coordinate error of the lesion is ≤1mm. The feature dimension mapping unit adopts a depth domain adaptive mapping algorithm to uniformly map the heterogeneous features of CT density values, MRI T2-weighted signals, and PET SUV metabolic values ​​to the same high-dimensional feature space, eliminating the fusion barrier caused by differences in data dimensions. The temporal synchronization unit automatically extracts the image acquisition timestamps for images at multiple time points before and after treatment, and synchronizes the acquisition time sequence of different modalities through a temporal calibration algorithm to ensure that the trend of lesion changes can be accurately compared in multiple modalities.

[0068] The cross-modal feature alignment module directly receives the standardized data, corrects the spatial misalignment of different modal images through the spatial registration unit, maps heterogeneous features such as CT density values, MRI signals, and PET metabolic values ​​to the same high-dimensional space through the feature dimension mapping unit, and calibrates the acquisition time sequence before and after treatment through the temporal synchronization unit, outputting multimodal feature data that is spatially aligned, dimensionally unified, and temporally synchronized.

[0069] The intelligent multimodal fusion module includes a disease-specific weight allocation unit, a redundant information filtering unit, and a two-stage fusion unit;

[0070] The disease-specific weight allocation unit, based on a deep learning sub-model with an attention mechanism, automatically learns the multimodal value weights of different diseases, avoiding the dilution of key information caused by average weighting. The redundant information filtering unit, through the mutual information entropy calculation sub-module, quantifies the information overlap between multimodal images, automatically removes duplicate information, and retains only complementary information, solving the problem of information redundancy in fusion. The two-stage fusion unit fuses the underlying lesion features, retains fine-grained information, and optimizes the fusion results by combining clinical diagnostic rules, avoiding information loss caused by a single fusion method and improving fusion accuracy.

[0071] The intelligent multimodal fusion module receives aligned feature data, automatically learns the modal value weights of different diseases through the disease-specific weight allocation unit, removes duplicate information and retains complementary information through the redundancy filtering unit (mutual information entropy calculation), and then achieves accurate integration of multimodal information through the two-stage fusion unit (low-level feature fusion and clinical rule optimization), outputting fusion feature data that is free of redundancy, highly complementary, and of high value.

[0072] The lesion feature extraction and quantification module includes a full-dimensional feature extraction unit, a feature quantification and annotation unit, and a key feature screening unit;

[0073] The full-dimensional feature extraction unit is based on an improved U-Net+Transformer hybrid model, which automatically extracts the anatomical features, functional features, and texture features of lesions, covering the core dimensions of clinical diagnosis; the feature quantification and annotation unit transforms abstract features into quantifiable indicators and generates a standardized feature data table; the key feature screening unit has a built-in authoritative medical guideline rule base, which automatically filters feature combinations that are strongly related to diagnosis through a feature importance ranking algorithm.

[0074] Based on this fusion feature, the lesion feature extraction and quantification module extracts the anatomical, functional, and textural features of the lesion through an improved U-Net+Transformer hybrid model. Then, the feature quantification annotation unit transforms the abstract features into standardized quantitative indicators. Finally, the key feature screening unit selects feature combinations that are strongly related to the diagnosis.

[0075] The interpretable decision modeling module includes a decision reasoning chain generation unit, a feature contribution visualization unit, and a clinical standard mapping unit;

[0076] The decision reasoning chain generation unit adopts a hybrid model of clinical rules and deep learning. It first generates preliminary reasoning logic based on the guideline rule base, and then optimizes the feature weights through deep learning to output a structured reasoning chain. The feature contribution visualization unit uses heatmaps, radar charts, and bar charts to intuitively show the role of each feature in the diagnostic results, helping doctors quickly understand the model's decision focus. The clinical standard mapping unit maps the features extracted by the model to authoritative medical guidelines, marking the diagnostic basis corresponding to the features.

[0077] The interpretable decision modeling module receives the quantified core features, generates a structured decision reasoning chain through a hybrid model of clinical rules and deep learning, displays the role of features in diagnosis through a feature contribution visualization unit, and maps features to authoritative guidelines through a clinical standard mapping unit, outputting traceable, verifiable, and clinically relevant decision-making basis.

[0078] The medical record modeling module includes a three-dimensional fusion modeling unit, a four-dimensional dynamic modeling unit, and a quantitative analysis unit;

[0079] The three-dimensional fusion modeling unit, based on the feature data after multimodal fusion, uses a volume rendering algorithm to automatically reconstruct a three-dimensional model of the lesion and surrounding tissues. It supports rotation, sectioning, and scaling operations, intuitively displaying the spatial relationship between the lesion and blood vessels, nerves, and organs, and assisting in surgical path planning. The four-dimensional dynamic modeling unit integrates multimodal images from different time points to generate a 3D, time-dynamic model, which displays the trend of lesion changes through animation, and assists in efficacy evaluation. The quantitative analysis unit automatically calculates quantitative indicators such as lesion volume, surface area, shortest distance to surrounding important structures, density, and average metabolic value, and generates an analysis report, providing accurate data support for radiotherapy target delineation and prognostic assessment.

[0080] The medical record modeling module simultaneously receives fusion features and quantitative indicators. It reconstructs a 3D model of the lesion and surrounding tissues through a 3D fusion modeling unit (volume rendering algorithm), generates a dynamic model of lesion changes through a 4D dynamic modeling unit, and calculates precise data such as lesion volume and distance from important structures through a quantitative analysis unit, outputting an intuitive, visual, and precisely quantified medical record model.

[0081] The clinical interactive verification module includes a custom adjustment unit, a similar case matching unit, and an operation log recording unit;

[0082] The custom adjustment unit provides a visual operation interface, allowing doctors to manually correct lesion boundaries, adjust feature weights, and supplement clinical information, with the model updating decision results in real time. The similar case matching unit, based on a multimodal feature similarity algorithm, retrieves modeling results, clinical diagnostic conclusions, and treatment plans for historical similar cases, providing references for difficult cases and assisting doctors in validating model decisions. The operation log recording unit automatically records the model decision-making process, doctor adjustment traces, final diagnostic conclusions, and modeling time information, forming a traceable operation log.

[0083] The clinical interactive validation module receives the visualized model and interpretable decision results from the medical record modeling. It provides a visual interface through a customizable adjustment unit, allowing doctors to manually correct lesion boundaries, adjust feature weights, and supplement clinical information. The model updates the decision results in real time. The similar case matching unit retrieves similar archived medical records for doctors' reference. The operation log recording unit records information throughout the entire process, including model decisions, doctor adjustments, and final conclusions. The adjusted feature data and corrected modeling results are fed back to the intelligent multimodal fusion module, lesion feature extraction and quantification module, and interpretable decision modeling module, forming a closed loop of model output, doctor validation, data feedback, and algorithm optimization.

[0084] The result output and archiving module includes a multi-format result generation unit, a clinical system integration unit, and an intelligent archive retrieval unit;

[0085] The multi-format result generation unit supports output in multiple formats, including 3D models, diagnostic reports, and visualization charts, to meet clinical and research needs. The clinical system interface unit has a built-in EMR system data writing interface, which automatically synchronizes the modeling results to the electronic medical record without secondary processing, adapts to the doctor's existing workflow, and avoids the modeling results from becoming disconnected from the clinical system. The intelligent archiving and retrieval unit classifies and archives medical record data according to disease type, patient ID, modeling time, and lesion type, and supports multi-condition combination retrieval.

[0086] The results output and archiving module receives the final modeling results after clinical validation. It adapts to clinical and research needs through a multi-format results generation unit and automatically synchronizes the results to the EMR electronic medical record through a clinical system integration unit, eliminating the need for secondary data entry. The intelligent archiving and retrieval unit categorizes and archives data by disease type, patient ID, and other dimensions. The archived data provides a historical case library for the similar case matching unit of the clinical interactive validation module and high-quality labeled data for subsequent algorithm iterations.

[0087] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0088] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A clinically validated multimodal precision medical record image modeling system, characterized in that, It includes modules for multimodal data access and standardization, cross-modal feature alignment, intelligent multimodal fusion, lesion feature extraction and quantification, interpretable decision modeling, medical record modeling, clinical interaction verification, and result output and archiving. The multimodal data access and standardization module provides a unified data foundation for multimodal fusion, avoiding misalignment or distortion of fused information due to differences in format, parameters, and quality. The cross-modal feature alignment module breaks down the fusion barriers caused by the different physical meanings and spatial misalignments of multimodal images, providing a prerequisite for subsequent accurate fusion; The intelligent multimodal fusion module fully leverages the complementary value of multimodal images, eliminates information redundancy or missing key features, and achieves more accurate multimodal information integration than traditional fusion algorithms. The lesion feature extraction and quantification module provides quantitative feature support for model interpretability, avoiding deep learning models that only output results without specific feature basis; The interpretable decision modeling module breaks down the barriers of deep learning models, allowing doctors to verify and trace the rationality of model decisions, thereby improving clinical adoption rates, especially suitable for comprehensive judgment of difficult cases; The medical record modeling module transforms the abstract information after multimodal fusion into an intuitive visual model, solving the problem of complex multimodal information and the difficulty for doctors to quickly integrate it, and assisting in accurate clinical decision-making. The clinical interactive verification module addresses the problem that model results are difficult to replace human judgment. Through the interactive participation of doctors, the modeling results are made more consistent with clinical practice, thereby improving the accuracy of diagnosis of difficult cases. The output and archiving module prevents the modeling results from becoming disconnected from the existing medical system, thereby improving the system's usability and the value of data reuse.

2. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The multimodal data access and standardization module includes a multi-format data compatibility unit, a medical system interface adaptation unit, an image preprocessing unit, and a privacy desensitization unit; The multi-format data compatibility unit supports the parsing of mainstream medical image formats such as DICOM, NIfTI, and PNG, and automatically identifies CT, MRI, PET, and pathological slide image types, solving the problem of heterogeneous multimodal image formats and enabling one-stop data access. The medical system interface adapter unit has built-in standardized interfaces for HIS, PACS, and EMR systems, enabling automatic import of images, electronic medical records, and pathology reports without manual input, thus opening up the clinical data flow channel. The image preprocessing unit includes a parameter standardization subunit and an artifact correction subunit. The parameter standardization subunit unifies the image resolution, layer thickness, and pixel value range to eliminate differences in scanning parameters between different devices. The artifact correction subunit uses adaptive median filtering and metal artifact correction algorithms to remove motion artifacts and interference from metal implants, thereby improving image quality. The privacy desensitization unit uses a reversible privacy shielding algorithm to automatically identify and shield patient name, medical record number, and ID card number privacy information, while retaining the image anatomical structure and lesion characteristics.

3. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The cross-modal feature alignment module includes a spatial registration unit, a feature dimension mapping unit, and a temporal synchronization unit; The spatial registration unit, based on an automatic anatomical landmark recognition algorithm and combined with B-spline elastic registration technology, adjusts the local pixel positions of images from different modalities to ensure that the spatial coordinate error of the lesion is ≤1mm. The feature dimension mapping unit adopts a depth domain adaptive mapping algorithm to uniformly map the heterogeneous features of CT density values, MRI T2-weighted signals, and PET SUV metabolic values ​​to the same high-dimensional feature space, eliminating the fusion barrier caused by differences in data dimensions. The temporal synchronization unit automatically extracts the image acquisition timestamps for images at multiple time points before and after treatment, and synchronizes the acquisition time sequence of different modalities through a temporal calibration algorithm to ensure that the trend of lesion changes can be accurately compared in multiple modalities.

4. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The intelligent multimodal fusion module includes a disease-specific weight allocation unit, a redundant information filtering unit, and a two-stage fusion unit; The disease-specific weight allocation unit is based on a deep learning sub-model with an attention mechanism, which automatically learns the multimodal value weights of different diseases to avoid the dilution of key information caused by average weighting; the redundant information filtering unit uses a mutual information entropy calculation sub-module to quantify the information overlap between multimodal images, automatically remove duplicate information, and retain only complementary information to solve the problem of information redundancy in fusion. The dual-stage fusion unit integrates underlying lesion features, retains fine-grained information, and optimizes the fusion results by combining clinical diagnostic rules, avoiding information loss caused by a single fusion method and improving fusion accuracy.

5. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The lesion feature extraction and quantification module includes a full-dimensional feature extraction unit, a feature quantification and annotation unit, and a key feature screening unit; The full-dimensional feature extraction unit is based on an improved U-Net+Transformer hybrid model, which automatically extracts the anatomical features, functional features, and texture features of lesions, covering the core dimensions of clinical diagnosis; the feature quantification and annotation unit transforms abstract features into quantifiable indicators and generates a standardized feature data table; the key feature screening unit has a built-in authoritative medical guideline rule base, which automatically filters feature combinations that are strongly related to diagnosis through a feature importance ranking algorithm.

6. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The interpretable decision modeling module includes a decision reasoning chain generation unit, a feature contribution visualization unit, and a clinical standard mapping unit; The decision reasoning chain generation unit adopts a hybrid model of clinical rules and deep learning. It first generates preliminary reasoning logic based on the guideline rule base, and then optimizes the feature weights through deep learning to output a structured reasoning chain. The feature contribution visualization unit uses heatmaps, radar charts, and bar charts to intuitively show the role of each feature in the diagnostic results, helping doctors quickly understand the model's decision focus. The clinical standard mapping unit maps the features extracted by the model to authoritative medical guidelines, marking the diagnostic basis corresponding to the features.

7. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The medical record modeling module includes a three-dimensional fusion modeling unit, a four-dimensional dynamic modeling unit, and a quantitative analysis unit; The three-dimensional fusion modeling unit, based on the feature data after multimodal fusion, uses a volume rendering algorithm to automatically reconstruct a three-dimensional model of the lesion and surrounding tissues. It supports rotation, sectioning, and scaling operations, intuitively displaying the spatial relationship between the lesion and blood vessels, nerves, and organs, and assisting in surgical path planning. The four-dimensional dynamic modeling unit integrates multimodal images from different time points to generate a 3D, time-dynamic model, which displays the trend of lesion changes through animation, and assists in efficacy evaluation. The quantitative analysis unit automatically calculates quantitative indicators such as lesion volume, surface area, shortest distance to surrounding important structures, density, and average metabolic value, and generates an analysis report, providing accurate data support for radiotherapy target delineation and prognostic assessment.

8. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The clinical interactive verification module includes a custom adjustment unit, a similar case matching unit, and an operation log recording unit; The custom adjustment unit provides a visual operation interface, allowing doctors to manually correct lesion boundaries, adjust feature weights, and supplement clinical information, with the model updating decision results in real time. The similar case matching unit, based on a multimodal feature similarity algorithm, retrieves modeling results, clinical diagnostic conclusions, and treatment plans for historical similar cases, providing references for difficult cases and assisting doctors in validating model decisions. The operation log recording unit automatically records the model decision-making process, doctor adjustment traces, final diagnostic conclusions, and modeling time information, forming a traceable operation log.

9. The multimodal precision medical record image modeling system with clinical validation according to claim 1, characterized in that: The result output and archiving module includes a multi-format result generation unit, a clinical system integration unit, and an intelligent archive retrieval unit; The multi-format result generation unit supports multi-format output of 3D models, diagnostic reports, and visualization charts to meet clinical and research needs. The clinical system interface unit has a built-in EMR system data writing interface, which automatically synchronizes the modeling results to the electronic medical record without secondary processing. It adapts to the doctor's existing workflow and avoids the modeling results from becoming disconnected from the clinical system. The intelligent archiving and retrieval unit classifies and archives medical record data according to disease type, patient ID, modeling time, and lesion type, and supports multi-condition combination retrieval.

Images