Lightweight multi-modal medical imaging intelligent diagnosis system based on mercuration platform
The lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform solves the problems of single information utilization, insufficient ability to identify subtle lesions, and high deployment costs. It realizes deep collaborative reasoning and efficient deployment of multimodal data, improves the comprehensiveness and accuracy of diagnosis, reduces the rate of missed diagnoses, and increases deployment flexibility.
Patent Information
- Application Number
- CN202511804438.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-03
AI Technical Summary
Existing medical AI diagnostic systems suffer from problems such as limited information utilization, insufficient ability to identify subtle lesions, reliance on computing power and high deployment costs, as well as poor system integration and generalization. They are unable to achieve deep multimodal fusion, accurate identification of subtle lesions, and efficient deployment in cloud-edge environments.
A lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform is adopted, including multimodal data preprocessing, cross-modal semantic fusion, cloud-edge collaborative reasoning and multi-scale lesion identification modules. Through cross-modal attention mechanism and feature pyramid network, deep collaborative reasoning of multimodal data and accurate identification of subtle lesions are achieved.
It enables deep collaborative reasoning of multimodal data, improves the comprehensiveness and accuracy of diagnosis, reduces the rate of missed diagnoses, and enhances deployment flexibility and diagnostic efficiency, demonstrating good clinical applicability and promotional value.
Smart Images
Figure CN121601169A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical artificial intelligence and high-performance computing technology, and in particular to an intelligent diagnostic system and method based on the Ascend computing platform, integrating multimodal medical data, and adopting a cloud-edge collaborative architecture. Background Technology
[0002] Medical imaging (such as CT, MRI, and X-rays) is a core basis for modern disease diagnosis. With the explosive growth of medical image data, radiologists face immense pressure in interpreting images, increasing the risk of misdiagnosis and missed diagnosis due to fatigue. Artificial intelligence technology, especially deep learning, has shown great potential in medical image analysis.
[0003] However, existing medical AI diagnostic systems still have significant bottlenecks:
[0004] Limited information utilization: Most systems only analyze single-modality imaging data (e.g., CT scans only), lacking comprehensive utilization of multimodal patient information (e.g., MRI, pathological sections, laboratory test results). A "semantic gap" exists between different modalities of data, making simple post-processing fusion insufficient for deep collaborative reasoning, thus limiting the comprehensiveness and accuracy of diagnosis.
[0005] Insufficient ability to identify subtle lesions: For early, small lesions (such as lung nodules smaller than 2mm), the feature extraction network of existing models has limited ability to capture multi-scale features, poor receptive field design, and is prone to ignoring key details, leading to missed detection.
[0006] High computational power dependence and high deployment costs: High-precision model calculations are complex and rely on high-end GPU computing power in the cloud. It is difficult to deploy them with low latency and low cost at the edge of grassroots hospitals with limited resources, which restricts the widespread application of high-quality medical AI resources.
[0007] Poor system integration and generalization: The integration with the hospital's existing PACS system is complex, and the model has poor adaptability to image data generated by different manufacturers and models of equipment, which affects the stability and accuracy of clinical applications.
[0008] Therefore, there is an urgent need in this field for an intelligent diagnostic solution that can achieve deep multimodal fusion, accurately identify subtle lesions, and be efficiently deployed in cloud-edge environments. Summary of the Invention
[0009] The purpose of this invention is to provide a lightweight, multimodal intelligent diagnostic method for medical imaging based on the Ascend platform, in order to solve the problems mentioned in the background art.
[0010] To achieve the above objectives, the present invention provides the following technical solution: a lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform, comprising:
[0011] The multimodal data preprocessing module is used to interface with medical information systems, receive and standardize multimodal medical data;
[0012] A cross-modal semantic fusion module, connected to the preprocessing module, is used to map standardized multimodal data to a unified semantic space and perform feature-level fusion through a cross-modal attention mechanism to generate deep semantic representations.
[0013] The cloud-edge collaborative reasoning module is connected to the semantic fusion module and includes a high-precision diagnostic model in the cloud and a lightweight diagnostic model at the edge. The cloud model is generated based on the Ascend AI processor, and the edge model is obtained by knowledge distillation and quantization of the cloud model. The module is configured to dynamically allocate reasoning tasks between the cloud and the edge according to the complexity of the diagnostic task.
[0014] A multi-scale lesion identification module is connected to the cloud-edge collaborative inference module. The module includes a feature pyramid network, dilated convolutional groups, and an attention mechanism to extract and enhance multi-scale features from the deep semantic representation in order to identify minute lesions and accurately segment lesion boundaries.
[0015] Preferably, the multimodal data preprocessing module specifically employs a domain adaptive method based on generative adversarial networks to standardize the image data, and uses an embedding layer to transform unstructured image data into feature vectors.
[0016] Preferably, the core of the cross-modal semantic fusion module is a cross-modal transformer, which includes a modality-specific encoder and a shared cross-attention encoder.
[0017] Preferably, in the cloud-edge collaborative reasoning module, the dynamic allocation of reasoning tasks specifically involves: for routine screening tasks, real-time reasoning is performed by the edge model; for complex case analysis tasks, data is uploaded from the edge to the cloud, and deep reasoning is performed by the cloud model.
[0018] Preferably, in the multi-scale lesion identification module, the feature pyramid network is a bidirectional feature pyramid network (BiFPN).
[0019] Preferably, the dilated convolution group contains multiple dilated convolutions with different dilation rates to process feature maps in parallel.
[0020] Preferably, the attention mechanism is a convolutional block attention module (CBAM).
[0021] Preferably, the multi-scale lesion identification module uses a joint loss function composed of DiceLoss and FocalLoss during training.
[0022] A lightweight, multimodal intelligent diagnostic method for medical imaging based on the Ascend platform includes the following steps:
[0023] Step S1: Receive and standardize multimodal medical data through the preprocessing module;
[0024] Step S2: Map the standardized multimodal data to a unified semantic space and perform feature-level fusion through the cross-modal semantic fusion module to generate a deep semantic representation;
[0025] Step S3: Using the cloud-edge collaborative reasoning module, based on the complexity of the diagnostic task, select to call the diagnostic model in the cloud or at the edge to reason about the deep semantic representation;
[0026] Step S4: Through the multi-scale lesion identification module, the features in the reasoning process are enhanced at multiple scales, and a diagnostic report containing lesion location, classification and segmentation results is output.
[0027] A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of a lightweight, multimodal, intelligent diagnostic method for medical imaging.
[0028] Compared with the prior art, the beneficial effects of the present invention are:
[0029] Comprehensive breakthrough in diagnosis: Through cross-modal unified semantic mapping technology, deep collaborative reasoning of multi-source heterogeneous data is realized, and the comprehensive diagnostic index MMMU-Med reaches 82.3 points, far exceeding the single-modal model.
[0030] Significantly improved recognition accuracy: Through multi-scale feature enhancement network, the sensitivity to small lesions is effectively improved, the accuracy of lung nodule detection reaches 97.3%, and the smallest nodule can be detected as small as 2mm. The missed diagnosis rate is reduced by 32% in clinical application.
[0031] Deployment flexibility and efficiency optimization: Based on the cloud-edge collaborative architecture of the Ascend platform, high-performance AI models can be deployed in primary medical institutions at low cost and low latency. The processing time for a single CT image is less than 0.3 seconds, which greatly improves diagnostic efficiency and increases doctors' work efficiency by 40%.
[0032] The system is highly practical: it has been seamlessly integrated with the PACS systems of multiple hospitals, and has assisted in the diagnosis of more than 10,000 cases, demonstrating good clinical applicability and promotional value. Attached Figure Description
[0033] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0034] Figure 1 This is a block diagram of the cloud-edge collaborative intelligent medical imaging diagnostic system based on the Ascend platform of the present invention;
[0035] Figure 2 This is a flowchart of the cloud-edge collaborative intelligent medical image diagnosis method based on the Ascend platform of the present invention. Detailed Implementation
[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0037] Please see Figures 1 to 2 The present invention adopts the following technical solution:
[0038] In a first aspect, the present invention provides a cloud-edge collaborative intelligent medical imaging diagnostic system based on the Ascend platform, comprising:
[0039] The multimodal data preprocessing module is used to interface with medical information systems, receive and standardize multimodal medical data;
[0040] A cross-modal semantic fusion module, connected to the preprocessing module, is used to map standardized multimodal data to a unified semantic space and perform feature-level fusion through a cross-modal attention mechanism to generate deep semantic representations.
[0041] The cloud-edge collaborative reasoning module is connected to the semantic fusion module and includes a high-precision diagnostic model in the cloud and a lightweight diagnostic model at the edge. The cloud model is generated based on the Ascend AI processor, and the edge model is obtained by knowledge distillation and quantization of the cloud model. The module is configured to dynamically allocate reasoning tasks between the cloud and the edge according to the complexity of the diagnostic task.
[0042] A multi-scale lesion identification module is connected to the cloud-edge collaborative inference module. The module includes a feature pyramid network, dilated convolutional groups, and an attention mechanism to extract and enhance multi-scale features from the deep semantic representation in order to identify minute lesions and accurately segment lesion boundaries.
[0043] Specifically, the multimodal data preprocessing module employs a domain adaptive method based on generative adversarial networks to standardize the image data and uses an embedding layer to transform unstructured image data into feature vectors.
[0044] Specifically, the core of the cross-modal semantic fusion module is a cross-modal transformer, which includes a modality-specific encoder and a shared cross-attention encoder. Deep fusion is achieved by calculating the correlation weights between features of different modalities.
[0045] Specifically, in the cloud-edge collaborative reasoning module, the dynamic allocation of reasoning tasks is as follows: for routine screening tasks, the edge model performs real-time reasoning; for complex case analysis tasks, the edge uploads data to the cloud, and the cloud model performs deep reasoning.
[0046] Specifically, in the multi-scale lesion identification module, the feature pyramid network is a bidirectional feature pyramid network used for bidirectional cross-scale feature fusion; the dilated convolution group contains a variety of dilated convolutions with different dilation rates to expand the network's receptive field; and the attention mechanism is a convolution block attention module used to enhance lesion region features along the channel and spatial dimensions.
[0047] Secondly, the present invention provides a medical image intelligent diagnosis method based on the above system, comprising the following steps:
[0048] Step S1: Receive and standardize multimodal medical data through the preprocessing module;
[0049] Step S2: Map the standardized multimodal data to a unified semantic space and perform feature-level fusion through the cross-modal semantic fusion module to generate a deep semantic representation;
[0050] Step S3: Using the cloud-edge collaborative reasoning module, based on the complexity of the diagnostic task, select to call the diagnostic model in the cloud or at the edge to reason about the deep semantic representation;
[0051] Step S4: Through the multi-scale lesion identification module, the features in the reasoning process are enhanced at multiple scales, and a diagnostic report containing lesion location, classification and segmentation results is output.
[0052] Example: Assisted diagnosis of pulmonary nodules based on this system
[0053] Data input (corresponding to step S1): The system automatically obtains a patient's chest CT image (DICOM format) from the hospital's PACS system and obtains the patient's carcinoembryonic antigen (CEA) test value from the Laboratory Information System (LIS).
[0054] Data processing and fusion (corresponding to step S2):
[0055] The preprocessing module uses a GAN model to standardize CT images and converts CEA values into feature vectors through an embedding layer.
[0056] The ViT encoder in the cross-modal fusion module extracts visual features from CT images, while the text encoder processes CEA features. The cross-attention encoder calculates the correlation between image features and test features, generating a fused deep semantic representation vector that simultaneously encodes the joint information of "minor shadows in the lungs" and "slightly elevated tumor markers".
[0057] Collaborative reasoning (corresponding to step S3): The cloud-edge collaborative module determines that "lung nodule screening" is a routine task, and therefore calls the lightweight model deployed on the Ascend 310 chip at the edge for real-time reasoning.
[0058] Lesion identification and output (corresponding to step S4): During inference, the multi-scale lesion identification module is activated. BiFPN extracts features from different network layers, the cavitary convolutional group captures the contextual information of the nodule, and the CBAM module focuses on specific areas of the lung lobe. The model finally outputs a diagnostic report: "A 3mm solid nodule was found in the upper lobe of the right lung. The nodule has a regular shape and a low risk of malignancy. Based on the CEA index, a follow-up examination is recommended in 6 months." This report is generated within 0.3 seconds and automatically written back to the PACS system.
[0059] The above embodiments demonstrate the effectiveness and practicality of this system in processing multimodal information and quickly and accurately identifying minute lesions.
[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A lightweight, multimodal intelligent diagnostic system for medical imaging based on the Ascend platform, characterized in that, include: The multimodal data preprocessing module is used to interface with medical information systems, receive and standardize multimodal medical data; A cross-modal semantic fusion module, connected to the preprocessing module, is used to map standardized multimodal data to a unified semantic space and perform feature-level fusion through a cross-modal attention mechanism to generate deep semantic representations. The cloud-edge collaborative reasoning module is connected to the semantic fusion module and includes a high-precision diagnostic model in the cloud and a lightweight diagnostic model at the edge. The cloud model is generated based on the Ascend AI processor, and the edge model is obtained by knowledge distillation and quantization of the cloud model. The module is configured to dynamically allocate reasoning tasks between the cloud and the edge according to the complexity of the diagnostic task. A multi-scale lesion identification module is connected to the cloud-edge collaborative inference module. The module includes a feature pyramid network, dilated convolutional groups, and an attention mechanism to extract and enhance multi-scale features from the deep semantic representation in order to identify minute lesions and accurately segment lesion boundaries.
2. The lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, The multimodal data preprocessing module specifically employs a domain adaptive method based on generative adversarial networks to standardize image data and uses an embedding layer to transform unstructured image data into feature vectors.
3. The lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, The core of the cross-modal semantic fusion module is a cross-modal transformer, which includes a modality-specific encoder and a shared cross-attention encoder.
4. The lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, In the cloud-edge collaborative reasoning module, the dynamic allocation of reasoning tasks specifically involves: for routine screening tasks, real-time reasoning is performed by the edge model; for complex case analysis tasks, data is uploaded from the edge to the cloud, where the cloud model performs deep reasoning.
5. A lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, In the multi-scale lesion identification module, the feature pyramid network is a bidirectional feature pyramid network (BiFPN).
6. The lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, The dilated convolutional group contains multiple dilated convolutions with different dilation rates, which process feature maps in parallel.
7. A lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, The attention mechanism is a convolutional block attention module (CBAM).
8. A lightweight multimodal medical imaging intelligent diagnostic system based on the Ascend platform according to claim 1, characterized in that, The multi-scale lesion identification module uses a joint loss function consisting of DiceLoss and FocalLoss during training.
9. A lightweight, multimodal intelligent diagnostic method for medical imaging based on the Ascend platform, characterized in that, The system described in any one of claims 1-8 includes the following steps: Step S1: Receive and standardize multimodal medical data through the preprocessing module; Step S2: Map the standardized multimodal data to a unified semantic space and perform feature-level fusion through the cross-modal semantic fusion module to generate a deep semantic representation; Step S3: Using the cloud-edge collaborative reasoning module, based on the complexity of the diagnostic task, select to call the diagnostic model in the cloud or at the edge to reason about the deep semantic representation; Step S4: Through the multi-scale lesion identification module, the features in the reasoning process are enhanced at multiple scales, and a diagnostic report containing lesion location, classification and segmentation results is output.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in claim 9.