Medical image analysis method based on deep learning

Abstract

The invention discloses a medical image analysis method based on deep learning, and belongs to the technical field of medical image analysis. The method comprises the following steps: automatically acquiring a medical image through a DICOM interface and carrying out standardized preprocessing; performing feature extraction and semantic matching on the preprocessed image and related patient text information by adopting a multi-modal deep learning model based on an OpenCLIP architecture to generate a preliminary analysis conclusion and confidence; matching the preliminary conclusion with a structured disease category vocabulary library, and outputting a structured analysis report containing disease category names, subcategories and confidence coefficients; and when the confidence coefficient is lower than a threshold value, automatically optimizing the disease lexical library and carrying out iterative training on the model. According to the invention, full-process automation of medical image analysis is realized, and the diagnosis efficiency, accuracy and system adaptability are effectively improved.

Description

A Deep Learning-Based Medical Image Analysis Method Technical Field

[0001] This invention relates to the field of medical image analysis technology, and in particular to a deep learning-based medical image analysis method. Background Technology

[0002] Medical image analysis is a crucial component of modern medical diagnosis. With the widespread use of medical imaging equipment such as CT and MRI, the amount of image data generated clinically has exploded. Traditional image interpretation mainly relies on manual reading, which suffers from low efficiency, strong subjectivity, and susceptibility to experience-based biases, making it difficult to meet the current clinical needs for large-scale, rapid, and accurate diagnosis.

[0003] In recent years, artificial intelligence technologies such as deep learning have made significant progress in the field of medical image analysis, giving rise to various automated or semi-automated assisted diagnostic solutions to improve diagnostic accuracy and efficiency. Existing solutions mainly fall into the following categories: 1. Systems based on traditional image processing: These solutions employ classic algorithms such as edge detection and threshold segmentation to perform preliminary identification of lesions in images. Their advantage lies in their simplicity, but their main disadvantage is poor adaptability to differences in imaging conditions and complex lesions, leading to a high rate of false positives and false negatives in practical applications.

[0004] 2. Deep Learning-Based Systems: Deep learning models, represented by convolutional neural networks, have achieved higher recognition accuracy through end-to-end learning and are currently a hot research topic. However, these systems have two major problems: First, the models are "black box" characteristics, with opaque decision-making processes and a lack of interpretability; second, once the models are solidified, it is difficult to flexibly and cost-effectively expand their knowledge scope, and they cannot quickly adapt to the customized needs of different clinical scenarios for new diseases or specific diagnostic terms.

[0005] 3. Knowledge-based and rule-based systems: Some solutions improve efficiency by integrating medical knowledge bases and using rule engines to generate structured reports. However, the analytical capabilities of these systems are limited by preset rules, resulting in insufficient intelligence. Furthermore, most existing systems face common engineering challenges, such as insufficient standardization support for images from multiple vendors, low batch processing efficiency, and inadequate integration with hospital information systems, making large-scale deployment in real-world workflows difficult.

[0006] In summary, existing technologies struggle to achieve a good balance between diagnostic accuracy, clinical adaptability, and system flexibility. Therefore, there is an urgent need in the field for an automated analysis system that integrates the advanced performance of deep learning with the scalability of knowledge bases and possesses strong engineering implementation capabilities. Summary of the Invention

[0007] To address the above problems, this invention provides a deep learning-based medical image analysis method that achieves automated analysis and accurate interpretation of medical images by integrating multimodal information and a structured knowledge base, and has the ability to continuously improve diagnostic accuracy and adaptability through iterative optimization.

[0008] This invention provides a deep learning-based medical image analysis method, comprising: S1, automatically acquiring medical images via a DICOM interface and preprocessing the medical images; S2, using a multimodal deep learning model to extract features and perform semantic matching on the preprocessed medical images and related patient text information to generate preliminary image analysis conclusions and confidence levels; S3, matching the preliminary image analysis conclusions with a disease vocabulary database to obtain a structured analysis report, wherein the disease vocabulary database defines disease names, descriptions, and subcategories in a structured data format.

[0009] Furthermore, S1 specifically includes: automatically acquiring images via a DICOM interface; adjusting the window width and window level of the acquired images; and normalizing and converting the images to ensure data consistency.

[0010] By adjusting window width and level, normalizing, and converting formats, data differences caused by different imaging devices and imaging parameters can be effectively eliminated, ensuring that the data input to the model has a high degree of consistency. This provides a stable and reliable input foundation for subsequent deep learning analysis, significantly improving the accuracy and robustness of model analysis.

[0011] Furthermore, the window width and window level adjustment of the acquired images specifically includes: lung window: window width 1500-2000HU, window level -450 to -600HU; mediastinal window: window width 250-350HU, window level 30-50HU; bone window: window width 1000-1500HU, window level 250-350HU; soft tissue window: window width 300-500HU, window level 40-60HU.

[0012] By specifically limiting the range of window width and window level adjustment parameters for different human tissues (such as lungs, mediastinum, bone, and soft tissue), the image display effect of different tissues can be optimized in a targeted manner, making key lesion features (such as lung nodules, fractures, and soft tissue masses) more prominent in the images, effectively reducing missed or false detections caused by improper display parameters, and improving the professionalism and effectiveness of the preprocessing stage.

[0013] Furthermore, the multimodal deep learning model is based on the OpenCLIP architecture and is used to map medical image features and patient text information into the same semantic space for similarity calculation.

[0014] Furthermore, S2 specifically includes: S21, inputting the medical image into the image encoder of the model to extract the image visual feature vector; S22, inputting the patient text information into the text encoder of the model to extract the text feature vector; S23, mapping the image feature vector and the text feature vector into the same high-dimensional semantic space, and generating the preliminary image analysis conclusion and the corresponding confidence level by calculating the cosine similarity between the two.

[0015] By limiting the multimodal deep learning model to the OpenCLIP architecture, its powerful vision-language alignment capability can be fully utilized to associate abstract medical image features with rich patient text information in a unified semantic space. This breaks through the limitations of single image analysis and achieves a deeper understanding that integrates clinical context, thereby generating more accurate preliminary analysis conclusions that are more in line with clinical practice.

[0016] Furthermore, the structured data format is JSON.

[0017] Furthermore, S3 specifically includes: calculating the semantic similarity between the preliminary image analysis conclusion and the descriptive text of each disease entry in the disease vocabulary; sorting according to semantic similarity, selecting the disease entry with the highest similarity and its sub-category as the best matching result; and outputting the best matching result to the downstream report generation module, including the matched disease name, sub-category and corresponding confidence level.

[0018] By calculating semantic similarity, ranking and selecting the best results, and providing structured output, a precise and automatic mapping from preliminary conclusions to standardized diagnostic terminology is achieved, ensuring the professionalism and standardization of the generated reports.

[0019] Furthermore, the method also includes: if the confidence level is lower than a preset threshold, optimizing and expanding the definition of the disease vocabulary database based on the matching results, and using the optimized disease vocabulary database to iteratively train the multimodal deep learning model.

[0020] By introducing a confidence-based feedback optimization mechanism, the system acquires the ability to learn and continuously evolve. When encountering difficult cases with low confidence levels, the system can automatically trigger iterative optimization of the vocabulary and model, thereby continuously expanding the scope and accuracy of disease identification. This effectively solves the problems of poor adaptability and difficulty in coping with new scenarios in existing systems, and has long-term application value.

[0021] Compared with existing technologies, the beneficial effects of this invention are: by constructing a fully automated medical image analysis solution, standardized diagnostic reports can be generated efficiently and accurately. The core of this invention lies in utilizing a multimodal deep learning model to fuse image and text information for comprehensive analysis, and combining it with a structured disease vocabulary database to achieve accurate mapping and interpretation of diagnostic results, significantly improving the accuracy and reliability of the analysis; simultaneously, it possesses self-optimization capabilities based on confidence feedback, continuously expanding the scope of disease identification and optimizing diagnostic accuracy, thereby effectively adapting to diverse clinical needs and realizing intelligent diagnostic assistance from automated processing to continuous evolution, possessing extremely high application value. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this drawing or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this drawing. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 is a flowchart of the method of the present invention; Figure 2 is the generated structured analysis report. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0025] This invention provides a medical image analysis method based on deep learning, as shown in Figure 1, which specifically includes the following steps: S1, automatically acquiring medical images through the DICOM interface and preprocessing the medical images.

[0026] Specifically, medical image data from multiple manufacturers and types of equipment is automatically acquired via the DICOM interface. The acquired images undergo preprocessing to ensure the standardization and consistency of the input data. The preprocessing process is as follows: The image type (e.g., CT, MR, DR) is automatically identified by parsing the metadata tags in the DICOM file. Based on the identified image type, corresponding preset processing parameters and algorithms are automatically invoked to address inherent differences from different equipment manufacturers. For CT images: Due to the different grayscale response curves of equipment from different manufacturers, the same tissue may have different pixel values. Therefore, by reading and applying the Modality LUT in the DICOM file, the device-related grayscale values ​​are mapped to a standardized Hornsfield unit value range, ensuring that the grayscale values ​​of CT images from different sources have a consistent physical meaning.

[0027] For MR imaging: Different manufacturers' MRI equipment uses different signal processing algorithms and sequence parameters, which can cause significant differences in image brightness and contrast. By applying a lookup table (LUT) to adjust the image's tone curve, its brightness and contrast are made to conform to a unified diagnostic display standard, eliminating the visual impact of manufacturer specificity.

[0028] Based on the identified image type and examination site, adjust the window width and level accordingly: Lung window: window width 1500-2000HU, window level -450 to -600HU; Mediastinal window: window width 250-350HU, window level 30-50HU; Bone window: window width 1000-1500HU, window level 250-350HU; Soft tissue window: window width 300-500HU, window level 40-60HU.

[0029] The image data is normalized and converted into a uniform format required by the model (e.g., converting DICOM to JPEG).

[0030] S2 employs a multimodal deep learning model to extract features and perform semantic matching on preprocessed medical images and related patient text information, generating preliminary image analysis conclusions and confidence levels.

[0031] Specifically, the multimodal deep learning model is based on the OpenCLIP architecture and consists of an image encoder and a text encoder. The image encoder uses a Vision Transformer architecture to extract deep visual features from medical images, while the text encoder uses a Transformer architecture to extract semantic features from patient text information (including age, examination site, examination items, and clinical diagnosis).

[0032] The preprocessed medical images are input into an image encoder based on the Vision Transformer architecture. This encoder first segments the 2D image into a series of image patches, converts them into sequential patch embedding vectors through a linear embedding layer, and introduces positional encoding to preserve spatial structure information. Subsequently, self-attention computation is performed through a multi-layer Transformer encoder to progressively extract and fuse global and local features, ultimately converging at the output to generate a comprehensive image visual feature vector that fully represents the semantic content of the image.

[0033] In parallel, structured patient text information (such as age, examination site, clinical diagnosis, etc.) is input into a Transformer-based text encoder. The text encoder first performs word segmentation and embedding processing on the input text, and then captures the semantic relationships within the text through a multi-layer self-attention mechanism, finally outputting a text feature vector that accurately reflects the patient's clinical background.

[0034] To achieve collaborative understanding of images and text, a shared projection layer is used to non-linearly map the visual feature vectors of images and text originating from different modalities to a unified, high-dimensional semantic space of the same dimension. Within this shared semantic space, the cosine similarity between two feature vectors is calculated to quantitatively assess the semantic consistency between medical image content and patient clinical information. This similarity score serves not only as a direct basis for generating preliminary image analysis conclusions (such as "the image manifestations are highly correlated with the description of 'pulmonary nodules'"), but is also normalized to a confidence score between 0 and 1 to characterize the reliability of the current analysis conclusions.

[0035] S3. The preliminary image analysis results are matched with the disease vocabulary database to obtain a structured analysis report, as shown in Figure 2. The disease vocabulary database defines the disease name, description and subcategories in a structured data format.

[0036] Specifically, the pre-defined disease vocabulary database uses JSON as the core structured data format. Each disease entry constitutes a complete data object, containing the following key fields: standard disease name (Chinese and English labels); comprehensive disease description (defining the clinical manifestations and imaging characteristics of the disease); and sub-category array (for fine-grained classification of the disease, with each sub-category containing a name and a description of specific medical characteristics).

[0037] The preliminary image analysis conclusions and the descriptive texts of each entry in the disease vocabulary database are converted into feature vectors in a high-dimensional semantic space. The cosine similarity algorithm is used to quantify the semantic association strength between the preliminary conclusions and each disease description. The entries are sorted based on similarity scores, and the disease entry with the highest score and its most relevant subcategory are selected as the best matching results. These results are then output to the downstream report generation module, including the matched disease name, subcategory, and corresponding confidence level.

[0038] To improve system adaptability and diagnostic accuracy, a model optimization mechanism was further designed. When the output confidence level is lower than a preset threshold, the system will automatically optimize and expand the definition of the disease vocabulary based on the matching results, and use the optimized disease vocabulary to iteratively train the multimodal deep learning model, thereby realizing the self-evolution of the system's diagnostic capabilities during continuous use.

[0039] Specifically, the system continuously monitors the confidence score of the analysis results. When the confidence score falls below a preset threshold, the optimization process is automatically triggered. The system will mark the current analysis case as a "difficult case sample" and fully save its medical imaging data, patient text information, preliminary analysis conclusions, and low-confidence matching results to form a training sample set to be optimized.

[0040] By comparing the semantic differences between preliminary conclusions and existing disease descriptions, we can identify medical concepts or feature descriptions that are not fully covered in the existing knowledge base, add subcategories to the corresponding entries in the disease vocabulary database, or expand the medical feature description text of existing categories.

[0041] The accumulated difficult examples are merged with the original training data to construct an enhanced training set; based on the expanded knowledge base content, the semantic understanding ability of the text encoder is retrained; by contrastive learning loss function, the ability of the image encoder to extract visual features of newly added medical concepts is enhanced; after training is completed, the model performance is evaluated on an independent validation set, and the improvement effect is confirmed before switching to the production environment.

[0042] It should be noted that the present invention is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments that have the same structure and perform the same effects as the technical concept within the scope of the present invention are included within the scope of the present invention. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of the present invention, are also included within the scope of the present invention.

Claims

1. A deep learning-based medical image analysis method, characterized in that, include: S1. Automatically acquire medical images via DICOM interface and preprocess the medical images; S2. Use a multimodal deep learning model to extract features and perform semantic matching on the preprocessed medical images and related patient text information to generate preliminary image analysis conclusions and confidence levels; S3. Match the preliminary image analysis conclusions with a disease vocabulary database to obtain a structured analysis report. The disease vocabulary database defines disease names, descriptions, and subcategories in a structured data format.

2. The medical image analysis method based on deep learning as described in claim 1, characterized in that, S1 specifically includes: automatically acquiring images via the DICOM interface; adjusting the window width and window level of the acquired images; and normalizing and converting the images to ensure data consistency.

3. The medical image analysis method based on deep learning as described in claim 2, characterized in that, The window width and window level adjustment of the acquired images specifically includes: lung window: window width 1500-2000HU, window level -450 to -600HU; mediastinal window: window width 250-350HU, window level 30-50HU; bone window: window width 1000-1500HU, window level 250-350HU; soft tissue window: window width 300-500HU, window level 40-60HU.

4. The medical image analysis method based on deep learning as described in claim 1, characterized in that, The multimodal deep learning model is based on the OpenCLIP architecture and is used to map medical image features and patient text information into the same semantic space for similarity calculation.

5. The medical image analysis method based on deep learning as described in claim 4, characterized in that, S2 specifically includes: S21, inputting the medical image into the image encoder of the model to extract the image visual feature vector; S22, inputting the patient text information into the text encoder of the model to extract the text feature vector; S23, mapping the image feature vector and the text feature vector into the same high-dimensional semantic space, and generating the preliminary image analysis conclusion and the corresponding confidence level by calculating the cosine similarity between the two.

6. The medical image analysis method based on deep learning as described in claim 1, characterized in that, The structured data format is JSON.

7. The medical image analysis method based on deep learning as described in claim 1, characterized in that, S3 specifically includes: calculating the semantic similarity between the preliminary image analysis conclusion and the descriptive text of each disease entry in the disease vocabulary; sorting according to semantic similarity, selecting the disease entry with the highest similarity and its sub-category as the best matching result; and outputting the best matching result to the downstream report generation module, including the matched disease name, sub-category and corresponding confidence level.

8. The medical image analysis method based on deep learning as described in claim 1, characterized in that, The method further includes: if the confidence level is lower than a preset threshold, optimizing and expanding the definition of the disease vocabulary database based on the matching results, and using the optimized disease vocabulary database to iteratively train the multimodal deep learning model.

Images