Medical image intelligent detection and auxiliary diagnosis system based on deep learning

Abstract

The invention relates to the technical field of medical images, in particular to a medical image intelligent detection and auxiliary diagnosis system based on deep learning, and the system comprises a data access module which is used for obtaining medical image data and clinical text data of a patient; the data fusion module is used for generating a focus feature vector and a text feature vector, and performing cross-modal alignment and fusion to generate a fusion feature vector; the diagnosis analysis module is used for executing focus detection, focus segmentation and focus classification tasks, generating a diagnosis result and generating a diagnosis label based on the diagnosis result; the decision generation module is used for mapping the generated execution result to a preset medical knowledge base and performing deep reasoning to generate an auxiliary diagnosis decision; the decision auditing module is used for performing confidence scoring on the auxiliary diagnosis decisions and selecting the auxiliary diagnosis decision with the highest confidence score as the final auxiliary decision; and the data visualization module is used for carrying out visualization processing on the auxiliary decision and the diagnosis result.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, and more particularly to a deep learning-based intelligent detection and auxiliary diagnosis system for medical images. Background Technology

[0002] In the rapid development of modern medical technology, medical imaging diagnosis, as a crucial link in the accurate diagnosis of diseases, has always been at the forefront of continuous exploration and innovation in the medical field. Traditional medical imaging diagnosis mainly relies on doctors' rich experience and professional knowledge to manually interpret images such as X-rays, CT scans, and MRIs. However, this method is not only inefficient and unable to meet the ever-increasing medical needs, but it is also highly susceptible to multiple factors such as the doctor's subjective factors, fatigue level, and differences in experience, leading to frequent misdiagnosis and missed diagnosis.

[0003] Currently, deep learning-based medical image analysis technologies have been widely applied, such as using convolutional neural networks for lesion identification. However, most models rely solely on a single image modality (such as CT or MRI) for feature learning, failing to effectively integrate complementary information from multimodal images (such as CT-MRI-PET) and patient clinical text data (such as medical history and laboratory indicators). This results in incomplete feature representation and affects diagnostic accuracy.

[0004] Furthermore, existing tools often treat lesion detection, segmentation, classification, and diagnostic recommendations as independent modules, resulting in lengthy processes. Errors in intermediate results are propagated at each level, affecting the accuracy of the final diagnosis and the efficiency of the system. The model's prediction results lack interpretability, making it difficult for doctors to understand the basis for a certain diagnosis and thus hindering trust and limiting its practical application in critical clinical decisions. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a deep learning-based intelligent detection and assisted diagnosis system for medical images.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A deep learning-based intelligent detection and assisted diagnosis system for medical images includes:

[0008] The data access module is used to acquire patients' multimodal medical image data and clinical text data;

[0009] The data fusion module is used to extract multimodal medical image data and clinical text data respectively, generate lesion feature vectors and text feature vectors, and perform cross-modal alignment and fusion to generate fused feature vectors;

[0010] The diagnostic analysis module is used to simultaneously perform lesion detection, lesion segmentation and lesion classification tasks based on the fused feature vector and generate diagnostic results, and generate diagnostic labels based on the diagnostic results, the diagnostic results including lesion annotation and feature heatmap;

[0011] The decision generation module is used to map the generated execution results to a pre-set medical knowledge base and perform deep reasoning to generate auxiliary diagnostic decisions.

[0012] The decision review module is used to score the confidence level of auxiliary diagnostic decisions and select the auxiliary diagnostic decision with the highest confidence level as the final auxiliary decision.

[0013] The data visualization module is used to visualize the results of decision support and diagnosis.

[0014] Preferably, the data fusion module includes:

[0015] The image feature extraction subunit is used to extract image feature maps of different modalities using a deep convolutional neural network and generate lesion feature vectors.

[0016] The clinical data embedding subunit is used to map clinical text data into text feature vectors through the embedding layer;

[0017] A feature alignment unit is used to connect to the image feature extraction subunit and the clinical data embedding subunit respectively, and is used to project the image feature map and the numerical feature vector onto a common semantic space.

[0018] The feature fusion unit is used to perform weighted fusion of the image feature map and the numerical feature vector that are in a common semantic space through an attention mechanism to generate the fused feature.

[0019] Preferably, the process of performing cross-modal alignment and fusion to generate a fused feature vector specifically includes:

[0020] The lesion feature vector and the text feature vector are projected into a common low-dimensional semantic space through fully connected layers or convolutional layers, respectively.

[0021] Using the projected text feature vector as the query and the projected lesion feature vector as the key and value, the cross-modal attention weight between the two is calculated, and clinical information-guided image features are generated.

[0022] The clinical information-guided image features are concatenated with the projected text feature vectors, and the concatenated features are input into a Transformer encoder for deep fusion, outputting the final fused features.

[0023] Preferably, the data visualization module includes:

[0024] The two-dimensional display subunit is used to present medical image slices on a two-dimensional plane and overlay lesion annotations, feature heatmaps, and confidence levels for auxiliary diagnostic decisions generated by the diagnostic analysis module and the diagnostic review module.

[0025] The 3D display subunit is used to perform 3D volume rendering or surface rendering based on a continuous sequence of medical image slices to reconstruct a 3D model of the target organ or lesion.

[0026] The display switching unit is used to communicate with the two-dimensional display subunit and the three-dimensional display subunit respectively, and is configured to switch and synchronize between the two-dimensional view and the three-dimensional view in response to user commands.

[0027] Preferably, the configuration is to switch and synchronize between two-dimensional and three-dimensional views in response to user commands, specifically including:

[0028] Receive view switching commands or point selection commands on the 3D model issued by the user through the interactive interface;

[0029] When the instruction is to switch from a 3D view to a 2D view, the corresponding original 2D image slice containing the selected point is automatically calculated and located based on the coordinates of the point selected by the user on the 3D model.

[0030] When the command is to switch from a 2D view to a 3D view, the viewpoint and zoom of the 3D model are automatically adjusted based on the spatial position of the currently displayed 2D slice in the original sequence, so that it focuses on the anatomical structure corresponding to the slice.

[0031] During the switching process, the lesion annotation, feature heatmap, and confidence level of auxiliary diagnostic decisions are displayed synchronously between the two views.

[0032] Preferably, the formula for calculating the confidence score for the auxiliary diagnostic decision is as follows:

[0033]

[0034] Where SCORE is the confidence score. Based on the maximum conditional probability of diagnostic labels in the medical record database, Sim(.) represents the cosine similarity between lesion features and disease prototype features, C represents the level of evidence obtained according to disease guideline clauses, and rank(a) represents the level of evidence. i ) represents the suggested disease grade, Φ(F) represents the lesion feature vector, and Ψ(a) represents the lesion feature vector. i Let be the original disease feature corresponding to the i-th diagnostic decision, and let a be the original disease feature. i Let be the i-th auxiliary diagnostic decision, D be the diagnostic label, γ be the dynamic weight, η be the guideline weight, K be the number of auxiliary diagnostic decisions, and δ be the conflict term.

[0035] Preferably, the step of mapping the generated execution results to a pre-set medical knowledge base and performing deep reasoning to generate auxiliary diagnostic decisions specifically includes:

[0036] The execution results are mapped to entities in a pre-built medical knowledge base, and all reasoning paths between entities and potential diseases are traversed.

[0037] Along each reasoning path, aggregate hypothetical evidence that supports or opposes the diagnostic decision, and calculate the overall support for each hypothesis based on predefined rules or probabilistic graphical models, while removing contradictory hypothetical evidence;

[0038] The overall support score of all hypothetical evidence is aggregated, and the reasoning path with the highest overall support score is taken as the reasoning path corresponding to the entity and the potential disease.

[0039] Repeat the steps until all entities and potential diseases are obtained, sort and aggregate all reasoning paths, and generate auxiliary diagnostic decisions based on the reasoning paths.

[0040] Preferably, mapping the execution result to entities in a pre-set medical knowledge base specifically includes:

[0041] The text feature vectors are transformed into word embedding vectors using a word vector model;

[0042] Calculate the cosine similarity between the word embedding vector and the entity name vector in the medical knowledge base;

[0043] Knowledge base entities with similarity exceeding a preset threshold are identified as target entities for successful mapping.

[0044] The beneficial effects of this invention are as follows:

[0045] This application breaks down data silos and forms a more comprehensive understanding of the disease by combining medical imaging data and clinical text data for cross-modal feature alignment and fusion.

[0046] By setting up a decision generation model, the time from image analysis to generating a structured report is shortened, reducing the workload of doctors. At the same time, it ensures the standardization and completeness of the report content. From data input to the generation of a complete structured report containing image findings, diagnostic opinions, differential diagnoses and recommendations, no manual intervention is required throughout the process. The intelligent switching and synchronous linkage of the visualization module between two-dimensional and three-dimensional modes enables doctors to quickly perform spatial positioning and morphological analysis, avoiding the cumbersome operation of frequently switching between multiple tools. Moreover, the decision generation module automatically matches image findings with the medical knowledge base to generate targeted diagnostic decisions, reducing the time doctors spend memorizing and searching for standard treatment plans.

[0047] By setting up confidence scores based on diagnostic decisions, doctors can quickly assess the reliability of the results and assist in decision-making. Attached Figure Description

[0048] Figure 1 This is a block diagram of a deep learning-based intelligent medical image detection and auxiliary diagnosis system in a specific embodiment of the present invention.

[0049] Figure 2 This is a flowchart illustrating the generation of fused feature vectors in a specific embodiment of the present invention;

[0050] Figure 3 This is a flowchart illustrating the switching process in a specific embodiment of the present invention;

[0051] Figure 4 This is a flowchart illustrating the auxiliary diagnostic decision generation process in a specific embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Please see Figures 1-4 As shown, this invention relates to a deep learning-based intelligent detection and assisted diagnosis system for medical images, comprising:

[0054] The data access module is used to acquire patients' multimodal medical image data and clinical text data;

[0055] Specifically, the data access module includes:

[0056] The image data access subunit is configured to acquire medical image data from multiple medical image sources, specifically including:

[0057] The DICOM protocol interface is configured to receive medical image sequences directly from the PACS system via the DICOM network protocol.

[0058] File system monitor, configured to monitor specified local or network storage directories, automatically identify and load newly stored DICOM files;

[0059] The API interface is configured to retrieve image data from third-party systems or cloud storage services via a RESTful API or HL7 FHIR interface, specifically including:

[0060] The HL7 interface engine is configured to parse HL7 messages sent from HIS, RIS, or LIS systems and extract patient clinical information from them.

[0061] The FHIR client is configured to query and retrieve structured clinical data from compliant electronic health records via the FHIR standard.

[0062] The natural language processing model is configured to perform named entity recognition and relation extraction on unstructured clinical texts, such as doctor's notes and discharge summaries, in order to extract key clinical entities and structure them.

[0063] The clinical data access subunit is configured to retrieve patients' clinical text data from the hospital information system.

[0064] The data preprocessing and standardization subunit is connected to the image data access subunit and the clinical data access subunit, respectively, and is configured to preprocess and standardize the acquired raw data.

[0065] The data fusion module is used to extract multimodal medical image data and clinical text data respectively, generate lesion feature vectors and text feature vectors, and perform cross-modal alignment and fusion to generate fused feature vectors. Specifically, it includes:

[0066] The image feature extraction subunit is used to extract image feature maps of different modalities using a deep convolutional neural network and generate lesion feature vectors.

[0067] The clinical data embedding subunit is used to map clinical text data into text feature vectors through the embedding layer;

[0068] A feature alignment unit is used to connect to the image feature extraction subunit and the clinical data embedding subunit respectively, and is used to project the image feature map and the numerical feature vector onto a common semantic space.

[0069] The feature fusion unit is used to perform weighted fusion of the image feature map and the numerical feature vector within a common semantic space through an attention mechanism to generate the fused feature, specifically including:

[0070] The lesion feature vector and the text feature vector are projected into a common low-dimensional semantic space through fully connected layers or convolutional layers, respectively.

[0071] Using the projected text feature vector as the query and the projected lesion feature vector as the key and value, the cross-modal attention weight is calculated, and clinical information-guided image features are generated.

[0072] The clinical information-guided image features are concatenated with the projected text feature vectors, and the concatenated features are input into a Transformer encoder for deep fusion, outputting the final fused features.

[0073] The diagnostic analysis module is used to simultaneously perform lesion detection, lesion segmentation and lesion classification tasks based on the fused feature vector and generate diagnostic results, and generate diagnostic labels based on the diagnostic results, the diagnostic results including lesion annotation and feature heatmap;

[0074] The decision generation module maps the generated execution results to a pre-set medical knowledge base and performs deep reasoning to generate auxiliary diagnostic decisions, specifically including:

[0075] The execution results are mapped to entities in a pre-set medical knowledge base, and all reasoning paths between entities and potential diseases are traversed. The medical knowledge base includes at least disease entities, image feature entities, clinical symptom entities, and the semantic relationships between them.

[0076] Along each reasoning path, aggregate hypothetical evidence that supports or opposes the diagnostic decision, and calculate the overall support for each hypothesis based on predefined rules or probabilistic graphical models, while removing contradictory hypothetical evidence;

[0077] The overall support score of all hypothetical evidence is aggregated, and the reasoning path with the highest overall support score is taken as the reasoning path corresponding to the entity and the potential disease.

[0078] Repeat the steps until all entities and potential diseases are obtained, sort and aggregate all reasoning paths, and generate auxiliary diagnostic decisions based on the reasoning paths.

[0079] The decision review module is used to score the confidence level of auxiliary diagnostic decisions and select the auxiliary diagnostic decision with the highest confidence level as the final auxiliary decision. The formula for calculating the confidence level of the auxiliary diagnostic decision is as follows:

[0080]

[0081] Where SCORE is the confidence score. Based on the maximum conditional probability of diagnostic labels in the medical record database, Sim(.) represents the cosine similarity between lesion features and disease prototype features, C represents the level of evidence obtained according to disease guideline clauses, and rank(a) represents the level of evidence. i ) represents the suggested disease grade, Φ(F) represents the lesion feature vector, and Ψ(a) represents the lesion feature vector. i Let be the original disease feature corresponding to the i-th diagnostic decision, and let a be the original disease feature. iLet be the i-th auxiliary diagnostic decision, D be the diagnostic label, γ be the dynamic weight, η be the guideline weight, K be the number of auxiliary diagnostic decisions, and δ be the conflict term.

[0082] The lesion feature vector Φ(F) is obtained by the data fusion module and through the deep feature extractor, Ψ(a i Templates are defined by medical experts, for example, for typical lung cancer features = [size > 10mm, spiculation +, enhancement > 15HU];

[0083] The data visualization module is used to visualize the results of decision support and diagnosis, and specifically includes:

[0084] The two-dimensional display subunit is used to present medical image slices on a two-dimensional plane and overlay lesion annotations, feature heatmaps, and confidence levels for auxiliary diagnostic decisions generated by the diagnostic analysis module and the diagnostic review module.

[0085] The 3D display subunit is used to perform 3D volume rendering or surface rendering based on a continuous sequence of medical image slices to reconstruct a 3D model of the target organ or lesion.

[0086] The display switching unit is used to communicate with the two-dimensional display subunit and the three-dimensional display subunit respectively, and is configured to switch and synchronize between the two-dimensional view and the three-dimensional view in response to user commands.

[0087] When receiving view switching commands or point selection commands on the 3D model issued by the user through the interactive interface;

[0088] When the instruction is to switch from a 3D view to a 2D view, the corresponding original 2D image slice containing the selected point is automatically calculated and located based on the coordinates of the point selected by the user on the 3D model.

[0089] When the command is to switch from a 2D view to a 3D view, the viewpoint and zoom of the 3D model are automatically adjusted based on the spatial position of the currently displayed 2D slice in the original sequence, so that it focuses on the anatomical structure corresponding to the slice.

[0090] During the switching process, the lesion annotation, feature heat map and confidence of auxiliary diagnostic decision are displayed synchronously between the two views. When the user browses different image slices, the cutting plane or view orientation of the three-dimensional model in the three-dimensional display subunit is updated in real time to match the spatial position of the current two-dimensional slice.

[0091] When a user rotates, translates, or scales the 3D model, the orientation indicator of the 2D slice sequence displayed in the 2D display subunit is updated in real time to reflect the current 3D viewpoint.

[0092] Example 2

[0093] Based on the above embodiment 1, in this embodiment, in order to improve the accuracy of the decision support of this application, after acquiring multimodal medical image data and clinical text data through the data access module, a confidence score is performed on the accessed multimodal medical image data and clinical text data to screen reliable data, specifically including:

[0094] Clinical text data is matched with a pre-built medical knowledge base to assess the degree of support of clinical evidence for imaging diagnostic results. Based on the degree of matching, a clinical consistency confidence value between 0 and 1 is output through a rule engine or a lightweight discriminant model.

[0095] By setting up an analysis network, multimodal medical image data is input into the analysis network. For each lesion to be classified, the classification head outputs a category probability distribution vector. The image classification confidence is taken from the probability value of the corresponding preferred diagnostic category in the vector.

[0096] The confidence scores for image classification and clinical consistency are calculated using a weighted fusion formula to generate a preliminary fusion confidence score. The preliminary fusion confidence score is then corrected based on the clinical context features of the case to obtain the final comprehensive confidence score.

[0097] The formula for the overall confidence level is:

[0098] S = α·A + β·B + γ·(A·B);

[0099] Where S is the overall confidence score, A is the image classification confidence score, B is the clinical consistency score, and α, β, and γ are preset weighting coefficients that are manually set.

[0100] When S is greater than the threshold, the corresponding multimodal medical image data and clinical text data are collected; when S is less than the threshold, the data is collected again.

[0101] Furthermore, this embodiment provides an example of using confidence scoring for assisted diagnostic decisions, specifically as follows:

[0102] 1. Decision-making for subsequent diagnosis of pulmonary nodules

[0103] The auxiliary diagnostic decision is:

[0104] a1: A CT scan will be performed in three months; a2: Further PET-CT scan;

[0105] Parameter values: γ = 0.6; η = 0.15;

[0106] Knowledge base statistics:

[0107] Feature matching accuracy: Sim(a1): Circular shape + small size matches benign feature template -0.92; Sim(a2): Lacks support from malignant features -0.35;

[0108] Level of Evidence for Guideline Clauses: rank(a1)=1, rank(a2)=2;

[0109]

[0110] The final diagnostic decision was followed up by CT scan 6 months later, with a reliability of 95.3%.

[0111] 2. Diagnostic decision-making based on breast mass biopsy

[0112] The diagnostic decision is as follows:

[0113] a1: Core needle biopsy; a2: Surgical excision;

[0114] Parameter values: γ = 0.75; η = 0.18;

[0115] Knowledge base statistics:

[0116] Feature matching degree: Sim(a1): BI-RADS 4C+ spiculation sign strongly suggests puncture -0.98; Sim(a2): size does not meet the standard for direct surgery -0.65;

[0117] Level of Evidence for Guideline Clauses: rank(a1)=2, rank(a2)=3;

[0118] Conflict detection: conflict between family history and conservative treatment - δ = 0.2;

[0119]

[0120] The final diagnostic decision output is a core needle biopsy.

[0121] Example 3

[0122] Based on the method described in Example 1, when generating auxiliary diagnostic decisions, the hypothetical evidence that supports or opposes the diagnostic decision is aggregated along each reasoning path, and the overall support of each hypothetical evidence is calculated based on predefined rules or probabilistic graphical models, while contradictory hypothetical evidence is removed.

[0123] Specifically, each "feature-disease" relationship in the knowledge base is assigned a weight, which is preset by experts or learned from data based on the strength of the medical specificity of the relationship;

[0124] The sum of the positive and negative results yields the overall support score for the hypothesis. The calculation formula is as follows:

[0125] Support S core(D i )=Σ(W p Strength p )-Σ(W n Strength o );

[0126] Where D i Let W represent the i-th piece of evidence for the hypothesis. p Strength represents the weight of the evidence supporting the i-th hypothesis. p W represents the strength of the evidence for the i-th hypothesis. n Strength represents the weight of the opposing evidence for the i-th hypothesis. o This represents the strength of the non-existence of the i-th hypothesis evidence.

[0127] For example: for lung cancer (D1)

[0128] Positive evidence:

[0129] Evidence 1: The feature “solid pulmonary nodules” has a weight of W = 0.8 and a strength of presence of this feature in the current patient of S = 0.9;

[0130] Evidence 2: The feature “lobulation sign”, with a weight W = 0.7, has an intensity S = 0.7 in the current patient;

[0131] Evidence 3: The feature “spiculated sign”, with a weight W = 0.8, has an intensity S = 0.6 in the current patient;

[0132] Negative evidence:

[0133] Evidence 1: Feature “no history of smoking”, weight W = 0.7, presence strength S = 1.0;

[0134] Evidence 2: The feature "no change in lesions over 2 years" has a weight of W = 0.8 and an intensity of presence of S = 0.9;

[0135] The overall support score for path D1 - lung cancer is as follows:

[0136] Support S core(D1)=0.8×0.9+0.7×0.7+0.8×0.6-0.6×1.0-0.8×0.9=0.37;

[0137] For pneumonia (D2)

[0138] Positive evidence:

[0139] Evidence 1: The feature “ground-glass opacity in the lungs”, with a weight of W = 0.8, has a strength of presence of S = 0.9 in the current patient;

[0140] Evidence 2: The feature “fever symptom” has a weight of W = 0.6, and its intensity in the current patient is S = 0.8;

[0141] Evidence 3: The feature “elevated white blood cell count” has a weight of W = 0.7, and its intensity in the current patient is S = 0.7.

[0142] Evidence 4: The feature “cough symptom”, with a weight of W = 0.5, has an intensity of S = 0.9 in the current patient;

[0143] Negative evidence: None

[0144] The overall support score for path D2 - pneumonia is as follows:

[0145] Support S core(D2)=0.8×0.9+0.6×0.8+0.7×0.7+0.5×0.9=2.14;

[0146] Then path D1 serves as the reasoning path corresponding to the entity and the potential disease. Based on lung cancer, it calls the corresponding treatment decision from the pre-set expert knowledge base to generate auxiliary diagnostic decisions.

[0147] Furthermore, in some other embodiments, deep reasoning of the reasoning path is also included, including:

[0148] Utilize the causal relationships between diseases and symptoms / features defined in the knowledge base;

[0149] For a highly supported diagnostic hypothesis, reason whether it can serve as a "cause" to reasonably explain most of the imaging features and clinical symptoms currently exhibited by the patient;

[0150] The explanatory power of the diagnostic hypothesis is incorporated as a weighting factor into the final overall support calculation, so that diagnostic hypotheses that can explain more evidence receive a higher ranking.

[0151] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A deep learning-based intelligent detection and assisted diagnosis system for medical images, characterized in that, include: The data access module is used to acquire patients' multimodal medical image data and clinical text data; The data fusion module is used to extract multimodal medical image data and clinical text data respectively, generate lesion feature vectors and text feature vectors, and perform cross-modal alignment and fusion to generate fused feature vectors; The diagnostic analysis module is used to simultaneously perform lesion detection, lesion segmentation and lesion classification tasks based on the fused feature vector and generate diagnostic results, and generate diagnostic labels based on the diagnostic results, the diagnostic results including lesion annotation and feature heatmap; The decision generation module is used to map the generated execution results to a pre-set medical knowledge base and perform deep reasoning to generate auxiliary diagnostic decisions. The decision review module is used to score the confidence level of auxiliary diagnostic decisions and select the auxiliary diagnostic decision with the highest confidence level as the final auxiliary decision. The data visualization module is used to visualize the results of decision support and diagnosis.

2. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 1, characterized in that, The data fusion module includes: The image feature extraction subunit is used to extract image feature maps of different modalities using a deep convolutional neural network and generate lesion feature vectors. The clinical data embedding subunit is used to map clinical text data into text feature vectors through the embedding layer; A feature alignment unit is used to connect to the image feature extraction subunit and the clinical data embedding subunit respectively, and is used to project the image feature map and the numerical feature vector onto a common semantic space. The feature fusion unit is used to perform weighted fusion of the image feature map and the numerical feature vector that are in a common semantic space through an attention mechanism to generate the fused feature.

3. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 1, characterized in that, The process of performing cross-modal alignment and fusion to generate a fused feature vector specifically includes: The lesion feature vector and the text feature vector are projected into a common low-dimensional semantic space through fully connected layers or convolutional layers, respectively. Using the projected text feature vector as the query and the projected lesion feature vector as the key and value, the cross-modal attention weight between the two is calculated, and clinical information-guided image features are generated. The clinical information-guided image features are concatenated with the projected text feature vectors, and the concatenated features are input into a Transformer encoder for deep fusion, outputting the final fused features.

4. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 1, characterized in that, The data visualization module includes: The two-dimensional display subunit is used to present medical image slices on a two-dimensional plane and overlay lesion annotations, feature heatmaps, and confidence levels for auxiliary diagnostic decisions generated by the diagnostic analysis module and the diagnostic review module. The 3D display subunit is used to perform 3D volume rendering or surface rendering based on a continuous sequence of medical image slices to reconstruct a 3D model of the target organ or lesion. The display switching unit is used to communicate with the two-dimensional display subunit and the three-dimensional display subunit respectively, and is configured to switch and synchronize between the two-dimensional view and the three-dimensional view in response to user commands.

5. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 4, characterized in that, The configuration is designed to respond to user commands and switch between two-dimensional and three-dimensional views, and to synchronize these views. Specifically, it includes: Receive view switching commands or point selection commands on the 3D model issued by the user through the interactive interface; When the instruction is to switch from a 3D view to a 2D view, the corresponding original 2D image slice containing the selected point is automatically calculated and located based on the coordinates of the point selected by the user on the 3D model. When the command is to switch from a 2D view to a 3D view, the viewpoint and zoom of the 3D model are automatically adjusted based on the spatial position of the currently displayed 2D slice in the original sequence, so that it focuses on the anatomical structure corresponding to the slice. During the switching process, the lesion annotation, feature heatmap, and confidence level of auxiliary diagnostic decisions are displayed synchronously between the two views.

6. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 1, characterized in that, The formula for calculating the confidence score of the auxiliary diagnostic decision is as follows: Where SCORE is the confidence score. Based on the maximum conditional probability of diagnostic labels in the medical record database, Sim(.) represents the cosine similarity between lesion features and disease prototype features, C represents the level of evidence obtained according to disease guideline clauses, and rank(a) represents the level of evidence. i ) represents the suggested disease grade, Φ(F) represents the lesion feature vector, and Ψ(a) represents the lesion feature vector. i Let be the original disease feature corresponding to the i-th diagnostic decision, and let a be the original disease feature. i Let be the i-th auxiliary diagnostic decision, D be the diagnostic label, γ be the dynamic weight, η be the guideline weight, K be the number of auxiliary diagnostic decisions, and δ be the conflict term.

7. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 1, characterized in that, The process of mapping the generated execution results to a pre-set medical knowledge base and performing deep reasoning to generate auxiliary diagnostic decisions specifically includes: The execution results are mapped to entities in a pre-built medical knowledge base, and all reasoning paths between entities and potential diseases are traversed. Along each reasoning path, aggregate hypothetical evidence that supports or opposes the diagnostic decision, and calculate the overall support for each hypothesis based on predefined rules or probabilistic graphical models, while removing contradictory hypothetical evidence; The overall support score of all hypothetical evidence is aggregated, and the reasoning path with the highest overall support score is taken as the reasoning path corresponding to the entity and the potential disease. Repeat the steps until all entities and potential diseases are obtained, sort and aggregate all reasoning paths, and generate auxiliary diagnostic decisions based on the reasoning paths.

8. The deep learning-based intelligent medical image detection and assisted diagnosis system according to claim 7, characterized in that, The mapping of the execution results to entities in a pre-built medical knowledge base specifically includes: The text feature vectors are transformed into word embedding vectors using a word vector model; Calculate the cosine similarity between the word embedding vector and the entity name vector in the medical knowledge base; Knowledge base entities with similarity exceeding a preset threshold are identified as target entities for successful mapping.

Images