Medical image analysis system based on mixed detection mask large model

Abstract

The invention discloses a medical image analysis system based on a mixed detection mask large model. The medical image analysis system comprises a data input module, an image processing and analysis module, a diagnosis suggestion module, a report generation module, a user interface module and a system management module. Wherein the image processing and analysis module is a mixed detection mask large model and comprises an anomaly detection sub-module and an image segmentation sub-module; the anomaly detection sub-module uses a DETR model to carry out abnormal region identification detection on a preprocessed medical image; the image segmentation sub-module uses a Mask2Former model to accurately segment the abnormal region identified by the anomaly detection sub-module to generate a segmentation mask; by means of the innovative mixed detection mask large model, the problems of high cost, model generalization, real-time performance, interpretability and the like in an existing medical image analysis technology are solved, and the accuracy and efficiency of medical image analysis and diagnosis can be improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image analysis and artificial intelligence technology, and more specifically to a medical image analysis system based on a hybrid detection mask large model. Background Art

[0002] At present, medical image analysis technology generally adopts traditional image processing technology and primary machine learning technology. Although these technologies have made progress in some aspects, they often perform poorly when processing complex or blurred images, especially in accurately identifying and segmenting fine structures in medical images. In addition, existing systems mostly rely on built-in software of high-end medical imaging equipment, which not only increases medical costs, but also limits the application of these advanced technologies in a wide range of areas. There are still several key problems in the current medical image analysis system:

[0003] Existing systems have difficulty adapting to images obtained from different types of medical imaging equipment because these devices vary greatly in image quality, resolution, and contrast, which greatly reduces their effectiveness in new environments. The real-time performance of existing systems often cannot meet the needs of clinical emergencies, especially when processing large amounts of data or high-resolution images. The system response time is too long to provide immediate diagnostic results. Existing systems often lack sufficient explainability, and medical professionals find it difficult to understand the decision-making process of AI models, which directly affects their trust and acceptance of system outputs.

[0004] Therefore, there is an urgent need to develop a medical image analysis system that is cost-effective, accurate, and easy to promote and use. Summary of the invention

[0005] In view of this, the present invention provides a medical image analysis system based on a hybrid detection mask large model that solves at least some of the above-mentioned technical problems. The medical image analysis system is cost-effective, accurate and easy to promote and use. It can process and analyze images from various medical imaging devices in real time, automatically detect abnormal areas in images, and accurately segment these areas for further quantitative analysis; and provide image diagnosis suggestions based on deep learning.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention provides a medical image analysis system based on a hybrid detection mask large model, the medical image analysis system comprising: a data input module, an image processing and analysis module, and a diagnosis suggestion module; wherein:

[0008] The data input module includes: an image acquisition interface and a data preprocessing unit, wherein: the image acquisition interface acquires medical image data from the medical imaging device in real time; the data preprocessing unit uses the image processing library to perform data preprocessing on the acquired medical image data;

[0009] The image processing and analysis module is a hybrid detection mask large model, including: an abnormality detection submodule, an image segmentation submodule and a feature extraction submodule, wherein: the abnormality detection submodule uses the DETR model to identify and detect abnormal areas of preprocessed medical images; the image segmentation submodule uses the Mask2Former model to accurately segment the abnormal areas identified by the abnormality detection submodule to generate segmentation masks; the feature extraction submodule is used to extract features of abnormal areas;

[0010] The diagnosis suggestion module uses a deep learning diagnosis model to process the output results of the hybrid detection mask large model and outputs diagnosis suggestions.

[0011] Preferably, the image acquisition interface acquires medical image data from a medical imaging device in real time, and the acquired medical image data includes: X-ray images, MRI images, CT images and ultrasonic images.

[0012] Preferably, the hybrid detection mask large model adopts a lightweight optimization strategy for model optimization; the lightweight optimization strategy includes: model quantization, parameter pruning, knowledge distillation, low-rank decomposition, lightweight network architecture, quantization-aware training, model compression and acceleration.

[0013] Preferably, a graph attention network module is also introduced into the hybrid detection mask large model to refine the edges of the detected abnormal areas.

[0014] Preferably, in the diagnosis suggestion module, annotated medical image data is used to train a deep learning diagnosis model, and the deep learning diagnosis model combines medical knowledge and pathological information to provide image diagnosis suggestions.

[0015] Preferably, the medical image analysis system also includes: a report generation module, which includes a report editor and a report review system, wherein: the report editor is used to automatically generate a diagnostic report based on the diagnostic suggestions output by the diagnostic suggestion module; the report review system is used to provide doctors with the function of reviewing and modifying reports.

[0016] Preferably, the medical image analysis system also includes: a user interface module, which includes: a doctor operation interface and a patient information management system, wherein: the doctor operation interface is used to provide doctors with the functions of viewing images, diagnosis suggestions and reports; the patient information management system is used to manage patient information and historical diagnosis records.

[0017] Preferably, the medical image analysis system further includes: a system management module, which includes: a system monitoring module and a data management module, wherein: the system monitoring module is used to monitor system performance and stability; the data management module is used to manage stored image data and diagnostic records.

[0018] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0019] 1. The present invention provides a medical image analysis system based on a hybrid detection mask large model. The medical image analysis system is cost-effective, accurate and easy to promote and use. It can process and analyze images from various medical imaging devices in real time, automatically detect abnormal areas in images, and accurately segment these areas for further quantitative analysis; and provide image diagnosis suggestions based on deep learning; it solves the problems of model generalization, real-time and interpretability in existing medical image analysis technologies.

[0020] 2. The medical image analysis system of the present invention improves the accuracy and efficiency of medical image diagnosis through accurate image segmentation and deep learning diagnosis suggestions using a hybrid detection mask large model (HDM).

[0021] 3. The present invention adopts a lightweight optimization strategy, including model quantization and parameter pruning, which significantly reduces the computational complexity and response time of the model and meets the needs of real-time diagnosis.

[0022] 4. The medical image analysis system of the present invention can run on low-power, low-latency devices, reducing dependence on high-end computing resources.

[0023] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0024] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0027] Figure 1 A schematic diagram of the overall framework of a medical image analysis system based on a hybrid detection mask large model provided in an embodiment of the present invention.

[0028] Figure 2 A schematic diagram of the working principle of the hybrid detection mask large model provided in an embodiment of the present invention.

[0029] Figure 3 A schematic diagram of the workflow of a data input module provided in an embodiment of the present invention.

[0030] Figure 4 A schematic diagram of the workflow of the image processing and analysis module provided in an embodiment of the present invention.

[0031] Figure 5 A schematic diagram of the workflow of the diagnosis suggestion module provided in an embodiment of the present invention.

[0032] Figure 6 A schematic diagram of the workflow of the report generation module provided in an embodiment of the present invention.

[0033] Figure 7 A functional schematic diagram of a user interface module provided in an embodiment of the present invention.

[0034] Figure 8 A functional schematic diagram of a system management module provided in an embodiment of the present invention.

[0035] Fig. 9 A functional schematic diagram of a lightweight optimization strategy provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations. In addition, various serial numbers, etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0037] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0038] See also Figure 1 As shown, an embodiment of the present invention provides a medical image analysis system based on a hybrid detection mask model (Hybrid DETR-Mask2Former, HDM). The core idea of ​​HDM is to integrate the object detection capability of DETR and the image segmentation capability of Mask2Former into a unified framework, and introduce an attention mechanism to enhance the model's ability to recognize and analyze key features in medical images. The system architecture of the medical image analysis system mainly includes: a data input module, an image processing and analysis module, a diagnosis suggestion module, a report generation module, a user interface module, and a system management module. The medical image analysis system can process and analyze images from various medical imaging devices in real time, including but not limited to X-ray, MRI, CT, and ultrasonic images. The main uses of the system include: automatically detecting abnormal areas in images, such as tumors, inflammation, or other pathological changes; accurately segmenting these areas for further quantitative analysis; providing image diagnosis suggestions based on deep learning; and generating detailed diagnostic reports for doctors' reference. The medical image analysis system is particularly suitable for emergency and intensive care scenarios that require rapid and accurate diagnostic support, as well as routine medical examinations and long-term disease management.

[0039] Further, see Figure 2 As shown, it shows the main working principle of the hybrid detection mask large model HDM:

[0040] Input Image: The model receives an image to be processed.

[0041] CNN backbone network: Use a convolutional neural network backbone network to extract preliminary features of the image.

[0042] Feature map context enhancement layer: enhances the context information of feature maps.

[0043] Enhanced feature map: Extract the enhanced feature map.

[0044] Transformer Encoder: Utilizes the Transformer architecture for anomaly detection and preliminary segmentation.

[0045] Prediction results: The model predicts the mask and category of each abnormal region.

[0046] Final mask and category: If the prediction result does not need to be refined, it is output directly.

[0047] DETR and Mask2Former attention refinement layer: If the prediction result needs to be refined, it enters this layer; combining the image detection capability of HDM-Detect (DETR model) and the image segmentation capability of HDM-Segment (Mask2Former model) to predict the location and category of the abnormal area in the image. The Graph Attention Network (GAT) module is introduced to refine the edges of the detected abnormal area to improve the segmentation accuracy; and refine the mask of the abnormal area.

[0048] Diagnosis suggestion layer: Based on the refined abnormal area mask, deep learning classifier and medical knowledge base are used to provide diagnosis suggestions for abnormal areas.

[0049] Output diagnosis report: The system outputs a detailed diagnosis report for the doctor's reference.

[0050] In an embodiment of the present invention, the entire model framework (HDM) is designed to be end-to-end trainable, allowing the model to collaboratively optimize the performance of each component during the training process.

[0051] The following is a detailed description of the specific implementation and working principle of a medical image analysis system based on a hybrid detection mask large model of the present invention. The functions and processes of each module of the system can be referred to in detail. Figure 3-Figure 8 As shown:

[0052] 1. Data input module:

[0053] Image acquisition interface: responsible for acquiring image data from medical imaging equipment (such as X-ray, MRI, CT, ultrasound) in real time.

[0054] Data preprocessing unit: Use image processing libraries (such as OpenCV) to perform operations such as denoising and contrast enhancement.

[0055] 2. Image processing and analysis module:

[0056] Anomaly detection submodule: The HDM-Detect model combines the Transformer architecture of DETR and the real-time performance of the YOLO series to perform abnormal area detection. DETR is an efficient real-time object detection system that can simultaneously predict the location and category of multiple objects in the entire image. This algorithm has significant speed advantages over traditional sliding window and region proposal methods. In the present invention, DETR is optimized to adapt to the characteristics of medical images, such as enhancing its detection capabilities for small objects and fuzzy boundaries. Implementation method: The preprocessed image is input into the HDM-Detect model (ie, optimized DETR), and the model outputs the location and category of the abnormal area. Advantages: HDM-Detect can provide high detection accuracy while maintaining high speed. The core formula is:

[0057]

[0058] Among them, Q, K, and V are query, key, and value matrices respectively. k is the dimension of the key.

[0059] HDM-Detect is formally expressed as:

[0060] Detected_Boundaries=σ(HDM-Detect(Preprocessed_Image))

[0061] Among them, σ is the sigmoid function used to convert the output into a probability distribution, and Detected_Boundaries is the boundary of the detected abnormal area.

[0062] Image segmentation submodule: Use the encoder-decoder structure of U-Net and the self-attention mechanism of Transformer to perform high-precision image segmentation. Implementation method: Input the abnormal area detected by HDM-Detect into HDM-Segment (i.e. Mask2Former model) to generate a high-precision segmentation mask. Advantages: The Mask2Former model captures long-distance dependencies through the Transformer architecture, effectively handles objects of various sizes and maintains the clarity and accuracy of boundaries. Image segmentation is formally expressed as:

[0063] Segmented_Mask=U-Net(HDM-Detected_Regions)

[0064] Among them, Segmented_Mask is the mask obtained by segmentation, and U-Net represents the forward propagation of the U-Net structure.

[0065] Feature extraction submodule: It is used to extract features of abnormal areas and provide data for subsequent analysis, including using the features of abnormal areas obtained by the feature extraction submodule for further processing and analysis through deep learning diagnostic models to generate accurate diagnostic recommendations. This includes disease classification, quantitative analysis, pathology information integration, personalized treatment plan formulation, prognosis assessment, and data mining, etc., aiming to provide doctors with comprehensive and reliable diagnostic support and assist in making treatment decisions.

[0066] 3. Diagnosis suggestion module:

[0067] Deep learning diagnostic model: Combines medical knowledge and pathological information to provide deep learning-based image diagnostic suggestions. Implementation method: Use a large amount of labeled medical image data to train the model, and the model output is the possible disease category and probability. Advantages: Combines the high accuracy of the deep learning model with the rich information of the knowledge base to improve the accuracy and credibility of the diagnosis. The diagnostic suggestion can be formally expressed as:

[0068] Diagnosis_Suggestion=softmax(DL_Model(Segmented_Mask))

[0069] Among them, Diagnosis_Suggestion is the diagnosis suggestion and DL_Model is the deep learning diagnosis model.

[0070] 4. Report generation module:

[0071] Report Editor: Automatically generate detailed diagnostic reports based on analysis results.

[0072] Report review system: Doctors can review and modify reports to ensure their accuracy.

[0073] 5. User interface module:

[0074] Physician interface: Intuitive user interface where doctors can view images, diagnostic suggestions and reports.

[0075] Patient Information Management System: Manage patient information and historical diagnosis records.

[0076] 6. System management module:

[0077] System Monitoring: Monitor system performance and stability.

[0078] Data Management: Manage stored image data and diagnostic records.

[0079] The lightweight optimization strategy adopted by the present invention is introduced below. The functional structure of the lightweight optimization strategy can be found in Fig. 9 As shown:

[0080] The present invention adopts a series of lightweight optimization strategies to lightweight the HDM model, including model quantization, parameter pruning, knowledge distillation, low-rank decomposition, lightweight network architecture, quantization-aware training, and model compression and acceleration. Among them:

[0081] Model quantization: This invention uses quantization technology to convert model parameters from 32-bit floating point numbers to 8-bit integers, significantly reducing the model size and the amount of calculation. The quantization formula is: Among them, B is the quantized value, R is the original floating point value, Z is the zero point (Zero Point), and S is the scale factor (Scale).

[0082] Parameter pruning: This paper uses structured pruning technology to remove unimportant weights in the neural network, thereby reducing the complexity of the model. The pruning formula is: W pruned =W⊙M where W is the original weight and M is a binary mask indicating which weights are retained.

[0083] Knowledge distillation: This paper uses knowledge distillation technology to train a small "student" model to imitate the behavior of a large "teacher" model, thereby reducing the model size while retaining the knowledge of the teacher model. The knowledge distillation formula is: L distill =λL student +(1-λ)L teacher Among them, L student and L teacher are the loss functions of the student model and the teacher model respectively, and λ is the balance parameter.

[0084] From the description of the above embodiments, those skilled in the art can know that the present invention provides a medical image analysis system based on a hybrid detection mask large model. The system first uses an abnormality detection module to quickly identify key objects in medical images, such as tumors, lesions, etc. This module can process images from different medical imaging devices and quickly output the location and preliminary identification information of the object. Next, the segmentation module accurately segments these identified objects and uses image processing technology to carefully depict the precise edges and related details of each object. The segmentation results are used for data analysis, and the possible nature and pathological state of the object are analyzed using a deep learning algorithm in combination with the patient's historical medical records and real-time data. Finally, the system generates a detailed diagnostic report, including image analysis results, diagnostic recommendations and their basis. This report will be displayed in the user interface and can be used by doctors for further analysis and decision-making.

[0085] The present invention constructs a medical image analysis system through an innovative hybrid detection mask large model (HDM), solving the problems of model generalization, real-time performance and interpretability in existing medical image analysis technologies. Specifically, the present invention has the following advantages:

[0086] Improving model generalization ability: The HDM model adopted in the present invention improves the adaptability and generalization ability of the model to images from different medical imaging devices by combining multiple deep learning technologies, such as DETR and Mask2Former.

[0087] Meeting real-time requirements: The lightweight optimization strategy of the present invention, including model quantization and parameter pruning, significantly reduces the computational complexity and response time of the model, meeting the needs of real-time diagnosis.

[0088] Improve model interpretability: The present invention provides an intuitive visualization of AI working logic, enabling medical professionals to easily understand the decision logic of the AI ​​model.

[0089] Reduce hardware resource requirements: Through lightweight network architecture and quantization-aware training, the HDM model of the present invention can run on low-power, low-latency devices, reducing dependence on high-end computing resources.

[0090] Improved diagnostic accuracy and efficiency: The HDM model of the present invention improves the accuracy and efficiency of medical image diagnosis through precise image segmentation and deep learning diagnostic suggestions.

[0091] Easy to promote and use: The system of the present invention is easy to operate, making the system suitable not only for professional medical environments, but also for resource-constrained areas and mobile devices.

[0092] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as software systems or computer-related products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components, and by means of a suitably programmed computer.

[0094] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0095] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image analysis system based on a hybrid detection mask large model, characterized in that: The medical image analysis system includes: a data input module, an image processing and analysis module, and a diagnosis suggestion module; wherein: The data input module includes: an image acquisition interface and a data preprocessing unit, wherein: the image acquisition interface acquires medical image data from the medical imaging device in real time; the data preprocessing unit uses the image processing library to perform data preprocessing on the acquired medical image data; The image processing and analysis module is a hybrid detection mask large model, including: an abnormality detection submodule, an image segmentation submodule and a feature extraction submodule, wherein: the abnormality detection submodule uses the DETR model to identify and detect abnormal areas of preprocessed medical images; the image segmentation submodule uses the Mask2Former model to accurately segment the abnormal areas identified by the abnormality detection submodule to generate segmentation masks; the feature extraction submodule is used to extract features of abnormal areas; The diagnosis suggestion module uses a deep learning diagnosis model to process the output results of the hybrid detection mask large model and outputs diagnosis suggestions.

2. A medical image analysis system based on a hybrid detection mask large model according to claim 1, characterized in that: The image acquisition interface acquires medical image data from medical imaging equipment in real time, and the acquired medical image data includes: X-ray images, MRI images, CT images and ultrasonic images.

3. A medical image analysis system based on a hybrid detection mask large model according to claim 1, characterized in that: The hybrid detection mask large model adopts a lightweight optimization strategy for model optimization; the lightweight optimization strategy includes: model quantization, parameter pruning, knowledge distillation, low-rank decomposition, lightweight network architecture, quantization-aware training, model compression and acceleration.

4. A medical image analysis system based on a hybrid detection mask large model according to claim 3, characterized in that: The hybrid detection mask model also introduces a graph attention network module to refine the edges of the detected abnormal areas.

5. A medical image analysis system based on a hybrid detection mask large model according to claim 1, characterized in that: In the diagnosis suggestion module, annotated medical image data is used to train a deep learning diagnosis model, and the deep learning diagnosis model combines medical knowledge and pathological information to provide image diagnosis suggestions.

6. A medical image analysis system based on a hybrid detection mask large model according to claim 1, characterized in that: The medical image analysis system also includes: a report generation module, which includes a report editor and a report review system, wherein: the report editor is used to automatically generate a diagnosis report based on the diagnosis advice output by the diagnosis advice module; the report review system is used to provide doctors with the function of reviewing and modifying reports.

7. A medical image analysis system based on a hybrid detection mask large model according to claim 6, characterized in that: The medical image analysis system also includes: a user interface module, which includes: a doctor operation interface and a patient information management system, wherein: the doctor operation interface is used to provide doctors with functions of viewing images, diagnosis suggestions and reports; the patient information management system is used to manage patient information and historical diagnosis records.

8. A medical image analysis system based on a hybrid detection mask large model according to claim 7, characterized in that: The medical image analysis system also includes: a system management module, which includes: a system monitoring module and a data management module, wherein: the system monitoring module is used to monitor system performance and stability; the data management module is used to manage stored image data and diagnosis records.

Images