Medical image analysis system based on cross-scale double-head self-attention

Abstract

According to the 3D medical image intelligent analysis system based on artificial intelligence, deep learning and machine vision technologies are fused, and automatic preprocessing, segmentation and three-dimensional visualization of multi-mode images such as CT and MRI are supported. The system core adopts SOTA models such as nnUNet, ResUNet and the like, key areas such as abdominal organs, tumors and the like are automatically identified, and doctors are assisted in accurate diagnosis. The platform is based on a Kubernetes architecture, supports containerized deployment, automatic capacity expansion and fault recovery, and guarantees high availability and performance. The system pays attention to data security, implements encryption and authority control, has an automatic model updating mechanism, and continuously keeps the advancement and accuracy of an algorithm.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an intelligent parsing system for 3D medical image data based on artificial intelligence (AI). With the rapid development of medical technology, the acquisition and analysis of medical image data are playing an increasingly important role in clinical diagnosis. This system, leveraging advanced machine learning and image processing techniques, enables automated parsing and analysis of medical image data, significantly improving the efficiency of medical image data utilization and the accuracy of diagnoses. Technical Background

[0002] In modern medicine, medical imaging technologies such as X-rays, CT scans, and MRIs have become essential tools for disease diagnosis and treatment planning. However, with the surge in the amount of medical imaging data, doctors face immense pressure to process and analyze vast quantities of imaging data. Traditional manual image reading is not only time-consuming and labor-intensive but also susceptible to subjective factors, leading to diagnostic errors. Furthermore, the uneven distribution of medical resources, particularly at primary healthcare facilities, and the lack of specialized imaging specialists, limits the quality and efficiency of medical services.

[0003] In order to solve the above problems, artificial intelligence technology has been introduced into the field of medical image analysis. The application of AI technology can not only improve the speed and accuracy of image data processing, but also assist doctors in discovering difficult-to-detect lesions through pattern recognition and deep learning algorithms, thereby improving the reliability of diagnosis. The 3D medical image data intelligent analysis system of the present invention, such as Figure 1 As shown in the figure, it is based on this background and aims to improve the parsing efficiency and diagnostic quality of medical imaging data through automated and intelligent methods, reduce the workload of doctors, optimize the allocation of medical resources, and promote the modernization and intelligence of medical services.

[0004] By applying the latest deep learning algorithms, such as nnUnet and ResUnet, the system of this invention can automatically segment relevant areas in medical images and perform three-dimensional reconstruction and visualization, allowing doctors to observe lesions and structures from multiple angles and levels. Furthermore, the system uses cloud computing technology to achieve distributed data processing and storage, improving the system's processing power and stability. Through the integrated application of these technologies, the system can provide doctors with faster and more accurate diagnostic support and provide patients with higher-quality medical services. Summary of the Invention

[0005] This invention relates to an intelligent medical image analysis system, specifically an artificial intelligence-based intelligent analysis system for 3D medical image data. The system aims to improve the efficiency and accuracy of medical image data processing, reduce physician workload, and enhance the quality of diagnosis and treatment. This system leverages deep learning and machine vision technologies to intelligently analyze and automatically process multimodal medical image data, such as CT and MRI. The system includes functions such as data preprocessing, automatic segmentation, and visualization, enabling physicians to make more accurate diagnoses and treatment plans.

[0006] Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solutions:

[0008] 1. Deep Learning Models: This system utilizes advanced deep learning models, including but not limited to state-of-the-art (SOTA) machine vision models such as nnUnet and ResUnet. These models are capable of intelligently analyzing multimodal medical imaging data, such as CT and MRI. Trained with extensive imaging data, these models can automatically identify and segment relevant areas in medical images, such as abdominal organs and tumors.

[0009] 2. Data preprocessing module: Before the deep learning model processes the image data, the data preprocessing module standardizes the input medical image data, including but not limited to normalization operations and image enhancement, to improve the model's generalization ability and processing efficiency.

[0010] 3. Image Segmentation Module: This module is the core of the system and can automatically segment and visualize relevant areas in medical images. For example, it can identify and segment 15 abdominal organs and organ tumors, providing doctors with intuitive image analysis results.

[0011] 4. User Interface: The system features a user-friendly interface, using a client-server architecture. The client uses ITK for visualization, while the server handles data augmentation and model inference. The user interface allows doctors to easily view segmentation results and conduct further analysis and diagnosis.

[0012] 5. Server-Side Architecture: On the server side, this system utilizes the cloud computing industry's leading Kubernetes distributed architecture, with containerized deployment of the relevant inference engines. This architecture enables automatic capacity expansion and disaster recovery, ensuring high performance and availability to meet the needs of medical institutions of varying sizes.

[0013] 6. Security and privacy protection: This system focuses on data security and privacy protection, and adopts strict data encryption and access control measures to ensure the safe transmission and storage of medical imaging data and protect patient privacy.

[0014] 7. System Updates and Maintenance: The system is designed with an automatic update mechanism that can promptly update the latest deep learning models and algorithms to ensure the system's advancement and accuracy. At the same time, the system provides a convenient maintenance interface to facilitate technicians to perform daily maintenance and troubleshooting.

[0015] Quality Control

[0016] To ensure the smooth operation of the Vision Medical Image Intelligent Analysis System, establish a high-quality brand image, and achieve positive market response, quality control is essential. We will explain the quality control aspects of this project from three perspectives: model accuracy, operational status monitoring, and fault monitoring and self-healing capabilities.

[0017] 1. Model accuracy monitoring indicators and improvement measures

[0018] In terms of segmentation accuracy, we use the Iou Score and Dice Score to evaluate the model's segmentation results. These values are used to determine the model's generalization ability, that is, its quality and accuracy in actual operations. Both the Iou and Dice coefficients measure the similarity between two sets. In the field of image segmentation, they are used to measure the similarity between the network segmentation results and the gold standard mask.

[0019] 1)IOU Score(Intersection over Union)

[0020] from Figure 2 As can be seen from the formula, the greater the overlap between the actual object area and the inferred area, the larger the IoU value. In other words, the larger the IoU value, the more accurate the inferred object area. The maximum IoU value is 1, at which point the actual object area and the inferred area completely overlap; the minimum IoU value is 0, at which point the actual object area and the inferred area do not overlap at all. IoU is actually a relatively strict evaluation metric; even a slight deviation between the actual area and the inferred area can result in a significantly smaller IoU value.

[0021] 2)Dice Score(dice similarity coefficient)

[0022] Dice Score is a metric function used to evaluate the similarity between two samples. Its value range is between 0 and 1. The larger the value, the more similar it is. The definition of Dice Score is as follows Figure 3 shown.

[0023] 3) Improvement measures: Model pre-training and fine-tuning (Pretrain & Finetune)

[0024] New data is continuously generated during operation. This cleaned data can be added to the model training set, increasing the data volume and improving the model's generalization capabilities. This is achieved by using the existing model as a pre-trained model and initializing the training parameters to increase model stability during training. The network layer of the existing model already extracts a large amount of image feature information. Using the existing model's parameters as the initialization for the new model allows for the reuse of a large amount of underlying feature information.

[0025] If new requirements for the final result of the model arise during use, you can finetune the model by adding an additional layer to the final output layer of the model to adjust the final output of the model (such as changing Figure 4 Fine-tuning can effectively avoid problems that may occur when training from scratch, such as model non-convergence, suboptimal parameters, low accuracy, poor model generalization ability, and overfitting.

[0026] 2. Operation status monitoring

[0027] We use the cloud computing industry's top Kubernetes distributed architecture and develop our own GPU monitoring plug-in to effectively monitor the running status of the AI inference engine and reliably record and visualize the running status of the server. Figure 5 shown.

[0028] There are some components in the Kubernetes cluster that can be used to monitor the running status of the cluster.

[0029] For example:

[0030] metrics-server: It obtains metrics data from Kubelet and cAdvisor, and then provides it to Dashboard, HPA controller, etc.

[0031] kube-state-metrics: It listens to the Kubernetes API server and generates metrics about the state of objects.

[0032] Prometheus: It can obtain core metrics from metrics-server and non-core metrics from other data sources (such as NodeExporter, etc.), and then build a monitoring and alarm system based on them.

[0033] 3. Fault monitoring and self-healing

[0034] 1) Health Check

[0035] Regularly check the status of the application or Pod inside the container to detect failures or crashes in time and automatically restart or rebuild the failed Pod, such as Figure 6 There are three types of health checks:

[0036] livenessProbe: Checks whether the application inside the container is alive and responding to requests.

[0037] readinessProbe: Checks whether the application inside the container is ready and can accept network traffic.

[0038] startupProbe: Checks if the application inside the container is starting up and waits for a period of time before the startup is complete.

[0039] 2) Pod and ReplicaSet

[0040] The ReplicaSet component will continuously check the feasibility of the Pods in the cluster. If a Pod fails or is terminated, the ReplicaSet will automatically start a new Pod to replace it. This ensures that the container application is always available at runtime.

[0041] 3) Controller and Failover

[0042] In a cluster, the controller is responsible for managing Pods and ReplicaSets, such as Deployment, StatefulSet, and DaemonSet.

[0043] The controller can monitor the status of Pods and ReplicaSets and perform failover or re-creation as needed. For example, if a Pod fails or is deleted, the Deployment controller can automatically create a new Pod and ensure that the application remains available at runtime.

[0044] In addition, the controller can also use the rolling deployment feature to ensure that there is no service interruption when updating the application. It switches the pods of the new version based on availability and load balancing policies to ensure that there are no failures during the application upgrade.

[0045] 4) Self-healing ability

[0046] Automatic rolling upgrades: This scheduling method updates all containers gradually to avoid momentary service interruptions and failures. A rolling update starts with the new version of the application container and then gradually stops the old version containers until all containers are updated.

[0047] Automatic scaling: The system automatically adjusts the number of replicas based on application load to ensure system availability. When the load becomes very high, it automatically increases the number of replicas; when the load becomes very low, it automatically decreases the number of replicas. This adaptive scaling mechanism ensures system stability and availability.

[0048] Automatic fault tolerance: By restarting Pods, containers, and nodes, you can ensure that the application can quickly recover to normal state when a failure occurs.

[0049] Automatic Configuration Updates: This system automatically updates application configurations, ensuring they have the latest configuration at runtime. This update is recursive and safe, ensuring all pods are successfully started without interrupting or losing any requests.

[0050] Automatic repair: The system has self-healing mechanisms that can automatically detect and repair failures or anomalies in Pods. These mechanisms include liveness and readiness probes, Pod health checks, etc.

[0051] When the system detects a container failure within a Pod, it automatically restarts the container through a self-healing mechanism to restore as many containers as possible to normal operation. If recovery is not possible, the entire Pod instance is killed. This mechanism eliminates the need for manual intervention by operations personnel, further enhancing automation.

[0052] Finally, starting from these three aspects, we collected user feedback and observed market reactions to ensure that the three quality control methods could achieve a high level of customer satisfaction. We even reached cooperation with hospitals to jointly explore deeper quality control methods based on user needs and opinions, so as to pursue higher segmentation accuracy, more stable operation status and stronger self-healing ability of the system.

[0053] Innovation

[0054] The algorithm models are diverse and the technical level is comprehensive. The AI-based intelligent analysis system for 3D medical imaging data uses state-of-the-art machine vision models such as nnUnet and ResUnet. It can intelligently analyze multimodal medical imaging data such as CT and MRI, automatically segment relevant areas, and visualize them. The algorithm of this project is partially based on the improved nnUnet. The improved nnUnet abandons the complex network structure and chooses the basic Unet as the basic structure. Compared with various previous UNet improvements, nnUNet focuses more on image preprocessing, can automatically determine the image modality and perform corresponding normalization operations, and resample different image voxel spacings based on cubic spline interpolation. This makes it extremely versatile.

[0055] With diverse functions and complete application performance, the project includes multiple functions such as basic medical image views, organ 3D imaging and segmentation, and organ quantitative analysis, which can effectively meet the needs of doctors in reading medical images, reduce their workload, and effectively improve diagnostic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] (1) Figure 1 This is the system interface diagram

[0057] (2) Figure 2 It is the IOU Score graph

[0058] (3) Figure 3 Dice Score chart

[0059] (4) Figure 4 Model improvement process diagram

[0060] (5) Figure 5 Is a visual server diagram

[0061] (6) Figure 6 It is a fault monitoring diagram.

Claims

1. An artificial intelligence-based intelligent analysis method for 3D medical imaging data, characterized by: The following steps are included: Step 1: Use the visual-semantic convolution module to process the input medical image to generate short-range features and long-range features. The short-range features focus on the image details, while the long-range features capture the global context of the image. Step 2: Through the U-shaped cross-attention Transformer structure, a dynamic multi-scale attention mechanism is used to interactively transfer short-range features and long-range features to achieve effective fusion and information sharing between features, thereby enhancing the overall quality of the image; In step three, the interacted features are projected back to the image feature space through a mapping convolution module to obtain an enhanced image, which is better than the original low-light image in terms of brightness, contrast and detail preservation.

2. The intelligent analysis method for medical image data according to claim 1, wherein: The visual-semantic convolution module adopts a depth-wise separable convolution structure to improve the efficiency of feature extraction and reduce computational complexity while maintaining high accuracy, thereby accelerating the forward propagation process of the model.

3. The intelligent analysis method for medical image data according to claim 1, wherein: The U-shaped cross-attention Transformer structure includes a dual multi-head self-attention (DMSA) module, which generates query, key and value vectors by independently mapping the input features, thereby realizing dynamic interaction and information integration between different features.

4. The intelligent analysis method for medical image data according to claim 3, wherein: The dual multi-head self-attention (DMSA) module crosses the key vectors of the two inputs to enhance the connectivity between them, thereby improving the effect of feature fusion and ensuring the naturalness of the enhanced image in terms of details and colors.

5. The intelligent analysis method for medical image data according to claim 1, wherein: The U-shaped cross-attention Transformer structure also includes a cross-scale dual multi-head self-attention (CSDMSA) module, which is specifically used to process cross-layer residual information to reduce the loss of image details and blurring that may occur during the downsampling process.

6. The intelligent analysis method for medical image data according to claim 1, wherein: The mapping convolution module effectively projects the interacted features back into the image feature space by performing a convolution operation on the interacted features to generate an enhanced image that is closer in visual quality to an image taken under normal lighting conditions.

7. The intelligent analysis method for medical image data according to claim 1, wherein: The method also includes an optimization step that combines perceptual loss and Charbonnier loss to optimize the perceptual effect and detail smoothness of image enhancement, ensuring high fidelity and high accuracy when enhancing low-light images.

8. The intelligent analysis method for medical image data according to claim 1, wherein: The method is trained and tested on multiple public datasets, including LOL-v1, LOL-v2, SID, SMID, and SDSD, to verify its applicability and effectiveness in different low-light environments.

9. The intelligent analysis method for medical image data according to claim 1, wherein: The method can effectively process different types of low-light images, including but not limited to traffic scenes taken at night, indoor low-light environments, and images under other complex lighting conditions, thereby improving the performance of computer vision systems in practical applications.

Images