Medical image analysis system

Abstract

A medical image analysis system includes an image processing module, a three-dimensional modeling module, a pre-training module, and a comparative statistical analysis module. The image processing module is configured to convert two-dimensional (2D) grayscale medical image information into NIfTI format and, based on the type of the 2D grayscale medical image information, perform corresponding normalization, data augmentation, and resampling to generate standardized modeling image information. The three-dimensional modeling module is configured to use a deep learning model to perform segmentation of organ structures in the standardized modeling image information according to definitions of functional anatomy, and to label and color the segmented organ structures, thereby generating a color three-dimensional model file. The pre-training module is configured to perform data augmentation and resampling for classification tasks based on nnU-Net, nnTrans, or SwinUNETR, and to build a lesion recognition deep learning model for each individual organ structure. The comparative statistical analysis module is configured to import the color three-dimensional model file into the lesion recognition deep learning model, mark positional information in the color three-dimensional model file according to extracted relevant organ structures and statistical analysis calculations, and generate an analysis report.

Description

Technical Field

[0001] A medical image analysis system, particularly a medical image analysis system that uses deep learning technology to generate color-marked three-dimensional models of medical images and further produces clinical auxiliary reports. Prior Technology

[0002] This invention claims priority to U.S. Provisional Case Nos. 63714875, 63714876, 63714878, and 63714879.

[0003] Traditional medical imaging, such as computed tomography (CT) or magnetic resonance imaging (MRI), primarily presents images in two-dimensional (2D) slices. While 2D images offer basic diagnostic capabilities, they still have many limitations, making it difficult to meet the high standards of precision medicine and surgical guidance required in modern clinical practice. Firstly, 2D images can only present information from specific sections, lacking an intuitive understanding of the overall three-dimensional structure of organs or lesions. Medical personnel must repeatedly compare and visualize multiple consecutive slices to infer the spatial relationships between tissues, increasing the interpretive burden and the risk of misdiagnosis.

[0004] Secondly, two-dimensional images cannot provide free adjustment of the viewing angle. For lesions located in complex anatomical areas (such as the skull base, pelvic cavity, and vascular plexus), the limited section plane often leads to incomplete information, which in turn affects clinical judgment. In addition, the lack of three-dimensionality in two-dimensional images is also not conducive to communication with patients or preoperative simulation, reducing their convenience for clinical and teaching applications.

[0005] Therefore, how to develop a medical image analysis system that can solve the above problems has become a problem to be solved in this technical field. Summary of the Invention

[0006] To address the aforementioned problems, according to one embodiment, the present invention provides a medical image analysis system, comprising multiple hardware modules consisting of electrically connected hardware circuits, which are signal-connected to a hospital host. The multiple hardware modules include an image processing module, a 3D modeling module, a pre-training module, and a comparative statistical analysis module.

[0007] The aforementioned image processing module receives multiple 2D black-and-white medical images from the hospital's mainframe, converts the image format of these images to NIfTI format, performs standardization processing according to the type of the images, and then performs data augmentation and resampling to generate a standard modeling image. The aforementioned 3D modeling module, communicatively connected to the image processing module, uses a deep learning model to partition the standard modeling image into organ structures based on the definition of functional anatomy. It then labels and colors the partitioned organ structures to generate a color 3D model file. The aforementioned pre-training module obtains multiple three-dimensional lesion training sample files for individual organ structures from the hospital's mainframe. Based on nnUnet, nnTrans, or SwinUNETR classification tasks, it performs data augmentation and resampling to establish a deep learning model for lesion identification in each organ structure. The multiple three-dimensional lesion sample files contain more than 1000 elements and have multiple lesion feature voxels, which are used for labeling and calculating the median of all the multiple three-dimensional lesion sample files during the resampling. The aforementioned comparative statistical analysis module imports the aforementioned color three-dimensional model files into the lesion identification deep learning model. Based on the extracted relevant organ structures and statistical analysis calculations, it marks the location information in the color three-dimensional model files and generates an analysis report.

[0008] According to this embodiment, preferably, the lesion identification method is based on adding a classification task to NNUNET. Currently, the classification task focuses on distinguishing whether the image is normal or abnormal, and it can be extended to multi-category classification.

[0009] According to this embodiment, preferably, when the above-mentioned 2D black and white medical image information is magnetic resonance imaging (MRI) image, the image processing module performs Z-score standardization processing on the MRI image.

[0010] According to this embodiment, preferably, when the above-mentioned 2D black and white medical image information is a computed tomography (CT) scan image, the standardization processing performed by the image processing module on the CT scan image is in the following order: collecting all samples, removing extreme values, and Z-score.

[0011] According to this embodiment, preferably, the image format of the above-mentioned 2D black and white medical image information is DICOM format.

[0012] According to this embodiment, preferably, the number of images in the above-mentioned 2D black and white medical image information used for conversion is greater than 10.

[0013] According to this embodiment, preferably, the minimum area that can be treated by the above-mentioned lesion feature voxel is 2 cm by 2 cm.

[0014] According to this embodiment, preferably, the above-mentioned medical image analysis system further includes a deep learning training database, which is communicatively connected to the image processing module and the 3D modeling module. It receives and stores the standard modeling image information from the image processing module and receives and stores the corresponding color 3D model files from the 3D modeling module, providing the training data set and validation data set required by the deep learning model.

[0015] According to this embodiment, preferably, the above-mentioned color three-dimensional model file is in stereolithography format.

[0016] According to this embodiment, preferably, the above-mentioned analysis report is generated by using a natural language model to generate the color three-dimensional model file and the marked location information.

[0017] The claimed benefits of this invention include: (1) Automatically performing organ structure partitioning and lesion detection for medical images, using a deep learning model to analyze and semantically segment the images, quickly generating anatomically corresponding three-dimensional models, and further distinguishing and representing different organ regions and lesions using color marking, thereby improving clinical interpretability, visual recognition, and diagnostic communication efficiency. (2) Generating auxiliary reports with structural information and statistical features, allowing physicians to quickly grasp the location, shape, volume, and spatial relationship of lesions with adjacent organs, serving as a basis for surgical planning and treatment. Simple Explanation of the Diagram

[0018] To make the above-described techniques and other objects, features, advantages, and embodiments of the present invention more apparent and understandable, the accompanying drawings are described below: Figure 1 is a schematic diagram of the architecture of a medical image analysis system according to one embodiment of the present invention. Figure 2 is a schematic diagram of a medical image according to one embodiment of the present invention. Figure 3 is a schematic diagram of the colored three-dimensional model made according to Figure 2. Figure 4 shows a three-dimensional lesion training sample file of one embodiment of the present invention. Figure 5 is a schematic diagram showing the analysis report of one embodiment of the present invention displayed on a medical image analysis device. Figure 6 is a flowchart illustrating a medical image analysis system according to one embodiment of the present invention. Implementation

[0019] To illustrate the various embodiments of the present invention in more detail, the following description is provided with reference to the accompanying drawings. It should be understood that when an element is referred to as "connected" or "disposed" on another element, it may indicate that the element is directly located on the other element, or that there may also be an intermediate element connecting the element to the other element. Conversely, when an element is referred to as "directly on another element" or "directly connected to another element," it is understood that this explicitly defines the absence of an intermediate element.

[0020] Referring to Figure 1, according to one embodiment, the present invention provides a medical image analysis system 100. Specifically, after converting the medical image format from DICOM to NIfTI format, the system performs preprocessing on the medical images. Then, a deep learning model 122 is used to partition the preprocessed image data into organ structures 124 and identify lesions. Furthermore, a color three-dimensional model is generated for physicians to use in surgical planning, disease interpretation, and clinical diagnosis. In addition, the medical image analysis system 100 also analyzes the color three-dimensional model to extract the spatial location and statistical characteristics of organs and lesions, generating an analysis report 144 to assist physicians in clinical decision-making.

[0021] The medical image analysis system 100 includes multiple hardware modules consisting of multiple electrically connected hardware circuits, which are signal-connected to a hospital host 160. The multiple hardware modules include an image processing module 110, a 3D modeling module 120, a pre-training module 130, a comparative statistical analysis module 140, and a deep learning training database 150.

[0022] Image processing module 110 receives 2D black-and-white medical image information 112 transmitted from hospital host 160. First, image processing module 110 converts the received 2D black-and-white medical image information 112 from its original DICOM format to NIfTI format. This format conversion helps to standardize image data formats, facilitating subsequent data processing and the application of deep learning model 122. After conversion, image processing module 110 further determines whether the received 2D black-and-white medical image information 112 is a computed tomography (CT) scan or magnetic resonance imaging (MRI). Depending on the image type, image processing module 110 performs corresponding standardization processing. For example, for CT images, it collects all samples, removes extreme values ​​and Z-scores, and for MRI images, it uses Z-score standardization to reduce inter-image differences. The number of images in the 2D black-and-white medical image information 112 to be converted needs to be greater than 10.

[0023] Furthermore, to improve the generalization ability of the deep learning model 122 and avoid overfitting, the image processing module 110 performs data augmentation techniques on the 2D black-and-white medical image information 112, including random rotation, scaling, deformation, mirror reversal, and adding Gaussian noise. In addition, to ensure the consistency of spatial resolution of all images, the image processing module 110 performs a resampling operation, standardizing the images according to the median of the voxel resolution in the training data.

[0024] Finally, the image processing module 110 generates standard modeling image information 114 that conforms to the specifications. This standard modeling image information 114 not only has high consistency and repeatability, but is also optimized for subsequent construction of three-dimensional models and deep learning analysis, thereby improving the accuracy and efficiency of medical image analysis.

[0025] Please refer to Figures 2 and 3. Figure 2 is a schematic diagram of a medical image according to one embodiment of the present invention. Figure 3 is a schematic diagram of a color three-dimensional model made according to Figure 2. Figure 2 is a medical image obtained from a full abdominal scan, as described in U.S. Provisional Application No. 63714879; Figure 3 is a three-dimensional model generated by three-dimensional modeling, i.e., identification and marking, based on the obtained medical image, as described in U.S. Provisional Application No. 63714879. The three-dimensional modeling module 120 is connected to the image processing module 110 and is used to receive preprocessed standard modeling image information 114, and to perform three-dimensional reconstruction and medical structural identification on this information through a deep learning model 122. The deep learning model 122 is nnU-Net, nnTrans, or SwinUNETR. nnU-Net, nnTrans, or SwinUNETR are medical image segmentation architectures with automated structure adjustment capabilities. They can automatically adjust the network layers and training parameters based on the spatial resolution, number of image channels, and anatomical features of the input image to optimize model performance.

[0026] The deep learning model 122 can analyze image content based on functional anatomical definitions (such as Couinaud liver lobe partitions and lung / bronchial partitions) to automatically partition organ structures 124. After partitioning and identification, the 3D modeling module 120 further visualizes the identified regions, assigning specific color markers to different organs to form a clinically visible color 3D model file. This color 3D model file is output in .stl 3D graphics format and can be viewed by physicians using software that supports 3D model operations, including rotation, cutting, and cross-sectional browsing, for diagnosis, surgical planning, and patient communication.

[0027] The organ structure includes the following 177 parts: spleen, right kidney, left kidney, gallbladder, liver, stomach, pancreas, right adrenal gland, left adrenal gland, left upper lobe of lung, left lower lobe of lung, right upper lobe of lung, right middle lobe of lung, right lower lobe of lung, esophagus, trachea, thyroid gland, small bowel, duodenum, colon, urinary bladder, prostate, left kidney cyst, right kidney cyst, sacrum, S1 vertebra, L5 vertebra, L4 vertebra, L3 vertebra, L2 vertebra, L1 vertebra, T12 vertebra, T11 vertebra, T10 vertebra, T9 vertebra, T8 vertebra, T7 vertebra, T6 vertebra, T5 vertebra, T4 vertebra, T3 vertebra, T2 vertebra, T1 vertebra, C7 vertebra, C6 vertebra, C5 vertebra, C4 vertebra, C3 vertebra, C2 vertebra, C1 vertebra, heart, aorta, pulmonary vein, brachiocephalic trunk, right subclavian artery, left subclavian artery, right common carotid artery, left common carotid artery, left brachiocephalic vein, right brachiocephalic vein, left atrial appendage, superior vena cava, inferior vena cava, portal vein and splenic vein, left iliac artery,iliac_artery_right、iliac_vena_left、iliac_vena_right、humerus_left、humerus_right、scapula_left、scapula_right、clavicula_left、clavicula_right、femur_left、femur_right、hip_left、hip_right、spinal_cord、gluteus_maximus_left、gluteus_maximus_right、gluteus_medius_left、gluteus_medius_right、gluteus_minimus_left、gluteus_minimus_right、autochthon_left、autochthon_right、iliopsoas_left、iliopsoas_right、brain、skull、rib_left_1、rib_left_2、rib_left_3、rib_left_4、rib_left_5、rib_left_6、rib_left_7、rib_left_8、rib_left_9、rib_left_10、rib_left_11、rib_left_12、rib_right_1、rib_right_2、rib_right_3、rib_right_4、rib_right_5、rib_right_6、rib_right_7、rib_right_8、rib_right_9、rib_right_10、rib_right_11、rib_right_12、sternum、costal_cartilages。 、

[0028] According to U.S. Provisional Application No. 63714879, the 3D modeling module 120 has the ability to automatically identify various parts, organs, and tissues of the human body. Through a layered design, it groups and classifies the identification results, allowing each organ and structure to be presented in an independent layer according to its functional anatomical framework. This layered design not only improves data organization and browsing efficiency but also enables subsequent clinical applications or model output to select and manipulate specific organs according to user needs, further supporting the conversion of 3D medical image models covering all parts of the body. The 3D modeling module 120 can automatically identify and convert major anatomical regions such as the brain, thoracic cavity, abdominal cavity, and pelvic cavity based on the areas displayed in the input image, and supports automatic regional modeling of several clinically common organs such as the liver, kidneys, pancreas, spleen, heart, lungs, and brain. In addition to organ identification, it can also perform more detailed classification of organs based on information such as specific features or tissue density changes, further identifying tissue blocks and even cell clusters. The 3D modeling module 120 can perform marking and high-contrast coloring to help doctors identify potential lesion areas and improve the efficiency and interpretability of clinical diagnosis.

[0029] Furthermore, according to U.S. Provisional Application No. 63714879, the deep learning model 122 of the 3D modeling module 120 is developed and trained based on nnUNet. nnUNet (no-new-Net) is an automated medical image segmentation architecture with self-adjusting capabilities in model structure, preprocessing procedures, and post-processing mechanisms, and can automatically configure network parameters and training strategies according to different image data. The training data of this invention comes from 2,500 CT and MRI image cases covering different body parts, all of which are common DICOM format medical image data in clinical practice.

[0030] To ensure the model's accuracy for clinical application and anatomical precision, all training data is manually annotated, with precise contour depictions of organs and structures in each region. After initial model training, the 3D modeling module 120 outputs predicted segmentation results, which are then interpreted and error-corrected by clinicians. The corrected results and tuning parameters are then re-imported into the model for iterative training to gradually improve its accuracy and clinical applicability.

[0031] Please refer to Figure 4, which illustrates a three-dimensional lesion training sample file according to one embodiment of the present invention. The pre-training module 130 of the present invention is designed to execute the training process of the deep learning model 122. It further processes and trains the model by retrieving and aggregating multiple three-dimensional lesion training sample files 132 corresponding to the structures 124 of different organs from the aforementioned hospital host computer 160. The number of training samples must be at least 1000, and each sample contains a representative lesion feature voxel 136 to improve the model's generalization ability and recognition accuracy.

[0032] In implementation, the pre-training module 130 performs data augmentation and resampling procedures required for the classification task based on deep learning architectures such as nnUNet, nnTrans, or SwinUNETR. Data augmentation procedures may include operations such as rotation, scaling, flipping, and contrast adjustment to simulate different clinical conditions and increase sample diversity. The resampling process standardizes the lesion feature voxels 136 of all lesion samples and calculates the median of all training samples as the resampling standard during integration to reduce the impact of spatial resolution differences on model training. The lesion feature voxels 136 may contain various quantifiable parameters to describe the morphology, density, and location characteristics of the lesions. For example, lesion volume reflects its total volume in a three-dimensional model and is expressed in mm³ or cm³; Hounsfield Units (HU) are used to quantify the density of lesion tissue in CT scans, with higher values ​​indicating greater tissue density; maximum diameter and axial diameter can be used to describe the size of the lesion in different directions, while surface area reflects the three-dimensional surface range of the lesion. The shape characteristics of the lesion can also be quantified, including roundness or irregularity to assess the regularity of the lesion's shape; texture or grain characteristics are used to describe the homogeneity or heterogeneity of the internal structure of the lesion; density distribution reflects the distribution of different HU values ​​within the lesion; and boundary features are used to determine the clarity of the lesion's boundaries.

[0033] For example, the lesion characteristic voxel 136 of liver tumor samples has an average CT value ranging from approximately 56 HU to 69 HU, an average volume ranging from approximately 36847 mm³ to 45036 mm³, and an average axial diameter ranging from approximately 2.97 cm to 3.63 cm. The lesion characteristic voxel 136 of colorectal tumor samples has an average CT value ranging from approximately 43 HU to 53 HU, an average volume ranging from approximately 39598 mm³ to 48397 mm³, and an average axial diameter ranging from approximately 3.69 cm to 4.51 cm. The definition of these multiple characteristic parameters can serve as a screening criterion for the deep learning model 122 during the training and inference stages, helping to exclude clinically irrelevant noise voxels and further improving the accuracy and stability of lesion segmentation and identification. The lesion characteristic voxel 136 of sarcopenia mainly represents muscle mass, used to describe the quantitative characteristics of the patient's muscle groups. Muscle loss can be defined by clinical criteria. When a man's muscle mass is less than 7.0 kg / m², or a woman's is less than 5.4 kg / m² (measured by dual-energy X-ray absorptiometry, DXA), he is considered to have insufficient muscle mass.

[0034] The minimum area that lesion feature voxels 136 can process is 2 cm by 2 cm. Taking tumor samples as an example, the average axial diameter of liver tumor experimental cases is approximately between 2.97 cm and 3.63 cm. The average axial diameter of colorectal tumor experimental cases is approximately between 3.69 cm and 4.51 cm. All of these results are greater than the set minimum area limit, indicating that the pre-training module 130 and the deep learning model 122 can effectively process lesion feature voxels within this range. By setting this minimum area limit, excessively small or clinically irrelevant noise areas can be excluded, further improving the accuracy and stability of lesion detection and segmentation.

[0035] Through the aforementioned method, the pre-training module 130 can establish highly recognizable deep learning models 134 for lesion identification for different organ structures 124, serving as the basis for 3D modeling. This implementation not only improves training efficiency and model stability but also effectively enhances the accuracy and reliability in clinical applications.

[0036] After multiple rounds of training and model optimization, the deep learning model 122 of this invention achieves an overall segmentation accuracy of over 85% on the validation dataset. This deep learning model 122 possesses stable and reliable 3D modeling and lesion identification capabilities, and can serve as the basis for subsequent color labeling, visualization modeling, and medical report generation.

[0037] The comparative statistical analysis module 140 is used to further enhance the clinical application value of lesion identification results. During implementation, the comparative statistical analysis module 140 will first receive and import the color three-dimensional model file produced by the aforementioned deep learning modeling and visualization processing, and input the file into the lesion identification deep learning model 134 previously established by the pre-training module 130 for further feature comparison and verification.

[0038] In the lesion identification method disclosed in this invention, the core technology used is a deep learning model 122 developed based on the nnU-Net architecture. A classification task is added to the deep learning model 122 to enhance the ability to distinguish lesion abnormalities, so as to determine whether the overall input image is "normal" or "abnormal".

[0039] Through the comparison procedure, the comparative statistical analysis module 140 can effectively extract features and perform statistical analysis on the organ structures 124 and lesion areas in the color three-dimensional model, including but not limited to: lesion location (such as the coordinates or slice location relative to the organ), volume (calculated in Voxel units), boundary clarity, density distribution, and relative distance to adjacent anatomical structures.

[0040] The comparative statistical analysis module 140 transforms the above analysis results into a structured data format and visualizes them in a color 3D model with dynamic markings and location information 142. Finally, the module automatically generates a complete analysis report 144, which may include a structural classification summary, quantitative information on lesions, and clinical reference annotations, providing clinicians with further information for disease interpretation, surgical evaluation, and treatment planning. This process significantly reduces the manpower required for traditional analysis and improves diagnostic efficiency and consistency.

[0041] The deep learning training database 150 is connected to the image processing module 110 and the 3D modeling module 120, and is used to collect and store the training and validation data required for the deep learning model 122. This database receives and stores pre-processed standard modeling image information 114 from the image processing module 110, and receives and stores corresponding color 3D model files from the 3D modeling module 120, serving as the basic dataset for training and validating the deep learning model 122.

[0042] Example 1

[0043] The image processing module 110 receives 2D black-and-white medical image information 112 transmitted from the hospital host 160, such as coronary CTA (Computed Tomography Angiography) images obtained after the administration of an angiographic contrast agent. It then converts the image format of the 2D black-and-white medical image information 112 to NIfTI format and performs standardization processing, including collecting all samples, removing extreme values, and applying Z-score. Afterward, the image processing module 110 performs data augmentation and resampling operations on the 2D black-and-white medical image information 112 to generate a standard modeling image information 114. This effectively improves the quality and consistency of the input image, enabling subsequent 3D modeling algorithms to more accurately extract structural contours and key information.

[0044] The 3D modeling module 120 receives standard modeling image information 114 preprocessed by the image processing module 110, and uses a deep learning model 122 based on the nnU-Net architecture to perform 3D reconstruction of the coronary arteries. This deep learning model 122 has automatic configuration and adjustment capabilities, optimizing network parameters based on the characteristics of the input images and defining the coronary artery structure according to functional anatomy. The reconstructed color 3D model (.stl format) clearly presents the overall structure of the heart, distinguishing blood vessels and myocardial tissue with different colors, thus improving visual recognition.

[0045] The comparative statistical analysis module 140 receives the aforementioned color 3D model and integrates it into the previously trained lesion identification deep learning model 134. Based on the organ structures 124 extracted from the color 3D model, the comparative statistical analysis module 140 performs statistical analysis and feature calculations, marking key lesion locations and their spatial distribution. For example, by comparing the characteristics of normal blood vessel structures with abnormal areas in the input model, the comparative statistical analysis module 140 identifies the location of blood vessel blockage and quantifies the extent and impact of the blockage. Finally, a detailed analysis report 144 is generated based on the analysis results, providing physicians with reference for surgical planning and clinical diagnosis. This analysis report 144 is generated using a natural language model from the color 3D model file and the marked location information 142.

[0046] Example 2

[0047] This embodiment uses the assessment and diagnosis of sarcopenia as an example to illustrate the image processing flow and technical effects of the system of the present invention.

[0048] Image processing module 110 receives 2D black-and-white medical image information of a female subject transmitted from hospital host 160. This 2D black-and-white medical image information refers to medical images acquired by MRI and CT, such as cross-sections of the third lumbar vertebra (L3 segment), femoral muscle groups, and trunk muscle groups such as the psoas major and erector spinae muscles, which are commonly used to assess skeletal muscle volume, muscle thickness, or muscle cross-sectional area. Image processing module 110 converts the image format of the 2D black-and-white medical image information 112 to NIfTI format and performs standardization processing on the 2D black-and-white medical image information 112, including collecting all samples, removing extreme values, and Z-score. Subsequently, image processing module 110 performs data augmentation and resampling operations on the 2D black-and-white medical image information 112 to generate a standard modeling image information 114, which can effectively improve the quality and consistency of the input image, enabling subsequent 3D modeling algorithms to more accurately extract structural contours and key information.

[0049] The 3D modeling module 120 receives standard modeling image information 114 preprocessed by the image processing module 110, and uses a deep learning model 122 based on the nnU-Net architecture to reconstruct the muscles in 3D. This deep learning model 122 has automatic configuration and adjustment capabilities, optimizing network parameters based on the characteristics of the input images and defining the coronary artery structure according to functional anatomy. The reconstructed color 3D model (.stl format) clearly presents the muscle structure, distinguishing different muscle groups with different colors, providing an intuitive visualization of muscle morphology and spatial distribution, which is beneficial for the quantitative analysis and clinical assessment of sarcopenia.

[0050] The comparative statistical analysis module 140 receives the aforementioned color 3D model and integrates it into the previously trained lesion identification deep learning model 134. Based on the muscle structure extracted from the color 3D model, the comparative statistical analysis module 140 performs statistical analysis and feature calculations, including indicators such as muscle volume, muscle thickness, cross-sectional area, and the proportion between muscle groups, to provide quantitative data to support the diagnosis and clinical decision-making of sarcopenia. For example, the comparative statistical analysis module 140 can identify the degree of muscle atrophy, fat infiltration, and imbalances between muscle groups by comparing the subject's muscle structure with baseline data of normal muscles. Finally, a detailed analysis report 144 is generated based on the analysis results, providing physicians with a reference for surgical planning and clinical diagnosis.

[0051] Please refer to Figure 5, which is a schematic diagram of an analysis report displayed on a medical image analysis device according to one embodiment of the present invention. The analysis report 144 is generated by a natural language model using the color three-dimensional model file and the marked location information 142. Simultaneously, the analysis report 144 includes quantitative indicators obtained from comparative statistical analysis, such as muscle volume, muscle thickness, cross-sectional area, and the proportion between muscle groups, to comprehensively present the sarcopenia assessment results and assist clinical decision-making. As shown in Figure 5, the analysis report 144 shows that the examinee's muscle mass is 5.6 kg / m², exceeding the threshold of 5.4 kg / m² for sarcopenia in women. Therefore, it is determined that their muscle mass is within the normal range and they do not have sarcopenia. Furthermore, the analysis report 144 can further display the muscle mass distribution and the relative proportions of each muscle group for clinicians' reference and can be used to track changes in muscle mass or evaluate treatment effects.

[0052] Please refer to Figure 6, which is a flowchart of a medical image analysis system 100 according to one embodiment of the present invention.

[0053] In step 200, after receiving the 2D black and white medical image information 112 transmitted by the hospital host 160, the image processing module 110 converts the image format of the 2D black and white medical image information 112 into NIfTI format.

[0054] In step 201, the image processing module 110 performs corresponding standardization processing according to the type of 2D black and white medical image information 112, and then performs data enhancement and resampling to generate standard modeling image information 114.

[0055] In step 202, the 3D modeling module 120 uses a deep learning model 122 to partition the standard modeling image information 114 into organ structures 124 according to the definition of functional anatomy.

[0056] In step 203, the 3D modeling module 120 marks and colors the organ structures 124 in the partitioned area to generate a colored 3D model file.

[0057] In step 204, the comparative statistical analysis module 140 imports the color three-dimensional model file into the lesion identification deep learning model 134, and marks the location information 142 in the color three-dimensional model file based on the extracted relevant organ structure 124 and statistical analysis calculation, and generates an analysis report 144.

[0058] In summary, this invention enables fully automated processing of medical images. Starting from receiving raw image data, it sequentially completes format conversion, preprocessing, 3D modeling, structural identification, lesion detection, and color marking. Furthermore, it generates visually identifiable 3D models and clinical auxiliary reports for physicians to use for lesion localization, surgical planning, disease description, and clinical diagnosis, effectively improving the practical value and diagnostic efficiency of medical images and reducing the burden of manual operation.

[0059] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0060] The above-described embodiments are merely illustrative examples and are not intended to limit the scope of the present invention. Any equivalent modifications or alterations made to them shall not depart from the spirit and scope of the present invention and shall be included within the scope of the patent application in this case.

[0061] 100: Medical Image Analysis System 110: Image Processing Module 112:2D Black and White Medical Imaging Information 114: Standard Modeling Image Information 120: 3D Modeling Module 122: Deep Learning Model 124: Organ Structure 130: Pre-training module 132: Three-dimensional lesion training sample archive 134: Deep Learning Model for Lesion Identification 136: Characteristic Voxels of Lesions 140: Comparative Statistical Analysis Module 142: Location Information 144: Analysis Report 150: Deep Learning Training Resource Library 160: Hospital Mainframe 200~204: Steps

Claims

1. A medical image analysis system comprising a plurality of hardware modules consisting of a plurality of electrically connected hardware circuits, signal-connected to a hospital host computer, the hardware modules comprising: An image processing module receives multiple 2D black-and-white medical images from the hospital's mainframe, converts the image format of the 2D black-and-white medical images to NIfTI format, performs corresponding standardization processing according to the type of the 2D black-and-white medical images, performs data augmentation and resampling on the 2D black-and-white medical images, and generates a standard modeling image. A 3D modeling module is connected to the image processing module, uses a deep learning model to partition the standard modeling image based on the definition of functional anatomy, and marks and colors the partitioned organ structures to generate a color 3D model file. A pre-training module obtains multiple three-dimensional lesion training sample files of individual organ structures from the hospital's mainframe. Based on nnUnet, nnTrans, or SwinUNETR classification tasks, it performs data augmentation and resampling to establish a lesion identification deep learning model for each organ structure. These three-dimensional lesion sample files contain more than 1000 files and have multiple lesion feature voxels, which are used for labeling and calculating the median of all three-dimensional lesion sample files during the resampling. A comparative statistical analysis module imports the color three-dimensional model file into the lesion identification deep learning model. Based on the extracted relevant organ structures and statistical analysis calculations, it marks the location information in the color three-dimensional model file and generates an analysis report.

2. The medical image analysis system as described in claim 1, wherein the lesion identification method is based on adding a classification task to NNUNET. Currently, the classification task focuses on distinguishing whether the image is normal or abnormal, and can be extended to multi-category classification.

3. The medical image analysis system as described in claim 1, wherein when the 2D black-and-white medical image information is a magnetic resonance imaging (MRI) image, the image processing module performs Z-score normalization processing on the MRI image.

4. The medical image analysis system as described in claim 1, wherein when the 2D black-and-white medical image information is a computed tomography (CT) image, the standardization processing performed by the image processing module on the CT image is, in sequence, collecting all samples, removing extreme values, and Z-score.

5. The medical image analysis system as described in claim 1, wherein the image format of the 2D black-and-white medical image information is DICOM format.

6. The medical image analysis system as described in claim 5, wherein the number of images in the 2D black-and-white medical image information used for conversion is greater than 10.

7. The medical image analysis system as claimed in claim 1, wherein the minimum area that the lesion feature voxel can process is 2 cm by 2 cm.

8. The medical image analysis system as described in claim 1 further includes a deep learning training database, which is communicatively connected to the image processing module and the 3D modeling module, receives and stores the standard modeling image information from the image processing module and receives and stores the corresponding color 3D model file from the 3D modeling module, and provides the training data set and validation data set required by the deep learning model.

9. The medical image analysis system as claimed in claim 1, wherein the color three-dimensional model file is in stereolithography format.

10. The medical image analysis system as claimed in claim 1, wherein the analysis report is generated by using a natural language model to analyze the color three-dimensional model file and the marked location information.

Images