Medical image analysis system

Abstract

A medical image analysis system, comprising an image processing module, a three-dimensional modeling module, a pre-training module, and a comparative statistical analysis module, wherein the image processing module converts the image format of 2D black-and-white medical image information into an NIfTI format, performs corresponding standardization processing on the basis of the type of the 2D black-and-white medical image information, and then performs data augmentation and resampling on the 2D black-and-white medical image information, such that standard modeling image information is generated; the three-dimensional modeling module uses a deep learning model to perform, on the basis of definitions of functional anatomy, organ structure segmentation processing on the standard modeling image information, and labeling and coloring the organ structures obtained from the segmentation, so as to generate a colored three-dimensional model file; the pre-training module performs data augmentation and resampling on the basis of nnUnet, nnTrans or SwinUNETR classification tasks, so as to establish a deep-learning lesion recognition model for individual organ structures; and the comparative statistical analysis module imports the colored three-dimensional model file into the deep-learning lesion recognition model, labels location information in the colored three-dimensional model file on the basis of related extracted organ structures and statistical analysis calculations, and generates an analysis report.

Description

Medical image analysis system Technical Field

[0001] A medical image analysis system, particularly a medical image analysis system that uses deep learning technology to generate color-marked three-dimensional models of medical images and further produces clinical auxiliary reports. Background Technology

[0002] This invention claims priority to U.S. Provisional Case Nos. 63714875, 63714876, 63714878, and 63714879.

[0003] Traditional medical imaging, such as computed tomography (CT) or magnetic resonance imaging (MRI), primarily presents image data in two-dimensional (2D) slices. While 2D images possess basic diagnostic capabilities, they still have numerous limitations, making it difficult to meet the high standards of precision medicine and surgical guidance required in modern clinical practice. Firstly, 2D images can only present information from specific sections, lacking an intuitive understanding of the overall three-dimensional structure of organs or lesions. Medical personnel must repeatedly compare and mentally reconstruct multiple consecutive slices to infer the spatial relationships between tissues, increasing the interpretive burden and the risk of misdiagnosis.

[0004] Secondly, two-dimensional images cannot provide free adjustment of the viewing angle. For lesions located in complex anatomical areas (such as the skull base, pelvic cavity, and vascular plexus), the limited section plane often leads to incomplete information, which in turn affects clinical judgment. In addition, the lack of three-dimensionality in two-dimensional images is also not conducive to communication with patients or preoperative simulation, reducing their convenience for clinical and teaching applications.

[0005] Therefore, how to develop a medical image analysis system that can solve the above problems has become a problem to be solved in this technical field. Summary of the Invention

[0006] To address the aforementioned problems, according to one embodiment, the present invention provides a medical image analysis system, comprising multiple hardware modules consisting of electrically connected hardware circuits, which are signal-connected to a hospital host. The multiple hardware modules include an image processing module, a 3D modeling module, a pre-training module, and a comparative statistical analysis module.

[0007] The aforementioned image processing module receives multiple 2D black-and-white medical images from the hospital's mainframe, converts the image format of these images to NIfTI format, performs standardization processing according to the type of the images, and then performs data augmentation and resampling to generate a standard modeling image. The aforementioned 3D modeling module, communicatively connected to the image processing module, uses a deep learning model to partition the standard modeling image into organ structures based on the definition of functional anatomy. It then labels and colors the partitioned organ structures to generate a color 3D model file. The aforementioned pre-training module obtains multiple three-dimensional lesion training sample files from the hospital's mainframe for individual organ structures. Based on nnUnet, nnTrans, or SwinUNETR classification tasks, it performs data augmentation and resampling to establish a deep learning model for lesion identification in each organ structure. The number of these multiple three-dimensional lesion sample files exceeds 1000, and each file contains multiple lesion feature voxels. These voxels are used in the resampling process to label and calculate the median of all the multiple three-dimensional lesion sample files. The aforementioned comparative statistical analysis module imports the aforementioned color three-dimensional model files into the lesion identification deep learning model. Based on the extracted relevant organ structures and statistical calculations, it marks the location information in the color three-dimensional model files and generates an analysis report.

[0008] According to this embodiment, preferably, the lesion identification method is based on adding a classification task to NNUNET. Currently, the classification task focuses on distinguishing whether the image is normal or abnormal, and it can be extended to multi-category classification in the future.

[0009] According to this embodiment, preferably, when the above-mentioned 2D black and white medical image information is a magnetic resonance imaging (MRI) image, the image processing module performs Z-score standardization processing on the MRI image.

[0010] According to this embodiment, preferably, when the above-mentioned 2D black and white medical image information is a computed tomography (CT) scan image, the standardization processing performed by the image processing module on the CT scan image is in the following order: collecting all samples, removing extreme values, and Z-score.

[0011] According to this embodiment, preferably, the image format of the above-mentioned 2D black and white medical image information is DICOM format.

[0012] According to this embodiment, preferably, the number of images used for conversion of the above-mentioned 2D black and white medical image information is greater than 10.

[0013] According to this embodiment, preferably, the minimum area that can be treated by the above-mentioned lesion feature voxel is 2 cm by 2 cm.

[0014] According to this embodiment, preferably, the above-mentioned medical image analysis system further includes a deep learning training database, which is communicatively connected to the image processing module and the 3D modeling module. It receives and stores the standard modeling image information from the image processing module and receives and stores the corresponding color 3D model file from the 3D modeling module, providing the training dataset and validation dataset required by the deep learning model.

[0015] According to this embodiment, preferably, the above-mentioned color three-dimensional model file is in stereolithography format.

[0016] According to this embodiment, preferably, the above-mentioned analysis report is generated by using a natural language model to generate the color three-dimensional model file and the marked location information.

[0017] The claimed benefits of this invention include: (1) automatically performing organ structure partitioning and lesion detection for medical images, using a deep learning model to analyze and semantically segment the images, quickly generating anatomically corresponding three-dimensional models, and further distinguishing and representing different organ regions and lesions using color marking, thereby improving clinical interpretability, visual recognition, and diagnostic communication efficiency. (2) generating auxiliary reports with structural information and statistical features, allowing physicians to quickly grasp the location, shape, volume, and spatial relationship of lesions with adjacent organs, serving as a basis for surgical planning and treatment. Attached Figure Description

[0018] To make the above-described techniques and other objects, features, advantages, and embodiments of the present invention more apparent and understandable, the accompanying drawings are described below:

[0019] Figure 1 is a schematic diagram of the architecture of a medical image analysis system according to one embodiment of the present invention.

[0020] Figure 2 is a schematic diagram of a medical image according to one embodiment of the present invention.

[0021] Figure 3 is a schematic diagram of the colored three-dimensional model made according to Figure 2.

[0022] Figure 4 shows a three-dimensional lesion training sample file of one embodiment of the present invention.

[0023] Figure 5 is a schematic diagram showing the analysis report of one embodiment of the present invention displayed on a medical image analysis device.

[0024] Figure 6 is a flowchart illustrating a medical image analysis system according to one embodiment of the present invention. Detailed Implementation

[0025] To illustrate the various embodiments of the present invention in more detail, the following description is provided with reference to the accompanying drawings. It should be understood that when a component is referred to as "connected" or "set" on another component, it may mean that the component is directly located on the other component, or that there may be an intermediate component connecting the component to the other component. Conversely, when a component is referred to as "directly on another component" or "directly connected to another component," it is understood that this explicitly defines the absence of an intermediate component.

[0026] Referring to Figure 1, according to one embodiment, the present invention provides a medical image analysis system 100. Specifically, after converting the medical image format from DICOM to NIfTI format, the system performs preprocessing on the medical images. Then, a deep learning model 122 is used to partition the preprocessed image data into organ structures 124 and identify lesions. Furthermore, a color three-dimensional model is generated for physicians to use in surgical planning, disease interpretation, and clinical diagnosis. In addition, the medical image analysis system 100 also analyzes the color three-dimensional model to extract the spatial location and statistical characteristics of organs and lesions, generating an analysis report 144 to assist physicians in clinical decision-making.

[0027] The medical image analysis system 100 includes multiple hardware modules consisting of multiple electrically connected hardware circuits, which are signal-connected to a hospital host 160. The multiple hardware modules include an image processing module 110, a 3D modeling module 120, a pre-training module 130, a comparative statistical analysis module 140, and a deep learning training database 150.

[0028] The image processing module 110 receives 2D black-and-white medical image information 112 transmitted from the hospital host 160. The image processing module 110 first converts the received 2D black-and-white medical image information 112 from its original DICOM format to NIfTI format. This format conversion helps to unify the image data format, facilitating subsequent data processing and the application of the deep learning model 122. After conversion, the image processing module 110 further determines whether the received 2D black-and-white medical image information 112 is a computed tomography (CT) scan or magnetic resonance imaging (MRI). Depending on the image type, the image processing module 110 performs corresponding standardization processing, such as collecting all samples, removing extreme values ​​and Z-scores for CT images, and using Z-score standardization for MRI images to reduce inter-image differences. The number of images in the 2D black-and-white medical image information 112 used for conversion needs to be greater than 10.

[0029] Furthermore, to improve the generalization ability of the deep learning model 122 and avoid overfitting, the image processing module 110 performs data augmentation techniques on the 2D black-and-white medical image information 112, including random rotation, scaling, deformation, mirror reversal, and adding Gaussian noise. In addition, to ensure the consistency of spatial resolution of all images, the image processing module 110 performs a resampling operation, standardizing the images according to the median voxel resolution in the training data.

[0030] Finally, the image processing module 110 generates standard modeling image information 114 that conforms to the specifications. This standard modeling image information 114 not only has high consistency and repeatability, but is also optimized for subsequent construction of three-dimensional models and deep learning analysis, thereby improving the accuracy and efficiency of medical image analysis.

[0031] Please refer to Figures 2 and 3. Figure 2 is a schematic diagram of a medical image according to one embodiment of the present invention. Figure 3 is a schematic diagram of a color three-dimensional model made according to Figure 2. Figure 2 is a medical image obtained from a full abdominal scan, as described in U.S. Provisional Application No. 63714879; Figure 3 is a three-dimensional model generated by three-dimensional modeling, i.e., identification and marking, based on the obtained medical image, as described in U.S. Provisional Application No. 63714879. The three-dimensional modeling module 120 is communicatively connected to the image processing module 110 and is used to receive preprocessed standard modeling image information 114, and to perform three-dimensional reconstruction and medical structural identification on this information through a deep learning model 122. The deep learning model 122 is nnU-Net, nnTrans, or SwinUNETR. nnU-Net, nnTrans, or SwinUNETR are medical image segmentation architectures with automated structure adjustment capabilities, which can automatically adjust the network layers and training parameters according to the spatial resolution, number of image channels, and anatomical features of the input image to optimize model performance.

[0032] The deep learning model 122 can analyze image content based on functional anatomical definitions (such as Couinaud liver lobe partitions and lung / bronchial partitions) to automatically partition organ structures 124. After partitioning and identification, the 3D modeling module 120 further visualizes the identified regions, assigning specific color markers to different organs to form a clinically visible color 3D model file. This color 3D model file is output in .stl 3D graphics format and can be viewed by physicians using software that supports 3D model operations, including rotation, cutting, and cross-sectional browsing, for diagnosis, surgical planning, and patient communication.

[0033] The organ structure includes the following 177 parts: spleen, right kidney, left kidney, gallbladder, liver, stomach, pancreas, right adrenal gland, left adrenal gland, left upper lobe of lung, left lower lobe of lung, right upper lobe of lung, right middle lobe of lung, right lower lobe of lung, esophagus, trachea, thyroid gland, small bowel, duodenum, colon, urinary bladder, prostate, left kidney cyst, right kidney cyst, sacrum, S1 vertebra, L5 vertebra, L4 vertebra, L3 vertebra, L2 vertebra, L1 vertebra, T12 vertebra, T11 vertebra, T10 vertebra, T9 vertebra, T8 vertebra, T7 vertebra, T6 vertebra, T5 vertebra, T4 vertebra, T3 vertebra, T2 vertebra, T1 vertebra, C7 vertebra, C6 vertebra, C5 vertebra, C4 vertebra, C3 vertebra, C2 vertebra, C1 vertebra, heart, aorta, pulmonary vein, brachiocephalic trunk, right subclavian artery, left subclavian artery, right common carotid artery, left common carotid artery, left brachiocephalic vein, right brachiocephalic vein, left atrial appendage, superior vena cava, inferior vena cava, portal vein and splenic vein, left iliac artery,iliac_artery_right、iliac_vena_left、iliac_vena_right、humerus_left、humerus_right、scapula_left、scapula_right、clavicula_left、clavicula_right、femur_left、femur_right、hip_left、hip_right、spinal_cord、gluteus_maximus_left、gluteus_maximus_right、gluteus_medius_left、gluteus_medius_right、gluteus_minimus_left、gluteus_minimus_right、autochthon_left、autochthon_right、iliopsoas_left、iliopsoas_right、brain、skull、rib_left_1、rib_left_2、rib_left_3、rib_left_4、rib_left_5、rib_left_6、rib_left_7、rib_left_8、rib_left_9、rib_left_10、rib_left_11、rib_left_12、rib_right_1、rib_right_2、rib_right_3、rib_right_4、rib_right_5、rib_right_6、rib_right_7、rib_right_8、rib_right_9、rib_right_10、rib_right_11、rib_right_12、sternum、costal_cartilages。、

[0034] According to U.S. Provisional Application No. 63714879, the 3D modeling module 120 has the ability to automatically identify various parts, organs, and tissues of the human body. Through a layered design, it groups and classifies the identification results, allowing each organ and structure to be presented in an independent layer according to its functional anatomical framework. This layered design not only improves data organization and browsing efficiency but also enables subsequent clinical applications or model outputs to select and manipulate specific organs according to user needs, further supporting the conversion of 3D medical image models covering all parts of the body. The 3D modeling module 120 can automatically identify and convert major anatomical regions such as the brain, thoracic cavity, abdominal cavity, and pelvic cavity based on the areas displayed in the input image, and supports automatic regional modeling of several clinically common organs such as the liver, kidneys, pancreas, spleen, heart, lungs, and brain. In addition to organ identification, it can also perform more detailed classification of organs based on information such as specific features or tissue density changes, further identifying tissue blocks and even cell clusters. The 3D modeling module 120 can perform marking and high-contrast coloring to help physicians identify potential lesion areas and improve the efficiency and interpretability of clinical diagnosis.

[0035] Furthermore, according to U.S. Provisional Application No. 63714879, the deep learning model 122 of the 3D modeling module 120 is developed and trained based on nnUNet. nnUNet (no-new-Net) is an automated medical image segmentation architecture with self-adjusting capabilities in model structure, preprocessing procedures, and post-processing mechanisms, and can automatically configure network parameters and training strategies according to different image data. The training data of this invention comes from 2,500 CT and MRI image cases covering different body parts, all of which are common clinical DICOM format medical image data.

[0036] To ensure the model's accuracy for clinical application and anatomical precision, all training data is manually annotated, precisely outlining the organs and structures of each part. After the initial model training is completed, the 3D modeling module 120 outputs predicted segmentation results, which are interpreted and error-corrected by clinicians. The corrected results and tuning parameters are then re-imported into the model for recursive training to gradually improve the model's accuracy and clinical applicability.

[0037] Please refer to Figure 4, which illustrates a three-dimensional lesion training sample file according to one embodiment of the present invention. The pre-training module 130 of the present invention is designed to execute the training process of the deep learning model 122. It further processes and trains the model by retrieving and aggregating multiple three-dimensional lesion training sample files 132 corresponding to the structures 124 of different organs from the aforementioned hospital host computer 160. The number of training samples must be at least 1000, and each sample contains representative lesion feature voxels 136 to improve the model's generalization ability and recognition accuracy.

[0038] In implementation, the pre-training module 130 performs data augmentation and resampling procedures required for the classification task based on deep learning architectures such as nnUNet, nnTrans, or SwinUNETR. Data augmentation procedures may include operations such as rotation, scaling, flipping, and contrast adjustment to simulate different clinical conditions and increase sample diversity. The resampling process standardizes the lesion feature voxels 136 of all lesion samples and calculates the median of all training samples as the resampling criterion during integration to reduce the impact of spatial resolution differences on model training. The lesion feature voxels 136 may contain various quantifiable parameters to describe the morphology, density, and location characteristics of the lesion. For example, the lesion volume can reflect its total volume in the three-dimensional model and is expressed in mm. 3 or cm 3 The Hounsfield Unit (HU) is used to quantify the density of lesions in CT scans; a higher HU value indicates greater tissue density. Maximum diameter and axial diameter describe the size of the lesion in different directions, while surface area reflects the three-dimensional surface extent of the lesion. The shape characteristics of the lesion can also be quantified, including roundness or irregularity to assess the regularity of the lesion's shape. Texture or grain characteristics describe the homogeneity or heterogeneity of the internal structure of the lesion. Density distribution reflects the distribution of different HU values ​​within the lesion, while boundary features determine the clarity of the lesion's boundaries.

[0039] For example, the characteristic voxel 136 of liver tumor samples has an average CT value ranging from approximately 56 HU to 69 HU and an average volume ranging from approximately 36,847 mm. 3 Up to 45036mm 3 The average axial diameter ranges from approximately 2.97 cm to 3.63 cm. The characteristic voxel 136 of the colorectal tumor samples has an average CT value ranging from approximately 43 HU to 53 HU and an average volume ranging from approximately 39598 mm². 3 Up to 48397mm 3The average axial diameter is approximately 3.69 cm to 4.51 cm. The definition of these multiple characteristic parameters can serve as a screening criterion for the deep learning model 122 during the training and inference stages, helping to eliminate clinically irrelevant noise voxels and further improving the accuracy and stability of lesion segmentation and identification. The lesion characteristic voxel 136 of sarcopenia is primarily muscle mass, used to describe the quantitative characteristics of the patient's muscle groups. Muscle mass reduction can be defined by clinical standards; when a male's muscle mass is below 7.0 kg / m²... 2 Or for women, below 5.4 kg / m 2 (Measured using a dual-energy X-ray absorptiometry (DXA) instrument), which is considered to indicate insufficient muscle mass.

[0040] The minimum area that lesion feature voxels 136 can process is 2 cm by 2 cm. Taking tumor samples as an example, the average axial diameter of liver tumor experimental cases is approximately between 2.97 cm and 3.63 cm. The average axial diameter of colorectal tumor experimental cases is approximately between 3.69 cm and 4.51 cm. All of these results are greater than the set minimum area limit, indicating that the pre-training module 130 and the deep learning model 122 can effectively process lesion feature voxels within this range. By setting this minimum area limit, excessively small or clinically irrelevant noise areas can be excluded, further improving the accuracy and stability of lesion detection and segmentation.

[0041] Through the aforementioned method, the pre-training module 130 can establish highly recognizable deep learning models 134 for lesion identification for different organ structures 124, serving as the basis for 3D modeling. This implementation not only improves training efficiency and model stability but also effectively enhances the accuracy and reliability in clinical applications.

[0042] After multiple rounds of training and model optimization, the deep learning model 122 of this invention achieves an overall segmentation accuracy of over 85% on the validation dataset. This deep learning model 122 possesses stable and reliable 3D modeling and lesion identification capabilities, and can serve as the basis for subsequent color labeling, visualization modeling, and medical report generation.

[0043] The comparative statistical analysis module 140 is used to further enhance the clinical application value of lesion identification results. During implementation, the comparative statistical analysis module 140 will first receive and import the color three-dimensional model file produced by the aforementioned deep learning modeling and visualization processing, and input the file into the lesion identification deep learning model 134 previously established by the pre-training module 130 for further feature comparison and verification.

[0044] In the lesion identification method disclosed in this invention, the core technology used is a deep learning model 122 developed based on the nnU-Net architecture, and a classification task is added to the deep learning model 122 to enhance the ability to distinguish lesion abnormalities, so as to determine whether the overall input image is "normal" or "abnormal".

[0045] Through the comparison procedure, the comparative statistical analysis module 140 can effectively extract features and perform statistical analysis on the organ structures 124 and lesion areas in the color three-dimensional model, including but not limited to: lesion location (such as the coordinates or slice location relative to the organ), volume (calculated in Voxel units), boundary clarity, density distribution, and relative distance to adjacent anatomical structures.

[0046] The comparative statistical analysis module 140 transforms the above analysis results into a structured data format and visualizes them in a color 3D model with dynamic markings and location information 142. Finally, this module automatically generates a complete analysis report 144, which may include a structural classification summary, quantitative lesion information, and clinical reference annotations, providing clinicians with further information for disease interpretation, surgical evaluation, and treatment planning. This process significantly reduces the manpower required for traditional analysis and improves diagnostic efficiency and consistency.

[0047] The deep learning training database 150 is connected to the image processing module 110 and the 3D modeling module 120, and is used to collect and store the training and validation data required for the deep learning model 122. This database receives and stores pre-processed standard modeling image information 114 from the image processing module 110, and receives and stores corresponding color 3D model files from the 3D modeling module 120, serving as the basic dataset for training and validating the deep learning model 122.

[0048] Example 1

[0049] The image processing module 110 receives 2D black-and-white medical image information 112 transmitted from the hospital host 160, such as coronary CTA (Computed Tomography Angiography) images obtained after the administration of an angiography agent. It then converts the image format of the 2D black-and-white medical image information 112 to NIfTI format and performs standardization processing, including collecting all samples, removing extreme values, and Z-score adjustment. Afterward, the image processing module 110 performs data augmentation and resampling operations on the 2D black-and-white medical image information 112 to generate a standard modeling image information 114. This effectively improves the quality and consistency of the input image, enabling subsequent 3D modeling algorithms to more accurately extract structural contours and key information.

[0050] The 3D modeling module 120 receives standard modeling image information 114 preprocessed by the image processing module 110, and uses a deep learning model 122 based on the nnU-Net architecture to perform 3D reconstruction of the coronary arteries. This deep learning model 122 has automatic configuration and adjustment capabilities, optimizing network parameters based on the characteristics of the input images and defining the coronary artery structure according to functional anatomy. The reconstructed color 3D model (.stl format) clearly presents the overall structure of the heart, distinguishing blood vessels and myocardial tissue with different colors, thus improving visual recognition.

[0051] The comparative statistical analysis module 140 receives the aforementioned color 3D model and inputs it into the previously trained lesion identification deep learning model 134. Based on the organ structures 124 extracted from the color 3D model, the comparative statistical analysis module 140 performs statistical analysis and feature calculations, marking key lesion locations and their spatial distribution. For example, by comparing the characteristics of normal blood vessel structures with abnormal areas in the input model, the comparative statistical analysis module 140 identifies the location of blood vessel blockage and quantifies the extent and impact of the blockage. Finally, a detailed analysis report 144 is generated based on the analysis results, providing reference for physicians' surgical planning and clinical diagnosis. This analysis report 144 is generated using a natural language model from the color 3D model file and the marked location information 142.

[0052] Example 2

[0053] This embodiment uses the assessment and diagnosis of sarcopenia as an example to illustrate the image processing flow and technical effects of the system of the present invention.

[0054] The image processing module 110 receives 2D black-and-white medical image information of a female subject transmitted from the hospital host 160. This 2D black-and-white medical image information refers to medical images acquired by MRI and CT, such as cross-sections of the third lumbar vertebra (L3 segment), femoral muscle groups, and trunk muscle groups such as the psoas major and erector spinae muscles, which are commonly used to assess skeletal muscle volume, muscle thickness, or muscle cross-sectional area. The image processing module 110 converts the image format of the 2D black-and-white medical image information 112 to NIfTI format and performs standardization processing on the 2D black-and-white medical image information 112, including collecting all samples, removing extreme values, and Z-score. Subsequently, the image processing module 110 performs data augmentation and resampling operations on the 2D black-and-white medical image information 112 to generate a standard modeling image information 114, which can effectively improve the quality and consistency of the input image, enabling subsequent 3D modeling algorithms to more accurately extract structural contours and key information.

[0055] The 3D modeling module 120 receives standard modeling image information 114 preprocessed by the image processing module 110, and uses a deep learning model 122 based on the nnU-Net architecture to reconstruct the muscles in 3D. This deep learning model 122 has automatic configuration and adjustment capabilities, optimizing network parameters based on the characteristics of the input images and defining the coronary artery structure according to functional anatomy. The reconstructed color 3D model (.stl format) clearly presents the muscle structure, distinguishing different muscle groups with different colors, providing an intuitive visualization of muscle morphology and spatial distribution, facilitating quantitative analysis and clinical assessment of sarcopenia.

[0056] The comparative statistical analysis module 140 receives the aforementioned color 3D model and integrates it into the previously trained lesion identification deep learning model 134. Based on the muscle structure extracted from the color 3D model, the comparative statistical analysis module 140 performs statistical analysis and feature calculations, including indicators such as muscle volume, muscle thickness, cross-sectional area, and the proportion between muscle groups, to provide quantitative data to support the diagnosis and clinical decision-making for sarcopenia. For example, the comparative statistical analysis module 140 can identify the degree of muscle atrophy, fat infiltration, and imbalances between muscle groups by comparing the subject's muscle structure with baseline data of normal muscles. Finally, a detailed analysis report 144 is generated based on the analysis results, providing physicians with a reference for surgical planning and clinical diagnosis.

[0057] Please refer to Figure 5, which is a schematic diagram of an analysis report displayed on a medical image analysis device according to one embodiment of the present invention. The analysis report 144 is generated by a natural language model using the color three-dimensional model file and the marked location information 142. Simultaneously, the analysis report 144 includes quantitative indicators obtained from comparative statistical analysis, such as muscle volume, muscle thickness, cross-sectional area, and the proportion between muscle groups, to comprehensively present the sarcopenia assessment results and assist clinical decision-making. As shown in Figure 5, the analysis report 144 shows that the examinee's muscle mass is 5.6 kg / m². 2 The muscle mass exceeded the threshold of 5.4 kg / m² for sarcopenia in women. Therefore, the individual's muscle mass was determined to be within the normal range and not indicative of sarcopenia. Furthermore, the analysis report 144 can further display the muscle mass distribution and the relative proportions of each muscle group for clinicians' reference, and can be used to track changes in muscle mass or assess treatment effectiveness.

[0058] Please refer to Figure 6, which is a flowchart of a medical image analysis system 100 according to one embodiment of the present invention.

[0059] In step 200, after receiving the 2D black and white medical image information 112 transmitted by the hospital host 160, the image processing module 110 converts the image format of the 2D black and white medical image information 112 into NIfTI format.

[0060] In step 201, the image processing module 110 performs corresponding standardization processing according to the type of 2D black and white medical image information 112, and then performs data enhancement and resampling to generate standard modeling image information 114.

[0061] In step 202, the 3D modeling module 120 uses a deep learning model 122 to partition the standard modeling image information 114 into organ structures 124 according to the definition of functional anatomy.

[0062] In step 203, the 3D modeling module 120 marks and colors the organ structures 124 in the partition, generating a colored 3D model file.

[0063] In step 204, the comparative statistical analysis module 140 imports the color three-dimensional model file into the lesion identification deep learning model 134, and marks the location information 142 in the color three-dimensional model file based on the extracted relevant organ structures 124 and statistical analysis calculation, and generates an analysis report 144.

[0064] In summary, this invention enables fully automated processing of medical images. Starting from receiving raw image data, it sequentially completes format conversion, preprocessing, 3D modeling, structural identification, lesion detection, and color marking. Furthermore, it generates visually identifiable 3D models and clinical auxiliary reports for physicians to use for lesion localization, surgical planning, disease description, and clinical diagnosis, effectively improving the practical value and diagnostic efficiency of medical images and reducing the burden of manual operation.

[0065] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0066] The above-described embodiments are merely illustrative examples and are not intended to limit the scope of the invention. Any equivalent modifications or alterations made to them shall not depart from the spirit and scope of the invention and shall be included in the claims of this application.

Claims

1. A medical image analysis system, comprising multiple hardware modules consisting of electrically connected hardware circuits, signal-connected to a hospital host computer, wherein the multiple hardware modules include: An image processing module receives multiple 2D black and white medical image information from the hospital host, converts the image format of the 2D black and white medical image information into NIfTI format, performs corresponding standardization processing according to the type of the 2D black and white medical image information, performs data enhancement and resampling on the 2D black and white medical image information, and generates a standard modeling image information. A 3D modeling module is communicatively connected to the image processing module. It uses a deep learning model to partition the standard modeling image information into organ structures according to the definition of functional anatomy, and marks and colors the partitioned organ structures to generate a color 3D model file. A pre-training module obtains multiple three-dimensional lesion training sample files of individual organ structures from the hospital host computer. Based on nnUnet, nnTrans, or SwinUNETR classification tasks, it performs data augmentation and resampling to establish a deep learning model for lesion identification of individual organ structures. The multiple three-dimensional lesion sample files contain more than 1000 samples and have multiple lesion feature voxels, which are used for labeling and calculating the median of all the multiple three-dimensional lesion sample files during the resampling process. A comparative statistical analysis module imports the color 3D model file into the lesion identification deep learning model, and based on the extracted relevant organ structures and statistical analysis calculations, marks the location information in the color 3D model file and generates an analysis report.

2. The medical image analysis system as described in claim 1, wherein the lesion identification method is based on adding a classification task to NNUNET. Currently, the classification task focuses on distinguishing whether the image is normal or abnormal, and can be extended to multi-category classification in the future.

3. The medical image analysis system as described in claim 1, wherein when the 2D black and white medical image information is a magnetic resonance imaging (MRI) image, the image processing module performs Z-score standardization processing on the computed tomography (CT) image.

4. The medical image analysis system as described in claim 1, wherein when the 2D black and white medical image information is a computed tomography image, the standardization processing performed by the image processing module on the magnetic resonance imaging image is in the following order: collecting all samples, removing extreme values, and Z-score.

5. The medical image analysis system as described in claim 1, wherein the image format of the 2D black and white medical image information is DICOM format.

6. The medical image analysis system as described in claim 5, wherein the number of images in the 2D black and white medical image information to be converted is greater than 10.

7. The medical image analysis system as described in claim 1, wherein the minimum area that the lesion feature voxel can process is 2 cm by 2 cm.

8. The medical image analysis system as described in claim 1 further includes a deep learning training database, which is communicatively connected to the image processing module and the 3D modeling module. The database receives and stores the standard modeling image information from the image processing module and receives and stores the corresponding color 3D stereoscopic model file from the 3D modeling module, providing the training dataset and validation dataset required by the deep learning model.

9. The medical image analysis system as described in claim 1, wherein the color three-dimensional model file is in stereolithography format.

10. The medical image analysis system of claim 1, wherein the analysis report is generated by using a natural language model to generate the color three-dimensional model file and the marked location information.

Images