Three-dimensional medical image segmentation method based on multi-modal data fusion
Through multimodal data fusion and 3D-UNet model adjustment training, the problem of insufficient accuracy of existing medical image segmentation methods in complex structure processing is solved, and higher image segmentation accuracy is achieved, providing clinical diagnosis with better results Reliable support.
Patent Information
- Application Number
- CN202411166994.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-17
AI Technical Summary
The existing medical imaging segmentation methods are insufficient in dealing with complex structures such as colon polyps, making it difficult to provide reliable clinical diagnostic support.
Using a three-dimensional medical image segmentation method based on multimodal data fusion, the abdominal CT data is processed through medical image three-dimensional reconstruction software, a three-dimensional model is generated, and a virtual endoscopy is performed on the model, and the pictures and data are intercepted with the 3D-UNet model. Feature fusion, adjust training to enhance the model's ability to recognize complex structures.
It improves the accuracy of medical imaging segmentation, provides more reliable technical support for clinical diagnosis, and enhances the model's ability to identify complex structures.
Smart Images
Figure CN119151955A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical image processing, and in particular relates to a three-dimensional medical image segmentation method based on multimodal data fusion. Background Art
[0002] Current medical image segmentation methods mainly rely on single-modality image data, such as CT, MRI, etc. These methods often face the problem of insufficient accuracy when dealing with complex structures such as colon polyps. Summary of the invention
[0003] In order to solve the above problems, this application designs a three-dimensional medical image segmentation method based on multimodal data fusion, in order to achieve the accuracy of medical image segmentation and provide more reliable technical support for clinical diagnosis.
[0004] A three-dimensional medical image segmentation method based on multimodal data fusion comprises the following steps:
[0005] Step S1, using medical imaging 3D reconstruction software to process abdominal CT data to generate an accurate 3D model;
[0006] Step S2, performing a virtual endoscopy on the colon on the three-dimensional model to obtain a clinically significant perspective;
[0007] Step S3, using the 3D-UNet model as the framework of the segmentation model, intercepting the three-dimensional model, intercepting the virtual endoscope image with the polyp and its corresponding sagittal, coronal, and axial three-view images, to form a multimodal data set;
[0008] Step S4: fuse the DICOM format image data of the abdominal CT with the corresponding three-dimensional model screenshot, introduce the multimodal data features of the screenshot, and adjust and train the 3D-UNet model to enhance the 3D-UNet model's ability to recognize complex structures.
[0009] Preferably, in step S1, the DICOM data of the abdominal CT needs to be standardized and normalized before being processed using medical imaging three-dimensional reconstruction software to reduce image differences under different equipment and conditions.
[0010] Preferably, the 3D-UNet model is also provided with a detail extraction module for acquiring texture, shape and depth information of the polyp.
[0011] Preferably, in step S4, the method for adjusting and training the 3D-UNet model is to adjust the network structure of the 3D-UNet and increase the depth of the convolutional layer.
[0012] Preferably, in step S1, it is necessary to ensure that each point on the three-dimensional model can find a corresponding point in the sagittal, coronal and axial views of the CT image.
[0013] Preferably, in step S3, the method of mapping corresponding points in the three viewing angles of sagittal, coronal and axial positions includes coordinate conversion and error correction;
[0014] The accuracy of the error correction is 0.01 mm.
[0015] The advantages and effects of this application are as follows:
[0016] The present application designs a 3D medical image segmentation method based on multimodal data fusion. First, medical image 3D reconstruction software is used to process abdominal CT data to generate an accurate 3D model. A virtual endoscopy of the colon is performed on the 3D model. Then, the 3D-UNet model is used as the framework of the segmentation model to intercept the 3D model, intercept the virtual endoscopy image with polyps and its corresponding sagittal, coronal and axial three-view images to form a multimodal data set. Finally, the DICOM format image data of the abdominal CT is fused with the corresponding 3D model interception image, the multimodal data features of the interception image are introduced, and the 3D-UNet model is adjusted and trained to enhance the 3D-UNet model's ability to recognize complex structures. This can achieve the accuracy of medical image segmentation and provide more reliable technical support for clinical diagnosis.
[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings as follows.
[0018] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.
[0020] Figure 1A flowchart of a 3D medical image segmentation method based on multimodal data fusion designed for this application. DETAILED DESCRIPTION
[0021] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all of the embodiments. In the following description, specific details such as specific configuration and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures is omitted in the embodiment.
[0022] It should be understood that the references to "one embodiment" or "this embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "one embodiment" or "this embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0023] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.
[0024] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that there can be two relationships. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship.
[0025] The term "at least one" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0026] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusions.
[0027] Example 1
[0028] Please refer to Figure 1 , this application designs a 3D medical image segmentation method based on multimodal data fusion, which includes the following steps:
[0029] Step S1, using medical imaging 3D reconstruction software to process abdominal CT data to generate an accurate 3D model;
[0030] Step S2, performing a virtual endoscopy on the colon on the three-dimensional model to obtain a clinically significant perspective;
[0031] Step S3, using the 3D-UNet model as the framework of the segmentation model, intercepting the three-dimensional model, intercepting the virtual endoscope image with the polyp and its corresponding sagittal, coronal, and axial three-view images, to form a multimodal data set;
[0032] Step S4: fuse the DICOM format image data of the abdominal CT with the corresponding three-dimensional model screenshot, introduce the multimodal data features of the screenshot, and adjust and train the 3D-UNet model to enhance the 3D-UNet model's ability to recognize complex structures.
[0033] Furthermore, in step S1, the DICOM data of the abdominal CT scan needs to be standardized and normalized before being processed using medical imaging three-dimensional reconstruction software to reduce image differences between different devices and conditions.
[0034] Furthermore, the 3D-UNet model is also provided with a detail extraction module for obtaining texture, shape and depth information of the polyp.
[0035] Furthermore, in step S4, the method for adjusting the training of the 3D-UNet model is to adjust the network structure of the 3D-UNet and increase the depth of the convolutional layer.
[0036] Furthermore, in step S1, it is necessary to ensure that each point on the three-dimensional model can find a corresponding point in the sagittal, coronal and axial views of the CT image.
[0037] Furthermore, in step S3, the method of mapping corresponding points in the three viewing angles of sagittal, coronal and axial positions includes coordinate conversion and error correction;
[0038] The accuracy of the error correction is 0.01 mm.
[0039] The above description is only the preferred embodiment of the present invention, which does not limit the protection scope of the present invention. For those skilled in the art, the present invention may have various modifications and changes. Within the spirit and principle of the present invention, the changes, modifications, replacements, integrations and parameter changes of these embodiments by conventional substitutions or capable of achieving the same functions without departing from the principles and spirit of the present invention all fall within the protection scope of the present invention.
Claims
1. A three-dimensional medical image segmentation method based on multimodal data fusion, characterized in that: The following steps are involved: Step S1, using medical imaging 3D reconstruction software to process abdominal CT data to generate an accurate 3D model; Step S2, performing a virtual endoscopy on the colon on the three-dimensional model to obtain a clinically significant perspective; Step S3, using the 3D-UNet model as the framework of the segmentation model, intercepting the three-dimensional model, intercepting the virtual endoscope image with the polyp and its corresponding sagittal, coronal, and axial three-view images, to form a multimodal data set; Step S4: fuse the DICOM format image data of the abdominal CT with the corresponding three-dimensional model screenshot, introduce the multimodal data features of the screenshot, and adjust and train the 3D-UNet model to enhance the 3D-UNet model's ability to recognize complex structures.
2. The three-dimensional medical image segmentation method based on multimodal data fusion according to claim 1, characterized in that: In step S1, the DICOM data of the abdominal CT scan needs to be standardized and normalized before being processed using medical imaging three-dimensional reconstruction software to reduce image differences between different devices and conditions.
3. The three-dimensional medical image segmentation method based on multimodal data fusion according to claim 1, characterized in that: The 3D-UNet model is also provided with a detail extraction module for obtaining the texture, shape and depth information of the polyp.
4. The three-dimensional medical image segmentation method based on multimodal data fusion according to claim 1, characterized in that: In step S4, the method for adjusting the training of the 3D-UNet model is to adjust the network structure of the 3D-UNet and increase the depth of the convolutional layer.
5. The three-dimensional medical image segmentation method based on multimodal data fusion according to claim 1, characterized in that: In step S1, it is necessary to ensure that each point on the three-dimensional model can find a corresponding point in the sagittal, coronal and axial views of the CT image.
6. The three-dimensional medical image segmentation method based on multimodal data fusion according to claim 1, characterized in that: In step S3, the method of mapping corresponding points in the three viewing angles of sagittal, coronal and axial positions includes coordinate conversion and error correction; The accuracy of the error correction is 0.01 mm.