Multi-organ segmentation method, device and equipment based on multi-task neural network model
Through the 2.5D-FADNet multi-task segmentation neural network based on the multi-task neural network model, the abdominal CT images are processed, which solves the problem of insufficient multi-organ segmentation accuracy in traditional methods, and achieves high-precision and fast multi-organ segmentation effect.
Patent Information
- Application Number
- CN202411396652.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-21
AI Technical Summary
Traditional medical image segmentation algorithms have poor accuracy in multi-organ segmentation, cannot accurately segment organ boundaries, and it is difficult to segment multiple organs at the same time.
The multi-task neural network model is used to process the abdominal CT images through the 2.5D-FADNet multi-task segmentation neural network model to output the segmentation results of thirteen organs. The model includes two network branches, each branch consisting of a multi-layer convolutional network, a downsampling network and an upsampling network. Through model training and verification, high-precision multi-organ segmentation results are obtained.
It realizes high-precision and rapid multi-organ segmentation, which can accurately identify and divide thirteen organs such as liver, kidney, spleen, and pancreas, and improves the accuracy and efficiency of medical image segmentation.
Smart Images
Figure CN119494844A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of deep learning intelligent recognition technology, and in particular, relates to a multi-organ segmentation method, device, electronic device and computer-readable storage medium based on a multi-task neural network model. Background Art
[0002] Traditional image segmentation algorithms segment medical images based on threshold segmentation, edge detection or region growing. These methods have poor segmentation accuracy, roughly segment the target area, cannot accurately segment the target area boundaries, and cannot perform multi-organ segmentation.
[0003] Therefore, how to perform multi-organ segmentation quickly and accurately is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0004] The embodiments of the present application provide a multi-organ segmentation method, device, electronic device and computer-readable storage medium based on a multi-task neural network model, which can perform multi-organ segmentation quickly and accurately.
[0005] In a first aspect, an embodiment of the present application provides a multi-organ segmentation method based on a multi-task neural network model, comprising:
[0006] Obtain abdominal CT images;
[0007] The abdominal CT image is input into the preset multi-task neural network model, and the multi-organ segmentation results are output respectively; among them, the multi-organ segmentation results include the segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
[0008] Optionally, the multi-task neural network model is obtained through model training based on the 2.5D-FADNet multi-task segmentation neural network. The 2.5D-FADNet multi-task segmentation neural network includes two network branches, each of which includes a multi-layer convolutional network, a downsampling network, and an upsampling network. The segmentation results of the thirteen organs are the fusion output of the results of the two network branches.
[0009] Optionally, before inputting the abdominal CT image into the preset multi-task neural network model, the method further includes:
[0010] Acquire an abdominal CT image dataset;
[0011] Annotating an abdominal region of an abdominal CT image in an abdominal CT image data set, and determining the abdominal region as a segmentation mask; wherein each segmentation mask corresponds to the abdominal CT image one by one;
[0012] Convert each abdominal CT image and its corresponding segmentation mask into PNG format;
[0013] All abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into training set, validation set and test set according to the preset ratio.
[0014] Optionally, after all the abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into a training set, a validation set, and a test set according to a preset ratio, the method further includes:
[0015] The training set is used to train the 2.5D-FADNet multi-task segmentation neural network model, and the validation set and test set are used for verification and testing to obtain a multi-task neural network model.
[0016] Optionally, the 2.5D-FADNet multi-task segmentation neural network model is trained using the training set, and the validation set and the test set are used for validation and testing to obtain a multi-task neural network model, including:
[0017] During model training, set the batch_size of training to 64;
[0018] Set the initial learning rate to 1e-4, and add a learning rate decay strategy. Every 5000 iterations, the learning rate decays to 0.9 of the previous learning rate.
[0019] Set the optimizer to Adam optimizer;
[0020] Set the loss function to DICE loss;
[0021] The training set and validation set are verified once every 1000 iterations. The early stopping method is used to determine the stopping time of network training and obtain a multi-task neural network model.
[0022] Optionally, multi-layer convolutional networks use residual convolution to reduce feature loss;
[0023] Among them, the residual convolution is a residual unit composed of 1x1, 3x3 and 1x1 convolution kernels.
[0024] Optional, downsampling network for feature extraction and reducing computational complexity, reducing the spatial dimension of the data;
[0025] Upsampling network is used for image super-resolution, image segmentation refinement, and increasing the spatial dimension of data.
[0026] In a second aspect, an embodiment of the present application provides a multi-organ segmentation device based on a multi-task neural network model, comprising:
[0027] An image acquisition module, used for acquiring abdominal CT images;
[0028] The segmentation result output module is used to input the abdominal CT image into a preset multi-task neural network model and output the multi-organ segmentation results respectively; wherein the multi-organ segmentation results include the segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
[0029] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
[0030] When the processor executes the computer program instructions, the multi-organ segmentation method based on the multi-task neural network model as shown in the first aspect is implemented.
[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the multi-organ segmentation method based on the multi-task neural network model as shown in the first aspect is implemented.
[0032] The multi-organ segmentation method based on the multi-task neural network model includes: acquiring an abdominal CT image; inputting the abdominal CT image into a preset multi-task neural network model, and outputting multi-organ segmentation results respectively; wherein the multi-organ segmentation results include: segmentation results of thirteen organs including liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0034] Figure 1 It is a flowchart of a multi-organ segmentation method based on a multi-task neural network model provided by an embodiment of the present application;
[0035] Figure 2 This is a schematic diagram of the 2.5D-FADNet multi-task segmentation neural network structure provided by an embodiment of the present application;
[0036] Figure 3 It is a structural schematic diagram of a multi-organ segmentation device based on a multi-task neural network model provided by an embodiment of the present application;
[0037] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0038] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0040] Traditional image segmentation algorithms segment medical images based on threshold segmentation, edge detection or region growing. These methods have poor segmentation accuracy, roughly segment the target area, and cannot accurately segment the target area boundaries.
[0041] In order to solve the problems of the prior art, the embodiments of the present application provide a multi-organ segmentation method, device, apparatus and computer-readable storage medium based on a multi-task neural network model. The multi-organ segmentation method based on a multi-task neural network model provided by the embodiments of the present application is first introduced below.
[0042] Figure 1 FIG. 1 is a flow chart of a multi-organ segmentation method based on a multi-task neural network model provided by an embodiment of the present application. Figure 1 As shown, the multi-organ segmentation method based on the multi-task neural network model includes:
[0043] S101, obtaining an abdominal CT image;
[0044] S102. Input the abdominal CT image into a preset multi-task neural network model, and output multi-organ segmentation results respectively; wherein the multi-organ segmentation results include segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
[0045] Figure 2 This is a schematic diagram of the 2.5D-FADNet multi-task segmentation neural network structure provided by an embodiment of the present application. In one embodiment, the multi-task neural network model is obtained through model training based on the 2.5D-FADNet multi-task segmentation neural network. The 2.5D-FADNet multi-task segmentation neural network includes two network branches, each of which includes a multi-layer convolutional network, a downsampling network, and an upsampling network. The segmentation results of the thirteen organs are the fusion outputs of the results of the two network branches.
[0046] In one embodiment, before inputting the abdominal CT image into the preset multi-task neural network model, the method further includes:
[0047] Acquire an abdominal CT image dataset;
[0048] Annotating an abdominal region of an abdominal CT image in an abdominal CT image data set, and determining the abdominal region as a segmentation mask; wherein each segmentation mask corresponds to the abdominal CT image one by one;
[0049] Convert each abdominal CT image and its corresponding segmentation mask into PNG format;
[0050] All abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into training set, validation set and test set according to the preset ratio.
[0051] In one embodiment, after all abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into a training set, a validation set, and a test set according to a preset ratio, the method further includes:
[0052] The training set is used to train the 2.5D-FADNet multi-task segmentation neural network model, and the validation set and test set are used for verification and testing to obtain a multi-task neural network model.
[0053] In one embodiment, the 2.5D-FADNet multi-task segmentation neural network is trained using a training set, and is verified and tested using a validation set and a test set to obtain a multi-task neural network model, including:
[0054] During model training, set the batch_size of training to 64;
[0055] Set the initial learning rate to 1e-4, and add a learning rate decay strategy. Every 5000 iterations, the learning rate decays to 0.9 of the previous learning rate.
[0056] Set the optimizer to Adam optimizer;
[0057] Set the loss function to DICE loss;
[0058] The training set and validation set are verified once every 1000 iterations. The early stopping method is used to determine the stopping time of network training and obtain a multi-task neural network model.
[0059] In one embodiment, the multi-layer convolutional network uses residual convolution to reduce feature loss;
[0060] Among them, the residual convolution is a residual unit composed of 1x1, 3x3 and 1x1 convolution kernels.
[0061] In one embodiment, a downsampling network is used for feature extraction and to reduce computational complexity, thereby reducing the spatial dimension of the data;
[0062] Upsampling network is used for image super-resolution, image segmentation refinement, and increasing the spatial dimension of data.
[0063] Figure 3 : is a schematic diagram of the structure of a multi-organ segmentation device based on a multi-task neural network model provided by an embodiment of the present application; the multi-organ segmentation device based on a multi-task neural network model comprises:
[0064] An image acquisition module 301 is used to acquire an abdominal CT image;
[0065] The segmentation result output module 302 is used to input the abdominal CT image into a preset multi-task neural network model and output multi-organ segmentation results respectively; wherein the multi-organ segmentation results include segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
[0066] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown.
[0067] The electronic device may include a processor 401 and a memory 402 storing computer program instructions.
[0068] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0069] Memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, memory 402 may be inside or outside of an electronic device. In a particular embodiment, memory 402 may be a non-volatile solid-state memory.
[0070] In one embodiment, the memory 402 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.
[0071] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any one of the multi-organ segmentation methods based on the multi-task neural network model in the above embodiments.
[0072] In one example, the electronic device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.
[0073] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0074] Bus 410 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 410 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0075] In addition, in combination with the multi-organ segmentation method based on the multi-task neural network model in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the multi-organ segmentation methods based on the multi-task neural network model in the above embodiment is implemented.
[0076] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0077] The functional modules shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0078] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0079] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.
[0080] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A multi-organ segmentation method based on a multi-task neural network model, characterized in that: include: Obtain abdominal CT images; The abdominal CT image is input into the preset multi-task neural network model, and the multi-organ segmentation results are output respectively; among them, the multi-organ segmentation results include the segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
2. The multi-organ segmentation method based on the multi-task neural network model according to claim 1, characterized in that: The multi-task neural network model is obtained through model training based on the 2.5D-FADNet multi-task segmentation neural network. The 2.5D-FADNet multi-task segmentation neural network includes two network branches, each of which includes a multi-layer convolutional network, a downsampling network, and an upsampling network. The segmentation results of the thirteen organs are the fusion output of the results of the two network branches.
3. The multi-organ segmentation method based on the multi-task neural network model according to claim 2, characterized in that: Before inputting the abdominal CT image into the preset multi-task neural network model, the method further includes: Acquire an abdominal CT image dataset; Annotating an abdominal region of an abdominal CT image in an abdominal CT image data set, and determining the abdominal region as a segmentation mask; wherein each segmentation mask corresponds to the abdominal CT image one by one; Convert each abdominal CT image and its corresponding segmentation mask into PNG format; All abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into training set, validation set and test set according to the preset ratio.
4. The multi-organ segmentation method based on the multi-task neural network model according to claim 3, characterized in that: After all the abdominal CT images converted into PNG format and their corresponding segmentation masks are divided into a training set, a validation set and a test set according to a preset ratio, the method further includes: The training set is used to train the 2.5D-FADNet multi-task segmentation neural network model, and the validation set and test set are used for verification and testing to obtain a multi-task neural network model.
5. The multi-organ segmentation method based on the multi-task neural network model according to claim 4, characterized in that: The 2.5D-FADNet multi-task segmentation neural network is trained using the training set, and the validation set and test set are used for verification and testing to obtain a multi-task neural network model, including: During model training, set the batch_size of training to 64; Set the initial learning rate to 1e-4, and add a learning rate decay strategy. Every 5000 iterations, the learning rate decays to 0.9 of the previous learning rate. Set the optimizer to Adam optimizer; Set the loss function to DICEloss; The training set and validation set are verified once every 1000 iterations. The early stopping method is used to determine the stopping time of network training and obtain a multi-task neural network model.
6. The multi-organ segmentation method based on the multi-task neural network model according to claim 2, characterized in that: Multi-layer convolutional networks use residual convolution to reduce feature loss; Among them, the residual convolution is a residual unit composed of 1x1, 3x3 and 1x1 convolution kernels.
7. The multi-organ segmentation method based on the multi-task neural network model according to claim 2, characterized in that: Downsampling network, used for feature extraction and reducing computational complexity, reducing the spatial dimension of data; Upsampling network is used for image super-resolution, image segmentation refinement, and increasing the spatial dimension of data.
8. A multi-organ segmentation device based on a multi-task neural network model, characterized in that: include: An image acquisition module, used for acquiring abdominal CT images; The segmentation result output module is used to input the abdominal CT image into a preset multi-task neural network model and output the multi-organ segmentation results respectively; wherein the multi-organ segmentation results include the segmentation results of thirteen organs: liver, right kidney, spleen, pancreas, aorta, inferior vena cava, right adrenal gland, left adrenal gland, gallbladder, esophagus, stomach, duodenum, and left kidney.
9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the multi-organ segmentation method based on the multi-task neural network model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the multi-organ segmentation method based on a multi-task neural network model as described in any one of claims 1 to 7.