Multi-map label fusion method and system for medical image, and storage medium
Through the multi-map label fusion method, labeled maps and differential homoembryonic registration technology, combined with expert prior knowledge, the problem of unclear edges in deep learning medical image segmentation is solved, significantly improving segmentation accuracy and clarity.
Patent Information
- Application Number
- CN202411095497.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-29
AI Technical Summary
When the medical image segmentation method based on deep learning lacks prior knowledge of the images to be segmented, it is prone to the situation where isolated discrete areas and lesions are not clear, resulting in poor segmentation effect.
By using the multi-map tag fusion method, differential homoembryonic registration and multi-layer convolution operations are performed by obtaining labeled maps, combining expert prior knowledge to improve the network's learning ability of organ or site characteristics, and generate more accurate segmentation. result.
The accuracy of deep learning models in medical image segmentation and the clarity of segmentation results are improved, especially in terms of segmentation effect at the edge of the lesion, which significantly improves segmentation accuracy.
Smart Images

Figure CN119048754A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical devices, and in particular to a method, system and storage medium for fusion of multi-atlas labels of medical images. Background Art
[0002] Medical image segmentation plays an important role in the field of medical image processing, and plays a key role in disease diagnosis, disease progression estimation, lesion site delineation, lesion targeted treatment, etc. based on medical images. With the development of deep learning technology, especially convolutional neural networks, the automatic segmentation method of medical images based on deep learning methods has higher segmentation accuracy and faster segmentation speed than traditional methods. At present, the medical image segmentation method based on deep learning is mainly improved based on the typical network structure UNet network. With the invention of Transformer network, the accuracy of the segmentation method has been improved to a certain extent. However, due to the lack of prior knowledge of the image to be segmented, the segmentation result has isolated discrete areas, especially in the picture with unclear edge contours of lesions or parts, the segmentation effect is greatly reduced. Summary of the invention
[0003] This patent proposes a multi-atlas label fusion method for deep registration to make up for the shortcomings of the current deep learning methods.
[0004] In a first aspect, the present invention provides a multi-atlas label fusion method for medical images, comprising the following steps:
[0005] Step 101: Obtain a labeled graph;
[0006] Step 102: Registration network processing;
[0007] Step 103: Integrate the generated velocity field;
[0008] Step 104: Generate a diffeomorphic transformation;
[0009] Step 105: Align the atlas image, the label image and the target image to be segmented;
[0010] Step 106: Acquire the target image to be segmented;
[0011] Step 107: multi-channel and multi-layer convolution operation;
[0012] Step 108: Generate a feature map and perform cascade processing;
[0013] Step 109: feature fusion and further learning;
[0014] Step 110: Output the predicted segmentation result through the softmax layer.
[0015] Furthermore, the labeled atlas includes a label image and an atlas image.
[0016] Furthermore, the step 107 specifically includes two branches, the upper branch input is the cascade feature map of the target image to be segmented and the feature map of the labeled atlas after registration, and the lower branch input is the cascade feature map of the labeled atlas feature map after registration.
[0017] Furthermore, the step 109 further performs feature summation operations on the 8th layer, the 11th layer and the 14th layer.
[0018] In a second aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the multi-atlas label fusion method as described in any one of the first aspects are implemented.
[0019] In a third aspect, the present invention provides a multi-atlas label fusion system, comprising:
[0020] one or more processors;
[0021] Memory; and
[0022] One or more computer programs, wherein the one or more computer programs are stored in the memory and are configured to be executed by the one or more processors, and when the processor executes the computer program, the steps of the multi-atlas label fusion method as described in any one of the first aspects are implemented.
[0023] This patent introduces a map containing segmentation labels and expert prior knowledge into a deep learning network, and topologically aligns the map with the image to be segmented through differential homeomorphism registration technology, thereby improving the network's ability to learn features of the same organs or parts in the image, thereby further improving the segmentation accuracy of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 : Schematic diagram of the multi-atlas label fusion model architecture of the present invention.
[0025] Figure 2 : Flowchart of the multi-atlas label fusion method of the present invention.
[0026] Figure 3 : Schematic diagram of the composition of the multi-atlas label fusion system of the present invention.
[0027] Figure 4 : Schematic diagram of the composition of the multi-atlas label fusion device of the present invention. DETAILED DESCRIPTION
[0028] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0029] refer to Figure 1 Schematic diagram of the multi-atlas label fusion model architecture of the present invention and reference Figure 3 Schematic diagram of the composition of the multi-atlas label fusion system of the present invention. The multi-atlas label fusion model proposed in the present invention includes two modules: a multi-atlas differential homeomorphism registration module 201 and a multi-path encoding and decoding module 202. The purpose of the multi-atlas differential homeomorphism registration module 201 is to align the labeled atlas (including the label image and the atlas image) to the target image to be segmented. Specifically, the input of the network is the atlas image and the image to be segmented, and the output is the velocity field. The velocity field generates a differential homeomorphism transformation after integration. The differential homeomorphism transformation has the performance of maintaining the topological structure before and after image registration. The differential homeomorphism transformation can act on the atlas image and the label image to align them with the image to be segmented. Then, the target image to be segmented and the transformed labeled atlas are subjected to multi-path and multi-layer Convolution operation, the generated feature map is further sent to the multi-path encoding and decoding module 202 through cascade operation for feature fusion. The multi-path encoding and decoding module 202 further learns the feature map generated by the multi-atlas differential homeomorphism registration module 201. The multi-path encoding and decoding module 202 includes an upper and lower branch. The upper branch input is the cascade feature map of the target image to be segmented and the cascade feature map of the labeled atlas feature map after registration. The lower branch is only the cascade feature map of the labeled atlas feature map after registration. The purpose is to further enhance the deep fusion of the target image features to be segmented and the multi-atlas features. The multi-path encoding and decoding module 202 performs feature summation operations at the 8th layer, the 11th layer and the 14th layer respectively, and finally outputs the predicted segmentation result through the softmax layer.
[0030] The multi-atlas label fusion model proposed in the present invention is trained in an end-to-end manner. The loss function specifically includes two parts: the multi-atlas registration network training loss function term and the segmentation loss function term, and its expression is shown in the following formula 1:
[0031]
[0032] in,
[0033] φ represents the diffeomorphic transformation φ=∫0 1 v t dt,p represents the predicted segmentation result.
[0034] refer to Figure 2Flow chart of the multi-atlas label fusion method of the present invention. The multi-atlas label fusion method of the present invention comprises the following steps:
[0035] Step 101: Obtain a labeled atlas, wherein the labeled atlas includes a label image and an atlas image;
[0036] Step 102: Registration network processing;
[0037] Step 103: Integrate the generated velocity field;
[0038] Step 104: Generate a diffeomorphic transformation;
[0039] Step 105: Align the atlas image, the label image and the target image to be segmented;
[0040] Step 106: Acquire the target image to be segmented;
[0041] Step 107: multi-path multi-layer convolution operation, specifically: the multi-path encoding and decoding module 202 includes an upper and lower branch, the upper branch input is a cascade feature map of the target image to be segmented and the feature map of the labeled spectrum after registration, and the lower branch is only a cascade feature map of the labeled spectrum feature map after registration;
[0042] Step 108: Generate a feature map and perform cascade processing;
[0043] Step 109: feature fusion and further learning, specifically: the multi-path encoding and decoding module 202 performs feature summation operations at the 8th layer, the 11th layer and the 14th layer respectively;
[0044] Step 110: Output the predicted segmentation result through the softmax layer.
[0045] refer to Figure 4 Schematic diagram of the composition of the multi-atlas label fusion device of the present invention. The multi-atlas label fusion device 10 of the present invention further includes one or more memories 20 and one or more processors 30, wherein the one or more computer programs are stored in the memories 20 and are configured to be executed by the one or more processors 30, and the processor 30 implements the steps of the multi-atlas label fusion device method when executing the computer program.
[0046] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0047] This patent introduces a map containing segmentation labels and expert prior knowledge into a deep learning network, and topologically aligns the map with the image to be segmented through differential homeomorphism registration technology, thereby improving the network's ability to learn features of the same organs or parts in the image, thereby further improving the segmentation accuracy of the deep learning model.
[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A multi-atlas label fusion method for medical images, It is characterized in that The following steps are involved: Step 101: Obtain a labeled graph; Step 102: Registration network processing; Step 103: Integrate the generated velocity field; Step 104: Generate a diffeomorphic transformation; Step 105: Align the atlas image, the label image and the target image to be segmented; Step 106: Acquire the target image to be segmented; Step 107: multi-channel and multi-layer convolution operation; Step 108: Generate a feature map and perform cascade processing; Step 109: feature fusion and further learning; Step 110: Output the predicted segmentation result through the softmax layer.
2. The multi-atlas label fusion method according to claim 1, It is characterized in that The labeled atlas includes a label image and an atlas image.
3. The multi-atlas label fusion method according to claim 1, It is characterized in that The step 107 specifically includes two branches, the upper branch input is the cascade feature map of the target image to be segmented and the feature map of the labeled atlas after registration, and the lower branch is the cascade feature map of the labeled atlas feature map after registration.
4. The multi-atlas label fusion method according to claim 1, It is characterized in that The step 109 further performs feature summation operations on the 8th, 11th and 14th layers.
5. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the steps of the multi-atlas label fusion method according to any one of claims 1 to 4 are implemented.
6. A multi-atlas label fusion system, include: one or more processors; Memory; as well as One or more computer programs, wherein the one or more computer programs are stored in the memory and are configured to be executed by the one or more processors, wherein the processor implements the steps of the multi-atlas label fusion method as described in any one of claims 1 to 4 when executing the computer program.
Citation Information
Patent Citations
Magnetic resonance image brain structure automatic dividing method based on statistics multi-map registration optimization
CN102646268A
Heart magnetic resonance image right ventricle segmentation calculation device and method based on multiple maps
CN109785340A
Image segmentation method and device based on multi-label fusion, and storage medium
CN115690426A
Image segmentation network training method, training device thereof and image segmentation method
CN115908973A
