Feature dimension raising method for 3D volumetric medical image segmentation
By organizing interneighbor data and master and secondary encoder feature extraction, integrating information and enhancing features, embedding 2D networks, the problems of high computing cost and poor versatility of existing 3D volume medical image segmentation methods are solved, and efficient and accurate 3D is achieved Volume medical image segmentation.
Patent Information
- Application Number
- CN202411052089.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
AI Technical Summary
The existing 3D volume medical image segmentation methods have high computational cost and training difficulty, and it is difficult to make full use of spatial information, poor versatility, and difficult to integrate useful information.
By organizing interneighbor data, master and secondary encoder feature extraction, interneighbor information integration and master feature enhancement, fusion features are obtained and embedded in the 2D network, upgrading the 2D segmentation network method so that it can directly connect to the 2D network without modification .
It significantly improves the accuracy and efficiency of 3D volume medical image segmentation, can make full use of the spatial information of volume data, and has a simple structure and strong versatility, so it can be easily embedded in 2D networks.
Smart Images

Figure CN119006482A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision and medical image processing, and in particular to a feature dimension enhancement method for 3D volume medical image segmentation. Background Art
[0002] Medical image segmentation has a wide range of applications in clinical medicine and medical research. It can be used for auxiliary diagnosis, pathological analysis, surgical planning, tumor localization and quantification, organ surface extraction and other tasks. Accurate medical image segmentation can help doctors better understand and analyze the patient's condition and provide more accurate diagnosis and treatment decisions. For example, fine segmentation of the heart can help diagnose dilated cardiomyopathy, myocardial infarction, valvular disease, etc. Deep learning has achieved great success in the field of image segmentation, and many segmentation methods have been proposed, including CNN-based methods, Transformer-based methods and other methods. But they segment medical images one by one, that is, in 2D images.
[0003] In recent years, 3D volumetric images such as computed tomography (CT) and magnetic resonance imaging (MRI) have been increasingly used in clinical practice. Although it is feasible to segment each volumetric image into several 2D slices and then use a 2D network method to segment these slices. However, this method does not fully utilize the spatial information of the volume data. Therefore, some 3D segmentation methods have been proposed. Although these methods can use the entire volume data as a sample to obtain spatial background information, 3D segmentation methods generally have high computational costs and training difficulties, which affect their clinical applications. There are also methods that combine the advantages of 2D and 3D methods, but these methods are usually complex in structure, poor in versatility, and difficult to integrate useful information. Therefore, there is an urgent need for a module and method that considers spatial information, has a simple structure, strong versatility, and is plug-and-play. Summary of the invention
[0004] In order to overcome the deficiencies in the prior art, the present invention provides a feature dimensionality upgrading method for 3D volumetric medical image segmentation, which is used to improve the performance of a 2D network method in 3D volumetric medical image segmentation. The module obtains fusion features by organizing inter-adjacent data, extracting features from primary and secondary encoders, integrating inter-adjacent information, and enhancing primary features, and then embeds them into a 2D network to upgrade the 2D segmentation network method. The 2D network can be directly connected without any modification, and plug-and-play is very efficient and has high accuracy.
[0005] In order to achieve the above-mentioned invention object, the technical solution adopted to solve the technical problem is as follows:
[0006] A feature dimension enhancement method for 3D volume medical image segmentation comprises the following steps:
[0007] Step 1, organizing inter-adjacent data: the 3D volume medical image is input into the module, the input 3D volume image is sliced and grouped, and every three adjacent 2D slices are merged into a three-channel sample; the middle slice is used as the main input and label, and the left and right slices are used as auxiliary inputs; the edge slices, that is, the start slice and the end slice, are directly copied into the three-channel sample;
[0008] Step 2, slice feature extraction stage: set three encoders, the middle encoder is the main one, and the two side encoders are auxiliary ones, the encoder structure is unified and the weights are shared;
[0009] Step 3, feature storage stage: set a specific feature map pool and store the extracted features in the feature map pool;
[0010] Step 4, inter-proximal information integration stage: the feature integration part of the design includes integration components and enhancement components;
[0011] Step 5, embedding / upgrading 2D network stage: the extracted fusion features are directly connected with the popular 2D segmentation network.
[0012] Furthermore, in step 1, the inter-adjacent data is organized as follows:
[0013] The size of a 3D volume image can be expressed as H×W×C z , where H, W and C z Represent the number of rows, columns and z-axis slices respectively. The three-channel adjacent slices combined along the z-axis are defined as:
[0014] C n ={c nl ,c nm ,c nr},
[0015] Where n is the index of the slice, 0≤n≤C z , c nl ,c nm and c nr Represents index slice C respectively n Left slice, middle slice and right slice; if n-1 or n+1 exceeds C z If c nl =c nm =c nr , adjacent slice combination C n The label is the middle channel slice c nm .
[0016] Furthermore, in step 2, the slice feature extraction, the intermediate encoder is responsible for extracting features from the intermediate slices m , while the encoders on both sides extract features e from the adjacent left and right slices respectively.l and e r .
[0017] Further, in step 3, the feature storage, for a thickness of C z 3D image, the extracted features are stored in the feature map pool, and the encoders with the same structure and shared weights can repeatedly call and update the features.
[0018] Furthermore, in step 4, the integration component uses the concat method to integrate the spatial information from adjacent features and uses a 1×1 convolution module to retain the channel dimension C. i , the processing method is mathematically expressed as follows:
[0019] e Ri =R BNRi (W(L(e li ,e mi ,e ri ))),
[0020]
[0021] Among them, the variable N represents the total number of levels present in the multi-level encoder, the variable i represents the level index of the multi-level encoder, L(·) represents concat concatenation, the function W(·) represents a 1×1 convolutional layer, and the function R CNR (·) Implemented a feature fusion mechanism involving convolution operations followed by batch normalization and RELU activation, E R Represents the output features from adjacent feature fusion components.
[0022] Furthermore, in step 4, the enhancement component uses residual connection to interconnect the intermediate encoder features with the fusion features to enhance the advantages of the intermediate encoder features.
[0023] Furthermore, in step 5, the 2D network is embedded / upgraded, and the extracted features of the module are directly used as the input of the 2D segmentation network, which is plug-and-play.
[0024] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art:
[0025] 1. The feature dimension-increasing module and method for 3D volume medical image segmentation of the present invention can improve the performance of segmenting 3D volume medical images using a 2D network method;
[0026] 2. The present invention uses a multi-level encoder and a fusion component to perform feature processing to achieve the utilization of spatial information of 3D volumetric medical images;
[0027] 3. The present invention cleverly draws on the RGB segmentation paradigm and proposes a new method for combining adjacent data;
[0028] 4. The weight sharing and feature storage strategies proposed in the present invention make the module and method of the present invention extremely efficient and can be easily embedded in a 2D segmentation network without any modification.
[0029] The present invention is used to embed in a 2D network method to improve the performance of 3D volumetric medical image segmentation. Compared with the traditional 2D network method, by using the feature dimensionality upgrading module and method for 3D volumetric medical image segmentation proposed by the present invention, the spatial information of the volumetric data can be fully utilized, the segmentation accuracy can be significantly improved, and the use is very efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0031] Figure 1 It is a flow chart of a feature dimension-raising method for 3D volume medical image segmentation according to the present invention;
[0032] Figure 2 is a schematic diagram of a network architecture of an embodiment of the present invention;
[0033] Figure 3 is a schematic diagram of the ACDC heart 3D medical image segmentation result according to an embodiment of the present invention;
[0034] Figure 4 It is a schematic diagram of the Synapse abdominal 3D medical image segmentation result according to an embodiment of the present invention. DETAILED DESCRIPTION
[0035] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] like Figure 1-4 As shown, this embodiment discloses a feature dimension enhancement method for 3D volume medical image segmentation, comprising the following steps:
[0037] Step 1, organizing inter-adjacent data: the 3D volume medical image is input into the module, the input 3D volume image is sliced and grouped, and every three adjacent 2D slices are merged into a three-channel sample; the middle slice is used as the main input and label, and the left and right slices are used as auxiliary inputs; the edge slices, that is, the start slice and the end slice, are directly copied into the three-channel sample;
[0038] Specifically, in step 1, the inter-proximal data is organized as follows:
[0039] The size of a 3D volume image can be expressed as H×W×C z , where H, W and C z Represent the number of rows, columns and z-axis slices respectively. The three-channel adjacent slices combined along the z-axis are defined as:
[0040] C n ={c nl ,c nm ,c nr},
[0041] Where n is the index of the slice, 0≤n≤C z , c nl ,c nm and c nr Represents index slice C respectively n Left slice, middle slice and right slice; if n-1 or n+1 exceeds C z If c nl =c nm =c nr , adjacent slice combination C n The label is the middle channel slice c nm .
[0042] Step 2, slice feature extraction stage: set three encoders, the middle encoder is the main one, and the two side encoders are auxiliary ones, the encoder structure is unified and the weights are shared;
[0043] Specifically, in step 2, the slice feature extraction, the intermediate encoder is responsible for extracting features from the intermediate slices m , while the encoders on both sides extract features e from the adjacent left and right slices respectively. l and e r .
[0044] Step 3, feature storage stage: set a specific feature map pool and store the extracted features in the feature map pool;
[0045] Specifically, in step 3, the feature storage, for a thickness of C z3D image, the extracted features are stored in the feature map pool, and the encoders with the same structure and shared weights can repeatedly call and update the features.
[0046] Step 4, inter-proximal information integration stage: the feature integration part of the design includes integration components and enhancement components;
[0047] Specifically, in step 4, the integration component uses the concat method to integrate the spatial information from adjacent features and uses a 1×1 convolution module to retain the channel dimension C i , the processing method is mathematically expressed as follows:
[0048] e Ri =R BNRi (W(L(e li ,e mi ,e ri ))),
[0049]
[0050] Among them, the variable N represents the total number of levels present in the multi-level encoder, the variable i represents the level index of the multi-level encoder, L(·) represents concat concatenation, the function W(·) represents a 1×1 convolutional layer, and the function R CNR (·) Implemented a feature fusion mechanism involving convolution operations followed by batch normalization and RELU activation, E R Represents the output features from adjacent feature fusion components.
[0051] Furthermore, in step 4, the enhancement component uses residual connection to interconnect the intermediate encoder features with the fusion features to enhance the advantages of the intermediate encoder features.
[0052] Step 5, embedding / upgrading 2D network stage: the extracted fusion features are directly connected with the popular 2D segmentation network.
[0053] Specifically, in step 5, the 2D network is embedded / upgraded, and the extracted features of the module are directly used as the input of the 2D segmentation network, which is plug-and-play.
[0054] This embodiment provides a feature dimensionality increase method for 3D volumetric medical image segmentation. The method takes a 3D volumetric medical image as input, and obtains fused features by organizing inter-adjacent data, extracting features of the primary and secondary encoders, integrating inter-adjacent information, and enhancing the primary features. The fused features obtained by this method can be conveniently used in combination with mainstream 2D networks, providing an effective solution for 3D medical image segmentation using 2D networks. At the same time, the present invention introduces weight sharing and feature storage strategies, making the method very efficient. Weight sharing can reduce the amount of network parameters, reduce model complexity, and improve the generalization ability of the model. The feature storage strategy allows previously calculated features to be stored and reused when processing each slice, thereby further improving computational efficiency. The present invention achieves excellent performance on various medical data sets, and has the advantages of high efficiency and high precision.
[0055] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A feature dimension-enhancing method for 3D volumetric medical image segmentation, characterized in that: The following steps are involved: Step 1, organizing inter-adjacent data: the 3D volume medical image is input into the module, the input 3D volume image is sliced and grouped, and every three adjacent 2D slices are merged into a three-channel sample; the middle slice is used as the main input and label, and the left and right slices are used as auxiliary inputs; the edge slices, that is, the start slice and the end slice, are directly copied into the three-channel sample; Step 2, slice feature extraction stage: set three encoders, the middle encoder is the main one, and the two side encoders are auxiliary ones, the encoder structure is unified and the weights are shared; Step 3, feature storage stage: set a specific feature map pool and store the extracted features in the feature map pool; Step 4, inter-proximal information integration stage: the feature integration part of the design includes integration components and enhancement components; Step 5, embedding / upgrading 2D network stage: the extracted fusion features are directly connected with the popular 2D segmentation network.
2. A feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 1, the inter-proximal data is organized as follows: The size of a 3D volume image can be expressed as H×W×C z , where H, W and C z Represent the number of rows, columns and z-axis slices respectively. The three-channel adjacent slices combined along the z-axis are defined as: C n ={c nl ,c nm ,c nr }, Where n is the index of the slice, 0≤n≤C z , c nl ,c nm and c nr Represents index slice C respectively n Left slice, middle slice and right slice; if n-1 or n+1 exceeds C z If c nl =c nm =c nr , adjacent slice combination C n The label is the middle channel slice c nm .
3. The feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 2, the slice feature extraction, the intermediate encoder is responsible for extracting features from the intermediate slices m , while the encoders on both sides extract features e from the adjacent left and right slices respectively. l and e r .
4. The feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 3, the feature storage, for a thickness of C z 3D image, the extracted features are stored in the feature map pool, and the encoders with the same structure and shared weights can repeatedly call and update the features.
5. The feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 4, the integration component uses the concat method to integrate the spatial information from adjacent features and uses a 1×1 convolution module to retain the channel dimension C i , the processing method is mathematically expressed as follows: have been Ri =R BNRi (W(L(e li ,have been mi ,have been ri ))), Wherein, variable N represents the total number of levels present in the multi-level encoder, variable i represents the level index of the multi-level encoder, L(·) represents concat concatenation, function W(·) represents a 1×1 convolutional layer, and function R CNR (·) Implemented a feature fusion mechanism involving convolution operations followed by batch normalization and RELU activation, E R Represents the output features from adjacent feature fusion components.
6. The feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 4, the enhancement component uses residual connection to interconnect the intermediate encoder features with the fusion features to enhance the advantages of the intermediate encoder features.
7. The feature dimension enhancement method for 3D volumetric medical image segmentation according to claim 1, characterized in that: In step 5, the 2D network is embedded / upgraded, and the extracted features of the module are directly used as the input of the 2D segmentation network, which is plug-and-play.
