Target segmentation method and device based on sequence feature learning
The target segmentation method addresses noise interference and data scarcity issues by enhancing and fusing features using a U-Net network with a sequence feature learning module, improving the accuracy and reducing false positives in small intestinal stromal tumor segmentation.
Patent Information
- Application Number
- CN202411174857.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-13
AI Technical Summary
Traditional medical segmentation models struggle with high false positive rates and low accuracy in segmenting small intestinal stromal tumors due to strong noise interference and limited labeled data, especially in plain CT images, leading to poor generalization ability.
A target segmentation method utilizing a U-Net network with a multi-scale feature enhancement module, small intestine branch segmentation decoder, fusion module, and sequence feature learning module to enhance and fuse features, learning sequence relationships between CT slices to improve segmentation accuracy.
The method reduces false positive rates and enhances segmentation accuracy by leveraging the attention of the small intestine region and introducing three-dimensional spatial information, improving the model's ability to classify and predict small intestinal stromal tumors.
Smart Images
Figure CN119131388A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular, to a method and device for target segmentation based on sequence feature learning. Background Art
[0002] Common imaging examination methods include computed tomography (CT), magnetic resonance imaging (MRI), and endoscopic ultrasound (EUS). Currently, the task of segmenting small intestinal stromal tumors in plain CT images has the following problems to be solved: (1) Traditional medical segmentation models cannot effectively classify the target and background noise in an environment with strong noise interference, which leads to a high false positive rate in the segmentation results. (2) Traditional medical segmentation models cannot effectively extract the boundary information of the target area, which leads to a large difference between the segmentation result and the target area, and the segmentation prediction accuracy is low. (3) Due to the confidentiality of medical images, there is less data on small intestinal stromal tumors in plain CT images, and even less labeled image data, which is extremely difficult to obtain. The deep learning model trained using small sample data has poor generalization ability. Summary of the invention
[0003] In order to overcome at least one deficiency in the prior art, the present application provides a target segmentation method and device based on sequence feature learning.
[0004] In a first aspect, a target segmentation method based on sequence feature learning is provided, comprising:
[0005] Construct a target segmentation model; the target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder, a fusion module, and a sequence feature learning module. The U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to enhance the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine region based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between the features of different slices at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output the target segmentation result based on the features containing sequence information;
[0006] Train the target segmentation model to obtain a trained target segmentation model;
[0007] The image to be segmented is input into the trained target segmentation model to obtain the target segmentation result.
[0008] In one embodiment, the encoder includes five coding layers, which are respectively recorded as the first coding layer, the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer; the multi-scale feature enhancement module includes four feature enhancement units; the outputs of the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer are respectively input into the four feature enhancement units for feature enhancement to obtain four enhanced features.
[0009] In one embodiment, the fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit;
[0010] The four enhanced features output by the multi-scale feature enhancement module are input into four fusion units respectively;
[0011] The four enhanced features output by the multi-scale feature enhancement module are input into the small intestine branch segmentation decoder to obtain the segmented features;
[0012] The segmented features are input to the first fusion unit, the output of the first fusion unit is input to the second fusion unit, the output of the second fusion unit is input to the third fusion unit, and the output of the third fusion unit is input to the fourth fusion unit.
[0013] In one embodiment, the sequence feature learning module includes three learning units, which are respectively recorded as a first learning unit, a second learning unit and a third learning unit;
[0014] The fusion module includes four fusion units, which are respectively denoted as the first fusion unit, the second fusion unit, the third fusion unit, and the fourth fusion unit; the output of each fusion unit includes 3 slice features;
[0015] The decoder includes four decoding layers, denoted as a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer;
[0016] The last slice feature output by the first fusion unit is input to the third learning unit, and the output of the second fusion unit is input to the third learning unit;
[0017] The output of the third learning unit is input into the second learning unit, and the output of the third fusion unit is input into the second learning unit;
[0018] The output of the second learning unit is input to the first learning unit, the output of the third fusion unit is input to the second learning unit; the output of the fourth fusion unit is input to the first learning unit;
[0019] The output of the first learning unit is concatenated with the feature of the last slice output by the fourth fusion unit, and the concatenated result is input into the third decoding layer;
[0020] The output of the second learning unit is concatenated with the feature of the last slice output by the third fusion unit, and the concatenated result is input into the second decoding layer;
[0021] The output of the third learning unit is concatenated with the feature of the last slice output by the second fusion unit, and the concatenated result is input into the first decoding layer;
[0022] The output of the first encoding layer is input to the fourth decoding layer;
[0023] The features of the last slice of the output of the first fusion unit are input to the first decoding layer, the output of the first decoding layer is input to the second decoding layer; the output of the second decoding layer is input to the third decoding layer; the output of the third decoding layer is input to the fourth decoding layer.
[0024] In one embodiment, the sequence feature learning module includes three learning units, and the input of each learning unit includes two parts. The first part is the output of the fusion unit, including the features of three slices, namely, feature X1, feature X2, and feature X3; the second part is the output of the previous learning unit or the output of the first fusion unit, namely, feature H0 and feature C0;
[0025] Each learning unit includes three Conv-LSTM units, which are recorded as the first Conv-LSTM unit, the second Conv-LSTM unit, and the third Conv-LSTM unit;
[0026] After feature H0 and feature X1 are concatenated, they are input into the first Conv-LSTM unit, and feature C0 is input into the first Conv-LSTM unit; the first Conv-LSTM unit outputs feature H1 and feature C1;
[0027] After the feature H1, feature H0, and feature X2 are concatenated, they are input into the second Conv-LSTM unit; after the feature C1 and feature C0 are concatenated, they are input into the second Conv-LSTM unit; the second Conv-LSTM unit outputs feature H2 and feature C2;
[0028] After feature H2, feature H1, feature H0, and feature X3 are concatenated, they are input into the third Conv-LSTM unit; after feature C2, feature C1, and feature C0 are concatenated, they are input into the third Conv-LSTM unit; the third Conv-LSTM unit outputs feature H3 and feature C3.
[0029] In a second aspect, a target segmentation device based on sequence feature learning is provided, comprising:
[0030] A model building module is used to build a target segmentation model; the target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder, a fusion module, and a sequence feature learning module, and the U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to enhance the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine region based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between the features of different slices at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output the target segmentation result based on the features containing sequence information;
[0031] A model training module is used to train the target segmentation model to obtain a trained target segmentation model;
[0032] The segmentation module is used to input the image to be segmented into the trained target segmentation model to obtain the target segmentation result.
[0033] In one embodiment, the encoder includes five coding layers, which are respectively recorded as the first coding layer, the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer; the multi-scale feature enhancement module includes four feature enhancement units; the outputs of the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer are respectively input into the four feature enhancement units for feature enhancement to obtain four enhanced features.
[0034] In one embodiment, the fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit;
[0035] The four enhanced features output by the multi-scale feature enhancement module are input into four fusion units respectively;
[0036] The four enhanced features output by the multi-scale feature enhancement module are input into the small intestine branch segmentation decoder to obtain the segmented features;
[0037] The segmented features are input to the first fusion unit, the output of the first fusion unit is input to the second fusion unit, the output of the second fusion unit is input to the third fusion unit, and the output of the third fusion unit is input to the fourth fusion unit.
[0038] In one embodiment, the sequence feature learning module includes three learning units, which are respectively recorded as a first learning unit, a second learning unit and a third learning unit;
[0039] The fusion module includes four fusion units, which are respectively denoted as the first fusion unit, the second fusion unit, the third fusion unit, and the fourth fusion unit; the output of each fusion unit includes 3 slice features;
[0040] The decoder includes four decoding layers, denoted as a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer;
[0041] The last slice feature output by the first fusion unit is input to the third learning unit, and the output of the second fusion unit is input to the third learning unit;
[0042] The output of the third learning unit is input into the second learning unit, and the output of the third fusion unit is input into the second learning unit;
[0043] The output of the second learning unit is input to the first learning unit, the output of the third fusion unit is input to the second learning unit; the output of the fourth fusion unit is input to the first learning unit;
[0044] The output of the first learning unit is concatenated with the feature of the last slice output by the fourth fusion unit, and the concatenated result is input into the third decoding layer;
[0045] The output of the second learning unit is concatenated with the feature of the last slice output by the third fusion unit, and the concatenated result is input into the second decoding layer;
[0046] The output of the third learning unit is concatenated with the feature of the last slice output by the second fusion unit, and the concatenated result is input into the first decoding layer;
[0047] The output of the first encoding layer is input to the fourth decoding layer;
[0048] The features of the last slice of the output of the first fusion unit are input to the first decoding layer, the output of the first decoding layer is input to the second decoding layer; the output of the second decoding layer is input to the third decoding layer; the output of the third decoding layer is input to the fourth decoding layer.
[0049] In one embodiment, the sequence feature learning module includes three learning units, and the input of each learning unit includes two parts. The first part is the output of the fusion unit, including the features of three slices, namely, feature X1, feature X2, and feature X3; the second part is the output of the previous learning unit or the output of the first fusion unit, namely, feature H0 and feature C0;
[0050] Each learning unit includes three Conv-LSTM units, which are recorded as the first Conv-LSTM unit, the second Conv-LSTM unit, and the third Conv-LSTM unit;
[0051] After feature H0 and feature X1 are concatenated, they are input into the first Conv-LSTM unit, and feature C0 is input into the first Conv-LSTM unit; the first Conv-LSTM unit outputs feature H1 and feature C1;
[0052] After the feature H1, feature H0, and feature X2 are concatenated, they are input into the second Conv-LSTM unit; after the feature C1 and feature C0 are concatenated, they are input into the second Conv-LSTM unit; the second Conv-LSTM unit outputs feature H2 and feature C2;
[0053] After feature H2, feature H1, feature H0, and feature X3 are concatenated, they are input into the third Conv-LSTM unit; after feature C2, feature C1, and feature C0 are concatenated, they are input into the third Conv-LSTM unit; the third Conv-LSTM unit outputs feature H3 and feature C3.
[0054] Compared with the prior art, the present application has the following beneficial effects: the present application fuses the segmented features and enhanced features through a fusion module, so that the model can use the attention of the small intestine region to guide the segmentation of small intestinal stromal tumors, reduce the impact of background noise on the segmentation, thereby improving the segmentation accuracy of the model and reducing the false positive rate. In addition, a sequence feature learning module is proposed, through which the sequence features between consecutive CT slices are learned, three-dimensional spatial information is introduced into the network, and the feature expression of the lesion area is enriched, thereby improving the segmentation accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] The present application may be better understood by referring to the following description given in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification. In the drawings:
[0056] Figure 1 A schematic diagram of the target segmentation model is shown;
[0057] Figure 2 A schematic diagram of the feature enhancement unit MSE is shown;
[0058] Figure 3 A schematic diagram of a small intestine branch segmentation decoder SID is shown;
[0059] Figure 4 A schematic diagram of a sequence feature learning module is shown;
[0060] Figure 5 The visual comparison results between the method of the present application and the existing model are shown. DETAILED DESCRIPTION
[0061] The exemplary embodiments of the present application will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions can be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.
[0062] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, only the device structure closely related to the scheme according to the present application is shown in the drawings, while other details that are not closely related to the present application are omitted.
[0063] It should be understood that the present application is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.
[0064] The present application embodiment provides a target segmentation method based on sequence feature learning, where the target may be, for example, a small intestinal stromal tumor, including:
[0065] Step S1, constructing a target segmentation model; Figure 1 A schematic diagram of the target segmentation model is shown in Figure 1 The target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder SID, a fusion module, and a sequence feature learning module. The U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to enhance the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine area based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between the features of different slices at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output the target segmentation result based on the features containing sequence information.
[0066] Here, the enhanced features extracted by the multi-scale feature enhancement module are applied to the small intestine branch segmentation decoder SID and the fusion module. The small intestine branch segmentation decoder SID is designed to solve the multi-level semantic consistency problem in small intestine segmentation. The output of the small intestine branch segmentation decoder SID will be used as the range of the small intestine area and input into the fusion module to assist the small intestinal stromal tumor segmentation task.
[0067] Step S2, training the target segmentation model to obtain a trained target segmentation model.
[0068] First, a model training data set was constructed. The samples in the model training data set were CT slices of patients with small intestinal stromal tumors. The input data during the model training process were 3 consecutive slices.
[0069] Here, the loss function Loss used in the model training process all for:
[0070]
[0071] Among them, DiceLoss si , BCELoss si DiceLoss is the Dice loss and binary cross entropy loss (BCELoss) constructed based on the output of the small intestine branch segmentation decoder SID. sist , BCELoss sist They are Dice loss and binary cross entropy loss (BCELoss) constructed based on the output of the target segmentation model, respectively.
[0072] Step S3, inputting the image to be segmented into the trained target segmentation model to obtain the target segmentation result.
[0073] This embodiment uses a fusion module to fuse the segmented features and enhanced features, so that the model can use the attention of the small intestine region to guide the segmentation of small intestinal stromal tumors, reduce the impact of background noise on the segmentation, thereby improving the segmentation accuracy of the model and reducing the false positive rate. In addition, a sequence feature learning module is proposed, through which the sequence features between consecutive CT slices are learned, three-dimensional spatial information is introduced into the network, and the feature expression of the lesion area is enriched, thereby improving the segmentation accuracy of the model.
[0074] In one embodiment, the encoder includes five encoding layers, which are respectively recorded as the first encoding layer, the second encoding layer, the third encoding layer, the fourth encoding layer and the fifth encoding layer; the multi-scale feature enhancement module includes four feature enhancement units MSE; the outputs E2, E3, E4, and E5 of the second encoding layer, the third encoding layer, the fourth encoding layer and the fifth encoding layer are respectively input into the four feature enhancement units MSE for feature enhancement to obtain four enhanced features. Here, Figure 2 A schematic diagram of the feature enhancement unit MSE is shown, where CBR is Conv+BN+RELU, i.e., convolution+batch normalization+RELU activation function, and DConv is dynamic convolution.
[0075] In one embodiment, the fusion module includes four fusion units SCB, which are respectively denoted as a first fusion unit SCB1, a second fusion unit SCB2, a third fusion unit SCB3, and a fourth fusion unit SCB4;
[0076] The four enhanced features output by the multi-scale feature enhancement module are respectively input into four fusion units, and each fusion unit realizes a fusion function, for example, the segmented features and enhanced features corresponding to each slice can be spliced;
[0077] The four enhanced features output by the multi-scale feature enhancement module are input into the small intestine branch segmentation decoder SID to obtain the segmented features; Figure 3 A schematic diagram of the small intestine branch segmentation decoder SID is shown.
[0078] The segmented features are input to the first fusion unit SCB1, the output S1 of the first fusion unit is input to the second fusion unit SCB2, the output S2 of the second fusion unit SCB2 is input to the third fusion unit SCB3, and the output S3 of the third fusion unit SCB3 is input to the fourth fusion unit SCB4.
[0079] In one embodiment, the sequence feature learning module includes three learning units MS-LSTM, which are respectively recorded as the first learning unit MS-LSTM1, the second learning unit MS-LSTM2 and the third learning unit MS-LSTM3;
[0080] The fusion module includes four fusion units SCB, which are respectively denoted as the first fusion unit SCB1, the second fusion unit SCB2, the third fusion unit SCB3, and the fourth fusion unit SCB4; the output of each fusion unit includes 3 slice features;
[0081] The decoder includes four decoding layers, denoted as a first decoding layer UP1, a second decoding layer UP2, a third decoding layer UP3 and a fourth decoding layer UP4;
[0082] The last slice feature of the output S1 of the first fusion unit SCB1 is input to the third learning unit MS-LSTM3, and the output S2 of the second fusion unit CB2 is input to the third learning unit MS-LSTM3;
[0083] The output of the third learning unit MS-LSTM3 is input to the second learning unit MS-LSTM2, and the output S3 of the third fusion unit MS-LSTM3 is input to the second learning unit MS-LSTM2;
[0084] The output of the second learning unit MS-LSTM2 is input to the first learning unit MS-LSTM1, the output S3 of the third fusion unit SCB3 is input to the second learning unit MS-LSTM2; the output S4 of the fourth fusion unit SCB4 is input to the first learning unit MS-LSTM1;
[0085] The output of the first learning unit MS-LSTM1 is concatenated with the feature of the last slice of the output S4 of the fourth fusion unit SCB4, and the concatenated result is input into the third decoding layer UP3;
[0086] The output of the second learning unit MS-LSTM2 is concatenated with the features of the last slice of the output S3 of the third fusion unit SCB3, and the concatenated result is input into the second decoding layer UP2;
[0087] The output of the third learning unit MS-LSTM3 is concatenated with the features of the last slice of the output S2 of the second fusion unit SCB2, and the concatenated result is input into the first decoding layer UP1;
[0088] The output of the first encoding layer E1 is input to the fourth decoding layer UP4;
[0089] The features of the last slice of the output S1 of the first fusion unit SCB1 are input to the first decoding layer UP1, the output of the first decoding layer UP1 is input to the second decoding layer UP2; the output of the second decoding layer UP2 is input to the third decoding layer UP3; the output of the third decoding layer UP3 is input to the fourth decoding layer UP4.
[0090] In this embodiment, three MS-LSTM units are set in parallel at the jump connection part of the U-Net network. In order to allow the MS-LSTM unit to learn sequence information, this embodiment uniformly sorts and organizes the features of the sequence slices according to different levels, and passes them to the MS-LSTM unit in the form of an array for the MS-LSTM unit to traverse and learn. For the feature E1 output of the first coding layer, since the feature size of this level is large, setting an MS-LSTM unit will increase a lot of calculations, so this embodiment does not set a sequence learning module at this level. The MS-LSTM unit processes the three slices in sequence, retains the feature part learned from the previous slice and applies it to the learning of the next slice, and so on, so that the sequence features are continuously enriched during the learning process. This method enables the MS-LSTM unit to fully learn the sequence relationship between the features of different slices at the same level, thereby providing richer spatial information for the prediction of the last slice. At the same time, the MS-LSTM unit transmits H x and C x The features of different levels are fused in a parameterized manner, where the high-level features containing sequence information are used as initialization parameters for the low-level MS-LSTM units, thereby providing more high-level semantic information for the low-level MS-LSTM units to improve the network's ability to learn global features. The output of each MS-LSTM unit is concatenated with the features of the last slice output by the fusion unit, and the concatenation result is used as the input of the decoder to provide sequence information for the final segmentation.
[0091] In one embodiment, Figure 4 The schematic diagram of the sequence feature learning module is shown. The sequence feature learning module includes three learning units, and the input of each learning unit includes two parts. The first part is the output of the fusion unit, including the features of three slices, namely feature X1, feature X2 and feature X3; the second part is the output of the previous learning unit or the output of the first fusion unit, namely feature H0 and feature C0;
[0092] Each learning unit includes three Conv-LSTM units (convolutional long short-term memory network), which are recorded as the first Conv-LSTM unit, the second Conv-LSTM unit, and the third Conv-LSTM unit;
[0093] After feature H0 and feature X1 are concatenated, they are input into the first Conv-LSTM unit, and feature C0 is input into the first Conv-LSTM unit; the first Conv-LSTM unit outputs feature H1 and feature C1;
[0094] After the feature H1, feature H0, and feature X2 are concatenated, they are input into the second Conv-LSTM unit; after the feature C1 and feature C0 are concatenated, they are input into the second Conv-LSTM unit; the second Conv-LSTM unit outputs feature H2 and feature C2;
[0095] After feature H2, feature H1, feature H0, and feature X3 are concatenated, they are input into the third Conv-LSTM unit; after feature C2, feature C1, and feature C0 are concatenated, they are input into the third Conv-LSTM unit; the third Conv-LSTM unit outputs feature H3 and feature C3.
[0096] In this embodiment, the designed MS-LSTM module based on the improved Conv-LSTM module is applied to the jump connection part of the U-Net network, and the MS-LSTM unit learns the continuous sequence information of the same-level features by recursively and continuously slicing the same-level features to improve the segmentation performance of the model. In order to further utilize the high-level semantic information of high-level features, this embodiment constructs a multi-level feature transfer MS-LSTM unit to fuse the continuous sequence information between different levels of features.
[0097] Since the sequence learning module is constantly accumulating sequence feature information during sequence learning, the prediction result of the last slice is often better than other slices in the sequence. For this reason, this embodiment chooses to output only the prediction result of the last slice in the slice group as the output of the network to reduce the number of network parameters and reduce the burden on the device.
[0098] In order to further verify the effectiveness of the method of this application, the following experimental analysis was carried out.
[0099] We designed a comparative experiment between the MS-LSTM module based on continuous slice input, the baseline model MTAF-Net using single slice input, and the best model SiNetV2. Table 1 shows the quantitative comparison results of the experiment. The visual comparison results are shown in Figure 1. Figure 5 shown.
[0100] Table 1
[0101]
[0102] Judging from the quantitative comparison results, the method of the present application has produced significant results for the task of segmenting small intestinal stromal tumors in plain CT images. Although the prediction accuracy of the baseline model is relatively high, the method of the present application has still achieved certain improvements on this basis, with the Dice coefficient and IoU indicators increased by 1.35% and 1.88% respectively. Compared with the SEN indicator, the method of the present application has improved the PRE indicator by 2.35%, which is a significant improvement. It is proved that the target segmentation method based on sequence feature learning in the present application can allow the model to predict the changes of small intestinal stromal tumors by learning the sequence information between slices, reducing the prediction probability of abnormal targets with continuous changes, thereby reducing the false positive rate.
[0103] Figure 5 The visual comparison results between the method of the present application and the existing model are shown. Figure 5 It can be seen that the method of the present application segmented fewer false positive parts in the four segmentation results shown, and has a stronger effect on suppressing false positives, which also confirms the conclusion drawn from the quantitative comparative analysis.
[0104] Using the same inventive concept as the target segmentation method based on sequence feature learning, this embodiment also provides a corresponding target segmentation device based on sequence feature learning, including:
[0105] A model building module is used to build a target segmentation model; the target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder, a fusion module, and a sequence feature learning module, and the U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to enhance the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine region based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between the features of different slices at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output the target segmentation result based on the features containing sequence information;
[0106] A model training module is used to train the target segmentation model to obtain a trained target segmentation model;
[0107] The segmentation module is used to input the image to be segmented into the trained target segmentation model to obtain the target segmentation result.
[0108] The target segmentation device based on sequence feature learning of this embodiment has the same inventive concept as the target segmentation method based on sequence feature learning mentioned above. Therefore, the specific implementation method of the device can be seen in the implementation example part of the target segmentation method based on sequence feature learning mentioned above, and its technical effect corresponds to the technical effect of the above method, which will not be repeated here.
[0109] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A target segmentation method based on sequence feature learning, characterized in that: include: Constructing a target segmentation model; the target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder, a fusion module, and a sequence feature learning module, and the U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to perform feature enhancement on the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine region based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between different slice features at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output a target segmentation result based on the features containing sequence information; Training the target segmentation model to obtain a trained target segmentation model; The image to be segmented is input into the trained target segmentation model to obtain a target segmentation result.
2. The method according to claim 1, characterized in that The encoder includes five encoding layers, which are respectively recorded as a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer and a fifth encoding layer; the multi-scale feature enhancement module includes four feature enhancement units; The outputs of the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer are respectively input into the four feature enhancement units for feature enhancement to obtain four enhanced features.
3. The method according to claim 1, characterized in that The fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit; The four enhanced features output by the multi-scale feature enhancement module are respectively input into the four fusion units; The four enhanced features output by the multi-scale feature enhancement module are input into the small intestine branch segmentation decoder to obtain segmented features; The segmented features are input to the first fusion unit, the output of the first fusion unit is input to the second fusion unit, the output of the second fusion unit is input to the third fusion unit, and the output of the third fusion unit is input to the fourth fusion unit.
4. The method according to claim 1, characterized in that The sequence feature learning module includes three learning units, which are respectively recorded as the first learning unit, the second learning unit and the third learning unit; The fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit; the output of each fusion unit includes 3 slice features; The decoder includes four decoding layers, denoted as a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer; The last slice feature output by the first fusion unit is input to the third learning unit, and the output of the second fusion unit is input to the third learning unit; The output of the third learning unit is input into the second learning unit, and the output of the third fusion unit is input into the second learning unit; The output of the second learning unit is input to the first learning unit, the output of the third fusion unit is input to the second learning unit; the output of the fourth fusion unit is input to the first learning unit; The output of the first learning unit is concatenated with the feature of the last slice output by the fourth fusion unit, and the concatenated result is input into the third decoding layer; The output of the second learning unit is concatenated with the feature of the last slice output by the third fusion unit, and the concatenated result is input into the second decoding layer; The output of the third learning unit is concatenated with the feature of the last slice output by the second fusion unit, and the concatenated result is input into the first decoding layer; The output of the first encoding layer is input to the fourth decoding layer; The feature of the last slice output by the first fusion unit is input to the first decoding layer, and the output of the first decoding layer is input to the second decoding layer; The output of the second decoding layer is input to the third decoding layer; The output of the third decoding layer is input to the fourth decoding layer.
5. The method according to claim 1, characterized in that The sequence feature learning module includes three learning units, and the input of each learning unit includes two parts. The first part is the output of the fusion unit, including the features of three slices, namely feature X1, feature X2 and feature X3; the second part is the output of the previous learning unit or the output of the first fusion unit, namely feature H0 and feature C0; Each of the learning units includes three Conv-LSTM units, which are recorded as the first Conv-LSTM unit, the second Conv-LSTM unit, and the third Conv-LSTM unit; After the feature H0 and the feature X1 are concatenated, they are input into the first Conv-LSTM unit, and the feature C0 is input into the first Conv-LSTM unit; the first Conv-LSTM unit outputs the feature H1 and the feature C1; The feature H1, the feature H0, and the feature X2 are concatenated and input into the second Conv-LSTM unit; the feature C1 and the feature C0 are concatenated and input into the second Conv-LSTM unit; the second Conv-LSTM unit outputs the feature H2 and the feature C2; The features H2, H1, H0, and X3 are concatenated and input into the third Conv-LSTM unit; the features C2, C1, and C0 are concatenated and input into the third Conv-LSTM unit; the third Conv-LSTM unit outputs features H3 and C3.
6. A target segmentation device based on sequence feature learning, characterized in that: include: A model building module is used to build a target segmentation model; the target segmentation model includes a U-Net network, a multi-scale feature enhancement module, a small intestine branch segmentation decoder, a fusion module, and a sequence feature learning module, and the U-Net network includes an encoder and a decoder; the multi-scale feature enhancement module is used to perform feature enhancement on the features output by the encoder to obtain enhanced features; the small intestine branch segmentation decoder is used to segment the small intestine region based on the enhanced features to obtain segmented features; the fusion module is used to fuse the segmented features and the enhanced features to obtain fused features; the sequence feature learning module is used to learn the sequence relationship between different slice features at the same level based on the fused features to obtain features containing sequence information; the decoder is used to output a target segmentation result based on the features containing sequence information; A model training module, used to train the target segmentation model to obtain a trained target segmentation model; The segmentation module is used to input the image to be segmented into the trained target segmentation model to obtain the target segmentation result.
7. The device according to claim 6, characterized in that The encoder includes five encoding layers, which are respectively recorded as a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer and a fifth encoding layer; the multi-scale feature enhancement module includes four feature enhancement units; The outputs of the second coding layer, the third coding layer, the fourth coding layer and the fifth coding layer are respectively input into the four feature enhancement units for feature enhancement to obtain four enhanced features.
8. The device according to claim 6, characterized in that The fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit; The four enhanced features output by the multi-scale feature enhancement module are respectively input into the four fusion units; The four enhanced features output by the multi-scale feature enhancement module are input into the small intestine branch segmentation decoder to obtain segmented features; The segmented features are input to the first fusion unit, the output of the first fusion unit is input to the second fusion unit, the output of the second fusion unit is input to the third fusion unit, and the output of the third fusion unit is input to the fourth fusion unit.
9. The device according to claim 1, characterized in that The sequence feature learning module includes three learning units, which are respectively recorded as the first learning unit, the second learning unit and the third learning unit; The fusion module includes four fusion units, which are respectively recorded as a first fusion unit, a second fusion unit, a third fusion unit, and a fourth fusion unit; the output of each fusion unit includes 3 slice features; The decoder includes four decoding layers, denoted as a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer; The last slice feature output by the first fusion unit is input to the third learning unit, and the output of the second fusion unit is input to the third learning unit; The output of the third learning unit is input into the second learning unit, and the output of the third fusion unit is input into the second learning unit; The output of the second learning unit is input to the first learning unit, the output of the third fusion unit is input to the second learning unit; the output of the fourth fusion unit is input to the first learning unit; The output of the first learning unit is concatenated with the feature of the last slice output by the fourth fusion unit, and the concatenated result is input into the third decoding layer; The output of the second learning unit is concatenated with the feature of the last slice output by the third fusion unit, and the concatenated result is input into the second decoding layer; The output of the third learning unit is concatenated with the feature of the last slice output by the second fusion unit, and the concatenated result is input into the first decoding layer; The output of the first encoding layer is input to the fourth decoding layer; The feature of the last slice output by the first fusion unit is input to the first decoding layer, and the output of the first decoding layer is input to the second decoding layer; The output of the second decoding layer is input to the third decoding layer; The output of the third decoding layer is input to the fourth decoding layer.
10. The device according to claim 1, characterized in that The sequence feature learning module includes three learning units, and the input of each learning unit includes two parts. The first part is the output of the fusion unit, including the features of three slices, namely feature X1, feature X2 and feature X3; the second part is the output of the previous learning unit or the output of the first fusion unit, namely feature H0 and feature C0; Each of the learning units includes three Conv-LSTM units, which are recorded as the first Conv-LSTM unit, the second Conv-LSTM unit, and the third Conv-LSTM unit; After the feature H0 and the feature X1 are concatenated, they are input into the first Conv-LSTM unit, and the feature C0 is input into the first Conv-LSTM unit; the first Conv-LSTM unit outputs the feature H1 and the feature C1; The feature H1, the feature H0, and the feature X2 are concatenated and input into the second Conv-LSTM unit; the feature C1 and the feature C0 are concatenated and input into the second Conv-LSTM unit; the second Conv-LSTM unit outputs the feature H2 and the feature C2; The features H2, H1, H0, and X3 are concatenated and input into the third Conv-LSTM unit; the features C2, C1, and C0 are concatenated and input into the third Conv-LSTM unit; the third Conv-LSTM unit outputs features H3 and C3.