Lesion region segmentation method and device based on normal tissue image information contrast

Abstract

The application relates to a lesion region segmentation method and device based on normal tissue image information contrast, and belongs to the technical field of image segmentation. The method and device are characterized in that: a to-be-segmented lesion image is input into a pre-constructed VEA network to obtain a single-mode normal image; the to-be-segmented lesion image and the single-mode normal image are input into an encoding-decoding network to obtain first image features extracted from the to-be-segmented lesion image and second image features extracted from the single-mode normal image; the first image features and the second image features are subjected to feature alignment and feature comparison to obtain an attention map; and the attention map is used to enhance lesion region features in the to-be-segmented lesion image to obtain a lesion region segmentation result of the to-be-segmented lesion image. Therefore, by comparing the lesion image with the normal image, the differences between the lesion image and the normal image are fully considered, the lesion is accurately segmented, and the precision of lesion region segmentation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, and particularly relates to a lesion region segmentation method and device based on normal tissue image information comparison. BACKGROUND

[0002] Accurate segmentation of a lesion region is an important part of diagnosing and treating tumor diseases. In related technologies, a deep neural network structure is usually used to segment a lesion region. For example, when a deep neural network structure is used to segment a lesion region, a complex network structure is usually used to improve the ability of the network to extract features of the lesion region. However, in reality, when a radiologist outlines a lesion region, the prior information of a normal anatomical structure is usually used as a reference. However, the current segmentation method based on a deep neural network usually ignores the prior information of the normal anatomical structure.

[0003] Therefore, how to effectively use the prior information of the normal anatomical structure to improve the accuracy of the segmentation of the lesion region has become a technical problem to be solved in the prior art. SUMMARY

[0004] Therefore, the present application aims to provide a lesion region segmentation method and device based on normal tissue image information comparison to overcome the problem of low accuracy of the current segmentation of a lesion region.

[0005] To achieve the above object, the present application adopts the following technical solutions:

[0006] On the one hand, a lesion region segmentation method based on normal tissue image information comparison comprises the following steps:

[0007] receiving a lesion image to be segmented, wherein the lesion image to be segmented is a multi-modal lesion image;

[0008] inputting a single-modal lesion image in the lesion image to be segmented into a pre-constructed VEA network to obtain a single-modal normal image, wherein the pre-constructed VEA network is obtained by training a real single-modal normal image;

[0009] inputting the lesion image to be segmented and the single-modal normal image into an encoding-decoding network to obtain a first image feature extracted from the lesion image to be segmented and a second image feature extracted from the single-modal normal image;

[0010] aligning and comparing the first image feature and the second image feature to obtain an attention map, wherein a region with a large weight in the attention map is an attention region, and the attention region is concentrated in a lesion region;

[0011] The lesion area feature of the to-be-segmented lesion image is enhanced by using the attention map, to obtain a lesion area segmentation result of the to-be-segmented lesion image.

[0012] Optionally, the loss function of the pre-constructed VEA network is:

[0013]

[0014]

[0015] wherein η>0 is a defined margin, I H i is a real normal image, I R i is a generated normal image, Z H is a real normal image feature, Z R is a generated normal image feature, L Reg is a regularization loss function, L Rec is a reconstruction loss function, and α and β are weights of the terms.

[0016] Optionally, the encoding-decoding network comprises a segmentation backbone network and a normal tissue reference network.

[0017] The segmentation backbone network comprises four convolution layers and four deconvolution layers, wherein in the four convolution layers, the convolution kernel size of each layer is 3*3, and the channel numbers of the four convolution layers are 64, 128, 256 and 512 respectively; in the four deconvolution layers, the deconvolution kernel size is 4*4, and the channel numbers of the four deconvolution layers are 512, 256, 128 and 64 respectively.

[0018] Optionally, the feature comparison of the first image feature and the second image feature to obtain an attention map comprises:

[0019] The first image feature and the second image feature are respectively input into a feature convolution layer to obtain a first convolution image feature and a second convolution image feature;

[0020] The first convolution image feature and the second convolution image feature are input into a SimSam network for feature alignment, and an attention map is obtained through an attention gate.

[0021] Optionally, the loss function of the SimSam network is:

[0022]

[0023] wherein,

[0024] wherein, and respectively, are a prediction multi-layer perception and a mapping multi-layer perception, F T,l is a feature of a to-be-segmented target image, F R,l is a normal image feature, and g and z are 1*1 convolution layers.

[0025] In another aspect, a lesion region segmentation device based on normal tissue image information comparison includes:

[0026] A receiving module is configured to receive a to-be-segmented lesion image, which is a multi-modal lesion image.

[0027] A conversion module is configured to input a single-modal lesion image in the to-be-segmented lesion image into a pre-constructed VEA network to obtain a single-modal normal image, wherein the pre-constructed VEA network is obtained by training on real single-modal normal images.

[0028] A feature extraction module is configured to input the to-be-segmented lesion image and the single-modal normal image into an encoding-decoding network to obtain a first image feature extracted from the to-be-segmented lesion image and a second image feature extracted from the single-modal normal image.

[0029] A comparison module is configured to perform feature alignment and feature comparison on the first image feature and the second image feature to obtain an attention map, wherein a region with a large weight in the attention map is an attention region, and the attention region is concentrated in a lesion region.

[0030] An enhancement module is configured to enhance a lesion region feature of the to-be-segmented lesion image using the attention map to obtain a lesion region segmentation result of the to-be-segmented lesion image.

[0031] Optionally, the comparison module is specifically configured to pass the first image feature and the second image feature through feature convolution layers to obtain a first convolution image feature and a second convolution image feature, input the first convolution image feature and the second convolution image feature into a SimSam network for feature alignment, and perform feature comparison through an attention gate to obtain an attention map.

[0032] Optionally, a loss function of the SimSam network is as follows:

[0033]

[0034] wherein,

[0035] wherein, and respectively, are a prediction multi-layer perception and a mapping multi-layer perception, F T,l is a feature of a to-be-segmented target image, F R,lFor normal image features, ζ and ɡ are 1*1 convolutional layers.

[0036] In another aspect, a lesion region segmentation device based on normal tissue image information comparison includes a processor and a memory, the processor being connected with the memory:

[0037] The processor is configured to call and execute a program stored in the memory.

[0038] The memory is configured to store the program, and the program is configured to execute at least the lesion region segmentation method based on normal tissue image information comparison.

[0039] The beneficial effects of the technical solutions of the present application include at least the following aspects:

[0040] It can be understood that the lesion region segmentation method based on normal tissue image information comparison provided by the embodiments of the present application can input the to-be-segmented lesion image into the pre-constructed VEA network to obtain a single-modality normal image; input the to-be-segmented lesion image and the single-modality normal image into the encoding-decoding network to obtain first image features extracted from the to-be-segmented lesion image and second image features extracted from the single-modality normal image; perform feature alignment and feature comparison on the first image features and the second image features to obtain an attention map; enhance the lesion area features in the attention map to obtain a lesion area segmentation result of the to-be-segmented lesion image. Therefore, by comparing the lesion image with the normal image, the technical solutions provided by the present application fully consider the differences between the lesion image and the normal image, thereby accurately segmenting the lesion and improving the accuracy of lesion region segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0042] Figure 1 A flowchart of a lesion region segmentation method based on normal tissue image information comparison provided by an embodiment of the present application;

[0043] Figure 2 A structure diagram of a deep neural network model of a lesion region segmentation method based on normal tissue image information comparison provided by an embodiment of the present application;

[0044] Figure 3 An IntroVAE network structure diagram provided by an embodiment of the present application, wherein,Figure 3 (a) is a schematic diagram of a training phase, Figure 3 (b) is a schematic diagram of a testing phase;

[0045] Figure 4 A structural schematic diagram of a feature comparison module provided by an embodiment of the present application;

[0046] Figure 5 A structural schematic diagram of a Simsam network provided by an embodiment of the present application;

[0047] Figure 6 A feature distribution schematic diagram in a feature comparison module at each level using SimSam and not using SimSam (Baseline-2) provided by an embodiment of the present application;

[0048] Figure 7 An attention map calculated in a feature comparison module at each level provided by an embodiment of the present application;

[0049] Figure 8 A structural schematic diagram of a lesion region segmentation device based on normal tissue image information comparison provided by an embodiment of the present application;

[0050] Figure 9 A structural schematic diagram of a lesion region segmentation device based on normal tissue image information comparison provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0052] To improve the accuracy of lesion region segmentation, an embodiment of the present application provides a lesion region segmentation method, device and equipment based on normal tissue image information comparison.

[0053] Figure 1 A flowchart of a lesion region segmentation method based on normal tissue image information comparison provided by an embodiment of the present application is shown in FIG. 1. Figure 1 The method provided by the embodiment of the present application can include the following steps:

[0054] Step S1, receiving a lesion image to be segmented I t The lesion image to be segmented is a multi-modal lesion image.

[0055] In a specific implementation process, a lesion region segmentation neural network model based on normal tissue image information comparison can be integrated, and the lesion region segmentation method based on normal tissue image information comparison provided in the embodiments of the present application is implemented through the neural network model. Figure 2 A structure diagram of a lesion region segmentation method deep neural network model based on normal tissue image information comparison provided in the embodiments of the present application is shown.

[0056] In the present application, any image to be segmented can be determined as a lesion image I t to be segmented. For example, the lesion region segmentation method based on normal tissue image information comparison is described in the present application taking brain tumor images as an example.

[0057] The lesion image I t to be segmented can be obtained by CT, nuclear magnetic resonance, etc. It can be understood that the lesion image to be segmented is a multi-modal lesion image.

[0058] In step S2, a single-modal lesion image in the lesion image to be segmented is input into a pre-constructed VEA network to obtain a single-modal normal image I n ; wherein the pre-constructed VEA network is obtained by training on real single-modal normal images.

[0059] After obtaining the lesion image I t to be segmented, the lesion image to be segmented is a multi-modal lesion image, a single-modal lesion image is selected from the multi-modal lesion image, and the single-modal lesion image is input into the pre-constructed VEA network to obtain a single-modal normal image I n . The pre-constructed VEA network can be an IntroVAE network structure.

[0060] Figure 3 An IntroVAE network structure provided in the embodiments of the present application is shown, wherein, Figure 3 (a) is a training stage diagram, Figure 3 (b) is a test stage diagram.

[0061] Referring to Figure 3 (a), in the training stage, a data set can be pre-acquired, and the data set is a real single-modal normal image I H i For any input real single-modal normal image I H i The decoder generates a feature Z H based on Z H The decoder generates a normal image I R i In order to improve the segmentation accuracy, I Hi and I R i consistency, thus, the reconstruction loss function is added:

[0062]

[0063] where I H i and I R i are the real single-modality normal image and the generated normal image respectively, Ω is the image region, and x is the voxel position in the image region.

[0064] In this application, in order to improve the segmentation accuracy, the distribution of Z H is regularized to a certain extent, and the distribution is controlled in the standard normal distribution, and the regularization constraint is:

[0065] L Reg (g H ) = KL (N (0, 1) || p (z H )) formula (2)

[0066] where KL represents the KL divergence formula, N (0, 1) is the standard normal distribution, and p (z H ) is the probability distribution of z H .

[0067] In some embodiments, in order to further enhance the authenticity of the generated normal image I , the decoder in IntroVAE can be regarded as a discriminator at the same time, which is used to distinguish the real normal image I H i from the generated normal image I R i , and the encoder can be regarded as a generator at the same time to generate the generated normal image I H that the discriminator cannot distinguish.

[0068] In this application, the loss function of the decoder and the encoder in IntroVAE is:

[0069]

[0070] where η>0 is the defined margin, I i R is the real normal image, I i H is the generated normal image, Z R is the feature of the real normal image, and Z RegL is a regularization loss function Rec L is a reconstruction loss function, and a and β are weights of terms.

[0071] The training data is used to train the IntroVAE network model using single-modality normal images until convergence. See Figure 3 (b) In the test phase, the input image is a lesion image corresponding to a single modality, and the output is a normal image.

[0072] Step S3, input the lesion image to be segmented (multi-modality lesion image) and single-modality normal image into the encoding-decoding network to obtain the first image feature F Tl extracted from the lesion image to be segmented (multi-modality lesion image), and the second image feature F Rl extracted from the single-modality normal image.

[0073] See Figure 2 , the lesion region segmentation method model based on comparison of normal tissue image information provided by the embodiments of the present application can include two sub-networks, i.e. segmentation backbone network A1 and normal tissue reference network A2, and finally obtain the segmentation result C1.

[0074] The segmentation backbone network includes 4 convolutional layers and 4 deconvolutional layers, wherein the convolutional kernel size of each layer in the 4 convolutional layers is 3*3, and the channel number of the 4 convolutional layers is 64, 128, 256, and 512 respectively; the deconvolutional kernel size of the 4 deconvolutional layers is 4*4, and the channel number of the 4 deconvolutional layers is 512, 256, 128, and 64 respectively. The last layer of the segmentation backbone network and the normal tissue reference network is a convolutional layer, and the convolutional kernel size can be 3*3 and the channel number is 1, i.e. through the last convolutional layer channel, the final segmentation result is obtained.

[0075] The multi-modality lesion image to be segmented I t is input into the encoder of the segmentation backbone network, and in the decoding stage, the first image feature F t (i.e. F T,l ) is extracted; the single-modality normal image I n is input into the encoder of the normal tissue reference network, and in the decoding stage, the second image feature F n (i.e. F R,l ) is extracted.

[0076] The normal tissue reference network mainly extracts normal region features F R,l from the input normal image, which is used for comparison with the features F T,l extracted from the segmentation backbone network, so as to obtain the attention map A l reflecting the inconsistent feature region, and is used to enhance F T,lThe network structure comprises 4 convolutional layers and 4 deconvolutional layers, and the number of convolutional kernels and the number of channels of each layer are the same as those of the segmentation backbone network.

[0077] In step S4, the first image features and the second image features are aligned and compared, and an attention map is obtained. In the attention map, the region with a large weight is an attention region, and the attention region is concentrated in the lesion region.

[0078] Specifically, the features of the normal region in the first image features and the second image features are aligned, so as to improve the feature comparison between the lesion and the normal region in the first image features and the second image features.

[0079] Figure 4 A structural diagram of a feature comparison module provided by an embodiment of the present application is shown.

[0080] Referring to Figure 4 , the first image features and the second image features are compared by the feature comparison module, and an attention map is obtained.

[0081] In step S5, the lesion region features of the to-be-segmented lesion image are enhanced by using the attention map, and a lesion region segmentation result of the to-be-segmented lesion image is obtained.

[0082] Referring to Figure 2 , the features F T,l obtained by each deconvolutional layer in the decoder (l=1, 2, 3, 4 layer number) are input to the feature comparison module, and compared with the features F R,l obtained by the same level deconvolutional layer in the normal tissue reference network (l=1, 2, 3, 4 layer number), to obtain an attention map A l In the attention map, the highlighted part is the region where the features F t and the features F n are inconsistent, that is, the normal tissue region and the lesion region features are inconsistent. That is, the attention region (highlighted part) with a large weight in the attention map is concentrated in the lesion region.

[0083] After obtaining the attention map, the attention map A l can be input into the decoder to enhance the features of the lesion region in F t , and finally obtain an accurate lesion region segmentation result. That is, A l is multiplied by F T,l to obtain an enhanced lesion region feature map F / T,l .

[0084] In some embodiments, referring to Figure 4 , the first image features and the second image features are compared to obtain an attention map, which comprises:

[0085] The first image feature F T,l and the second image feature F R,l are respectively input into a feature convolution layer to obtain a first convolution image feature ζ(F Tl ) and a second convolution image feature g(F Rl );

[0086] The first convolution image feature and the second convolution image feature are input into a SimSam network for feature alignment, and an attention map is obtained after feature comparison through an attention gate.

[0087] Figure 5 A structural schematic diagram of the Simsam network provided by the embodiment of the application is shown.

[0088] For example, in the feature comparison module, as shown in Figure 4 and Figure 5 , the input of the feature comparison module is the features F Tl and F Rl from the deconvolution layer of each level (l level) of the segmentation backbone network and the normal tissue reference network, and the features ζ(F Tl ) and g(F Rl ) are obtained through two 1*1 convolution layers. Considering that ζ(F Tl ) and g(F Rl ) are extracted from multi-modal and single-modal images respectively, in order to make the features of the normal region in ζ(F Tl ) and g(F Rl ) as consistent as possible, g(F Rl ) can be used to highlight the features of the lesion region in ζ(F Tl ), and a SimSam module (as shown in Figure 5 ) is used for feature alignment. First, according to the manually segmented label image, the feature vector at each position in ζ(F Tl ) can be divided into normal region features and lesion region features, and then a Simsam loss function is added:

[0089]

[0090] wherein,

[0091]

[0092] and are a prediction multi-layer perceptron (Prediction MLP) and a projection multi-layer perceptron (Projection MLP), respectively. Due to the constraint of L Sim , ζ(F Tl ) and g(FRl ) of the normal region approaches to be consistent. It is noted that in the training process, in order to prevent the gradient of SimSam from being returned to the segmentation backbone network, so that the backbone network is disturbed by the feature consistency target, a stop gradient operation is applied at ζ(F Tl ) as shown in FIG. 8. Figure 3 b, based on ζ(F Tl ) and g(F Rl ), the attention map A l is obtained using the subsequent ReLU, 1*1 convolution layer and sigmoid activation layer.

[0093]

[0094] Where σ1 is the ReLU layer, σ2 is the Sigmoid layer, ψ, ζ and g are 1*1 convolution layers, F T,l is the feature of the image to be segmented, and F R,l is the feature of the normal image.

[0095] It can be understood that by using the lesion region segmentation method based on comparison of normal tissue image information provided in the embodiments of the present application, the single modality normal image is obtained by inputting the lesion image to be segmented into the pre-constructed VEA network; the first image feature extracted from the lesion image to be segmented and the second image feature extracted from the single modality normal image are obtained by inputting the lesion image to be segmented and the single modality normal image into the encoding-decoding network; the attention map is obtained by comparing the first image feature and the second image feature; and the lesion region segmentation result of the lesion image to be segmented is obtained by using the attention region in the attention map to enhance the lesion region feature in the lesion image to be segmented. Therefore, by using the technical solution provided in the present application, the lesion image is compared with the normal image, the differences between the lesion image and the normal image are fully considered, and the lesion is accurately segmented, thereby improving the accuracy of the lesion region segmentation.

[0096] In order to verify the segmentation effect of the lesion region segmentation method based on comparison of normal tissue image information provided in the embodiments of the present application, an experimental embodiment is further provided in the embodiments of the present application.

[0097] In the experimental embodiment, the public brain tumor data set (BraTS) and the private data set are used to test the lesion region segmentation method based on comparison of normal tissue image information, and the mainstream method (including V-Net, Attention U-Net, nnU-Net) is compared. Ablation experiments are also conducted, including Baseline-1 (pure segmentation backbone network) and Baseline-2 (without SimSam module).

[0098] The specific data information is as follows: the public data set includes multi-modal magnetic resonance brain tumor images of 335 patients, the private data set includes multi-modal magnetic resonance brain tumor images of 104 patients. In addition, the public normal brain data set (IXI) is used for training of the IntroVAE network, which includes single-modal T1 magnetic resonance images of 581 healthy people. A 5-fold cross-validation method is used to test all image segmentation models, the input image is a 2.5D image (2K+1 sections), K=2 in the method, that is, the input data of each modality includes 5 sections, and the output segmentation result is the lesion area of the center section. The IntroVAE network is trained separately, and the 2D section is used, and the training parameters are: batch size 120, maximum epoch 200, and α, β, η in formula (3) are respectively set as 0.5, 0.04 and 120. After the IntroVAE training converges, the parameters are fixed and integrated into the training of the proposed segmentation network. The training parameters of all tested segmentation network models are Batch size 4 and maximum epoch 300. The Dice coefficient, sensitivity and Hausdorff distance are used to quantify the segmentation accuracy, and the specific experimental results (test data) are as follows:

[0099] Table 1 Segmentation results of public data set

[0100] Method Dice coefficient Sensitivity Hausdorff distance V-Net 0.884±0.070 0.857±0.116 5.625±5.164 Attention U-Net 0.894±0.065 0.874±0.118 5.292±5.996 nnU-Net 0.905±0.072 0.895±0.098 6.045±4.215 Baseline-1 0.902±0.049 0.898±0.089 6.227±4.206 Baseline-2 0.903±0.053 0.903±0.081 6.124±4.506 Method 0.919±0.046 0.919±0.080 3.843±3.045

[0101] Referring to Table 1, the Dice coefficient results obtained by the lesion region segmentation method based on comparison of normal tissue image information provided in the embodiments of the present application are subjected to Wilcoxon signed rank test for significance detection with other methods, and the p values are respectively: 1.017x10 -3 (Comparison with V-Net), 0.022 (Comparison with Atten.U-Net), 0.008 (Comparison with nnU-Net), 2.798x10 -7 (Comparison with Baseline-1), and 4.224x10 -5 (Comparison with Baseline-2), therefore, the lesion region segmentation method based on comparison of normal tissue image information provided in the embodiments of the present application has a significant improvement in segmentation accuracy (p<0.05) compared with other comparison methods.

[0102] Table 2 Segmentation results of private data set

[0103] Dice coefficient Sensitivity Hausdorff distance V-Net Attention U-Net 0.853±0.050 0.832±0.084 5.850±1.768 nnU-Net 0.863±0.052 0.853±0.087 5.861±2.335 Baseline-1 0.874±0.074 0.860±0.122 6.072±3.528 Baseline-2 0.877±0.038 0.876±0.063 4.735±2.034 Method 0.879±0.036 0.877±0.047 4.625±1.730 Figure 6 0.897±0.037 0.909±0.055 4.287±1.814

[0104] Referring to Table 2, the Dice coefficient results obtained by the lesion region segmentation method provided by the embodiment of the application based on comparison of normal tissue image information are subjected to significance detection with other methods by using the Wilcoxon signed rank test, and the p values are respectively: 3.815x10 -6 (Comparison with V-Net), 9.537x10 -6 (Comparison with Atten.U-Net), 0.018 (Comparison with nnU-Net), 7.796x10 -4 (Comparison with Baseline-1), and 2.204x10 -3 (Comparison with Baseline-2), so the segmentation accuracy of the method of the application is significantly improved (p<0.05) compared with other comparison methods.

[0105] Figure 6 A feature distribution diagram in the feature comparison module of each level provided by the embodiment of the application is used with and without SimSam (Baseline-2).

[0106] In this embodiment, in order to intuitively show the role of SimSam in the feature comparison module, the features of the feature comparison module without SimSam (i.e., the features in Baseline-2) and the features of the feature comparison module with SimSam (i.e., the features of the method of the application) are subjected to PCA dimension reduction, and the three-dimensional points after dimension reduction are plotted on the image, Figure 6 showing the feature point distribution of different regions. Referring to Figure 7 It can be clearly seen that in the technical solution provided by the embodiment of the application, with the increase of the level, the consistency of the feature distribution of the normal region in the lesion image and the normal image is stronger than that of Baseline-2.

[0107] Figure 7 The attention map calculated in the feature comparison module of each level provided by the embodiment of the application is shown in Figure 8 It can be seen that compared with Baseline-2, the highlighted part of the attention map is more concentrated in the tumor region by using the method recorded in the embodiment of the application.

[0108] Based on a general inventive concept, the embodiment of the application also provides a lesion region segmentation device based on comparison of normal tissue image information.

[0109] Figure 8 The structure diagram of the lesion region segmentation device based on comparison of normal tissue image information provided by the embodiment of the application is shown in the embodiment of the application. The device provided by the embodiment of the application is used to implement the above-mentioned method embodiment. Referring to Figure 9 The device provided by the application can include the following structures:

[0110] The receiving module 81 is configured to receive a to-be-segmented lesion image, the to-be-segmented lesion image being a multi-modal lesion image;

[0111] The conversion module 82 is configured to input a single-modal lesion image in the to-be-segmented lesion image into a pre-constructed VEA network to obtain a single-modal normal image; the pre-constructed VEA network is obtained by training on real single-modal normal images.

[0112] The feature extraction module 83 is configured to input the to-be-segmented lesion image and the single-modal normal image into an encoding-decoding network to obtain first image features extracted from the to-be-segmented lesion image and second image features extracted from the single-modal normal image.

[0113] The comparison module 84 is configured to perform feature alignment and feature comparison on the first image features and the second image features to obtain an attention map; in the attention map, a region with a large weight is an attention region, and the attention region is concentrated in a lesion region.

[0114] The enhancement module 85 is configured to enhance a lesion region feature of the to-be-segmented lesion image by using the attention map to obtain a lesion region segmentation result of the to-be-segmented lesion image.

[0115] Optionally, the comparison module is specifically configured to pass the first image features and the second image features through a feature convolution layer to obtain first convolution image features and second convolution image features; and input the first convolution image features and the second convolution image features into a SimSam network to perform feature alignment, and perform feature comparison through an attention gate to obtain the attention map.

[0116] Optionally, a loss function of the SimSam network is as follows:

[0117]

[0118] wherein,

[0119] wherein, and are a prediction multi-layer perceptron and a mapping multi-layer perceptron respectively, F T,l is a to-be-segmented target image feature, F R,l is a normal image feature, and ζ and ɡ are 1*1 convolution layers.

[0120] As to the apparatus in the above-described embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described herein in detail.

[0121] Based on one general inventive concept, the embodiments of the present application also provide a lesion region segmentation device based on normal tissue image information comparison.

[0122] Figure 9 A structural schematic diagram of a lesion region segmentation device based on normal tissue image information comparison provided by an embodiment of the present application is shown in FIG. 1. As shown in the figure, the device of the present embodiment comprises a processor 91 and a memory 92, and the processor 91 is connected with the memory 92. The processor 91 is configured to invoke and execute a program stored in the memory 92, and the memory 92 is configured to store the program, which is used at least to execute the lesion region segmentation method based on normal tissue image information comparison in the above embodiment. ​

[0123] The specific implementation of the lesion region segmentation device based on normal tissue image information comparison provided by the embodiment of the present application can refer to the implementation of the lesion region segmentation method based on normal tissue image information comparison of any of the above embodiments, which will not be repeated here.

[0124] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the content not described in detail in some embodiments can be referred to the same or similar content in other embodiments.

[0125] It should be noted that, in the description of the present application, the terms "first", "second", etc. are only used for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0126] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) in the process, and the various embodiments of the present application include additional implementations in which the order of steps can be different, including use of a different order of executing hardware steps, or a different order of executing software steps, or both, depending upon the functionality involved, as would be understood by one of ordinary skill in the art.

[0127] It should be understood that each part of the present application can be realized by hardware, software, firmware or their combination. In the above implementation, a plurality of steps or methods can be realized by software or firmware stored in the memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another implementation, it can be realized by any one or their combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0128] ​Those skilled in the art can understand that all or part of the steps of the method carried out by the above-mentioned embodiments can be instructed by a program to complete the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0129] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically independently, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0130] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0131] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0132] Although the embodiments of the present application have been shown and described above, it can be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application. Those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A lesion region segmentation method based on normal tissue image information contrast, characterized in that, The method comprises the steps of: receiving a lesion image to be segmented, wherein the lesion image to be segmented is a multi-modal lesion image; inputting a single-modal lesion image in the lesion image to be segmented into a pre-constructed VEA network to obtain a single-modal normal image, wherein the pre-constructed VEA network is obtained by training a real single-modal normal image; inputting the lesion image to be segmented and the single-modal normal image into an encoding-decoding network to obtain a first image feature extracted from the lesion image to be segmented and a second image feature extracted from the single-modal normal image; inputting the first image feature and the second image feature into a feature convolution layer to obtain a first convolution image feature and a second convolution image feature; inputting the first convolution image feature and the second convolution image feature into a SimSam network for feature alignment, and obtaining an attention map after feature comparison through an attention gate, wherein a region with a large weight in the attention map is an attention region, and the attention region is concentrated in a lesion region; enhancing a lesion region feature of the lesion image to be segmented by using the attention map to obtain a lesion region segmentation result of the lesion image to be segmented.

2. The method of claim 1, wherein, The loss function of the pre-constructed VEA network is: where I H i is the real normal image, I R i is the generated normal image, Z H is the real normal image feature, Z R is the generated normal image feature, L Reg is the regularization loss function, L Rec is the reconstruction loss function, and a, b are the weights of the terms.

3. The method of claim 1, wherein, The encoding-decoding network comprises a segmentation backbone network and a normal tissue reference network. The segmentation backbone network comprises four convolution layers and four deconvolution layers, wherein the convolution kernel size of each layer in the four convolution layers is 3*3, the channel numbers of the four convolution layers are 64, 128, 256 and 512 respectively, the deconvolution kernel size of each layer in the four deconvolution layers is 4*4, and the channel numbers of the four deconvolution layers are 512, 256, 128 and 64 respectively.

4. The method of claim 1, wherein, The loss function of the SimSam network is: wherein wherein, with are a predictive multilayer perceptron and a mapping multilayer perceptron, respectively, F T,l is a feature of the target image to be segmented, F R,l is a normal image feature, and ɡ and ζ are 1*1 convolutional layers.

5. A lesion region segmentation apparatus based on normal tissue image information contrast, characterized by, The method comprises the steps of: a receiving module configured to receive a lesion image to be segmented, wherein the lesion image to be segmented is a multi-modal lesion image; a conversion module configured to input a single-modal lesion image in the lesion image to be segmented into a pre-constructed VEA network to obtain a single-modal normal image, wherein the pre-constructed VEA network is obtained by training a real single-modal normal image; a feature extraction module configured to input the lesion image to be segmented and the single-modal normal image into an encoding-decoding network to obtain a first image feature extracted from the lesion image to be segmented and a second image feature extracted from the single-modal normal image; a comparison module configured to perform feature alignment and feature comparison on the first image feature and the second image feature to obtain an attention map, wherein a region with a large weight in the attention map is an attention region, and the attention region is concentrated in a lesion region; and specifically configured to input the first image feature and the second image feature into a feature convolution layer to obtain a first convolution image feature and a second convolution image feature; input the first convolution image feature and the second convolution image feature into a SimSam network for feature alignment, and obtain an attention map after feature comparison through an attention gate. An enhancement module is configured to enhance the lesion region features of the lesion image to be segmented by using the attention map, so as to obtain a lesion region segmentation result of the lesion image to be segmented.

6. The apparatus of claim 5, wherein, The loss function of the SimSam network is: wherein wherein, with respectively a predictive multilayer perceptron and a mapping multilayer perceptron, F T,l is a feature of the target image to be segmented, F R,l is a normal image feature, g and z are 1*1 convolutional layers.

7. A lesion region segmentation apparatus based on normal tissue image information contrast, characterized by, The device comprises a processor and a memory, and the processor is connected to the memory: The processor is configured to call and execute a program stored in the memory. The memory is configured to store the program, and the program is at least used to execute the lesion region segmentation method based on the normal tissue image information contrast according to any one of claims 1-4.

Images