Pancreatic tumor segmentation method based on easily-confused region information mining

Abstract

The application relates to a pancreatic tumor segmentation method based on easily confused region information mining, which is based on a pancreatic tumor segmentation model, the model adopts a multi-scale U-Net network, three convolution kernels with different sizes are adopted in an encoder to extract features, feature maps extracted by the three convolution kernels contain context information of different scales, so that the performance of pancreatic tumor segmentation for small tumors is improved, in addition, the model adopts a positioning and focusing module, long-distance dependence is obtained through an attention module to position potential tumors, then interference information is searched and removed, and the initial segmentation result is gradually refined, so that the problem that the pancreatic tumor segmentation effect is poor due to the similarity with background tissues is solved.

Description

Technical Field

[0001] This application relates to the field of medical image segmentation technology, specifically to a pancreatic tumor segmentation method based on the mining of easily confused region information. Background Technology

[0002] Pancreatic cancer is a highly malignant disease that causes more than 330,000 deaths worldwide each year (the seventh leading cause of cancer death). Because early diagnosis is difficult, pancreatic cancer often spreads to other parts of the body by the time it is diagnosed, resulting in a poor prognosis and a 5-year survival rate of less than 8%.

[0003] In recent years, with the emergence of deep learning, many excellent neural networks have emerged, improving the accuracy and speed of segmentation and detection tasks for various organs and tumors. For example, methods based on intermediate modalities to capture the similarity between MIR and CT images enhance the discriminative power of high-level features and provide rich intermediate representations for pancreatic cancer segmentation. Another method uses deep neural networks with different input scales for tumor segmentation. First, a network with a larger input scale is used for coarse segmentation. Then, a network with a smaller input scale is used to find pancreatic tumors that were ignored in the coarse segmentation results for candidate regions. A post-processing algorithm is designed to remove erroneous values ​​and reduce missegmentation. A self-supervised learning 3D tumor segmentation method based on prior knowledge is implemented by combining scale discrimination and self-recovery. While these methods have achieved some success, their segmentation performance is greatly limited by the small size and unclear boundaries of pancreatic tumors. First, compared to the human body, pancreatic tumors are very small, and the network used for segmentation has low sensitivity to small-scale targets, resulting in low prediction accuracy, especially regarding their boundaries. Secondly, when the target and background tissues are highly similar, the network may be confused by background areas that have a similar appearance to the target or by cluttered camouflage areas in the background, resulting in false negatives and false positives. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a pancreatic tumor segmentation method based on the mining of easily confused region information.

[0005] Firstly, a pancreatic tumor segmentation model is provided, including: a multi-scale U-Net network, focusing on the network structure and localization module;

[0006] The multi-scale U-Net network includes an encoder structure and a decoder structure. The decoder structure includes multiple decoder layers. The image to be segmented is input into the multi-scale U-Net network, and each decoder layer outputs a layer feature.

[0007] The localization module is used to output the potential location of the tumor based on the layer features output by the first decoder layer of the decoder structure;

[0008] The focused network structure is used to output the accurate location of the tumor based on the layer features output by the decoder structure and the potential location of the tumor.

[0009] In one embodiment, the decoder structure includes multiple decoder layers connected in sequence, each decoder layer employing three different convolutional kernels to output image features; the output of the previous encoder layer serves as the input of the next encoder layer, and the output of the last encoder layer serves as the input of the decoder structure.

[0010] In one embodiment, multiple decoder layers are connected sequentially, with the output of the previous decoder layer serving as the input of the next decoder layer.

[0011] The output of the first decoder layer in a multi-decoder layer is the layer feature output by the first decoder layer.

[0012] The layer features output by the decoder layers other than the first decoder are obtained in the following way:

[0013] The result obtained by convolving the output of the decoder layer is denoted as the convolution result. The convolution result is then merged with the output of the encoder layer corresponding to the decoder layer to obtain the layer feature.

[0014] In one embodiment, the positioning module includes a channel attention block and a spatial attention block;

[0015] In the decoder structure, the layer features output by the first decoder layer are convolved and batch normalized before being input into the localization module, which outputs the potential location of the tumor.

[0016] In one embodiment, the focusing network structure includes multiple focusing modules, and each of the multiple focusing modules corresponds to a multiple decoder layer;

[0017] The first focusing module is used to output the tumor location prediction result of the first focusing module based on the layer features output by the corresponding decoder layer, the layer features output by the first decoder layer, and the potential location of the tumor output by the localization module.

[0018] The other focusing modules are used to output the tumor location prediction result based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction result output by the previous focusing module; the tumor location prediction result output by the last focusing module is the accurate location of the tumor.

[0019] In one embodiment, other focusing modules are used to output tumor location prediction results based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction results output by the previous focusing module, including:

[0020] The focus module includes the context exploration module;

[0021] For the current focusing module, the tumor location prediction result output by the previous focusing module is upsampled to obtain the first upsampled result image;

[0022] The layer features output by the encoder layer corresponding to the current focusing module are convolved and batch normalized, and then multiplied with the first upsampled result image and the inverse image of the upsampled result image, respectively, to obtain the first multiplication result and the second multiplication result;

[0023] The first and second multiplication results are input into the context exploration module, which outputs false positive and false negative interference respectively.

[0024] The layer features output from the previous decoder layer are convolved, batch normalized, and then upsampled to obtain the second upsampled result.

[0025] False positive interference and the second upsampling result are processed using element-based subtraction. The result is then batch normalized to obtain the first processing result.

[0026] The false negative interference and the first processing result are subjected to element-based addition. After batch normalization, the result is used to obtain a refined feature map.

[0027] Convolution is performed on the refined feature map to obtain the tumor location prediction result.

[0028] In one embodiment, the context exploration module includes multiple context exploration branches, a fusion unit, and a first convolutional unit; each context exploration branch includes a convolutional unit for channel reduction, a convolutional unit for local feature extraction, and a context-aware dilated convolutional unit connected in sequence.

[0029] The input features are fed into multiple context exploration branches, and the output of the previous context exploration branch is used as the input of the next context exploration branch.

[0030] The output of each context exploration branch is input into the fusion unit for fusion.

[0031] The output of the fusion unit is input to the first convolution unit, and the output of the first convolution unit is either a false positive or a false negative.

[0032] Secondly, a pancreatic tumor segmentation method based on easily confused region information mining is provided, including:

[0033] The image to be segmented is input into the pancreatic tumor segmentation model, which outputs the accurate location of the tumor; the pancreatic tumor segmentation model is the pancreatic tumor segmentation model described above.

[0034] Thirdly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the aforementioned pancreatic tumor segmentation method based on the mining of easily confused region information.

[0035] Fourthly, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the aforementioned pancreatic tumor segmentation method based on the mining of easily confused region information.

[0036] Compared with existing technologies, this application has the following advantages: The pancreatic tumor segmentation method based on easily confused region information mining in this application is based on a pancreatic tumor segmentation model. The model adopts a multi-scale U-Net network, and three convolutional kernels of different sizes are used in the encoder to extract features. The extracted feature maps contain contextual information of different scales, thus improving the performance of pancreatic tumor segmentation for small tumors. In addition, the model adopts localization and focusing modules. First, the long-distance dependency is obtained through the attention module to locate potential tumors. Then, interference information is searched and removed to gradually refine the initial segmentation results, solving the problem of poor segmentation effect of pancreatic tumors that are similar to background tissues. Attached Figure Description

[0037] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0038] Figure 1 A schematic diagram of the structure of a pancreatic tumor segmentation model according to an embodiment of this application is shown;

[0039] Figure 2 A schematic diagram of the focusing module is shown;

[0040] Figure 3 The diagram shows a comparison of the segmentation results between this application and existing methods. Detailed Implementation

[0041] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0042] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0043] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0044] To address the problems of inaccurate localization of small-scale pancreatic tumors and false positives and false negatives that often occur during the segmentation of pancreatic tumors with ambiguous boundaries in existing technologies, this application provides a pancreatic tumor segmentation method based on the mining of easily confused region information. This method performs tumor segmentation based on a pancreatic tumor segmentation model, which improves the segmentation performance of small pancreatic tumors and has better tumor segmentation results.

[0045] This application provides a pancreatic tumor segmentation model. Figure 1 A schematic diagram of the structure of a pancreatic tumor segmentation model according to an embodiment of this application is shown. See also: Figure 1 The model includes: a multi-scale U-Net network, focusing on the network structure and localization module.

[0046] The multi-scale U-Net network includes an encoder structure and a decoder structure. The decoder structure includes multiple decoder layers. The image to be segmented is input into the multi-scale U-Net network, and each decoder layer outputs a layer feature.

[0047] The localization module is used to output the potential location of the tumor based on the layer features output by the first decoder layer of the decoder structure;

[0048] The focused network structure is used to output the accurate location of the tumor based on the layer features output by the decoder structure and the potential location of the tumor.

[0049] In this embodiment, the multi-scale U-Net network uses three different sizes of convolutional kernels to extract feature information at different scales; the localization module uses channel and spatial attention mechanisms to initially determine the potential location of the tumor; and the focusing module, based on the initial location information, discovers and removes easily confused information to more accurately obtain the specific location of the target.

[0050] In one embodiment, the decoder structure includes multiple decoder layers connected in sequence. Each decoder layer uses three different convolutional kernels to output image features; the output of the previous encoder layer serves as the input to the next encoder layer, and the output of the last encoder layer serves as the input to the decoder structure. See also Figure 1The multiple decoder layers are D4, D3, D2, D1, and D0, with D4 designated as the first decoder layer, and the multiple encoder layers are E0, E1, E2, and E3.

[0051] In this embodiment, each encoder layer includes two encoder units. One encoder unit uses a 3×3 convolution kernel, and the other encoder unit uses 1×1 and 5×5 convolution kernels to capture image features at three different scales. Here, a 1×1 convolution kernel is introduced to increase the model's representational power.

[0052] In one embodiment, multiple decoder layers are connected sequentially, with the output of the previous decoder layer serving as the input of the next. The output of the first decoder layer is its layer feature. The layer features of the other decoder layers are obtained as follows: the output of each decoder layer is convolved, and this result is denoted as the convolution result. The convolution result is then merged with the output of the corresponding encoder layer to obtain the layer feature. Specifically, the output of the first decoder layer D4 is the layer feature; the output of the second decoder layer D3 is convolved and then merged with the output of the encoder layer E3 to obtain the layer feature of the second decoder layer D3, and so on.

[0053] In one embodiment, the localization module includes a channel attention block and a spatial attention block; the layer features output from the first decoder layer D4 are convolved and batch normalized before being input to the localization module, which outputs the potential location of the tumor. See also Figure 1 Convolution and batch normalization are adopted. Figure 1 The CBR module in the CBR module also includes R e LU activation function.

[0054] In this embodiment, the localization module acquires long-range dependencies in terms of channel and spatial location, enhancing the semantic representation of high-level features from a global perspective. The channel attention block outputs a channel attention map, which is then input to the spatial attention block. The output is processed by a 7×7 convolutional kernel with a stride of 3 to locate the potential location of the tumor.

[0055] In one embodiment, the focusing network structure includes multiple focusing modules, which can be set to four, denoted as F1, F2, F3, and F4. The multiple focusing modules correspond one-to-one with multiple decoder layers; that is, F1 corresponds to D3, F2 corresponds to D2, F3 corresponds to D1, and F4 corresponds to D0.

[0056] The first focusing module F1 is used to output the tumor location prediction result of the first focusing module F1 based on the layer features output by the corresponding decoder layer D3, the layer features output by the first decoder layer D4, and the potential location of the tumor output by the localization module.

[0057] The other focusing modules are used to output the tumor location prediction result based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction result output by the previous focusing module; the tumor location prediction result output by the last focusing module is the accurate location of the tumor. For example, the second focusing module F2 outputs the tumor location prediction result based on the layer features output by decoder layer D2, the layer features output by decoder layer D3, and the tumor location prediction result output by the first focusing module F1.

[0058] In one embodiment, Figure 2 A schematic diagram of the focusing module is shown; see [link / reference]. Figure 2 Other focusing modules are used to output tumor location prediction results based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction results output by the previous focusing module, including:

[0059] The focus module includes the context exploration module;

[0060] For the current focusing module, the tumor location prediction result output by the previous focusing module is upsampled to obtain the first upsampled result image;

[0061] The layer features output by the encoder layer corresponding to the current focusing module are convolved and batch normalized, and then multiplied with the first upsampled result image and the inverse image of the upsampled result image, respectively, to obtain the first multiplication result F. fa The result of multiplying the second product F fb ;

[0062] The result of the first multiplication F fa The result of multiplying the second product F fb The inputs are respectively fed into the context exploration module, and the false positive interference F is output respectively. fpd and false negative interference F fnd ;

[0063] The layer features output from the previous decoder layer are convolved, batch normalized, and then upsampled to obtain the second upsampled result F. up ;

[0064] False positive interference F fpd Second upsampling result F up Using element-based subtraction, the result is batch normalized to obtain the first processed result F. r ;

[0065] False negative interference F fnd And the first processing result F r After element-based addition, the result is batch normalized to obtain the refined feature map F′. r ;

[0066] For the refined feature map F′ r Convolution is performed to obtain the tumor location prediction result. Here, a convolutional layer with a kernel size of 7×7 and a stride of 3 is used for convolution.

[0067] In the above embodiments, if the current focusing module is the first focusing module, the structure of the focusing module is the same. The specific input is that the tumor location prediction result output by the previous focusing module is upsampled, and the potential location of the tumor output by the positioning module is upsampled.

[0068] In one embodiment, the context exploration module includes multiple context exploration branches, a concatenation unit, and a first convolutional unit. Here, the first convolutional unit includes a 3×3 convolution, batch normalization, and a ReLU activation function. Each context exploration branch includes, in sequence, a convolutional unit for channel reduction, a convolutional unit for local feature extraction, and a context-aware dilated convolutional unit. The convolutional unit for channel reduction includes a 3×3 convolution, and the convolutional unit for local feature extraction includes k... i ×k i convolution, k i The values ​​can be 1, 3, 5, or 7, and the dilated convolution unit includes a 3×3 dilated convolution.

[0069] The input features are fed into multiple context exploration branches, with the output of the previous context exploration branch serving as the input to the next context exploration branch; here, the input feature is the result of the first multiplication, F. fa Or the result of the second multiplication F fb .

[0070] The output of each context exploration branch is input to the fusion unit for fusion, which allows contextual information to be obtained over a wider range.

[0071] The output of the fusion unit is input to the first convolution unit, and the output of the first convolution unit is either a false positive or a false negative.

[0072] This application also provides a pancreatic tumor segmentation method based on easily confused region information mining, including:

[0073] The image to be segmented is input into the pancreatic tumor segmentation model, which outputs the accurate location of the tumor. The pancreatic tumor segmentation model is the pancreatic tumor segmentation model according to the above embodiment.

[0074] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described pancreatic tumor segmentation method based on the mining of easily confused region information.

[0075] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the above-described pancreatic tumor segmentation method based on easily confused region information mining.

[0076] To further verify the effectiveness of the model and method of this application in tumor segmentation, a tumor segmentation experiment was conducted using the method of this application. Figure 3 The diagram shows a comparison of the segmentation results between this application and existing methods. Figure 3 As can be seen, the green outline represents the actual tumor segmentation line, while the red outline represents the tumor segmentation results of different methods. The figure shows that this application can more accurately locate smaller pancreatic tumors. Furthermore, for tumors with low contrast to surrounding tissues, this application, to a certain extent, solves the false positive and false negative problems that easily occur in traditional networks.

[0077] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A pancreatic tumor segmentation method based on easily confused region information mining, characterized in that, include: The image to be segmented is input into the pancreatic tumor segmentation model, which outputs the accurate location of the tumor. The pancreatic tumor segmentation model includes: a multi-scale U-Net network, a focused network structure, and a localization module; The multi-scale U-Net network includes an encoder structure and a decoder structure. The decoder structure includes multiple decoder layers. The image to be segmented is input into the multi-scale U-Net network, and each decoder layer outputs a layer feature. Each decoder layer uses three different convolutional kernels. The localization module is used to output the potential location of the tumor based on the layer features output by the first decoder layer of the decoder structure; The focused network structure is used to output the accurate location of the tumor based on the layer features output by the decoder structure and the potential location of the tumor. The positioning module includes a channel attention block and a spatial attention block; The layer features output by the first decoder layer of the decoder structure are convolved and batch normalized before being input into the localization module, which outputs the potential location of the tumor. The focused network structure includes multiple focused modules, and each of the multiple focused modules corresponds one-to-one with multiple decoder layers other than the first decoder layer. The first focusing module is used to output the tumor location prediction result of the first focusing module based on the layer features output by the corresponding decoder layer, the layer features output by the first decoder layer, and the potential location of the tumor output by the localization module. Other focusing modules are used to output tumor location prediction results based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction results output by the previous focusing module; the tumor location prediction results output by the last focusing module are the accurate locations of the tumor. The other focusing modules are used to output tumor location prediction results based on the layer features output by the corresponding decoder layer, the layer features output by the previous decoder layer, and the tumor location prediction results output by the previous focusing module, including: The focusing module includes a context exploration module; For the current focusing module, the tumor location prediction result output by the previous focusing module is upsampled to obtain the first upsampled result image; The layer features output by the decoder layer corresponding to the current focusing module are convolved and batch normalized, and then multiplied with the first upsampled result image and the reverse image of the first upsampled result image, respectively, to obtain the first multiplication result and the second multiplication result. The first multiplication result and the second multiplication result are respectively input into the context exploration module, and false positive interference and false negative interference are respectively output; The layer features output from the previous decoder layer are convolved, batch normalized, and then upsampled to obtain the second upsampling result. The false positive interference and the second upsampling result are processed using element-based subtraction, and the result is batch normalized to obtain the first processing result. The false negative interference and the first processing result are subjected to element-based addition, and the result is batch normalized to obtain a refined feature map. The refined feature map is convolved to obtain the tumor location prediction result.

2. The method as described in claim 1, characterized in that, The encoder structure includes multiple encoder layers connected in sequence. Each encoder layer uses three different convolution kernels to output image features. The output of the previous encoder layer serves as the input of the next encoder layer, and the output of the last encoder layer serves as the input of the decoder structure.

3. The method as described in claim 1, characterized in that, The multiple decoder layers are connected in sequence, with the output of the previous decoder layer serving as the input of the next decoder layer. The output of the first decoder layer in the plurality of decoder layers is the layer feature output by the first decoder layer; The layer features output by the other decoder layers besides the first decoder are obtained in the following way: The result obtained by convolving the output of the decoder layer is denoted as the convolution result; the convolution result is merged with the output of the encoder layer corresponding to the decoder layer to obtain the layer feature.

4. The method as described in claim 1, characterized in that, The context exploration module includes multiple context exploration branches, a fusion unit, and a first convolutional unit; each context exploration branch includes a convolutional unit for channel reduction, a convolutional unit for local feature extraction, and a context-aware dilated convolutional unit connected in sequence. The input features are fed into multiple context exploration branches, and the output of the previous context exploration branch is used as the input of the next context exploration branch. The output of each context exploration branch is input into the fusion unit for fusion. The output of the fusion unit is input to the first convolution unit, and the output of the first convolution unit is the false positive interference or the false negative interference.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the pancreatic tumor segmentation method based on easily confused region information mining as described in claim 1.

6. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, implements the pancreatic tumor segmentation method based on easily confused region information mining as described in claim 1.

Images