A deep learning-based automatic segmentation method for pediatric kidney and pyelonephritis in computed tomography

Abstract

The present invention discloses a method for automatic segmentation of children's kidney and pyelonephritis based on deep learning applied to computer tomography, including a local feature extraction stage, a global feature extraction stage, a decoding upsampling stage, and a segmentation stage. The beneficial effects of the present invention are: designing a dense skip connection mode of long connections and short connections, superimposing the extracted multi-layer local features at the current node and transmitting them to the decoder end; designing an adaptive attention module, based on the extracted local features and global features, fusing the two and passing through the adaptive attention module, so that the model learns the characteristics of children's kidney and pyelonephritis, and improves the segmentation accuracy of the model for the kidney and pyelonephritis edges.

Description

Technical Field

[0001] The present invention relates to an automatic segmentation method, in particular to an automatic segmentation method for children's kidneys and pyelonephritis based on deep learning and applied to computer tomography, belonging to the technical field of medical image segmentation. Background Art

[0002] Pyelonephritis is a common upper urinary tract infection, which is caused by various pathogenic microorganisms invading the renal pelvis and renal parenchyma, including acute pyelonephritis and chronic pyelonephritis. This disease can occur at any age. For children, the incidence rate is higher in infants under 3 years old. Acute pyelonephritis can generally be cured with proper treatment. If it is not treated properly, it may progress to chronic pyelonephritis, leading to persistent renal damage and scarring, and then causing hypertension and chronic renal failure. Ultrasound is the initial screening method for kidney disease, but due to the large image noise and low resolution, ultrasound often cannot detect lesions for early kidney disease. Renal biopsy is the gold standard for the diagnosis of kidney disease, but it is an invasive examination and may cause complications such as perinephric hematoma, hematuria, and dysuria, which is not accepted by most patients, especially children.

[0003] Computed tomography has become an important means of clinical diagnosis and evaluation of kidney disease due to its advantages of being non-invasive, high image quality, and being able to comprehensively evaluate the kidney and its surrounding spatial structures. Among them, dynamic enhanced scanning of computed tomography has a good effect on displaying kidney lesions. Accurate lesion segmentation is of great significance for the clinical diagnosis and prognosis of pyelonephritis. For the prior art, such as a whole heart segmentation method, device, equipment and medium based on computed tomography 3D images disclosed in publication number CN115760894A, the first stage is to detect the bounding box of the heart by the FasterR-CNN network, and the second stage is to input the original CT image of the heart aligned with the bounding box into 3DU-Net for heart substructure segmentation. The CIoU loss function is used. The experimental results show that the scheme achieves the most advanced segmentation effect on the 2017 Multimodal Whole Heart Segmentation Challenge (MM-WHS) dataset, reaching an average Dice score of 91.1%. At the same time, the segmentation time of a single cardiac CT image has been greatly improved from several minutes to less than 6 seconds, achieving faster and more accurate automatic segmentation of the whole heart. Since computed tomography examinations generate a large amount of imaging data, manual segmentation of kidney lesions from computed tomography images is a time-consuming and labor-intensive process, and is prone to differences in human assessment. Therefore, the development of automatic segmentation methods for pediatric kidney lesions has become an active research field in medical image analysis. Summary of the invention

[0004] The purpose of the present invention is to provide a method for automatic segmentation of children's kidneys and pyelonephritis based on deep learning applied to computed tomography in order to solve the problem that doctors need a lot of time to look at a single film in clinical practice and the problem that doctors may miss some pyelonephritis after looking at the film for a long time, so as to assist clinicians in their diagnosis and lay the foundation for the subsequent calculation of children's kidney volume and pyelonephritis volume.

[0005] The present invention achieves the above-mentioned object through the following technical scheme: a method for automatic segmentation of children's kidneys and pyelonephritis based on deep learning applied to computer tomography, comprising the following steps:

[0006] S1, local feature extraction stage: according to the children's kidney and pyelonephritis images to be segmented, the local features of the input image are obtained by local feature extraction;

[0007] S2, global feature extraction stage; the output of the local feature extraction stage is used as the input of the global feature extraction stage to extract the global feature information of the image;

[0008] S3, decoding upsampling stage; taking the output of the global feature extraction stage as input, after feature fusion with the output of the local feature extraction stage, the model learns the important information in the fused features and filters the redundant information through the designed adaptive attention module;

[0009] S4, segmentation stage; use convolution to convert the input into the segmented category for segmentation.

[0010] As a further solution of the present invention: the local feature extraction network structure includes five levels, specifically including:

[0011] The first level is a "convolutional layer 1 + group normalization + activation function + maximum pooling" network;

[0012] The second level is Block1, which contains three convolutional layer networks with the same 3-layer settings;

[0013] The third level is Black2, which contains four convolutional layer networks with the same 3-layer settings;

[0014] The fourth level is Block3, which contains 9 convolutional layer networks with the same 3-layer settings;

[0015] The fifth level is convolutional layer 2.

[0016] As a further solution of the present invention: the global feature extraction network structure specifically includes:

[0017] First, the channel size of the input image is changed through the convolution layer, then it is flattened and feature-concatenated with the position embedding layer, and then the generalization ability of the network is enhanced through the random inactivation layer; it passes through layer normalization and multi-head self-attention layer in turn, and then tensor concatenation with the random inactivation layer is performed, and then layer normalization and multi-layer perceptron are performed, and the output of the multi-layer perceptron is concatenated with the output of the previous tensor concatenation and finally output.

[0018] As a further solution of the present invention: the calculation formula of the position embedding layer is:

[0019]

[0020] in, is the location where the block is embedded, E pos ∈R N×D Is the embedding position of the current block.

[0021] As a further solution of the present invention: the calculation formula of the tensor splicing is:

[0022] z l ′=MSA(LN(z l-1 ))+z l-1 (2)

[0023] z l =MLP(LN(z l ′))+z l ′ (3)

[0024] Where l is the output of the current layer, LN(·) represents the layer normalization operator, and z l The encoded representation of the image.

[0025] As a further solution of the present invention: the decoding upsampling stage specifically includes:

[0026] Given an input image x∈R with spatial dimensions H and W and channels C H×W×C , the image x is first sent to the feature extraction module consisting of three blocks; the first block consists of three convolutional layers and one pooling layer, the second block enters four convolutional layers and one pooling layer, and the last block enters nine convolutional layers and one pooling layer. This design not only makes the receptive domain gradually increase from top to bottom, but also reduces the number of parameters by reducing the size of the feature map, thereby improving the extraction of local spatial information while enhancing the computational efficiency of the model.

[0027] As a further solution of the present invention: by introducing the idea of ​​dense connection, a new skip connection path is introduced to integrate U-shaped structures of different sizes into one network, providing feature maps of different scales for the decoder, so that the aggregation layer can decide how to fuse the various feature maps carried in the skip connection with the decoder feature map. The dense skip connection ensures that all prior feature maps are accumulated and reach the current node through the convolution blocks on each skip path. The input of each node is expressed as follows:

[0028]

[0029] Among them, H(·) represents the convolution operation, D(·) is downsampling, u(·) is upsampling, and x i,j Represents node X i,j The output of , i represents the encoder index downsampling layer, and j represents the skip connection index j-th convolutional layer of the dense block.

[0030] As a further solution of the present invention: the adaptive attention module is composed of two parts: an adaptive channel attention module and a spatial attention module;

[0031] The calculation formula of the adaptive channel attention module is as follows:

[0032]

[0033] in, and Represent the average pooling feature and the maximum pooling feature respectively, σ represents the activation function, C1D k represents a one-dimensional convolution involving k parameters, k is the convolution kernel size, C represents the number of channels, |·| odd Indicates that the convolution kernel k can only be an odd number. γ and b are set to 2 and 1 in this paper, which are mainly used to change the ratio between the number of channels C and the size of the convolution kernel. W0∈R C / r×C ,W1∈R C×C / r , MLP weights W0 and W1 are shared, and W0 is preceded by a ReLU activation function;

[0034] The calculation formula of the spatial attention feature map is as follows:

[0035] M s (F) = σ(f 7×7 ([AvgPool(F);MaxPool(F)])) (8)

[0036]

[0037] Among them, σ represents the activation function, f 7×7 represents a convolution operation with a filter size of 7×7,

[0038] The beneficial effects of the present invention are: designing a dense skip connection mode of long connections and short connections, and superimposing the extracted multi-layer local features at the current node and transmitting them to the decoder end;

[0039] An adaptive attention module is designed. Based on the extracted local features and global features, the two are fused and passed through the adaptive attention module, so that the model can learn the characteristics of children's kidneys and pyelonephritis, thereby improving the segmentation accuracy of the model for the edges of the kidneys and pyelonephritis. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the process of the present invention;

[0041] Figure 2 It is a schematic diagram of the local feature extraction structure of the present invention;

[0042] Figure 3 It is a schematic diagram of the global feature extraction structure of the present invention;

[0043] Figure 4 It is a structural schematic diagram of the upsampling stage of the present invention;

[0044] Figure 5 Schematic diagram of the adaptive attention structure of the present invention;

[0045] Figure 6 This is a segmentation effect diagram of the existing method of the present invention;

[0046] Figure 7 This is the segmentation effect diagram of the adaptive attention module of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] like Figures 1 to 5 As shown, a method for automatic segmentation of kidney and pyelonephritis in children based on deep learning applied to computed tomography includes the following steps:

[0050] S1, local feature extraction stage: according to the children's kidney and pyelonephritis images to be segmented, the local features of the input image are obtained by local feature extraction;

[0051] S2, global feature extraction stage; the output of the local feature extraction stage is used as the input of the global feature extraction stage to extract the global feature information of the image;

[0052] S3, decoding upsampling stage; taking the output of the global feature extraction stage as input, after feature fusion with the output of the local feature extraction stage, the model learns the important information in the fused features and filters the redundant information through the designed adaptive attention module;

[0053] S4, segmentation stage; use convolution to convert the input into the segmented category for segmentation.

[0054] Embodiment 2

[0055] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0056] The local feature extraction network structure contains five levels, including:

[0057] The first level is a "convolutional layer 1 + group normalization + activation function + maximum pooling" network;

[0058] The second level is Block1, which contains three convolutional layer networks with the same 3-layer settings;

[0059] The third level is Black2, which contains four convolutional layer networks with the same 3-layer settings;

[0060] The fourth level is Block3, which contains 9 convolutional layer networks with the same 3-layer settings;

[0061] The fifth level is convolutional layer 2.

[0062] Embodiment 3

[0063] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0064] The global feature extraction network structure specifically includes:

[0065] First, the channel size of the input image is changed through the convolution layer, then it is flattened and feature-concatenated with the position embedding layer, and then the generalization ability of the network is enhanced through the random inactivation layer; it passes through layer normalization and multi-head self-attention layer in turn, and then tensor concatenation with the random inactivation layer is performed, and then layer normalization and multi-layer perceptron are performed, and the output of the multi-layer perceptron is concatenated with the output of the previous tensor concatenation and finally output.

[0066] The calculation formula of the position embedding layer is:

[0067]

[0068] in, is the location where the block is embedded, E pos ∈R N×D Is the embedding position of the current block.

[0069] The calculation formula for tensor splicing is:

[0070] z l ′=MSA(LN(z l-1 ))+z l-1 (2)

[0071] z l =MLP(LN(z l ′))+z l ′ (3)

[0072] Where l is the output of the current layer, LN(·) represents the layer normalization operator, and z l The encoded representation of the image.

[0073] Embodiment 4

[0074] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0075] The decoding upsampling stage specifically includes:

[0076] Given an input image x∈R with spatial dimensions H and W and channels C H×W×C , the image x is first sent to the feature extraction module consisting of three blocks; the first block consists of three convolutional layers and one pooling layer, the second block enters four convolutional layers and one pooling layer, and the last block enters nine convolutional layers and one pooling layer. This design not only makes the receptive domain gradually increase from top to bottom, but also reduces the number of parameters by reducing the size of the feature map, thereby improving the extraction of local spatial information while improving the computational efficiency of the model.

[0077] By introducing the idea of ​​dense connection and a new skip connection path, U-shaped structures of different sizes are integrated into one network, providing feature maps of different scales for the decoder, so that the aggregation layer can decide how to fuse the various feature maps carried in the skip connection with the decoder feature map. Dense skip connections ensure that all prior feature maps are accumulated and reach the current node through the convolution blocks on each skip path. The input of each node is represented as follows:

[0078]

[0079] Among them, H(·) represents the convolution operation, D(·) is downsampling, u(·) is upsampling, and x i,j Represents node X i,jThe output of , i represents the encoder index downsampling layer, and j represents the skip connection index j-th convolutional layer of the dense block.

[0080] Embodiment 5

[0081] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0082] In S3, the adaptive attention module consists of two parts: the adaptive channel attention module and the spatial attention module. By integrating the attention modules of the channel dimension and the spatial dimension, meaningful features can be enhanced and useless features can be suppressed, so that the model can learn more accurate target edge information, thereby improving the segmentation accuracy.

[0083] The calculation formula of the adaptive channel attention module is as follows:

[0084]

[0085] in, and Represent the average pooling feature and the maximum pooling feature respectively, σ represents the activation function, C1D k represents a one-dimensional convolution involving k parameters, k is the convolution kernel size, C represents the number of channels, |·| odd Indicates that the convolution kernel k can only be an odd number. γ and b are set to 2 and 1 in this paper, which are mainly used to change the ratio between the number of channels C and the size of the convolution kernel. W0∈R C / r×C ,W1∈R C×C / r , MLP weights W0 and W1 are shared, and W0 is preceded by a ReLU activation function;

[0086] The calculation formula of the spatial attention feature map is as follows:

[0087] M s (F) = σ(f 7×7 ([AvgPool(F);MaxPool(F)])) (8)

[0088]

[0089] Among them, σ represents the activation function, f 7×7 represents a convolution operation with a filter size of 7×7,

[0090] Its parameter settings are as follows:

[0091] Tiers Dimensions of input and output channels Convolution kernel size Step Length Conv (16,3,512,512)×1 3 1

[0092] Embodiment 6

[0093] like Figure 6 to Figure 7As shown in the figure, a deep learning-based automatic segmentation method for pediatric kidney and pyelonephritis in computed tomography is applied. Figure 7 In the , other methods have the phenomenon of mis-segmentation or missed segmentation. The designed adaptive attention module can effectively alleviate the over-segmentation or missed segmentation of other methods. With the help of long and short connections and adaptive attention modules, the model is helpful for accurate segmentation, and excellent performance is achieved in the segmentation of children's kidneys and pyelonephritis.

[0094] Working principle: Long and short connections are used to establish dense skip connections in the encoding and decoding stages, increase the information flow between local feature layers, and improve the model's segmentation accuracy for the kidney and pyelonephritis area.

[0095] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0096] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for automatic segmentation of pediatric kidneys and pyelonephritis based on deep learning applied to computed tomography, characterized by: The following steps are involved: S1, local feature extraction stage; According to the children's kidney and pyelonephritis images to be segmented, local features of the input image are obtained by extracting local features; S2, global feature extraction stage; The output of the local feature extraction stage is used as the input of the global feature extraction stage to extract the global feature information of the image; S3, decoding upsampling stage; taking the output of the global feature extraction stage as input, after feature fusion with the output of the local feature extraction stage, the model learns the important information in the fused features and filters the redundant information through the designed adaptive attention module; S4, segmentation stage; use convolution to convert the input into the segmented category for segmentation; In S3, the adaptive attention module consists of two parts: an adaptive channel attention module and a spatial attention module; The calculation formula of the adaptive channel attention module is as follows: M c (F)=σ(C1D k (MLP(AvgPool(F)))+c1D k (MLP(MaxPool(F)))) (5) in, and Represent the average pooling feature and the maximum pooling feature respectively, σ represents the activation function, C1D k represents a one-dimensional convolution involving k parameters, k is the convolution kernel size, C represents the number of channels, |·| odd Indicates that the convolution kernel k can only be an odd number. γ and b are set to 2 and 1 in this paper, which are mainly used to change the ratio between the number of channels C and the size of the convolution kernel. W0∈R C / r×C ,W1∈R C×C / r , the MLP weights W0 and W1 are shared, and W0 is preceded by a ReLU activation function.

2. The method for automatic segmentation of children's kidneys and pyelonephritis based on deep learning applied to computed tomography according to claim 1, characterized in that: In S1, the local feature extraction network structure includes five levels, specifically including: The first level is a "convolutional layer 1 + group normalization + activation function + maximum pooling" network; The second level is Block1, which contains three convolutional layer networks with the same 3-layer settings; The third level is Black2, which contains four convolutional layer networks with the same 3-layer settings; The fourth level is Block3, which contains 9 convolutional layer networks with the same 3-layer settings; The fifth level is convolutional layer 2.

3. The method for automatic segmentation of children's kidney and pyelonephritis based on deep learning applied to computed tomography according to claim 1, characterized in that: In S2, the global feature extraction network structure specifically includes: First, the channel size of the input image is changed through the convolution layer, then it is flattened and feature-concatenated with the position embedding layer, and then the generalization ability of the network is enhanced through the random inactivation layer; it passes through layer normalization and multi-head self-attention layer in turn, and then tensor concatenation with the random inactivation layer is performed, and then layer normalization and multi-layer perceptron are performed, and the output of the multi-layer perceptron is concatenated with the output of the previous tensor concatenation and finally output.

4. The method for automatic segmentation of kidney and pyelonephritis in children based on deep learning applied to computed tomography according to claim 3, characterized in that: The calculation formula of the position embedding layer is: in, is the location where the block is embedded, E pos ∈R N×D Is the embedding position of the current block.

5. The method for automatic segmentation of children's kidneys and pyelonephritis based on deep learning applied to computed tomography according to claim 3, characterized in that: The calculation formula for tensor splicing is: z l ′=MSA(LN(z l-1 ))+z l-1 (2) With l =MLP(LN(z l ′))+z l ′ (3) Where l is the output of the current layer, LN(·) represents the layer normalization operator, and z l The encoded representation of the image.

6. The method for automatic segmentation of kidney and pyelonephritis in children based on deep learning applied to computed tomography according to claim 1, characterized in that: The S3 specifically includes: Given an input image x∈R with spatial dimensions H and W and channels C H×W×C , the image x is first fed into a feature extraction module consisting of three blocks; the first block consists of three convolutional layers and one pooling layer, the second block enters four convolutional layers and one pooling layer, and the last block enters nine convolutional layers and one pooling layer.

7. The method for automatic segmentation of kidney and pyelonephritis in children based on deep learning applied to computed tomography according to claim 6, characterized in that: By introducing the idea of ​​dense connection and a new skip connection path, U-shaped structures of different sizes are integrated into one network, providing feature maps of different scales for the decoder, so that the aggregation layer can decide how to fuse the various feature maps carried in the skip connection with the decoder feature map. Dense skip connections ensure that all prior feature maps are accumulated and reach the current node through the convolution blocks on each skip path. The input of each node is represented as follows: Among them, H(·) represents the convolution operation, D(·) is downsampling, u(·) is upsampling, and x i,j Represents node X i,j The output of , i represents the encoder index downsampling layer, and j represents the skip connection index j-th convolutional layer of the dense block.

8. The method for automatic segmentation of kidney and pyelonephritis in children based on deep learning applied to computed tomography according to claim 1, characterized in that: The calculation formula of the spatial attention feature map is as follows: M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])) (8) Among them, σ represents the activation function, f 7×7 represents a convolution operation with a filter size of 7×7,

Images