Pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features
By fusing regional context and high- and low-order semantic features in MRI images, a segmentation method was developed that addresses the issues of loss of fine structures and slow speed in MRI anterior visual pathway segmentation, achieving efficient and accurate anterior visual pathway segmentation.
Patent Information
- Application Number
- CN202311400418.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-26
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-10-26
AI Technical Summary
Existing deep learning-based MRI anterior visual pathway segmentation methods suffer from problems such as loss of fine structures in the anterior visual pathway, discontinuous segmentation results, fragmentation, and slow segmentation speed.
We employ a segmentation method that integrates regional context and high- and low-order semantic features. By building a deep learning hybrid model, we use deep separable convolutional layers with large kernels and regular convolutional layers alternately for feature extraction. We combine jump connectors to fuse high- and low-order semantic information and use learnable regular convolutional operations to replace pooling operations.
It improves the speed and accuracy of front visual pathway segmentation, generates more accurate high-resolution segmentation maps, maintains the alternating extraction of long-distance dependencies and local information, and reduces the consumption of computing resources.
Smart Images

Figure CN117315257B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of deep learning and intelligent medical image analysis technology, specifically to a pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features. Background Technology
[0002] The anterior visual pathway is the physiological basis of vision, responsible for transmitting all visual signals from the retina to the brain. It consists of a pair of optic nerves (our most important cranial nerves), the optic chiasm, and a pair of visual pathways. Precise and robust segmentation of the anterior visual pathway can help neurosurgeons with preoperative planning, prevent unintentional penetration of critical visual systems, and provide valuable reference information in the diagnosis and treatment of diseases such as optic gliomas.
[0003] Magnetic Resonance Imaging (MRI) is an important non-invasive imaging technique that produces detailed three-dimensional anatomical images. It is also an important tool used by physicians in analyzing and diagnosing diseases related to the anterior visual pathway and in planning surgical procedures. However, segmenting the anterior visual pathway in MRI images presents three challenges: First, the anterior visual pathway has a slender shape, a structure that changes continuously along the path, and lacks clear integration with adjacent anatomical structures; second, the anterior visual pathway structure in MRI images generally exhibits low contrast and blurred boundaries; third, the visual pathway occupies a very small proportion of the total head volume data. Traditional image processing algorithms are not effective in this field, often exhibiting problems such as unclear segmentation boundaries, undersegmentation, and oversegmentation. Therefore, using artificial intelligence algorithms to automatically segment the anterior visual pathway from MRI images has become the mainstream approach.
[0004] In recent years, deep learning, as an important branch of artificial intelligence, is data-driven and automatically and efficiently extracts features from raw images without human intervention. In the field of medical image analysis, it is widely used in tasks such as medical image segmentation because it can detect subtle differences in imaging features before doctors diagnose diseases, providing information for treatment. Convolutional Neural Networks (CNNs) have achieved great success in medical image segmentation, achieving superior segmentation results compared to traditional machine learning.
[0005] Currently, deep learning-based MRI anterior visual pathway segmentation methods take into account the combination of image intensity information with prior information on shape and position, as well as the influence of different modal data on the network segmentation accuracy. However, the inventors of this application have found through research that current deep learning-based MRI anterior visual pathway segmentation methods still have the following shortcomings: (1) The use of repetitive, continuous and single-size small convolutional kernels means that there is still much room for improvement in the network's modeling of long-distance dependencies; (2) The use of pooling layers without learnable parameters will inevitably ignore or lose detailed features during the jump translation process, resulting in blurring of the fine structure of the segmentation map; (3) The network depth is relatively deep and the number of network parameters is large, which results in a large amount of computational resources being consumed and a slow segmentation speed. Summary of the Invention
[0006] To address the technical problems of lost fine structures in the anterior visual pathway of MRI, discontinuities and breaks in the segmentation results, and slow segmentation speed in existing deep learning-based methods for anterior visual pathway segmentation, this invention provides a method for anterior visual pathway segmentation that integrates regional context and high- and low-order semantic features. This method constructs a deep learning hybrid model, alternately using deep separable convolutional layers with large kernels to model long-range dependencies and regular convolutional layers to capture detailed information, thus fusing contextual information from different regions. A jumper connector then fuses high- and low-order semantic information. Simultaneously, regular convolutional operations are used instead of pooling operations for downsampling, thereby reducing the loss of fine structures in the anterior visual pathway segmentation image.
[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0008] A pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features includes the following steps:
[0009] S1. Construction of a 3D MRI anterior visual pathway segmentation network model with an encoding and decoding fully convolutional network structure:
[0010] S11. The 3D MRI pre-visual pathway segmentation network with a fully convolutional network structure includes a feature encoder, a feature decoder, a skip connector, and a classifier. The feature encoder first performs coarse extraction of image features through non-overlapping convolutions, and then performs layer-by-layer extraction of image features at different scales through alternating convolutions with depthwise separable convolutional layers with large kernels and regular convolutional layers to obtain high-order and low-order semantic features, thus completing the alternating extraction and fusion of contexts of two different regions: long-distance dependencies and local information. The feature decoder fuses high-order and low-order semantic features layer by layer through transposed convolutions and skip connectors to achieve the same size as the original image to be segmented. The classifier maps the multi-channel feature map output by the feature decoder to the required number of segmentation categories and generates pixel-level segmentation results.
[0011] S12. The feature encoder includes a non-overlapping projection layer and four feature encoding layer groups arranged sequentially. The first feature encoding layer group consists of a depthwise separable convolutional layer with a large kernel. The second, third, and fourth feature encoding layer groups consist of a regular convolutional layer and a depthwise separable convolutional layer with a large kernel arranged sequentially. The skip connector includes five skip connection layer groups arranged sequentially. The first skip connection layer group consists of a convolutional layer. The second, third, fourth, and fifth skip connection layer groups consist of two convolutional layers arranged sequentially. The feature decoder includes four feature decoding layer groups arranged sequentially. Each feature decoding layer group consists of a transposed convolutional layer. The classifier consists of a class prediction layer and a softmax regression layer. The softmax regression layer converts the class prediction score into a probability distribution.
[0012] S2. Network model training and parameter optimization:
[0013] S21. Dataset Preparation: The original DICOM format MRI brain images collected from the hospital were cropped from 224×256×192 to 224×256×32 three-dimensional MRI data, and converted and saved in NIFTI format using ITK-SNAP software. Then, experts labeled the morphological features of the anterior visual pathway on this dataset, and each labeled label was saved separately as a three-dimensional image in NIFTI format, thus completing the preprocessing of the collected MRI brain images.
[0014] S22. Training dataset partitioning: The labeled dataset is divided into a training set and a validation set in a 4:1 ratio, and a 5-fold cross-validation method is used to train the segmentation network model.
[0015] S23. Network parameter initialization: The parameters of the 3D MRI anterior visual pathway segmentation network model built in step S1 are initialized using the Xavier method.
[0016] S24. The preprocessed MRI brain image is used as a single-channel input network, and feature vectors are generated through forward computation of the network.
[0017] S25. A combination of the dice loss function and the cross-entropy loss function is used as the objective optimization function of the segmentation network, defined as follows:
[0018]
[0019]
[0020]
[0021] in, It is the hybrid segmentation loss function, i.e., the objective optimization function of the segmentation network, in the segmentation network learning process. Represents the dice loss function. Represents the cross-entropy loss function. The weight value representing the loss from the dice roll. The weights representing the cross-entropy loss are... and These represent the actual labels and the automatic segmentation results, respectively.
[0022] S26, in order to The goal is to minimize the network model parameters using stochastic gradient descent and backpropagation algorithms. To obtain the optimal network model parameters ;
[0023] S3. Automatic semantic segmentation of the front visual pathway:
[0024] S31. Use the learned optimal network model parameters An automatic semantic segmentation network for the anterior visual pathway of 3D MRI was constructed.
[0025] S32. The original size of the MRI brain image is cropped from 224×256×192 to 224×256×32, and used as a single-channel input 3D MRI pre-visual pathway automatic semantic segmentation network.
[0026] S33. Generate a pixel category prediction probability distribution map through forward computation of the network, use a classifier to predict the pixel category, and generate two pixel-level label prediction probability distribution maps with a size of 224×256×32, which represent the probability of each pixel in the front visual path and background categories, respectively.
[0027] S34. Take the index of the component with the highest probability of each pixel as the pixel category label to form the final MRI pre-visual pathway segmentation map.
[0028] Further, in step S12, the kernel size of the non-overlapping projection layer is 2×2×2, and the stride is 2. The depthwise separable convolutional layer with large kernels consists of a depthwise convolutional layer with a kernel size of 9×9×9, a stride of 1, and padding of 4, and two pointwise convolutional layers with kernel size of 1×1×1 and a stride of 1. The depthwise separable convolutional layer with large kernels first performs channelwise convolution on the feature map with n channels using n kernels of size 9×9×9 to fuse spatial information. Then, it is concatenated according to the channel dimension, and pointwise convolution is performed using 4×n kernels of size 1×1×1 to expand the number of channels to four times the original number of channels. Then, pointwise convolution is performed using n kernels of size 1×1×1 to shrink the number of channels back to the original number of channels n, thus fusing channel information. The kernel size of the regular convolutional layer is 3×3×3, the stride is 2, and the padding is 4. The first skip connection layer group consists of a 3×3×3 convolutional layer with a stride of 1 and padding of 1. This convolutional layer is responsible for feature encoding and increasing the number of channels in the feature map. The second, third, fourth, and fifth skip connection layer groups consist of two convolutional layers with a kernel size of 3×3×3, a stride of 1, and padding of 1. The first convolutional layer is responsible for feature encoding and increasing the number of channels in the feature map, while the second convolutional layer is responsible for feature decoding and reducing the number of channels in the feature map after concatenation by channel dimension. The transposed convolutional layer has a kernel size of 2×2×2 and a stride of 2, performing a 2x upsampling to increase the resolution of the feature map, reducing the number of output channels to half of the original, and concatenating it with the transferred feature encoding map of the same resolution. The class prediction layer has a kernel size of 1×1×1 and a stride of 1.
[0029] Furthermore, in step S22, data augmentation is performed on the training images in the training set by randomly flipping, randomly rotating, and changing the intensity of the images.
[0030] Furthermore, the forward computation of the network in step S24 includes convolution operations, nonlinear activation, batch normalization, and probability value transformation.
[0031] Furthermore, in the convolution operation, the output feature map corresponding to any convolution kernel The following formula is used for calculation:
[0032]
[0033] in, Represents a nonlinear excitation function. Indicates the first The bias terms corresponding to each convolution kernel Indicates the input channel index number. Indicates the number of input channels. Indicates the first The first convolutional kernel Channel weight matrix, It's a convolution operation. Indicates the first One input channel image.
[0034] Furthermore, the nonlinear activation uses Gaussian error linear units (GELUs) as feature maps. The nonlinear activation function, the Gaussian error linear unit (GELU), is defined as follows:
[0035]
[0036] in, Represents the Gaussian error function. This represents a feature map.
[0037] Furthermore, the batch normalization is used to calculate the mean and variance of each mini-batch and pull them back to a standard normal distribution with a mean of 0 and a variance of 1, as specifically defined below:
[0038] , ,
[0039] ,
[0040] in, It is a batch collection. It is the average of a batch. It is the variance of a batch. It is the first in a batch of data Feature values of each sample Yes Normalization, It is a small positive number to prevent the denominator from being zero. , These are scaling and translation variables, It is the first The value obtained after normalizing the data.
[0041] Furthermore, the probability value conversion uses The function transforms the predicted class scores into a probability distribution. The function is defined as follows:
[0042]
[0043] in, It belongs to the category of The probability of a class , It is the pixel in the final output of the network at the 1st position. , No. Predicted scores for each class Represents the number of categories.
[0044] Furthermore, in step S26, when using the stochastic gradient descent optimizer for iteration, the batch size (number of samples used in each iteration) is set to 4, and the initial learning rate is set to 1e. -4 The training is stopped if the loss on the validation set does not decrease after 30 rounds using an early stopping technique.
[0045] Compared with existing technologies, the pre-visual pathway segmentation method provided by this invention, which integrates regional context and high- and low-order semantic features, has the following advantages: (1) It uses a deep neural network for 3D MRI pre-visual pathway segmentation, without the need to extract image blocks. It inputs the entire image for end-to-end training, which is simple; (2) It can complete the automatic segmentation of the pre-visual pathway in one forward computation, which can improve the segmentation speed of the pre-visual pathway; (3) It adopts deep convolution with large convolution kernels and pointwise convolution, which enables the network to model long-distance dependencies while maintaining a low number of parameters and computational cost, further improving the segmentation speed and making full use of computing resources; (4) The network model architecture of alternating deep separable convolutional layers with large convolution kernels and convolutional layers with regular small convolution kernels enables long-distance dependencies and local information to be extracted alternately and preserved to the greatest extent; (5) It uses learnable regular convolution operations to replace pooling operations to complete downsampling operations, thereby improving the resolution of the segmentation map, making the pre-visual pathway segmentation boundary more accurate, generating a fine-grained segmentation map with more segmentation details, and thus obtaining a more accurate high-resolution pre-visual pathway segmentation map. Attached Figure Description
[0046] Figure 1 This is a technical roadmap for the pre-visual pathway segmentation method that integrates regional context and high- and low-order semantic features provided by the present invention.
[0047] Figure 2 This is a schematic diagram of the 3D MRI anterior visual pathway segmentation network model provided by the present invention.
[0048] Figure 3 This is a schematic diagram of the depth-separable convolutional layer structure provided by the present invention. Detailed Implementation
[0049] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.
[0050] Please refer to Figures 1 to 3As shown, this invention provides a pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features, comprising the following steps:
[0051] S1. Construction of a 3D MRI anterior visual pathway segmentation network model with an encoding and decoding fully convolutional network structure:
[0052] The S11 3D MRI anterior visual pathway segmentation network is mainly used to generate anterior visual pathway segmentation maps of input 3D MRI images. This fully convolutional network structure includes a feature encoder, a feature decoder, a skip connector, and a classifier. The feature encoder first performs coarse extraction of image features through non-overlapping convolutions, then alternates between depthwise separable convolutional layers with large kernels and regular convolutional layers to extract high-order and low-order semantic features layer by layer at different scales. The receptive field of the feature map continuously expands while the resolution continuously decreases, completing the alternating extraction and fusion of contexts of different region sizes: long-distance dependencies and local information. The feature decoder, through transposed convolutions and skip connectors, fuses high-order and low-order semantic features layer by layer, continuously improving the feature map resolution to the same size as the original image to be segmented. The classifier maps the multi-channel feature map output by the feature decoder to the required number of segmentation categories and generates pixel-level segmentation results.
[0053] S12. The feature encoder includes a non-overlapping projection layer and four feature encoding layer groups arranged sequentially. The first feature encoding layer group consists of a depthwise separable convolutional layer with a large convolutional kernel. The second, third, and fourth feature encoding layer groups consist of a regular convolutional layer and a depthwise separable convolutional layer with a large convolutional kernel arranged sequentially. The skip connector includes five skip connection layer groups arranged sequentially. The first skip connection layer group consists of a convolutional layer. The second, third, fourth, and fifth skip connection layer groups consist of two convolutional layers arranged sequentially. The feature decoder includes four feature decoding layer groups arranged sequentially. Each feature decoding layer group consists of a transposed convolutional layer. The classifier consists of a class prediction layer and a softmax regression layer. The softmax regression layer converts the class prediction score into a probability distribution.
[0054] In a specific embodiment, the kernel size of the non-overlapping projection layer in step S12 is 2×2×2, and the stride is 2. The depthwise separable convolutional layer with large kernels consists of a depthwise convolutional layer with a kernel size of 9×9×9, a stride of 1, and padding of 4, and two pointwise convolutional layers with a kernel size of 1×1×1 and a stride of 1. The convolution operation of the depthwise separable convolutional layer with large kernels is as follows: Figure 3As shown, firstly, n 9×9×9 convolutional kernels are used to perform channel-wise convolution on the feature map with n channels to fuse spatial information. Then, they are concatenated according to the channel dimension. Next, 4×n 1×1×1 convolutional kernels are used for pointwise convolution to expand the number of channels to four times the original number of channels. Then, n 1×1×1 convolutional kernels are used for pointwise convolution to shrink the number of channels back to the original number n, fusing channel information. The regular convolutional layer has a 3×3×3 kernel size, a stride of 2, and padding of 1, performing a 2x downsampling to reduce resolution and remove redundant features in a learnable manner. The first skip connection layer consists of a 3×3×3 kernel with a stride of... The first layer consists of convolutional layers with padding of 1, which are responsible for feature encoding and increasing the number of channels in the feature map. The second, third, fourth, and fifth skip connection layers consist of two convolutional layers with kernel size of 3×3×3, stride of 1, and padding of 1. The first convolutional layer is responsible for feature encoding and increasing the number of channels in the feature map, while the second convolutional layer is responsible for feature decoding and reducing the number of channels in the feature map after concatenation by channel dimension. The transposed convolutional layer has a kernel size of 2×2×2 and a stride of 2, which performs upsampling by 2x to improve the resolution of the feature map, reduces the number of output channels to half of the original, and concatenates it with the transferred feature encoding map of the same resolution. The class prediction layer has a kernel size of 1×1×1 and a stride of 1. The entire segmentation network takes a 1-channel input, with each channel being a 224×256×32 3D MRI image; the network outputs a 2-channel output, with each channel being 224×256×32, representing the anterior visual pathway segmentation map and the background map, respectively. Detailed model parameters are shown in Table 1 below.
[0055] Table 1. Parameters of the 3D MRI Anterior Visual Pathway Segmentation Network Model
[0056] Serial Number Layer name nuclear size Step length Padding Number of convolution kernels enter Output Function 1 Convolutional layer 1_1+BN 2×2×2 2 — 48 1×224×256×32 48×112×128×16 Feature encoding 2 Depth-first convolution 2_1+BN 9×9×9 1 4 48 48×112×128×16 48×112×128×16 Feature encoding 3 Pointwise convolutional layer 2_1+BN 1×1×1 1 — 192 48×112×128×16 192×112×128×16 Feature encoding 4 Pointwise convolutional layer 2_2+GELU 1×1×1 1 — 48 192×112×128×16 48×112×128×16 Feature encoding 5 Convolutional layer 3_1+BN 3×3×3 2 1 96 48×112×128×16 96×56×64×8 Feature encoding 6 Depth-wise convolution 4_1+BN 9×9×9 1 4 96 96×56×64×8 96×56×64×8 Feature encoding 7 Pointwise convolutional layers 4_2+BN 1×1×1 1 — 384 96×56×64×8 384×56×64×8 Feature encoding 8 Pointwise convolutional layers 4_3+GELU 1×1×1 1 — 96 384×56×64×8 96×56×64×8 Feature encoding 9 5_1+BN convolutional layers 3×3×3 2 1 192 96×56×64×8 192×28×32×4 Feature encoding 10 Depth-wise convolution 6_1+BN 9×9×9 1 4 192 192×28×32×4 192×28×32×4 Feature encoding 11 Pointwise convolutional layers 6_2+GELU 1×1×1 1 — 768 192×28×32×4 768×28×32×4 Feature encoding 12 Pointwise convolutional layers 6_3+GELU 1×1×1 1 — 192 768×28×32×4 192×28×32×4 Feature encoding 13 Convolutional layer 7_1+BN 3×3×3 2 1 384 192×28×32×4 384×14×16×2 Feature encoding 14 Depth-first convolution 8_1+BN 9×9×9 1 4 384 384×14×16×2 384×14×16×2 Feature Decoding 15 8_2+GELU pointwise convolutional layers 1×1×1 1 — 1536 384×14×16×2 1536×14×16×2 Feature encoding 16 8_3+GELU pointwise convolutional layers 1×1×1 1 — 384 1536×14×16×2 384×14×16×2 Feature encoding 17 Convolutional layer 9_1+GELU 3×3×3 1 1 768 384×14×16×2 768×14×16×2 Feature encoding 18 Transposed convolution 2×2×2 2 — 384 768×14×16×2 384×28×32×4 Feature Decoding 19 10 convolutional layers + GELU 3×3×3 1 1 384 192×28×32×4 384×28×32×4 Feature encoding 20 102+GELU convolutional layers 3×3×3 1 1 384 768×28×32×4 384×28×32×4 Feature Decoding 21 Transposed convolution 2×2×2 2 — 192 384×28×32×4 192×56×64×8 Feature Decoding 22 Convolutional layer 11_1+GELU 3×3×3 1 1 192 96×56×64×8 192×56×64×8 Feature encoding 23 Convolutional layer 11_2+GELU 3×3×3 1 1 192 384×56×64×8 192×56×64×8 Feature Decoding 24 Transposed convolution 2×2×2 2 — 96 192×56×64×8 96×112×128×16 Feature Decoding 25 Convolutional layer 12_1+GELU 3×3×3 1 1 96 48×112×128×16 96×112×128×16 Feature encoding 26 Convolutional layer 12_2+GELU 3×3×3 1 1 96 192×112×128×16 96×112×128×16 Feature Decoding 27 Transposed convolution 2×2×2 2 — 48 96×112×128×16 48×224×256×32 Feature Decoding 28 Convolutional layer 13_1+GELU 3×3×3 1 1 48 1×224×256×32 48×224×256×32 Feature encoding 29 Convolutional layer 13_2+GELU 3×3×3 1 1 48 96×224×256×32 48×224×256×32 Feature Decoding 30 Category prediction layer + softmax 1×1×1 1 — 2 48×224×256×32 2×224×256×32 Probability Prediction
[0057] In Table 1 above, in order to ensure that the feature map size remains unchanged during the convolution process, Padding=1 is set, that is, 0 is used to fill the area around the image during convolution.
[0058] S2. Network model training and parameter optimization:
[0059] S21. Dataset Preparation: The original DICOM format MRI brain images collected from the hospital were cropped from 224×256×192 to 224×256×32 three-dimensional MRI data, and converted and saved in NIFTI format using ITK-SNAP software. Then, experts labeled the morphological features of the anterior visual pathway on this dataset, and each labeled label was saved separately as a three-dimensional image in NIFTI format, thus completing the preprocessing of the acquired MRI brain images.
[0060] S22. Training Dataset Partitioning: The labeled dataset is divided into a training set and a validation set in a 4:1 ratio. A 5-fold cross-validation method is used to train the segmentation network model. As a specific embodiment, the inventors of this application obtained 161 3D MRI brain images with segmentation labels. The training samples and their corresponding labels are both 224×256×32 pixels. Data augmentation techniques such as random flipping, random rotation, and changing image intensity are used on the training images in the training set to enable the network model to acquire more useful information, thereby improving the model's generalization ability.
[0061] S23. Network parameter initialization: The parameters of the 3D MRI anterior visual pathway segmentation network model built in step S1 are initialized using the Xavier method.
[0062] S24. The preprocessed MRI brain image is used as a single-channel input network, and feature vectors are generated through forward computation of the network. In a specific embodiment, the forward computation of the network includes convolution operations, nonlinear activation, batch normalization, and probability value transformation.
[0063] Convolution operation: The output feature map corresponding to any convolution kernel The following formula is used for calculation:
[0064]
[0065] in, Represents a nonlinear excitation function. Indicates the first The bias terms corresponding to each convolution kernel Indicates the input channel index number. Indicates the number of input channels. Indicates the first The first convolutional kernel Channel weight matrix, It's a convolution operation. Indicates the first One input channel image.
[0066] Nonlinear activation: Gaussian error linear unit (GELU) is used as the feature map. The nonlinear activation function, the Gaussian error linear unit (GELU), is defined as follows:
[0067]
[0068] in, Represents the Gaussian error function. This represents a feature map.
[0069] Batch Normalization (BN) is a method that makes neural network training faster and more stable. It is used to calculate the mean and variance of each mini-batch and bring them back to a standard normal distribution with a mean of 0 and a variance of 1. The specific definition is as follows:
[0070] , ,
[0071] ,
[0072] in, It is a batch collection. It is the average of a batch. It is the variance of a batch. It is the first in a batch of data Feature values of each sample Yes Normalization, It is a small positive number to prevent the denominator from being zero. , These are scaling and translation variables, It is the first The value obtained after normalizing the data.
[0073] Probability value conversion: using The function transforms the predicted class scores into a probability distribution. The function is defined as follows:
[0074]
[0075] in, It belongs to the category of The probability of a class , It is the pixel in the final output of the network at the 1st position. , No. Predicted scores for each class This represents the number of categories, which is 2 in this example.
[0076] S25. A combination of the dice loss function and the cross-entropy loss function is used as the objective optimization function of the segmentation network, defined as follows:
[0077]
[0078]
[0079]
[0080] in, It is the hybrid segmentation loss function, i.e., the objective optimization function of the segmentation network, in the segmentation network learning process. Represents the dice loss function. Represents the cross-entropy loss function. The weight value representing the loss from the dice roll. The weights representing the cross-entropy loss are... and These represent the real labels and the automatic segmentation results, respectively. and These two values should not be less than 0, and are usually set to 1.
[0081] S26, in order to The goal is to minimize the network model parameters using stochastic gradient descent and backpropagation algorithms. To obtain the optimal network model parameters This step uses the Stochastic Gradient Descent (SGD) optimizer for optimization computation. Each parameter update utilizes a subset of samples, thus reducing convergence volatility and the variance of parameter updates, resulting in more stable updates. Compared to full gradient descent, it not only increases the speed of each learning iteration but also eliminates memory bottlenecks, allowing for efficient computation using matrix operations. As a specific example, in this step, when using the stochastic gradient descent optimizer for iteration, the batch size (number of samples used per iteration) is set to 4, and the initial learning rate is set to 1e. -4 The training is stopped if the loss on the validation set does not decrease after 30 rounds using an early stopping technique.
[0082] S3. Automatic semantic segmentation of the front visual pathway:
[0083] S31. Use the learned optimal network model parameters An automatic semantic segmentation network for the anterior visual pathway of 3D MRI was constructed.
[0084] S32. The original size of the MRI brain image is cropped from 224×256×192 to 224×256×32, and used as a single-channel input 3D MRI pre-visual pathway automatic semantic segmentation network.
[0085] S33. Generate a pixel category prediction probability distribution map through forward computation of the network, use a classifier to predict the pixel category, and generate two pixel-level label prediction probability distribution maps with a size of 224×256×32, which represent the probability of each pixel in the front visual path and background categories, respectively.
[0086] S34. Take the index of the component with the highest probability of each pixel as the pixel category label to form the final MRI pre-visual pathway segmentation map.
[0087] Compared with existing technologies, the pre-visual pathway segmentation method provided by this invention, which integrates regional context and high- and low-order semantic features, has the following advantages: (1) It uses a deep neural network for 3D MRI pre-visual pathway segmentation, without the need to extract image blocks. The entire image is input for end-to-end training, which is simple; (2) The pre-visual pathway can be automatically segmented in one forward computation, which can improve the segmentation speed of the pre-visual pathway; (3) The use of deep convolution with large kernels and pointwise convolution enables the network to model long-range dependencies while maintaining a low number of parameters and computational cost, further improving the segmentation speed and making full use of computing resources; (4) The network model architecture of alternating deep separable convolutional layers with large kernels and convolutional layers with regular small kernels enables long-range dependencies and local information to be extracted alternately and preserved to the greatest extent; (5) The use of learnable regular convolution operations instead of pooling operations to complete the downsampling operation improves the resolution of the segmentation map, makes the pre-visual pathway segmentation boundary more accurate, and generates a fine-grained segmentation map with more segmentation details, thereby obtaining a more accurate high-resolution pre-visual pathway segmentation map.
[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features, characterized in that, Includes the following steps: S1. Construction of a 3D MRI anterior visual pathway segmentation network model with an encoding and decoding fully convolutional network structure: S11. The 3D MRI pre-visual pathway segmentation network with a fully convolutional network structure includes a feature encoder, a feature decoder, a skip connector, and a classifier. The feature encoder first performs coarse extraction of image features through non-overlapping convolution, and then performs layer-by-layer extraction of image features at different scales through alternating convolutions of depthwise separable convolutional layers with large convolutional kernels and regular convolutional layers to obtain high-order semantic features and low-order semantic features, thereby completing the alternating extraction and fusion of contexts of two different regions: long-distance dependencies and local information. The feature decoder fuses high-order semantic features with low-order semantic features layer by layer through transposed convolution and skip connectors, achieving the same size as the original image to be segmented; the classifier maps the multi-channel feature map output by the feature decoder to the required number of segmentation categories and generates pixel-level segmentation results. S12. The feature encoder includes a non-overlapping projection layer and four feature encoding layer groups arranged sequentially. The first feature encoding layer group consists of a depthwise separable convolutional layer with a large kernel. The second, third, and fourth feature encoding layer groups consist of a regular convolutional layer and a depthwise separable convolutional layer with a large kernel arranged sequentially. The skip connector includes five skip connection layer groups arranged sequentially. The first skip connection layer group consists of a convolutional layer. The second, third, fourth, and fifth skip connection layer groups consist of two convolutional layers arranged sequentially. The feature decoder includes four feature decoding layer groups arranged sequentially. Each feature decoding layer group consists of a transposed convolutional layer. The classifier consists of a class prediction layer and a softmax regression layer. The softmax regression layer converts the class prediction score into a probability distribution. S2. Network model training and parameter optimization: S21. Dataset Preparation: The original DICOM format MRI brain images collected from the hospital were cropped from 224×256×192 to 224×256×32 three-dimensional MRI data, and converted and saved in NIFTI format using ITK-SNAP software. Then, experts labeled the morphological features of the anterior visual pathway on this dataset, and each labeled label was saved separately as a three-dimensional image in NIFTI format, thus completing the preprocessing of the collected MRI brain images. S22. Training dataset partitioning: The labeled dataset is divided into training and validation sets in a 4:1 ratio, and the segmentation network model is trained using 5-fold cross-validation. S23. Network parameter initialization: The parameters of the 3D MRI anterior visual pathway segmentation network model built in step S1 are initialized using the Xavier method. S24. The preprocessed MRI brain image is used as a single-channel input network, and feature vectors are generated through forward computation of the network. S25. A combination of the dice loss function and the cross-entropy loss function is used as the objective optimization function of the segmentation network, defined as follows: in, It is the hybrid segmentation loss function, i.e., the objective optimization function of the segmentation network, in the segmentation network learning process. Represents the dice loss function. Represents the cross-entropy loss function. The weight value representing the loss from the dice roll. The weights representing the cross-entropy loss are... and These represent the actual labels and the automatic segmentation results, respectively. S26, in order to The goal is to minimize the network model parameters using stochastic gradient descent and backpropagation algorithms. To obtain the optimal network model parameters ; S3. Automatic semantic segmentation of the front visual pathway: S31. Use the learned optimal network model parameters An automatic semantic segmentation network for the anterior visual pathway of 3D MRI was constructed. S32. The original size of the MRI brain image is cropped from 224×256×192 to 224×256×32, and used as a single-channel input 3D MRI pre-visual pathway automatic semantic segmentation network. S33. Generate a pixel category prediction probability distribution map through forward computation of the network, use a classifier to predict the pixel category, and generate two pixel-level label prediction probability distribution maps with a size of 224×256×32, which represent the probability of each pixel in the front visual path and background categories, respectively. S34. Take the index of the component with the highest probability of each pixel as the pixel category label to form the final MRI pre-visual pathway segmentation map.
2. The pre-visual pathway segmentation method that integrates region context and high- and low-order semantic features according to claim 1, characterized in that, In step S12, the non-overlapping projection layer has a 2×2×2 kernel size and a stride of 2. The depthwise separable convolutional layer with large kernels consists of a depthwise convolutional layer with a kernel size of 9×9×9, a stride of 1, and padding of 4, and two pointwise convolutional layers with kernel sizes of 1×1×1 and strides of 1. The depthwise separable convolutional layer with large kernels first performs channel-wise convolution on the feature map with n channels using n 9×9×9 kernels to fuse spatial information. Then, the layers are concatenated along the channel dimension, and pointwise convolution is performed using 4×n convolutional kernels of size 1×1×1 to expand the number of channels to four times the original number of channels. Then, pointwise convolution is performed using n convolutional kernels of size 1×1×1 to shrink the number of channels back to the original number of channels n, thus fusing the channel information. The kernel size of the regular convolutional layer is 3×3×3, the stride is 2, and the padding is 1, which performs 2x downsampling to reduce the resolution and remove redundant features in a learnable way. The first skip connection layer group consists of a convolutional layer with a kernel size of 3×3×3, a stride of 1, and padding of 1. This convolutional layer is responsible for feature encoding and increasing the number of channels in the feature map. The second, third, fourth, and fifth skip connection layers consist of two convolutional layers with a kernel size of 3×3×3, a stride of 1, and padding of 1. The first convolutional layer is responsible for feature encoding and increasing the number of channels in the feature map, while the second convolutional layer is responsible for feature decoding and reducing the number of channels in the feature map after concatenation by channel dimension. The transposed convolutional layer has a kernel size of 2×2×2 and a stride of 2, performs a 2x upsampling to increase the resolution of the feature map, reduces the number of output channels to half of the original, and concatenates it with the transferred feature encoding map of the same resolution. The class prediction layer has a kernel size of 1×1×1 and a stride of 1.
3. The pre-visual pathway segmentation method according to claim 1, which integrates region context and high- and low-order semantic features, is characterized in that... In step S22, data augmentation is performed on the training images in the training set by randomly flipping, randomly rotating, and changing the intensity of the images.
4. The pre-visual pathway segmentation method according to claim 1, which integrates region context and high- and low-order semantic features, is characterized in that... The forward computation of the network in step S24 includes convolution operations, nonlinear activation, batch normalization, and probability value transformation.
5. The pre-visual pathway segmentation method according to claim 4, which integrates region context and high- and low-order semantic features, is characterized in that... In the convolution operation, the output feature map corresponding to any convolution kernel The following formula is used for calculation: in, Represents a nonlinear excitation function. Indicates the first The bias terms corresponding to each convolution kernel Indicates the input channel index number. Indicates the number of input channels. Indicates the first The first convolutional kernel Channel weight matrix, It's a convolution operation. Indicates the first One input channel image.
6. The pre-visual pathway segmentation method according to claim 4, which integrates region context and high- and low-order semantic features, is characterized in that... The nonlinear activation uses Gaussian error linear units (GELUs) as feature maps. The nonlinear activation function, the Gaussian error linear unit (GELU), is defined as follows: in, Represents the Gaussian error function. This represents a feature map.
7. The previsual pathway segmentation method according to claim 4, which integrates region context and high- and low-order semantic features, is characterized in that... The batch normalization is used to calculate the mean and variance of each mini-batch and pull them back to a standard normal distribution with a mean of 0 and a variance of 1, as defined below: , , , in, It is a batch collection. It is the average of a batch. It is the variance of a batch. It is the first in a batch of data Feature values of each sample Yes Normalization, It is a small positive number to prevent the denominator from being zero. , These are scaling and translation variables, It is the first The value obtained after normalizing the data.
8. The pre-visual pathway segmentation method according to claim 4, which integrates region context and high- and low-order semantic features, is characterized in that... The probability value conversion uses The function transforms the predicted category scores into a probability distribution. The function is defined as follows: in, It belongs to the category of The probability of a class , It is the pixel in the final output of the network at the 1st position. , No. Predicted scores for each class Represents the number of categories.
9. The previsual pathway segmentation method according to claim 1, which integrates region context and high- and low-order semantic features, is characterized in that... The batch size used in each iteration of the random gradient descent optimizer used in step S26 is set to 4, and the initial learning rate is set to 1e -4 Early stopping is used, and the training is stopped when the loss on the validation set does not decrease for 30 rounds.
Citation Information
Patent Citations
CT image medial rectus and optic nerve segmentation method and system
CN112184720A
Semantic segmentation method based on improved multi-scale full convolutional network
CN114220098A
