Medical image segmentation method based on full convolutional neural network

Abstract

The invention discloses a medical image segmentation method based on a full convolutional neural network, and the method comprises the steps: carrying out the preprocessing of a medical image through a deep learning algorithm for the complex anatomical structure and pathological features of the medical image, enhancing the key features in the image, and providing a richer information basis for the subsequent feature extraction; in the feature extraction stage of the full convolutional neural network, an attention mechanism module and a spatial pyramid pooling module are introduced to learn importance weights of different areas in an image, weight distribution of a convolution kernel is adaptively adjusted, and features with finer granularity and discrimination capability are extracted. The up-sampling process is optimized through a deconvolution operation and a jump connection structure, a conditional random field model is introduced to carry out post-processing optimization, a multi-scale feature fusion module is constructed, and the generalization ability of the network is improved by adopting an adversarial learning strategy. According to the invention, the accuracy and robustness of medical image segmentation are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a medical image segmentation method based on a fully convolutional neural network. Background Art

[0002] In the field of medical image segmentation, fully convolutional neural networks face a key technical challenge. Since medical images usually have complex anatomical structures and pathological features, traditional fully convolutional neural networks often find it difficult to capture sufficiently rich and fine-grained feature information in the feature extraction stage. This results in the network being unable to accurately restore the spatial details of the original image during the subsequent upsampling and pixel-level classification process, thus affecting the accuracy of the segmentation results. Specifically, when the network extracts features from medical images, the continuous operations of the convolution layer and the pooling layer will inevitably lose some spatial information. Although this information loss is acceptable in general image segmentation tasks, it may cause key anatomical structures or lesion areas to be ignored in medical image segmentation. At the same time, the deconvolution or transposed convolution operations used in the upsampling process are also difficult to fully restore the lost spatial details, further affecting the accuracy of pixel-level classification. This technical challenge highlights the uniqueness and complexity of the medical image segmentation task, requiring researchers to explore more sophisticated and intelligent feature extraction, upsampling, and classification strategies in order to obtain more accurate and reliable segmentation results, providing strong support for subsequent diagnosis and treatment. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a medical image segmentation method based on a fully convolutional neural network, comprising: Preprocess medical images through deep learning algorithms to enhance key image features; In the process of feature extraction based on the fully convolutional neural network, the attention mechanism module is introduced to learn the importance weights of different areas in the image, adaptively adjust the weight distribution of the convolution kernel, and extract fine-grained and discriminative image features; Multi-scale feature fusion and context information capture are performed through the spatial pyramid pooling module embedded in the fully convolutional neural network; The fused feature map is obtained by multi-scale feature fusion, and the fused feature map is gradually restored to the same spatial size as the original image through upsampling operation; Based on the upsampled feature map, each pixel is classified through convolution operation to obtain the pixel-level segmentation result; An adversarial learning strategy is introduced. By constructing a generator and a discriminator network, the segmentation results of the fully convolutional neural network are adversarially trained with the real annotations to obtain the final segmentation results.

[0004] Preferably, the process of preprocessing the medical image by a deep learning algorithm to enhance the key features of the image includes: Acquire medical image data and use deep learning algorithms to analyze and identify complex anatomical structures and pathological features in the images; Preprocess medical images, adjust the grayscale distribution of images through histogram equalization algorithm, and enhance the contrast of images; According to the characteristics of the image, an adaptive contrast adjustment algorithm is used to specifically enhance the contrast of local areas of the image and highlight the pathological features.

[0005] Preferably, the process of introducing the attention mechanism module to extract and obtain fine-grained and discriminative image features includes: Construct a fully convolutional neural network model, introduce an attention mechanism module in the feature extraction stage, generate an attention weight matrix by calculating the importance weights of different regions in the medical image, perform element-wise multiplication of the attention weight matrix with the convolution kernel weight matrix, and obtain an adjusted convolution kernel weight distribution; The medical image is convolved using the adjusted convolution kernel to extract the features of the key anatomical structures and lesion areas in the image, and obtain fine-grained and discriminative feature maps and corresponding image features.

[0006] Preferably, the process of multi-scale feature fusion and context information capture by embedding a spatial pyramid pooling module in a fully convolutional neural network includes: Based on the spatial pyramid pooling module, multi-scale pooling operations are performed on fine-grained and discriminative image features to obtain feature representations at different scales; Multi-scale feature fusion is performed based on the feature representations at different scales, and context information in the feature map is captured at the same time to enhance the semantic representation capability of the features.

[0007] Preferably, the process of classifying each pixel through a convolution operation based on the feature map obtained by upsampling to obtain a pixel-level segmentation result includes: Deconvolution is used to replace the traditional transposed convolution operation for upsampling, and the convolution kernel parameters in the upsampling process are learned; at the same time, a skip connection structure is introduced to fuse the high-resolution features of the shallow network with the high-level semantic features of the deep network to obtain the upsampling result; Then, pixel-level classification is performed. The spatial relationship and category dependency between pixels are modeled through the conditional random field model. The segmentation results output by the full convolutional neural network are post-processed and optimized to smooth the segmentation boundaries and eliminate isolated misclassified pixels. A multi-scale feature fusion module is constructed to extract multi-scale feature maps at different levels of the full convolutional neural network. Through feature fusion operations, feature information of different scales is integrated to obtain a more comprehensive and robust feature representation.

[0008] Preferably, the process of using a deconvolution operation to replace a traditional transposed convolution operation for upsampling includes: The input low-resolution image is used to extract deep semantic features using a pre-trained convolutional neural network; According to a preset upsampling multiple, upsampling the deep semantic features through a deconvolution operation to obtain a first upsampling feature map; Obtain shallow high-resolution features corresponding to the input image, and fuse the shallow high-resolution features with the first up-sampled feature map through a skip connection to obtain a second up-sampled feature map; According to the second up-sampled feature map, further improving the resolution through a deconvolution operation to obtain a third up-sampled feature map; According to the third up-sampled feature map, a convolution operation is used to perform detail enhancement to obtain a fourth up-sampled feature map; According to the fourth up-sampled feature map, converting into a high-resolution output image through a pixel rearrangement operation; According to the high-resolution output image and the original high-resolution image, the reconstruction error is calculated by the loss function, and the error back-propagation algorithm is used to optimize the convolution kernel parameters of the deconvolution and convolution operations.

[0009] Preferably, the process of performing pixel level classification includes: The pixel-level segmentation results output by the fully convolutional neural network are used as the input of the conditional random field model; Construct a conditional random field model, define pixel node and edge feature functions, and model the spatial relationship and category dependency between pixels; The spatial relationship feature function considers the relative position of pixels in the image, and the category-dependent feature function considers the consistency of category labels of adjacent pixels; In the conditional random field model, the optimized pixel labels are obtained by maximum a posteriori probability inference; The inference process considers both local features of pixels and global contextual information, smooths segmentation boundaries, and eliminates isolated misclassified pixels.

[0010] Preferably, the inference process further includes post-processing the inference results of the conditional random field model; The post-processing process includes: Connected domain analysis and morphological operations are used to remove small isolated areas and fill the holes in the segmentation results. Reconstruct the boundary contour of the segmentation result based on the spatial position of the pixel and the optimized category label; Through edge detection and contour tracing algorithms, the boundaries of the segmented objects are obtained, and the optimized segmentation results are evaluated to calculate the pixel-level classification accuracy and intersection-over-union ratio indicators; Compare the optimized segmentation results with the original output of the fully convolutional neural network to quantify the improvement of the conditional random field model; Iteratively optimize the parameters of the conditional random field model, and find the optimal feature function weights and inference algorithm parameters through cross-validation and grid search.

[0011] Preferably, a multi-scale feature fusion module is constructed to extract multi-scale feature maps at different levels of the full convolutional neural network. The process of integrating feature information of different scales through feature fusion operation includes: According to different levels of the fully convolutional neural network, feature maps of several scales are obtained, wherein the feature maps contain feature information of different scales; A splicing operation is performed on the feature maps of the plurality of scales to splice the feature maps of different scales in the channel dimension to obtain a spliced ​​multi-scale feature map; A weighted operation is performed on the spliced ​​multi-scale feature map, and weighted fusion is performed on the different scale features in the multi-scale feature map through the weight parameters obtained by learning, so as to obtain a fused multi-scale feature representation.

[0012] Preferably, an adversarial learning strategy is introduced, and by constructing a generator and a discriminator network, the segmentation result of the full convolutional neural network is subjected to adversarial training with the real annotation. The process of obtaining the final segmentation result includes: A fully convolutional neural network is constructed as the generator network. The multi-scale features of the medical image are extracted through the convolution layer and the pooling layer. The spatial resolution of the feature map is restored through the upsampling layer to obtain the pixel-level segmentation result. Construct a discriminator network, extract the segmentation results and true annotation features generated by the generator network through the convolution layer, and determine whether the segmentation results are true annotations through the fully connected layer to obtain the binary classification results; During the training process, the generator network and the discriminator network are trained alternately. The goal of the generator network is to generate realistic segmentation results to deceive the discriminator, and the goal of the discriminator network is to determine whether the segmentation results are real annotations. In the adversarial training process, a gradient penalty term is introduced to limit the gradient norm of the discriminator network, and data enhancement technology is used to expand the training samples. By rotating, translating and scaling medical images, diversified training samples are generated. Through adversarial training, the generator network and the discriminator network compete with each other and eventually reach a Nash equilibrium, making it difficult to distinguish the segmentation results generated by the generator network from the real annotations, and the discriminator network also difficult to distinguish the authenticity. During the testing phase, the trained generator network is used to perform segmentation prediction on unknown medical images, and post-processing techniques are used to optimize the segmentation results to obtain the final medical image segmentation results.

[0013] Compared with the prior art, the present invention has the following advantages and technical effects: The present invention discloses a medical image segmentation method based on deep learning. Aiming at the complex anatomical structures and pathological features in medical images, the present invention uses image enhancement technology to pre-process medical images, and introduces an attention mechanism and a spatial pyramid pooling module in a fully convolutional neural network to extract more fine-grained and discriminative features. The upsampling process is optimized by deconvolution operations and jump connection structures, a conditional random field model is introduced for post-processing optimization, a multi-scale feature fusion module is constructed, and an adversarial learning strategy is used to improve the generalization ability of the network. This method can effectively improve the accuracy and robustness of medical image segmentation, especially in the recognition of complex anatomical structures and lesion areas, and provides reliable technical support for subsequent auxiliary diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a method flow of an embodiment of the present invention; DETAILED DESCRIPTION

[0015] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0016] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0017] like Figure 1 As shown, this embodiment provides a medical image segmentation method based on a fully convolutional neural network, including: Preprocess medical images through deep learning algorithms to enhance key image features; In the process of feature extraction based on the fully convolutional neural network, the attention mechanism module is introduced to learn the importance weights of different areas in the image, adaptively adjust the weight distribution of the convolution kernel, and extract fine-grained and discriminative image features; Through the spatial pyramid pooling module embedded in the fully convolutional neural network, multi-scale feature fusion and context information capture of fine-grained and discriminative image features are performed; The fused feature map is obtained by multi-scale feature fusion, and the fused feature map is gradually restored to the same spatial size as the original image through upsampling operation; Based on the upsampled feature map, each pixel is classified through convolution operation to obtain the pixel-level segmentation result; An adversarial learning strategy is introduced. By constructing a generator and a discriminator network, the segmentation results of the fully convolutional neural network are adversarially trained with the real annotations to obtain the final segmentation results.

[0018] Furthermore, the process of preprocessing medical images through deep learning algorithms and enhancing key features of images includes: Acquire medical image data and use deep learning algorithms to analyze and identify complex anatomical structures and pathological features in the images; Preprocess medical images, adjust the grayscale distribution of images through histogram equalization algorithm, and enhance the contrast of images; According to the characteristics of the image, an adaptive contrast adjustment algorithm is used to specifically enhance the contrast of local areas of the image and highlight the pathological features.

[0019] In this embodiment, the key anatomical structures and pathological features of the medical images processed by the image enhancement technology are significantly enhanced.

[0020] More specifically, in order to obtain medical image data, this embodiment extracts medical images such as CT and MRI from the hospital's image storage system, or obtains them from public medical image data sets such as the LIDC-IDRI lung nodule data set. After obtaining the medical image, the image is first preprocessed, and the grayscale distribution of the image is adjusted using a histogram equalization algorithm. Histogram equalization enhances the contrast of the image by mapping the grayscale value of the image to a uniformly distributed interval. For example, for an 8-bit grayscale image with a grayscale value range of 0-255, histogram equalization converts the grayscale distribution of the original image into a uniform distribution, so that the number of pixels at each grayscale level is as equal as possible. On this basis, according to the characteristics of medical images, an adaptive contrast adjustment algorithm is used to dynamically adjust the intensity of contrast enhancement according to the grayscale statistical information of the local area of ​​the image. The adaptive contrast adjustment algorithm of this embodiment includes adaptive histogram equalization (AHE) and contrast-limited adaptive histogram equalization (CLAHE). The CLAHE algorithm avoids excessive enhancement of noise and artifacts by limiting the contrast magnification of the local histogram. After image enhancement processing, the key anatomical structures and pathological features in medical images become more obvious and prominent.

[0021] Furthermore, the process of introducing the attention mechanism module to extract fine-grained and discriminative image features includes: Construct a fully convolutional neural network model and introduce an attention mechanism module in the feature extraction stage. In the attention mechanism module, the importance weights of different regions in the medical image are calculated to generate an attention weight matrix. The attention weight matrix is ​​element-wise multiplied with the convolution kernel weight matrix to obtain the adjusted convolution kernel weight distribution. The medical image is convolved using the adjusted convolution kernel to extract the features of the key anatomical structures and lesion areas in the image, and obtain fine-grained and discriminative feature maps and corresponding image features.

[0022] More specifically, when constructing a fully convolutional neural network model, a medical image dataset is first obtained and normalized to scale the pixel values ​​to between 0 and 1. A Gaussian filter algorithm is then applied for denoising to reduce random noise in the image. Next, the preprocessed image is input into a fully convolutional neural network, which includes multiple convolutional layers and pooling layers to extract basic features of the image. In the feature extraction stage, an attention mechanism module is introduced, which calculates the importance weight of each region through the Softmax function, and these weights form an attention weight matrix. For example, for a 128x128 feature map, the attention mechanism outputs a weight matrix of the same size, in which each element represents the importance of the corresponding position of the original feature map. Then, this attention weight matrix is ​​element-wise multiplied with the weight matrix of the convolution kernel to obtain the adjusted convolution kernel weight distribution. In this way, when performing convolution operations, the network can focus more on key areas in the image, such as lesions or abnormal structures, and extract more discriminative features.

[0023] This embodiment introduces an attention mechanism in the feature extraction process to focus on the key areas in the image. Common attention mechanisms include channel attention and spatial attention. By learning the importance weights of different channels and spatial positions in the feature map, the feature representation is adaptively adjusted to highlight important areas and features. Finally, the extracted image features are input into a classifier or detector for disease diagnosis and lesion location. For image classification tasks, a softmax classifier can be used to map image features to different categories. For image localization tasks, a region proposal network (RPN) can be used to generate candidate regions, and a convolutional neural network can be used to classify and regress the candidate regions to obtain accurate target locations and sizes. Through the analysis and recognition of deep learning algorithms, high-precision medical image diagnosis and lesion detection can be achieved, providing auxiliary decision support for clinicians.

[0024] Furthermore, by embedding the spatial pyramid pooling module in the fully convolutional neural network, the process of multi-scale feature fusion and context information capture includes: Based on the spatial pyramid pooling module, multi-scale pooling operations are performed on fine-grained and discriminative image features to obtain feature representations at different scales; Based on the feature representation at different scales, multi-scale feature fusion is performed to capture the contextual information in the feature map and enhance the semantic representation ability of the feature. The fused features can restore the spatial detail information of the image as much as possible while retaining the high-level semantic information.

[0025] More specifically, in order to solve the problem of spatial information loss in the feature extraction process, a spatial pyramid pooling module is embedded after the third pooling layer of the fully convolutional neural network. This module performs multi-scale pooling operations on the input 64x64 feature map, using 2x2, 4x4 and 8x8 pooling windows respectively, to obtain feature representations at three different scales, with sizes of 32x32, 16x16 and 8x8 respectively. Then, the features of these three scales are spliced ​​to obtain a 56x56 fused feature map, while capturing the contextual information in the feature map to enhance the semantic representation ability of the feature. Next, through three upsampling operations, the fused feature map is gradually restored to the same 512x512 spatial size as the original image. During the upsampling process, the bilinear interpolation algorithm is used to enlarge the feature map, and the features are refined through convolution operations. Finally, on the upsampled 512x512 feature map, a 1x1 convolution kernel is used to classify each pixel to obtain a pixel-level segmentation result. By embedding the spatial pyramid pooling module and multi-scale feature fusion, this method can restore the spatial details of the image as much as possible while retaining high-level semantic information, thereby improving the accuracy of medical image segmentation.

[0026] Based on the feature map obtained by upsampling, each pixel is classified through convolution operation to obtain the pixel-level segmentation result. The process includes: Based on the feature map obtained by upsampling, a convolutional neural network is used to classify each pixel in the feature map. By setting the size and number of convolution kernels, the local features of the pixels are extracted and the category to which the pixels belong is determined. Based on the pixel classification results obtained by the convolution operation, the image is segmented at the pixel level. The feature vector output by the convolution is normalized using the softmax function to obtain the probability distribution of each pixel belonging to each category. Based on the pixel classification probability, the category to which each pixel belongs is determined to obtain the segmentation result of the image. The size of the feature map is increased by upsampling so that the segmentation result matches the resolution of the original image. The pixel categories in the segmentation result are mapped to corresponding colors or labels to obtain the final segmented image.

[0027] More specifically, in the fully convolutional neural network, the upsampled feature map is convolved by setting the convolution kernel size to 3x3 and the number to 64 to extract the local features of each pixel. This step is to ensure that the detailed information around each pixel in the image can be accurately captured, so as to more accurately classify the pixels. Next, the output of the convolution layer is normalized using the softmax function, and the obtained feature vector is converted into a probability distribution. In this process, each element in the output vector of each pixel represents the probability that the pixel belongs to a certain category. The category of each pixel is determined according to the probability, so as to achieve accurate segmentation of the image. Finally, the final image segmentation result is generated by converting each pixel category in the classification result into the corresponding color or label. In this process, if the resolution of the original image is 1920x1080, the upsampling operation needs to adjust the feature map size to this resolution to ensure that the segmentation result has the same size as the original image, so that the segmentation result can be directly compared with the original image or further analyzed and processed.

[0028] Furthermore, based on the feature map obtained by upsampling, each pixel is classified through a convolution operation to obtain a pixel-level segmentation result, which includes: Deconvolution is used to replace the traditional transposed convolution operation for upsampling, and the convolution kernel parameters in the upsampling process are learned; at the same time, a skip connection structure is introduced to fuse the high-resolution features of the shallow network with the high-level semantic features of the deep network to obtain the upsampling result; Then, pixel-level classification is performed. The spatial relationship and category dependency between pixels are modeled through the conditional random field model. The segmentation results output by the full convolutional neural network are post-processed and optimized to smooth the segmentation boundaries and eliminate isolated misclassified pixels.

[0029] A multi-scale feature fusion module is constructed to extract multi-scale feature maps at different levels of the full convolutional neural network. Through feature fusion operations, feature information of different scales is integrated to obtain a more comprehensive and robust feature representation.

[0030] Furthermore, the process of using the deconvolution operation to replace the traditional transposed convolution operation for upsampling includes: The input low-resolution image is used to extract deep semantic features using a pre-trained convolutional neural network; According to a preset upsampling multiple, the deep semantic features are upsampled by a deconvolution operation to obtain a first upsampled feature map; Obtain shallow high-resolution features corresponding to the input image, and fuse the shallow high-resolution features with the first up-sampled feature map through a skip connection to obtain a second up-sampled feature map; According to the second up-sampled feature map, further improving the resolution through a deconvolution operation to obtain a third up-sampled feature map; According to the third up-sampled feature map, a convolution operation is used to perform detail enhancement to obtain a fourth up-sampled feature map; According to the fourth up-sampled feature map, converting into a high-resolution output image through a pixel rearrangement operation; According to the high-resolution output image and the original high-resolution image, the reconstruction error is calculated by the loss function, and the error back-propagation algorithm is used to optimize the convolution kernel parameters of the deconvolution and convolution operations.

[0031] More specifically, given a low-resolution input image of size 256×256, a pre-trained VGG-19 convolutional neural network is used to extract its features, and a set of 512-dimensional deep semantic feature vectors is obtained. According to the preset upsampling multiple of 4, the deep semantic features are upsampled to a spatial size of 1024×1024 through deconvolution operation to obtain the first upsampled feature map. At the same time, the high-resolution features of the input image are extracted by the shallow convolution layer of the VGG-19 network, and they are fused element by element with the first upsampled feature map through skip connection to obtain the fused second upsampled feature map. Based on the second upsampled feature map, a deconvolution operation with a step size of 2 is performed to further increase the resolution of the feature map to 2048×2048 to obtain the third upsampled feature map. For the third upsampled feature map, three 3×3 convolutional layers are used to enhance its details, with the number of convolution kernels being 256, 128 and 3 respectively, and finally a fourth upsampled feature map of the same size as the target high-resolution image is obtained. Using the pixel rearrangement operation, each pixel in the fourth upsampled feature map is rearranged into a 2048×2048 high-resolution output image according to the preset mapping relationship. The reconstruction error between the high-resolution output image and the original high-resolution image is measured by the mean square error loss function, and the convolution kernel parameters in the deconvolution and convolution operations are iteratively optimized with a learning rate of 0001 using the Adam optimization algorithm to minimize the reconstruction error. After 50 epochs of training, the super-resolution reconstruction network can well learn the mapping relationship from low resolution to high resolution, effectively improving the visual quality of the image.

[0032] Furthermore, the process of performing pixel-level classification includes: The pixel-level segmentation results output by the fully convolutional neural network are used as the input of the conditional random field model; Construct a conditional random field model, define pixel node and edge feature functions, and model the spatial relationship and category dependency between pixels; The spatial relationship feature function considers the relative position of pixels in the image, and the category-dependent feature function considers the consistency of category labels of adjacent pixels; In the conditional random field model, the optimized pixel labels are obtained by maximum a posteriori probability inference; The inference process considers both local features of pixels and global contextual information, smooths segmentation boundaries, and eliminates isolated misclassified pixels.

[0033] Furthermore, the inference process also includes post-processing the inference results of the conditional random field model; the post-processing process includes: Connected domain analysis and morphological operations are used to remove small isolated areas and fill the holes in the segmentation results. Reconstruct the boundary contour of the segmentation result based on the spatial position of the pixel and the optimized category label; Through edge detection and contour tracing algorithms, the boundaries of the segmented objects are obtained, and the optimized segmentation results are evaluated to calculate the pixel-level classification accuracy and intersection-over-union ratio indicators; Compare the optimized segmentation results with the original output of the fully convolutional neural network to quantify the improvement of the conditional random field model; Iteratively optimize the parameters of the conditional random field model, and find the optimal feature function weights and inference algorithm parameters through cross-validation and grid search.

[0034] More specifically, the pixel-level segmentation results are obtained from the fully convolutional neural network as the input of the conditional random field model. When constructing the conditional random field model, the pixel node feature function is defined, considering the RGB color value and gradient amplitude of the pixel, and the edge feature function considers the Euclidean distance and color difference between pixels. The optimized pixel label is obtained by maximum a posteriori probability inference, and the mean field algorithm is used for approximate inference, and the number of iterations is set to 10. During the inference process, the local feature weight of the pixel is set to 6, and the global context information weight is set to 4 to balance local consistency and global information. When post-processing the inference results, the connected domain analysis is used, and the area threshold is set to 50 pixels. Isolated areas smaller than the threshold are removed; the morphological closing operation is used, and the structure element size is 3x3 to fill small holes. The boundary of the segmented object is extracted by the Canny edge detection algorithm, and the standard deviation of the Gaussian filter is 5, the low threshold is 50, and the high threshold is 150. When evaluating the segmentation results, pixel accuracy and average IoU are used as indicators, and the optimal parameter combination is selected through 5-fold cross validation. During the iterative optimization process, the feature function weights are determined by grid search, with a search range of 1 to 0 and a step size of 1; the number of iterations and convergence threshold of the inference algorithm are automatically adjusted through Bayesian optimization to improve the segmentation performance.

[0035] Furthermore, a multi-scale feature fusion module is constructed to extract multi-scale feature maps at different levels of the full convolutional neural network. The process of integrating feature information of different scales through feature fusion operation includes: According to different levels of the fully convolutional neural network, feature maps of several scales are obtained, where the feature maps contain feature information of different scales; A splicing operation is performed on feature maps of several scales, and feature maps of different scales are spliced ​​in the channel dimension to obtain a spliced ​​multi-scale feature map; A weighted operation is applied to the spliced ​​multi-scale feature map. The different scale features in the multi-scale feature map are weighted fused through the learned weight parameters to obtain the fused multi-scale feature representation.

[0036] According to the fused multi-scale feature representation, the feature extraction and classifier of the convolutional neural network are used to determine whether there is a lesion area in the image. If there is a lesion area, its location is determined. According to the determined location of the lesion area, the feature vector of the corresponding position is extracted from the fused multi-scale feature representation as the feature representation of the lesion area. The attention mechanism is used to adaptively adjust the weights of each scale feature in the multi-scale feature representation according to the feature representation of the lesion area, highlight the features of the lesion area, and suppress the features of the non-lesion area. The adjusted multi-scale feature representation is input into the subsequent network layer, and the final lesion area segmentation result is obtained through further feature transformation and classifier.

[0037] More specifically, in a fully convolutional neural network, feature information of multiple scales is obtained by extracting feature maps of different levels. For example, in a VGG-16 network, the outputs of the 3rd, 4th, and 5th convolutional blocks can be selected as multi-scale feature maps, whose scales are 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. These feature maps of different scales are spliced ​​in the channel dimension to obtain a feature map containing multi-scale information, and the number of channels is 1024. For the spliced ​​multi-scale feature maps, a weighted fusion method is used to adaptively adjust the importance of features of different scales through the weight parameters obtained by learning. For example, an attention mechanism can be used to dynamically generate a weight coefficient for each scale based on the feature representation of the lesion area, and the multi-scale feature maps are weighted summed to obtain a fused feature representation. The fused multi-scale feature representation contains semantic information and detail information at different scales, and can better characterize the characteristics of the lesion area. Based on the fused feature representation, the feature extraction and classifier of the convolutional neural network are used to determine whether there is a lesion area in the image. For example, two fully connected layers and a Softmax activation function can be used to classify feature vectors and output the probability of the lesion area. If a lesion area is detected, the feature vector of the corresponding position is extracted on the fused multi-scale feature representation according to the position coordinates output by the classifier as the feature representation of the lesion area for subsequent segmentation tasks. In order to further highlight the features of the lesion area and suppress the interference of the non-lesion area, an attention mechanism can be used to adaptively adjust the weights of the features of each scale in the multi-scale feature representation according to the feature representation of the lesion area. For example, through a fully connected layer and a Sigmoid activation function, the attention weight of each scale is generated, and the multi-scale feature map is weighted to obtain the adjusted feature representation. Finally, the adjusted multi-scale feature representation is input into the subsequent network layer, and after several convolutional layers and upsampling layers, a segmentation result of the same size as the input image is obtained. Through this method of multi-scale feature fusion and attention mechanism, semantic information and spatial details at different levels can be effectively combined to improve the segmentation accuracy of the lesion area.

[0038] Furthermore, an adversarial learning strategy is introduced. By constructing a generator and a discriminator network, the segmentation results of the full convolutional neural network are trained against the real annotations. The process of obtaining the final segmentation result includes: A fully convolutional neural network is constructed as the generator network. The multi-scale features of the medical image are extracted through the convolution layer and the pooling layer. The spatial resolution of the feature map is restored through the upsampling layer to obtain the pixel-level segmentation result. Construct a discriminator network, extract the segmentation results and true annotation features generated by the generator network through the convolution layer, and determine whether the segmentation results are true annotations through the fully connected layer to obtain the binary classification results; During the training process, the generator network and the discriminator network are trained alternately. The goal of the generator network is to generate realistic segmentation results to deceive the discriminator, and the goal of the discriminator network is to determine whether the segmentation results are real annotations. In the adversarial training process, a gradient penalty term is introduced to limit the gradient norm of the discriminator network, and data enhancement technology is used to expand the training samples. By rotating, translating and scaling medical images, diversified training samples are generated. Through adversarial training, the generator network and the discriminator network compete with each other and eventually reach a Nash equilibrium, making it difficult to distinguish the segmentation results generated by the generator network from the real annotations, and the discriminator network also difficult to distinguish the authenticity. During the testing phase, the trained generator network is used to perform segmentation prediction on unknown medical images, and post-processing techniques are used to optimize the segmentation results to obtain the final medical image segmentation results.

[0039] More specifically, in the generator network of the fully convolutional neural network, the features of medical images at different scales are extracted by setting convolution layers with convolution kernel sizes of 3x3, 5x5 and 7x7 respectively. After the convolution layer, the maximum pooling operation is used, with a pooling kernel size of 2x2 and a step size of 2 to achieve downsampling of the feature map size. In the upsampling layer, the transposed convolution operation is used, with a convolution kernel size of 4x4 and a step size of 2 to restore the spatial resolution of the feature map to the same as the input medical image. The discriminator network uses 5 convolution layers to extract features, with a convolution kernel size of 4x4 and a step size of 2. The LeakyReLU activation function is applied after each convolution layer. Finally, the binary classification results are output through the fully connected layer and the Sigmoid activation function. In each iteration, the discriminator network is trained first, and then the generator network is trained. The learning rates of the two are set to 0002 and 0001 respectively. At the same time, the gradient penalty term is introduced in the loss function of the discriminator network, and the penalty factor is set to 10 to limit the gradient norm of the discriminator network to no more than 1. When performing data augmentation on medical images, the random rotation angle ranges from -10° to 10°, the random translation distance ranges from -10% to 10% of the image width and height, and the random scaling factor ranges from 8 to 2. In the post-processing stage, small areas with an area less than 100 pixels are removed through connected domain analysis, and small holes in the segmentation results are filled using morphological closing operations, and the structural element is set to a 3x3 rectangle.

[0040] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A medical image segmentation method based on a fully convolutional neural network, characterized in that: include: Preprocess medical images through deep learning algorithms to enhance key image features; In the process of feature extraction based on the fully convolutional neural network, the attention mechanism module is introduced to learn the importance weights of different areas in the image, adaptively adjust the weight distribution of the convolution kernel, and extract fine-grained and discriminative image features; Multi-scale feature fusion and context information capture are performed through the spatial pyramid pooling module embedded in the fully convolutional neural network; The fused feature map is obtained by multi-scale feature fusion, and the fused feature map is gradually restored to the same spatial size as the original image through upsampling operation; Based on the upsampled feature map, each pixel is classified through convolution operation to obtain the pixel-level segmentation result; An adversarial learning strategy is introduced. By constructing a generator and a discriminator network, the segmentation results of the fully convolutional neural network are adversarially trained with the real annotations to obtain the final segmentation results.

2. The medical image segmentation method based on a fully convolutional neural network according to claim 1, characterized in that: The process of preprocessing medical images and enhancing key features of images through deep learning algorithms includes: Acquire medical image data and use deep learning algorithms to analyze and identify complex anatomical structures and pathological features in the images; Preprocess medical images, adjust the grayscale distribution of images through histogram equalization algorithm, and enhance the contrast of images; According to the characteristics of the image, an adaptive contrast adjustment algorithm is used to specifically enhance the contrast of local areas of the image and highlight the pathological features.

3. The medical image segmentation method based on a fully convolutional neural network according to claim 1, characterized in that: The process of introducing the attention mechanism module to extract fine-grained and discriminative image features includes: Construct a fully convolutional neural network model, introduce an attention mechanism module in the feature extraction stage, generate an attention weight matrix by calculating the importance weights of different regions in the medical image, perform element-wise multiplication of the attention weight matrix with the convolution kernel weight matrix, and obtain an adjusted convolution kernel weight distribution; The medical image is convolved using the adjusted convolution kernel to extract the features of the key anatomical structures and lesion areas in the image, and obtain fine-grained and discriminative feature maps and corresponding image features.

4. The medical image segmentation method based on a fully convolutional neural network according to claim 1, characterized in that: The process of multi-scale feature fusion and context information capture by embedding the spatial pyramid pooling module in the fully convolutional neural network includes: Based on the spatial pyramid pooling module, multi-scale pooling operations are performed on fine-grained and discriminative image features to obtain feature representations at different scales; Multi-scale feature fusion is performed based on the feature representations at different scales, and context information in the feature map is captured at the same time to enhance the semantic representation capability of the features.

5. The medical image segmentation method based on a fully convolutional neural network according to claim 1, characterized in that: The process of classifying each pixel through a convolution operation based on the feature map obtained by upsampling to obtain a pixel-level segmentation result includes: Deconvolution is used to replace the traditional transposed convolution operation for upsampling, and the convolution kernel parameters in the upsampling process are learned; at the same time, a skip connection structure is introduced to fuse the high-resolution features of the shallow network with the high-level semantic features of the deep network to obtain the upsampling result; Then, pixel-level classification is performed. The spatial relationship and category dependency between pixels are modeled through the conditional random field model. The segmentation results output by the full convolutional neural network are post-processed and optimized to smooth the segmentation boundaries and eliminate isolated misclassified pixels. A multi-scale feature fusion module is constructed to extract multi-scale feature maps at different levels of the full convolutional neural network. Through feature fusion operations, feature information of different scales is integrated to obtain a more comprehensive and robust feature representation.

6. The medical image segmentation method based on a fully convolutional neural network according to claim 5, characterized in that: The process of using deconvolution operation instead of traditional transposed convolution operation for upsampling includes: The input low-resolution image is used to extract deep semantic features using a pre-trained convolutional neural network; According to a preset upsampling multiple, upsampling the deep semantic features through a deconvolution operation to obtain a first upsampling feature map; Obtain shallow high-resolution features corresponding to the input image, and fuse the shallow high-resolution features with the first up-sampled feature map through a skip connection to obtain a second up-sampled feature map; According to the second up-sampled feature map, further improving the resolution through a deconvolution operation to obtain a third up-sampled feature map; According to the third up-sampled feature map, a convolution operation is used to perform detail enhancement to obtain a fourth up-sampled feature map; According to the fourth up-sampled feature map, converting into a high-resolution output image through a pixel rearrangement operation; According to the high-resolution output image and the original high-resolution image, the reconstruction error is calculated by the loss function, and the error back-propagation algorithm is used to optimize the convolution kernel parameters of the deconvolution and convolution operations.

7. The medical image segmentation method based on a fully convolutional neural network according to claim 5, characterized in that: The process of performing pixel level classification includes: The pixel-level segmentation results output by the fully convolutional neural network are used as the input of the conditional random field model; Construct a conditional random field model, define pixel node and edge feature functions, and model the spatial relationship and category dependency between pixels; The spatial relationship feature function considers the relative position of pixels in the image, and the category-dependent feature function considers the consistency of category labels of adjacent pixels; In the conditional random field model, the optimized pixel labels are obtained by maximum a posteriori probability inference; The inference process considers both local features of pixels and global contextual information, smooths segmentation boundaries, and eliminates isolated misclassified pixels.

8. The medical image segmentation method based on a fully convolutional neural network according to claim 7, characterized in that: The inference process also includes post-processing the inference results of the conditional random field model; The post-processing process includes: Connected domain analysis and morphological operations are used to remove small isolated areas and fill the holes in the segmentation results. Reconstruct the boundary contour of the segmentation result based on the spatial position of the pixel and the optimized category label; Through edge detection and contour tracing algorithms, the boundaries of the segmented objects are obtained, and the optimized segmentation results are evaluated to calculate the pixel-level classification accuracy and intersection-over-union ratio indicators; Compare the optimized segmentation results with the original output of the fully convolutional neural network to quantify the improvement of the conditional random field model; Iteratively optimize the parameters of the conditional random field model, and find the optimal feature function weights and inference algorithm parameters through cross-validation and grid search.

9. The medical image segmentation method based on a fully convolutional neural network according to claim 5, characterized in that: Construct a multi-scale feature fusion module to extract multi-scale feature maps at different levels of the full convolutional neural network. The process of integrating feature information of different scales through feature fusion operations includes: According to different levels of the fully convolutional neural network, feature maps of several scales are obtained, wherein the feature maps contain feature information of different scales; A splicing operation is performed on the feature maps of the plurality of scales to splice the feature maps of different scales in the channel dimension to obtain a spliced ​​multi-scale feature map; A weighted operation is performed on the spliced ​​multi-scale feature map, and weighted fusion is performed on the different scale features in the multi-scale feature map through the weight parameters obtained by learning, so as to obtain a fused multi-scale feature representation.

10. The medical image segmentation method based on a fully convolutional neural network according to claim 1, characterized in that: The adversarial learning strategy is introduced. By constructing the generator and discriminator networks, the segmentation results of the full convolutional neural network are trained against the real annotations. The process of obtaining the final segmentation result includes: A fully convolutional neural network is constructed as the generator network. The multi-scale features of the medical image are extracted through the convolution layer and the pooling layer. The spatial resolution of the feature map is restored through the upsampling layer to obtain the pixel-level segmentation result. Construct a discriminator network, extract the segmentation results and true annotation features generated by the generator network through the convolution layer, and determine whether the segmentation results are true annotations through the fully connected layer to obtain the binary classification results; During the training process, the generator network and the discriminator network are trained alternately. The goal of the generator network is to generate realistic segmentation results to deceive the discriminator, and the goal of the discriminator network is to determine whether the segmentation results are real annotations. In the adversarial training process, a gradient penalty term is introduced to limit the gradient norm of the discriminator network, and data enhancement technology is used to expand the training samples. By rotating, translating and scaling medical images, diversified training samples are generated. Through adversarial training, the generator network and the discriminator network compete with each other and eventually reach a Nash equilibrium, making it difficult to distinguish the segmentation results generated by the generator network from the real annotations, and the discriminator network also difficult to distinguish the authenticity. During the testing phase, the trained generator network is used to perform segmentation prediction on unknown medical images, and post-processing techniques are used to optimize the segmentation results to obtain the final medical image segmentation results.

Images