A medical image segmentation method, system and device based on deep learning

Abstract

The application belongs to the field of image processing, and particularly relates to a medical image segmentation method, system and device based on deep learning. The method comprises the following steps: S1: constructing an image segmentation network model comprising a feature extraction module, a U-net network and a feature fusion module. The feature extraction module adopts a ResNext101 network, and the U-net network model comprises an encoder unit and a decoder unit. The feature fusion module is used for fusing the features extracted by the feature extraction module and the encoder. S2: obtaining an original data set and classifying the original data set to obtain a training set and a test set. S3: setting a loss function used for training the network model. S4: training the network model by using the aforementioned training set and the loss function. S5: testing the network model by using the test set, and saving the network model with the best test performance for segmenting medical images. The application solves the problems of poor universality of the existing image segmentation method and inaccurate segmentation results of medical images with large scales.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to a medical image segmentation method, system, and apparatus based on deep learning. Background Technology

[0002] Medical images are an important analytical tool in clinical medical diagnosis and treatment. Medical image segmentation technology can quickly locate lesion areas and segment image textures, assisting medical staff in making accurate diagnoses. However, traditional medical image segmentation methods still have problems such as insufficient datasets, high image segmentation difficulty, and insufficient segmentation accuracy. Achieving accurate segmentation of medical images with low data volume has become an urgent problem to be solved.

[0003] Existing medical image segmentation techniques are mainly divided into traditional image segmentation methods and deep learning-based segmentation methods. Traditional image segmentation methods mainly include threshold-based segmentation methods, edge-based segmentation methods, region-based segmentation methods, and so on. These traditional methods can only be applied to specific datasets and specific segmentation requirements, and do not have universality for segmenting various medical image datasets. Deep learning-based methods mainly use segmentation networks based on the U-Net architecture. Through the encoder-decoder structure, the receptive field of pixels is expanded and restored to the original image size. Information from shallow and deep spatial layers of the image is fused to obtain the segmentation result of the target region. However, for medical images with large scale variations, U-Net-based medical image segmentation methods still produce inaccurate segmentation results. Summary of the Invention

[0004] To address the issues of poor universality and inaccurate segmentation results for large-scale medical images in existing image segmentation methods, this invention provides a medical image segmentation method, system, and apparatus based on deep learning.

[0005] This invention is achieved using the following technical solution:

[0006] A deep learning-based medical image segmentation method includes the following steps:

[0007] S1: Construct an image segmentation network model, which includes a feature extraction module, a U-net network, and a feature fusion module. The feature extraction module uses the ResNext101 network. The input to the feature extraction model is the medical image to be segmented, and the output is the first feature extracted by the ResNext101 network at different scales. The U-net network model includes an encoder unit and a decoder unit. The input to the encoder unit is the medical image to be segmented, and the output is the second feature at different scales. The feature fusion module performs convolution, pooling, and concatenation on the first and second features to obtain fused features. The fused features are jointly input to the decoder unit of the U-net network model through skip connections. The input to the decoder unit includes the fused features and the upsampled output of the previous layer in the decoder; the final output of the decoder unit is the binarized medical image segmentation result.

[0008] S2: Obtain the original dataset for training the image segmentation network model, classify the original dataset to obtain the training set and the test set.

[0009] S3: Set the loss function for joint optimization of the parameters of the image segmentation network model. The loss function includes the traditional cross-entropy loss function and a loss function based on the correlation of image pixels.

[0010] S4: Using the training set from step S2 and the loss function from step S3, train the constructed image segmentation network model to obtain a network model with fully optimized parameters, and save the model as the final segmentation network.

[0011] S5: Using the test set from step S2, test the segmentation network that has been trained and saved in step S4. Determine the segmentation performance of the image segmentation network model based on the test results. Save the network model with the best test performance and use this network model to segment the medical image to be segmented.

[0012] The medical image segmentation method provided in this invention constructs a novel image segmentation network model. This model uses the decoder-encoder structure of the U-net network as the base network and adds a ResNext101 network for feature extraction from the medical image to be segmented. The features extracted by the ResNext101 network and the features extracted by the decoder in the U-net network are fused and used as the joint input of the encoder. This invention provides an image segmentation network that, by fusing different features at multiple scales, considers the spatial relationships of image features, increases the receptive field of feature pixels, and combines the original information in the image, resulting in more refined segmentation results. It also overcomes the limitation of the conventional U-net network on the size of the input medical image.

[0013] As a further improvement of the present invention, in step S1, the first feature extracted by the ResNext101 network includes four feature maps of different scales, namely m1, m2, m3, and m4. The encoder unit of the U-Net network includes four convolutional pooling layer modules of different scales, thereby extracting four feature maps of the second feature of different scales, namely n1, n2, n3, and n4. The sizes of the four feature maps in the first and second features correspond sequentially.

[0014] As a further improvement of the present invention, the processing procedure of the feature fusion module in step S1 is as follows:

[0015] First, the second feature input to the encoding layer is subjected to 1×1, 3×3, and 5×5 convolution operations, respectively. Then, the results of these three convolution operations are concatenated, followed by 3×3 convolution and pooling. Next, the pooling result is concatenated with the corresponding first feature extracted by the ResNext101 network in the channel space. Finally, the concatenated result is subjected to a 3×3 convolution to obtain the fused feature.

[0016] As a further improvement of the present invention, the expression for the feature fusion processing is as follows:

[0017] p i =Conv3{Cat[Conv1(n i ),Conv3(n i ),Conv5(n i )]}

[0018] out i =Conv3{Cat[pool(p i ),m i ]}

[0019] Here, Conv1, Conv3, and Conv5 represent convolution operations with kernel sizes of 1×1, 3×3, and 5×5, respectively. i The values ​​i = 1, 2, 3, 4 represent the second features extracted by the encoder units of the U-Net network at different scales. Cat represents the concatenation of feature maps in the channel space; pool represents the pooling operation.

[0020] As a further improvement of this invention, the output of the image segmentation network model includes two parts: one part is the segmentation result of the output medical image, and the other part is the feature map after the medical image and the first layer of the encoder unit are combined. The latter is the feature map result used to calculate the image pixel correlation loss function, i.e., the embedding layer feature mentioned below.

[0021] As a further improvement of the present invention, in step S3, the cross-entropy loss function is used to optimize the global network model parameters. The goal of the loss is to make the segmentation result close to the given label result, and then the entire network parameters are optimized through backpropagation. The expression of the cross-entropy loss function is as follows:

[0022]

[0023] In the above formula, y i p(y) represents the output of medical image segmentation. i ) represents the probability that the segmentation prediction is correct in the corresponding segmentation result.

[0024] As a further improvement of the present invention, in step S4, the loss function based on the image pixel correlation is used to calculate the correlation between each pixel and its surrounding neighborhood. Pixels in the same class should generate similar feature spaces; for any two pixel indices i and j, the corresponding feature is output as e. i and e j The corresponding object labels are I i and I j The optimization goal is to reduce the differences between features with the same label, while increasing the differences between features with different labels. The expression for the loss function of image pixel correlation is as follows:

[0025]

[0026] In the above formula, ||·|| represents the vector norm, and α and β represent the threshold values ​​for feature proximity, with threshold values ​​of 0.5 and 2 respectively.

[0027] To evaluate the overall embedding layer features, it is necessary to calculate the relationship between each feature and its surrounding features, and then sum all the losses using the following formula:

[0028]

[0029] In the above formula, j∈N i Let i represent the spatial neighborhood of the iterative index. In the experiment, the correlation degree of each pixel is calculated using the eight-neighbor features. This loss is minimized by stochastic gradient descent, thereby optimizing the segmentation result of the network model.

[0030] During the training of the model on the network, this invention uses a joint loss function. One part is the cross-entropy loss function, which can highlight the distance difference between feature pixels and label pixels to optimize the overall network structure parameters. The other part is the novel pixel correlation loss, which can consider the correlation between pixel neighborhoods in the output segmentation result. The purpose is to increase the inter-class difference of pixels while reducing the intra-class difference of pixels, thus sharpening the segmentation effect of the experiment.

[0031] This invention also includes a deep learning-based medical image segmentation system. This system uses the aforementioned deep learning-based medical image segmentation method to segment an input medical image to be segmented, thereby obtaining the segmentation result of the target region. The medical image segmentation system includes: an image acquisition module, a feature extraction module, a U-net network model, a feature fusion module, and an optimization processing module.

[0032] The image acquisition module includes an optimization channel and an application channel. The optimization channel is used to acquire a dataset of medical images for training or testing, and then uses the dataset to train and test other modules in the medical image segmentation system, optimizing the parameters of each network model in the medical image segmentation system. The application channel is used to acquire the medical image to be segmented, and then uses the optimized network model to segment the medical image.

[0033] The feature extraction module employs the ResNext101 network as its core feature extraction network. This module performs multi-level feature extraction on the input medical image, resulting in first feature maps at different scales.

[0034] The U-net network model comprises encoder units and decoder units. The encoder unit includes multiple convolutional pooling layers at different scales. The input to the encoder unit is the medical image to be segmented, and the output is multiple second feature maps at different scales. The first and second feature maps are fused according to their corresponding scales and used as part of the input to the decoder unit. The decoder unit's input includes the encoder unit's output and the feature fusion result. The decoder unit's output consists of two parts: the segmentation result of the medical image, and the feature map resulting from the joint processing of the medical image and the first layer of the encoder unit.

[0035] The feature fusion module first performs 1×1, 3×3, and 5×5 convolution operations on the second feature map. Then, it concatenates the results of these three convolution operations, performing 3×3 convolution and pooling sequentially. Next, it concatenates the pooling result with the corresponding scale of the second feature map in the channel space, and finally performs a 3×3 convolution on the concatenated result to obtain the desired fused feature. This fused feature is then input into the decoder unit of the U-net network model as joint input via skip connections.

[0036] The optimization module uses cross-entropy loss function and loss function based on image pixel correlation as loss functions for the medical image segmentation system. It uses the dataset obtained in the optimization channel to train and test the medical image segmentation system, while optimizing the overall network structure parameters and the final segmentation effect to obtain the optimized network model after training.

[0037] As a further improvement of this invention, the first feature map extracted by the feature extraction module includes four scales, and the second feature map extracted by the encoder unit of the U-net network model also includes four scales. Furthermore, the scale levels in the first and second feature maps correspond one-to-one.

[0038] The present invention also includes a deep learning-based medical image segmentation apparatus, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the deep learning-based medical image segmentation method described above.

[0039] The technical solution provided by this invention has the following beneficial effects:

[0040] This invention provides a deep learning-based medical image segmentation method that constructs an improved U-net network combining the ResNext101 network. This network model overcomes the limitations of traditional U-shaped networks on the size of the input image and solves the problem of inaccurate segmentation results when the image scale varies significantly in the field of medical image segmentation. Furthermore, the constructed network model introduces a feature fusion module to fuse the features extracted by the encoders of the ResNext101 and U-net networks. This allows the network model to consider the spatial feature relationships in the image during processing, increase the receptive field of feature pixels, and combine the original information in the image, resulting in more refined segmentation results.

[0041] This invention also employs a joint loss function, including both cross-entropy and image pixel correlation loss functions, to optimize and train the constructed network model. During training, the former optimizes the overall network structure parameters by highlighting the distance difference between feature pixels and label pixels. The latter considers the correlation between pixel neighborhoods in the segmentation results, increasing inter-class differences while reducing intra-class differences, thereby sharpening the image segmentation performance of the network model.

[0042] After testing on datasets from different sources, it was found that the image segmentation network model constructed in the method provided by this invention also has good generalization performance, showing good segmentation results for various types of medical images. It is suitable for application in clinical diagnosis, providing assistance to doctors in the diagnostic process. Attached Figure Description

[0043] Figure 1 This is a flowchart of the steps of a deep learning-based medical image segmentation method provided in Embodiment 1 of the present invention.

[0044] Figure 2 This is a simplified diagram of the framework of the image segmentation network model constructed in Embodiment 1 of the present invention.

[0045] Figure 3 This is a schematic diagram of the feature fusion module in Embodiment 1 of the present invention.

[0046] Figure 4 This is a schematic diagram of the modules of the deep learning-based medical image segmentation system provided in Embodiment 2 of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Example 1

[0049] This embodiment provides a deep learning-based medical image segmentation method, such as... Figure 1 As shown, this medical image segmentation method includes the following steps:

[0050] S1: Construct an image segmentation network model, which includes a feature extraction module, a U-net network, and a feature fusion module. The feature extraction module uses the ResNext101 network. The input of the feature extraction model is the medical image to be segmented, and the output is the first feature at different scales extracted by the ResNext101 network. For example... Figure 2As shown, the U-net network model includes encoder units and decoder units. The encoder unit takes the medical image to be segmented as input and outputs second features at different scales. The feature fusion module performs convolution, pooling, and concatenation on the first and second features to obtain fused features. The fused features are jointly input to the decoder unit of the U-net network model through skip connections. The decoder unit's input includes the fused features and the upsampled output of the previous layer in the decoder; the output of each layer of the decoder unit is the result of concatenation, convolution, and upsampling of each feature. The final output of the entire decoder module is a binarized image.

[0051] The ResNext101 network extracts four different scale feature maps for the first feature: m1, m2, m3, and m4. The U-Net network's encoder unit includes four convolutional pooling layer modules of different scales, which in turn extract four different scale feature maps for the second feature: n1, n2, n3, and n4. The sizes of the four feature maps in the first and second features correspond sequentially.

[0052] like Figure 3 As shown, the processing procedure of the feature fusion module is as follows:

[0053] First, the second feature input to the encoding layer is subjected to 1×1, 3×3, and 5×5 convolution operations, respectively. Then, the results of these three convolution operations are concatenated, followed by 3×3 convolution and pooling. Next, the pooling result is concatenated with the corresponding first feature extracted by the ResNext101 network in the channel space. Finally, the concatenated result is subjected to a 3×3 convolution to obtain the fused feature.

[0054] The expression for the feature fusion process is as follows:

[0055] p i =Conv3{Cat[Conv1(n i ),Conv3(n i ),Conv5(n i )]}

[0056] out i =Conv3{Cat[pool(p i ),m i ]}

[0057] Here, Conv1, Conv3, and Conv5 represent convolution operations with kernel sizes of 1×1, 3×3, and 5×5, respectively. iLet i = 1, 2, 3, 4, representing the second features extracted by the encoder units of the U-Net network at different scales. Let Cat represent the concatenation of feature maps in the channel space; pool represents the pooling operation.

[0058] The output of the image segmentation network model consists of two parts: the segmentation result of the output medical image, and the feature map resulting from the joint processing of the medical image and the first layer of the encoder unit. The latter is the feature map result used to calculate the image pixel correlation loss function, i.e., the embedding layer features mentioned below. The specific transfer relationship of the feature data is as follows: Figure 2 As shown.

[0059] S2: Obtain the original dataset for training the image segmentation network model, classify the original dataset to obtain the training set and the test set.

[0060] S3: Set the loss function for joint optimization of the parameters of the image segmentation network model. The loss function includes the traditional cross-entropy loss function and a loss function based on the correlation of image pixels.

[0061] S4: Using the training set from step S2 and the loss function from step S3, train the constructed image segmentation network model to obtain a network model with fully optimized parameters, and save the model as the final segmentation network.

[0062] S5: Using the test set from step S2, test the segmentation network that has been trained and saved in step S4. Determine the segmentation performance of the image segmentation network model based on the test results. Save the network model with the best test performance and use this network model to segment the medical image to be segmented.

[0063] The following section details the processing steps of the image segmentation network model in this embodiment, providing a clearer explanation of the characteristics of this network model.

[0064] First, the input medical image to be segmented is fed into the ResNext101 network for feature extraction. The feature maps extracted by the ResNext101 network include four different layers, namely m1, m2, m3 and m4 in order from shallow to deep. These four layers of feature maps are respectively the outputs of the four different layers of the U-net decoder in the image segmentation network model in this embodiment.

[0065] Secondly, the input medical image to be segmented is processed by four convolutional pooling layers of different scales in the encoder of the U-net network, resulting in four feature maps of different scales: n1, n2, n3, and n4. The output of the encoder in the U-net network and the output of the preceding ResNext101 network are both input into the feature fusion module for feature fusion processing.

[0066] In the feature fusion module, such as Figure 3 As shown, the feature fusion module first performs 1×1, 3×3, and 5×5 convolution operations on the feature map input from the encoder to expand the receptive field. Then, it concatenates the three convolutional results before performing a 3×3 convolution and pooling operation. The feature after pooling is P. i The output feature P i Features at different scales extracted by the ResNext101 network i Feature concatenation is performed in the channel space. Finally, a 3×3 convolution process is used to restore the original input channel size. The final fused feature output by the feature fusion module is out. i In this embodiment, a residual network is used during feature fusion to preserve the detailed information of the features, while also facilitating backpropagation and optimization of network parameters.

[0067] In this embodiment, as Figure 2 As shown, the fused feature output by the feature fusion module is out. i By using skip connections, the network connects to the corresponding decoder in the U-net network for processing, thus preserving detailed information in the features.

[0068] Next, this embodiment also employs two different loss functions jointly to optimize the parameters of the image segmentation network model. Specifically, for the segmentation result res of the medical image, the traditional cross-entropy loss is used to calculate the pixel-by-pixel difference, as shown in the following formula:

[0069]

[0070] Among them, y i p(y) represents the output of medical image segmentation. i ) represents the probability that the segmentation prediction is correct in the corresponding segmentation result.

[0071] The goal of this loss function is to make the segmentation result close to the given label result, and then optimize the entire network parameters through backpropagation.

[0072] Furthermore, this embodiment proposes a loss function based on image pixel correlation to calculate the correlation between each pixel and its surrounding neighborhood, ensuring that pixels in the same class generate similar feature spaces. For any two pixel indices i and j, the corresponding feature output is e. i and e j The corresponding object labels are I i and I jThe goal of this loss function optimization is to reduce the differences between features with the same label, while increasing the differences between features with different labels. The invention uses α and β to represent the threshold values ​​for feature proximity, and the optimized loss function is shown below:

[0073]

[0074] Where ||·|| represents the vector norm, and the threshold values ​​for α and β are 0.5 and 2, respectively.

[0075] After optimization using this loss function, the image segmentation network model can shorten the distance between features when they are similar, and widen the distance between features when they are different, thereby achieving a sharpening effect and making the segmentation results more refined.

[0076] To evaluate the overall embedding layer features, it is necessary to calculate the relationship between each feature and its surrounding features, and then sum all the losses, as shown in the following formula:

[0077]

[0078] In the above formula, j∈N i Let i represent the spatial neighborhood of the iterative index. In the experiment, the correlation degree of each pixel is calculated using the eight-neighbor features. This loss is minimized by stochastic gradient descent, thereby optimizing the segmentation result of the model.

[0079] After the image segmentation network model is constructed, it still needs to be trained and tested to optimize the network model parameters. In the overall training process of the model in this embodiment, the input medical image first passes through a pre-trained ResNext101 network to extract feature maps of different scales, which serve as part of the input to the feature fusion module. In this embodiment, using a pre-trained model can greatly shorten the training time and improve the efficiency of the experiment. Another part is the training of the U-Net main network model. The medical image is input into the U-Net network, moving from a shallow feature space to a deep feature space, giving the features context awareness. The feature scale is restored by upsampling through the decoder, and finally, the segmentation result is obtained. In this embodiment, the cross-entropy loss function is used to optimize the global network model parameters, and the loss function of image pixel correlation is used to optimize some network model parameters. The combination of the two makes the segmentation result of the image segmentation network model more refined and accurate.

[0080] The testing process involves using the pre-trained network model to test medical images in the test set. The overall process is consistent with the above. Once the segmentation results are obtained, it is no longer necessary to optimize the network parameters using the proposed loss function, thus completing the output of the test results. The final model performance test results of this embodiment show that the medical image segmentation network model provided in this embodiment can achieve excellent segmentation results for various different medical image datasets, and the image segmentation accuracy is better than that of the traditional U-shaped network.

[0081] Example 2

[0082] This embodiment provides a deep learning-based medical image segmentation system. This system uses the deep learning-based medical image segmentation method described in Embodiment 1 to segment an input medical image to be segmented, thereby obtaining the segmentation result of the target region. Figure 4 As shown, the medical image segmentation system includes: an image acquisition module, a feature extraction module, a U-net network model, a feature fusion module, and an optimization processing module.

[0083] The image acquisition module includes an optimization channel and an application channel. The optimization channel is used to acquire a dataset of medical images for training or testing, and then uses the dataset to train and test other modules in the medical image segmentation system, optimizing the parameters of each network model in the medical image segmentation system. The application channel is used to acquire the medical image to be segmented, and then uses the optimized network model to segment the medical image.

[0084] The feature extraction module employs the ResNext101 network as its core feature extraction network. This module performs multi-level feature extraction on the input medical image, resulting in first feature maps at different scales.

[0085] The U-net network model comprises encoder units and decoder units. The encoder unit includes multiple convolutional pooling layers at different scales. The input to the encoder unit is the medical image to be segmented, and the output is multiple second feature maps at different scales. The first and second feature maps are fused according to their corresponding scales and used as part of the input to the decoder unit. The decoder unit's input includes the encoder unit's output and the feature fusion result. The decoder unit's output consists of two parts: the segmentation result of the medical image, and the feature map resulting from the joint processing of the medical image and the first layer of the encoder unit.

[0086] The feature fusion module first performs 1×1, 3×3, and 5×5 convolution operations on the second feature map. Then, it concatenates the results of these three convolution operations, performing 3×3 convolution and pooling sequentially. Next, it concatenates the pooling result with the corresponding scale of the second feature map in the channel space, and finally performs a 3×3 convolution on the concatenated result to obtain the desired fused feature. This fused feature is then input into the decoder unit of the U-net network model as joint input via skip connections.

[0087] The optimization module uses cross-entropy loss function and loss function based on image pixel correlation as loss functions for the medical image segmentation system. It uses the dataset obtained in the optimization channel to train and test the medical image segmentation system, while optimizing the overall network structure parameters and the final segmentation effect to obtain the optimized network model after training.

[0088] The first feature map extracted by the feature extraction module has four scales, and the second feature map extracted by the encoder unit of the U-net network model also has four scales. Furthermore, the scale levels in the first and second feature maps correspond one-to-one.

[0089] Example 3

[0090] This embodiment provides a deep learning-based medical image segmentation device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the deep learning-based medical image segmentation method as described in Embodiment 1.

[0091] The computer device can be a smartphone, tablet, laptop, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc., capable of executing programs. The computer device in this embodiment includes, but is not limited to, a memory and a processor that can communicate with each other via a system bus.

[0092] In this embodiment, the memory (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory can also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0093] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of a computer device. In this embodiment, the processor is used to run program code stored in memory or process data to implement the processing procedure of the deep learning-based medical image segmentation method described in the foregoing embodiments, thereby obtaining the segmentation result of the target region in the given medical image.

[0094] The medical image segmentation device based on deep learning provided by this city should at least include, or be connected to, a display module for displaying the input medical image to be segmented and the segmentation results of the image segmentation network model.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical image segmentation method based on deep learning, characterized in that, The medical image segmentation method includes the following steps: S1: Construct an image segmentation network model, which includes a feature extraction module, a U-net network, and a feature fusion module; the feature extraction module adopts the ResNext101 network, the input of the feature extraction model is the medical image to be segmented, and the output is the first feature at different scales extracted by the ResNext101 network; The U-net network model includes an encoder unit and a decoder unit. The encoder unit takes the medical image to be segmented as input and outputs second features at different scales. The feature fusion module performs convolution, pooling, and concatenation on the first and second features to obtain fused features. The fused features are jointly input to the decoder unit of the U-net network model via skip connections. The decoder unit's input includes the fused features and the upsampled output of the previous layer in the decoder. The final output of the decoder unit is a binarized medical image segmentation result. The processing procedure of the feature fusion module is as follows: First, the second feature input to the encoder unit is subjected to 1×1, 3×3, and 5×5 convolution operations respectively; then, the processing results of the three convolution operations are concatenated, and 3×3 convolution and pooling are performed sequentially; next, the pooling result and the corresponding first feature extracted by the ResNext101 network are concatenated in the channel space, and finally, the concatenated result is subjected to 3×3 convolution to obtain the fused feature; S2: Obtain the original dataset for training the image segmentation network model, classify the original dataset to obtain the training set and the test set; S3: Set the loss function for joint optimization of the parameters of the image segmentation network model. The loss function includes the traditional cross-entropy loss function and a loss function based on the correlation of image pixels. The expression for the loss function of the image pixel correlation is as follows: In the above formula, ||·|| represents the vector norm, and α and β represent the threshold values ​​for feature proximity, with threshold values ​​of 0.5 and 2 respectively; e i and e j Let I represent the output features of any two pixel indices i and j, respectively. i and I j These represent object labels for any two pixel indices i and j, respectively. To evaluate the overall embedding layer features, it is necessary to calculate the relationship between each feature and its surrounding features, and then sum all the losses using the following formula: In the above formula, N i The spatial neighborhood of the iteration index i is represented. In the experiment, the correlation degree of the features is calculated using the eight-neighbor features of each pixel. This loss is minimized by stochastic gradient descent, thereby optimizing the segmentation result of the network model. S4: Using the training set from step S2 and the loss function from step S3, train the constructed image segmentation network model to obtain a network model with fully optimized parameters, and save the model as the final segmentation network. S5: Using the test set in step S2, test the segmentation network that has been trained and saved in step S4, and determine the segmentation performance of the image segmentation network model based on the test results; save the network model with the best test performance, and use the network model to segment the medical image to be segmented.

2. The deep learning-based medical image segmentation method as described in claim 1, characterized in that: In step S1, the first feature extracted by the ResNext101 network includes four feature maps of different scales, namely m1, m2, m3 and m4; the encoder unit of the U-Net network includes four convolutional pooling layer modules of different scales, and the feature maps of the second feature extracted at four different scales are n1, n2, n3 and n4 respectively; the sizes of the four feature maps in the first feature and the second feature correspond to each other in sequence.

3. The deep learning-based medical image segmentation method as described in claim 1, characterized in that: Feature fusion The expression for the processing procedure is as follows: p i =Conv3{Cat[Conv1(n i ),Conv3(n i ),Conv5(n i )]} out i =Conv3{Cat[pool(p i ),m i ]} Where Conv1, Conv3, and Conv5 represent convolution operations with kernel sizes of 1×1, 3×3, and 5×5, respectively; n i The values ​​represent the second features extracted by the encoder units of the U-Net network at different scales, i = 1, 2, 3, 4; Cat represents the concatenation of feature maps in the channel space; pool represents the pooling operation.

4. The deep learning-based medical image segmentation method as described in claim 1, characterized in that: The output of the image segmentation network model includes two parts: one is the segmentation result of the output medical image, and the other is the feature map after the medical image is combined with the first layer of the encoder unit. This feature map is used to calculate the loss function of the image pixel correlation.

5. The deep learning-based medical image segmentation method as described in claim 4, characterized in that: In step S3, the cross-entropy loss function is used to optimize the global network model parameters. The goal of the loss is to make the segmentation result close to the given label result, and then the entire network parameters are optimized through backpropagation. The expression of the cross-entropy loss function is as follows: In the above formula, y i p(y) represents the output of medical image segmentation. i ) represents the probability that the segmentation prediction is correct in the corresponding segmentation result.

6. The deep learning-based medical image segmentation method as described in claim 1, characterized in that: In step S4, a loss function based on image pixel correlation is used to calculate the correlation between each pixel and its surrounding neighborhood, so that pixels in the same class generate similar feature spaces; for any two pixel indices i and j, the corresponding feature is output as e. i and e j The corresponding object labels are I i and I j The goal of optimization is to reduce the differences between features with the same label, while increasing the differences between features with different labels.

7. A medical image segmentation system based on deep learning, characterized in that, It is used to segment an input medical image to be segmented using the deep learning-based medical image segmentation method as described in any one of claims 1-6, thereby obtaining the segmentation result of the target region; the medical image segmentation system includes: The image acquisition module includes an optimization channel and an application channel. The optimization channel is used to acquire a dataset of medical images for training or testing, and then uses the dataset to train and test other modules in the medical image segmentation system, optimizing the parameters of each network model in the medical image segmentation system. The application channel is used to acquire the medical image to be segmented, and then uses the optimized network model to segment the medical image. The feature extraction module employs the core network of the ResNext101 network for feature extraction. This module is used to perform multi-level feature extraction on the input medical image, thereby obtaining first feature maps at different scales. The U-net network model includes an encoder unit and a decoder unit. The encoder unit includes multiple convolutional pooling layers of different scales. The input of the encoder unit is the medical image to be segmented, and the output is multiple second feature maps of different scales. The first and second feature maps are fused according to their corresponding scales and used as part of the input of the decoder unit. The input of the decoder unit includes the output of the encoder unit and the result of feature fusion. The output of the decoder unit includes two parts: the segmentation result of the medical image and the feature map after the medical image and the first layer of the encoder unit are combined. The feature fusion module performs 1×1, 3×3, and 5×5 convolution operations on the second feature map, respectively. Then, it concatenates the results of the three convolution operations, performing 3×3 convolution and pooling sequentially. Next, it concatenates the pooling result with the corresponding scale of the second feature map in the channel space, and finally performs a 3×3 convolution on the concatenated result to obtain the desired fused feature. This fused feature is input to the decoder unit of the U-net network model as joint input via skip connections. The optimization processing module uses cross-entropy loss function and loss function based on image pixel correlation as loss functions for the medical image segmentation system. It uses the dataset obtained in the optimization channel to train and test the medical image segmentation system, while optimizing the overall network structure parameters and the final segmentation effect, to obtain the optimized network model after training.

8. The deep learning-based medical image segmentation system as described in claim 7, characterized in that: The first feature map extracted by the feature extraction module has four scales, and the second feature map extracted by the encoder unit of the U-net network model also has four scales; and the scale levels in the first feature map and the second feature map correspond one-to-one.

9. A medical image segmentation device based on deep learning, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the deep learning-based medical image segmentation method as described in any one of claims 1 to 6.

Images