Image processing method, and training method and device of image segmentation network

Abstract

The application provides an image processing method and a training method and device of an image segmentation network. The method comprises: performing Inception convolution on an MRI image by using an Inception convolution module of the image segmentation network to obtain first image features; performing dilated convolution on the first image features by using a dilated convolution module of the image segmentation network to obtain second image features; and segmenting a tumor lesion area from the MRI image based on the second image features, wherein the tumor lesion area is a region in which tumor lesion tissue is imaged. In this way, more contour information and detail information of the MRI image can be obtained, the segmentation accuracy of the image is improved, the depth of the network is increased, the adaptability of the network to scales and the nonlinear capability of the network are enhanced, and the segmentation accuracy of the MRI image is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical technology, and in particular to an image processing method, an image segmentation network training method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] Brain tumors refer to various tumors located in the brain, and their growth rate and location determine their impact on the nervous system. According to surveys, the number of brain tumor patients is increasing year by year, and the situation is extremely serious. Advances in computer science and imaging technology have driven the development of the intelligent healthcare industry. Magnetic Resonance Imaging (MRI) images and deep learning technology are effective means to achieve automatic segmentation of brain tumors; automatically and accurately segmenting tumor regions from MRI images is one of the key steps in establishing computer-aided diagnostic systems for automatic brain tumor segmentation in clinical practice.

[0003] In related technologies, brain tumor MRI image segmentation methods include: semi-automatic image segmentation methods and fully automatic image segmentation methods.

[0004] For the semi-automatic image segmentation method, a small amount of radiologist involvement is required during the segmentation of MRI images. Pixel-level fine annotation is not necessary. Typically, the radiologist manually selects initial seed points or marks the Region of Interest (ROI). Then, an image thresholding algorithm based on grayscale histograms, combined with a Gaussian equation, calculates the grayscale range of the tumor within the ROI, thereby obtaining the tumor region through thresholding. Alternatively, the ROI boundary can be used as the initial contour of the target region, employing the Snake algorithm, where external and internal energy terms jointly control contour changes, continuously approximating the true tumor contour. However, this method, requiring manual selection of initial seed points or ROIs, is tedious for radiologists and heavily reliant on clinical experience. When faced with the automated diagnosis of large-scale MRI images, this method becomes extremely time-consuming and highly susceptible to human error.

[0005] The fully automated image segmentation method described above requires no manual intervention. It directly transforms and extracts image features through deeply stacked nonlinear modules, outputting high-dimensional feature representations and automatically segmenting tumor regions. However, this fully automated method is trained through fully supervised learning, requiring a large amount of labeled training data to effectively avoid model overfitting and ensure the robustness of the neural network model. Obtaining a large number of well-labeled medical images is difficult, and the data labeling process is also challenging, often requiring physicians with over ten years of experience. A small number of labels affects the accuracy of the neural network modules, while large-scale image data labeling incurs significant human and financial costs.

[0006] However, due to the large range of size variations in tumor lesions, the two methods mentioned above cannot simultaneously accommodate the segmentation of tumor lesions of different sizes, which can easily lead to inaccurate segmentation. Summary of the Invention

[0007] This invention provides an image processing method, an image segmentation network training method, an apparatus, an electronic device, and a storage medium.

[0008] The technical solution of this invention is implemented as follows:

[0009] In a first aspect, embodiments of the present invention provide an image processing method, comprising:

[0010] For MRI images, Inception convolution is performed using the Inception convolution module of the image segmentation network to obtain the first image features;

[0011] The first image features are dilated by using the dilated convolution module of the image segmentation network to obtain the second image features;

[0012] Based on the second image features, the tumor lesion region is segmented from the MRI image, wherein the tumor lesion region is the area where the pixels of the tumor lesion tissue are located.

[0013] Secondly, embodiments of the present invention provide a training method for an image segmentation network, comprising:

[0014] Obtain a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples;

[0015] The MRI images of the training sample set are input into a first network that has been pre-trained using labeled training data to obtain the predicted segmentation information output by the first network; wherein, the predicted segmentation information includes: the predicted segmentation information of the labeled training samples;

[0016] A first segmentation quality score is obtained for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the second network using the predicted segmentation information of the first network.

[0017] The first loss value of the first network is determined based on the difference between the predicted segmentation information of the labeled training samples and the labels.

[0018] Based on the first segmentation quality score and the labels of the labeled training samples, the second loss value of the second network is determined;

[0019] Based on the first loss value and the second loss value, determine whether to continue training the first network.

[0020] Thirdly, embodiments of the present invention provide an image processing apparatus, the apparatus comprising:

[0021] The first convolutional module is used to perform Inception convolution on MRI images using the Inception convolutional module of the image segmentation network to obtain the first image features.

[0022] The second convolution module is used to perform dilated convolution on the first image features using the dilated convolution module of the image segmentation network to obtain the second image features;

[0023] The segmentation module is used to segment the tumor lesion region from the MRI image based on the second image features, wherein the tumor lesion region is the region where the pixels of the tumor lesion tissue are located.

[0024] Fourthly, embodiments of the present invention provide a training apparatus for an image segmentation network, the training apparatus comprising:

[0025] An acquisition module is used to acquire a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples;

[0026] The first network module is used to input the MRI images of the training sample set into a first network that has been pre-trained using labeled training data, and to obtain the predicted segmentation information output by the first network; wherein, the predicted segmentation information includes: the predicted segmentation information of the labeled training samples;

[0027] The second network module is used to obtain a first segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the labeled training samples through the second network.

[0028] The loss determination module is used to determine a first loss value for the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels; to determine a second loss value for the second network based on the first segmentation quality score and the labels of the labeled training samples; and to determine whether to continue training the first network based on the first loss value and the second loss value.

[0029] Fifthly, embodiments of the present invention provide an electronic device, comprising:

[0030] Memory, used to store executable instructions;

[0031] The processor, when executing executable instructions stored in the memory, implements the image processing method provided by one or more of the foregoing technical solutions.

[0032] Sixthly, embodiments of the present invention provide an electronic device, comprising:

[0033] Memory, used to store executable instructions;

[0034] The processor, when executing executable instructions stored in the memory, implements the training method of the image segmentation network provided by one or more of the foregoing technical solutions.

[0035] In a seventh aspect, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions; the computer-executable instructions, when executed by a processor, are capable of implementing the image processing method provided by one or more of the aforementioned technical solutions.

[0036] Eighthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions; when executed by a processor, the computer-executable instructions can implement the training method of the image segmentation network provided by one or more of the foregoing technical solutions.

[0037] This invention provides an image processing method, an image segmentation network training method, an apparatus, an electronic device, and a storage medium. By inputting an MRI image into a trained image segmentation network, the Inception convolution module in the image segmentation network performs Inception convolution on the MRI image to obtain a first image feature; then, a dilated convolution module performs dilated convolution on the first image feature to obtain a second image feature; based on the second image feature, the tumor lesion region is segmented from the MRI image.

[0038] By performing Inception convolution and dilated convolution on MRI images using the Inception convolution module and the dilated convolution module, on the one hand, it is possible to extract multi-scale image features from the MRI images by utilizing convolution kernels of different sizes in the Inception convolution module, thereby obtaining more contour and detail information from the MRI images and improving the segmentation accuracy of the images; on the other hand, it can increase the depth of the network, enhance the network's adaptability to scale and the network's nonlinearity.

[0039] On the other hand, the dilated convolution module can be used to increase the receptive field while maintaining the same image resolution, thus adapting to tumor lesions of different sizes in MRI images. Therefore, by directly performing Inception convolution and dilated convolution on MRI images, the segmentation accuracy of MRI images can be effectively improved. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present invention;

[0041] Figure 2 This is a flowchart illustrating a training method for an image segmentation network provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of a training device for an image segmentation network provided in an embodiment of the present invention;

[0044] Figure 5 A flowchart illustrating an image processing method provided for this example;

[0045] Figure 6 This is a schematic diagram of the network architecture of an image segmentation network provided in this example;

[0046] Figure 7 This example provides a schematic diagram of a network architecture for network discrimination.

[0047] Figure 8 This is a schematic diagram of a generative adversarial network architecture provided in this example. Detailed Implementation

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] This invention provides an image processing method. Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, it includes the following steps:

[0050] Step 101: For the MRI image, perform Inception convolution using the Inception convolution module of the image segmentation network to obtain the first image features;

[0051] Step 102: Perform dilated convolution on the first image features using the dilated convolution module of the image segmentation network to obtain the second image features;

[0052] Step 103: Based on the second image features, segment the tumor lesion region from the MRI image, wherein the tumor lesion region is the region where the pixels of the tumor lesion tissue are located.

[0053] The image processing method involved in the embodiments of the present invention can be applied to electronic devices; here, the electronic devices include devices such as terminals or servers, the terminal can be a mobile phone, tablet computer, laptop computer, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers.

[0054] In step 101, the first image features can be obtained by inputting the MRI image into a pre-trained image segmentation network and performing Inception convolution by the Inception convolution module of the image segmentation network.

[0055] It should be noted that the Inception convolution module can perform multiple convolution operations or pooling operations in parallel and concatenate all the output results into a single feature map, thereby improving the utilization of network resources and increasing the width and depth of the network without changing the amount of computation.

[0056] In some embodiments, step 101 may include:

[0057] Based on the sliding operation of a preset-sized sliding window on the MRI image, multiple images to be segmented are determined;

[0058] The Inception convolution module of the image segmentation network is used to perform Inception convolution on the multiple images to be segmented to obtain the first image features;

[0059] It should be noted that since the size of the MRI images obtained using medical imaging equipment may not be the same as the size that the image segmentation network adapts to during training, multiple images of a preset size can be extracted from the MRI images by sliding a preset-size sliding window on the MRI images, thereby making the size of the images to be segmented adapt to the network model.

[0060] In this embodiment of the invention, there may be overlapping parts between adjacent images to be segmented, and all images to be segmented may include information from the complete MRI image. In this way, not only can the accuracy of the prediction results obtained after the MRI image is input into the image segmentation network be guaranteed, but also the completeness of the final result can be guaranteed.

[0061] In step 102, the first image features are dilated by the dilated convolution module to obtain the second image features after expanding the receptive field.

[0062] Dilated convolution modules can extract multi-scale features, functioning as both convolutional and pooling layers. While ensuring the output image size is the same as the input image size, they expand the receptive field of the convolution kernel, extracting deeper information while preserving shallower details. In other words, by performing dilated convolution on image features, the utilization rate of pixels within the receptive field is increased, ensuring all pixels participate in the computation and fully utilizing the input image features, resulting in more accurate predictions.

[0063] Here, in the convolutional operation of a neural network, the receptive field is the size of the region on the input feature map that a pixel in the output feature map of each layer of the neural network maps to. In other words, the receptive field is the size of the element in the output of a certain layer of the neural network mapped to the input image. Generally, the receptive field size of the input features of the first convolutional layer is equal to the size of the convolutional kernel, while the receptive field size of higher convolutional layers is related to the kernel size and stride of all previous layers. Therefore, based on different receptive fields, different levels of information can be captured, thereby achieving the purpose of extracting feature information at different scales.

[0064] It's important to note that the conventional method for expanding the receptive field is to use max pooling layers to pool image features, but this reduces image resolution. Dilated convolution, on the other hand, builds upon conventional convolution by inserting zero-valued holes around each parameter in the convolution kernel, thus increasing the receptive field without sacrificing feature map resolution.

[0065] In step 103, the second image feature may be a multi-scale image feature used to indicate the tumor lesion region in the MRI image.

[0066] In this embodiment of the invention, the second image features at different scales output by the dilated convolution module can be fused to obtain multi-channel image features, and channel transformation processing can be performed on the multi-channel image features to obtain predictive segmentation information.

[0067] For example, second image features at different scales can be stitched together to obtain multi-channel image features; a 1×1 convolutional layer is used to transform the channels so that the predicted segmentation information output by the 1×1 convolutional layer is consistent with the number of channels of the input image of the image segmentation network.

[0068] The predicted segmentation information can also be upsampled to obtain a segmentation probability map with the same size as the input MRI image; and the MRI image can be segmented according to the segmentation probability map to obtain the tumor lesion area in the MRI image.

[0069] Optionally, the Inception convolution module has N Inception convolution layers, and the convolution kernels of different Inception convolution layers are different; different Inception convolution layers are used to extract the first image features at different scales.

[0070] Here, N is a positive integer greater than or equal to 1;

[0071] In this embodiment of the invention, the convolution kernels of different Inception convolution layers in the Inception convolution module are different. By using convolution kernels of different scales to perform Inception convolution on MRI images, image features of multiple scales of MRI images can be extracted simultaneously.

[0072] It should be noted that the Inception convolution module divides the input into multiple branches by applying different convolution kernels, and uses Inception convolution layers with different kernels to extract image features of different scales from the multiple branches respectively.

[0073] Optionally, the dilated convolution module includes: N dilated convolution dense sub-modules;

[0074] The Inception convolutional layer and the dilated convolutional dense submodule are interspersed in the image segmentation network. The dilated convolutional dense submodule is used to perform dilated convolution processing on the first image features at different scales.

[0075] Here, N is a positive integer greater than or equal to 1; it is understood that the number of dilated convolution dense sub-modules in the dilated convolution module is the same as the number of Inception convolution layers in the Inception convolution module.

[0076] In this embodiment of the invention, the Inception convolutional layer and the dilated convolutional dense submodule are interspersed in the image segmentation network. The Inception convolutional layer performs Inception convolution on the input MRI image to obtain first image features at different scales. The dilated convolutional dense submodule performs dilated convolution on the first image features at different scales output by the Inception convolutional layer to obtain second image features at multiple scales.

[0077] It should be noted that the dense submodule consists of multiple convolutional layers. Each dense submodule uses the same number of output channels, but during forward propagation, the output of each dense submodule is merged with its output in the channel dimension to form the next dense submodule.

[0078] Optionally, the dilated convolution module of the image segmentation network is used to perform dilated convolution on the first image features to obtain the second image features, including:

[0079] Perform dilated convolution on the received first image features to obtain the dilated convolution result;

[0080] The dilated convolution result is subjected to batch normalization (BN) to obtain the normalized result;

[0081] The normalization result is rectified using a leakage rectifier function to obtain the second image feature.

[0082] In this embodiment of the invention, the step of batch normalizing the dilated convolution results to obtain normalized results includes:

[0083] Obtain the mean and standard deviation of the dilated convolution results;

[0084] The dilated convolution result is normalized based on the mean and standard deviation.

[0085] Based on preset learning parameters, the normalized dilated convolution result is reconstructed to obtain the normalized result.

[0086] Here, the preset learning parameters can be set according to actual needs;

[0087] In this embodiment of the invention, the dilated convolution results are first normalized using the mean and standard deviation, so that the output results of each dense dilated convolution submodule have the same data distribution, thereby improving the generalization ability of the network and accelerating the convergence speed. Then, the normalized linear dilated convolution results are reconstructed using preset learning parameters, so that the normalized results can better match the true distribution of the dilated convolution results and ensure the nonlinear expressive ability of the model.

[0088] For example, the normalization process of the dilated convolution result based on the mean and variance may include:

[0089] Subtract the mean from the dilated convolution result to obtain the first dilated convolution result;

[0090] Divide the first dilated convolution result by the variance to obtain the dilated convolution result after batch normalization.

[0091] It should be noted that the Leaky Rectified Linear Unit (Leaky-ReLU) function, as an activation function, can retain some negative axis values ​​while correcting the data distribution, so that the information of the negative axis is not completely lost. This not only avoids the gradient vanishing problem, but also reduces the occurrence of silent neurons.

[0092] Optionally, the image processing method further includes:

[0093] The MRI images were subjected to standard convolution processing to obtain the standard convolution result;

[0094] For the MRI images, the Inception convolution module of the image segmentation network is used to perform Inception convolution to obtain multiple first image features at different scales, including:

[0095] The Inception convolution module is used to perform Inception convolution on the standard convolution result based on different convolution kernels to obtain multiple first image features at different scales.

[0096] In this embodiment of the invention, the MRI image is subjected to standard convolution processing to obtain a standard convolution result for indicating low-dimensional features of the MRI image; the standard convolution result is input into the Inception convolution module, and the standard convolution result is subjected to Inception convolution processing using different convolution kernels in the Inception convolution module to obtain first image features at different scales.

[0097] Here, the low-dimensional features are used to indicate the global contour information of the MRI image, such as edges and lines; the first image feature can be a high-dimensional feature used to indicate the detailed information of the MRI image.

[0098] The following embodiments of the present invention provide a training method for an image segmentation network, such as... Figure 2 As shown, Figure 2 This is a flowchart illustrating a training method for an image segmentation network provided in an embodiment of the present invention. The method includes:

[0099] Step 201: Obtain a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples.

[0100] Step 202: Input the MRI images of the training sample set into the first network that has been pre-trained using labeled training data to obtain the predicted segmentation information output by the first network; wherein, the predicted segmentation information includes: the predicted segmentation information of the labeled training samples;

[0101] Step 203: Obtain a first segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the labeled training samples.

[0102] Step 204: Determine the first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels;

[0103] Step 205: Based on the first segmentation quality score and the labels of the labeled training samples, determine the second loss value of the second network;

[0104] Step 206: Based on the first loss value and the second loss value, determine whether to continue training the first network.

[0105] In step 201, a large number of MRI images are acquired, and the MRI images are divided into labeled training samples and unlabeled training samples based on whether they carry labels. The labels are used to indicate tumor lesion areas in the MRI images of the training samples.

[0106] It should be noted that the training sample set consists of MRI images used to adjust the training parameters in the first and second networks; in this embodiment of the invention, the training sample set includes labeled training samples and unlabeled training samples.

[0107] In related technologies, image segmentation networks are trained using fully supervised learning, requiring a large number of labeled MRI images for training to effectively prevent overfitting and ensure robustness. However, in the medical field, labeling MRI images is difficult, resulting in a limited number of labeled images that can negatively impact the accuracy of the image segmentation network. To address this, this invention pre-trains the image segmentation network using labeled MRI images. Then, through a discriminant network, it trains the pre-trained network using both unlabeled and labeled MRI images in a semi-supervised learning manner. This fully utilizes a large number of unlabeled MRI images, improving the accuracy of the image segmentation network.

[0108] In step 202, the predicted segmentation information is used to indicate the tumor lesion area in the MRI image.

[0109] The first network is a network model trained in a fully supervised manner using labeled training data; MRI images from the training sample set are input into the first network to obtain the predicted segmentation information output by the first network;

[0110] In step 203, the MRI image and the corresponding predicted segmentation information are input into the second network to obtain the first segmentation quality score corresponding to the predicted segmentation information.

[0111] In this embodiment of the invention, the first segmentation quality score is used to indicate the accuracy of the predicted segmentation result of the labeled training samples output by the first network. For example, if the first segmentation quality score is 1, it indicates that the segmentation quality of the predicted segmentation information output by the first network is good; if the first segmentation quality score is 0, it indicates that the segmentation quality of the predicted segmentation information output by the first network is poor.

[0112] During the training process of the image segmentation network, the second network can be pre-trained using the predicted segmentation information and labels of labeled training samples.

[0113] The labeled training sample MRI images, the corresponding labels of the MRI images, and the predicted segmentation information can be input into the second network to obtain a first segmentation quality score of the predicted segmentation information corresponding to the labeled MRI images output by the second network; the first segmentation quality score and the predicted segmentation information are used to adjust the training parameters in the second network.

[0114] In step 204, the first loss value of the first network can be determined by using the difference between the predicted segmentation information of the labeled training samples and the labels, as well as the first segmentation quality score of the labeled training samples.

[0115] In this embodiment of the invention, the first network and the second network are the generator subnetwork and the discriminator subnetwork in the generative adversarial network, respectively. Through adversarial learning between the first network and the second network, the output of the first network is used to guide the second network to output a more accurate segmentation quality score, and the segmentation quality score output by the second network is used to prompt the first network to improve the accuracy of image segmentation.

[0116] Therefore, when determining the first loss function value of the first network, it is necessary to consider not only the difference between the predicted segmentation information of the labeled training samples and the labels, but also the first segmentation quality score of the labeled training samples.

[0117] In step 205, for labeled training samples, the MRI image of the labeled training sample, the predicted segmentation information of the MRI image, and the label are input into the second network, and the second loss value of the second network is determined based on the first segmentation quality score for the predicted segmentation information and the segmentation quality score for the label output by the second network.

[0118] In this embodiment of the invention, during the training process of the image segmentation network, the second loss value of the second network can be determined first using the predicted segmentation information and labels of the labeled training samples; the second network can be pre-trained using the second loss value; and the pre-trained second network can be used to re-evaluate the segmentation quality of the labeled and unlabeled training samples, thereby training the first network based on the first segmentation quality score of the labeled training samples and the second segmentation quality score of the unlabeled training samples.

[0119] In step 206, it is determined whether the first loss value and the second loss value meet the stop training condition. If the first loss value or the second loss value does not meet the stop training condition, the training parameters in the first network and the second network are optimized until both the first loss value and the second loss value meet the stop training condition, and the image segmentation network is obtained.

[0120] Here, the stopping training condition can be that the training reaches the required number of iterations and / or the first loss function and the second loss function converge. The number of iterations can be determined based on the number of MRI images in the training sample set and the initial network size; this invention does not specifically limit this number.

[0121] Optionally, the training method for the image segmentation network further includes:

[0122] A second segmentation quality score is obtained for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the second network based on the unlabeled training samples.

[0123] Determining the first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels includes:

[0124] The first loss value is obtained based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score and the second segmentation quality score.

[0125] In this embodiment of the invention, the second segmentation quality score is used to indicate the accuracy of the predicted segmentation result of the unlabeled training samples output by the first network.

[0126] The MRI images from the unlabeled training samples are input into a first network to obtain the predicted segmentation information of the unlabeled training samples output by the first network; the predicted segmentation information and the MRI images are then input into a second network to obtain a second segmentation quality score output by the second network.

[0127] It should be noted that, in practical applications, the number of labeled MRI images is limited, and training the first network using a fully supervised approach requires a large amount of labeled training data to effectively improve the accuracy of the first network.

[0128] To address this, this embodiment of the invention employs a semi-supervised approach. Unlabeled and labeled training samples are input into a first network to obtain predicted segmentation information for both unlabeled and labeled training samples. A second network is then used to obtain a first segmentation quality score for the labeled predicted segmentation information and a second segmentation quality score for the unlabeled predicted segmentation information. The difference between the predicted segmentation information and the labels in the labeled training data, along with the first and second segmentation quality scores, is used to determine the loss function value of the first network, thereby adjusting the training parameters in the first network.

[0129] Optionally, obtaining the first loss value based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score, and the second segmentation quality score includes:

[0130] Based on the first segmentation quality score, the loss value of the second network is determined;

[0131] Based on the loss value of the second network, adjust the training parameters in the second network;

[0132] Using the adjusted second network, the predicted segmentation information of labeled training samples and the predicted segmentation information of unlabeled training samples are evaluated to obtain a first segmentation quality score for the predicted segmentation information of labeled training samples and a second segmentation quality score for the predicted segmentation information of unlabeled training samples.

[0133] The first loss value of the first network is determined based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score, and the second segmentation quality score.

[0134] In this embodiment of the invention, the first network and the second network are trained adversarially. The output of the first network can be used to guide the second network to output a more accurate segmentation quality score; the segmentation quality score output by the second network can be used to prompt the first network to improve the accuracy of image segmentation.

[0135] The loss value of the second network is determined by using the first segmentation quality score and the difference between the predicted segmentation information and the label of the labeled training samples; and the second network is pre-trained based on the loss value of the second network; the labeled and unlabeled predicted segmentation information output by the first network are respectively input into the pre-trained second network, and the loss value of the first network is determined by using the first and second segmentation quality scores output by the second network, the predicted segmentation information and the label; thus, the first network can be trained using labeled and unlabeled training samples to obtain an image segmentation network.

[0136] Optionally, determining the first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels, a first segmentation quality score, and a second segmentation quality score includes:

[0137] Based on the predicted segmentation information of the labeled training samples and the labels, determine the segmentation loss function value corresponding to the labels;

[0138] Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score;

[0139] Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score;

[0140] The first loss value of the first network is obtained by weighted fusion of the segmentation loss function value, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score.

[0141] In this embodiment of the invention, the first loss function of the first network consists of a segmentation loss function and an adversarial loss function; the segmentation loss function value corresponding to the label is used to indicate the difference between the labeled training sample and the label. The adversarial loss function may include: the binary cross-entropy corresponding to the first segmentation quality score and the binary cross-entropy corresponding to the second segmentation quality score;

[0142] Wherein, the binary cross-entropy corresponding to the first segmentation quality score is used to determine the loss value of the first segmentation quality score corresponding to the labeled training sample; the binary cross-entropy corresponding to the second segmentation quality score is used to determine the loss value of the second segmentation quality score corresponding to the unlabeled training sample.

[0143] It should be noted that the segmentation loss function improves the basic segmentation performance of the first network by utilizing labeled training samples; the binary cross-entropy corresponding to the first segmentation quality score and the binary cross-entropy corresponding to the second segmentation quality score characterize the adversarial relationship between the predicted segmentation information output by the first network and the real label; thereby making the labeled training samples and unlabeled training samples closer to the labels.

[0144] The weights corresponding to the segmentation loss function value, the binary cross-entropy of the first segmentation quality score, and the binary cross-entropy of the second segmentation quality score can be set according to actual needs.

[0145] In some embodiments, the first loss function of the first network can be expressed by the following equation:

[0146]

[0147] Wherein, the l G (·) represents the first loss function of the first network; the I l The I is a labeled MRI image; ul The image is an unlabeled MRI image; the y l ' represents the predicted segmentation information corresponding to a labeled MRI image; the y ul ' represents the predicted segmentation information corresponding to the unlabeled MRI image; the θ G The weight parameters to be trained in the first network; the l seg (·) represents the segmentation loss function of the first network; the The weights of the binary cross-entropy of the first segmentation quality score; The weights of the binary cross-entropy of the second segmentation quality score; the l DCE (·) represents the Dice cross-entropy loss function; the D(I) l ,y l D(I) is the first segmentation quality score output by the second network; the D(I) is the first segmentation quality score output by the second network. ul ,y ul ′) represents the second segmentation quality score output by the second network.

[0148] Optionally, determining the second loss value of the second network based on the first segmentation quality score and the labels of the labeled training samples includes:

[0149] Based on the first segmentation quality score, the labels of the labeled training samples, and the second segmentation quality score, a second loss value for the second network is determined.

[0150] In this embodiment of the invention, the first network outputs predicted segmentation information for labeled MRI images and predicted segmentation information for unlabeled MRI images; the labeled MRI images, along with their corresponding labels and predicted segmentation information, are input into a second network to obtain a first segmentation quality score corresponding to the labeled predicted segmentation information and a segmentation quality score corresponding to the labels, output by the second network; the unlabeled MRI images, along with their corresponding predicted segmentation information, are input into the second network to obtain a second segmentation quality score corresponding to the unlabeled predicted segmentation information, output by the second network.

[0151] Based on the segmentation quality score corresponding to the label, the first segmentation quality score, and the second segmentation quality score, the differences between the labeled predicted segmentation result, the unlabeled predicted segmentation result, and the MRI image, as well as the differences between the label and the MRI image, are determined, thereby determining the second loss value of the second network.

[0152] Optionally, determining the second loss value of the second network based on the first segmentation quality score, the labels of the labeled training samples, and the second segmentation quality score includes:

[0153] Based on the segmentation quality score corresponding to the label, determine the binary cross-entropy corresponding to the label;

[0154] Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score;

[0155] Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score;

[0156] The binary cross-entropy corresponding to the label, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score are weighted and fused to obtain the second loss value of the second network.

[0157] In this embodiment of the invention, a second network is used to determine the segmentation quality score and a first segmentation quality score corresponding to the label based on the labeled MRI image and the label. The binary cross-entropy of the segmentation quality score corresponding to the label, the binary cross-entropy of the first segmentation quality score, and the binary cross-entropy of the second segmentation quality score are then used.

[0158] In some embodiments, the second loss function of the second network can be expressed by the following equation:

[0159]

[0160] Wherein, the l D (·) represents the second loss function of the second network; the θ D These are the weight parameters to be trained in the second network; The weights of the binary cross-entropy of the first segmentation quality score; The weight of the binary cross-entropy of the second segmentation quality score.

[0161] Optionally, the training method for the image segmentation network further includes:

[0162] Labeled training data is input into the initial network to obtain the predicted segmentation information output by the initial network;

[0163] The segmentation loss function value of the initial network is determined based on the difference between the predicted segmentation information and the label;

[0164] Based on the segmentation loss function value of the initial network, the training parameters of the initial network are adjusted to obtain the first network.

[0165] In this embodiment of the invention, a first network is obtained by training an initial network using labeled training data in a fully supervised manner.

[0166] The training parameters of the initial network may include one or more of the following parameters: number of iterations, batch size, image size, learning rate, and learning rate decay value.

[0167] Here, the number of iterations refers to the number of training iterations across all training images in the training sample set, generally determined by the number of training images and the initial network size; for example, the number of iterations can be set to 1500. The batch size represents the number of images loaded into the network during one forward propagation; for example, the batch size can be set to 16. The image size refers to the size that the image needs to be adjusted to before training, changing the image size according to a predefined length and width; for example, the image size can be set to 224×224. The learning rate represents the rate at which the model learns. If the learning rate is set too small, the trained network model will converge too slowly; if the learning rate is set too large, the loss function will oscillate. Therefore, to avoid setting an unreasonable learning rate during training, a dynamic adjustment of the learning rate is usually adopted. By setting a learning rate decay value, the learning rate decreases exponentially with the number of training rounds, converging the learning step size of gradient descent.

[0168] By inputting training data of labeled MRI images into an initial network, predictive segmentation information for indicating tumor lesion regions in the MRI images is obtained from the output of the initial network. Based on the difference between the predicted segmentation information and the labels, the segmentation loss function value of the initial network is calculated. The backpropagation gradient is calculated based on the segmentation loss function value, and the training parameters in the initial network are updated. The above steps are repeated until the number of iterations is reached or the loss function converges, thus obtaining the first network.

[0169] Optionally, determining the segmentation loss function value of the initial network based on the difference between the predicted segmentation information and the labeled information includes:

[0170] Based on the predicted segmentation information and the labeled information, the binary cross-entropy and Dice cross-entropy of the predicted segmentation information and the labeled information are determined respectively.

[0171] The binary cross-entropy and the Dice cross-entropy are weighted and fused to obtain the segmentation loss function value of the initial network.

[0172] In this embodiment of the invention, the segmentation loss function of the initial network consists of a Dice cross-entropy loss function and a binary cross-entropy loss function. The Dice cross-entropy loss function is used to determine the similarity between two tumor lesion regions in the MRI image. The binary cross-entropy loss function is used to determine the loss value of each pixel in the MRI image, which has a good effect on the segmentation details of MRI images with unclear boundaries and uneven gray levels.

[0173] By weighted fusion of the binary cross-entropy and Dice cross-entropy of the predicted segmentation information and the labeled information, it is possible to take into account the different scales and shapes of the tumor lesion region in the MRI image, as well as the influence of the imaging quality of the MRI image itself (e.g., gray-level unevenness, noise, etc.) on the boundary segmentation of the tumor lesion region.

[0174] Here, the weights of the binary cross-entropy and the Dice cross-entropy can be set according to actual needs. For example, the weights of the binary cross-entropy and the Dice cross-entropy can both be set to 0.5.

[0175] In some embodiments, the segmentation loss function of the initial network can be expressed by the following equation:

[0176] l seg (y′,y;θ G )=αl DCE (y′,y;θ G )+βl BCE (y′,y;θ G );

[0177] Wherein, the l seg (·) represents the segmentation loss function of the initial network; y represents the label of the MRI image; y′ represents the predicted segmentation information of the MRI image; θ G The initial network's training weights are defined as follows: α is the weight of the Dice cross-entropy; l DCE (·) represents the Dice cross-entropy loss function; β represents the weight of the binary cross-entropy; l BCE (·) represents the binary cross-entropy loss function.

[0178] Optionally, obtaining a segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the labeled training samples through the second network includes:

[0179] The predicted segmentation information and the labeled training samples are input into the second network. The image intensity attention module and geometric boundary attention module of the second network are used to process the predicted segmentation information to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image of the labeled training samples, the contour features of the tumor lesion area and the contour features of the background area.

[0180] Based on the intensity signal difference features, the contour features of the tumor lesion area, and the contour features of the background area, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0181] In this embodiment of the invention, the image intensity attention module is used to extract intensity information difference features between the tumor lesion area and the background area in the MRI image based on the predicted segmentation information and the training samples; the geometric boundary attention module is used to extract the contour features of the tumor lesion area and the contour features of the background area in the MRI image.

[0182] It should be noted that by using the image intensity attention module and the geometric boundary attention module, the information contained in the tumor lesion area and the background area in the MRI image is fully utilized to increase the contrast between the tumor lesion area and the background area. This makes it easier for the second network to use the intensity signal difference features, the contour features of the tumor lesion area and the contour features of the background area to more effectively identify the tumor lesion area, thereby determining the accuracy of the predicted segmentation information.

[0183] Based on the MRI image, image features of the MRI image are extracted; based on the predicted segmentation information of the MRI image, intensity signal difference features, contour features of the tumor lesion area, and contour features of the background area are extracted; using the intensity signal difference features and contour features of the MRI image and the predicted segmentation information, the difference between the MRI image and the predicted segmentation information is determined, thereby determining the segmentation quality score corresponding to the predicted segmentation information.

[0184] Optionally, the image intensity attention module and geometric boundary attention module of the second network process the predicted segmentation information to obtain the intensity information difference features between the tumor lesion region and the background region of the MRI image of the labeled training sample, the contour features of the tumor lesion region, and the contour features of the background region, including:

[0185] The predicted segmentation information is subjected to standard convolution processing to obtain the tumor lesion region features and the background region features of the MRI image, respectively.

[0186] The features of the tumor lesion area and the background area are classified to obtain the attention information of the tumor lesion area and the attention information of the background area in the MRI image.

[0187] The MRI image, the attention information corresponding to the tumor lesion area, and the attention information of the background area are fused to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image, the contour features of the tumor lesion area, and the contour features of the background area.

[0188] In this embodiment of the invention, the tumor lesion region features and the background region features in the MRI image are extracted by performing parallel standard convolution processing on the predicted segmentation information; and the tumor lesion region features and the background region features are processed by activation functions to obtain the attention information corresponding to the tumor lesion region and the attention information corresponding to the background region.

[0189] The fusion of the MRI image, the attention information corresponding to the tumor lesion region, and the attention information of the background region may include:

[0190] The MRI images are convolved to obtain their image features.

[0191] The first attention feature is obtained by multiplying the image features of the MRI image with the attention information corresponding to the tumor lesion area;

[0192] The second attention feature is obtained by multiplying the image features of the MRI image with the attention information corresponding to the background region.

[0193] In this embodiment of the disclosure, the first attention feature is an image feature that incorporates attention information corresponding to the tumor lesion area; the second attention feature is an image feature that incorporates attention information corresponding to the background area.

[0194] The first attention feature can be used to indicate the contour features of the tumor lesion area, and the second attention feature can be used to indicate the contour features of the background area; and the difference between the first attention feature and the second attention feature can be used to determine the intensity information difference features between the tumor lesion area and the background area.

[0195] Optionally, determining the segmentation quality score corresponding to the predicted segmentation information based on the intensity signal difference features, the contour features of the tumor lesion region, and the contour features of the background region includes:

[0196] The intensity signal difference features, the contour features of the tumor lesion region, and the contour features of the background region are cascaded to obtain dual attention features;

[0197] Based on the dual attention feature, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0198] In this embodiment of the invention, feature fusion can be performed by the first attention feature and the second attention feature to achieve cascaded processing of the intensity signal difference feature, the contour feature of the tumor lesion area and the contour feature of the background area, thereby obtaining dual attention features.

[0199] For example, feature fusion of the first attention feature and the second attention feature can be achieved by adding the first attention feature and the second attention feature element by element.

[0200] It should be noted that, based on the dual attention feature, the information contained in the background region and the information contained in the tumor lesion region can be fully utilized to increase the contrast between the background region and the tumor lesion region, thereby facilitating the identification of the tumor lesion region.

[0201] A third image feature is obtained by convolving the dual attention features; this third image feature can be used to indicate the probability that the predicted segmentation information is a labeled sample. Based on the third image probability, the segmentation quality score corresponding to the predicted segmentation information is obtained.

[0202] Optionally, determining the segmentation quality score corresponding to the predicted segmentation information based on the dual attention features includes:

[0203] For the dual attention features, Inception convolution is performed using the Inception convolution module of the second network to obtain third image features at different scales;

[0204] The third image feature is dilated by the dilated convolution module of the second network to obtain the fourth image feature;

[0205] Based on the fourth image feature, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0206] In this embodiment of the invention, the Inception convolution module has N Inception convolutional layers, and the convolutional kernels of different Inception convolutional layers are different; here, N is a positive integer greater than or equal to 1.

[0207] Third image features of different scales are extracted from the dual attention features through different Inception convolutional layers in the Inception convolutional module.

[0208] The dilated convolution module has N dilated convolution dense sub-modules. The Inception convolutional layer and the dilated convolution dense sub-modules are interspersed in the second network. The dilated convolution dense sub-modules perform dilated convolution processing on the third image features of different scales output by the Inception convolutional layer to obtain the fourth image features.

[0209] Optionally, the training method for the image segmentation network further includes:

[0210] The MRI images in the training sample set are cropped to obtain images of the tumor lesion area in the MRI images;

[0211] The image of the tumor lesion area is subjected to grayscale normalization.

[0212] The image of the tumor lesion area after grayscale normalization is subjected to geometric transformation to obtain the enhanced MRI image.

[0213] The step of inputting the MRI images of the training sample set into a first network pre-trained using labeled training data includes:

[0214] The enhanced MRI image is input into the first network.

[0215] In this embodiment of the invention, the cropping process may include: cropping the regions in the MRI image that do not contain tumor information, as well as the boundary regions and background regions in the MRI image;

[0216] By cropping the MRI images in the training sample set, cropped MRI images are obtained. A sliding window of a preset size is then slid across the cropped MRI images to obtain multiple images of preset sizes to be segmented.

[0217] Here, the preset size of the sliding window is the same as the image size adapted by the first network. For example, the preset size is 224×224.

[0218] The grayscale normalization process refers to setting the grayscale value of each pixel in the image to be processed within a specific range. In this embodiment of the invention, grayscale normalization can be performed on the multiple tumor lesion region images to make the grayscale value distribution of the multiple tumor lesion region images similar.

[0219] The grayscale normalization processing of the image of the tumor lesion region may include:

[0220] Based on the tumor lesion area image, determine the maximum and minimum pixel values ​​in the tumor lesion area image, as well as the difference information between the maximum and minimum pixel values;

[0221] The pixel difference corresponding to each pixel is determined based on the pixel value of each pixel in the image of the tumor lesion area and the minimum pixel value;

[0222] The grayscale normalized tumor lesion region image is determined based on the ratio of the pixel difference corresponding to each pixel point to the difference information.

[0223] Geometric transformation of the tumor lesion region image can be achieved by performing operations on the grayscale-normalized tumor lesion region image with a geometric transformation function. Here, the specific form of the geometric transformation function is not limited; it can be a geometric transformation function representing horizontal flipping, a geometric transformation function representing vertical flipping, etc.

[0224] It should be noted that training an image segmentation network requires a large number of labeled training samples, but in practice, the number of labeled training samples is limited. In order to make the training sample set as diverse as possible and improve the generalization ability of the image segmentation network, it is necessary to perform geometric transformation processing on the MRI images in the training sample set to achieve data augmentation of the training sample set.

[0225] The following is an embodiment of the present invention providing an image processing apparatus 30, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of an image processing apparatus provided in an embodiment of the present invention. The image processing apparatus includes:

[0226] The first convolutional module 31 is used to perform Inception convolution on MRI images using the Inception convolutional module of the image segmentation network to obtain the first image features.

[0227] The second convolution module 32 is used to perform dilated convolution on the first image features using the dilated convolution module of the image segmentation network to obtain the second image features;

[0228] The segmentation module 33 is used to segment the tumor lesion region from the MRI image based on the second image features, wherein the tumor lesion region is the region where the pixels of the tumor lesion tissue are located.

[0229] Optionally, the Inception convolution module has N Inception convolution layers, and the convolution kernels of different Inception convolution layers are different; different Inception convolution layers are used to extract the first image features at different scales.

[0230] Optionally, the dilated convolution module includes: N dilated convolution dense sub-modules;

[0231] The Inception convolutional layer and the dilated convolutional dense submodule are interspersed in the image segmentation network. The dilated convolutional dense submodule is used to perform dilated convolution processing on the first image features at different scales.

[0232] Optionally, the second convolutional module is specifically used for:

[0233] Perform dilated convolution on the received first image features to obtain the dilated convolution result;

[0234] The dilated convolution results are batch normalized to obtain normalized results;

[0235] The normalization result is rectified using a leakage rectifier function to obtain the second image feature.

[0236] Optionally, the image processing device further includes: a third convolution module, used to perform standard convolution processing on the MRI image to obtain a standard convolution result;

[0237] The first convolution module is used to perform Inception convolution on the standard convolution result based on different convolution kernels using the Inception convolution module to obtain multiple first image features at different scales.

[0238] The following embodiments of the present invention provide a training device 40 for an image segmentation network, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a training device for an image segmentation network provided in an embodiment of the present invention. The training device for the image segmentation network includes:

[0239] The acquisition module 41 is used to acquire a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples.

[0240] The first network module 42 is used to input the MRI images of the training sample set into the first network that has been pre-trained using labeled training data, and to obtain the predicted segmentation information output by the first network; wherein, the predicted segmentation information includes: the predicted segmentation information of the labeled training samples;

[0241] The second network module 43 is used to obtain a first segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the labeled training samples through the second network.

[0242] The loss determination module 44 is used to determine a first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels; determine a second loss value of the second network based on the first segmentation quality score and the labels of the labeled training samples; and determine whether to continue training the first network based on the first loss value and the second loss value.

[0243] Optionally, the second network module is further configured to obtain a second segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the unlabeled training samples.

[0244] The loss determination module is used to obtain the first loss value based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score and the second segmentation quality score.

[0245] Optionally, the loss determination module is specifically used for:

[0246] Based on the first segmentation quality score, the loss value of the second network is determined;

[0247] Based on the loss value of the second network, adjust the training parameters in the second network;

[0248] Using the adjusted second network, the predicted segmentation information of labeled training samples and the predicted segmentation information of unlabeled training samples are evaluated to obtain a first segmentation quality score for the predicted segmentation information of labeled training samples and a second segmentation quality score for the predicted segmentation information of unlabeled training samples.

[0249] The first loss value of the first network is determined based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score, and the second segmentation quality score.

[0250] Optionally, the loss determination module is used to:

[0251] Based on the predicted segmentation information of the labeled training samples and the labels, determine the segmentation loss function value corresponding to the labels;

[0252] Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score;

[0253] Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score;

[0254] The first loss value of the first network is obtained by weighted fusion of the segmentation loss function value, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score.

[0255] Optionally, the loss determination module is used to:

[0256] Based on the first segmentation quality score, the labels of the labeled training samples, and the second segmentation quality score, a second loss value for the second network is determined.

[0257] Optionally, the loss determination module is specifically used for:

[0258] Based on the segmentation quality score corresponding to the label, determine the binary cross-entropy corresponding to the label;

[0259] Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score;

[0260] Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score;

[0261] The binary cross-entropy corresponding to the label, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score are weighted and fused to obtain the second loss value of the second network.

[0262] Optionally, the training apparatus for the image segmentation network further includes: a pre-training module, used for:

[0263] Labeled training data is input into the initial network to obtain the predicted segmentation information output by the initial network;

[0264] The segmentation loss function value of the initial network is determined based on the difference between the predicted segmentation information and the label;

[0265] Based on the segmentation loss function value of the initial network, the training parameters of the initial network are adjusted to obtain the first network.

[0266] Optionally, the pre-training module is further configured to:

[0267] Based on the predicted segmentation information and the labeled information, the binary cross-entropy and Dice cross-entropy of the predicted segmentation information and the labeled information are determined respectively.

[0268] The binary cross-entropy and the Dice cross-entropy are weighted and fused to obtain the segmentation loss function value of the initial network.

[0269] Optionally, the second network module is used for:

[0270] The predicted segmentation information and the labeled training samples are input into the second network. The image intensity attention module and geometric boundary attention module of the second network are used to process the predicted segmentation information to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image of the labeled training samples, the contour features of the tumor lesion area and the contour features of the background area.

[0271] Based on the intensity signal difference features, the contour features of the tumor lesion area, and the contour features of the background area, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0272] Optionally, the second network module is specifically used for:

[0273] The predicted segmentation information is subjected to standard convolution processing to obtain the tumor lesion region features and the background region features of the MRI image, respectively.

[0274] The features of the tumor lesion area and the background area are classified to obtain the attention information of the tumor lesion area and the attention information of the background area in the MRI image.

[0275] The MRI image, the attention information corresponding to the tumor lesion area, and the attention information of the background area are fused to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image, the contour features of the tumor lesion area, and the contour features of the background area.

[0276] Optionally, the second network module is specifically used for:

[0277] The intensity signal difference features, the contour features of the tumor lesion region, and the contour features of the background region are cascaded to obtain dual attention features;

[0278] Based on the dual attention feature, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0279] Optionally, the second network module is further specifically used for:

[0280] For the dual attention features, Inception convolution is performed using the Inception convolution module of the second network to obtain third image features at different scales;

[0281] The third image feature is dilated by the dilated convolution module of the second network to obtain the fourth image feature;

[0282] Based on the fourth image feature, the segmentation quality score corresponding to the predicted segmentation information is determined.

[0283] Optionally, the apparatus further includes: a preprocessing module, for:

[0284] The MRI images in the training sample set are cropped to obtain images of the tumor lesion area in the MRI images;

[0285] The image of the tumor lesion area is subjected to grayscale normalization.

[0286] The image of the tumor lesion area after grayscale normalization is subjected to geometric transformation to obtain the enhanced MRI image.

[0287] The first network module is used to input the enhanced MRI image into the first network.

[0288] Based on the above embodiments of the present invention, an exemplary application of the embodiments of the present invention in a practical application scenario will be described below.

[0289] This example provides an image processing method, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating an image processing method provided for this example. The specific steps are as follows:

[0290] Step 501: Obtain a training sample set of MRI images and preprocess the MRI images;

[0291] A large number of MRI images are acquired to form a training sample set, which includes labeled and unlabeled training samples. The MRI images in the training sample set are cropped to remove regions containing no tumor information, as well as boundary and background information with pixel values ​​of 0. The cropped MRI images are then uniformly scaled to 224×224. The scaled MRI images are then subjected to grayscale normalization, compressing the grayscale value range to [-1, 1] to ensure a similar grayscale distribution across all MRI images. Finally, the grayscale-normalized MRI images undergo geometric transformation.

[0292] It should be noted that training neural networks in a fully supervised manner requires a large number of labeled training samples. However, in practical applications, the number of labeled MRI images is limited. Therefore, in order to diversify the training sample set as much as possible and improve the generalization ability of the neural network, geometric transformation of MRI images can be performed to augment the training sample set.

[0293] Here, the geometric transformation process may include: random angle rotation, horizontal flipping, and vertical flipping, etc.

[0294] Step 502: Construct a semi-supervised generative adversarial network;

[0295] In this embodiment of the invention, the semi-supervised generative adversarial network includes an image segmentation network and a discriminant network; wherein the image segmentation network is used to map random noise to a real image data distribution; and the discriminant network is used to determine whether the input image belongs to a real sample or a generated image sample. By using a semi-supervised generative adversarial network, a large number of unlabeled training samples can be effectively utilized, improving the accuracy of the image segmentation network and significantly reducing the manpower and material costs associated with data annotation.

[0296] In this embodiment of the invention, a semi-supervised generative adversarial network can be constructed by building a first network and a second network respectively.

[0297] Here, the first network is a basic segmentation network trained on the initial network based on labeled training samples. The initial network can adopt an encoder-decoder structure.

[0298] In the encoder section, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the network architecture of an image segmentation network provided in this example. A standard convolutional layer is set at the beginning of the initial network. This standard convolutional layer performs standard convolution on the input MRI image to extract low-dimensional features from the MRI image. Specifically, the kernel size of the standard convolutional layer is 5×5, and the stride is 2.

[0299] The Inception convolution module, located at the output of the standard convolutional layer in the initial network, performs Inception convolution based on the low-dimensional features of the MRI image to obtain the first image feature; and then performs dilated convolution on the first image feature through the dilated convolution module to obtain the second image feature.

[0300] It should be noted that, due to the unclear boundaries of tumor lesions in brain tumor MRI images, the images are blurry and have uneven grayscale distribution. In order to more effectively merge the low-level feature information of MRI images and generate higher-level feature information,

[0301] Here, the Inception convolutional module consists of multiple different Inception convolutional layers, each with a different convolutional kernel. For example, the Inception convolutional module can be composed of Inception convolutional layers with kernels of 1×1, 3×3, and 5×5. The dilated convolutional layer includes multiple dilated convolutional dense sub-modules; each dilated convolutional dense sub-module includes: a dilated convolutional layer, a BN layer, and a Leaky-ReLU function.

[0302] The Inception convolution module can perform Inception convolution on low-dimensional features of MRI images through multiple different Inception convolution layers to obtain first image features at different scales.

[0303] The dilated convolution module can perform dilated convolution on the received first image features through a dilated convolution layer to obtain a dilated convolution result; the BN layer can perform batch normalization on the dilated convolution result to obtain a normalized result; and the Leaky-ReLU function can be used to rectify the normalized result to obtain a second image feature.

[0304] In this example, the dilated convolutional layer in each dilated convolutional dense submodule can be a dilated convolutional layer with a kernel size of 3×3, a magnification parameter of 2, and a stride of 1; the receptive field is increased by dilated convolution of the first image features to adapt to tumor lesion regions of different sizes.

[0305] It should be noted that each dense dilated convolution submodule in the dilated convolution module is formed by concatenating the outputs of all layers in its feedforward network channel by channel. The dilated convolution layers allow for the perception of a larger field of view with fewer downsampling iterations. The dense dilated convolution submodules can access image features from lower-dimensional layers to help generate image features from higher-dimensional layers, ensuring gradient propagation to deeper layers. This embodiment of the invention considers the potentially significant differences in the size of tumor lesions in different MRI images. Through the Inception convolution module and the dilated convolution module, multi-scale image features can be captured, more accurately representing tumor lesion areas and adapting to the diverse appearances of tumor lesions in MRI images.

[0306] For the decoder section, a standard convolutional layer, a batch normalization (BN) layer, and a Leaky-ReLU layer can be set at the output of the dilated convolutional module. The second image features output by the dilated convolutional module are merged through the standard convolutional layer, the BN layer, and the Leaky-ReLU layer to obtain a segmentation probability map. A convolutional layer with a kernel size of 1×1 is used to perform channel transformation processing on the segmentation probability map so that the number of channels in the segmentation probability map is consistent with that of the input MRI image. An upsampling layer is then used to upsample the segmentation probability map, restoring the size of the segmentation probability map to be the same as that of the input MRI image.

[0307] The second network is a discriminant network used to determine whether the segmentation probability map distribution output by the first network satisfies the true label. The second network requires two inputs: a brain tumor MRI image and the tumor segmentation result corresponding to the brain tumor MRI image. In practical applications, for labeled training samples, the brain tumor MRI image, the label, and the segmentation probability map corresponding to the brain tumor MRI image output by the image segmentation network are required; while for unlabeled training samples, only the brain tumor MRI image and the segmentation probability map corresponding to the brain tumor MRI image output by the image segmentation network are needed.

[0308] like Figure 7 As shown, Figure 7 This is a schematic diagram of a discriminative network architecture provided in this example. A dual attention module is set at the beginning of the second network: an image intensity attention module and a geometric boundary attention module. The image intensity attention module is used to acquire the intensity signal difference between the tumor lesion region and the background region in the MRI image; the geometric boundary attention module is used to acquire the boundary features and geometric deformation features of the tumor lesion region in the MRI image. Through the dual attention modules, important features in the tumor lesion region and the background region of the MRI image can be extracted respectively, thereby effectively evaluating the quality of the predicted segmentation result output by the image segmentation network based on these important features.

[0309] Using a dual attention module, image features are first extracted from the brain tumor MRI image; then, two parallel convolutional layers are used to extract features of the tumor lesion region and the background region in the segmentation probability map, respectively; a sigmoid function is used to activate the features of the tumor lesion region and the background region output by the two parallel convolutional layers, respectively, to obtain attention maps of the tumor lesion region and the background region.

[0310] By combining the image features of the brain tumor MRI image with the attention map of the tumor lesion area and the attention map of the background area in an element-wise multiplication manner, attention features for indicating information of the tumor lesion area and reverse attention features for indicating information of the background area are obtained.

[0311] Deep image features that indicate detailed information in the segmentation probability map are extracted from the attention features and reverse attention features output by the dual attention module through the Inception convolution module and the dilated convolution module.

[0312] The extracted deep image features are converted into feature vectors by a global average pooling layer and input into a fully connected layer. The fully connected layer classifies the features based on the feature vectors and outputs the final evaluation score.

[0313] Step 503: Using labeled training samples, pre-train the initial network to obtain the first network;

[0314] The initial network is trained using labeled training samples. In each iteration of the training phase, preprocessed image data is input into the initial network, and the segmentation loss function value of the initial network is determined based on the difference between the predicted segmentation probability map output by the initial network and the label.

[0315] Specifically, the batch size of the initial network is set to 16, and the number of iterations of the initial network is set to 1500. In order to train the initial network and minimize the segmentation loss function value of the initial network, the Adam optimizer can be used to optimize the initial network. The parameters of the Adam optimizer are set to β1 = 0.9, β2 = 0.999, and the initial learning rate is set to 0.01.

[0316] The initial network's segmentation loss function consists of two parts: the Dice cross-entropy loss function and the binary cross-entropy loss function. The Dice cross-entropy loss function identifies the similarity between the contours of two tumor target regions in a brain tumor MRI image; however, when both tumor target regions are very small, the gradient changes drastically, making training difficult to converge. The binary cross-entropy loss function calculates the logarithmic loss of each pixel in the image, and it has a better effect on the segmentation details of brain tumor MRI images with unclear boundaries and uneven gray levels. By weighting the Dice cross-entropy loss function and the binary cross-entropy loss function, both the different scales and shapes of the tumor lesion region in the MRI image and the influence of the MRI image quality itself (uneven gray levels, noise, etc.) on the boundary segmentation of the tumor lesion region can be better achieved, enabling fully supervised training of the initial network for segmenting labeled MRI images. In continuous iterative training, by minimizing the segmentation loss function value, the segmentation probability map generated by the initial network is made closer to the label, thus obtaining the first network.

[0317] Step 504: Train the generative adversarial network using labeled training samples and unlabeled training samples.

[0318] The first and second networks are trained using labeled and unlabeled training samples, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of a generative adversarial network architecture provided in this example. In each iteration of the training phase, preprocessed image data is input into a first network, and the segmentation probability map output by the first network and the image data are input into a second network to obtain a quality score output by the second network. Based on the difference between the segmentation probability map and the label, as well as the quality score, a first loss value for the first network and a second loss value for the second network are determined.

[0319] The batch size is set to 16 and the number of iterations is set to 1500. In order to train the first network and the second network and minimize the first loss value of the first network and the second loss value of the second network, the Adam optimizer can be used to optimize the initial network. The parameters of the Adam optimizer are set to β1 = 0.9 and β2 = 0.999, and the initial learning rate of the first network and the second network is set to 0.01.

[0320] A first network, pre-trained on labeled training samples, is used as the generator in a generative adversarial network (GAN). Labeled and unlabeled training samples are input into the first network to obtain segmentation probability maps corresponding to the labeled training samples and predicted segmentation probability maps corresponding to the unlabeled training samples. Then, a second network is used as the discriminator in the GAN. The segmentation probability maps, labeled and unlabeled training samples are input into the second network to obtain a quality score. The segmentation results evaluated by the second network include the labels of the labeled training samples and the predicted segmentation probability maps output by the first network. Using the quality score output by the second network, adversarial learning is further performed between the first and second networks. Through adversarial training, the second network improves its ability to evaluate segmentation quality and, in turn, guides the first network to output more accurate segmentation results.

[0321] For training the second network, it requires two inputs: a brain tumor MRI image and the corresponding tumor segmentation result. For labeled training samples, the tumor segmentation result path has two sources: one is the label of the tumor lesion area annotated by the radiologist; the other is the segmentation probability map output by the first network. For unlabeled training samples, the tumor segmentation result path only includes the segmentation probability map output by the first network.

[0322] Based on the input image from the second network, a dual attention module (i.e., an image intensity attention module and a geometric boundary attention module) extracts dual attention features from the input image. Then, an Inception convolution module and a dilated convolution module extract deep features from the mixed features proposed by the dual attention module. A global average pooling layer then transforms the deep features into a feature vector; finally, a fully connected layer outputs a quality score based on the feature vector.

[0323] In this example, the loss function for the first network consists of a fully supervised segmentation loss function and an adversarial loss function. The main function of the segmentation loss function is to learn from the training samples of labeled MRI images, determining the basic segmentation performance of the generative adversarial network. The adversarial loss function establishes an adversarial relationship between the segmentation probability map predicted by the first network and the ground truth labels, making the distribution of the predicted segmentation probability maps of both labeled and unlabeled training samples closer to the labels. The loss function for the second network enables it to effectively distinguish between the predicted segmentation probability map output by the first network and the ground truth labels.

[0324] Step 505: Input the MRI image to be segmented into the trained image segmentation network to obtain the segmentation probability map output by the image segmentation network. Based on the segmentation probability map, determine the tumor lesion area in the MRI image.

[0325] This invention also provides an electronic device, the electronic device comprising:

[0326] Memory, used to store executable instructions;

[0327] The processor, when executing executable instructions stored in the memory, implements the image processing method provided by one or more of the aforementioned technical solutions.

[0328] The memory may include various types of storage media for data storage. In this embodiment, the storage media included in the memory are at least partially non-volatile storage media, which can be used to store the computer program.

[0329] The processor may include: a central processing unit, a microprocessor, a digital signal processor, an application processor, an application-specific integrated circuit, or a programmable array, etc., and can be used for image processing methods of one or more of the aforementioned technical solutions through computer programs.

[0330] In this embodiment, the processor can be connected to the memory via an internal device bus such as an integrated circuit bus.

[0331] This invention also provides an electronic device, the electronic device comprising:

[0332] Memory, used to store executable instructions;

[0333] The processor, when executing executable instructions stored in the memory, implements the training method of the image segmentation network provided by one or more of the aforementioned technical solutions.

[0334] This invention also provides a computer storage medium storing a computer program. When executed by a processor, the computer program performs the image processing method provided by one or more of the aforementioned technical solutions. For example, it can execute... Figure 1 The method shown.

[0335] This invention also provides a computer storage medium storing a computer program. When executed by a processor, the computer program performs a training method for the image segmentation network provided by one or more of the aforementioned technical solutions. For example, it can execute... Figure 2 The method shown.

[0336] The computer storage medium provided in this embodiment of the invention includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Optionally, the computer storage medium may be a non-transient storage medium. Here, non-transient storage medium can also be referred to as non-volatile storage medium.

[0337] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, include: For MRI images, Inception convolution is performed using the Inception convolution module of the image segmentation network to obtain the first image features; The first image features are dilated by using the dilated convolution module of the image segmentation network to obtain the second image features; Based on the second image features, the tumor lesion region is segmented from the MRI image, wherein the tumor lesion region is the region where the pixels of the tumor lesion tissue image are located; The Inception convolution module includes multiple Inception convolution layers, each with a different convolution kernel; the dilated convolution module includes multiple dilated convolution dense sub-modules; the Inception convolution layers and the dilated convolution dense sub-modules are interspersed in the image segmentation network. For the MRI image, the Inception convolution module of the image segmentation network is used to perform Inception convolution to obtain the first image features, including: For the MRI image, Inception convolution is performed using multiple Inception convolution layers within the Inception convolution module to obtain the first image features at different scales; The method of using the dilated convolution module of the image segmentation network to perform dilated convolution on the first image features to obtain the second image features includes: The first image features at different scales are processed by dilated convolution through the dilated convolution dense submodule to obtain the second image features.

2. The method according to claim 1, characterized in that, The method of using the dilated convolution module of the image segmentation network to perform dilated convolution on the first image features to obtain the second image features includes: Perform dilated convolution on the received first image features to obtain the dilated convolution result; The dilated convolution results are batch normalized to obtain normalized results; The normalization result is rectified using a leakage rectifier function to obtain the second image feature.

3. The method according to claim 1, characterized in that, The method further includes: The MRI images are subjected to standard convolution processing to obtain standard convolution results; For the MRI images, the Inception convolution module of the image segmentation network is used to perform Inception convolution to obtain multiple first image features at different scales, including: The Inception convolution module is used to perform Inception convolution on the standard convolution result based on different convolution kernels to obtain multiple first image features at different scales.

4. A training method for an image segmentation network, characterized in that, include: Obtain a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples; The MRI images of the training sample set are input into a first network pre-trained using labeled training data to obtain the predicted segmentation information output by the first network. The predicted segmentation information includes the predicted segmentation information of the labeled training samples. The first network is trained on an initial network based on labeled training data. The initial network includes an encoder and a decoder. The decoder includes standard convolutional layers, BN layers, Leaky-ReLU, 1*1 convolutional layers, and upsampling layers. The encoder includes standard convolutional layers, an Inception convolutional module, and a dilated convolutional module. The Inception convolutional module includes multiple Inception convolutional layers. The Inception convolutional layers have different convolutional kernels; the dilated convolution module includes multiple dilated convolution dense sub-modules; the Inception convolutional layers and the dilated convolution dense sub-modules are interspersed; wherein, the standard convolutional layer is used to perform standard convolution on the input MRI image to extract low-dimensional features of the MRI image; the Inception convolutional layer is used to perform Inception convolution on the low-dimensional features of the input MRI image to obtain first image features at different scales; the dilated convolution module is used to perform dilated convolution on the first image features at different scales to obtain second image features; the second image features are used to determine the predicted segmentation information output by the initial network; A first segmentation quality score is obtained for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the second network using the predicted segmentation information of the first network. The first loss value of the first network is determined based on the difference between the predicted segmentation information of the labeled training samples and the labels. Based on the first segmentation quality score and the labels of the labeled training samples, the second loss value of the second network is determined; Based on the first loss value and the second loss value, determine whether to continue training the first network.

5. The method according to claim 4, characterized in that, The method further includes: A second segmentation quality score is obtained for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the predicted segmentation information of the second network based on the unlabeled training samples. Determining the first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels includes: The first loss value is obtained based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score and the second segmentation quality score.

6. The method according to claim 5, characterized in that, The step of obtaining the first loss value based on the predicted segmentation information of the labeled training samples and the difference between the labels, the first segmentation quality score, and the second segmentation quality score includes: Based on the first segmentation quality score, the loss value of the second network is determined; Based on the loss value of the second network, adjust the training parameters in the second network; Using the adjusted second network, the predicted segmentation information of labeled training samples and the predicted segmentation information of unlabeled training samples are evaluated to obtain a first segmentation quality score for the predicted segmentation information of labeled training samples and a second segmentation quality score for the predicted segmentation information of unlabeled training samples. The first loss value of the first network is determined based on the difference between the predicted segmentation information of the labeled training samples and the labels, the first segmentation quality score, and the second segmentation quality score.

7. The method according to claim 6, characterized in that, The step of determining the first loss value of the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels, a first segmentation quality score, and a second segmentation quality score includes: Based on the predicted segmentation information of the labeled training samples and the labels, determine the segmentation loss function value corresponding to the labels; Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score; Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score; The first loss value of the first network is obtained by weighted fusion of the segmentation loss function value, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score.

8. The method according to claim 5, characterized in that, Determining the second loss value of the second network based on the first segmentation quality score and the labels of the labeled training samples includes: Based on the first segmentation quality score, the labels of the labeled training samples, and the second segmentation quality score, a second loss value for the second network is determined.

9. The method according to claim 8, characterized in that, The step of determining the second loss value of the second network based on the first segmentation quality score, the labels of the labeled training samples, and the second segmentation quality score includes: Based on the segmentation quality score corresponding to the label, determine the binary cross-entropy corresponding to the label; Based on the first segmentation quality score, determine the binary cross-entropy corresponding to the first segmentation quality score; Based on the second segmentation quality score, determine the binary cross-entropy corresponding to the second segmentation quality score; The binary cross-entropy corresponding to the label, the binary cross-entropy corresponding to the first segmentation quality score, and the binary cross-entropy corresponding to the second segmentation quality score are weighted and fused to obtain the second loss value of the second network.

10. The method according to claim 4, characterized in that, The method further includes: Labeled training data is input into the initial network to obtain the predicted segmentation information output by the initial network; The segmentation loss function value of the initial network is determined based on the difference between the predicted segmentation information and the label; Based on the segmentation loss function value of the initial network, the training parameters of the initial network are adjusted to obtain the first network.

11. The method according to claim 10, characterized in that, The step of determining the segmentation loss function value of the initial network based on the difference between the predicted segmentation information and the label includes: Based on the predicted segmentation information and the label, the binary cross-entropy and Dice cross-entropy of the predicted segmentation information and the label are determined respectively. The binary cross-entropy and the Dice cross-entropy are weighted and fused to obtain the segmentation loss function value of the initial network.

12. The method according to claim 4, characterized in that, The step of obtaining a first segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the second network using the labeled training samples includes: The predicted segmentation information and the labeled training samples are input into the second network. The image intensity attention module and geometric boundary attention module of the second network are used to process the predicted segmentation information to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image of the labeled training samples, the contour features of the tumor lesion area and the contour features of the background area. Based on the intensity information difference features, the contour features of the tumor lesion area, and the contour features of the background area, a first segmentation quality score corresponding to the predicted segmentation information is determined.

13. The method according to claim 12, characterized in that, The image intensity attention module and geometric boundary attention module of the second network process the predicted segmentation information to obtain the intensity information difference features between the tumor lesion region and the background region of the MRI image of the labeled training sample, the contour features of the tumor lesion region and the contour features of the background region, including: The predicted segmentation information is subjected to standard convolution processing to obtain the tumor lesion region features and background region features of the MRI image, respectively. The features of the tumor lesion area and the features of the background area are classified to obtain the attention information of the tumor lesion area and the attention information of the background area in the MRI image. The MRI image, the attention information corresponding to the tumor lesion area, and the attention information of the background area are fused to obtain the intensity information difference features between the tumor lesion area and the background area of ​​the MRI image, the contour features of the tumor lesion area, and the contour features of the background area.

14. The method according to claim 12, characterized in that, The step of determining the first segmentation quality score corresponding to the predicted segmentation information based on the intensity information difference features, the contour features of the tumor lesion region, and the contour features of the background region includes: The intensity information difference features, the contour features of the tumor lesion region, and the contour features of the background region are cascaded to obtain dual attention features; Based on the dual attention feature, a first segmentation quality score corresponding to the predicted segmentation information is determined.

15. The method according to claim 14, characterized in that, The step of determining the first segmentation quality score corresponding to the predicted segmentation information based on the dual attention features includes: For the dual attention features, Inception convolution is performed using the Inception convolution module of the second network to obtain third image features at different scales; The third image feature is dilated by the dilated convolution module of the second network to obtain the fourth image feature; Based on the fourth image feature, a first segmentation quality score corresponding to the predicted segmentation information is determined.

16. The method according to claim 4, characterized in that, The method further includes: The MRI images in the training sample set are cropped to obtain images of the tumor lesion area in the MRI images; The image of the tumor lesion area is subjected to grayscale normalization. The image of the tumor lesion area after grayscale normalization is subjected to geometric transformation to obtain the enhanced MRI image. The step of inputting the MRI images of the training sample set into a first network pre-trained using labeled training data includes: The enhanced MRI image is input into the first network.

17. An image processing apparatus, characterized in that, include: The first convolutional module is used to perform Inception convolution on MRI images using the Inception convolutional module of the image segmentation network to obtain the first image features. The second convolution module is used to perform dilated convolution on the first image features using the dilated convolution module of the image segmentation network to obtain the second image features; The segmentation module is used to segment the tumor lesion region from the MRI image based on the second image features, wherein the tumor lesion region is the region where the pixels of the tumor lesion tissue image are located; The Inception convolution module includes multiple Inception convolution layers, each with a different convolution kernel; the dilated convolution module includes multiple dilated convolution dense sub-modules; the Inception convolution layers and the dilated convolution dense sub-modules are interspersed in the image segmentation network. The first convolution module is used to perform Inception convolution on the MRI image using multiple Inception convolution layers within the Inception convolution module to obtain the first image features at different scales. The second convolution module is used to perform dilated convolution processing on the first image features at different scales through the dilated convolution dense submodule to obtain the second image features.

18. A training device for an image segmentation network, characterized in that, include: An acquisition module is used to acquire a training sample set of MRI images; wherein the training sample set includes: labeled training samples and unlabeled training samples; A first network module is used to input MRI images from the training sample set into a first network pre-trained using labeled training data to obtain predicted segmentation information output by the first network; wherein, the predicted segmentation information includes: predicted segmentation information of labeled training samples; the first network is trained on an initial network based on labeled training data; the initial network includes an encoder part and a decoder part; wherein, the decoder part includes: standard convolutional layers, BN layers, Leaky-ReLU, convolutional layers with a kernel size of 1*1, and upsampling layers; the encoder part includes standard convolutional layers, an Inception convolutional module, and a dilated convolutional module; the Inception convolutional module includes multiple Inception convolutional layers; The multiple Inception convolutional layers have different convolutional kernels; the dilated convolution module includes multiple dilated convolution dense sub-modules; the Inception convolutional layers and the dilated convolution dense sub-modules are interspersed; wherein, the standard convolutional layer is used to perform standard convolution on the input MRI image to extract low-dimensional features of the MRI image; the Inception convolutional layer is used to perform Inception convolution on the low-dimensional features of the input MRI image to obtain first image features at different scales; the dilated convolution module is used to perform dilated convolution on the first image features at different scales to obtain second image features; the second image features are used to determine the predicted segmentation information output by the initial network; The second network module is used to obtain a first segmentation quality score for the predicted segmentation information of the first network based on the predicted segmentation information of the labeled training samples through the second network. The loss determination module is used to determine a first loss value for the first network based on the difference between the predicted segmentation information of the labeled training samples and the labels; to determine a second loss value for the second network based on the first segmentation quality score and the labels of the labeled training samples; and to determine whether to continue training the first network based on the first loss value and the second loss value.

19. An electronic device comprising: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the image processing method as described in any one of claims 1-3.

20. An electronic device, comprising: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the training method of the image segmentation network as described in any one of claims 4-16.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions, which, when executed by a processor, implement the image processing method as described in any one of claims 1-3.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions, which, when executed by a processor, implement the training method for the image segmentation network as described in any one of claims 4-16.

Images