Training method of image segmentation model, and image segmentation method and device

Abstract

The invention provides a training method of an image segmentation model, and an image segmentation method, device and equipment. The training method comprises the following steps: acquiring a to-be-trained medical image and a segmentation result of the to-be-trained medical image; training the image segmentation model according to the to-be-trained medical image and the segmentation result of the to-be-trained medical image until a preset condition is reached, and obtaining a trained image segmentation model; the model comprises an encoder module and a decoder module, wherein the encoder module comprises a first convolutional layer, a pooling layer and a plurality of efficient pyramid segmentation attention units which are connected in sequence; and the efficient pyramid segmentation attention unit comprises a second convolutional layer, a pyramid extrusion attention module and a third convolutional layer which are connected in sequence. The pyramid extrusion attention module is adopted in the encoder module, so that the feature extraction capability of the encoder module is improved, the image segmentation model is enabled to pay more attention to a target object with significant features, and the segmentation accuracy of the image segmentation model is improved.

Description

Image segmentation model training method, image segmentation method and device Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a training method for an image segmentation model, an image segmentation method, a device and equipment. Background Art

[0002] Currently, segmenting target objects in medical images is one of the important areas in medical image processing. For example, cell segmentation is the basic prerequisite for identifying and counting cell images.

[0003] In related technologies, segmentation of target objects in medical images can be achieved using deep learning-based segmentation algorithms. However, current deep learning-based segmentation algorithms cannot effectively handle the boundaries of target objects, resulting in inaccurate segmentation results.

[0004] Summary of the Invention

[0005] In view of this, embodiments of the present application provide a training method for an image segmentation model, an image segmentation method, an apparatus, and an equipment to improve the segmentation accuracy of a target object.

[0006] To solve the above problems, the technical solutions provided in the embodiments of the present application are as follows:

[0007] A method for training an image segmentation model, the method comprising:

[0008] Acquire a medical image to be trained and a segmentation result of the medical image to be trained;

[0009] Training the image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met, thereby obtaining a trained image segmentation model;

[0010] The model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

[0011] In one possible implementation, the pyramid squeezing attention module includes:

[0012] The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group to generate N feature maps, and merge the N feature maps to generate a merged feature map; N is a positive integer;

[0013] an attention unit, configured to perform an attention operation on the merged feature map to generate a feature map after the attention operation;

[0014] The activation unit uses a softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.

[0015] In a possible implementation, obtaining a medical image to be trained and a segmentation result of the medical image to be trained includes:

[0016] Obtaining medical images to be trained;

[0017] Using a preset image segmentation algorithm to extract the target object, the target object outline, and the image background in the medical image to be trained;

[0018] The classification label of each pixel of the to-be-trained medical image belonging to the target object, the target object contour or the image background is determined as the segmentation result of the to-be-trained medical image.

[0019] In a possible implementation, the training of the image segmentation model according to the to-be-trained medical image and the segmentation result of the to-be-trained medical image until a preset condition is met to obtain a trained image segmentation model includes:

[0020] Inputting the medical image to be trained into an image segmentation model to obtain a predicted classification result for each pixel in the medical image to be trained;

[0021] Calculating a loss value based on a predicted classification result of each pixel in the to-be-trained medical image and a classification label of each pixel in the to-be-trained medical image as belonging to the target object, the target object outline, or the image background, and training the image segmentation model based on the loss value;

[0022] Repeating the steps of inputting the medical image to be trained into the image segmentation model, obtaining the predicted classification results of each pixel in the medical image to be trained, and subsequent steps until a preset condition is met, thereby obtaining a trained image segmentation model;

[0023] Among them, when calculating the loss value, the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the target object, and the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the image background.

[0024] An image segmentation method, comprising:

[0025] Inputting the medical image to be segmented into an image segmentation model to obtain an initial segmentation result for each pixel in the medical image to be segmented, wherein the initial segmentation result includes the pixel belonging to the target object, the target object outline, or the image background; the image segmentation model is trained according to the above-mentioned image segmentation model training method;

[0026] The pixel points belonging to the target object or the outline of the target object in the initial segmentation result are determined as pixel points belonging to the target object, and the segmentation result of the medical image to be segmented is output.

[0027] In a possible implementation, the method further includes:

[0028] A watershed algorithm is used to filter target objects whose areas are smaller than a threshold in the segmentation results of the medical image to be segmented, and / or to repair incomplete target objects.

[0029] A training device for an image segmentation model, comprising:

[0030] An acquisition unit, configured to acquire a medical image to be trained and a segmentation result of the medical image to be trained;

[0031] A training unit, configured to train an image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained, until a preset condition is met, thereby obtaining a trained image segmentation model;

[0032] The model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

[0033] An image segmentation device, comprising:

[0034] An input unit, configured to input the medical image to be segmented into an image segmentation model to obtain an initial segmentation result for each pixel in the medical image to be segmented, wherein the initial segmentation result includes pixels belonging to a target object, a target object outline, or an image background; the image segmentation model is trained according to the above-mentioned image segmentation model training method;

[0035] The determining unit is configured to determine the pixel points belonging to the target object or the outline of the target object in the initial segmentation result as the pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

[0036] A training device for an image segmentation model comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the training method for the image segmentation model as described above is implemented.

[0037] An image segmentation device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image segmentation method described above is implemented.

[0038] A computer-readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to execute the training method for the image segmentation model or the image segmentation method as described above.

[0039] It can be seen that the embodiments of the present application have the following beneficial effects:

[0040] The image segmentation model generated by training in the embodiment of the present application consists of an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and a plurality of efficient pyramid segmentation attention units connected in sequence. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence. That is, the pyramid squeezing attention module is adopted in the encoder module, which improves the feature extraction capability of the encoder module, enables the image segmentation model to pay more attention to target objects with significant features, and thus improves the segmentation accuracy of the image segmentation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0042] FIG2 is a flow chart of a method for training an image segmentation model according to an embodiment of the present application;

[0043] FIG3 is a schematic diagram of the structure of an image segmentation model in an embodiment of the present application;

[0044] FIG4 is a schematic diagram of the structure of an efficient pyramid segmentation attention unit in an embodiment of the present application;

[0045] FIG5 is a schematic diagram of the structure of an image segmentation model in an embodiment of the present application;

[0046] FIG6 is a schematic diagram of the structure of a pyramid squeezing attention module in an embodiment of the present application;

[0047] FIG7 is a flow chart of an image segmentation method provided in an embodiment of the present application;

[0048] FIG8 is a diagram showing the segmentation effect of the image segmentation model of an embodiment of the present application on a mouse brain image;

[0049] FIG9 is a diagram showing the segmentation effect of the image segmentation model of an embodiment of the present application on a monkey brain image;

[0050] FIG10 is a diagram showing the segmentation effect of the image segmentation model of an embodiment of the present application on a salamander brain image;

[0051] Figure 11 shows the segmentation results of various algorithms on mouse brain images;

[0052] FIG12 is a schematic diagram of the structure of an image segmentation model training device provided in an embodiment of the present application;

[0053] FIG13 is a schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0055] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0056] Currently, segmenting target objects in medical images is a key area of ​​medical image processing. For example, segmenting cells, organs, and tissues in medical images is crucial. However, current deep learning-based segmentation algorithms cannot effectively handle the boundaries of target objects, resulting in inaccurate segmentation results.

[0057] Taking cell segmentation as an example, cell segmentation is the fundamental prerequisite for identifying and counting cell images. However, due to the complexity of the segmented cell images, uneven illumination in the microscope images, and grayscale variations in the target objects themselves, segmented images can still exhibit some issues, such as cell adhesion and overlap.

[0058] Based on this, the embodiments of the present application provide a training method, an image segmentation method, an apparatus and a device for an image segmentation model, wherein the image segmentation model generated by training is composed of an encoder module and a decoder module. Among them, the encoder module includes a first convolutional layer, a pooling layer, and a plurality of efficient pyramid segmentation attention units connected in sequence. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module and a third convolutional layer connected in sequence. That is, the pyramid squeezing attention module is adopted in the encoder module, which improves the feature extraction capability of the encoder module and enables the image segmentation model to pay more attention to the target object with significant features. The segmentation of the target object in the medical image is realized based on the deep learning attention mechanism, thereby improving the accuracy of image segmentation.

[0059] To facilitate understanding of the image segmentation model training method and image segmentation method provided in the embodiments of the present application, the following description is provided in conjunction with the scenario example shown in Figure 1. Referring to Figure 1, this figure is a schematic diagram of the framework of an exemplary application scenario provided in the embodiments of the present application. As an optional example, this method can be applied to a terminal device or a server.

[0060] In practical applications, the image segmentation model can be trained based on the medical image to be trained and the segmentation results of the medical image to be trained. In one possible implementation, the segmentation results of the medical image to be trained can be considered as label data, which can include classification labels for each pixel in the medical image to be trained, such as the target object, the target object outline, or the image background. The medical image to be trained is input into the image segmentation model to obtain the predicted classification results of each pixel in the medical image to be trained. The predicted classification results of each pixel in the medical image to be trained and the segmentation results of the medical image to be trained are used to calculate the loss value, and the image segmentation model is adjusted using the loss value. The image segmentation model is then adjusted using the next medical image to be trained and the segmentation results of the medical image to be trained until the training is completed, thereby obtaining a trained image segmentation model.

[0061] Those skilled in the art will appreciate that the framework diagram shown in FIG1 is merely an example in which the embodiments of the present application may be implemented, and the scope of application of the embodiments of the present application is not limited in any way by the framework.

[0062] Based on the above description, the training method of the image segmentation model and the image segmentation method provided in the embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0063] Referring to FIG. 2 , which is a flowchart of a method for training an image segmentation model provided in an embodiment of the present application, as shown in FIG. 2 , the method for training an image segmentation model may include steps S201-S202:

[0064] S201: Obtain a medical image to be trained and a segmentation result of the medical image to be trained.

[0065] The medical images to be trained are images used to train the image segmentation model. They can be images obtained under a microscope or scanned by medical equipment, such as X-ray images or CT images. The type of medical images to be trained can be selected based on the target object to be segmented by the image segmentation model. For example, if the target object is a cell, the medical images to be trained are images obtained under a microscope. If the target object is an organ or tissue, the medical images to be trained are images obtained by scanning with medical equipment. The embodiments of the present application do not limit the type of medical images to be trained.

[0066] The segmentation results of the training medical images can be considered as label data for training the image segmentation model. The segmentation results of the training medical images can include classification labels indicating whether each pixel in the training medical image belongs to the target object, the target object outline, or the image background. The segmentation results of the training medical images can be manually annotated. However, to improve the efficiency of obtaining label data, the segmentation results of the training medical images can also be obtained using a pre-set image segmentation algorithm.

[0067] In a possible implementation, the specific implementation of S201 of obtaining the medical image to be trained and the segmentation result of the medical image to be trained may include A1-A3:

[0068] A1: Obtain the medical images to be trained.

[0069] A2: Use a preset image segmentation algorithm to extract the target object, target object outline, and image background in the medical image to be trained.

[0070] After acquiring the training medical image, a preset image segmentation algorithm can be used to segment the training medical image to determine whether each pixel in the training medical image belongs to the target object, the target object outline, or the image background. For example, if the training medical image is a microscope image containing cells, the DeepCell and CellProfiler algorithms can be used to process the training medical image to extract the cells, cell outlines, and image background.

[0071] A3: Determine the classification label of each pixel of the to-be-trained medical image as belonging to the target object, the target object outline, or the image background as the segmentation result of the to-be-trained medical image.

[0072] After the medical image to be trained is segmented, the classification label of each pixel of the medical image to be trained belonging to the target object, the target object outline or the image background can be determined as the segmentation result of the medical image to be trained, that is, the label data of the training image segmentation model is obtained.

[0073] By determining the segmentation results of the medical images to be trained through a preset image segmentation algorithm, a large number of segmentation results corresponding to the medical images to be trained can be quickly obtained.

[0074] S202: The image segmentation model is trained according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met, thereby obtaining a trained image segmentation model.

[0075] By inputting the medical image to be trained into the image segmentation model, an image prediction segmentation result can be obtained. The image prediction segmentation result is compared with the segmentation result of the medical image to be trained, and the loss value is calculated. The image segmentation model can be adjusted according to the loss value, specifically, the model parameters of the image segmentation model can be adjusted.

[0076] After adjusting the image segmentation model, it is necessary to determine whether the preset conditions are met. When the preset conditions are met, the image segmentation model training is completed and the trained image segmentation model is obtained. If the preset conditions are not met, the medical image to be trained is input into the image segmentation model again, and the image segmentation model is adjusted according to the loss value until the preset conditions are met. As an optional example, the preset condition is to reach a preset number of training times. As another optional example, the preset condition is that the loss value reaches a preset loss threshold. The preset number of training times and the preset loss threshold can be set according to actual conditions and are not limited here.

[0077] In a possible implementation, S202 trains the image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met. The specific implementation of obtaining the trained image segmentation model may include B1-B3:

[0078] B1: Input the medical image to be trained into the image segmentation model to obtain the predicted classification results of each pixel in the medical image to be trained.

[0079] By inputting the medical image to be trained into the image segmentation model, a predicted image segmentation result can be obtained, which can be specifically the predicted classification result of each pixel in the medical image to be trained. The predicted classification result can include whether it belongs to the target object, the target object outline, or the image background.

[0080] B2: Calculate the loss value based on the predicted classification results of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained, whether it belongs to the target object, the target object outline or the image background, and train the image segmentation model based on the loss value.

[0081] The predicted classification results are compared pixel by pixel in the training medical image with the classification label of whether the pixel belongs to the target object, the target object outline, or the image background. The loss value is calculated based on the comparison results of each pixel in the training medical image, and the image segmentation model is trained based on the loss value. In other words, the model parameters of the image segmentation model are adjusted based on the loss value.

[0082] B3: Repeat the steps of inputting the medical image to be trained into the image segmentation model, obtaining the predicted classification results of each pixel in the medical image to be trained, and subsequent steps until the preset conditions are met and the trained image segmentation model is obtained.

[0083] The medical image to be trained is input into the image segmentation model again. The loss value is calculated based on the predicted classification results of each pixel in the medical image to be trained and the classification label of each pixel in the medical image to be trained, whether it belongs to the target object, the target object outline or the image background. The image segmentation model is trained according to the loss value until the preset conditions are met to obtain a trained image segmentation model.

[0084] Among them, when calculating the loss value, the loss weight of the classification label as the target object contour is greater than the loss weight of the classification label as the target object, and the loss weight of the classification label as the target object contour is greater than the loss weight of the classification label as the image background.

[0085] Because the target object contour plays a relatively important role in image segmentation, the loss weight for the target object contour should be greater than the loss weights for the target object and image background when calculating the loss value. For example, the loss weight for the target object and image background is 0.2, and the loss weight for the target object contour is 0.6. In practical applications, the loss function for the target object and image background can be cross entropy, and the loss function for the target object contour can be focal loss.

[0086] In the embodiment of the present application, the structure of the image segmentation model is improved. See FIG3 , which shows a schematic diagram of the structure of the image segmentation model in the embodiment of the present application.

[0087] The image segmentation model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

[0088] In an embodiment of the present application, the Unet network structure is improved. Based on the Resnet network structure, the last fc (Full Connection) layer is removed, and the 3x3 convolution of each residual block in the Resnet network structure is replaced by a pyramid squeeze attention module (PSA Module). The improved residual block is named Epsablock (i.e., efficient pyramid segmentation attention unit). Referring to Figure 4, a structural diagram of Epsablock is shown. The efficient pyramid segmentation attention unit (Epsablock) includes a second convolutional layer, a pyramid squeeze attention module (PSA Module) and a third convolutional layer connected in sequence. Among them, the second convolutional layer and the third convolutional layer can be 1x1 convolution (Conv 1x1).

[0089] Referring to Figure 5, a schematic diagram of the structure of the image segmentation model in practical applications is shown. Epsablock replaces the original residual block as an improved resnet. The improved resnet includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units (Epsablock) connected in sequence. The improved resnet is used as the encoder of the Unet network structure, and the decoder is not modified. The improved Unet network structure is the image segmentation model of this application.

[0090] In one possible implementation, the pyramid squeeze attention module includes:

[0091] The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group to generate N feature maps, and merge the N feature maps to generate a merged feature map; N is a positive integer;

[0092] The attention unit is used to perform an attention operation on the merged feature map to generate a feature map after the attention operation;

[0093] The activation unit uses the softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.

[0094] See Figure 6, which shows the original schematic diagram of the pyramid squeeze attention module (PSA module). The slicing unit first divides the input features into N groups using 1x1 convolution, for example, N is 4. Convolution operations with different convolution kernel sizes are performed on each group, and the convolution kernel size of each group increases in sequence, for example, the convolution kernel size of each group is 3, 5, 7, and 9 in sequence. After the convolution operations of different sizes, the N feature maps generated are merged to generate a merged feature map.

[0095] The output of the segmentation unit is passed through the attention unit to obtain the channel attention value, that is, the feature map after the attention operation. The purpose of this is to obtain attention weights for feature maps of different scales. This approach integrates contextual information at different scales and produces better pixel-level attention. Finally, the activation unit performs softmax normalization on the feature map after the attention operation and performs a channel-wise product operation with the merged feature map to output the final feature map.

[0096] The PSA module allows the image segmentation model to pay more attention to targets with significant features.

[0097] The image segmentation model generated by training in the embodiment of the present application consists of an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and a plurality of efficient pyramid segmentation attention units connected in sequence. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence. That is, the pyramid squeezing attention module is adopted in the encoder module, which improves the feature extraction capability of the encoder module, enables the image segmentation model to pay more attention to target objects with significant features, and thus improves the segmentation accuracy of the image segmentation model.

[0098] Based on the embodiment of the training method of the image segmentation model provided above, the embodiment of the present application further provides an image segmentation method. Referring to FIG7 , FIG7 is a flow chart of an image segmentation method provided by the embodiment of the present application. As shown in FIG7 , the method includes S701-S702:

[0099] S701: Input the medical image to be segmented into the image segmentation model to obtain the initial segmentation results of each pixel in the medical image to be segmented, the initial segmentation results including the target object, the target object outline or the image background; wherein the image segmentation model is trained according to the above-mentioned image segmentation model training method.

[0100] By inputting the medical image to be segmented into the above-mentioned image segmentation model, a probability value of each pixel in the medical image to be segmented belonging to the target object, the target object outline, or the image background can be obtained. Based on whether the probability value is greater than a threshold, an initial segmentation result for each pixel in the medical image to be segmented is determined, i.e., it is determined that each pixel in the medical image to be segmented belongs to the target object, the target object outline, or the image background. In practical applications, the medical image to be segmented can be expanded into a three-channel image (e.g., RGB three-channel) and input into the image segmentation model to obtain an initial segmentation result for each pixel in each channel image, i.e., each pixel in each channel image belongs to the target object, the target object outline, or the image background. The initial segmentation results of each pixel in each channel image are then combined to determine the initial segmentation result for each pixel in the medical image to be segmented.

[0101] The image segmentation model is trained according to the image segmentation model training method provided in the above embodiment of the present application. For relevant instructions, please refer to the above embodiment and will not be repeated here.

[0102] S702: Determine the pixel points belonging to the target object or the outline of the target object in the initial segmentation result as the pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

[0103] The pixel points belonging to the target object or the target object outline are fused to achieve the segmentation of the target object, that is, the pixel points belonging to the target object or the target object outline are determined as the pixel points belonging to the target object, and the image segmentation is performed to output the segmentation result of the medical image to be segmented.

[0104] In the embodiment of the present application, the watershed algorithm may also be used to filter target objects whose areas are smaller than a threshold in the segmentation results of the medical image to be segmented, and / or to repair incomplete target objects.

[0105] By filtering out target objects with an area smaller than a threshold and / or repairing incomplete target objects, the target objects can be corrected, completing post-processing of the segmentation results of the medical image to be segmented and improving image segmentation accuracy. For example, when the target objects are cells, algorithms such as cell filtering and correction can filter out very small cells and correct incomplete cells, resulting in more accurate cell segmentation.

[0106] The image segmentation model used in the embodiment of the present application adopts a pyramid squeezing attention module in the encoder module, which improves the feature extraction capability of the encoder module and enables the image segmentation model to pay more attention to target objects with significant features, thereby improving the segmentation accuracy of the image segmentation model.

[0107] The present invention is applied to cell segmentation in spatiotemporal omics. As shown in Figure 8, the segmentation effect of the image segmentation model of the present invention on a mouse brain image is demonstrated. A portion of the mouse brain image in the above figure is captured for demonstration. As can be seen from the lower left figure, there is cell adhesion in the mouse brain image, and some cell boundaries are severely blurred. The image in the lower right figure uses the image segmentation model proposed in the present invention to segment independent cells from the adhered cells and provide relatively reliable boundaries.

[0108] Refer to Figures 9 and 10, which are the segmentation results of the image segmentation model provided in the embodiments of the present application on monkey brain images and salamander brain images, respectively. Parts are cut out from the original images for display. From the results, it can be seen that the image segmentation model provided in the embodiments of the present application can also process other images, indicating that the image segmentation model is generalizable.

[0109] See Figure 11 for the segmentation results of various algorithms on a mouse brain image. The left figure is a portion of the mouse brain image, the middle figure is the segmentation result of the mouse brain image using the original Unet network, and the right figure is the segmentation result of the mouse brain image using the watershed algorithm. Compared with the segmentation results of the image segmentation model proposed in the embodiment of the present application (see Figure 8), the segmentation results of the Unet network and the image segmentation model proposed in the embodiment of the present application are similar in the segmentation of clear cells. However, the Unet network's ability to process boundaries in the case of adherent cells is not as good as the image segmentation model proposed in the embodiment of the present application. The watershed algorithm has a poor ability to process boundaries, over-segmenting cells and forcing cells to be divided into multiple cells.

[0110] Table 1 further compares the Unet network and the image segmentation model proposed in this embodiment of the present invention. The quantitative metrics of the Unet network and the image segmentation model proposed in this embodiment of the present invention on five mouse brain images are compared. These metrics include the Dice coefficient, Intersection over Union (IoU), precision, and recall.

[0111] It can be seen from the results that the image segmentation model proposed in the embodiment of the present application has better segmentation performance.

[0112] Table 1

[0113]

[0114] In summary, the image segmentation model proposed in the embodiment of the present application has better segmentation effect than the Unet network and the watershed algorithm.

[0115] Based on the image segmentation model training method provided in the above method embodiment, the embodiment of the present application also provides an image segmentation model training device, and the image segmentation model training device will be described below with reference to the accompanying drawings.

[0116] See Figure 12, which is a schematic diagram of the structure of a training device for an image segmentation model provided in an embodiment of the present application. As shown in Figure 12, the training device for the image segmentation model includes:

[0117] An acquiring unit 1201 is configured to acquire a medical image to be trained and a segmentation result of the medical image to be trained;

[0118] A training unit 1202 is configured to train an image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met, thereby obtaining a trained image segmentation model;

[0119] The model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

[0120] In one possible implementation, the pyramid squeezing attention module includes:

[0121] The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group to generate N feature maps, and merge the N feature maps to generate a merged feature map; N is a positive integer;

[0122] an attention unit, configured to perform an attention operation on the merged feature map to generate a feature map after the attention operation;

[0123] The activation unit uses a softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.

[0124] In a possible implementation, the acquiring unit includes:

[0125] An acquisition subunit, used for acquiring medical images to be trained;

[0126] an extraction subunit, configured to extract the target object, the target object contour, and the image background in the medical image to be trained using a preset image segmentation algorithm;

[0127] The determination subunit is configured to determine the classification label of each pixel of the to-be-trained medical image belonging to the target object, the target object contour, or the image background as the segmentation result of the to-be-trained medical image.

[0128] In a possible implementation, the training unit includes:

[0129] An input subunit, configured to input the medical image to be trained into an image segmentation model to obtain a predicted classification result for each pixel in the medical image to be trained;

[0130] a training subunit, configured to calculate a loss value based on a predicted classification result of each pixel in the medical image to be trained and a classification label of each pixel in the medical image to be trained, that is, whether it belongs to the target object, the target object outline, or the image background, and train the image segmentation model based on the loss value;

[0131] a loop subunit, configured to return to the input subunit and the training subunit to execute inputting the medical image to be trained into the image segmentation model, obtaining a predicted classification result for each pixel in the medical image to be trained, and subsequent steps until a preset condition is met, thereby obtaining a trained image segmentation model;

[0132] Among them, when calculating the loss value, the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the target object, and the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the image background.

[0133] It should be noted that the technical details of the image segmentation model training device provided in the embodiments of the present application can be found in the relevant embodiments of the above-mentioned image segmentation model training method, which will not be repeated here.

[0134] Based on the image segmentation method provided in the above method embodiment, the embodiment of the present application further provides an image segmentation device, which will be described below with reference to the accompanying drawings.

[0135] Referring to FIG13 , FIG13 is a schematic diagram of the structure of an image segmentation device provided in an embodiment of the present application. As shown in FIG13 , the image segmentation device includes:

[0136] An input unit 1301 is configured to input a medical image to be segmented into an image segmentation model to obtain an initial segmentation result for each pixel in the medical image to be segmented, wherein the initial segmentation result includes a pixel belonging to a target object, a target object outline, or an image background; the image segmentation model is trained according to the above-mentioned image segmentation model training method;

[0137] The determining unit 1302 is configured to determine the pixel points in the initial segmentation result belonging to the target object or the outline of the target object as the pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

[0138] In a possible implementation, the apparatus further includes:

[0139] The correction unit is used to filter target objects with an area smaller than a threshold in the segmentation result of the medical image to be segmented using a watershed algorithm, and / or to repair incomplete target objects.

[0140] It should be noted that the technical details of the image segmentation device provided in the embodiments of the present application can be found in the relevant embodiments of the above-mentioned image segmentation method, which will not be repeated here.

[0141] In addition, an embodiment of the present application also provides a training device for an image segmentation model, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the training method for the image segmentation model as described above is implemented.

[0142] An embodiment of the present application further provides an image segmentation device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image segmentation method described above is implemented.

[0143] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device executes the training method of the image segmentation model as described above, or executes the image segmentation method as described above.

[0144] The image segmentation model generated by training in the embodiment of the present application consists of an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and a plurality of efficient pyramid segmentation attention units connected in sequence. The efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence. That is, the pyramid squeezing attention module is adopted in the encoder module, which improves the feature extraction capability of the encoder module, enables the image segmentation model to pay more attention to target objects with significant features, and thus improves the segmentation accuracy of the image segmentation model.

[0145] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0146] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0147] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0148] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0149] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for an image segmentation model, characterized in that: The method comprises: Acquire a medical image to be trained and a segmentation result of the medical image to be trained; Training the image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met, thereby obtaining a trained image segmentation model; The model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

2. The method according to claim 1, characterized in that The pyramid squeeze attention module includes: The slicing unit is used to divide the input features into N groups, perform convolution operations with different convolution kernel sizes on each group to generate N feature maps, and merge the N feature maps to generate a merged feature map; N is a positive integer; an attention unit, configured to perform an attention operation on the merged feature map to generate a feature map after the attention operation; The activation unit uses a softmax function to activate the feature map after the attention operation, performs a dot multiplication operation with the merged feature map, and outputs the final feature map.

3. The method according to claim 1, characterized in that The obtaining of the medical image to be trained and the segmentation result of the medical image to be trained includes: Obtaining medical images to be trained; Using a preset image segmentation algorithm to extract the target object, the target object outline, and the image background in the medical image to be trained; The classification label of each pixel of the to-be-trained medical image belonging to the target object, the target object contour or the image background is determined as the segmentation result of the to-be-trained medical image.

4. The method according to claim 3, characterized in that The image segmentation model is trained according to the medical image to be trained and the segmentation result of the medical image to be trained until a preset condition is met to obtain a trained image segmentation model, including: Inputting the medical image to be trained into an image segmentation model to obtain a predicted classification result for each pixel in the medical image to be trained; Calculating a loss value based on a predicted classification result of each pixel in the to-be-trained medical image and a classification label of each pixel in the to-be-trained medical image as belonging to the target object, the target object outline, or the image background, and training the image segmentation model based on the loss value; Repeating the steps of inputting the medical image to be trained into the image segmentation model, obtaining the predicted classification results of each pixel in the medical image to be trained, and subsequent steps until a preset condition is met, thereby obtaining a trained image segmentation model; Among them, when calculating the loss value, the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the target object, and the loss weight of the classification label being the target object contour is greater than the loss weight of the classification label being the image background.

5. An image segmentation method, characterized in that: The method comprises: Inputting the medical image to be segmented into an image segmentation model to obtain an initial segmentation result for each pixel in the medical image to be segmented, wherein the initial segmentation result includes pixels belonging to a target object, a target object outline, or an image background; the image segmentation model is trained using the image segmentation model training method according to any one of claims 1 to 4; The pixel points belonging to the target object or the outline of the target object in the initial segmentation result are determined as pixel points belonging to the target object, and the segmentation result of the medical image to be segmented is output.

6. The method according to claim 5, characterized in that The method further comprises: A watershed algorithm is used to filter target objects whose areas are smaller than a threshold in the segmentation results of the medical image to be segmented, and / or to repair incomplete target objects.

7. A training device for an image segmentation model, characterized in that: The device comprises: An acquisition unit, configured to acquire a medical image to be trained and a segmentation result of the medical image to be trained; A training unit, configured to train an image segmentation model according to the medical image to be trained and the segmentation result of the medical image to be trained, until a preset condition is met, thereby obtaining a trained image segmentation model; The model includes an encoder module and a decoder module. The encoder module includes a first convolutional layer, a pooling layer, and multiple efficient pyramid segmentation attention units connected in sequence; the efficient pyramid segmentation attention unit includes a second convolutional layer, a pyramid squeezing attention module, and a third convolutional layer connected in sequence.

8. An image segmentation device, characterized in that: The device comprises: an input unit, configured to input a medical image to be segmented into an image segmentation model to obtain an initial segmentation result for each pixel in the medical image to be segmented, wherein the initial segmentation result includes a pixel belonging to a target object, a target object outline, or an image background; the image segmentation model is trained according to the image segmentation model training method according to any one of claims 1 to 4; The determining unit is configured to determine the pixel points belonging to the target object or the outline of the target object in the initial segmentation result as the pixel points belonging to the target object, and output the segmentation result of the medical image to be segmented.

9. A training device for an image segmentation model, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for the image segmentation model according to any one of claims 1 to 4 is implemented.

10. An image segmentation device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image segmentation method according to any one of claims 5 to 6 is implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a terminal device, cause the terminal device to execute the image segmentation model training method described in any one of claims 1 to 4, or to execute the image segmentation method described in any one of claims 5 to 6.

Images