Cervical cell segmentation network training method, cervical cell segmentation method, device

Abstract

The present disclosure relates to a method for training a cervical cell segmentation network, a cervical cell segmentation method, and an apparatus. The method includes: adding an attention module to the connection structure between the encoder and the decoder of a U-shaped network to obtain a U-shaped attention network, where the U-shaped attention network is used to determine the nucleus region in a cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module; connecting each downsampling module and the first residual module in the encoder of the U-shaped attention network with dilated convolution to obtain a cervical cell segmentation network, where the cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex hierarchical feature information and the simple hierarchical feature information in the cervical cell image obtained through the dilation factor of the dilated convolution. By using this method, it is possible to accurately extract the abnormally shaped nuclei and abnormally shaped cells in the cervical cell image without introducing a large number of parameters and computational amounts.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image segmentation, and particularly to a method for training a cervical cell segmentation network, a method for segmenting cervical cells, and a device. Background Art

[0002] Cervical cancer is one of the cancers with the highest mortality rate among women, with more than 300,000 deaths each year. In cervical cytological diagnosis, the analysis of the component characteristics of the cell nucleus and cytoplasm in cells is mainly carried out. Compared with the cytoplasm, the component analysis of the cell nucleus is more reliable and has higher diagnostic value.

[0003] With the continuous development of deep learning, cervical cell nucleus segmentation technology has emerged. At present, the existing nucleus segmentation methods mainly include traditional image processing methods and deep learning-based methods. Among them, the traditional image processing methods use low-level features designed manually, resulting in low segmentation accuracy and unsatisfactory segmentation effects. The existing deep learning methods lack consideration of the abnormal cell shape and size, nuclear-cytoplasm contrast, and chromatin distribution, resulting in inaccurate extraction of abnormally shaped cell nuclei. Therefore, there are certain segmentation limitations. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method for training a cervical cell segmentation network, a method for segmenting cervical cells, and a device that can improve the segmentation accuracy and accurately extract abnormally shaped cell nuclei.

[0005] In a first aspect, the present disclosure provides a method for training a cervical cell segmentation network, the method including:

[0006] Adding a first residual module to the encoder of the U-shaped network, replacing the downsampling module in the encoder with a second residual module, and adding a second residual module connected to each upsampling module in the decoder to the decoder of the U-shaped network, where the first residual module and the second residual module are used to increase the training speed;

[0007] Adding an attention module to the connection structure between the encoder and the decoder of the U-shaped network to obtain a U-shaped attention network, where the U-shaped attention network is used to determine the cell nucleus region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module;

[0008] Connecting each downsampling module and the first residual module in the encoder of the U-shaped attention network to a dilated convolution to obtain a cervical cell segmentation network, where the cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex-level feature information and simple-level feature information in the cervical cell image obtained through the dilation factor of the dilated convolution.

[0009] In one embodiment, the attention module includes a channel attention module and a spatial attention module. Adding the attention module to the connection structure of the encoder and decoder of the U-shaped network includes:

[0010] Inputting the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into the attention module;

[0011] Obtaining the importance of each feature channel in the cervical cell image through the channel attention module;

[0012] Obtaining the importance of each spatial region in the cervical cell image through the spatial attention module;

[0013] Determining the nucleus region in the cervical cell image according to the importance of each feature channel in the cervical cell image obtained by the channel attention module and the importance of each spatial region obtained by the spatial attention module.

[0014] In one embodiment, the obtaining the importance of each feature channel in the cervical cell image through the channel attention module includes:

[0015] Inputting the cervical cell image into the global average pooling layer, the first fully connected layer, the rectified linear activation function layer, the second fully connected layer, and the sigmoid activation function layer of the channel attention module in sequence, and compressing the parameters of the cervical cell image to obtain the weights of each feature channel;

[0016] Multiplying the weights of each feature channel by the corresponding feature channels in the cervical cell image to obtain the importance of each feature channel.

[0017] In one embodiment, the obtaining the importance of each spatial region in the cervical cell image through the spatial attention module includes:

[0018] Inputting the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into two identical dimension transformation convolutions in the spatial attention module respectively, and converting the cervical cell image into the same dimension;

[0019] Performing a normalization operation on the cervical cell image after conversion through the rectified linear activation function, the dimension transformation convolution, and the sigmoid activation function in the spatial attention module to obtain an attention coefficient map;

[0020] Multiply the attention coefficient map with the cervical cell image to obtain the importance of each spatial region in the cervical cell image.

[0021] In one embodiment, it is characterized in that the channel attention module and the spatial attention module in the attention module are connected in parallel or in sequence.

[0022] In one embodiment, the first residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, two batch normalization layers, and an activation function layer; the second residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, three batch normalization layers, and two activation function layers.

[0023] In one embodiment, the dilation factor includes a complex feature dilation factor and a simple feature dilation factor; the connection of each downsampling module and the first residual module in the encoder of the U-shaped attention network to the dilated convolution includes:

[0024] Connect the first residual module and the first downsampling module in the encoder of the U-shaped attention network to the corresponding dilated convolution of the complex feature dilation factor;

[0025] Obtain the complex hierarchical feature information of the cervical cell image through the dilated convolution of the complex feature dilation factor;

[0026] Connect the second downsampling module in the encoder of the U-shaped attention network to the dilated convolution corresponding to the simple feature dilation factor;

[0027] Obtain the simple hierarchical feature information of the cervical cell image through the dilated convolution of the simple feature dilation factor;

[0028] Use the complex hierarchical feature information and the simple hierarchical feature information to determine abnormal cells in the cervical cell image.

[0029] In one embodiment, the method further includes:

[0030] Input the cervical cell image set into the cervical cell segmentation network, and adjust and train the cervical cell image set of the cervical cell segmentation network through a preset combined loss function, where the combined loss function is obtained by combining a focal loss function and a difference loss function.

[0031] In one embodiment, the calculation formula of the combined loss function includes:

[0032] Loss = w dice L dice + w focal L focal

[0033] Among them, w dice is the weight of the difference loss function, and L dice is the difference loss function; w focal is the weight of the focal loss function, and L focal is the focal loss function.

[0034] In one embodiment, before inputting the cervical cell image set into the cervical cell segmentation network, it further includes:

[0035] Performing image augmentation on the cervical cell images to obtain a cervical cell image set, and the methods of the image augmentation include: performing random rotation according to a preset random rotation angle range, performing random translation in the horizontal and vertical directions according to a preset translation range, performing random scaling according to a preset scaling range, and performing random flipping according to a preset flipping range.

[0036] In a second aspect, the present disclosure also provides a cervical cell segmentation method, and the method includes:

[0037] Obtaining a cervical cell image to be segmented;

[0038] Inputting the cervical cell image to be segmented into the cervical cell segmentation network trained by any one of the above cervical cell segmentation network training methods to segment the cervical cell image, and obtaining the nucleus image of the cervical cells in the cervical cell image to be segmented.

[0039] In a third aspect, the present disclosure also provides a cervical cell segmentation network training device, and the device includes:

[0040] A residual addition module, configured to add a first residual module to the encoder of the U-shaped network, replace the downsampling module in the encoder with a second residual module, and add a second residual module connected to each upsampling module in the decoder to the decoder of the U-shaped network, and the first residual module and the second residual module are used to increase the training speed;

[0041] An attention addition module, configured to add an attention module to the connection structure of the encoder and decoder of the U-shaped network to obtain a U-shaped attention network, and the U-shaped attention network is used to determine the nucleus region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module;

[0042] Atrous convolution connection module, which is used to connect each downsampling module and the first residual module in the encoder of the U-shaped attention network with atrous convolution to obtain a cervical cell segmentation network. The cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex hierarchical feature information and simple hierarchical feature information in the cervical cell image obtained by the dilation factor of the atrous convolution.

[0043] Fourthly, the present disclosure also provides a cervical cell segmentation device, which is characterized in that the device includes:

[0044] Cervical cell image acquisition module, which is used to acquire the cervical cell image to be segmented;

[0045] Cervical cell image segmentation module, which is used to input the cervical cell image to be segmented into the cervical cell segmentation network trained by any one of the above cervical cell segmentation network training methods to segment the cervical cell image, and obtain the nucleus image of the cervical cells in the cervical cell image to be segmented.

[0046] Fifthly, the present disclosure also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0047] Sixthly, the present disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0048] Seventhly, the present disclosure also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0049] In the above cervical cell segmentation network training method, cervical cell segmentation method, and device, the attention module in the cervical cell segmentation network can solve the problems of low contrast between the cell nucleus and cytoplasm, irregular distribution of chromatin in the smear, etc. caused by irregular changes in the shape, color, and size of abnormal cells, interference from impurities such as inflammatory factors, etc. in the cervical cell image without introducing a large number of parameters and computational complexity. By introducing skip connections in the U-net network, the features of the cervical cell image obtained by the downsampling module of the encoder are spliced into the feature maps of the cervical cell image obtained by the upsampling module of each stage of the decoder. In this way, the fusion of the underlying position detail information and the high-level semantic information is achieved during the upsampling process, which can improve the segmentation accuracy. And through the dilated convolution in the cervical cell segmentation network, different dilation factors can be set, so as to determine the corresponding cell nucleus in the abnormal cell region based on the complex overall information and detailed local information in the cervical cell image, improve the segmentation effect of the details of the cervical cell image and the edge of the cell nucleus in the cervical cell image, and can effectively process abnormal cells of different shapes and sizes. Description of the Drawings

[0050] Figure 1 It is a schematic diagram of the application environment of the cervical cell segmentation network training method in an embodiment;

[0051] Figure 2 It is a schematic flowchart of the cervical cell segmentation network training method in an embodiment;

[0052] Figure 3 It is a schematic diagram of the U-shaped attention network structure in an embodiment;

[0053] Figure 4 It is a schematic diagram of the cervical cell segmentation network structure in an embodiment;

[0054] Figure 5 It is a schematic flowchart of step S204 in an embodiment;

[0055] Figure 6 It is a schematic flowchart of step S504 in an embodiment;

[0056] Figure 7 It is a schematic diagram of the channel attention module structure in an embodiment;

[0057] Figure 8 It is a schematic flowchart of step S506 in an embodiment;

[0058] Figure 9 It is a schematic diagram of the spatial attention module structure in an embodiment;

[0059] Figure 10 It is a schematic diagram of the first residual module structure in an embodiment;

[0060] Figure 11 Schematic diagram of the structure of the second residual module in an embodiment;

[0061] Figure 12 Schematic diagram of the process of step S204 in an embodiment;

[0062] Figure 13 Schematic diagram of the process of the cervical cell segmentation network training method in another embodiment;

[0063] Figure 14 Schematic block diagram of the structure of the cervical cell segmentation network training device in an embodiment;

[0064] Figure 15 Schematic diagram of the internal structure of a computer device in an embodiment. Detailed implementation manners

[0065] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0067] Currently, the methods for segmenting the nuclei of cervical cells generally include: (1) manually segmenting cervical cell smears; (2) segmenting cervical cell nuclei based on traditional image processing methods, such as clustering algorithms, threshold algorithms, watershed algorithms, graph cut algorithms, etc.; (3) segmenting cervical cell nuclei by deep learning methods, such as algorithms combining convolutional networks and graph methods, algorithms combining fully convolutional networks and fully connected conditions, etc.

[0068] However, the current method of manually segmenting cervical cell smears requires a large number of experienced professionals, which is time-consuming and error-prone. Based on traditional image processing methods, this method is based on prior assumptions such as the circular boundary and low intensity of cervical cell nuclei to manually design and extract low-level features. However, abnormal cell nuclei usually exhibit irregular shapes, and these features may not cover all nuclear structures. Moreover, the manually designed low-level features themselves lack detailed structural information, resulting in unsatisfactory segmentation effects. In the case of deep learning-based methods, due to the irregular changes in the shape, color, and size of abnormal cells in cervical cell smears, interference from impurities such as inflammatory factors, the low contrast between cell nuclei and cytoplasm, and the irregular distribution of chromatin in the smear, the extraction of the nuclei of abnormal cells is inaccurate.

[0069] Therefore, to solve the above problems, the embodiments of the present disclosure provide a method for training a cervical cell segmentation network, which can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. The terminal 102 adds a first residual module to the encoder in the U-shaped network of the server 104. The terminal 102 replaces the downsampling module in the encoder with a second residual module. The terminal 102 adds a second residual module connected to each upsampling module in the decoder to the decoder in the U-shaped network of the server 104. The terminal 102 adds an attention module to the connection structure between the encoder and the decoder in the U-shaped network of the server 104 to obtain a U-shaped attention network. The U-shaped attention network obtained by the terminal 102 after adding the attention module can determine the nuclear region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module. The terminal 102 connects each downsampling of the encoder of the U-shaped attention network obtained after adding the attention module with a dilated convolution with a dilation factor, thereby obtaining a cervical cell segmentation network. The cervical cell segmentation network can determine the abnormal cell region in the cervical cell image according to the complex-level feature information and simple-level feature information in the cervical cell image obtained by the dilation factor. Therefore, the cervical cell segmentation network can determine the nuclear region in the cervical cell image and the abnormal cell region in the cervical cell image. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smartphones, tablet computers, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers. It should be noted that this method can also be used alone for the terminal or the server and can be implemented alone through the terminal or the server.

[0070] In one embodiment, as Figure 2As shown, a method for training a cervical cell segmentation network is provided. Taking the application of this method to the Figure 1 terminal in

[0071] as an example, the method includes the following steps:

[0072] Among them, the U-shaped network can usually be a U-net network. U-net is a classic network design method and has a large number of applications in image segmentation tasks. The encoder of the U-shaped network can usually be a general term for the downsampling part of the U-net. The decoder of the U-shaped network can usually be a general term for the upsampling part of the U-net.

[0073] Specifically, the encoder mainly includes four downsampling modules. Generally, it is considered that 4 layers of downsampling can obtain better results. If the network depth is further increased, it will cause the network to be difficult to converge. The decoder corresponds to the encoder structure and includes four upsampling modules. Add a first residual module to the encoder of the U-shaped network, and replace the downsampling module in the encoder with a second residual module. Usually, the first residual module replaces the convolution and max-pooling layers of the downsampling module in the encoder of the U-shaped network. Add a second residual module to the decoder in the U-shaped network and connect it to each upsampling module in the decoder. The size of the cervical cell image is gradually restored by connecting the upsampling module through the second residual module. After the last second residual module in the decoder, a convolution layer and an activation function layer are connected. The problem that the data distribution of the intermediate layer changes during the training process is solved through the network architecture layers existing in the first residual module and the second residual module, so as to prevent gradient disappearance or explosion and accelerate the training speed.

[0074] S204, add an attention module to the connection structure between the encoder and decoder of the U-shaped network to obtain a U-shaped attention network. The U-shaped attention network is used to determine the nucleus region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module.

[0075] Among them, the attention module is usually a module used to focus on local information, which can usually be used to locate the information of interest and suppress the useless information. The results are usually displayed in the form of a probability map or a probability feature vector. The mechanism of the attention module can be generally understood as follows: using some networks to calculate a weight, operating this weight with the image, changing this image, and obtaining the image with enhanced attention. The importance of the spatial region usually represents the region of interest in the cervical cell image, usually the region of cells or the region of cell nuclei. The importance of the feature channel usually represents the importance degree of the feature channel in the cervical cell image.

[0076] Specifically, as Figure 3 shown, in the structure of the skip connection between the encoder and the decoder of the U-shaped network, adding an attention module can be to connect the first residual module of the first layer of the encoder of the U-shaped network with the attention module, and the upsampling module of the second layer in the corresponding decoder is connected with the attention module through the second residual module. Connect the second residual module of the second layer of the encoder of the U-shaped network with the attention module, and the upsampling module of the third layer in the corresponding decoder is connected with this attention module through the second residual module, and so on. When reaching the bottom layer of the U-shaped network, the downsampling module of the last layer in the encoder is connected with the attention module connected to the downsampling module of the previous layer. The feature information of the cervical cell image in the attention module and the upsampling module is merged through the merging layer. After inputting the cervical cell image into the U-shaped attention network, the importance degree of the feature channel and the importance degree of the spatial region in the cervical cell image can be obtained through the attention module. Furthermore, the region of interest in the cervical cell image can be determined through the importance degree of the feature channel and the importance degree of the spatial region, and then the region of interest can be identified and distinguished to determine the region of cell nuclei in the cervical cell image.

[0077] S206, connect each downsampling module and the first residual module in the encoder of the U-shaped attention network with the dilated convolution to obtain a cervical cell segmentation network, and the cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex-level feature information and the simple-level feature information obtained through the dilation factor.

[0078] Among them, dilated convolution is usually a convolution method that can expand the receptive field and is very useful in detection and segmentation tasks. On the one hand, after the receptive field is expanded, large targets can be detected and segmented. On the other hand, with high resolution, the target can be accurately located. It can also capture multi-scale context information: dilated convolution has a parameter to set the dilation rate. Therefore, when different dilation rates are set, the receptive fields will be different, that is, multi-scale information is obtained. Simple hierarchical feature information can be relatively simple local detail information in the relatively shallow layer of the cervical cell image. Complex hierarchical feature information can be accurate and complex overall information in the cervical cell image.

[0079] Specifically, each downsampling module in the encoder of the obtained U-shaped attention network is respectively connected to a dilated convolution. After connection, a new U-shaped attention network is obtained, as Figure 4 shown. This new U-shaped attention network is set as the cervical cell segmentation network. The left side is the encoder part, which can include a first residual module and four second residual modules connected in sequence. And the first residual module and each second residual module are both connected to a dilated convolution. After connection, each dilated convolution is connected to an attention module. The right side is the decoder part. The decoder includes four upsampling modules. Each upsampling module is connected to a second residual module in the decoder. After the upsampling module in the decoder passes through the second residual module, it is connected to the attention module of the upper layer. The output of the upper-layer attention module is sent to the merging layer of this layer. The feature information extracted by the upsampling module and the attention module is merged through the merging layer. The last second residual module in the decoder usually can be that after the topmost second residual module, a convolutional layer and an activation function layer are connected, and classification is performed through the convolutional layer and the activation function layer.

[0080] After the cervical cell image is input into the cervical cell segmentation network, the features of the cervical cell image are extracted through the downsampling module, and then the complex hierarchical feature information and simple hierarchical feature information in the cervical cell image are extracted through the dilated convolution. Furthermore, information exchange of the complex hierarchical feature information and the simple hierarchical feature information is carried out, so as to determine the abnormal cell region in the cervical cell image. After the abnormal cell region is determined, because an attention module is also added to the cervical cell segmentation network, the corresponding cell nucleus in the abnormal cell region can be determined through the attention module.

[0081] The traditional method to solve the problems of irregular changes in the shape, color, and size of abnormal cells in cervical cell images, interference from impurities such as inflammatory factors, resulting in low contrast between the cell nucleus and cytoplasm, and irregular distribution of chromatin in the smear is to obtain the region of interest using a coarse-grained model and then use a second model to perform fine segmentation in the region of interest. However, this will lead to an increase in the number of parameters and the amount of computation. In the above-mentioned cervical cell segmentation network training method, the attention module in the cervical cell segmentation network can solve the above problems without introducing a large number of parameters and the amount of computation. Since the skip connection is introduced through the U-net network to splice the features of the cervical cell image obtained by the downsampling module of the encoder into the feature map of the cervical cell image obtained by the upsampling module of the decoder at each stage. In this way, the fusion of the underlying position detail information and the high-level semantic information is achieved during the upsampling process, which can improve the segmentation accuracy. And through the dilated convolution in the cervical cell segmentation network, different dilation factors can be set, so as to determine the corresponding cell nucleus in the abnormal cell region through the complex overall information and detailed local information in the cervical cell image, improve the segmentation effect of the details of the cervical cell image and the edge of the cell nucleus in the cervical cell image, and can effectively process abnormal cells of different shapes and sizes. And through the first residual module and the second residual module, the problem of training degradation can be solved to a certain extent, and the training speed can be increased.

[0082] In one embodiment, the attention module includes a channel attention module and a spatial attention module.

[0083] Among them, the channel attention module can determine the importance between various features in the cervical cell image, and for different tasks, feature allocation can be performed according to the input. The spatial attention module can locate different regions in the cervical cell image and perform some transformations or obtain weights.

[0084] As Figure 5 shown, adding an attention module to the connection structure of the encoder and decoder of the U-shaped network includes:

[0085] S502, input the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into the attention module.

[0086] Specifically, the input cervical cell image of the attention module mainly comes from the cervical cell image input after passing through the second residual module in the upsampling module of the decoder and the input cervical cell image of the downsampling module in the previous layer of the encoder corresponding to this decoder.

[0087] S504, obtain the importance of each feature channel in the cervical cell image through the channel attention module.

[0088] Specifically, the cervical cell image is input into the channel attention module, and the cervical cell image is processed through various architecture layers in the channel attention module, such as the fully connected layer, pooling layer, etc., so as to obtain the weights of each feature channel in the cervical cell image, and then the importance of each feature channel in the cervical cell image can be determined according to the weights of each feature channel.

[0089] S506, obtain the importance of each spatial region in the cervical cell image through the spatial attention module.

[0090] Specifically, the cervical cell image is input into the spatial attention module, and the cervical cell image is processed through various architectures in the spatial attention module, such as dimension transformation convolution, activation function, etc., so as to obtain the attention coefficient map, and the importance of each spatial region in the cervical cell image can be determined according to the attention coefficient map.

[0091] S508, determine the nucleus region in the cervical cell image according to the importance of each feature channel in the cervical cell image obtained by the channel attention module and the importance of each spatial region obtained by the spatial attention module.

[0092] Specifically, by synthesizing the importance of each feature channel and the importance of each spatial region in the cervical cell image, the part with higher importance in the cervical cell image can be obtained, and the nucleus region in the cervical cell image can be determined through this part with higher importance.

[0093] In this embodiment, by combining the channel attention module and the spatial attention module, the region of interest in the cervical cell image can be determined more accurately, and then the problems of irregular changes in the shape, color and size of abnormal cells in the cervical cell image, interference by impurities such as inflammatory factors, resulting in low contrast between the nucleus and cytoplasm, and irregular distribution of chromatin in the smear can be solved.

[0094] In one embodiment, as Figure 6 shown, the obtaining the importance of each feature channel in the cervical cell image through the channel attention module includes:

[0095] S602, input the cervical cell image into the global average pooling layer, the first fully connected layer, the rectified linear activation function layer, the second fully connected layer, and the sigmoid activation function layer of the channel attention module in sequence, and compress the parameters of the cervical cell image to obtain the weights of each feature channel.

[0096] Among them, the global average pooling layer can add up all the pixel values in the input cervical cell images from different places and calculate the average to obtain a value, that is, use this value to represent the corresponding cervical cell image. The rectified linear activation function layer can be a RELU (Rectified Linear Unit) activation function layer. The RELU activation function can increase the non-linearity of the neural network model, overcome the problem of gradient disappearance, and speed up the training speed. The sigmoid activation function layer usually refers to the sigmoid function, which is used as the activation function of the neural network to map the variable to the range between 0 and 1.

[0097] Specifically, the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder are both input to the global average pooling layer of the channel attention module. The global average pooling layer processes the cervical cell images input from two places, thereby outputting a cervical cell image. Then, through the first fully connected layer, the rectified linear activation function layer, and the second fully connected layer, the parameters of the output cervical cell image are compressed. Finally, the weights of each feature channel in the cervical cell image are obtained through the sigmoid activation function layer.

[0098] S604, multiply the weights of each feature channel by the corresponding feature channels in the cervical cell image to obtain the importance of each feature channel.

[0099] Specifically, multiply the weights of each feature channel by the corresponding feature channels in the cervical cell image to obtain the importance of each feature channel.

[0100] In some embodiments, such as Figure 7As shown, the input is a cervical cell image of W×H×C. H and W respectively represent the height and width of the image features, and C represents the feature channels. There are a total of C feature channels, and then a spatial global average pooling is performed. A scalar is obtained for each channel, and the output is 1×1×C. Then it is sent into a two-layer fully connected layer and a RELU (Rectified Linear Unit) activation function layer. The first fully connected layer changes 1×1×C to 1×1×C / r, and then inputs it into the RELU (Rectified Linear Unit) activation function layer. The output size is still 1×1×C / r, and then it is input into the second fully connected layer, and the output is 1×1×C while keeping the size unchanged. Then, through a Sigmoid function, C weights between 0 and 1 are obtained, in the format of 1×1×C, as the weight coefficients of each of the C feature channels. The scale operation multiplies the obtained weight coefficients of each channel by the image channels of the corresponding feature channels of W×H×C, usually multiplying with all the feature channels, so as to enhance the important feature channels and weaken the unimportant feature channels, so that the extracted feature channels have stronger directivity.

[0101] In one embodiment, as Figure 8 shown, obtaining the importance of each spatial region in the cervical cell image through the spatial attention module includes:

[0102] S802, input the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into two identical dimension conversion convolutions in the spatial attention module respectively, and convert the cervical cell image into the same dimension.

[0103] Specifically, input the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into two identical dimension conversion convolutions respectively. The dimension conversion convolution can be a 1×1 convolution. Two 1×1 convolutions are used to make the two parts of the input have the same dimension information.

[0104] S804, perform a normalization operation on the cervical cell image of the same dimension converted by the dimension conversion convolution through the linear correction activation function, dimension conversion convolution and S-shaped activation function in the spatial attention module to obtain an attention coefficient map.

[0105] Specifically, the cervical coefficient images with the same dimensional information after being transformed by 1×1 convolution are normalized for the cervical cell images through a ReLU activation function, 1×1 convolution, and a sigmoid function. After the normalization operation, different values are obtained, and these different values can be used as the attention coefficient map.

[0106] S806, multiply the attention coefficient map with the cervical cell image to obtain the importance of each spatial region in the cervical cell image.

[0107] Specifically, multiply the attention coefficient map with the corresponding pixels in the cervical cells to obtain the importance of each spatial region in the cervical cells.

[0108] In some embodiments, as Figure 9 shown, g is the cervical cell image from the decoder, and x is the cervical cell image from the encoder. Then, since their dimensions are different and the dimension from the encoder is larger, it is restored to the same dimension using 1×1 convolution. After passing the cervical cell image through 1×1 convolution, it is input into two routes. One route is that the information enters the localization network, including a linear rectifier activation function, a dimensionality conversion convolution, an S-shaped activation function, and an upsampling layer to output the attention coefficient map. The other route is that the convolved cervical cell image is directly multiplied by the output attention coefficient map.

[0109] Specifically, the localization network will generate a set of parameters, and this set of parameters can be used as the parameters of a grid generator (gridgenerator) to generate a sampling signal, which can be the attention coefficient map. After multiplying with the cervical cell image, the importance of each spatial region in the cervical cells can be obtained.

[0110] In one embodiment, the channel attention module and the spatial attention module in the attention module are connected in parallel or in sequence.

[0111] Specifically, the channel attention module and the spatial attention module can be connected in a parallel manner, or the channel attention module can be in the front and the spatial attention module can be in the back for sequential connection, or the channel attention module can be in the back and the spatial attention module can be in the front for sequential connection. Among them, those skilled in the art have found through actual verification that when the spatial attention module and the channel attention module are combined, the sequential connection method with the channel attention module placed in the front can achieve better results.

[0112] In one embodiment, the first residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, a batch normalization layer, and an activation function layer. The second residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, three batch normalization layers, and two activation function layers.

[0113] Specifically, as Figure 10 shown, the first residual module is generally composed of two first-scale convolutional layers, which can be 3×3 convolutional layers, one second-scale convolutional layer, which can be 1×1 convolutional layer, a batch normalization layer, and a RELU activation function layer. The stride of both the 3×3 convolutional layer and the 1×1 convolutional layer is 1. Finally, the image output by the upper 3×3 convolutional layer and the image output by the lower batch normalization layer are added pixel by pixel to output the result.

[0114] As Figure 11 shown, the second residual module is used to replace the convolutional and max-pooling layers of the downsampling module of the U-net network. The second residual module adds a batch normalization layer and a RELU activation function layer compared with the first residual module, and the stride of the first 3×3 convolution and 1×1 convolution is 2, and the stride of the second 3×3 convolution is 1. Finally, the image output by the upper 3×3 convolutional layer and the image output by the lower batch normalization layer are added pixel by pixel to output the result. The difference between the second residual module in the decoder and the second residual module in the encoder is that the stride of all convolutional layers of the second residual module in the decoder is 1, and the feature map size is gradually restored by connecting the upsampling module.

[0115] In this embodiment, by introducing the first residual module and the second residual module, the problems of the change of the intermediate layer data distribution and the training degradation during the training process can be solved to prevent the gradient disappearance or explosion and accelerate the training speed.

[0116] In one embodiment, the dilation factor includes a complex feature dilation factor and a simple feature dilation factor.

[0117] Among them, the complex feature dilation factor can be obtained by setting a larger dilation rate. The simple feature dilation factor can be obtained by setting a smaller dilation rate. The complex feature dilation factor will have a larger receptive field.

[0118] As Figure 12 shown, connecting each downsampling module and the first residual module in the encoder of the U-shaped attention network with the dilated convolution includes:

[0119] S1202, connecting the first downsampling module and the first residual module in the encoder of the U-shaped attention network with the dilated convolution corresponding to the complex feature dilation factor.

[0120] Specifically, connect the first downsampling module and the first residual module in the encoder of the U-shaped attention network to the corresponding dilated convolution of the complex feature dilation factor. The first downsampling module can be the downsampling module in the upper layer of the encoder of the U-shaped attention network. In some embodiments, if there are four downsampling modules in the U-shaped attention network, the first downsampling module can be set to the two downsampling modules in the upper layer. It should be noted that those skilled in the art can select and set the first downsampling module according to the architecture of the U-shaped attention network and actual requirements.

[0121] In some embodiments, the complex feature dilation factors can be set to 5, 4, and 3. The first residual module in the upper layer of the U-shaped attention network can be connected to the complex feature dilation factor set to 5. The first downsampling module in the first layer of the upper layer of the U-shaped attention network can be connected to the complex feature dilation factor set to 4. The second downsampling module in the second layer of the upper layer of the U-shaped attention network can be connected to the complex feature dilation factor set to 3.

[0122] S1204. Obtain the complex hierarchical feature information of the cervical cell image through the dilated convolution of the complex feature dilation factor.

[0123] Specifically, capture the global context semantic information in the cervical cell image through the dilated convolution of the complex feature dilation factor. The global context information can be the complex hierarchical feature information.

[0124] S1206. Connect the second downsampling module in the encoder of the U-shaped attention network to the dilated convolution corresponding to the simple feature dilation factor.

[0125] Specifically, connect the second downsampling module in the encoder of the U-shaped attention network to the corresponding dilated convolution of the simple feature dilation factor. The second downsampling module can be the downsampling module in the lower layer of the encoder of the U-shaped attention network. In some embodiments, if there are four downsampling modules in the U-shaped attention network, the first downsampling module can be set to the two downsampling modules in the lower layer. It can be the downsampling modules in the third and fourth layers of the U-shaped attention network.

[0126] It should be noted that those skilled in the art can select and set the second downsampling module according to the architecture of the U-shaped attention network and actual requirements.

[0127] In some embodiments, the simple feature dilation factors can be set to 2 and 1. For the lower layer of the U-shaped attention network, which can be the downsampling module in the third layer of the U-shaped attention network, connect it to the simple feature dilation factor set to 2. Connect the downsampling module in the fourth layer of the U-shaped attention network to the complex feature dilation factor set to 1.

[0128] S1208, obtain the simple hierarchical feature information of the cervical cell image through the dilated convolution of the simple feature dilation factor.

[0129] Specifically, the local shallow information of the cervical cell image can be obtained through the dilated convolution of the simple feature dilation factor, which can be the relatively simple local detail information in the relatively shallow layer of the cervical cell image.

[0130] S1210, use the complex hierarchical feature information and the simple hierarchical feature information to determine the abnormal cells in the cervical cell image.

[0131] Specifically, by exchanging the information of the complex hierarchical feature information and the simple hierarchical feature information, more feature details in the cervical cell image can be obtained, and then the abnormal cells in the cervical cell image can be determined.

[0132] In this embodiment, through the dilated convolution in the cervical cell segmentation network, dilation factors of different sizes can be set, so as to obtain the complex overall information and detailed local information in the cervical cell image, determine the corresponding cell nucleus in the abnormal cell region, improve the details of the cervical cell image and the segmentation effect of the cell nucleus edge in the cervical cell image, and can effectively process abnormal cells of different shapes and sizes. Moreover, by using the structure of adding the simple feature dilation factor after the complex feature dilation factor, the problem that the spatial connection of adjacent neurons becomes weak and the ability of the deep network to extract local structure information is reduced caused by using multiple dilated convolutions at the same time can be solved, preventing the loss of local information in the cervical cell image and the irrelevance of the long-distance information obtained. And after cascading the dilated convolution to the residual module of the encoder, the multi-scale features are combined with a larger receptive field at the same level, effectively solving the problem that the receptive field of the convolution kernel in the network is limited and cannot meet the feature extraction requirements of abnormal cell nuclei of various shapes and sizes.

[0133] In one embodiment, the method further includes: inputting the cervical cell image set into the cervical cell segmentation network, and adjusting and training the cervical cell image set of the cervical cell segmentation network through a preset combined loss function, where the combined loss function is obtained by combining a focal loss function and a difference loss function. Then, a cervical cell segmentation network trained after adjusting the cervical cell image set through the combined loss function is obtained.

[0134] Among them, the focal loss function can be focal loss, which is usually used to solve the extreme imbalance between the foreground class and the background class in the training stage of object detection, and the difference loss function can be dice loss, which is usually more suitable for the situation of extremely uneven samples.

[0135] The calculation formula of the combined loss function includes:

[0136] Loss=wdice L dice + w focal L focal

[0137] Among them, w dice is the weight of the difference loss function, and L dice is the difference loss function; w focal is the weight of the focal loss function, and L focal is the focal loss function, and Loss is the combined loss function.

[0138] Among them, those skilled in the art can selectively adjust and / or set the weights of the difference loss function and the focal loss function according to actual test results or operation experience, so as to better solve the problem of extreme imbalance between the foreground class and the background class in the training stage of object detection.

[0139] In this embodiment, since in cervical cell images, the part of the cell nucleus often accounts for a very small proportion, it is extremely vulnerable to the imbalance of training samples in cervical cell images, resulting in slow convergence and unstable training of the network during the training process. The combined loss function combines the focal loss and the dice loss, and strengthens the attention to difficult samples through the focal loss and the dice loss respectively, which can solve the problems of sample imbalance, slow convergence and unstable training during the training process.

[0140] In one embodiment, before inputting the cervical cell image set into the cervical cell segmentation network, it further includes:

[0141] Augment the cervical cell images to obtain a cervical cell image set. The methods of image augmentation may include: randomly rotating according to a preset random rotation angle range, randomly translating in the horizontal and vertical directions according to a preset translation range, randomly scaling according to a preset scaling range, randomly flipping according to a preset flipping range, etc.

[0142] Among them, the preset random rotation angle range, the preset translation range, the preset scaling range, and the preset flipping range can all be selected and set by those skilled in the art according to the actual situation, and are not limited in this embodiment. It should be noted that the methods of image augmentation are only exemplified by the above several methods in this embodiment, but there are many other ways in the actual process of augmenting cervical cell images, such as image cropping, image interpolation, adding noise, performing contrast transformation, etc.

[0143] In this embodiment, since the medical image data (cervical cell images) is limited, overfitting is likely to occur when training the cervical cell segmentation network. Therefore, it is necessary to augment the cervical cell image set through image augmentation methods.

[0144] In another embodiment, as Figure 13 shown, the present disclosure also provides a method for training a cervical cell segmentation network, the method comprising:

[0145] S1301, performing image augmentation on cervical cell images to obtain a set of cervical cell images.

[0146] S1302, adding a first residual module to the encoder of the U-shaped network and replacing the downsampling module in the encoder with a second residual module.

[0147] S1303, adding a second residual module connected to each upsampling module in the decoder to the decoder of the U-shaped network.

[0148] S1304, adding an attention module to the connection structure between the encoder and the decoder of the U-shaped network to obtain a U-shaped attention network.

[0149] S1305, connecting each downsampling module and the first residual module in the encoder of the U-shaped attention network to dilated convolution to obtain a cervical cell segmentation network.

[0150] S1306, inputting the set of cervical cell images into the cervical cell segmentation network, and adjusting and training the set of cervical cell images of the cervical cell segmentation network through a preset combined loss function to obtain a final cervical cell segmentation network.

[0151] In one embodiment, the present disclosure also provides a cervical cell segmentation method, the method comprising the following steps:

[0152] Obtaining a cervical cell image to be segmented;

[0153] Inputting the cervical cell image to be segmented into the cervical cell segmentation network trained in the above embodiment to segment the cervical cell image, and obtaining a nuclear image of the cervical cells in the cervical cell image to be segmented.

[0154] For the specific implementation manner of the cervical cell segmentation method, reference may be made to the embodiment of the cervical cell segmentation network training method in the above text, which will not be elaborated herein.

[0155] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in rotation with at least a part of other steps or steps or stages in other steps.

[0156] Based on the same inventive concept, an embodiment of the present disclosure further provides a cervical cell segmentation network training device for implementing the above-mentioned cervical cell segmentation network training method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations and implementation manners in one or more embodiments of the following cervical cell segmentation network training device can refer to the embodiments of the cervical cell segmentation network training method in the above text, and will not be repeated here.

[0157] In one embodiment, as Figure 14 shown, a cervical cell segmentation network training device 1400 is provided, including: a residual addition module 1402, an attention addition module 1404, and a dilated convolution connection module 1406, where:

[0158] The residual addition module 1402 is configured to add a first residual module to the encoder of the U-shaped network, replace the downsampling module in the encoder with a second residual module, and add a second residual module connected to each upsampling module in the decoder to the decoder of the U-shaped network. The first residual module and the second residual module are used to increase the training speed.

[0159] The attention addition module 1404 is configured to add an attention module to the connection structure between the encoder and the decoder of the U-shaped network to obtain a U-shaped attention network. The U-shaped attention network is used to determine the nucleus region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module.

[0160] The dilated convolution connection module 1406 is used to connect each downsampling module and the first residual module in the encoder of the U-shaped attention network through dilated convolution, so as to obtain a cervical cell segmentation network. The cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex hierarchical feature information and simple hierarchical feature information in the cervical cell image obtained through the dilation factor of the dilated convolution.

[0161] In an embodiment of the device, the attention module includes a channel attention module and a spatial attention module. The attention addition module 1404 includes: an image input module, a feature channel module, a spatial region module, and a cell nucleus region determination module, where:

[0162] The image input module is used to input the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the corresponding downsampling module in the previous layer of the encoder corresponding to the upsampling module in the decoder into the attention module.

[0163] The feature channel module is used to obtain the importance of each feature channel in the cervical cell image through the channel attention module.

[0164] The spatial region module is used to obtain the importance of each spatial region in the cervical cell image through the spatial attention module.

[0165] The cell nucleus region determination module is used to determine the cell nucleus region in the cervical cell image according to the importance of each feature channel in the cervical cell image obtained through the channel attention module and the importance of each spatial region obtained through the spatial attention module.

[0166] In an embodiment of the device, the feature channel module includes: a weight calculation module and a feature importance calculation module, where:

[0167] The weight calculation module is used to input the cervical cell image into the global average pooling layer, the first fully connected layer, the rectified linear activation function layer, the second fully connected layer, and the sigmoid activation function layer of the channel attention module in sequence, and compress the parameters of the cervical cell image to obtain the weights of each feature channel.

[0168] The feature importance calculation module is used to multiply the weights of each feature channel by the corresponding feature channels in the cervical cell image to obtain the importance of each feature channel.

[0169] In an embodiment of the device, the spatial region module includes: a dimension conversion module, a normalization module, and a spatial importance calculation module, where:

[0170] A dimension conversion module is used to input the cervical cell image of the upsampling module in the decoder and the cervical cell image of the downsampling module in the previous layer encoder corresponding to the upsampling module in the decoder into two identical dimension conversion convolutions in the spatial attention module respectively, and convert the cervical cell images into the same dimension.

[0171] A normalization module is used to perform a normalization operation on the cervical cell images of the same dimension after being converted by the dimension conversion convolution through a linear rectification activation function, a dimension conversion convolution, and an S-shaped activation function in the spatial attention module to obtain an attention coefficient map.

[0172] A spatial importance calculation module is used to multiply the attention coefficient map by the cervical cell image to obtain the importance of each spatial region in the cervical cell image.

[0173] In an embodiment of the device, the channel attention module and the spatial attention module in the attention module are connected in parallel or in sequence.

[0174] In an embodiment of the device, the first residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, two batch normalization layers, and an activation function layer; the second residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, three batch normalization layers, and two activation function layers.

[0175] In an embodiment of the device, the dilated convolution connection module 1406 includes: a first dilated convolution connection module, a complex information acquisition module, a second dilated convolution connection module, a simple information acquisition module, and an abnormal cell determination module, where:

[0176] The first dilated convolution connection module is used to connect the first residual module and the first downsampling module in the encoder of the U-shaped attention network to the corresponding dilated convolution of the complex feature dilation factor.

[0177] The complex information acquisition module is used to obtain the complex hierarchical feature information of the cervical cell image through the dilated convolution of the complex feature dilation factor.

[0178] The second dilated convolution connection module is used to connect the second downsampling module in the encoder of the U-shaped attention network to the dilated convolution corresponding to the simple feature dilation factor.

[0179] The simple information acquisition module is used to obtain the simple hierarchical feature information of the cervical cell image through the dilated convolution of the simple feature dilation factor.

[0180] An abnormal cell determination module, configured to determine abnormal cells in the cervical cell image by using the complex hierarchical feature information and the simple hierarchical feature information.

[0181] In one embodiment of the apparatus, the apparatus further includes: an image set input module, configured to input a cervical cell image set into the cervical cell segmentation network.

[0182] A combined loss function adjustment module, configured to adjust the cervical cell image set for training the cervical cell segmentation network by a preset combined loss function, where the combined loss function is obtained by combining a focal loss function and a difference loss function.

[0183] In one embodiment of the apparatus, the calculation formula of the combined loss function includes:

[0184] Loss = w dice L dice + w focal L focal

[0185] where w dice is the weight of the difference loss function, and L dice is the difference loss function; w focal is the weight of the focal loss function, and L focal is the focal loss function.

[0186] In one embodiment of the apparatus, the apparatus further includes: an image augmentation module, configured to perform image augmentation on a cervical cell image to obtain a cervical cell image set, where the method of image augmentation includes: performing random rotation according to a preset random rotation angle range, performing random translation in the horizontal and vertical directions according to a preset translation range, performing random scaling according to a preset scaling range, and performing random flipping according to a preset flipping range.

[0187] The present disclosure further provides a cervical cell segmentation apparatus, where the apparatus includes:

[0188] A cervical cell image acquisition module, configured to acquire a cervical cell image to be segmented;

[0189] A cervical cell image segmentation module, configured to segment the cervical cell image to be segmented in the cervical cell segmentation network trained by the above method embodiments to obtain a nucleus image of the cervical cells in the cervical cell image to be segmented.

[0190] Each module in the above-mentioned cervical cell segmentation network training device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0191] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 15 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a cervical cell segmentation network training method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0192] Those skilled in the art can understand that Figure 15 the structure shown in

[0193] is only a block diagram of some structures related to the solution of the present disclosure, and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0194] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in each of the above method embodiments.

[0195] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps in each of the above method embodiments.

[0196] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided by the present disclosure can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided by the present disclosure can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0197] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0198] The above-described embodiments merely represent several implementation manners of the present disclosure. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present disclosure. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present disclosure, several modifications and improvements can still be made, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the appended claims.

Claims

1. A method for training a cervical cell segmentation network, characterized in that, The method includes: Adding a first residual module to the encoder of the U-shaped network, replacing the downsampling module in the encoder with a second residual module, and adding a second residual module connected to each upsampling module in the decoder of the U-shaped network. The first residual module and the second residual module are used to increase the training speed; Adding an attention module to the connection structure between the encoder and the decoder of the U-shaped network to obtain a U-shaped attention network, which is used to determine the nuclear region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module; Connecting each second residual module and the first residual module in the encoder of the U-shaped attention network with dilated convolution to obtain a cervical cell segmentation network, which is used to determine the abnormal cell region in the cervical cell image according to the complex-level feature information and simple-level feature information in the cervical cell image obtained through the dilation factor of the dilated convolution.

2. The cervical cell segmentation network training method according to claim 1, wherein The attention module includes a channel attention module and a spatial attention module. Adding an attention module to the connection structure between the encoder and the decoder of the U-shaped network includes: Inputting the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the second residual module in the corresponding upper-layer encoder of the upsampling module in the decoder into the attention module; Obtaining the importance of each feature channel in the cervical cell image through the channel attention module; Obtaining the importance of each spatial region in the cervical cell image through the spatial attention module; Determining the nuclear region in the cervical cell image according to the importance of each feature channel in the cervical cell image obtained through the channel attention module and the importance of each spatial region obtained through the spatial attention module.

3. The cervical cell segmentation network training method according to claim 2, characterized in that The obtaining the importance of each feature channel in the cervical cell image through the channel attention module includes: Sequentially inputting the cervical cell image into the global average pooling layer, the first fully connected layer, the rectified linear activation function layer, the second fully connected layer, and the sigmoid activation function layer of the channel attention module, and compressing the parameters of the cervical cell image to obtain the weights of each feature channel; Multiplying the weights of each feature channel by the corresponding feature channels in the cervical cell image to obtain the importance of each feature channel.

4. The cervical cell segmentation network training method according to claim 2, wherein The obtaining the importance of each spatial region in the cervical cell image through the spatial attention module includes: Inputting the cervical cell image input to the upsampling module in the decoder and the cervical cell image of the second residual module in the corresponding upper-layer encoder of the upsampling module in the decoder into two identical dimension transformation convolutions in the spatial attention module respectively, and converting the cervical cell image into the same dimension; The cervical cell images with the same dimension after being transformed by the dimension transformation convolution are normalized through the linear rectification activation function, dimension transformation convolution, and sigmoid activation function in the spatial attention module to obtain an attention coefficient map; Multiply the attention coefficient map with the cervical cell images to obtain the importance of each spatial region in the cervical cell images.

5. The method for training a cervical cell segmentation network according to any one of claims 2-4, characterized in that, In the attention module, the channel attention module and the spatial attention module are connected in parallel or in sequence.

6. The cervical cell segmentation network training method according to claim 2, wherein The first residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, two batch normalization layers, and an activation function layer; the second residual module includes: two first-scale convolutional layers, a second-scale convolutional layer, three batch normalization layers, and two activation function layers.

7. The cervical cell segmentation network training method according to claim 1, wherein The method further includes: Input a cervical cell image set into the cervical cell segmentation network, and adjust and train the cervical cell image set of the cervical cell segmentation network through a preset combined loss function, where the combined loss function is obtained by combining a focal loss function and a difference loss function.

8. The cervical cell segmentation network training method according to claim 7, characterized in that The calculation formula of the combined loss function includes: Loss=w dice L dice +w focal L focal Among them, w dice is the weight of the difference loss function, and L dice is the difference loss function; w focal is the weight of the focal loss function, and L focal is the focal loss function.

9. The cervical cell segmentation network training method according to claim 7, wherein, Before inputting the cervical cell image set into the cervical cell segmentation network, it further includes: Perform image augmentation on the cervical cell images to obtain a cervical cell image set, and the method of image augmentation includes: randomly rotating according to a preset random rotation angle range, randomly translating in the horizontal and vertical directions according to a preset translation range, randomly scaling according to a preset scaling range, and randomly flipping according to a preset flipping range.

10. A method for cervical cell segmentation, characterized in that, The method includes: Obtain a cervical cell image to be segmented; Input the cervical cell image to be segmented into the cervical cell segmentation network trained according to any one of claims 1 to 9 to segment the cervical cell image, and obtain a nucleus image of the cervical cells in the cervical cell image to be segmented.

11. A training device for a cervical cell segmentation network, characterized in that, The device includes: A residual addition module, configured to add a first residual module to the encoder of the U-shaped network, replace the downsampling module in the encoder with a second residual module, and add a second residual module connected to each upsampling module in the decoder to the decoder of the U-shaped network, where the first residual module and the second residual module are used to increase the training speed; An attention addition module, configured to add an attention module to the connection structure of the encoder and decoder of the U-shaped network to obtain a U-shaped attention network, where the U-shaped attention network is used to determine the nucleus region in the cervical cell image according to the importance of the cervical cell feature channels and the importance of the spatial regions obtained through the attention module; A dilated convolution connection module, configured to connect each second residual module and the first residual module in the encoder of the U-shaped attention network with dilated convolution to obtain a cervical cell segmentation network, where the cervical cell segmentation network is used to determine the abnormal cell region in the cervical cell image according to the complex hierarchical feature information and simple hierarchical feature information in the cervical cell image obtained through the dilation factor of the dilated convolution.

12. A cervical cell segmentation device, characterized in that, The device includes: The cervical cell image acquisition module is used to acquire cervical cell images to be segmented; The cervical cell image segmentation module is used to input the cervical cell images to be segmented into the cervical cell segmentation network trained according to any one of claims 1 to 9 to segment the cervical cell images, so as to obtain the cell nuclei of cervical cells in the cervical cell images to be segmented.

13. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 9.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 9.

Images