Deep learning based automatic classification and segmentation method for squamous epithelial tumor cell pictures

Abstract

The application discloses a kind of based on deep learning squamous epithelial tumor cell picture automatic classification segmentation method.Method includes: obtaining enhanced section picture set;Training image set is constructed;Domain adaptive nucleus multi-classification segmentation network is constructed;Training image set is input into nucleus classification segmentation network and carries out first round training, obtains pre-training domain adaptive nucleus multi-classification segmentation network;Enhanced section picture set and training image set are input in second round training, and domain adaptive nucleus multi-classification segmentation network is trained;To be classified and segmented squamous epithelial cell tumor section picture is input into training completed domain adaptive nucleus multi-classification segmentation network and handles, realizes to the automatic classification segmentation of to be classified and segmented squamous epithelial cell tumor section picture.The method of the application can effectively extract squamous epithelial tumor cell image features, realize the automatic classification segmentation of the nucleus of each cell in image, improve the recognition accuracy of cell in image.

Description

TECHNICAL FIELD

[0001] The application relates to an automatic classification and segmentation method, in particular to an automatic classification and segmentation method for squamous epithelial tumor cell pictures based on deep learning. BACKGROUND

[0002] Tumor cells and their differentiation, inflammatory cell infiltration and other unique characteristics can be observed in pathological sections of squamous epithelial tumors, and the occurrence, growth and metastasis of tumors are closely related to the internal and external environment of tumor cells. Meanwhile, the tumor microenvironment also affects the prognosis of the tumor. The reading of tumor pathological sections and the segmentation of cell nuclei are helpful for the classification and statistics of tumor cells, inflammatory cells and keratinization, interbridge and nuclear division images, and have important guiding significance for the diagnosis of squamous epithelial tumors, tumor staging and typing and the selection of treatment plans.

[0003] Traditional pathological section observation mainly relies on manual reading. This method is time-consuming and laborious, and is highly subjective, and often requires experienced pathologists to make correct judgments. In recent years, computer-aided pathology has emerged, aiming to provide fast, repeatable and quantitative judgments. With the continuous development of deep learning technology, many attempts have been made to study the application of deep learning technology in the automatic detection of cancer in digital histology images. At present, some deep learning technologies only distinguish the benign and malignant cells by segmenting the cell nuclei. Due to the large area of the whole pathological section, some deep learning technologies only read a part of the whole section for cell classification and recognition. Due to the anatomical diversity of histological sections, some areas may still be misclassified, especially the areas near the normal edge of the tumor and the areas with few cell nuclei. SUMMARY

[0004] In order to solve the problems in the background art, the application provides an automatic classification and segmentation method for squamous epithelial tumor cell pictures based on deep learning.

[0005] The technical scheme adopted by the application is:

[0006] The automatic classification and segmentation method of the application comprises the following steps:

[0007] Step 1) Collect a plurality of squamous epithelial tumor section pictures, uniformly segment the squamous epithelial tumor section pictures after pretreatment and labeling, obtain the labeled squamous epithelial tumor section pictures, and construct a labeled section picture set; and perform data enhancement processing on the labeled section picture set to obtain an enhanced section picture set.

[0008] Step 2) Obtain a plurality of squamous epithelial tumor slice pictures and their masks in the CoNSeP dataset, and construct a training image set after the same data enhancement processing in step 1) is performed on each squamous epithelial tumor slice picture and its mask in the obtained CoNSeP dataset.

[0009] Step 3) Construct a domain adaptive cell nucleus multi-classification segmentation network, which includes a cell nucleus classification and segmentation network and two domain adaptive networks DANet, the cell nucleus classification and segmentation network includes an input layer, a classification branch, a segmentation branch and a regression branch, the input layer is connected to the classification branch, the segmentation branch and the regression branch respectively, and the classification branch and the segmentation branch are connected to the two domain adaptive networks DANet.

[0010] Step 4) Input the training image set into the cell nucleus classification and segmentation network of the domain adaptive cell nucleus multi-classification segmentation network for first round training, obtain the loss value of the cell nucleus classification and segmentation network in the first round training, propagate the overall loss value to the cell nucleus classification and segmentation network by the back propagation method, update the network parameters of the cell nucleus classification and segmentation network by the gradient descent method, optimize the cell nucleus classification and segmentation network by the ADAM optimizer, and finally obtain a pre-trained domain adaptive cell nucleus multi-classification segmentation network.

[0011] Step 5) Input the enhanced slice picture set and the training image set into the pre-trained domain adaptive cell nucleus multi-classification segmentation network for second round training, calculate the binary cross-entropy loss of the output results of the two domain adaptive networks DANet in the second round training, optimize the pre-trained domain adaptive cell nucleus multi-classification segmentation network by the ADAM optimizer until the binary cross-entropy loss is less than a preset value to complete the second round training, and obtain a trained domain adaptive cell nucleus multi-classification segmentation network.

[0012] Step 6) Obtain a squamous epithelial tumor slice picture to be classified and segmented, input the squamous epithelial tumor slice picture to be classified and segmented into the trained domain adaptive cell nucleus multi-classification segmentation network for processing, output the segmentation results and classes of the cell nuclei of each cell in the squamous epithelial tumor slice picture to be classified and segmented after the processing, and realize automatic classification and segmentation of the squamous epithelial tumor slice picture to be classified and segmented.

[0013] In step 1), each squamous epithelial tumor section picture is preprocessed, labeled, and divided into a preset size, specifically, the nucleus of each cell in each squamous epithelial tumor section picture is labeled in terms of category and contour, and each labeled squamous epithelial tumor section picture is uniformly cut into a plurality of square labeled squamous epithelial tumor section pictures of the same size; if the area of the non-nucleus region in the labeled squamous epithelial tumor section picture exceeds a preset value, the labeled squamous epithelial tumor section picture is excluded.

[0014] The cell category of each cell in the squamous epithelial tumor section picture is specifically epithelial cells, spindle cells, inflammatory cells, and other cells.

[0015] In step 1), the labeled section picture set is subjected to data enhancement processing, specifically, each labeled squamous epithelial tumor section picture in the labeled section picture set is subjected to data enhancement processing, the data enhancement processing includes affine transformation, random cropping size and scaling to the original size, horizontal and vertical flipping, Gaussian blur, and contrast enhancement processing, each labeled squamous epithelial tumor section picture obtains a plurality of enhanced squamous epithelial tumor section pictures after being subjected to various data enhancement processing, and each enhanced squamous epithelial tumor section picture obtained by the labeled section picture set constitutes an enhanced section picture set.

[0016] In step 3), the input layer of the nucleus classification and segmentation network of the domain adaptive nucleus multi-classification and segmentation network includes a first 7x7 convolution, three first residual modules, four second residual modules, six third residual modules, three fourth residual modules, and a first 1x1 convolution connected in sequence; the structures of the classification branch, the segmentation branch, and the regression branch of the nucleus classification and segmentation network are the same, and include a first up-sampling module, a first 5x5 convolution, eight first dense modules, a second 1x1 convolution, a second up-sampling module, a second 5x5 convolution, four second dense modules, a third 1x1 convolution, a third up-sampling module, a third 5x5 convolution, and a fourth 1x1 convolution connected in sequence; the outputs of the first 1x1 convolution of the input layer of the nucleus classification and segmentation network are respectively input to the first up-sampling module of the classification branch, the segmentation branch, and the regression branch; the outputs of the fourth 1x1 convolution of the classification branch and the segmentation branch are respectively input to the two domain adaptive networks DANet.

[0017] The structures of the two domain adaptive networks DANet are the same, and the domain adaptive network DANet includes a second 7x7 convolution, a fifth residual module, a sixth residual module, a seventh residual module, and an eighth residual module connected in sequence, and the outputs of the fifth residual module, the sixth residual module, the seventh residual module, and the eighth residual module are subjected to splicing processing and then output as the output of the domain adaptive network DANet.

[0018] In the step 5), the binary cross-entropy loss of the output results of the two domain adaptive networks DANet is calculated in the second round of training, specifically, for each domain adaptive network DANet, the binary cross-entropy loss between the output results of the enhanced slice picture set processed by the domain adaptive cell nucleus multi-classification segmentation network in the domain adaptive network DANet and the output results of the training image set processed by the domain adaptive cell nucleus multi-classification segmentation network in the domain adaptive network DANet is calculated by the multi-scale loss function, until the binary cross-entropy loss is less than the preset value to complete the second round of training. Each domain adaptive network DANet can determine whether the input corresponding to the output result belongs to the enhanced slice picture set or the training image set, that is, whether it belongs to the target domain or the source domain.

[0019] In the step 6), the segmentation results and categories of each cell in the processed output to-be-classified and segmented squamous epithelial cell tumor slice picture are output, specifically, a domain adaptive network DANet connected by the classification branch of the nucleus classification segmentation network of the domain adaptive cell nucleus multi-classification segmentation network outputs the classification image of the to-be-classified and segmented squamous epithelial cell tumor slice picture, the cell categories of each cell are marked in the classification image, and in specific implementation, the cell categories of each cell are distinguished by the color of the cell nucleus in the classification image; a domain adaptive network DANet connected by the segmentation branch outputs the segmentation image of the to-be-classified and segmented squamous epithelial cell tumor slice picture, the pixel point positions of all cell nuclei are marked in the segmentation image, in specific implementation, the pixel point positions of all cell nuclei in the segmentation image are judged as 1, and the non-cell nucleus positions are judged as 0, that is, a gray image is formed; the regression image of the to-be-classified and segmented squamous epithelial cell tumor slice picture is output by the regression branch, for each cell nucleus, the regression image includes a horizontal distance map and a vertical distance map, the horizontal and vertical distances of the pixel point positions of the cell nucleus from the cell nucleus center are marked in the horizontal distance map and the vertical distance map respectively, the differentiation of each cell nucleus is realized, and automatic classification and segmentation of the to-be-classified and segmented squamous epithelial cell tumor slice picture is realized according to the output classification image, segmentation image and regression image.

[0020] The beneficial effects of the present application are:

[0021] 1) The method of the present application divides the task into cell nucleus classification and segmentation and domain adaptation, solves the domain bias problem caused by pathological images of different diseases, and adopts a domain adaptive network that can effectively capture the semantic information difference between different domains, learn domain-invariant features, and achieve self-adaptation between different domains.

[0022] 2) The method can effectively extract the image features of squamous epithelial tumor cells, realize the automatic classification and segmentation of the cell nuclei in the image, effectively reduce the workload of cell nucleus labeling, and improve the recognition accuracy of each cell in the image. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The figure is a schematic diagram of the domain adaptive cell nucleus multi-classification segmentation network structure of the application.

[0024] Figure 2 The figure is a schematic diagram of the domain adaptive network DANet structure of the application.

[0025] Figure 3 The figure is a schematic diagram of the squamous epithelial cell tumor slice picture to be classified and segmented of the application.

[0026] Figure 4 (a) of the figure is the classification image output by the domain adaptive network DANet of the application.

[0027] Figure 4 (b) of the figure is the segmentation image output by the domain adaptive network DANet of the application.

[0028] Figure 5 (a) of the figure is the horizontal distance map output by the regression branch of the application.

[0029] Figure 5 (b) of the figure is the vertical distance map output by the regression branch of the application. DETAILED DESCRIPTION

[0030] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0031] The automatic classification and segmentation method of the application includes the following steps:

[0032] Step 1) Collect several squamous epithelial tumor slice pictures, uniformly segment the preprocessed and labeled squamous epithelial tumor slice pictures, obtain each labeled squamous epithelial tumor slice picture, and construct a labeled slice picture set; perform data enhancement processing on the labeled slice picture set to obtain an enhanced slice picture set.

[0033] In step 1), each squamous epithelial tumor section picture is preprocessed, labeled, and uniformly cut into a plurality of square labeled squamous epithelial tumor section pictures of the same size. The cell nucleus of each cell in each squamous epithelial tumor section picture is labeled in terms of category and contour. If the area of a non-nucleus region in a labeled squamous epithelial tumor section picture exceeds a preset value, the labeled squamous epithelial tumor section picture is excluded. The cell category of each cell in the squamous epithelial tumor section picture is specifically epithelial cells, spindle cells, inflammatory cells, and other cells.

[0034] In step 1), the labeled section picture set is subjected to data augmentation processing. Specifically, each labeled squamous epithelial tumor section picture in the labeled section picture set is subjected to data augmentation processing. The data augmentation processing includes affine transformation, random cropping size and scaling to the original size, horizontal and vertical flipping, Gaussian blur, and contrast enhancement processing. Each labeled squamous epithelial tumor section picture obtains a plurality of augmented squamous epithelial tumor section pictures after being subjected to various data augmentation processing. The augmented squamous epithelial tumor section pictures obtained from the labeled section picture set constitute an augmented section picture set.

[0035] In step 2), a plurality of squamous epithelial tumor section pictures and their masks in the CoNSeP dataset are obtained. Each squamous epithelial tumor section picture and its mask in the CoNSeP dataset are subjected to the same data augmentation processing in step 1) to construct a training image set.

[0036] In step 3), a domain adaptive cell nucleus multi-classification segmentation network is constructed. The domain adaptive cell nucleus multi-classification segmentation network includes a cell nucleus classification and segmentation network and two domain adaptive networks DANet. The cell nucleus classification and segmentation network includes an input layer, a classification branch, a segmentation branch, and a regression branch. The input layer is connected to the classification branch, the segmentation branch, and the regression branch, respectively. The classification branch and the segmentation branch are connected to the two domain adaptive networks DANet.

[0037] In step 3), as shown in FIG. 6, the domain adaptive cell nucleus multi-classification segmentation network includes a cell nucleus classification and segmentation network and two domain adaptive networks DANet. Figure 1As shown, the input layer of the nucleus classification and segmentation network of the domain adaptive nucleus multi-classification and segmentation network comprises a first 7x7 convolution, three first residual modules, four second residual modules, six third residual modules, three fourth residual modules and a first 1x1 convolution connected in sequence; the structures of the classification branch, the segmentation branch and the regression branch of the nucleus classification and segmentation network are all the same, comprising a first up-sampling module, a first 5x5 convolution, eight first dense modules, a second 1x1 convolution, a second up-sampling module, a second 5x5 convolution, four second dense modules, a third 1x1 convolution, a third up-sampling module, a third 5x5 convolution and a fourth 1x1 convolution connected in sequence; the outputs of the first 1x1 convolution of the input layer of the nucleus classification and segmentation network are respectively taken as the inputs of the first up-sampling module of the classification branch, the segmentation branch and the regression branch; the outputs of the fourth 1x1 convolution of the classification branch and the segmentation branch are respectively taken as the inputs of the two domain adaptive networks DANet.

[0038] As shown in Figure 2 The structures of the two domain adaptive networks DANet are all the same, and the domain adaptive network DANet comprises a second 7x7 convolution, a fifth residual module, a sixth residual module, a seventh residual module and an eighth residual module connected in sequence, and the outputs of the fifth residual module, the sixth residual module, the seventh residual module and the eighth residual module are subjected to splicing processing and then taken as the output of the domain adaptive network DANet.

[0039] Step 4) inputting the training image set into the nucleus classification and segmentation network of the domain adaptive nucleus multi-classification and segmentation network for first round training, obtaining the loss value of the nucleus classification and segmentation network in the first round training, reversely propagating the overall loss value to the nucleus classification and segmentation network by the back propagation method, updating the network parameters of the nucleus classification and segmentation network by the gradient descent method, simultaneously optimizing the nucleus classification and segmentation network by the ADAM optimizer, and finally obtaining the pre-trained domain adaptive nucleus multi-classification and segmentation network.

[0040] Step 5) inputting the enhanced slice image set and the training image set into the pre-trained domain adaptive nucleus multi-classification and segmentation network for second round training, calculating the binary cross-entropy loss of the output results of the two domain adaptive networks DANet in the second round training, simultaneously optimizing the pre-trained domain adaptive nucleus multi-classification and segmentation network by the ADAM optimizer, until the binary cross-entropy loss is less than a preset value, completing the second round training, and obtaining the trained domain adaptive nucleus multi-classification and segmentation network.

[0041] In step 5), the binary cross-entropy loss of the output results of the two domain adaptation networks DANet in the second round of training is calculated, specifically, for each domain adaptation network DANet, the binary cross-entropy loss between the output results of the enhanced slice image set processed by the domain adaptation cell nucleus multi-classification segmentation network in the domain adaptation network DANet and the output results of the training image set processed by the domain adaptation cell nucleus multi-classification segmentation network in the domain adaptation network DANet is calculated by the multi-scale loss function, until the binary cross-entropy loss is less than the preset value to complete the second round of training. Each domain adaptation network DANet can determine whether the input corresponding to the output result belongs to the enhanced slice image set or the training image set, that is, whether it belongs to the target domain or the source domain.

[0042] In step 6), the slice image of the squamous epithelial cell tumor to be classified and segmented is obtained, and the slice image of the squamous epithelial cell tumor to be classified and segmented is input into the trained domain adaptation cell nucleus multi-classification segmentation network for processing, and the segmentation results and classes of the cell nuclei of each cell in the slice image of the squamous epithelial cell tumor to be classified and segmented are output after processing, so as to realize automatic classification and segmentation of the slice image of the squamous epithelial cell tumor to be classified and segmented.

[0043] In step 6), the segmentation results and classes of each cell in the slice image of the squamous epithelial cell tumor to be classified and segmented are output after processing, specifically, a domain adaptation network DANet connected by the classification branch of the cell nucleus classification and segmentation network of the domain adaptation cell nucleus multi-classification segmentation network outputs a classification image of the slice image of the squamous epithelial cell tumor to be classified and segmented, and the cell classes of each cell are marked in the classification image, and in specific implementation, the cell classes are distinguished by the color of the cell nucleus in the classification image; a domain adaptation network DANet connected by the segmentation branch outputs a segmentation image of the slice image of the squamous epithelial cell tumor to be classified and segmented, and the pixel point positions of all cell nuclei are marked in the segmentation image, and in specific implementation, the pixel point positions of all cell nuclei in the segmentation image are judged as 1, and the non-cell nucleus positions are judged as 0, that is, a gray image is formed; a regression image of the slice image of the squamous epithelial cell tumor to be classified and segmented is output by the regression branch, and for each cell nucleus, the regression image includes a horizontal distance map and a vertical distance map, and the horizontal and vertical distances of the pixel point positions of the cell nucleus from the cell nucleus center are marked in the horizontal distance map and the vertical distance map, respectively, to realize the differentiation of each cell nucleus, and automatic classification and segmentation of the slice image of the squamous epithelial cell tumor to be classified and segmented is realized according to the output classification image, segmentation image and regression image.

[0044] The specific implementation of the present application is as follows:

[0045] In step 1), the collected several pictures of squamous epithelial tumor sections correspond to the sections of weakly annotated squamous epithelial tumor data, that is, hematoxylin-eosin stained histopathological data; the pre-processing annotation is specifically to annotate the tumor range using ImageScope software, that is, to annotate the nuclei of each cell; since the size of the annotated full field digital section is large, it cannot be directly input into the network for training without processing, therefore, the histopathological section is segmented by using the annotation range, including cutting the section into multiple 1000x1000 size images, if the area of the non-nucleus region in the annotated squamous epithelial tumor section picture exceeds the general area of the image itself, then the annotated squamous epithelial tumor section picture is excluded.

[0046] In step 3), the step length of the second 7x7 convolution in the domain adaptive network DANet is 2, and the padding number is 3.

[0047] In step 4), the first round of training, 50 rounds of training, the loss value of the calculated nucleus classification and segmentation network Specifically as follows:

[0048]

[0049]

[0050]

[0051] Among them, respectively represent the loss values of the classification branch, the segmentation branch and the regression branch of the nucleus classification and segmentation network; λ1, λ2 and λ3 respectively represent the first control weight, the second control weight and the third control weight; represents the cross-entropy loss calculated by the classification branch or the segmentation branch; represents the Dice coefficient loss, which is used to balance the effect of each branch; and respectively represent the prediction result and the true value label of the classification branch or the segmentation branch in the i th round; represents the mean square error of the regression branch; and respectively represent the horizontal distance map and the vertical distance map predicted by the regression branch; the gradient is calculated in the process of back propagation according to the loss value, and Adam is used as the optimizer, the initial learning rate is 1e-3, and the network weight parameters are optimized by continuous iteration.

[0052] In step 6), the obtained squamous epithelial tumor section picture to be classified and segmented is as follows Figure 3As shown, the classified segmented squamous epithelial tumor section picture is input into the trained domain adaptive cell nucleus multi-classification segmentation network for processing, and the segmentation result and class of the cell nucleus of each cell in the classified segmented squamous epithelial tumor section picture are output after processing, such as Figure 4 As shown in (a) and (b) of FIG. 13, the classification image and the segmentation image output by the domain adaptive network DANet are respectively shown; as Figure 5 As shown in (a) and (b) of FIG. 13, the classification image and the segmentation image output by the domain adaptive network DANet are respectively shown; as

[0053] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of improvements and modifications can be made, and these improvements and modifications should be considered within the scope of the present application.

Claims

1. A deep learning-based automatic classification and segmentation method for squamous epithelial tumor cell pictures, characterized by: The method comprises the following steps: Step 1) Collecting a plurality of squamous cell carcinoma slice images, uniformly dividing the squamous cell carcinoma slice images after pretreatment and labeling, obtaining labeled squamous cell carcinoma slice images and constructing a labeled slice image set; The labeled slice image set is subjected to data enhancement processing to obtain an enhanced slice image set; Step 2) Obtaining a plurality of squamous cell carcinoma slice images and masks in the CoNSeP dataset, and constructing a training image set after the same data enhancement processing of the squamous cell carcinoma slice images and masks in the CoNSeP dataset in step 1); Step 3) Constructing a domain adaptive cell nucleus multi-classification segmentation network, the domain adaptive cell nucleus multi-classification segmentation network comprising a cell nucleus classification and segmentation network and two domain adaptive networks DANet, the cell nucleus classification and segmentation network comprising an input layer, a classification branch, a segmentation branch and a regression branch, the input layer being connected to the classification branch, the segmentation branch and the regression branch respectively, and the classification branch and the segmentation branch being connected to the two domain adaptive networks DANet respectively; Step 4) Inputting the training image set into the cell nucleus classification and segmentation network of the domain adaptive cell nucleus multi-classification segmentation network for first round training, obtaining a loss value of the cell nucleus classification and segmentation network in the first round training, reversely propagating the overall loss value to the cell nucleus classification and segmentation network by a back propagation method, updating network parameters of the cell nucleus classification and segmentation network by a gradient descent method, optimizing the cell nucleus classification and segmentation network by an ADAM optimizer and finally obtaining a pre-trained domain adaptive cell nucleus multi-classification segmentation network; Step 5) Inputting the enhanced slice image set and the training image set into the pre-trained domain adaptive cell nucleus multi-classification segmentation network for second round training, calculating a binary cross-entropy loss of output results of the two domain adaptive networks DANet in the second round training, optimizing the pre-trained domain adaptive cell nucleus multi-classification segmentation network by an ADAM optimizer until the binary cross-entropy loss is less than a preset value to complete the second round training and obtain a trained domain adaptive cell nucleus multi-classification segmentation network; Step 6) Obtaining a squamous cell carcinoma slice image to be classified and segmented, inputting the squamous cell carcinoma slice image to be classified and segmented into the trained domain adaptive cell nucleus multi-classification segmentation network for processing, outputting a segmentation result and a class of each cell nucleus in the squamous cell carcinoma slice image to be classified and segmented after the processing and realizing automatic classification and segmentation of the squamous cell carcinoma slice image to be classified and segmented; In step 4), the first round of training, the loss value of the calculated cell nucleus classification segmentation network is trained for several rounds The specific process is as follows: wherein, 、 、 respectively represent the loss values of the classification branch, the segmentation branch and the regression branch of the nucleus classification and segmentation network; 、 and respectively represent the first control weight, the second control weight and the third control weight; represent the cross-entropy loss calculated by the classification branch or the segmentation branch; represent the Dice coefficient loss, which is used to balance the role of each branch; and respectively represent the prediction result and the true value label of the classification branch or the segmentation branch in the i-th round; represent the mean square error of the regression branch; and respectively represent the horizontal distance map and the vertical distance map predicted by the regression branch; the gradients are calculated in the back propagation process according to the loss values, and Adam is used as the optimizer with an initial learning rate of 1e-3, and the network weight parameters are optimized through continuous iteration.

2. The method of claim 1, wherein the method is based on deep learning. In step 1), each cell nucleus in each squamous cell carcinoma slice image is labeled in class and contour, and each labeled squamous cell carcinoma slice image is uniformly cut into a plurality of square labeled squamous cell carcinoma slice images of the same size.

3. The method of claim 1, wherein the method is based on deep learning. The cell class of each cell in the squamous cell carcinoma slice image is specifically epithelial cells, spindle cells, inflammatory cells and other cells.

4. The method of claim 1, wherein the method is based on deep learning. In the step 1), the data augmentation processing is performed on the labeled slice picture set, specifically, each labeled squamous epithelial tumor slice picture in the labeled slice picture set is subjected to data augmentation processing, the data augmentation processing includes affine transformation, random cropping size and scaling to the original size, horizontal and vertical flipping, Gaussian blur and contrast enhancement processing, and each labeled squamous epithelial tumor slice picture obtains a plurality of enhanced squamous epithelial tumor slice pictures after being subjected to various data augmentation processing, and each enhanced squamous epithelial tumor slice picture obtained by the labeled slice picture set constitutes an enhanced slice picture set.

5. The method of claim 1, wherein the method is based on deep learning. In the step 3), the input layer of the nucleus classification and segmentation network of the domain adaptive nucleus multi-classification and segmentation network comprises a first 7*7 convolution, three first residual modules, four second residual modules, six third residual modules, three fourth residual modules and a first 1*1 convolution connected in sequence; the structures of the classification branch, the segmentation branch and the regression branch of the nucleus classification and segmentation network are the same, and comprise a first up-sampling module, a first 5*5 convolution, eight first dense modules, a second 1*1 convolution, a second up-sampling module, a second 5*5 convolution, four second dense modules, a third 1*1 convolution, a third up-sampling module, a third 5*5 convolution and a fourth 1*1 convolution connected in sequence; the outputs of the first 1*1 convolution of the input layer of the nucleus classification and segmentation network are respectively taken as the inputs of the first up-sampling module of the classification branch, the segmentation branch and the regression branch; the outputs of the fourth 1*1 convolution of the classification branch and the segmentation branch are respectively taken as the inputs of the two domain adaptive networks DANet; the structures of the two domain adaptive networks DANet are the same, and the domain adaptive network DANet comprises a second 7*7 convolution, a fifth residual module, a sixth residual module, a seventh residual module and an eighth residual module connected in sequence, and the outputs of the fifth residual module, the sixth residual module, the seventh residual module and the eighth residual module are subjected to splicing processing and then taken as the output of the domain adaptive network DANet.

6. The method of claim 1, wherein the method is based on deep learning. In the step 5), the binary cross-entropy loss of the output results of the two domain adaptive networks DANet is calculated in the second round of training, specifically, for each domain adaptive network DANet, the binary cross-entropy loss between each output result of the enhanced slice picture set processed by the domain adaptive nucleus multi-classification and segmentation network in the domain adaptive network DANet and each output result of the training image set processed by the domain adaptive nucleus multi-classification and segmentation network in the domain adaptive network DANet is calculated through a multi-scale loss function, and the second round of training is completed until the binary cross-entropy loss is less than a preset value.

7. The method of claim 1, wherein the method is based on deep learning. In the step 6), the segmentation result and the category of each cell in the picture of the segmented squamous epithelial cell tumor section to be classified are output after processing, specifically, a classification image of the picture of the segmented squamous epithelial cell tumor section to be classified is output by a domain adaptive network DANet connected with a classification branch of a nucleus classification and segmentation network of the domain adaptive nucleus multi-classification and segmentation network, and the classification image marks the cell category of each cell; a segmentation image of the picture of the segmented squamous epithelial cell tumor section to be classified is output by a domain adaptive network DANet connected with a segmentation branch, and the segmentation image marks the pixel point position of all the cell nuclei; and a regression image of the picture of the segmented squamous epithelial cell tumor section to be classified is output by a regression branch, and the regression image includes a horizontal distance map and a vertical distance map for the cell nucleus of each cell, the horizontal distance map and the vertical distance map mark the horizontal and vertical distances of the pixel point position of the cell nucleus from the nucleus center, the cell nuclei are distinguished, and the automatic classification and segmentation of the picture of the segmented squamous epithelial cell tumor section to be classified are realized according to the output classification image, the segmentation image and the regression image.

Images