A TransUNet-based method for detecting lesion cells in whole-section pathological sections of FCD type II.

Abstract

This invention discloses a method for detecting lesion cells in whole-section pathological slides of FCD type II based on TransUNet, belonging to the field of image processing technology. The method includes the following steps: S1: Segmenting the scanned FCD type II pathological slide images and establishing a database; S2: Constructing a lesion detection model based on TransUNet; S3: Manually selecting the database into training, validation, and test sets, and training the lesion detection model; S4: Segmenting the acquired slide images to be detected and inputting them into the lesion detection model to obtain lesion cell images, and using the OpenCV processing library to restore the lesion cell images back to the slide images to obtain the location of the lesion cells in the slide images. This invention, by segmenting the pathological slide images and establishing a database, can reduce the training load and shorten the detection time when training and using the lesion detection model. Furthermore, the entire detection model has a simple structure, low computational resource consumption, fast model operation speed, and excellent robustness and accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for detecting lesion cells in FCD type II pathological whole slides based on TransUNet. Background Technology

[0002] Focal Cortical Dysplasia (FCD) type II is the most common cause of refractory epilepsy in children. It is a focal developmental brain malformation characterized by disordered cortical stratification, often accompanied by abnormal lesion cells, requiring pathological diagnosis and treatment. Currently, FCD type II is diagnosed based on pathological changes in the malformed neuronal cell structure. Therefore, the whole-section pathological section method of lesion cell segmentation helps to discover specific pathological markers, thereby identifying the pathological changes in FCD type II, which is of great significance for the diagnosis and treatment of the disease.

[0003] Traditionally, the detection of lesion cells relies primarily on manual labeling. However, pathological slides are often large, and manual labeling is prone to errors and incomplete lesion cell counts, leading to low accuracy. Existing technologies utilize neural network models for image segmentation to reduce the difficulty of lesion cell labeling. However, these models require pathologists to manually delineate lesion cell boundaries to construct the cell dataset, which is challenging to obtain. Furthermore, to ensure detection / segmentation accuracy, the network model setup is complex, resulting in a large number of parameters, high demands on computing power, and significant training difficulties. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention provides a TransUNet-based method for detecting lesion cells in FCD type II pathological whole slides, which boasts high accuracy and efficiency in identifying lesion cells.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A method for detecting lesion cells in whole FCD type II pathological sections based on TransUNet is provided, which includes the following steps:

[0007] S1: Segment the FCD type II pathological slide images obtained from the scan and establish a database;

[0008] S2: Constructing a lesion detection model based on TransUNet;

[0009] The lesion detection model consists of a downsampling group, an encoder group, and an upsampling group connected in sequence. The downsampling group consists of three downsampling modules connected in sequence. The encoder group consists of four encoder modules connected in sequence, and each encoder module consists of a multi-head attention mechanism, a feedforward network, and a normalization layer connected in sequence. The upsampling group consists of four upsampling modules connected in sequence.

[0010] The downsampling module is used to extract feature maps from the obtained pathological images and convert the feature maps into semantic sequences;

[0011] The encoder module is used to extract semantic features from the semantic sequence;

[0012] The upsampling module is used to reconstruct the predicted feature map based on semantic features. During the reconstruction process, the upsampling module concatenates the feature map extracted by the downsampling module into the predicted feature map.

[0013] The upsampling module consists of two convolutional layers and one upsampling layer;

[0014] The downsampling module of the lesion detection model has 3 input channels; after the first layer of downsampling, the number of channels is 64, after the second layer of downsampling, the number of channels is 256, after the third layer of downsampling, the number of channels is 512, and after the fourth layer of downsampling, the number of channels is 1024.

[0015] S3: The manually selected database is divided into training, validation, and test sets to train the lesion detection model; the loss function used for training is:

[0016]

[0017] Where C is the number of label categories; N is the number of pixels in the image; y n，c p is the label value of the pixel; n , c The predicted value for a pixel;

[0018] S4: After segmenting the acquired slice image to be detected, input it into the lesion detection model to obtain the lesion cell image, and use the OpenCV processing library to restore the lesion cell image to the slice image to obtain the position of the lesion cell in the slice image.

[0019] Furthermore, step S1 includes the following specific steps:

[0020] A1: Staining specific vimentins in FCD type II pathological slides;

[0021] A2: Cut the FCD type II pathological slide image into several image blocks of 512×512 pixels in order from top to bottom and from left to right; the original image size parameters are: 40000×40000 pixels, 24-bit bit depth, 300dpi horizontal / vertical resolution.

[0022] A3: Use QuPath annotation software to delineate the boundaries of lesion cells in several image blocks to obtain lesion cell boundary maps;

[0023] A4: Binarization of the boundary map of the diseased cells yields a binary image;

[0024] A5: Use the binarized image as the label value, and the image patch and its corresponding binarized image as elements to form a dataset.

[0025] Furthermore, between steps A2 and A3, there is also step A21: discarding the frame blocks in the image that do not contain cellular tissue.

[0026] Furthermore, a downsampling module is set up between the downsampling group and the encoder group. The added downsampling module can extract deeper feature information, while reducing the image size sent to the encoder group and reducing the computational load of the model.

[0027] The beneficial effects of this invention are as follows:

[0028] This invention establishes a database by segmenting pathological slide images. When using the lesion detection model for training and detection, it can reduce the training load and shorten the detection time. Furthermore, the entire detection model has a simple structure, low computational resource consumption, fast model operation speed, and excellent robustness and accuracy.

[0029] Pathological slide images are segmented using high-precision and high-efficiency segmentation methods to establish a database. The segmented images are then used to train a prediction model, greatly reducing the training difficulty. After the model is trained, the slide images to be detected are segmented using the same method and then fed into the prediction model, shortening the overall prediction time and ensuring high accuracy. Finally, by restoring the location of the diseased cells in the slide images, the distribution of the diseased cells can be obtained, helping doctors to more accurately diagnose and treat FCD type II cases, improving diagnostic accuracy and treatment effectiveness. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the process of the present invention;

[0031] Figure 2 A flowchart illustrating the process of establishing a database;

[0032] Figure 3 This is a schematic diagram of the lesion detection model. Detailed Implementation

[0033] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0034] like Figure 1-3 As shown, a method for detecting lesion cells in whole FCD type II pathological sections based on TransUNet includes the following steps:

[0035] S1: Segment the FCD type II pathological slide images obtained from the scan and establish a database;

[0036] S2: Constructing a lesion detection model based on TransUNet;

[0037] The lesion detection model consists of a downsampling group, an encoder group, and an upsampling group connected in sequence. The downsampling group consists of three downsampling modules connected in sequence. The encoder group consists of four encoder modules (visiontransformer) connected in sequence. Each encoder module consists of a multi-head attention mechanism, a feedforward network, and a normalization layer connected in sequence. The upsampling group consists of four upsampling modules connected in sequence. A downsampling module is set between the downsampling group and the encoder group.

[0038] The downsampling module is used to extract feature maps from the obtained pathological images and convert the feature maps into semantic sequences;

[0039] The encoder module is used to extract semantic features from the semantic sequence;

[0040] The upsampling module is used to reconstruct the predicted feature map based on semantic features. During the reconstruction process, the upsampling module concatenates the feature map extracted by the downsampling module into the predicted feature map. Using the texture features from the downsampling in the upsampling can effectively improve the accuracy and robustness of the segmentation.

[0041] The upsampling module consists of two convolutional layers and one upsampling layer;

[0042] The downsampling module of the lesion detection model has 3 input channels; after the first layer of downsampling, the number of channels is 64, after the second layer of downsampling, the number of channels is 256, after the third layer of downsampling, the number of channels is 512, and after the fourth layer of downsampling, the number of channels is 1024.

[0043] S3: The manually selected database is divided into training, validation, and test sets to train the lesion detection model; the loss function used for training is:

[0044]

[0045] Where C is the number of label categories; N is the number of pixels in the image; y n，c p is the label value of the pixel; n , c The predicted value for a pixel;

[0046] S4: After segmenting the acquired slice image to be detected, input it into the lesion detection model to obtain the lesion cell image, and use the OpenCV processing library to restore the lesion cell image to the slice image to obtain the position of the lesion cell in the slice image.

[0047] In specific implementation, the slice image to be detected is segmented through steps A1-A2.1 to obtain several image blocks of size 512×512×3. The image blocks to be detected are then processed by the downsampling group to extract features to obtain a feature map of size 16×16×1024. The feature map is then flattened into 1024 one-dimensional semantic sequences. The semantic sequences are then processed by the encoder group to extract semantic features.

[0048] Step S1 includes the following specific steps:

[0049] A1: Staining specific vimentins in FCD type II pathological slides;

[0050] A2: Cut the FCD type II pathological slide image into several image blocks of 512×512 pixels in order from top to bottom and from left to right; the original image size parameters are: 40000×40000 pixels, 24-bit bit depth, 300dpi horizontal / vertical resolution.

[0051] In practice, the FCD type II pathological slide image is in TAMP format, and the FCD type II pathological slide image is converted to svs format before cutting.

[0052] A21: Use the Histolab tool in Python to discard bounding boxes in images that do not contain cellular tissue;

[0053] A3: Use QuPath annotation software to delineate the boundaries of lesion cells in several image blocks to obtain lesion cell boundary maps;

[0054] A4: Binarization of the boundary map of the diseased cells yields a binary image;

[0055] A5: Use the binarized image as the label value, and the image patch and its corresponding binarized image as elements to form a dataset.

[0056] Data Validation

[0057] The same slice image to be detected was input into the UNet model, the TransUNet model and the lesion detection model of the present invention for prediction. The Dice similarity coefficient, sensitivity index, IOU index, accuracy, HD_95 index and model operation time of each model were statistically analyzed to obtain the test results comparison table shown in Table 1.

[0058] Table 1

[0059]

[0060] As shown in Table 1, the prediction results of the UNet model are different from those of the TransUNet model and the lesion detection model. However, the prediction model proposed in this invention has a higher sensitivity index and HD_95 index than the TransUNet model. Moreover, the difference between the Dice similarity coefficient, IOU index and accuracy and those of the TransUNet model is small. The model budget time is greatly reduced, and the prediction results can be obtained more quickly.

Claims

1. A TransUNet-based FCD type II pathological whole-section lesion cell detection method, characterized by, The method comprises the following steps: S1: a database is established after the FCD II pathological section image obtained by scanning is segmented; S2: a lesion detection model is constructed based on TransUNet; The lesion detection model comprises a down-sampling group, an encoder group and an up-sampling group connected in sequence; the down-sampling group comprises three down-sampling modules connected in sequence; the encoder group comprises four encoder modules connected in sequence, and each encoder module comprises a multi-head attention mechanism, a feedforward network and a normalization layer connected in sequence; the up-sampling group comprises four up-sampling modules connected in sequence; The down-sampling module is used for extracting a feature map of a pathological image and converting the feature map into a semantic sequence; The encoder module is used for extracting semantic features according to the semantic sequence; The up-sampling module is used for reconstructing a predicted feature map according to the semantic features, and in the reconstruction process, the up-sampling module splices the feature map extracted by the down-sampling module into the predicted feature map; The up-sampling module comprises two convolution layers and one up-sampling layer; The input channel number of the down-sampling module of the lesion detection model is 3; the channel number after the first layer of down-sampling is 64, the channel number after the second layer of down-sampling is 256, the channel number after the third layer of down-sampling is 512, and the channel number after the fourth layer of down-sampling is 1024; S3: the database is manually screened into a training set, a validation set and a test set, and the lesion detection model is trained; the loss function used in the training is: where C is the number of classes of the label; N is the number of pixels of the image; y n，c is the label value of the pixel; p n，c is the predicted value of the pixel; S4: the collected slice image to be detected is segmented and input into the lesion detection model to obtain a lesion cell image, and the lesion cell image is restored to the slice image by using an OpenCV processing library to obtain the position of the lesion cell in the slice image.

2. The TransUNet-based FCD type II pathology whole-slide lesion cell detection method of claim 1, wherein, The step S1 comprises the following specific steps: A1: a specific vimentin of the FCD II pathological section image is dyed; A2: the FCD II pathological section image is cut to obtain a plurality of image blocks with a size of 512*512 pixels in a sequence from top to bottom and from left to right; the size parameters of the original image are: 40000*40000 pixels, 24-bit depth, 300 dpi horizontal / vertical resolution; A3: the boundaries of the lesion cells in the plurality of image blocks are respectively outlined to obtain lesion cell boundary images by using a Qupath labeling software; A4: the lesion cell boundary images are binarized to obtain binarized images; A5: the binarized images are taken as label values, and the image blocks and the corresponding binarized images are taken as elements to form a data set.

3. The TransUNet-based FCD type II pathology whole-slide lesion cell detection method of claim 2, wherein, The step A2 and A3 further comprise a step A21: discarding the image frame blocks that do not contain cell tissues.

4. The TransUNet-based FCD type II pathology whole-slide lesion cell detection method of claim 1, wherein, A down-sampling module is arranged between the down-sampling group and the encoder group.

Images