Method and device for constructing thyroid cell pathological image information region classification model

Abstract

The application provides a thyroid cell pathological image information region classification model construction method and device. In the preparation of training data, the image blocks segmented from the panoramic pathological image are divided into a primary data set and an advanced data set, and the pathological experts add annotation labels to each image block in the advanced data set. In the training stage, the primary data set is used to preliminarily train a first-level network to train the model to recognize the cell features in the image block, then the parameters of the first-level network after training are used as the initial parameters of a second-level network, and the second-level network is fine-tuned using the advanced data set to obtain a thyroid cell pathological image information region classification model for identifying useful information regions. In this way, the annotation labels based on the pathological experts help to deepen the understanding of the neural network for the useful information region, and secondly, the two-level training fine-tuning scheme can comprehensively learn the objective cell features and the clinical prior knowledge of the pathological experts, effectively improving the performance of the model.

Description

TECHNICAL FIELD

[0001] The application relates to a thyroid cell pathological image information region classification model construction method and device, and belongs to the technical field of neural network models. BACKGROUND

[0002] Fine needle aspiration (FNA) is a routine diagnostic method recommended by the current domestic and foreign thyroid cancer diagnosis and treatment guidelines for preoperative diagnosis of benign and malignant thyroid nodules. However, due to the limitations of thyroid FNA, the number of useful cells (thyroid follicular epithelial cells) in a pathological image is not large and is scattered, and a pathologist will spend a lot of time in finding useful cells, which is a labor-intensive work. In recent years, convolutional neural networks have been applied in the field of thyroid cell pathological image analysis, which can effectively reduce the workload of doctors.

[0003] At present, there are mainly two methods based on convolutional neural networks: 1. analysis based on whole slide image (WSI); 2. using threshold segmentation method in image processing technology to extract the organized area of each WSI and then analyzing.

[0004] However, method 1 requires high GPU capacity due to the high image resolution and large data volume of WSI, and a thyroid cell pathological WSI contains a large number of background areas (red blood cells, blank, stains, etc.), which will weaken the efficiency and accuracy of convolutional neural network analysis. Method 2 uses image color threshold segmentation method to extract the organized area, which does not conform to the quality evaluation process of thyroid fine needle aspiration cell pathological sections by pathologists in clinical practice, and cannot be well compatible with clinical practice. SUMMARY

[0005] The application provides a thyroid cell pathological image information region classification model construction method and device to solve the problems of low efficiency and accuracy of convolutional neural networks or poor compatibility with clinical practice in existing thyroid cell pathological image information analysis methods.

[0006] In a first aspect, the embodiments of the application provide a thyroid cell pathological image information region classification model construction method, comprising:

[0007] Obtaining a preset number of whole slide pathological images of thyroid cells to establish a thyroid cell pathological image data set; wherein the whole slide pathological images are obtained by scanning a cell pathological smear by a digital pathological section scanner, and the cell pathological smear is stained with hematoxylin-eosin;

[0008] Each panoramic pathological image in the thyroid cell pathological image dataset is preprocessed to remove blemishes from the images and normalize the red, green and blue channel values ​​of the panoramic pathological images to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent.

[0009] Each preprocessed panoramic pathological image is divided into multiple image blocks of a preset size;

[0010] All image patches are divided into a primary dataset and a high-level dataset according to a preset ratio, and annotation labels are obtained by pathology experts who manually annotate each image patch in the high-level dataset; wherein, the annotation labels include image patches with useful information and image patches with background information.

[0011] The pre-constructed first-level network is trained using the primary dataset to obtain the trained first-level network. The first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part; perform binarization processing on the hematoxylin-stained part; based on the result of the binarization processing, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei; and use the contour detection method to obtain a two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector.

[0012] The secondary network is fine-tuned using the high-level dataset to obtain a thyroid cell pathology image information region classification model, which is used to identify whether the input image patch is a useful information region; wherein, the secondary network uses the parameters of the trained primary network as initialization parameters.

[0013] Based on the above method, optionally, the obtained panoramic pathological images are output in SVS format to establish a thyroid cell pathological image dataset.

[0014] Based on the above method, optionally, the step of dividing the preprocessed panoramic pathological images into multiple image blocks of a preset size includes:

[0015] The Python interface of Openslide-3.4.1 is called, and its cropping function is used to crop each of the panoramic pathological images to obtain multiple image blocks of a preset size.

[0016] Based on the above method, optionally, the step of dividing all image patches into a primary dataset and a high-level dataset according to a preset ratio includes:

[0017] Based on the above method, optionally, all image patches can be randomly divided into a primary dataset and a high-level dataset at a 1:1 ratio.

[0018] Based on the above method, optionally, an image block containing at least 10 nucleated cells is a useful information image block, and an image block containing fewer than 10 nucleated cells is a background information image block.

[0019] Based on the above methods, optionally, after training the thyroid cell pathology image information region classification model, the following further steps may be taken:

[0020] Reacquire a set number of image blocks containing annotation labels;

[0021] The newly acquired image patches are input into the trained thyroid cell pathology image information region classification model to obtain the recognition results.

[0022] The recognition results are compared with the annotation labels corresponding to the input image blocks to evaluate the performance of the thyroid cell pathology image information region classification model.

[0023] Secondly, embodiments of this application also provide a device for constructing a classification model of thyroid cell pathological image information regions, comprising:

[0024] The acquisition module is used to acquire a preset number of panoramic pathological images of thyroid cells to establish a thyroid cell pathological image dataset; wherein, the panoramic pathological images are obtained by scanning cell pathological smears with a digital pathological slide scanner, and the cell pathological smears are stained with hematoxylin-eosin;

[0025] The preprocessing module is used to preprocess each of the panoramic pathological images in the thyroid cell pathological image dataset to remove blemishes in the images and normalize and map the red, green and blue three-channel values ​​of the panoramic pathological images to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent.

[0026] The segmentation module is used to segment each preprocessed panoramic pathological image into multiple image blocks of a preset size;

[0027] The partitioning module is used to divide all image blocks into a primary dataset and a high-level dataset according to a preset ratio, and to obtain annotation labels obtained by pathology experts through manual annotation of each image block in the high-level dataset; wherein, the annotation labels include image blocks with useful information and image blocks with background information.

[0028] The initial training module is used to train the pre-constructed first-level network using the primary dataset to obtain the trained first-level network. The first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part; perform binarization processing on the hematoxylin-stained part; based on the result of the binarization processing, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei; and use the contour detection method to obtain a two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector.

[0029] The secondary fine-tuning module is used to fine-tune the secondary network using the high-level dataset to obtain a thyroid cell pathology image information region classification model, which is used to identify whether the input image patch is a useful information region; wherein, the secondary network uses the parameters of the trained primary network as initialization parameters.

[0030] The technical solution provided in this application has at least the following beneficial effects:

[0031] This application provides a method and apparatus for constructing a thyroid cell pathology image information region classification model. In preparing training data, image patches obtained by segmenting panoramic pathology images are divided into a primary dataset and a high-level dataset. Pathology experts add annotation labels to each image patch in the high-level dataset. During the training phase, a primary network is first trained using the primary dataset to improve the model's ability to recognize cell features in input image patches. Then, the parameters of the trained primary network are used as the initial parameters of a secondary network, which is fine-tuned using the high-level dataset. Finally, the desired thyroid cell pathology image information region classification model is obtained, enabling the model to identify whether an input image patch is a useful information region. Based on this scheme, the annotation of image patches by pathology experts helps deepen the neural network's understanding of useful information regions in thyroid cell pathology images. Furthermore, the two-level training and fine-tuning scheme can comprehensively learn objective cell features and the prior clinical knowledge of pathology experts, effectively improving the model's performance. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Furthermore, these drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments.

[0033] Figure 1 A flowchart illustrating a method for constructing a thyroid cell pathological image information region classification model according to an embodiment of this application;

[0034] Figure 2A schematic diagram illustrating the principle of a method for constructing a thyroid cell pathological image information region classification model according to an embodiment of this application;

[0035] Figure 3 A schematic diagram of a network architecture with two training and fine-tuning steps provided in one embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the structure of a thyroid cell pathological image information region classification model construction device provided in one embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0038] As described in the background section, after obtaining thyroid cell pathological images based on FNA technology, convolutional neural networks can be used to analyze the thyroid cell pathological images in order to reduce the workload of doctors.

[0039] However, existing methods for analyzing thyroid cell pathological images based on convolutional neural networks have problems such as low efficiency and accuracy or poor compatibility with clinical applications.

[0040] To address the aforementioned issues, this application provides a scheme for constructing a classification model for thyroid cell pathological image information regions. This model enhances the neural network's understanding of useful information regions in thyroid cell pathological images based on prior clinical knowledge. Furthermore, a two-level training and fine-tuning scheme comprehensively trains the model's ability to learn objective cell features and prior clinical knowledge, effectively improving model performance. The following examples and embodiments provide non-limiting descriptions of the specific implementation scheme.

[0041] First, some embodiments of this application provide a method for constructing a thyroid cell pathology image information region classification model. Using this method, a thyroid cell pathology image information region classification model can be trained. This model can identify and classify input images to determine whether the input image contains useful thyroid cell information.

[0042] Reference Figure 1 and Figure 2 , Figure 1This is a flowchart illustrating a method for constructing a regional classification model of thyroid cell pathological image information provided in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the principle of a method for constructing a thyroid cell pathological image information region classification model provided in an embodiment of this application. Figure 1 and Figure 2 As shown, the method for constructing a thyroid cell pathological image information region classification model in this embodiment includes the following steps:

[0043] Step S101: Obtain a preset number of panoramic pathological images of thyroid cells to establish a thyroid cell pathological image dataset; wherein, the panoramic pathological images are obtained by scanning cell pathological smears with a digital pathological slide scanner, and the cell pathological smears are stained with hematoxylin-eosin.

[0044] Specifically, in this embodiment, the panoramic pathological image is obtained by scanning the cytopathological smear using a digital pathological slide scanner. The cytopathological smear can be prepared after obtaining thyroid cells using fine-needle aspiration biopsy (FNA) technology. The cytopathological smear is stained with hematoxylin and eosin to distinguish between the cell nucleus and extranuclear cytoplasm.

[0045] Hematoxylin-eosin staining, or HE staining for short, is the most basic and widely used technique in histology, embryology, and pathology teaching. Hematoxylin is an alkaline staining solution, which primarily colors the chromatin in the cell nucleus and nucleic acids in the cytoplasm with a purple-blue hue; eosin is an acidic dye, which primarily colors the components of the cytoplasm and extracellular matrix with a red hue.

[0046] A predetermined number of panoramic pathological images of thyroid cells are acquired to establish a thyroid cell pathological image dataset, which will then be used to train a classification model in subsequent steps.

[0047] In some embodiments, the scanned panoramic pathology images are output in SVS format to create a thyroid cell pathology image dataset. Specifically, SVS (Scanned Virtual Slide) format is a file format used for storing and transmitting digital pathology images. SVS format has many advantages, such as the ability to store very large images, making it suitable for high-resolution digital pathology images; it also supports lossless compression, preserving the original image quality without introducing compression loss; and it can segment images into multiple levels, each with a different resolution. This multi-level storage method makes viewing images at different resolutions more efficient, allowing for the selection of appropriate resolutions for display and analysis as needed.

[0048] Step S102: Preprocess each panoramic pathological image in the thyroid cell pathological image dataset to remove blemishes from the images and normalize the red, green and blue channel values ​​of the panoramic pathological images to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent.

[0049] Specifically, removing blemishes from panoramic pathology images can improve image quality. More specifically, because the cells in the image have been stained, they will appear in a different color than the blemishes. Based on this, blemishes can be removed by adjusting the red, green, and blue channel values ​​of the panoramic pathology image using set rules. The principle is to set the pixel values ​​of pixels whose color values ​​do not conform to the characteristics of stained cells as background values ​​to remove blemishes.

[0050] Furthermore, normalizing the red, green, and blue channel values ​​of panoramic pathology images to standardized parameters (i.e., a specific range of pixel values) can ensure that the color distribution of all panoramic pathology images is consistent, which helps to reduce the adverse effects of color distribution differences on model training.

[0051] Of course, it is understandable that, in addition to the above preprocessing content, other preprocessing can be performed as needed, and there are no restrictions on this.

[0052] Step S103: Divide each preprocessed panoramic pathological image into multiple image blocks of a preset size.

[0053] Specifically, because panoramic pathological images have high resolution and contain a large amount of data, directly using them as training data input into the model would place high demands on the GPU capacity of the training device, making it difficult to implement in practice. Furthermore, a single panoramic pathological image contains a large number of background regions, and the redundant information in these background regions can weaken the efficiency and accuracy of convolutional neural network analysis, hindering model training. Therefore, in this embodiment, the preprocessed panoramic pathological image is divided into multiple small image patches of a preset size. During subsequent training, these image patches are used as model input, thereby reducing the GPU capacity requirements. Simultaneously, useful parts can be further filtered from the large number of image patches, reducing the interference of useless redundant information on the model training process.

[0054] In some embodiments, the specific implementation process of step S103 may include: calling the Python interface of Openslide-3.4.1 and using its cropping function to crop each panoramic pathological image to obtain multiple image blocks of a preset size.

[0055] Specifically, Openslide-3.4.1 is a version of the Openslide library, an open-source library for reading and processing large mirror-image scans. It provides a powerful set of tools and functions for loading, viewing, cropping, scaling, and analyzing high-resolution mirror-image scans in Python. Based on the Openslide library, processing and analyzing large mirror-image scans (such as the panoramic thyroid pathology image in this case) in a Python environment becomes easier and more efficient.

[0056] Step S104: Divide all image blocks into primary and advanced datasets according to a preset ratio, and obtain annotation labels obtained by pathology experts who manually annotate each image block in the advanced dataset; wherein, the annotation labels include image blocks with useful information and image blocks with background information.

[0057] Specifically, before training the model, this step first divides the large number of image patches obtained in the previous steps into two categories: a primary dataset and a high-level dataset, which are used to train the model's ability to learn different information.

[0058] The advanced dataset includes annotations manually provided by pathologists. These annotations are divided into two categories: useful information patches and background information patches. If an image patch is labeled as a useful information patch, it indicates that the patch contains sufficient useful cells (thyroid follicular epithelial cells). Similarly, if an image patch is labeled as a background information patch, it indicates that the patch mainly contains background information (e.g., blank areas or other cells). Adding these annotations effectively trains the model to recognize useful information. The primary dataset, mainly used to train the model's ability to recognize cellular features, does not require additional annotations.

[0059] In some embodiments, when dividing the primary and advanced datasets, all image patches can be randomly assigned to the primary and advanced datasets in a 1:1 ratio. This random division reduces human influence. Furthermore, a 1:1 ratio allows the model to better balance the learning capabilities of both datasets. Of course, this ratio can be adjusted according to actual needs.

[0060] In addition, in some embodiments, when adding annotation labels, image blocks containing at least 10 nucleated cells are designated as useful information image blocks, while image blocks containing fewer than 10 nucleated cells are designated as background information image blocks.

[0061] Specifically, current clinical assessment of thyroid cell pathology slide quality is typically based on cell clusters. If a cell pathology slide contains 6 cell clusters, and each cluster contains at least 10 cells, the slide is considered "satisfactory." Therefore, in this embodiment, the definition of useful information image blocks is based on the aforementioned prior clinical knowledge. The image block classification criteria are: image blocks containing ≥10 nucleated cells are labeled as useful information image blocks, and image blocks containing <10 nucleated cells are labeled as background information image blocks. Additionally, two or more experienced pathologists (≥10 years of experience) can be selected for manual labeling and annotation. For image blocks where multiple pathologists have differing opinions, the final labeling will be determined through consultation.

[0062] Step S105: Train the pre-constructed first-level network using the primary dataset to obtain the trained first-level network; wherein, the first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part, perform binarization processing on the hematoxylin-stained part, and based on the result of the binarization processing, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei, and use the contour detection method to obtain the two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector.

[0063] Specifically, a suitable convolutional neural network algorithm can be pre-selected to construct the first-level network. For example, ResNet (residual network) and DenseNet (dense connection network).

[0064] The first-level network is first trained using a primary dataset, i.e., the input is image patches without annotation labels. The first-level network performs deconvolution on the input image patches to separate the hematoxylin-stained and eosin-stained parts, that is, it identifies and separates the cell nuclei and other parts stained with hematoxylin (H) and the cytoplasm and other parts stained with eosin (E) from the image patches.

[0065] Next, the hematoxylin-stained portion was binarized, that is, the cell nucleus in the hematoxylin-stained portion was set to different pixel values ​​from other parts. The main purpose was to distinguish the cell nucleus from other parts (such as the nucleic acid portion in the cytoplasm) so as to better identify the cell nucleus.

[0066] Next, based on the binarization results, the watershed algorithm is used to separate adhered or partially overlapping cell nuclei. The basic principle of the watershed algorithm is as follows: First, all pixels in the gradient image are classified according to their grayscale values, and a geodesic distance threshold is set. Then, the pixel with the smallest grayscale value is found, and the threshold is increased starting from the minimum value. During this increase, the geodesic distance between the pixel with the smallest grayscale value and the current pixel is calculated. If the distance is less than the set threshold, these pixels are "flooded"; otherwise, a "dam" is set on these pixels. This classifies these neighboring pixels. The watershed algorithm can more accurately identify the number of cell nuclei, avoiding the misidentification of adhered or partially overlapping cell nuclei as a single cell nucleus.

[0067] Finally, a contour detection method is used to obtain two-dimensional vectors of the number of nuclei (i.e., the total number of cell nuclei) and the total area from each image patch. Cell features can then be identified based on these two-dimensional vectors. Specifically, a suitable contour detection method can be selected from the OpenCV library to detect cell nuclei contours, thereby obtaining the number of cell nuclei and their total area. Ultimately, based on the two-dimensional vectors of the number of nuclei and the total area of ​​each image patch, the first-level network can accurately identify which parts of the image patch are cell nuclei.

[0068] Because the low-level dataset lacks labeled annotations, the first-level network trained on it attempts to group similar samples together and establish associations between them. That is, if two samples are very close in the input space, they are likely to have similar outputs. Therefore, by utilizing the unlabeled low-level dataset as training samples, the model can learn cellular features of input image patches over a wider range, improving its predictive ability for inputs without labels. As the first-level network gradually learns cellular features in image patches during training, it can better generalize to new, unseen image patches.

[0069] Step S106: Fine-tune the secondary network using the high-level dataset to obtain a thyroid cell pathology image information region classification model, which is used to identify whether the input image patch is a useful information region; wherein, the secondary network uses the parameters of the trained primary network as initialization parameters.

[0070] Specifically, fine-tuning the secondary network is essentially a further training of the trained primary network, with different sample datasets used in the two training sessions to emphasize different aspects of the model's processing and recognition capabilities. The primary training of the primary network aims to enable the model to accurately identify cellular features (cell nuclei) within image patches, while the training of the secondary network aims to enable the model to identify whether the entire input image patch contains useful information, i.e., whether it contains a sufficient number of useful cells, based on annotations from pathologists. A schematic diagram of the network architecture for the two training and fine-tuning sessions is shown below.Figure 3 As shown.

[0071] Therefore, based on the above scheme, the annotation of image patches by pathology experts helps the neural network to understand the useful information regions of thyroid cell pathology images. Secondly, the two-level training fine-tuning scheme can comprehensively learn objective cell features and the clinical prior knowledge of pathology experts, effectively improving the performance of the model.

[0072] In practical applications of the trained thyroid cell pathology image information region classification model, users can segment the panoramic pathology image into multiple image patches (the same size as the training patch size) using the same method. Each patch is then input into the model, which performs recognition and classification, outputting the recognition result. The result includes whether the corresponding image patch is a useful information region. Based on this, further analysis can be performed using manual or other automated methods, such as benign / malignant diagnosis and identification of suspicious malignant image regions.

[0073] Furthermore, in some embodiments, after training the thyroid cytopathology image information region classification model, the above method may also include:

[0074] Reacquire a set number of image patches containing annotation labels; input the reacquired image patches into the trained thyroid cell pathology image information region classification model to obtain the recognition results; compare the recognition results with the annotation labels corresponding to the input image patches to evaluate the performance of the thyroid cell pathology image information region classification model.

[0075] The image blocks containing annotation labels can be processed using the methods described in steps S101-104 above. The image blocks containing annotation labels are then input into a trained thyroid cell pathology image information region classification model. This model automatically outputs the recognition results, which are then compared with the annotation labels added by pathologists. The performance of the thyroid cell pathology image information region classification model is evaluated based on its accuracy. If the accuracy is below a set threshold, further training can be performed until a model that meets the requirements is obtained.

[0076] Furthermore, embodiments of this application also provide a device for constructing a regional classification model for thyroid cell pathological image information. For example... Figure 4 As shown, the device includes:

[0077] The acquisition module 41 is used to acquire a preset number of panoramic pathological images of thyroid cells to establish a thyroid cell pathological image dataset; wherein, the panoramic pathological images are obtained by scanning cell pathological smears with a digital pathological slide scanner and the cell pathological smears are stained with hematoxylin-eosin.

[0078] The preprocessing module 42 is used to preprocess each panoramic pathological image in the thyroid cell pathological image dataset to remove blemishes in the image and normalize and map the red, green and blue three-channel values ​​of the panoramic pathological image to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent.

[0079] The segmentation module 43 is used to segment each preprocessed panoramic pathological image into multiple image blocks of a preset size;

[0080] The partitioning module 44 is used to partition all image blocks into a primary dataset and a high-level dataset according to a preset ratio, and to obtain annotation labels obtained by pathology experts who manually annotate each image block in the high-level dataset; wherein, the annotation labels include image blocks with useful information and image blocks with background information.

[0081] The initial training module 45 is used to train the pre-constructed first-level network using the primary dataset to obtain the trained first-level network. The first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part; perform binarization on the hematoxylin-stained part; based on the result of the binarization, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei; and use the contour detection method to obtain the two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector.

[0082] The secondary fine-tuning module 46 is used to fine-tune the secondary network using the high-level dataset to obtain a classification model of thyroid cell pathology image information regions, which is used to identify whether the input image patch is a useful information region; wherein, the secondary network uses the parameters of the trained primary network as initialization parameters.

[0083] The specific implementation methods of each module of the above-mentioned thyroid cell pathological image information region classification model construction device can be referred to the corresponding content in the aforementioned method embodiments, and will not be repeated here.

[0084] Based on the modules in the aforementioned device, a classification model for thyroid cell pathology image information regions can be constructed. Furthermore, annotations of image patches by pathologists help deepen the neural network's understanding of useful information regions in thyroid cell pathology images. Secondly, the two-level training and fine-tuning scheme can comprehensively learn objective cell characteristics and the prior clinical knowledge of pathologists, effectively improving the model's performance.

[0085] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0086] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0087] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0088] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0089] Furthermore, the functional units in the various embodiments of this invention can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0090] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0091] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for constructing a classification model of thyroid cell pathological image information regions, characterized in that, include: A predetermined number of panoramic pathological images of thyroid cells are acquired to establish a thyroid cell pathological image dataset; wherein, the panoramic pathological images are obtained by scanning cell pathological smears with a digital pathological slide scanner, and the cell pathological smears are stained with hematoxylin-eosin; Each panoramic pathological image in the thyroid cell pathological image dataset is preprocessed to remove blemishes from the images and normalize the red, green and blue channel values ​​of the panoramic pathological images to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent. Each preprocessed panoramic pathological image is divided into multiple image blocks of a preset size; All image blocks are divided into primary and advanced datasets according to a preset ratio, and annotation labels are obtained by pathology experts who manually annotate each image block in the advanced dataset. The annotation labels include useful information image blocks and background information image blocks. Image blocks containing at least 10 nucleated cells are useful information image blocks, and image blocks containing fewer than 10 nucleated cells are background information image blocks. The pre-constructed first-level network is trained using the primary dataset to obtain the trained first-level network. The first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part; perform binarization processing on the hematoxylin-stained part; based on the result of the binarization processing, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei; and use the contour detection method to obtain a two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector. The secondary network is fine-tuned using the high-level dataset to obtain a thyroid cell pathology image information region classification model, which is used to identify whether the input image patch is a useful information region; wherein, the secondary network uses the parameters of the trained primary network as initialization parameters; Furthermore, after training the thyroid cell pathology image information region classification model, a set number of image patches containing annotation labels are reacquired. The newly acquired image patches are input into the trained thyroid cell pathology image information region classification model to obtain the recognition results. The recognition results are compared with the annotation labels corresponding to the input image blocks to evaluate the performance of the thyroid cell pathology image information region classification model.

2. A device for constructing a classification model of thyroid cell pathological image information regions, characterized in that, include: The acquisition module is used to acquire a preset number of panoramic pathological images of thyroid cells to establish a thyroid cell pathological image dataset; wherein, the panoramic pathological images are obtained by scanning cell pathological smears with a digital pathological slide scanner, and the cell pathological smears are stained with hematoxylin-eosin; The preprocessing module is used to preprocess each of the panoramic pathological images in the thyroid cell pathological image dataset to remove blemishes in the images and normalize and map the red, green and blue three-channel values ​​of the panoramic pathological images to standardized parameters to ensure that the color distribution of all panoramic pathological images is consistent. The segmentation module is used to segment each preprocessed panoramic pathological image into multiple image blocks of a preset size; The segmentation module is used to divide all image blocks into a primary dataset and a high-level dataset according to a preset ratio, and to obtain annotation labels obtained by pathology experts through manual annotation of each image block in the high-level dataset; wherein, the annotation labels include useful information image blocks and background information image blocks, image blocks containing at least 10 nucleated cells are useful information image blocks, and image blocks containing fewer than 10 nucleated cells are background information image blocks. The initial training module is used to train the pre-constructed first-level network using the primary dataset to obtain the trained first-level network. The first-level network is used to: perform deconvolution on the input image patch to separate the hematoxylin-stained part and the eosin-stained part; perform binarization processing on the hematoxylin-stained part; based on the result of the binarization processing, use the watershed algorithm to separate the adhered or partially overlapping cell nuclei; and use the contour detection method to obtain a two-dimensional vector of the number of nuclei and the total area from each image patch, and then identify cell features based on the two-dimensional vector. A secondary fine-tuning module is used to fine-tune the secondary network using the high-level dataset to obtain a thyroid cell pathology image information region classification model, which is used to identify whether the input image patch is a useful information region. The secondary network uses the parameters of the trained primary network as initialization parameters. After training the thyroid cell pathology image information region classification model, a set number of image patches with annotation labels are re-acquired. The re-acquired image patches are input into the trained thyroid cell pathology image information region classification model to obtain recognition results. The recognition results are compared with the annotation labels corresponding to the input image patches to evaluate the performance of the thyroid cell pathology image information region classification model.

Images