Urinary tract epithelial cell combined classification method, device and equipment and storage medium

Abstract

The invention discloses a urinary tract epithelial cell joint classification method, device and equipment and a storage medium, and the method comprises the steps: obtaining a liquid-based slice image and a fluorescence in-situ hybridization slice image of urinary tract epithelial cells, dividing the liquid-based slice image into a plurality of first sub-images, and dividing the fluorescence in-situ hybridization slice image into a plurality of second sub-images; obtaining a first depth feature of each first sub-image, determining a first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determining a first statistical result of the liquid-based slice image according to each first classification result; acquiring a second depth feature of each second sub-image, determining a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determining a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; and determining a classification result of the urinary tract epithelial cells according to the first depth feature, the first statistical result, the second depth feature and the second statistical result.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, equipment and storage medium for joint classification of urothelial cells. Background Art

[0002] Urothelial carcinoma is a common malignant tumor, and histopathology is generally used as the standard for diagnosis. However, since surgery often causes trauma and requires recovery, if there is a stable, accurate and non-surgical diagnostic method, it can help doctors make a better diagnosis before surgery. Urine cytology and urine fluorescence in situ hybridization are both non-invasive screening methods based on urine. They have the advantage of being non-invasive and are important means of screening and detecting urothelial carcinoma. In actual diagnosis, urine cytology liquid-based sections and urine fluorescence in situ hybridization sections are experimented and diagnosed from the same sample, and finally a comprehensive diagnostic conclusion is obtained. In recent years, with the development of computer vision technology, a number of computer vision-based diagnostic systems have begun to emerge in the field of cytopathology.

[0003] In the prior art, a urothelial cell slide can be scanned into a slide image by a digital scanner, and the slide image can be divided into multiple sub-images, and the classification results of urothelial cells can be integrated according to the classification results of the sub-images. However, since the sub-image only represents a local image in the slide image, it cannot fully reflect the information of the entire image, resulting in low accuracy of the classification results of urothelial cells. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for joint classification of urothelial cells, so as to fuse the features of each sub-image in the liquid-based slice image and the fluorescent in situ hybridization slice image of urothelial cells and the statistical results, accurately analyze the category of urothelial cells, and solve the problem of low classification accuracy of urothelial cells caused by inferring the category of urothelial cells only by sub-image category in the prior art.

[0005] In a first aspect, the present application provides a combined classification method for urothelial cells, comprising: Acquire a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, divide the liquid-based slice image into a plurality of first sub-images, and divide the fluorescence in situ hybridization slice image into a plurality of second sub-images; Acquire a first depth feature of each of the first sub-images, determine a first classification result corresponding to the first sub-image based on the first depth feature of each of the first sub-images, and determine a first statistical result of the liquid-based slice image according to each of the first classification results; Acquire a second depth feature of each second sub-image, determine a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determine a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; A classification result of the urothelial cells is determined based on the first depth feature, the first statistical result, the second depth feature, and the second statistical result.

[0006] In a second aspect, the present application provides a urothelial cell combined classification device, comprising: A first image division module is configured to obtain a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, divide the liquid-based slice image into a plurality of first sub-images, and divide the fluorescence in situ hybridization slice image into a plurality of second sub-images; A first image processing module is configured to obtain a first depth feature of each of the first sub-images, determine a first classification result corresponding to the first sub-image based on the first depth feature of each of the first sub-images, and determine a first statistical result of the liquid-based slice image according to each of the first classification results; a second image processing module, configured to obtain a second depth feature of each of the second sub-images, determine a second classification result corresponding to the second sub-image based on the second depth feature of each of the second sub-images, and determine a second statistical result of the fluorescence in situ hybridization slice image according to each of the second classification results; The first cell classification module is configured to determine the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature and the second statistical result.

[0007] In a third aspect, the present application provides a urothelial cell combined classification device, comprising: One or more processors; a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the urothelial cell joint classification method as described in the first aspect.

[0008] In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform the method for joint classification of urothelial cells as described in the first aspect.

[0009] In the present application, by acquiring liquid-based slice images and fluorescence in situ hybridization slice images of urothelial cells, the liquid-based slice images are divided into multiple first sub-images, and the fluorescence in situ hybridization slice images are divided into multiple second sub-images; the first depth feature of each first sub-image is acquired, and the first classification result corresponding to the first sub-image is determined based on the first depth feature of each first sub-image, and the first statistical result of the liquid-based slice image is determined according to each first classification result; the second depth feature of each second sub-image is acquired, and the second classification result corresponding to the second sub-image is determined based on the second depth feature of each second sub-image, and the second statistical result of the fluorescence in situ hybridization slice image is determined according to each second classification result; the classification result of the urothelial cells is determined according to the first depth feature, the first statistical result, the second depth feature and the second statistical result. Through the above-mentioned technical means, the features of each sub-image in the liquid-based section image and the fluorescence in situ hybridization section image of the urothelial cells and the classification statistical results can be fused. The fusion result not only includes the features that can reflect the information of the whole image, but also includes the features of different types of images and classification statistical results. The fusion result contains rich characteristic information of the urothelial cells. The category of the urothelial cells can be accurately analyzed based on the fusion result, which solves the problem of low classification accuracy of the urothelial cells caused by inferring the category of the urothelial cells only by the sub-image category in the prior art, and effectively improves the classification accuracy of the urothelial cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flow chart of a method for combined classification of urothelial cells provided in an embodiment of the present application; Figure 2 is a flow chart for identifying the category of a digital image provided by an embodiment of the present application; Figure 3 is a flowchart of determining a first statistical result provided by an embodiment of the present application; Figure 4 is a flow chart for determining a second statistical result provided by an embodiment of the present application; Figure 5 is a flow chart for determining the classification results of urothelial cells provided in an embodiment of the present application; Figure 6 is a flow chart of classifying urothelial cells based on a single digital image provided in an embodiment of the present application; Figure 7 is a schematic diagram of the operation process of the combined diagnosis algorithm provided in the embodiment of the present application; Figure 8 It is a structural schematic diagram of a urothelial cell combined classification device provided in an embodiment of the present application; Fig. 9 It is a structural schematic diagram of a urothelial cell combined classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only part of the present application is shown in the accompanying drawings, but not all of the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0012] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0013] In a more common existing implementation, a urothelial cell slide is scanned into a slide image by a digital scanner, the slide image is divided into multiple sub-images, and the classification results of the urothelial cells are integrated according to the classification results of the sub-images. For example, when the lesion ratio of the sub-image is greater than or equal to a preset ratio threshold, it is determined that the urothelial cells have lesions, and when the lesion ratio of the sub-image is less than the preset ratio threshold, it is determined that the urothelial cells have no lesions. However, since the sub-image only represents a local image in the slide image, it cannot fully reflect the information of the entire image, resulting in low accuracy of the classification results of urothelial cells.

[0014] To solve the above problems, this embodiment provides a joint classification method for urothelial cells, which fuses the features of each sub-image in the liquid-based slice image and the fluorescent in situ hybridization slice image of the urothelial cells and the statistical results, accurately analyzes the category of the urothelial cells, and improves the classification accuracy of the urothelial cells.

[0015] The urothelial cell joint classification method provided in this embodiment can be performed by a urothelial cell joint classification device, which can be implemented by software and / or hardware, and the urothelial cell joint classification device can be composed of two or more physical entities, or can be composed of one physical entity. For example, the urothelial cell joint classification device can be a computer device with strong processing power such as a computer and a server. Among them, the server can be implemented by an independent server or a server cluster composed of multiple servers.

[0016] The urothelial cell joint classification device is installed with at least one type of operating system, and the urothelial cell joint classification device can install at least one application based on the operating system, and the application can be an application that comes with the operating system, or an application downloaded from a third-party device or server. In this embodiment, the urothelial cell joint classification device is installed with at least an application that can execute the urothelial cell joint classification method.

[0017] For ease of understanding, this embodiment is described by taking a computer as an example of a subject that executes the urothelial cell joint classification method.

[0018] Figure 1 A flowchart of a combined classification method for urothelial cells provided in an embodiment of the present application is given. Figure 1 , the combined classification method of urothelial cells specifically includes: S110, acquiring a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, dividing the liquid-based slice image into a plurality of first sub-images, and dividing the fluorescence in situ hybridization slice image into a plurality of second sub-images.

[0019] Among them, the liquid-based slice image is a digital image obtained by scanning the urine cytology liquid-based slice with a digital scanner, and the fluorescent in situ hybridization slice image is a digital image obtained by scanning the urine fluorescent in situ hybridization slice with a digital scanner. This embodiment aims to jointly diagnose the liquid-based slice image and the fluorescent in situ hybridization slice image of urothelial cells to improve the classification accuracy of urothelial cells. Exemplarily, after the staff uses a digital scanner to scan the urine cytology liquid-based slice and the urine fluorescent in situ hybridization slice into liquid-based slice images and fluorescent in situ hybridization slice images, respectively, the liquid-based slice image and the fluorescent in situ hybridization slice image are uploaded to the computer, and the computer performs visual analysis on the received liquid-based slice image and fluorescent in situ hybridization slice image to diagnose the category of urothelial cells corresponding to the liquid-based slice image and the fluorescent in situ hybridization slice image.

[0020] It should be noted that, since the liquid-based slice image and the fluorescent in situ hybridization slice image contain different image contents, different deep learning models will be used to perform visual analysis on the liquid-based slice image and the fluorescent in situ hybridization slice image. Accordingly, when the computer obtains the two digital images uploaded by the staff, it is necessary to identify which digital image is the liquid-based slice image and which digital image is the fluorescent in situ hybridization slice image. In this embodiment, the staff can scan the liquid-based slice image and the fluorescent in situ hybridization slice image with a digital scanner, mark the corresponding image category for the liquid-based slice image, mark the corresponding image category for the fluorescent in situ hybridization slice image, and upload the liquid-based slice image and the corresponding marked category and the fluorescent in situ hybridization slice image and the corresponding marked category to the computer. The computer can determine which digital image is the liquid-based slice image and which digital image is the fluorescent in situ hybridization slice image based on the received two digital images and the corresponding marked categories. For example, assuming that digital image A is a liquid-based slice image and digital image B is a fluorescence in situ hybridization slice image, the staff will mark the category of digital image A as a liquid-based slice image and the category of digital image B as a fluorescence in situ hybridization slice image, and upload digital image A and the corresponding marked categories and digital image B and the corresponding marked categories to the computer. The computer determines that digital image A is a liquid-based slice image based on the category marked in digital image A, and determines that digital image A is a fluorescence in situ hybridization slice image based on the category marked in digital image B.

[0021] In another embodiment, in order to save the step of manually marking the digital image category and save labor costs, the computer can perform content analysis on the received digital image to identify the liquid-based slice image and the fluorescent in situ hybridization slice image in the two received digital images. Figure 2 : is a flowchart of identifying the category of a digital image provided by an embodiment of the present application. Figure 2 As shown, the step of identifying the category of the digital image specifically includes S1101-S1102: S1101. Acquire two digital images of urothelial cells, and reduce each digital image to a thumbnail.

[0022] Exemplarily, the computer obtains two digital images uploaded by the staff, and can reduce the digital images to thumbnails based on the preset size of the thumbnails. Alternatively, the reduction ratio is preset based on the size ratio of the thumbnail to the digital image, and the digital image is reduced to a thumbnail based on the preset reduction ratio. It can be understood that the size of the digital image is relatively large, and it is usually composed of 50,000 to 200,000*50,000 to 200,000 pixels. If the content analysis is performed directly on the original size of the digital image, it will seriously affect the computing efficiency of the computer. The content of the liquid-based section image and the fluorescent in situ hybridization section image is quite different. Therefore, when the digital image is reduced to a thumbnail, the category of the digital image can also be accurately identified based on the deep features extracted from the thumbnail, thereby ensuring the accuracy of image recognition and improving the recognition efficiency. Among them, the size of the thumbnail can be set according to actual needs and is not limited here.

[0023] S1102: Acquire a third depth feature of each thumbnail, and identify a liquid-based slice image and a fluorescent in situ hybridization slice image in two digital images based on each third depth feature.

[0024] Exemplarily, a thumbnail is input into a pre-trained image classification model, and the image classification model identifies the corresponding input thumbnail as a liquid-based slice image or a fluorescent in situ hybridization slice image. The image classification model includes a first feature extraction network and an image classification network, the first feature extraction network may be a deep convolutional network or a deep attention network, and the image classification network may be a fully connected layer. After the thumbnail of the digital image is input into the pre-trained image classification model, the third depth feature in the corresponding input thumbnail is extracted by the first feature extraction network, and the third depth feature is input into the image classification network. The image classification network determines the image category of the digital image as a liquid-based slice image or a fluorescent in situ hybridization slice image based on the third depth feature.

[0025] In this embodiment, thumbnails of fluorescent in situ hybridization slice images and liquid-based slice images of various urothelial cells are used as first sample images in advance, the categories of the first sample images are marked, and an image classification model is trained using the first sample images and the corresponding image categories, so that the trained image classification model has the function of identifying the image category of the thumbnail.

[0026] This embodiment obtains the third depth feature of each digital image and accurately identifies the image category of the digital image according to the third depth feature, so as to subsequently perform different visual analyses on images of different categories to ensure the accuracy of diagnosis of urothelial cells.

[0027] After determining which digital image is a liquid-based slice image and which digital image is a fluorescence in situ hybridization slice image, the liquid-based slice image can be slidably cropped through a window of a preset size, and the image block cropped by the window each time is used as the first sub-image. After traversing the entire liquid-based slice image, multiple first sub-images corresponding to the liquid-based slice image are obtained. Among them, the sliding step length of the window is equal to the size of the window, that is, multiple first sub-images can be spliced ​​into a complete liquid-based slice image, and there is no overlapping area between adjacent first sub-images. Similarly, the fluorescence in situ hybridization slice image is slidably cropped through a window of a preset size, and the image block cropped by the window each time is used as the second sub-image. After traversing the entire fluorescence in situ hybridization slice image, multiple second sub-images corresponding to the fluorescence in situ hybridization slice image are obtained. Among them, the sliding step length of the window is equal to the size of the window, that is, multiple second sub-images can be spliced ​​into a complete fluorescence in situ hybridization slice image, and there is no overlapping area between adjacent second sub-images. The second sub-image and the first sub-image are intercepted by windows of the same size, so the first sub-image and the second sub-image have the same size. It can be understood that the size of liquid-based slice images and fluorescence in situ hybridization slice images is relatively large, and they need to be divided into multiple sub-images during classification processing so as to classify the liquid-based slice images and fluorescence in situ hybridization slice images through the sub-images. Otherwise, the operating pressure of the computer will be relatively large, affecting the computing efficiency of the computer.

[0028] S120, obtaining a first depth feature of each first sub-image, determining a first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determining a first statistical result of the liquid-based slice image according to each first classification result.

[0029] Exemplarily, the first sub-image is input into a pre-trained first cell classification model, the first depth feature of the first sub-image is extracted by the first cell classification model, and the first classification result of the first sub-image is determined based on the first depth feature. Wherein, the first cell classification model includes a second feature extraction network and a first cell classification network, the second feature extraction network is used to extract the first depth feature of the first sub-image, the second feature extraction network can be a deep convolutional network or a deep attention network, the first cell classification network can be a fully connected layer, and the first cell classification network is used to determine the first classification result of the first sub-image. In this embodiment, the sub-images divided in advance by the liquid-based slice images of each urothelial cell are used as the second sample image, and the categories of the second sample image are marked, and the categories include cell lesions, cells transitioning from normal to lesions, and cells without lesions. The first cell classification model is trained by the second sample image and the corresponding categories, so that the trained first cell classification model has the function of classifying the first sub-image.

[0030] Furthermore, after the first classification result of each first sub-image is obtained, the number of first sub-images of each category in the corresponding liquid-based slice image is counted according to the first classification result of each first sub-image to obtain a first statistical result.

[0031] In this embodiment, Figure 3 is a flow chart of determining the first statistical result provided by an embodiment of the present application. Figure 3 As shown, the step of determining the first statistical result specifically includes S1201-S1202: S1201. Input a first depth feature of a first sub-image into a pre-trained first cell classification network to obtain a first classification result of the first sub-image output by the first cell classification network, wherein the first classification result is cell pathology, cell transition, or cell non-pathology.

[0032] Exemplarily, after obtaining the first depth feature of the first sub-image through the second feature extraction network, the first depth feature is input into the first cell classification network, and the first cell classification network determines that the cell state of the first sub-image is pathological based on the first depth feature, and transitions from normal to pathological or non-pathological, and then outputs the first classification result of the first sub-image.

[0033] S1202: Count the number of first sub-images corresponding to each first classification result to obtain a first statistical result of the liquid-based slice image.

[0034] For example, the number of first sub-images whose first classification results are cell lesions is M, the number of first sub-images whose first classification results are cell transition is N, and the number of first sub-images whose first classification results are cell non-lesions is K. M, N, and K are used to generate a vector As the first statistical result.

[0035] This embodiment uses a pre-trained first cell classification network to quickly determine a first classification result of the first sub-image based on a first depth feature, and determines a first statistical result based on the first classification result, so as to subsequently accurately classify urothelial cells based on the first statistical result.

[0036] S130, obtaining a second depth feature of each second sub-image, determining a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determining a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result.

[0037] Exemplarily, the second sub-image is input into a pre-trained second cell classification model, the second depth feature of the second sub-image is extracted by the second cell classification model, and the second classification result of the second sub-image is determined based on the second depth feature. Wherein, the second cell classification model includes a third feature extraction network and a second cell classification network, the third feature extraction network is used to extract the second depth feature of the second sub-image, the third feature extraction network can be a deep convolutional network or a deep attention network, the second cell classification network can be a fully connected layer, and the second cell classification network is used to determine the second classification result of the second sub-image. In this embodiment, the sub-images divided into by the fluorescence in situ hybridization slice images of each urothelial cell are used as the third sample image in advance, and the categories of the third sample image are marked, and the categories include cell lesions and cells without lesions. The second cell classification model is trained by the third sample image and the corresponding category, so that the trained second cell classification model has the function of classifying the first sub-image. Further, after obtaining the second classification result of each second sub-image, the number of second sub-images of each category in the corresponding fluorescence in situ hybridization slice image is counted according to the second classification result of each second sub-image, and the second statistical result is obtained.

[0038] In another embodiment, cells in the fluorescence in situ hybridization slice image will present a specific fluorescence signal, and the signal colors of diseased cells and non-disease cells are different. In this regard, in order to improve the classification accuracy of the fluorescence in situ hybridization slice image, the fluorescence in situ hybridization slice image can be processed by cell segmentation to determine the segmented cells in the fluorescence in situ hybridization slice image, and the category of the corresponding segmented cells can be determined according to the fluorescence signal characteristics of the segmented cells, and the second statistical result of the fluorescence in situ hybridization slice image can be determined according to the category of each segmented cell. Exemplarily, Figure 4 is a flow chart of determining the second statistical result provided by an embodiment of the present application. Figure 4 As shown, the step of determining the second statistical result specifically includes S1301-S1303: S1301. Input the second depth feature of the second sub-image into a pre-trained cell segmentation network to obtain segmented cells in the second sub-image output by the cell segmentation network.

[0039] Exemplarily, the second sub-image is input into a pre-trained cell segmentation model, the second depth feature of the second sub-image is extracted by the cell segmentation model, and the segmented cells are determined in the second sub-image based on the second depth feature. Wherein, the cell segmentation model includes a fourth feature extraction network and a cell segmentation network, the fourth feature extraction network is used to extract the second depth feature of the second sub-image, the fourth feature extraction network can be a deep convolutional network or a deep attention network, the cell segmentation network can be a fully connected layer, and the cell segmentation network is used to segment cells in the second sub-image. In this embodiment, the sub-images divided in advance by the fluorescent in situ hybridization slice images of each urothelial cell are used as the fourth sample image, the cell mask is marked in the fourth sample image, and the cell segmentation model is trained by the fourth sample image and the corresponding marked cell mask, so that the trained cell segmentation model has the function of performing cell segmentation on the second sub-image.

[0040] It should be noted that the segmented cells output by the cell segmentation network are actually cell masks. The size of the cell mask is the same as that of the second sub-image. The cell mask can be used to filter pixels other than the segmented cells in the second sub-image. That is, after superimposing the second sub-image with the cell mask, an image containing only the segmented cells but with the same size as the second sub-image can be obtained.

[0041] S1302, obtaining fluorescence signal characteristics of the segmented cells in the second sub-image, and determining a second classification result of each segmented cell in the second sub-image according to the fluorescence signal characteristics, wherein the second classification result includes diseased cells and non-lesioned cells.

[0042] Exemplarily, the image obtained by filtering the second sub-image through the cell mask is input into the pre-trained third cell classification model, and the third cell classification model extracts the fluorescence signal features of the segmented cells in the input image, and determines whether the segmented cells are diseased cells or non-disease cells based on the fluorescence signal features. Wherein, the third cell classification model includes a fifth feature extraction network and a third cell classification network, the fifth feature extraction network is used to extract the fluorescence signal features of the segmented cells, the fifth feature extraction network can be a deep convolutional network or a deep attention network, the third cell classification network can be a fully connected layer, and the third cell classification network is used to identify the segmented cells as diseased cells or non-disease cells. In this embodiment, each sub-image processed by the cell mask is used as the fifth sample image in advance, and the category of each segmented cell in the fifth sample image is marked. The third cell classification model is trained by the fifth sample image and the corresponding marked category, so that the trained third cell classification network has the function of classifying the segmented cells.

[0043] S1303 , counting the number of segmented cells corresponding to each second classification result to obtain a second statistical result of the fluorescence in situ hybridization slice image.

[0044] For example, the number of segmented cells with cytopathic results as the second classification is Q, and the number of segmented cells with non-cytopathic results as the second classification is P. Q and P are used to generate a vector As the second statistical result.

[0045] This embodiment uses a pre-trained cell segmentation network to quickly determine the segmented cells in the first sub-image based on the second depth features, determines a second classification result of the segmented cells based on the fluorescence signal characteristics of the segmented cells, and determines a second statistical result based on the second classification result, so as to subsequently accurately classify the urothelial cells based on the second statistical result.

[0046] It should be noted that in the field of urothelial cell classification, there is a lack of technical means for classifying urothelial cells based on fluorescence in situ hybridization sections. This embodiment creatively proposes to classify urothelial cells based on fluorescence in situ hybridization sections while jointly classifying urothelial cells based on liquid-based sections and fluorescence in situ hybridization sections, which effectively improves the classification accuracy of urothelial cells.

[0047] S140: Determine a classification result of urothelial cells according to the first depth feature, the first statistical result, the second depth feature, and the second statistical result.

[0048] Exemplarily, the first depth features of each first sub-image, the first statistical result of the liquid-based slice image, the second depth features of each second sub-image, and the second statistical result of the fluorescence in situ hybridization slice image are fused, and the fusion result is analyzed to determine the classification result of the urothelial cells. It can be understood that the first depth features of each first sub-image and the second depth features of each second sub-image fused in the fusion result respectively reflect the full image information of the liquid-based slice image and the fluorescence in situ hybridization slice image, so the fusion result not only includes the features reflecting the full image information, but also includes the features of different types of digital images and classification statistical results. The fusion result contains rich feature information of urothelial cells. Based on the fusion result, the category of urothelial cells can be accurately analyzed, and the classification accuracy of urothelial cells can be effectively improved.

[0049] In this embodiment, Figure 5 : is a flow chart of determining the classification results of urothelial cells provided in the embodiment of the present application. Figure 5 As shown, the step of determining the classification result of urothelial cells specifically includes S1401-S1403: S1401. Merge the first depth feature, the second depth feature, and the fluorescence signal feature to obtain a fourth depth feature.

[0050] Exemplarily, the second depth feature and the fluorescence signal feature are feature merged, and the merged feature is then feature merged with the first depth feature to obtain a fourth depth feature. The second depth feature and the fluorescence signal feature can be feature merged by channel stacking, that is, if the second depth feature is a feature map of X channels and the fluorescence signal feature is a feature map of Y channels, then the second depth feature and the fluorescence signal feature are feature merged to obtain a new feature map of X+Y channels. The new feature map is feature merged with the first depth feature by channel stacking to obtain a fourth depth feature. If the first depth feature is a feature map of Z channels, then the fourth depth feature is a feature map of X+Y+Z. It should be noted that the first depth feature, the second depth feature, and the fluorescence signal feature have the same size, so these features can be merged by channel stacking.

[0051] S1402: Perform vector merging of the first statistical result and the second statistical result to obtain a first statistical vector.

[0052] For example, the first statistical result And the second statistical results Perform vector merging to obtain the first statistical vector .

[0053] S1403: Determine a classification result of urothelial cells according to the fourth depth feature and the first statistical vector.

[0054] Exemplarily, the fourth depth feature and the first statistical vector are fused, and the classification result of the urothelial cells is determined based on the fusion result. In this embodiment, the features characterizing the first sub-image and the second sub-image are merged to obtain a fourth depth feature containing full image information and different types of image information, and the first statistical result and the second statistical result characterizing the classification of the liquid-based slice image and the fluorescent in situ hybridization slice image are merged to obtain a first statistical vector containing classification statistical results of different types of images. The classification results of the urothelial cells are accurately analyzed based on the first statistical vector and the fourth depth feature to improve the classification accuracy of the urothelial cells.

[0055] In this embodiment, the fourth depth feature and the first statistical vector can be input into a preset gated fusion network to obtain a fusion feature output by the gated fusion network; the fusion feature is input into a preset diagnostic classification network to obtain the classification result of the urothelial cells output by the diagnostic classification network. Among them, the gated fusion network can fuse the corresponding input fourth depth feature and the first statistical vector through a learning gating mechanism, and capture the dependency between the fourth depth feature and the first statistical vector to output a fusion feature that can more accurately characterize the full image information, different types of image information and classification statistical information. The diagnostic classification network can be a fully connected layer, and the fusion feature is input into the diagnostic classification network. The diagnostic classification network can determine that the urothelial cells are diseased cells, transitioning from normal to diseased cells and non-disease cells based on the fusion feature. This embodiment extracts the input features through the gated fusion network and optimizes the combination to improve the comprehensiveness and accuracy of the feature representation, thereby improving the accuracy of the classification. The fusion features are efficiently classified by the diagnostic classification network to further improve the accuracy and efficiency of the classification.

[0056] It should be noted that the process of classifying and processing the urothelial cells in steps S110-S140 when the liquid-based slice images and fluorescent in situ hybridization slice images of the urothelial cells are obtained can be regarded as the joint diagnosis algorithm proposed in this embodiment. However, it is not excluded that in some scenarios, only liquid-based slice images or fluorescent in situ hybridization slice images can be obtained. In this regard, this embodiment proposes that when a single digital image is obtained, the depth features and statistical results of the sub-image corresponding to the digital image are fused with the preset zero features and zero statistical results, and the classification results of the urothelial cells are determined based on the fusion results, so that the urothelial cells can be classified and processed by the joint diagnosis algorithm even when only a single digital image is obtained, thereby improving the applicability of the joint diagnosis algorithm.

[0057] In this embodiment, Figure 6 Schematic diagram of the classification of urothelial cells based on a single digital image provided by the embodiment of the present application. Figure 6 As shown, the step of classifying urothelial cells based on a single digital image specifically includes S210-S240: S210 . When a liquid-based slice image or a fluorescence in situ hybridization slice image of urothelial cells is obtained, divide the liquid-based slice image or the fluorescence in situ hybridization slice image into a plurality of third sub-images.

[0058] Exemplarily, after acquiring a digital image uploaded by a staff member, the computer reduces the digital image to a thumbnail, inputs the thumbnail into a pre-trained image classification model, and identifies the corresponding input thumbnail as a liquid-based slice image or a fluorescent in situ hybridization slice image through the image classification model. The digital image is slidably cropped through a window of a preset size, and the image block cropped by the window each time is used as the third sub-image. After traversing the entire digital image, multiple third sub-images corresponding to the digital image are obtained.

[0059] S220, obtaining the fifth depth feature of each third sub-image, determining a third classification result corresponding to the third sub-image based on the fifth depth feature of each third sub-image, and determining a third statistical result of the liquid-based slice image or the fluorescent in situ hybridization slice image according to each third classification result.

[0060] Exemplarily, if the digital image is a liquid-based slice image, the third sub-image is input into a pre-trained first cell classification model, a fifth depth feature of the third sub-image is obtained through a second feature extraction network in the first cell classification model, and the fifth depth feature is input into a first cell classification network in the first cell classification model, and a third classification result of the third sub-image is determined through the first cell classification network. According to the third classification results of each third sub-image, the number of third sub-images of each category in the corresponding liquid-based slice image is counted to obtain a third statistical result.

[0061] If the digital image is a fluorescence in situ hybridization slice image, the third sub-image is input into a pre-trained cell segmentation model, the fifth depth feature of the fourth sub-image is obtained through the fourth feature extraction network in the cell segmentation model, and the fifth depth feature is input into the cell segmentation network in the cell segmentation model, and the cell mask is output through the cell segmentation network, and the third sub-image is superimposed with the cell mask to obtain an image containing only segmented cells but with the same size as the third sub-image. The image is input into a pre-trained third cell classification model, the fluorescence signal feature of the segmented cells in the image is obtained through the fifth feature extraction network in the third cell classification model, the fluorescence signal feature is input into the third cell classification network in the third cell classification model, and the third classification result of the segmented cells is determined through the third cell classification network. According to the third classification result of each segmented cell, the number of segmented cells of each category in the corresponding fluorescence in situ hybridization slice image is counted to obtain a third statistical result.

[0062] S230: Merge the fifth depth feature with the preset zero feature to obtain a sixth depth feature, and perform vector merging on the third statistical result with the preset zero statistical result to obtain a second statistical vector.

[0063] Exemplarily, based on the channel superposition method, the fifth depth feature and the preset zero feature are feature merged to obtain the sixth depth feature, and the third statistical result and the preset zero statistical result are vector-concatenated to obtain the second statistical vector.

[0064] S240: Determine a classification result of urothelial cells according to the sixth depth feature and the second statistical vector.

[0065] Exemplarily, the sixth depth feature and the second statistical vector are input into a preset gated fusion network to obtain a fusion feature output by the gated fusion network, and the fusion feature is input into a preset diagnostic classification network to obtain a classification result of urothelial cells output by the diagnostic classification network. In this embodiment, when only one digital image is acquired, the features of each sub-image of the digital image are merged with the preset zero feature, and the classification statistical result of the digital image is merged with the preset zero statistical result, so that the classification result of the urothelial cells can be determined based on the merged features and statistical results through the gated fusion network and the diagnostic classification network, so that the joint diagnosis algorithm can also take into account the situation where only a single digital image is acquired, thereby improving the applicability of the joint diagnosis algorithm.

[0066] In order to more intuitively understand the operation process of the joint diagnosis algorithm, this embodiment uses Figure 7 The operation process of the combined diagnosis algorithm shown in FIG. Figure 7As shown, after acquiring a digital image, the digital image is identified as a liquid-based slice image or a fluorescent in situ hybridization slice image by an image classification model. If there is no liquid-based slice image or fluorescent in situ hybridization slice image in the digital image, a preset zero feature and zero statistical result are obtained. The liquid-based slice image is divided into a plurality of first sub-images by a sliding window cropping algorithm, and the first sub-image is input into a first cell classification model to obtain a first depth feature and a first classification result output by the first cell classification model, and a first statistical result is determined according to the first classification result. The fluorescent in situ hybridization slice image is divided into a plurality of second sub-images by a sliding window cropping algorithm, and the second sub-image is input into a cell segmentation model to obtain a second depth feature and segmented cells output by the cell segmentation model, and the segmented cells are input into a third cell classification model to obtain a fluorescent signal feature and a second classification result output by the third cell classification model, and a second statistical result is determined according to the second classification result. In the case that the digital image contains liquid-based slice images and fluorescence in situ hybridization slice images, the first depth feature, the second depth feature and the fluorescence signal feature are feature merged, the first statistical result and the second statistical result are vector merged, the merged features and vectors are input into the gated fusion system, the gated fusion system fuses the input features and vectors to output fusion features, the fusion features are input into the diagnostic classification network, and the diagnostic classification network outputs the classification results of urothelial cells. In the case that the digital image does not contain liquid-based slice images or fluorescence in situ hybridization slice images, the first depth feature or the second depth feature and the fluorescence signal feature are feature merged with the pre-acquired zero feature, the first statistical result or the second statistical result is vector merged with the pre-acquired zero statistical result, the merged features and vectors are input into the gated fusion system, the gated fusion system fuses the input features and vectors to output fusion features, the fusion features are input into the diagnostic classification network, and the diagnostic classification network outputs the classification results of urothelial cells. Figure 7 It can be seen from the operation process shown that no matter whether the computer acquires one digital image or two digital images, it can eventually generate features and vectors for input into the gated fusion network, so as to determine the classification results of urothelial cells through the gated fusion network and the diagnostic classification network, so that the combined diagnosis algorithm can also take into account the situation where only a single digital image is acquired, thereby improving the applicability of the combined diagnosis algorithm.

[0067] In summary, the combined classification method of urothelial cells provided in the embodiments of the present application obtains liquid-based slice images and fluorescence in situ hybridization slice images of urothelial cells, divides the liquid-based slice images into multiple first sub-images, and divides the fluorescence in situ hybridization slice images into multiple second sub-images; obtains a first depth feature of each first sub-image, determines a first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determines a first statistical result of the liquid-based slice image according to each first classification result; obtains a second depth feature of each second sub-image, determines a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determines a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; determines the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature, and the second statistical result. Through the above-mentioned technical means, the features of each sub-image in the liquid-based section image and the fluorescence in situ hybridization section image of the urothelial cells and the classification statistical results can be fused. The fusion result not only includes the features that can reflect the information of the whole image, but also includes the features of different types of images and classification statistical results. The fusion result contains rich characteristic information of the urothelial cells. The category of the urothelial cells can be accurately analyzed based on the fusion result, which solves the problem of low classification accuracy of the urothelial cells caused by inferring the category of the urothelial cells only by the sub-image category in the prior art, and effectively improves the classification accuracy of the urothelial cells.

[0068] Based on the above embodiments, Figure 8 This is a schematic diagram of the structure of a urothelial cell combined classification device provided in an embodiment of the present application. Figure 8 The urothelial cell joint classification device provided in this embodiment specifically includes: a first image segmentation module 31, a first image processing module 32, a second image processing module 33 and a first cell classification module 34.

[0069] The first image segmentation module 31 is configured to obtain a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, divide the liquid-based slice image into a plurality of first sub-images, and divide the fluorescence in situ hybridization slice image into a plurality of second sub-images; The first image processing module 32 is configured to obtain a first depth feature of each first sub-image, determine a first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determine a first statistical result of the liquid-based slice image according to each first classification result; The second image processing module 33 is configured to obtain a second depth feature of each second sub-image, determine a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determine a second statistical result of the fluorescent in situ hybridization slice image according to each second classification result; The first cell classification module 34 is configured to determine the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature and the second statistical result.

[0070] Based on the above embodiment, the first image segmentation module 31 includes: a scaling unit, configured to acquire two digital images of urothelial cells and reduce each digital image to a thumbnail; an image recognition unit, configured to acquire a third depth feature of each thumbnail and identify a liquid-based slice image and a fluorescent in situ hybridization slice image in the two digital images based on each third depth feature.

[0071] Based on the above embodiment, the first image processing module 32 includes: a first classification unit, configured to input the first depth feature of the first sub-image into a pre-trained first cell classification network, and obtain a first classification result of the first sub-image output by the first cell classification network, and the first classification result is cell lesion, cell transition or cell non-lesion; a first statistical unit, configured to count the number of first sub-images corresponding to each first classification result, and obtain a first statistical result of the liquid-based slice image.

[0072] On the basis of the above embodiment, the second image processing module 33 includes: a segmentation unit, configured to input the second depth feature of the second sub-image into a pre-trained cell segmentation network to obtain segmented cells in the second sub-image output by the cell segmentation network; a second classification unit, configured to obtain the fluorescence signal characteristics of the segmented cells in the second sub-image, and determine the second classification results of each segmented cell in the second sub-image according to the fluorescence signal characteristics, and the second classification results include lesions and non-lesions; a second statistical unit, configured to count the number of segmented cells corresponding to each second classification result, and obtain the second statistical result of the fluorescence in situ hybridization slice image.

[0073] Based on the above embodiment, the first cell classification module 34 includes: a feature merging unit, configured to perform feature merging on the first depth feature, the second depth feature and the fluorescence signal feature to obtain a fourth depth feature; a statistical merging unit, configured to perform vector merging on the first statistical result and the second statistical result to obtain a first statistical vector; and a third classification unit, configured to determine the classification result of the urothelial cells based on the fourth depth feature and the first statistical vector.

[0074] Based on the above embodiment, the third classification unit includes: a feature fusion subunit, configured to input the fourth depth feature and the first statistical vector into a preset gated fusion network to obtain a fusion feature output by the gated fusion network; a cell classification subunit, configured to input the fusion feature into a preset diagnostic classification network to obtain a classification result of urothelial cells output by the diagnostic classification network.

[0075] On the basis of the above embodiments, the urothelial cell joint classification device also includes: a second image division module, configured to divide the liquid-based slice image or the fluorescent in situ hybridization slice image of the urothelial cells into multiple third sub-images when the liquid-based slice image or the fluorescent in situ hybridization slice image of the urothelial cells is obtained; a third image processing module, configured to obtain the fifth depth feature of each third sub-image, determine the third classification result of the corresponding third sub-image based on the fifth depth feature of each third sub-image, and determine the third statistical result of the liquid-based slice image or the fluorescent in situ hybridization slice image according to each third classification result; a merging processing module, configured to perform feature merging of the fifth depth feature and the preset zero feature to obtain a sixth depth feature, and perform vector merging of the third statistical result and the preset zero statistical result to obtain a second statistical vector; a second cell classification module, configured to determine the classification result of the urothelial cells based on the sixth depth feature and the second statistical vector.

[0076] As described above, the urothelial cell joint classification device provided in the embodiment of the present application obtains the liquid-based slice image and the fluorescence in situ hybridization slice image of the urothelial cells, divides the liquid-based slice image into multiple first sub-images, and divides the fluorescence in situ hybridization slice image into multiple second sub-images; obtains the first depth feature of each first sub-image, determines the first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determines the first statistical result of the liquid-based slice image according to each first classification result; obtains the second depth feature of each second sub-image, determines the second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determines the second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; determines the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature, and the second statistical result. Through the above-mentioned technical means, the features of each sub-image in the liquid-based section image and the fluorescence in situ hybridization section image of the urothelial cells and the classification statistical results can be fused. The fusion result not only includes the features that can reflect the information of the whole image, but also includes the features of different types of images and classification statistical results. The fusion result contains rich characteristic information of the urothelial cells. The category of the urothelial cells can be accurately analyzed based on the fusion result, which solves the problem of low classification accuracy of the urothelial cells caused by inferring the category of the urothelial cells only by the sub-image category in the prior art, and effectively improves the classification accuracy of the urothelial cells.

[0077] The urothelial cell joint classification device provided in the embodiment of the present application can be used to execute the urothelial cell joint classification method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0078] Fig. 9 is a schematic diagram of the structure of a urothelial cell combined classification device provided in an embodiment of the present application, with reference to Fig. 9 The urothelial cell joint classification device includes: a processor 41, a memory 42, a communication device 43, an input device 44, and an output device 45. The number of processors 41 in the urothelial cell joint classification device can be one or more, and the number of memories 42 in the urothelial cell joint classification device can be one or more. The processor 41, memory 42, communication device 43, input device 44, and output device 45 of the urothelial cell joint classification device can be connected via a bus or other means.

[0079] The memory 42, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the urothelial cell joint classification method of any embodiment of the present application (for example, the first image segmentation module 31, the first image processing module 32, the second image processing module 33 and the first cell classification module 34). The memory 42 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0080] The communication device 43 is used for data transmission.

[0081] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 42, that is, realizes the above-mentioned urothelial cell joint classification method.

[0082] The input device 44 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 45 may include a display device such as a display screen.

[0083] The above-mentioned urothelial cell joint classification device can be used to execute the urothelial cell joint classification method provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0084] An embodiment of the present application also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute a method for joint classification of urothelial cells. The method for joint classification of urothelial cells includes: obtaining a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, dividing the liquid-based slice image into a plurality of first sub-images, and dividing the fluorescence in situ hybridization slice image into a plurality of second sub-images; obtaining a first depth feature of each first sub-image, determining a first classification result corresponding to the first sub-image based on the first depth feature of each first sub-image, and determining a first statistical result of the liquid-based slice image according to each first classification result; obtaining a second depth feature of each second sub-image, determining a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determining a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; determining the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature, and the second statistical result.

[0085] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0086] Of course, the storage medium containing computer executable instructions provided in the embodiments of the present application is not limited to the above-mentioned urothelial cell joint classification method, and the computer executable instructions can also execute the related operations in the urothelial cell joint classification method provided in any embodiment of the present application.

[0087] The urothelial cell joint classification device, urothelial cell joint classification system, storage medium and urothelial cell joint classification equipment provided in the above embodiments can execute the urothelial cell joint classification method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the urothelial cell joint classification method provided in any embodiment of the present application.

[0088] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A combined classification method for urothelial cells, characterized in that: include: Acquire a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, divide the liquid-based slice image into a plurality of first sub-images, and divide the fluorescence in situ hybridization slice image into a plurality of second sub-images; Acquire a first depth feature of each of the first sub-images, determine a first classification result corresponding to the first sub-image based on the first depth feature of each of the first sub-images, and determine a first statistical result of the liquid-based slice image according to each of the first classification results; Acquire a second depth feature of each second sub-image, determine a second classification result corresponding to the second sub-image based on the second depth feature of each second sub-image, and determine a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result; A classification result of the urothelial cells is determined based on the first depth feature, the first statistical result, the second depth feature, and the second statistical result.

2. The combined classification method of urothelial cells according to claim 1, characterized in that: The step of determining a second classification result corresponding to each second sub-image based on the second depth feature of each second sub-image, and determining a second statistical result of the fluorescence in situ hybridization slice image according to each second classification result, comprises: Inputting the second depth feature of the second sub-image into a pre-trained cell segmentation network to obtain segmented cells in the second sub-image output by the cell segmentation network; Acquire fluorescence signal features of the segmented cells in the second sub-image, and determine a second classification result of each segmented cell in the second sub-image according to the fluorescence signal features, wherein the second classification result includes a lesion and a non-lesion; The number of segmented cells corresponding to each of the second classification results is counted to obtain a second statistical result of the fluorescence in situ hybridization slice image.

3. The combined classification method of urothelial cells according to claim 2, characterized in that: Determining the classification result of the urothelial cells according to the first depth feature, the first statistical result, the second depth feature, and the second statistical result includes: Merging the first depth feature, the second depth feature, and the fluorescence signal feature to obtain a fourth depth feature; Performing vector merging of the first statistical result and the second statistical result to obtain a first statistical vector; A classification result of the urothelial cells is determined according to the fourth depth feature and the first statistical vector.

4. The combined classification method of urothelial cells according to claim 3, characterized in that: Determining the classification result of the urothelial cells according to the fourth depth feature and the first statistical vector includes: Inputting the fourth deep feature and the first statistical vector into a preset gated fusion network to obtain a fusion feature output by the gated fusion network; The fusion feature is input into a preset diagnosis classification network to obtain a classification result of urothelial cells output by the diagnosis classification network.

5. The combined classification method of urothelial cells according to claim 1, characterized in that: The determining a first classification result corresponding to the first sub-image based on the first depth feature of each of the first sub-images, and determining a first statistical result of the liquid-based slice image according to each of the first classification results, includes: Inputting a first depth feature of the first sub-image into a pre-trained first cell classification network to obtain a first classification result of the first sub-image output by the first cell classification network, wherein the first classification result is cell pathological changes, cell transition, or cell non-pathological changes; The number of first sub-images corresponding to each of the first classification results is counted to obtain a first statistical result of the liquid-based slice image.

6. The combined classification method of urothelial cells according to claim 1, characterized in that: The method further comprises: When a liquid-based slice image or a fluorescent in situ hybridization slice image of urothelial cells is obtained, dividing the liquid-based slice image or the fluorescent in situ hybridization slice image into a plurality of third sub-images; Acquire a fifth depth feature of each of the third sub-images, determine a third classification result corresponding to the third sub-image based on the fifth depth feature of each of the third sub-images, and determine a third statistical result of the liquid-based slice image or the fluorescent in situ hybridization slice image according to each of the third classification results; Performing feature merging of the fifth depth feature with a preset zero feature to obtain a sixth depth feature, and performing vector merging of the third statistical result with a preset zero statistical result to obtain a second statistical vector; A classification result of the urothelial cells is determined according to the sixth depth feature and the second statistical vector.

7. The combined classification method of urothelial cells according to claim 1, characterized in that: The method of obtaining liquid-based slice images and fluorescent in situ hybridization slice images of urothelial cells comprises: Acquire two digital images of the urothelial cells, and reduce each of the digital images to a thumbnail; A third depth feature of each of the thumbnails is obtained, and a liquid-based slice image and a fluorescent in situ hybridization slice image are identified in the two digital images based on each of the third depth features.

8. A combined urothelial cell classification device, characterized in that: include: A first image division module is configured to obtain a liquid-based slice image and a fluorescence in situ hybridization slice image of urothelial cells, divide the liquid-based slice image into a plurality of first sub-images, and divide the fluorescence in situ hybridization slice image into a plurality of second sub-images; A first image processing module is configured to obtain a first depth feature of each of the first sub-images, determine a first classification result corresponding to the first sub-image based on the first depth feature of each of the first sub-images, and determine a first statistical result of the liquid-based slice image according to each of the first classification results; a second image processing module, configured to obtain a second depth feature of each of the second sub-images, determine a second classification result corresponding to the second sub-image based on the second depth feature of each of the second sub-images, and determine a second statistical result of the fluorescence in situ hybridization slice image according to each of the second classification results; The first cell classification module is configured to determine the classification result of the urothelial cells based on the first depth feature, the first statistical result, the second depth feature and the second statistical result.

9. A combined urothelial cell classification device, characterized in that: include: one or more processors; A storage device stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the urothelial cell joint classification method as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the urothelial cell joint classification method as described in any one of claims 1-7 when executed by a computer processor.

Images