Nasal polyp pathological section image recognition method and device, electronic equipment and storage medium
By employing domain generalization models, cell classification models, and tissue region segmentation models, the problem of insufficient cell-level accuracy in nasal polyp pathological slide image recognition in existing technologies has been solved, achieving highly accurate identification of nasal polyp inflammation types.
Patent Information
- Application Number
- CN202610087406.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-22
- Publication Date
- 2026-05-08
AI Technical Summary
Existing AI-based methods for identifying nasal polyps are mainly used to identify pathological subtypes, lacking precise identification at the cellular level and having insufficient ability to identify densely cellular regions.
A domain generalization model was used to generalize the pathological slide images of nasal polyps. Combined with a cell classification model and a tissue region segmentation model, cell types and region areas were statistically analyzed to determine the inflammation type of nasal polyps.
It improves the accuracy and generalization of identifying nasal polyp inflammation types, enabling the identification of images from different source domains and enhancing the results of cell category classification and tissue region segmentation.
Smart Images
Figure CN121999285A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of artificial intelligence and image processing technology. Specifically, it relates to methods, apparatus, electronic devices, and storage media for recognizing nasal polyp pathological slide images. Background Technology
[0002] Nasal polyps are a common nasal disease, formed by the gradual drooping of extremely edematous nasal and sinus mucosa under the influence of gravity. Nasal polyps are mainly detected through three methods: (1) Nasal endoscopy: observing viscous or mucopurulent secretions originating from the middle meatus and olfactory cleft, nasal mucosal congestion, edema, or polyps. This method is usually only used as a preliminary screening method. (2) Imaging examination: sinus CT scans can show inflammatory lesions of the ostiomeatal complex and / or sinus mucosa. (3) Pathological examination: is the gold standard of various examinations and can specifically determine the pathological type of nasal polyps.
[0003] With the advancement of artificial intelligence and advanced image processing technologies, AI-assisted diagnostic technologies based on digital images are innovatively integrated and developed across various medical specialties. AI technology has gradually become a powerful auxiliary tool for accurate diagnosis in the medical field. Digital pathology is an important component of AI-assisted diagnosis, which involves using digital pathology scanners to convert physical pathological tissue slides into digital pathological slides, which are then viewed and analyzed on a monitor using software. Common digital pathology image analysis tasks include the following four main categories: 1) Slide classification: such as molecular subtyping of endometrial cancer; 2) Target detection: such as thin-layer liquid-based cytology (TCT) lesion cell detection; 3) Region segmentation: such as glandular segmentation; 4) Cell detection: such as PD-L1 scoring of tumor cells, etc.
[0004] However, existing AI-based methods for identifying nasal polyps are mainly used to identify pathological subtypes, lacking precise identification at the cellular level, and due to algorithmic limitations, they are insufficient in identifying densely cellular regions. Summary of the Invention
[0005] This disclosure provides a method, apparatus, electronic device, and storage medium for recognizing nasal polyp pathological slide images.
[0006] According to one aspect of this disclosure, a method for recognizing nasal polyp pathological section images is provided, comprising: A domain generalization model is used to generalize the pathological slice image of nasal polyps in the first field of view of the first detection object to obtain the domain-invariant feature map of nasal polyps pathology. A cell classification model was used to classify the cell types of the invariant feature map of the nasal polyp pathological domain to obtain the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. A tissue region segmentation model was used to segment the invariant feature map of the nasal polyp pathological domain to obtain the epithelial region, vascular region and glandular region in the invariant feature map of the nasal polyp pathological domain. Based on the cell type of each cell in the invariant feature map of the nasal polyp pathological domain, the number of cells of each cell type in the first field of view is counted. Based on the epithelial region, vascular region, and glandular region in the invariant feature map of the pathological domain of nasal polyps, the area of the epithelial region, vascular region, and glandular region under the first field of view are statistically analyzed. Based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region, the nasal polyp inflammation type of the first test subject is determined.
[0007] According to another aspect of this disclosure, a device for recognizing nasal polyp pathological slide images is provided, comprising: The domain generalization module is used to perform domain generalization on the nasal polyp pathological slice image under the first field of view of the first detection object using the domain generalization model, so as to obtain the nasal polyp pathological domain invariant feature map. The cell classification module is used to classify the cell categories of the invariant feature map of the nasal polyp pathological domain using a cell classification model, so as to obtain the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The region segmentation module is used to perform tissue region segmentation on the invariant feature map of the nasal polyp pathological domain using a tissue region segmentation model, so as to obtain the epithelial region, blood vessel region and glandular region in the invariant feature map of the nasal polyp pathological domain. The first statistical module is used to count the number of cells of each cell type in the first field of view based on the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The second statistical module is used to calculate the area of the epithelial region, the area of the vascular region, and the area of the glandular region in the first field of view based on the epithelial region, the vascular region, and the glandular region in the invariant feature map of the nasal polyp pathological domain. The inflammation type determination module is used to determine the nasal polyp inflammation type of the first detection object based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region.
[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and The memory is communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the nasal polyp pathological slide image recognition methods in the embodiments of this disclosure.
[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the nasal polyp pathological slide image recognition methods according to embodiments of this disclosure.
[0010] According to the technology disclosed herein, a domain generalization model is used to generalize the nasal polyp pathological slide image of the first detection object under the first field of view, obtaining a domain-invariant feature map of nasal polyp pathology; a cell classification model is used to classify the cell types in the domain-invariant feature map of nasal polyp pathology, obtaining the cell types of each cell in the domain-invariant feature map of nasal polyp pathology; a tissue region segmentation model is used to segment the tissue regions in the domain-invariant feature map of nasal polyp pathology, obtaining the epithelial region, vascular region, and glandular region in the domain-invariant feature map of nasal polyp pathology; based on the cell types of each cell in the domain-invariant feature map of nasal polyp pathology, the number of cells of each cell type under the first field of view is counted; based on the epithelial region, vascular region, and glandular region in the domain-invariant feature map of nasal polyp pathology, the area of the epithelial region, the area of the vascular region, and the area of the glandular region under the first field of view are counted; based on the number of cells of each cell type under the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region, the inflammation type of the nasal polyp of the first detection object is determined. By first performing domain generalization on the nasal polyp pathological slide image, and then performing cell type classification and tissue region segmentation, images from different source domains can be identified. Furthermore, by using the cell classification results and tissue region segmentation results of nasal polyps in the same field of view, the inflammation type of nasal polyps in the tested subjects can be distinguished, which can improve the accuracy of identifying the inflammation type of nasal polyps.
[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a flowchart of a method for recognizing nasal polyp pathological slide images according to an embodiment of the present disclosure; Figure 2 This is a flowchart of the training and testing process of a domain generalization model according to an embodiment of the present disclosure; Figure 3This is a schematic diagram of a cell classification model according to an embodiment of the present disclosure; Figure 4 This is a structural block diagram of a nasal polyp pathological slide image recognition device according to an embodiment of the present disclosure; Figure 5 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0013] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0014] Figure 1 This is a flowchart of a method for recognizing nasal polyp pathological slide images according to an embodiment of the present invention.
[0015] like Figure 1 As shown, the method for recognizing nasal polyp pathological section images may include: S110, using a domain generalization model, performs domain generalization on the pathological slice image of nasal polyps in the first field of view of the first detection object to obtain the domain-invariant feature map of nasal polyps pathology. S120, using a cell classification model, classifies the cell types of each cell in the invariant feature map of the pathological domain of nasal polyps, and obtains the cell type of each cell in the invariant feature map of the pathological domain of nasal polyps. S130, using a tissue region segmentation model, the invariant feature map of the nasal polyp pathological domain is segmented into tissue regions to obtain the epithelial region, vascular region and glandular region in the invariant feature map of the nasal polyp pathological domain. S140, based on the cell types of each cell in the invariant feature map of the pathological domain of nasal polyps, count the number of cells of each cell type in the first field of view; S150, based on the epithelial region, vascular region and glandular region in the pathological domain invariant feature map of nasal polyps, the area of the epithelial region, vascular region and glandular region in the first field of view are statistically analyzed. S160, based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region, the nasal polyp inflammation type of the first test subject is determined.
[0016] For example, the domain generalization model includes a first encoder composed of a backbone network and a first decoder. The domain generalization model can be trained using nasal polyp pathological slide image samples from different source domains, so that the trained domain generalization model can extract domain-invariant features from nasal polyp pathological slide image samples from different source domains.
[0017] For example, the cell classification model can use a point-to-point network (P2PNet) or a network structure improved from a point-to-point network. For instance, during training, the learning rate can be set to 0.0001, the batch size to 4, and 500 training iterations can be performed.
[0018] For example, the organization region segmentation model can be a DeepLabv3+ model, or a model improved from the DeepLabv3+ model. For instance, during training, the learning rate can be set to 0.007, the batch size to 4, and 500 training rounds can be performed.
[0019] For example, preprocessing of training samples may include scaling the training samples according to the ratio of the distance per pixel (MPP) at the current magnification to the distance per pixel at the target, and then using the scaled images for training. For instance, for a 40x magnification nasal polyp pathology whole slide image with a magnification of 0.11 and a 0.25x magnification nasal polyp pathology whole slide digital pathology block image, the scaling ratio is calculated as nasal polyp pathology whole slide image / nasal polyp pathology whole slide small square image. The whole slide digital pathology image is then pre-cropped to 3840×2160 pixels and resized using linear interpolation to generate a nasal polyp pathology whole slide block image with a resolution of 3840×2160 and a distance per pixel of 0.25.
[0020] For example, cell types may include plasma cells, neutrophils, eosinophils, and lymphocytes.
[0021] For example, the first field of view can be 40× magnification and 3840×2160 resolution, or 20× magnification and 3840×2160 resolution.
[0022] For example, based on the number of cells of each cell type in the first field of view, the proportion of each cell type in the first field of view is counted; based on the area of the epithelial region, the area of the vascular region, and the area of the glandular region in the first field of view, the proportion of the epithelial region, the proportion of the vascular region, and the proportion of the glandular region in the first field of view are counted. Thus, by using the number and proportion of cells of each cell type in the first field of view, the area and proportion of the epithelial region, the area and proportion of the vascular region, and the area and proportion of the glandular region, the nasal polyp inflammation type of the first test subject is determined.
[0023] For example, if there are more than 10 eosinophils under high power field (HPF: 400x), the nasal polyp inflammation type of the first test subject can be identified as eosinophilic nasal polyp inflammation type, and the recurrence rate is the highest.
[0024] For example, nasal polyp inflammation types can be classified according to the predominance of cell type, and recurrence rates can be determined using the area and proportion of epithelial regions, vascular regions, and glandular regions. For instance, a small epithelial region combined with a large glandular region results in a high recurrence rate.
[0025] According to the above implementation method, domain generalization is first performed on the pathological slide images of nasal polyps, followed by cell type classification and tissue region segmentation. This allows for the identification of images from different source domains, improving the generalization of nasal polyp inflammation type identification. Furthermore, by using the cell type classification results and tissue region segmentation results of nasal polyps in the same field of view to distinguish the nasal polyp inflammation type of the test subject, the accuracy of nasal polyp inflammation type identification can be improved.
[0026] In one implementation, the domain generalization model includes a first encoder composed of a backbone network and a first decoder. The method includes: inputting a nasal polyp pathological slice image sample from a second field of view of a second detection object into the backbone network to obtain low-order features provided by a second network and high-order features provided by a fifth network in the backbone network, wherein the backbone network includes five networks with decreasing network sizes from input to output; performing random convolution on the low-order features to obtain perturbation features; extracting features from the perturbation features through a leaky hybrid network to obtain enhanced features; concatenating the high-order features with the enhanced features and inputting the concatenated features into the first decoder to obtain a predicted domain-invariant feature map output by the first decoder; and adjusting the network parameters of the backbone network, the first decoder, and the leaky hybrid network based on the difference between the predicted domain-invariant feature map and the true domain-invariant feature map corresponding to the nasal polyp pathological slice image sample.
[0027] For example, a receptive field attention convolutional network can be used to perform random convolution on low-order features to obtain perturbed features.
[0028] like Figure 2 As shown, the blue arrows represent the training process. For multiple nasal polyp pathological slice image samples from different source domains, the domain generalization model, the field-attention convolutional network, and the leakage hybrid network are trained. Each training iteration can execute the above steps until the training objective or the required number of training iterations is reached. Then, step S110 is executed using the domain generalization model. The red arrows represent the testing or application process of the domain generalization model.
[0029] According to the above implementation method, by using a field-attention convolutional network and a leakage hybrid network, and training a domain generalization model with multiple nasal polyp pathological slice image samples from different source domains, the domain generalization model can extract domain-invariant features from multiple nasal polyp pathological slice image samples from different source domains, thereby improving the generalization of nasal polyp inflammation type identification.
[0030] In one implementation, the method further includes: using a classifier to perform domain classification on high-order features to obtain the predicted domain source of the nasal polyp pathological slide image sample; and adjusting the network parameters of the classifier and the first encoder based on the difference between the predicted domain source and the corresponding true domain source of the nasal polyp pathological slide image sample.
[0031] like Figure 2 As shown, a classifier can also be used to classify the higher-order features into domains, obtaining the predicted domain source of the nasal polyp pathological slide image sample, for example, any one of A, B, C, and D. Then, the network parameters of the classifier and the first encoder are adjusted using the difference between the predicted domain source and the corresponding true domain source of the nasal polyp pathological slide image sample. This adjustment process can be part of the training process described above.
[0032] According to the above implementation method, a classifier is used to perform domain classification on the high-order features to obtain the predicted domain source of the nasal polyp pathological slice image sample. Based on the difference between the predicted domain source of the nasal polyp pathological slice image sample and the corresponding real domain source, the network parameters of the classifier and the first encoder are adjusted, which can improve the accuracy of the encoder in the domain generalization model.
[0033] In one implementation, a domain generalization model is used to generalize the nasal polyp pathological slice image under the first field of view of the first detection object to obtain a domain-invariant feature map of nasal polyp pathology. This includes: performing pixel clustering on the nasal polyp pathological slice image under the first field of view of the detection object to obtain foreground cell pixel regions and background fragment pixel regions; segmenting the nasal polyp pathological slice image into multiple rectangular images; extracting rectangular images that intersect with the foreground cell pixel regions from the multiple rectangular images, and using the extracted rectangular images to constitute the foreground cell image region; and using a domain generalization model to generalize the foreground cell image region to obtain a domain-invariant feature map of nasal polyp pathology.
[0034] Understandably, a typical pathological section contains three main components: background, debris, and cellular regions. A significant characteristic of cellular regions is that the hue of each pixel differs from that of the background and debris regions. Therefore, a clustering approach can be used to cluster the individual pixels in a nasal polyp pathological section image, resulting in foreground cellular pixel regions and background debris pixel regions.
[0035] For example, the k-means algorithm can be used to cluster the pixels in the pathological slice image of nasal polyps to obtain the foreground cell pixel region and the background fragment pixel region.
[0036] For example, a sliding window method is used to segment the nasal polyp pathological slide image into multiple rectangular images. Then, it is determined whether each rectangular image intersects with the foreground cell pixel region, and all intersecting rectangular images are extracted to form the foreground cell image region. Rectangular images that do not intersect with the foreground cell pixel region are discarded.
[0037] According to the above implementation method, the foreground cell pixel region and the background fragment pixel region can be separated first by clustering. Then, the multiple rectangular images obtained from the segmentation of the nasal polyp pathological slide image are compared with the foreground cell pixel region to determine whether they intersect. The rectangular images that intersect with the foreground cell pixel region are extracted, and the extracted rectangular images constitute the foreground cell image region. In this way, the interference of brightness and saturation changes caused by different whole-slide scanners can be reduced, and the accuracy of the subsequent domain generalization model in extracting domain-invariant features can be improved.
[0038] In one implementation, a domain generalization model is used to generalize the foreground cell image region to obtain a domain-invariant feature map of nasal polyps pathology. This includes: inputting the foreground cell image region into a first encoder in the domain generalization model to obtain high-order features output by the first encoder; and inputting the high-order features into a first decoder in the domain generalization model to obtain a domain-invariant feature map of nasal polyps pathology output by the first decoder.
[0039] In this example, such as Figure 2 The red arrows indicate the data processing flow, through which the pathological domain invariant feature map of nasal polyps can be obtained.
[0040] In one implementation, such as Figure 3As shown, the cell classification model includes a feature extraction module, an attention module, a linear regression module, a linear classification module, and a one-to-one matching module. The feature extraction module, comprising a feature pyramid network, uses the cell classification model to classify cell types from the invariant feature map of the nasal polyp pathology domain, obtaining the cell type of each cell in the invariant feature map. This includes: inputting the invariant feature map of the nasal polyp pathology domain into the feature extraction module to obtain the features of each pyramid layer output by the feature pyramid network; inputting the features of each pyramid layer into the attention module to obtain the attention aggregation features output by the attention module; inputting the attention aggregation features into the linear regression module to obtain the linear regression features of each cell output by the linear regression module; inputting the attention aggregation features into the linear classification module to obtain the linear classification features of each cell output by the linear classification module; convolving the linear regression features and linear classification features of each cell to obtain the convolutional features of each cell; and using the one-to-one matching module to perform one-to-one matching and classification of the convolutional features of each cell to obtain the cell type of each cell.
[0041] For example, the feature extraction module includes a residual network and a feature pyramid network, so that features of each pyramid layer can be extracted.
[0042] For example, the attention module can further remove background region features from the pathological domain invariant feature map of nasal polyps, leaving only cellular region features.
[0043] For example, the one-to-one matching module can use the Hungarian one-to-one matching algorithm to predict each point (cell point) in the feature map, and discard redundant points directly to avoid duplicate detection.
[0044] According to the above implementation method, the invariant feature map of the nasal polyp pathological domain is processed sequentially by the feature extraction module, attention module, linear regression module, linear classification module and one-to-one matching module, so that the cell type of each cell in the map can be accurately obtained.
[0045] Figure 4 This is a structural block diagram of a nasal polyp pathological slide image recognition device according to an embodiment of the present disclosure.
[0046] like Figure 4 As shown, the nasal polyp pathology slide image recognition device includes: The domain generalization module 410 is used to perform domain generalization on the nasal polyp pathological slice image under the first field of view of the first detection object using a domain generalization model, so as to obtain the nasal polyp pathological domain invariant feature map. The cell classification module 420 is used to classify the cell categories of the invariant feature map of the nasal polyp pathological domain using a cell classification model, so as to obtain the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The region segmentation module 430 is used to perform tissue region segmentation on the invariant feature map of the nasal polyp pathological domain using a tissue region segmentation model, so as to obtain the epithelial region, blood vessel region and glandular region in the invariant feature map of the nasal polyp pathological domain. The first statistical module 440 is used to count the number of cells of each cell type in the first field of view based on the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The second statistical module 450 is used to calculate the area of the epithelial region, the area of the vascular region, and the area of the glandular region in the first field of view based on the epithelial region, the vascular region, and the glandular region in the invariant feature map of the nasal polyp pathological domain. The inflammation type determination module 460 is used to determine the nasal polyp inflammation type of the first detection object based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region.
[0047] In one embodiment, the domain generalization model includes a first encoder composed of a backbone network and a first decoder, and the device further includes a domain generalization model training module, specifically used for: The pathological slice image sample of nasal polyps under the second field of view of the second detection object is input into the backbone network to obtain the low-order features provided by the second network and the high-order features provided by the fifth network in the backbone network. The backbone network includes 5 networks from input to output with decreasing network size. Random convolution is performed on the low-order features to obtain perturbation features; Enhanced features are obtained by extracting features from the perturbation features using a leaky hybrid network; The higher-order features are concatenated with the enhanced features, and the resulting features are input into the first decoder to obtain the prediction domain invariant feature map output by the first decoder. Based on the difference between the predicted domain invariant feature map and the real domain invariant feature map corresponding to the nasal polyp pathological slice image sample, the network parameters of the backbone network, the first decoder and the leaky hybrid network are adjusted.
[0048] In one implementation, the domain generalization model training module is further configured to: A classifier is used to perform domain classification on the higher-order features to obtain the predicted domain source of the nasal polyp pathological slice image sample; Based on the difference between the predicted domain source and the corresponding real domain source of the nasal polyp pathological slice image sample, the network parameters of the classifier and the first encoder are adjusted.
[0049] In one implementation, the domain generalization module 410 is specifically used for: Pixel clustering is performed on the pathological section image of nasal polyps in the first field of view of the detected object to obtain the foreground cell pixel region and the background debris pixel region; The pathological section image of the nasal polyp was segmented into multiple rectangular images; Extract the rectangular images that intersect with the foreground cell pixel region from the plurality of rectangular images, and use the extracted rectangular images to form the foreground cell image region; A domain generalization model is used to generalize the foreground cell image region to obtain the domain-invariant feature map of the nasal polyp pathology.
[0050] In one implementation, the step of employing a domain generalization model to generalize the foreground cell image region to obtain the domain-invariant feature map of the nasal polyp pathology includes: The foreground cell image region is input into the first encoder in the domain generalization model to obtain the high-order features output by the first encoder. The higher-order features are input into the first decoder in the domain generalization model to obtain the pathological domain invariant feature map of nasal polyps output by the first decoder.
[0051] In one embodiment, the cell classification model includes a feature extraction module, an attention module, a linear regression module, a linear classification module, and a one-to-one matching module, wherein the feature extraction module includes a feature pyramid network, and the cell classification module 420 is specifically used for: The pathological domain invariant feature map of the nasal polyp is input into the feature extraction module to obtain the features of each pyramid layer output by the feature pyramid network in the feature extraction module; Each of the tower layer features is input into the attention module to obtain the attention aggregation features output by the attention module; The attention aggregation feature is input into the linear regression module to obtain the linear regression feature of each cell output by the linear regression module; The attention aggregation feature is input into the linear classification module to obtain the linear classification feature of each cell output by the linear classification module; The linear regression features and linear classification features of each cell are convolved to obtain the convolutional features of each cell. The convolutional features of each cell are then matched and classified one-to-one by the one-to-one matching module to obtain the cell type of each cell.
[0052] In one implementation, the tissue region segmentation model uses the DeepLabv3+ model.
[0053] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.
[0054] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0055] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0056] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Figure 5 As shown, the electronic device includes a memory 410 and a processor 420. The memory 410 stores a computer program that can run on the processor 420. There can be one or more memories 410 and processors 420. The memory 410 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the methods provided in the above-described method embodiments. The electronic device may also include a communication interface 430 for communicating with external devices and performing data exchange and transmission.
[0057] If the memory 410, processor 420, and communication interface 430 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0058] Optionally, in a specific implementation, if the memory 410, processor 420 and communication interface 430 are integrated on a single chip, the memory 410, processor 420 and communication interface 430 can communicate with each other through an internal interface.
[0059] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.
[0060] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).
[0061] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure may be non-volatile storage media; in other words, they may be non-transient storage media.
[0062] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0063] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0064] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.
[0065] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.
[0066] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.
Claims
1. A method for recognizing pathological sections of nasal polyps, characterized in that, include: A domain generalization model is used to generalize the pathological slice image of nasal polyps in the first field of view of the first detection object to obtain the domain-invariant feature map of nasal polyps pathology. A cell classification model was used to classify the cell types of the invariant feature map of the nasal polyp pathological domain to obtain the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. A tissue region segmentation model was used to segment the invariant feature map of the nasal polyp pathological domain to obtain the epithelial region, vascular region and glandular region in the invariant feature map of the nasal polyp pathological domain. Based on the cell type of each cell in the invariant feature map of the nasal polyp pathological domain, the number of cells of each cell type in the first field of view is counted. Based on the epithelial region, vascular region, and glandular region in the invariant feature map of the pathological domain of nasal polyps, the area of the epithelial region, vascular region, and glandular region under the first field of view are statistically analyzed. Based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region, the nasal polyp inflammation type of the first test subject is determined.
2. The method according to claim 1, characterized in that, The domain generalization model includes a first encoder composed of a backbone network and a first decoder, and the method includes: The pathological slice image sample of nasal polyps under the second field of view of the second detection object is input into the backbone network to obtain the low-order features provided by the second network and the high-order features provided by the fifth network in the backbone network. The backbone network includes 5 networks from input to output with decreasing network size. Random convolution is performed on the low-order features to obtain perturbation features; Enhanced features are obtained by extracting features from the perturbation features using a leaky hybrid network; The higher-order features are concatenated with the enhanced features, and the resulting features are input into the first decoder to obtain the prediction domain invariant feature map output by the first decoder. Based on the difference between the predicted domain invariant feature map and the real domain invariant feature map corresponding to the nasal polyp pathological slice image sample, the network parameters of the backbone network, the first decoder and the leaky hybrid network are adjusted.
3. The method according to claim 2, characterized in that, Also includes: A classifier is used to perform domain classification on the higher-order features to obtain the predicted domain source of the nasal polyp pathological slice image sample; Based on the difference between the predicted domain source and the corresponding real domain source of the nasal polyp pathological slice image sample, the network parameters of the classifier and the first encoder are adjusted.
4. The method according to claim 2 or 3, characterized in that, The method employs a domain generalization model to generalize the nasal polyp pathological slice image under the first field of view of the first detection object, obtaining a domain-invariant feature map of nasal polyp pathology, including: Pixel clustering is performed on the pathological section image of nasal polyps in the first field of view of the detected object to obtain the foreground cell pixel region and the background debris pixel region; The pathological section image of the nasal polyp was segmented into multiple rectangular images; Extract the rectangular images that intersect with the foreground cell pixel region from the plurality of rectangular images, and use the extracted rectangular images to form the foreground cell image region; A domain generalization model is used to generalize the foreground cell image region to obtain the domain-invariant feature map of the nasal polyp pathology.
5. The method according to claim 4, characterized in that, The method employs a domain generalization model to generalize the foreground cell image region, obtaining the domain-invariant feature map of the nasal polyp pathology, including: The foreground cell image region is input into the first encoder in the domain generalization model to obtain the high-order features output by the first encoder. The higher-order features are input into the first decoder in the domain generalization model to obtain the pathological domain invariant feature map of nasal polyps output by the first decoder.
6. The method according to claim 1, characterized in that, The cell classification model includes a feature extraction module, an attention module, a linear regression module, a linear classification module, and a one-to-one matching module. The feature extraction module includes a feature pyramid network. The cell classification model is used to classify cell types in the nasal polyp pathological domain invariant feature map, obtaining the cell types of each cell in the nasal polyp pathological domain invariant feature map, including: The pathological domain invariant feature map of the nasal polyp is input into the feature extraction module to obtain the features of each pyramid layer output by the feature pyramid network in the feature extraction module; Each of the tower layer features is input into the attention module to obtain the attention aggregation features output by the attention module; The attention aggregation feature is input into the linear regression module to obtain the linear regression feature of each cell output by the linear regression module; The attention aggregation feature is input into the linear classification module to obtain the linear classification feature of each cell output by the linear classification module; The linear regression features and linear classification features of each cell are convolved to obtain the convolutional features of each cell. The convolutional features of each cell are then matched and classified one-to-one by the one-to-one matching module to obtain the cell type of each cell.
7. The method according to claim 1, characterized in that, The tissue region segmentation model uses the DeepLabv3+ model.
8. A device for recognizing nasal polyp pathological slide images, characterized in that, include: The domain generalization module is used to perform domain generalization on the nasal polyp pathological slice image under the first field of view of the first detection object using the domain generalization model, so as to obtain the nasal polyp pathological domain invariant feature map. The cell classification module is used to classify the cell categories of the invariant feature map of the nasal polyp pathological domain using a cell classification model, so as to obtain the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The region segmentation module is used to perform tissue region segmentation on the invariant feature map of the nasal polyp pathological domain using a tissue region segmentation model, so as to obtain the epithelial region, blood vessel region and glandular region in the invariant feature map of the nasal polyp pathological domain. The first statistical module is used to count the number of cells of each cell type in the first field of view based on the cell type of each cell in the invariant feature map of the nasal polyp pathological domain. The second statistical module is used to calculate the area of the epithelial region, the area of the vascular region, and the area of the glandular region in the first field of view based on the epithelial region, the vascular region, and the glandular region in the invariant feature map of the nasal polyp pathological domain. The inflammation type determination module is used to determine the nasal polyp inflammation type of the first detection object based on the number of cells of each cell type in the first field of view, as well as the area of the epithelial region, the area of the vascular region, and the area of the glandular region.
9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.