Visual analysis method and device supporting fine subtype identification of multiple types of immune cells
Patent Information
- Application Number
- CN202510937731.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2026-09-04
AI Technical Summary
[0017]本发明的目的是解决现有技术中在血液图像中仅能识别免疫细胞大类、无法精细划分功能亚型、计数准确率低、系统集成度差的问题,特别是在面对不同染色方法和采图设备条件时模型泛化能力不足、自动化程度不高的缺陷,提出了一种基于多尺度视觉Transformer(ViT)结构的免疫细胞精细亚型识别与计数的方法及系统
[0063] This invention enables the specific identification and classification of immune cell subtypes in blood, particularly the precise differentiation of multiple T cell and B cell subtypes, thereby significantly improving the granularity and accuracy of blood immune cell image analysis. This invention employs a multi-scale visual Transformer architecture, which can efficiently extract cell morphology and texture features, achieving detailed classification at the single-cell level without relying on complex pre-labeled or experimentally perturbed data.
Smart Images

Figure CN122695608A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and image analysis technology, and particularly to a visual analysis method, device, electronic device, computer-readable storage medium, and computer program product that supports the precise identification of multiple immune cell subtypes. It is especially suitable for the detection, subtype classification, quantity statistics, and automatic report generation of immune cells in blood smear images, and is widely used in medical and research scenarios such as hematological disease diagnosis, immune status assessment, infection and immunotherapy monitoring. Background Technology
[0002] The types and proportions of immune cells in the blood are crucial for assessing the body's immune status and diagnosing hematologic diseases, infectious diseases, and tumor immune responses. Traditional immune cell identification primarily relies on two methods: First, manual microscopic observation. In clinical practice, pathologists stain blood smears (e.g., Wright-Giemsa stain) and then observe and count major white blood cell types such as neutrophils, eosinophils, basophils, lymphocytes, and monocytes under a microscope. This method is cumbersome, subjective, inefficient, and struggles to accurately identify certain cell subtypes (such as naive CD8+ T cells and Treg cells). Second, flow cytometry.
[0003] Although flow cytometry is highly accurate in identifying immune cell subtypes, the equipment is expensive, the operation is complex, and it cannot provide morphological information of cells, making it difficult to apply to primary healthcare institutions and large-scale screening.
[0004] To improve recognition efficiency and objectivity, researchers have recently begun to explore image recognition and artificial intelligence methods for the automatic identification of immune cells in blood smear images. These methods typically utilize convolutional neural networks (CNNs) to build classification models, achieving preliminary identification of the five major white blood cell types (neutrophils, eosinophils, basophils, lymphocytes, and monocytes). For example, some models based on ResNet, DenseNet, and MobileNet have been used for white blood cell image classification tasks, achieving considerable accuracy at the cellular level. However, these methods generally suffer from the following shortcomings: First, the classification granularity is relatively coarse; most methods are limited to the five-part differential of white blood cells and cannot identify more fine-grained T cells, B cells, NK cells, and their functional states (such as naive, memory, activated, exhausted, etc.). Second, they lack spatial hierarchy perception capabilities; subtle morphological differences between cell subtypes depend on structural perception at different scales, while traditional CNNs have limited ability to model long-distance context. Third, they lack integrated automatic counting and reporting systems; existing research focuses mainly on image classification tasks and has failed to achieve clinically usable counting functions and structured output. Furthermore, they exhibit poor robustness to low resolution or different image qualities, and have insufficient generalization ability. Therefore, there is an urgent need for a novel visual analysis system that can support multi-scale feature extraction, refined immune cell subtype identification, automatic count, and generation of analysis reports. This system should maintain morphological interpretability while possessing clinical feasibility to improve the practicality and accuracy of intelligent blood image analysis in clinical and research scenarios.
[0005] Although AI-based image analysis methods have been initially applied to the identification of white blood cells in blood smears in recent years, existing technologies still have several key problems that make it difficult to meet the actual clinical needs for precise identification of multiple immune cell subtypes. These problems are mainly reflected in the following aspects:
[0006] (1) The number of types that can be identified is limited and cannot cover a variety of fine subtypes of immune cells.
[0007] Most current image recognition methods only support the recognition of five major types of white blood cells: neutrophils, eosinophils, monocytes, lymphocytes, and basophils. Their ability to recognize T cells, B cells, NK cells, and their functional subtypes (such as naive, memory, activated, and exhausted cells) is severely inadequate in medical research and clinical immune monitoring.
[0008] This is because traditional methods mainly rely on macroscopic morphological characteristics for classification, making it difficult to distinguish between fine subtypes of cells that are extremely similar in morphology but have significant functional differences.
[0009] (2) The feature extraction method is singular and lacks the ability to model multi-scale structures.
[0010] Most existing methods use classic convolutional neural networks (such as ResNet and DenseNet) for feature extraction, but such networks have limited ability to model long-distance dependencies, and the fixed receptive field makes it difficult to fully capture the microscopic features of cells at multiple scales (such as nuclear structure, cytoplasmic distribution, and staining density).
[0011] Therefore, it is difficult to identify functional T cells or immune cells in different activation states with slight structural differences.
[0012] (3) It is impossible to achieve accurate statistics and structured output of cell count.
[0013] Most existing image classification methods only support the classification of single cell images and cannot identify and count immune cells in large-format smear images as a whole, and lack the ability to generate automated and structured reports.
[0014] This makes it difficult for the technology to replace manual counting in real clinical scenarios, reducing its application value.
[0015] (4) The model has poor robustness and is difficult to adapt to staining differences or equipment changes.
[0016] Different laboratories, equipment, or staining procedures can result in images exhibiting varying colors, backgrounds, or sharpness. Traditional model training relies on large amounts of labeled data, but its generalization ability is insufficient, and its accuracy drops significantly when data distribution changes, affecting the model's generalizability. Currently, there is a lack of a complete visual analysis system that can handle everything from image input, cell identification, subtype classification, and quantity statistics to report generation, forcing physicians to manually integrate multiple tools, resulting in low analysis efficiency and a high risk of errors. Summary of the Invention
[0017] The purpose of this invention is to address the problems of existing technologies that can only identify broad categories of immune cells in blood images, cannot finely classify functional subtypes, have low counting accuracy, and poor system integration. In particular, these technologies suffer from insufficient model generalization ability and low automation when faced with different staining methods and image acquisition equipment. This invention proposes a method and system for fine-grained identification and counting of immune cell subtypes based on a multi-scale visual Transformer (ViT) structure. This system integrates cell detection, fine-grained subtype identification, quantity statistics, and structured report output, supporting the identification and analysis of more than 20 types of immune cells and their functional states, including CD4. + / CD8 +T cells, naive / memory / exhausted subsets, B cells, NK cells, plasma cells, etc. By introducing a multi-scale ViT architecture, the model's ability to model complex morphologies and fine structures is effectively enhanced, significantly improving the detection, recognition, and counting accuracy in large-format blood smear images. Furthermore, the structured reporting module outputs immunographic information that can be used for clinical reference. It boasts advantages such as high accuracy, high automation, strong generalization, and scalability, meeting the practical needs of high-throughput, refined immunoassay analysis in clinical settings.
[0018] To address the shortcomings of existing technologies, such as Figure 2 As shown, this invention proposes a visual analysis method that supports the fine-grained identification of multiple immune cell subtypes, including:
[0019] The initial step involves acquiring multiple blood cell images labeled with the target cell category, scaling these blood cell images to multiple specified sizes, and performing data augmentation to obtain multiple images of the blood cell image at different scales as training images.
[0020] The encoding step involves constructing a multi-scale visual ViT model, including an encoder and a fully connected layer. The encoder employs a multi-scale visual network, where training images at each scale are divided into blocks and input into the encoder to obtain image features for each image block. These image features belonging to the same training image are then fused to obtain a unified image embedding for each training image. The unified image embeddings of all training images belonging to the same blood cell image are then fused to obtain the final features of the blood cell image. These final features are then input into the fully connected layer for cell type classification to obtain the predicted probability of the blood cell image belonging to each cell type.
[0021] The training steps involve constructing a loss function based on the predicted probability and the target cell category to train the multi-scale visual ViT model, thereby obtaining the visual analysis model.
[0022] The detection step involves inputting the blood smear image to be visually analyzed into the visual analysis model to obtain the cell category of each cell in the image.
[0023] The aforementioned visual analysis method supporting fine subtype identification of multiple immune cells includes the following detection step:
[0024] The automatic counting step divides the blood smear image into multiple local regions and counts the number of cells of each cell type in each local region; it obtains a recognition result C that includes the cell types and their spatial locations in the local region; it performs spatial clustering on the cell locations in recognition result C and outputs a deduplicated recognition result D; based on recognition result D, it counts each cell type separately to obtain the number of cells of each cell type in the local region and the total number of cells, forming a statistical result E; it organizes the statistical result E to generate an automatic counting report, which includes the number of cells of each type, their proportion, statistical charts, and spatial distribution information.
[0025] The aforementioned visual analysis method supporting fine subtype identification of multiple immune cells, wherein the encoding step includes:
[0026] The s-th scale image of the i-th training image is I. i s Through the visual network E at the s-th scale in the encoder s Map it to fixed-dimensional image features:
[0027] z i s =E s (I i s )
[0028] Fuse features from various images using attention-weighted methods or direct stitching:
[0029] zi=∑α s ·z i s
[0030] Where α s The weighting coefficients for the s-th scale are obtained through learning or set according to the image resolution;
[0031] After completing multi-scale feature extraction and self-attention encoding, the image features of the training images at all scales are as follows:
[0032] z1, z2, ..., zL
[0033] Where zi represents the Tranformer encoded feature at the i-th scale; the image features of all training images at all scales are merged into this final feature z. final :
[0034]
[0035] Final feature z final The data is fed into a fully connected layer for cell type classification; the output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function.
[0036] y = Softmax(W × z) final +b)
[0037] Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for each type of blood cell.
[0038] The aforementioned visual analysis method supports the precise identification of multiple immune cell subtypes, wherein the cell types include: erythrocytes, lymphocytes, monocytes, neutrophils, eosinophils, and basophils.
[0039] like Figure 3 As shown, the present invention also proposes a visual analysis device that supports the fine subtype identification of multiple immune cells, including:
[0040] The initial module acquires multiple blood cell images labeled with the target cell category, scales the blood cell images to multiple specified sizes, and performs data augmentation to obtain multiple images of the blood cell images at different scales as training images;
[0041] The encoding module constructs a multi-scale visual ViT model, including an encoder and a fully connected layer. The encoder uses a multi-scale visual network, where training images at each scale are divided into blocks and input into the encoder to obtain image features for each image block. These image features belonging to the same training image are then fused to obtain a unified image embedding for each training image. The unified image embeddings of all training images belonging to the same blood cell image are then fused to obtain the final features of the blood cell image. The final features are then input into the fully connected layer for cell type classification to obtain the predicted probability of the blood cell image belonging to each cell type.
[0042] The training module constructs a loss function based on the predicted probability and the target cell category to train the multi-scale visual ViT model, thereby obtaining the visual analysis model.
[0043] The detection module inputs the blood smear image to be visually analyzed into the visual analysis model to obtain the cell category of each cell in the image.
[0044] The aforementioned visual analysis device supporting fine subtype recognition of multiple immune cells, wherein the detection module includes:
[0045] The automatic counting module divides the blood smear image into multiple local regions and counts the number of cells of each cell type in each local region. It obtains a recognition result C that includes the cell types and their spatial locations in the local region. The cell locations in the recognition result C are spatially clustered, and the deduplicated recognition result D is output. Based on the recognition result D, each cell type is counted separately to obtain the number of cells of each cell type in the local region and the total number of cells, forming a statistical result E. The statistical result E is compiled to generate an automatic counting report, which includes the number of cells of each type, their proportion, statistical charts, and spatial distribution information.
[0046] The aforementioned visual analysis device supporting fine subtype identification of multiple immune cells, wherein the encoding module includes:
[0047] The s-th scale image of the i-th training image is I. i s Through the visual network E at the s-th scale in the encoder s Map it to fixed-dimensional image features:
[0048] z i s =E s (I i s )
[0049] Fuse features from various images using attention-weighted methods or direct stitching:
[0050] zi=∑α s ·z i s
[0051] Where α s The weighting coefficients for the s-th scale are obtained through learning or set according to the image resolution;
[0052] After completing multi-scale feature extraction and self-attention encoding, the image features of the training images at all scales are as follows:
[0053] z1, z2, ..., zL
[0054] Where zi represents the Tranformer encoded feature at the i-th scale; the image features of all training images at all scales are merged into this final feature z. final :
[0055]
[0056] Final feature z final The data is fed into a fully connected layer for cell type classification; the output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function.
[0057] y = Softmax(W × z) final +b)
[0058] Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for each type of blood cell.
[0059] The present invention also proposes an electronic device, including the aforementioned visual analysis device supporting the fine subtype identification of multiple immune cells, the electronic device being connected to an information display device, the information display device being used to display the cell category using user-set display parameters, attributes or through an artificial intelligence model.
[0060] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the visual analysis method supporting the fine subtype identification of multiple immune cells.
[0061] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the visual analysis method supporting the fine subtype identification of multiple immune cells.
[0062] As can be seen from the above solutions, the advantages of the present invention are:
[0063] This invention enables the specific identification and classification of immune cell subtypes in blood, particularly the precise differentiation of multiple T cell and B cell subtypes, thereby significantly improving the granularity and accuracy of blood immune cell image analysis. This invention employs a multi-scale visual Transformer architecture, which can efficiently extract cell morphology and texture features, achieving detailed classification at the single-cell level without relying on complex pre-labeled or experimentally perturbed data.
[0064] The method of this invention, through the tight integration of cell detection and subtype identification modules, supports automated batch image processing, significantly improving the efficiency of immune cell classification, reducing human error, and is applicable to various blood samples and clinical testing environments. Its structured output and visualization system help clinicians and researchers quickly obtain immune status and cell subtype ratios, aiding in disease diagnosis and efficacy evaluation.
[0065] Compared with existing technologies, this invention eliminates the need for expensive single-cell sequencing or complex biological experiments, enabling high-throughput, multi-subtype immune cell identification based on standard blood image data, thus saving experimental costs and time. The technology boasts stable performance, wide applicability, and excellent scalability and clinical application potential, particularly valuable in tumor immune monitoring, infection immune analysis, and autoimmune disease research. Attached Figure Description
[0066] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;
[0067] Figure 2 This is an overall flowchart of the present invention;
[0068] Figure 3 This is a block diagram of the device of the present invention;
[0069] Figure 4 This is a schematic diagram of the structure of the first electronic device of the present invention;
[0070] Figure 5 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;
[0071] Figure 6 This is a schematic diagram of the structure of the second electronic device of the present invention.
[0072] Figure label:
[0073] A - First electronic device;
[0074] B- A visual analysis device that supports the precise identification of multiple immune cell subtypes;
[0075] C-Data acquisition equipment;
[0076] D-Information display device;
[0077] 1000 - Second electronic device;
[0078] Ⅰ-Computational Unit;
[0079] II-ROM;
[0080] III-RAM;
[0081] N-bus;
[0082] V-Interface;
[0083] VI - Input Unit;
[0084] VII - Output Unit;
[0085] VIII - Storage medium;
[0086] IX - Communication Unit. Detailed Implementation
[0087] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.
[0088] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0089] The processor described in this invention is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of this invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).
[0090] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory, and by calling data stored in memory.
[0091] In a specific implementation, as one example, the processor may include one or more CPUs. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include servers, desktop computers, laptops, smartphones, tablets, embedded computers, etc., where the embedded computer includes vehicles and robots, etc.
[0092] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.
[0093] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0094] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0095] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0096] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.
[0097] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0098] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0100] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0102] This invention aims to address key issues in existing technologies, such as limited identification types, insufficient feature extraction capabilities, and lack of statistical and reporting functions. It proposes an intelligent analysis system based on a multi-scale visual Transformer (ViT) structure, capable of automatically detecting multiple types of immune cells, identifying fine subtypes, statistically analyzing their numbers, and outputting structured reports in blood smear images. The system supports high-precision identification of more than 20 types of immune cells and their functional states, possesses multi-scale feature modeling capabilities, and effectively improves the resolution of complex microstructures. Simultaneously, it can achieve cell localization and counting in large-format images, automatically generating structured diagnostic reports, significantly improving analysis efficiency. Furthermore, the system exhibits good robustness and transferability, adapting to various staining methods and image acquisition conditions, and possesses broad potential for clinical application.
[0103] In their research on fine-grained subtype identification and counting of immune cells based on blood images, the inventors discovered two key technical shortcomings in existing technologies: First, existing methods primarily focus on the classification of broad white blood cell groups (such as neutrophils, monocytes, and lymphocytes), failing to further identify their fine-grained subtypes, particularly their functional states (such as activation, exhaustion, naivety, and memory). Second, when performing accurate cell detection and counting in large-format blood smear images, the models are often insensitive to complex morphological changes, leading to missed detections and misclassifications. The root cause of these problems lies in the fact that existing algorithms are mostly based on traditional CNN structures, lacking the ability to jointly model local contextual and global structural information between cells. This results in poor model generalization ability, especially when faced with staining differences and variations in image acquisition conditions.
[0104] After in-depth research on multi-scale feature analysis of blood cell images, the performance of the Transformer architecture in visual tasks, and the morphological details of immune subtypes, the inventors discovered that employing a multi-scale visual Transformer (ViT) structure with both local perception and global modeling capabilities can effectively alleviate the aforementioned technical bottlenecks. This structure can simultaneously capture the fine internal structure of cells and the contextual relationships between cells, significantly improving the accuracy of cell subtype recognition. Furthermore, through hierarchical feature aggregation and spatial attention mechanisms, it enhances the model's ability to perceive minute cell features, thus adapting it to cell detection and counting tasks in large-format images.
[0105] This invention improves upon traditional cell classification methods through multi-dimensional technological advancements, specifically including: introducing a multi-scale ViT structure to enhance the model's adaptability to cell morphological diversity; designing a unified detection and classification framework to integrate cell identification, localization, and counting; constructing a cell state knowledge base to support a labeling system for over 20 immune cell subtypes and functional states; and developing a structured report module to automatically output cell type distribution and quantity statistics. Through these technological innovations, this invention achieves a high-throughput, automated, and interpretable immune cell analysis workflow from blood images, significantly enhancing the system's practicality, scalability, and clinical value.
[0106] In summary, to achieve the above-mentioned technical effects, the present invention proposes the following key technical points:
[0107] Key Point 1: A method for fine identification of blood immune cells based on multi-scale visual Transformer. This method includes an image preprocessing module, a visual Transformer feature extraction module, and a subtype classification module. The image preprocessing module performs color normalization and cell region extraction on blood images; the visual Transformer module extracts cell morphology features under multi-scale windows; the classification module identifies immune cell subtypes based on the extracted features, particularly including the ability to identify fine subtypes of T cells and B cells; Technical effect: This method can finely identify more than 20 immune cell subtypes, including but not limited to:
[0108] T cell subtype: CD4 + T cells (including naive CD4+, central memory, effector memory, and regulatory T cells (Tregs)) and CD8+ T cells (including naive CD8+, exhausted T cells, and effector CD8+ T cells) + ), γδT cells, etc.;
[0109] B cell subtypes: naive B cells, memory B cells, plasmablasts, plasma cells, etc.
[0110] Other cell types include NK cells, monocytes, dendritic cells, and neutrophils.
[0111] This method can construct high-resolution immune cell atlases at the single-cell image level, accurately identify changes in immune status, and significantly improve the granularity and accuracy of immune cell classification in clinical images.
[0112] Key Point 2: A comprehensive system integrating detection and classification, comprising a cell detection submodule, an identification submodule, a quantity statistics submodule, and a structured output module. The system first uses a detection network to detect and locate individual cells in an image. Then, the identification module determines the subtype, and the quantity statistics module calculates the number and proportion of each subtype, ultimately generating a structured immunoassay report. Technical Benefits: The system supports automatic detection and subtype identification of all single cells in an image, exhibiting high-precision classification capabilities, particularly in distinguishing between T cells (CD4+, CD8+, Treg, etc.) and B cells (memory, plasma, etc.). The system can output the number and proportion of each subtype of cells, making it suitable for automated immunoassay tasks in clinical peripheral blood and bone marrow smears, improving analysis efficiency and reducing human error.
[0113] Key Point 3: An immune analysis system based on the fusion of clinical and image data, including modules for data uploading, model prediction, result storage, and visualization. The system can upload user-collected blood images or access existing sample libraries to perform immune cell identification and subtype statistics. The result storage module saves predicted data and analysis reports, while the visualization module provides interactive displays of cell distribution, proportions, and automatic cell counting. Technical Benefits: This system supports batch blood image processing and immune phenotypic analysis, and can integrate patient baseline information and laboratory indicators to provide researchers and clinicians with personalized immune status assessments and auxiliary judgments on disease progression. It possesses good scalability and versatility.
[0114] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.
[0115] like Figure 1 As shown, this invention proposes a method for blood cell image recognition based on a multi-scale Vision Transformer (ViT) model. This method improves the recognition accuracy of cell images by constructing a multi-scale feature extraction architecture and combining it with the global modeling capabilities of the Transformer network. The technical implementation steps of this invention are described in detail below.
[0116] 1) Training data collection and preprocessing module
[0117] The blood cell image data used in the training of this invention mainly comes from peripheral blood samples that have undergone flow cytometry sorting. Specific steps include: a) using a flow cytometer to perform multi-parameter detection and sorting of cells in the blood samples, achieving high-purity sorting of various subtypes of blood cells (T cells, B cells, NK cells) based on cell size, morphology, and surface markers. Each cell subpopulation corresponds to separate cell image data after sorting, ensuring accurate labeling and category differentiation of the image samples. b) preparing standard blood smears from the sorted cells and imaging them using a high-resolution microscope to obtain multi-scale image data at different magnifications. The images undergo color correction, contrast enhancement, and noise reduction to ensure consistent image quality and reliable recognition.
[0118] The blood cell image dataset contains Giemsa staining images of multiple different cell types. Each image is H×W×C in size, where H represents the image height, W represents the image width, and C is the number of channels (C=3 for RGB images). To facilitate model training, all images require the following preprocessing steps:
[0119] 1. Size standardization: Scaling the input image to a specified size.
[0120] 2. Data augmentation: Data augmentation is performed through techniques such as rotation, flipping, cropping, and color transformation to improve the robustness of the model.
[0121] 2) Multi-scale image encoder
[0122] The multi-scale image encoder is an encoder structure that can serve as an encoder for a pre-trained multi-scale Vision Transformer (ViT) network. This encoder aims to capture the feature differences of different cell types in blood cell images at different resolutions. The input is a multi-scale version of the blood cell image, such as an image pyramid constructed at different magnifications (original, 40×, 100X, etc.). The multi-scale image encoder is used to extract semantically rich and structurally stable deep feature representations from blood cell images at different resolutions. It can effectively capture the morphological differences of different cell types at multiple granular scales, enhancing the model's ability to identify heterogeneous cells. Utilizing the self-attention mechanism in the Transformer structure, the multi-scale image encoder can model long-distance dependencies between any locations in the image, thereby establishing cross-regional semantic connections between different cell structures. Compared to traditional convolutional networks that can only focus on local neighborhood features, the Transformer possesses global context awareness, effectively identifying subtle but discriminative morphological changes in high-resolution images, significantly improving the accuracy of identifying complex or morphologically similar cell types (such as T cells and B cells).
[0123] Given the image of the i-th image sample at the s-th scale, represented as I i s Then through encoder E s Map it to a fixed-dimensional image feature embedding vector:
[0124] z i s =E s (I i s (1)
[0125] Where E s For the s-th scale ViT sub-encoder, it can be a parameter-shared or non-shared structure.
[0126] Subsequently, attention-weighted or direct concatenation methods are used to fuse features at various scales, and a unified image embedding z is calculated. i :
[0127] z i =∑α s ·z i s (2)
[0128] Where α sThe weighting coefficients for the s-th scale can be learned or set according to the image resolution. Ultimately, this image embedding represents the integration of cell morphology features at different scales in the blood image, enhancing the model's ability to distinguish between large cells (such as monocytes) and small cells (such as platelets).
[0129] Because images at different resolutions have a consistent structure and are easily scalable, this multi-scale image encoder possesses good versatility and scalability, making it applicable to blood cell image recognition tasks under different magnifications and imaging devices. This invention weakens the dominant role of image color in model discrimination by introducing a multi-scale visual Transformer structure, combining structure-driven feature extraction capabilities with a global attention mechanism. Simultaneously, through color perturbation enhancement during training and standardization strategies during preprocessing, the model's adaptability to images with different laboratory staining variations is effectively improved, achieving accurate classification of color-inconsistent cell images. Specific key technologies include:
[0130] 1. Multi-scale visual Transformer structure
[0131] The Transformer's global attention mechanism (self-attention) not only focuses on color itself but also extracts semantic features from information such as structure, texture, and morphology. Therefore, compared to convolutional networks, it is less dependent on color intensity. Specifically, the Transformer models the long-distance relationships between cell structure and distribution in an image across regions, weakening the dominant role of color in feature extraction and improving the model's stability under color shift conditions.
[0132] 2. Multi-scale image input strategy
[0133] The input image is processed at multiple scales by constructing an image pyramid (e.g., original magnification, 40×, 100×), enabling the model to acquire cell morphology, boundaries, and internal structure information at multiple scales. This multi-granularity analysis approach compensates for the lack or shift in color information to some extent. Even if staining differences lead to different color distributions, multi-scale structure extraction can still ensure that the model acquires stable morphological features such as edges and karyotypes, thereby improving generalization ability.
[0134] 3. Color perturbation training in data augmentation mechanisms
[0135] During training, this application can introduce image enhancement strategies such as color dithering and brightness contrast perturbation, allowing the model to "see" similar cells with different colors in advance. This is equivalent to performing domain generalization training. By enhancing the training data through random color perturbation, the model learns to ignore irrelevant color changes and focus on structural features.
[0136] After completing multi-scale feature extraction and self-attention encoding, the outputs at all scales are:
[0137] z1, z2, ..., zL
[0138] Where zi represents the Tranformer encoded feature at the i-th scale. To fuse multi-scale information, this invention employs a weighted summation feature fusion strategy, merging feature vectors from all scales into a unified global representation for subsequent cell type discrimination. Specifically, the fusion operation is defined as follows:
[0139]
[0140] This fusion strategy effectively integrates local details and overall structural information from different scales while maintaining computational efficiency, thereby enhancing the model's expressive power in heterogeneous cell recognition tasks.
[0141] It is important to note that
[0142] Formula (2) is the self-attention fusion within the Transformer. The operation location is within the Transformer module at each scale, where multi-head self-attention is performed. The fusion mechanism is to dynamically weight and fuse information between different patches through self-attention (especially the interaction between long-distance tokens). It reflects the essential capabilities of the Transformer: such as modeling the nuclear-cytoplasmic boundary relationship within a cell, or the spatial pattern of the cell-background. Formula (3) mainly targets the modeling of spatial relationships within a single-scale image, that is, Formula (3) is the cross-scale feature fusion after the Transformer encoder. The operation location is after all scales are encoded, where z1, z2, ..., zL are fused. The fusion mechanism is: such as z = ∑w i *z i The representation vectors at different scales are weighted using learnable or manually set weights. Objective: To integrate image understanding at various resolutions, such as cell boundary information at a 40× scale and nuclear chromatin texture at a 100× scale. Final input for classification: A unified global representation is formed and input into the fully connected (FC) classifier.
[0143] The fused feature z final The data is fed into a fully connected layer for cell type classification. The output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function.
[0144] y = Softmax(W × z) final +b) (4)
[0145] Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for the six major types of blood cells (red blood cells, lymphocytes, monocytes, neutrophils, eosinophils, and basophils).
[0146] Finally, after classifying each cell type in all input image regions, the model will count the number of cells of each type to form a cell distribution vector for generating a structured report, which will assist in subsequent clinical diagnosis or drug screening analysis.
[0147] 3) Training and Testing
[0148] The model described in this invention is based on a multi-scale visual Transformer, combining cell morphology and background context information to achieve automatic identification and classification of blood cells. The training objective of the model is to minimize the multi-class cross-entropy loss, defined as follows:
[0149]
[0150] Where N represents the number of samples, C represents the number of cell types, and y ic Indicates the real label, p ic The probability predicted by the model, ∈ is a small constant introduced to prevent numerical instability, which is set to 1×10 by default. -6 .
[0151] To prevent data leakage and improve model generalization ability, all image data were strictly divided according to patient ID and acquisition batch to ensure no overlap between the training, validation, and test sets. The data partition ratio was 6:2:2, and the model used a five-fold cross-validation strategy for stability assessment.
[0152] During training, the AdamW optimizer was used with an initial learning rate of 1e-4, dynamically adjusted using a cosine annealing scheduling strategy. The batch size was 32, with a maximum of 50 training epochs, and training was stopped early if there was no improvement in validation set performance. Data augmentation strategies included rotation, flipping, color jittering, and blur simulation to improve the model's robustness to different microscopic imaging conditions.
[0153] 4) Automatic blood cell counting and reporting module
[0154] The automatic counting module provides a workflow for counting the number of five major cell types based on the ViT model described above. In the first step, the multi-scale ViT model divides the image into multiple local regions and classifies each cell into six cell types (erythrocytes, neutrophils, eosinophils, basophils, monocytes, and lymphocytes). The class label and spatial coordinates of each cell are output, yielding the identification result C. Based on this, spatial clustering is performed on the cell locations in the identification result C, and non-maximum suppression (NMS) is used to remove overlap detections, ensuring that each cell is uniquely counted, outputting the deduplicated cell identification result D. Based on the deduplicated cell identification result D, the six cell types are counted separately to obtain the number of each cell type and the total number of cells, forming the statistical result E. The statistical result E is then compiled to generate an automatic counting report, F, which includes the number and percentage of each cell type, statistical charts, and spatial distribution information for clinical or research use.
[0155] The reasons for necessitating partitioning for statistical analysis include: the image size being significantly larger than the cell size (an entire image can reach tens of thousands of pixels, making it difficult for GPU memory to process the entire image at once); the high number and density of cells (hundreds or thousands of cells overlap and intersperse in a high-magnification field of view, making direct detection prone to overdetection or underdetection); and the fixed resolution of the ViT model input (Vision Transformer typically accepts fixed-size patches or patch embeddings, such as 224×224), thus requiring sliding windows or region slicing for partitioning. The purpose of partitioning is to: control computational resource consumption; maintain standardized model input; model cell features locally and then integrate them globally using NMS to avoid redundant statistics.
[0156] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.
[0157] like Figure 3 As shown, the present invention also proposes a visual analysis device that supports the fine subtype identification of multiple immune cells, including:
[0158] The initial module acquires multiple blood cell images labeled with the target cell category, scales the blood cell images to multiple specified sizes, and performs data augmentation to obtain multiple images of the blood cell images at different scales as training images;
[0159] The encoding module constructs a multi-scale visual ViT model, including an encoder and a fully connected layer. The encoder uses a multi-scale visual network, where training images at each scale are divided into blocks and input into the encoder to obtain image features for each image block. These image features belonging to the same training image are then fused to obtain a unified image embedding for each training image. The unified image embeddings of all training images belonging to the same blood cell image are then fused to obtain the final features of the blood cell image. The final features are then input into the fully connected layer for cell type classification to obtain the predicted probability of the blood cell image belonging to each cell type.
[0160] The training module constructs a loss function based on the predicted probability and the target cell category to train the multi-scale visual ViT model, thereby obtaining the visual analysis model.
[0161] The detection module inputs the blood smear image to be visually analyzed into the visual analysis model to obtain the cell category of each cell in the image.
[0162] The aforementioned visual analysis device supporting fine subtype recognition of multiple immune cells, wherein the detection module includes:
[0163] The automatic counting module divides the blood smear image into multiple local regions and counts the number of cells of each cell type in each local region. It obtains a recognition result C that includes the cell types and their spatial locations in the local region. The cell locations in the recognition result C are spatially clustered, and the deduplicated recognition result D is output. Based on the recognition result D, each cell type is counted separately to obtain the number of cells of each cell type in the local region and the total number of cells, forming a statistical result E. The statistical result E is compiled to generate an automatic counting report, which includes the number of cells of each type, their proportion, statistical charts, and spatial distribution information.
[0164] The aforementioned visual analysis device supporting fine subtype identification of multiple immune cells, wherein the encoding module includes:
[0165] The s-th scale image of the i-th training image is I. i s Through the visual network E at the s-th scale in the encoder s Map it to fixed-dimensional image features:
[0166] z i s =E s (I i s )
[0167] Fuse features from various images using attention-weighted methods or direct stitching:
[0168] zi=∑α s ·zi s
[0169] Where α s The weighting coefficients for the s-th scale are obtained through learning or set according to the image resolution;
[0170] After completing multi-scale feature extraction and self-attention encoding, the image features of the training images at all scales are as follows:
[0171] z1, z2, ..., zL
[0172] Where zi represents the Tranformer encoded feature at the i-th scale; the image features of all training images at all scales are merged into this final feature z. final :
[0173]
[0174] Final feature z final The data is fed into a fully connected layer for cell type classification; the output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function.
[0175] y = Softmax(W × z) final +b)
[0176] Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for each type of blood cell.
[0177] like Figure 3 As shown, in another embodiment of the present invention, a first electronic device A is also provided, which includes the aforementioned visual analysis device B that supports the fine subtype identification of multiple immune cells.
[0178] like Figure 4 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D via wired or wireless information transmission schemes. The data acquisition device C is used to acquire blood cell images, and the information display device D is used to display the cell categories or predicted probabilities obtained by the present invention.
[0179] Information display device D can process and organize the data output by the first electronic device A based on an information display mechanism to improve the readability of the data. This information display mechanism can be manually preset, for example, visualizing the data output by the first electronic device A. It can present the user with specified key information, such as specific cell types or cell locations, based on user-defined display parameters and / or attributes. Display parameters could be, for example, the data range to be displayed, and display attributes could be, for example, the font, color, or whether scrolling is enabled. This allows the user to understand this information more quickly without having to access secondary pages or scroll through pages, saving user effort. Alternatively, this information display mechanism can be an artificial intelligence (AI) display model, which can learn the user's key information interests based on previous usage habits, such as viewing time, number of clicks, and number of edits, and then automatically present the user with rich and necessary key information.
[0180] The present invention also provides a computer program product, which includes a computer program that can be stored on a readable storage medium. When the computer program is executed by a processor, the computer is able to execute the visual analysis method provided by the above methods that supports the identification of fine subtypes of multiple immune cells.
[0181] In another embodiment, the present invention also proposes a storage medium VIII for storing a computer program that performs the visual analysis method supporting the fine subtype identification of multiple immune cells. It should be understood that the storage medium in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0182] Figure 5 A schematic block diagram of a second electronic device 1000 that can be used to implement embodiments of the present invention is shown. The second electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein. The second electronic device 1000 may be the same as or different from the first electronic device A.
[0183] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from storage medium VIII into random access memory (RAM) III. The RAM III may also store various programs and data required for the operation of the device 1000. The computing unit I, ROM II, and RAM III are interconnected via bus IV. An input / output (I / O) interface V is also connected to bus IV.
[0184] Multiple components in the second electronic device 1000 are connected to I / O interface V, including: input unit VI, such as a keyboard, mouse, etc.; output unit VII, such as various types of displays, speakers, etc.; storage medium VIII, such as a disk, optical disk, etc.; and communication unit IX, such as a network card, modem, wireless transceiver, etc. Communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0185] The computing unit I can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to perform methods by any other suitable means (e.g., by means of firmware).
[0186] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.
Claims
1. A visual analysis method supporting the precise identification of multiple immune cell subtypes, characterized in that, include: The initial step involves acquiring multiple blood cell images labeled with the target cell category, scaling these blood cell images to multiple specified sizes, and performing data augmentation to obtain multiple images of the blood cell image at different scales as training images. The encoding step involves constructing a multi-scale visual ViT model, which includes an encoder and a fully connected layer. The encoder uses a multi-scale visual network. The training images at each scale are divided into blocks and input into the encoder to obtain the image features of each image block. The image features belonging to the same training image are fused to obtain a unified image embedding for each training image. The unified image embeddings of all training images belonging to the same blood cell image are fused to obtain the final features of the blood cell image. The final features are input into a fully connected layer for cell type classification, and the predicted probability of the blood cell image belonging to each cell type is obtained. The training steps involve constructing a loss function based on the predicted probability and the target cell category to train the multi-scale visual ViT model, thereby obtaining the visual analysis model. The detection step involves inputting the blood smear image to be visually analyzed into the visual analysis model to obtain the cell category of each cell in the image.
2. The visual analysis method supporting fine subtype identification of multiple immune cells as described in claim 1, characterized in that, The testing steps include: The automatic counting step divides the blood smear image into multiple local regions and counts the number of cells of each cell type in each local region; it obtains a recognition result C that includes the cell types and their spatial locations in the local region; it performs spatial clustering on the cell locations in recognition result C and outputs a deduplicated recognition result D; based on recognition result D, it counts each cell type separately to obtain the number of cells of each cell type in the local region and the total number of cells, forming a statistical result E; it organizes the statistical result E to generate an automatic counting report, which includes the number of cells of each type, their proportion, statistical charts, and spatial distribution information.
3. The visual analysis method supporting fine subtype identification of multiple immune cells as described in claim 1, characterized in that, The encoding steps include: The s-th scale image of the i-th training image is I. i s Through the visual network E at the s-th scale in the encoder s Map it to fixed-dimensional image features: z i s =E s (I i s ) Fuse features from various images using attention-weighted methods or direct stitching: zi=∑α s ·from i s Where α s The weighting coefficients for the s-th scale are obtained through learning or set according to the image resolution; After completing multi-scale feature extraction and self-attention encoding, the image features of the training images at all scales are as follows: z1, z2, ..., zL Where zi represents the Tranformer encoded feature at the i-th scale; the image features of all training images at all scales are merged into this final feature z. final : Final feature z final The data is fed into a fully connected layer for cell type classification; the output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function. y=Softmax(W×z final +b) Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for each type of blood cell.
4. The visual analysis method supporting fine subtype identification of multiple immune cells as described in claim 1, characterized in that, This cell category includes: erythrocytes, lymphocytes, monocytes, neutrophils, eosinophils, and basophils.
5. A visual analysis device supporting the identification of fine subtypes of multiple immune cells, characterized in that, include: The initial module acquires multiple blood cell images labeled with the target cell category, scales the blood cell images to multiple specified sizes, and performs data augmentation to obtain multiple images of the blood cell images at different scales as training images; The encoding module constructs a multi-scale visual ViT model including an encoder and a fully connected layer. The encoder uses a multi-scale visual network. The training images at each scale are divided into blocks and input into the encoder to obtain the image features of each image block. The image features belonging to the same training image are fused to obtain a unified image embedding for each training image. The unified image embeddings of all training images belonging to the same blood cell image are fused to obtain the final features of the blood cell image. The final features are input into a fully connected layer for cell type classification, and the predicted probability of the blood cell image belonging to each cell type is obtained. The training module constructs a loss function based on the predicted probability and the target cell category to train the multi-scale visual ViT model, thereby obtaining the visual analysis model. The detection module inputs the blood smear image to be visually analyzed into the visual analysis model to obtain the cell category of each cell in the image.
6. The visual analysis device supporting fine subtype recognition of multiple immune cells as described in claim 5, characterized in that, The detection module includes: The automatic counting module divides the blood smear image into multiple local regions and counts the number of cells of each cell type in each local region. It obtains a recognition result C that includes the cell types and their spatial locations in the local region. The cell locations in the recognition result C are spatially clustered, and the deduplicated recognition result D is output. Based on the recognition result D, each cell type is counted separately to obtain the number of cells of each cell type in the local region and the total number of cells, forming a statistical result E. The statistical result E is compiled to generate an automatic counting report, which includes the number of cells of each type, their proportion, statistical charts, and spatial distribution information.
7. The visual analysis device supporting fine subtype recognition of multiple immune cells as described in claim 5, characterized in that, The encoding module includes: The s-th scale image of the i-th training image is I. i s Through the visual network E at the s-th scale in the encoder s Map it to fixed-dimensional image features: z i s =E s (I i s ) Fuse features from various images using attention-weighted methods or direct stitching: zi=∑α s ·from i s Where α s The weighting coefficients for the s-th scale are obtained through learning or set according to the image resolution; After completing multi-scale feature extraction and self-attention encoding, the image features of the training images at all scales are as follows: z1, z2, ..., zL Where zi represents the Tranformer encoded feature at the i-th scale; the image features of all training images at all scales are merged into this final feature z. final : Final feature z final The data is fed into a fully connected layer for cell type classification; the output of this fully connected layer is converted into a probability distribution for each cell type using a softmax function. y=Softmax(W×z final +b) Where W and b are the learnable weight matrix and bias vector of the classification layer, respectively, and y represents the model's predicted probability for each type of blood cell.
8. An electronic device, characterized in that, The invention includes a visual analysis device as described in claims 5-7 that supports the fine subtype identification of multiple immune cells. The electronic device may be connected to an information display device, which is used to display the cell category using user-set display parameters, attributes, or through an artificial intelligence model.
9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the visual analysis method for supporting fine subtype identification of multiple immune cells as described in any one of claims 1-4.
10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the visual analysis method for supporting fine subtype identification of multiple immune cells as described in any one of claims 1-4.
