A multi-domain unbalanced blood cell classification method, device and storage medium

Abstract

The application discloses a multi-domain unbalanced blood cell classification method and device and a storage medium. The method comprises the following steps: obtaining a cell image to be classified; inputting the cell image to be classified into a blood cell classification model to obtain a blood cell classification result; wherein the blood cell classification model is obtained through the following steps: obtaining a multi-domain unbalanced blood cell image set; performing dyeing enhancement processing on the multi-domain unbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image; performing local alignment processing according to the blood cell image and the domain transformation sample set corresponding to the blood cell image to calculate a local alignment loss; performing domain invariant contrast learning processing to calculate a domain invariant supervised contrast loss; and performing multi-stage training processing on a preset deep learning model according to the local alignment loss and the domain invariant supervised contrast loss to obtain the blood cell classification model. The application realizes blood cell classification and improves the classification accuracy. The application can be widely applied to the technical field of image recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus and storage medium for classifying multi-domain unbalanced blood cells. Background Technology

[0002] Blood cells, also known as hemocytes, are cells found in the blood and can be distributed throughout the body via blood flow. Traditional blood cell classification involves manually identifying blood cells using a microscope, a repetitive, labor-intensive, and time-consuming process. Furthermore, in traditional deep learning-based classification models, blood cell images collected from different laboratories often exhibit significant domain shifts and data imbalances. Domain shifts lead to decreased generalization performance of traditional classification models, resulting in poor performance on unseen datasets, while data imbalances cause them to perform poorly on minority classes, resulting in low classification accuracy.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for classifying multi-domain unbalanced blood cells, which effectively improves classification accuracy and model precision.

[0005] On one hand, embodiments of the present invention provide a method for classifying multi-domain unbalanced blood cells, including the following steps:

[0006] Acquire images of cells to be classified;

[0007] Input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result;

[0008] The blood cell classification model is obtained through the following steps:

[0009] Acquire a multi-domain imbalanced blood cell image set, which includes several blood cell images;

[0010] The multi-domain unbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image, wherein the domain transformation sample set includes several domain transformation samples.

[0011] Based on the blood cell image and the corresponding domain transform sample set, local alignment processing is performed, and local alignment loss is calculated.

[0012] Based on the blood cell image and the corresponding domain transformation sample set, perform domain-invariant contrastive learning processing and calculate the domain-invariant supervised contrastive loss.

[0013] Based on the local alignment loss and the domain-invariant supervised contrast loss, the preset deep learning model is subjected to multi-stage training to obtain the blood cell classification model.

[0014] In some embodiments, the staining enhancement processing of the multi-domain imbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image includes:

[0015] Image decomposition is performed on each of the blood cell images in the multi-domain imbalanced blood cell image set to obtain the corresponding density map and staining matrix;

[0016] Based on the number of clusters, multiple coloring matrices are clustered using a preset clustering method to obtain several clusters;

[0017] Set domain labels for the blood cell images corresponding to the staining matrix in each cluster;

[0018] A transformed image set is selected from the multi-domain unbalanced blood cell image set. The transformed image set includes a preset number of transformed images, and the domain labels of each transformed image in the transformed image set are different.

[0019] Select one transformed image from the set of transformed images as the image to be normalized;

[0020] The density map corresponding to the blood cell image and the density map corresponding to the image to be normalized are normalized to obtain a normalized map;

[0021] Calculate the domain transformation sample based on the normalized image, the illumination light intensity, and the staining matrix corresponding to the image to be normalized;

[0022] Multiple domain transformation samples are combined to obtain the domain transformation sample set.

[0023] In some embodiments, the step of performing local alignment processing based on the blood cell image and the domain transform sample set corresponding to the blood cell image, and calculating the local alignment loss, includes:

[0024] Feature extraction is performed on the blood cell image to obtain a blood cell feature map;

[0025] Feature extraction is performed on each domain transform sample in the domain transform sample set to obtain the corresponding domain transform feature map;

[0026] The blood cell feature map is mapped using a nonlinear multilayer perceptron to obtain the blood cell feature embedding;

[0027] The domain transform feature map is mapped using a nonlinear multilayer perceptron to obtain the domain transform feature embedding.

[0028] Calculate pixel-level alignment loss based on the blood cell feature embedding and the domain transformation feature embedding;

[0029] The local alignment loss is calculated based on the number of image samples, the spatial location index of the feature map, and the pixel-level alignment loss.

[0030] In some embodiments, the step of performing domain-invariant contrastive learning processing based on the blood cell image and the corresponding domain-transformed sample set, and calculating the domain-invariant supervised contrastive loss, includes:

[0031] Obtain the blood cell feature projection of the blood cell image and the domain transformation feature projection corresponding to each domain transformation sample in the domain transformation sample set;

[0032] L2 normalization was performed on the blood cell feature projections;

[0033] The domain transformation feature projection is L2 normalized;

[0034] The average feature is calculated based on the L2-normalized blood cell feature projection and the L2-normalized domain transformation feature projection.

[0035] Calculate positive samples based on class labels;

[0036] The domain-invariant supervised contrast loss is calculated based on the temperature coefficient, the average feature, and the positive sample.

[0037] In some embodiments, the step of performing multi-stage training on a preset deep learning model based on the local alignment loss and the domain-invariant supervised contrast loss to obtain the blood cell classification model includes:

[0038] The loss during the representation learning stage is calculated based on the loss ratio hyperparameter, the local alignment loss, and the domain-invariant supervised contrast loss.

[0039] The preset deep learning model is trained in the first stage using the representation learning stage loss.

[0040] Calculate the loss during the classification training phase based on the number of categories, hot-coded labels, and model prediction probability distribution;

[0041] The blood cell classification model is obtained by using the classification training stage loss to train the preset deep learning model after the first stage of training in the second stage.

[0042] In some embodiments, the method further includes:

[0043] A performance evaluation experiment was conducted on the blood cell classification model.

[0044] The performance evaluation experiment of the blood cell classification model includes the following steps:

[0045] Set the number of clusters, the loss ratio hyperparameter, the learning rate, and the number of image samples;

[0046] Based on the number of iterations, the number of clusters, the loss ratio hyperparameter, the learning rate, and the number of image samples, the preset experimental model is trained to obtain the target experimental model.

[0047] The target experimental model is used to classify the experimental dataset to obtain the performance evaluation experimental results.

[0048] In some embodiments, calculating the pixel-level alignment loss based on the blood cell feature embedding and the domain transform feature embedding includes:

[0049] Based on the blood cell feature embedding and the domain transform feature embedding, the pixel-level alignment loss is calculated using the pixel-level alignment loss calculation formula, which is as follows:

[0050]

[0051] In the formula, E is the pixel-level alignment loss. i,p Embedding of the blood cell features, The domain transformation feature embedding is given, where k is the number of clusters and ||·|2 is the L2 norm.

[0052] On the other hand, embodiments of the present invention provide a multi-domain unbalanced blood cell sorting device, comprising:

[0053] The first module is used to acquire images of cells to be classified.

[0054] The second module is used to input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result.

[0055] The blood cell classification model is obtained through the following steps:

[0056] Acquire a multi-domain imbalanced blood cell image set, which includes several blood cell images;

[0057] The multi-domain unbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image, wherein the domain transformation sample set includes several domain transformation samples.

[0058] Based on the blood cell image and the corresponding domain transform sample set, local alignment processing is performed, and local alignment loss is calculated.

[0059] Based on the blood cell image and the corresponding domain transformation sample set, perform domain-invariant contrastive learning processing and calculate the domain-invariant supervised contrastive loss.

[0060] Based on the local alignment loss and the domain-invariant supervised contrast loss, the preset deep learning model is subjected to multi-stage training to obtain the blood cell classification model.

[0061] On the other hand, embodiments of the present invention provide a computer device, including:

[0062] At least one processor;

[0063] At least one memory for storing at least one program;

[0064] When the at least one program is executed by the at least one processor, the at least one processor implements the method.

[0065] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described thereon.

[0066] The beneficial effects of this invention are as follows:

[0067] This invention first acquires images of cells to be classified, then inputs these images into a blood cell classification model to obtain blood cell classification results, thereby achieving blood cell classification and improving classification accuracy. The blood cell classification model is obtained through the following steps: First, a multi-domain imbalanced blood cell image set is acquired. Then, the multi-domain imbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image. Next, based on the blood cell image and its corresponding domain transformation sample set, local alignment processing is performed, and the local alignment loss is calculated. Then, domain-invariant contrastive learning processing is performed, and the domain-invariant supervised contrastive loss is calculated. Finally, based on the local alignment loss and the domain-invariant supervised contrastive loss, a pre-set deep learning model is trained in multiple stages to obtain the blood cell classification model, thereby improving model accuracy.

[0068] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart of a multi-domain unbalanced blood cell classification method according to an embodiment of the present invention;

[0071] Figure 2 This is a flowchart illustrating an embodiment of the present invention for obtaining a blood cell classification model;

[0072] Figure 3 This is a schematic diagram of a staining enhancement process according to an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of a local alignment process according to an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of a domain-invariant contrastive learning process according to an embodiment of the present invention;

[0075] Figure 6 This is a diagram of a blood cell classification network model architecture according to an embodiment of the present invention;

[0076] Figure 7 This is a schematic diagram illustrating the average results of a comparative experiment of different methods on a public dataset according to an embodiment of the present invention;

[0077] Figure 8 This is a schematic diagram illustrating the average results of a comparative experiment of different methods on a non-public dataset according to an embodiment of the present invention;

[0078] Figure 9 This is a schematic diagram illustrating the visual comparison results of a blood cell classification model prediction according to an embodiment of the present invention;

[0079] Figure 10 This is a schematic diagram of the structure of a multi-domain unbalanced blood cell sorting device according to an embodiment of the present invention;

[0080] Figure 11 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0082] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0083] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0085] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0086] Blood cells, also known as "blood spheres," are cells found in the blood and distributed throughout the body by blood flow. In mammals, blood cells mainly consist of three types: Red blood cells: Their primary function is to transport oxygen. White blood cells: They primarily play an immune role. When pathogens invade the body, white blood cells can cross capillary walls, concentrate at the site of invasion, and engulf the pathogens. Platelets: They play a crucial role in hemostasis. Blood cells make up approximately 45% of the blood volume and include red blood cells, white blood cells, and platelets. Under normal physiological conditions, blood cells and platelets have a certain morphological structure and a relatively stable number.

[0087] In related technologies, blood cell classification, as a common cell analysis method, can include cells from leukemia, anemia, infections, autoimmune disorders, and other blood-related diseases. Therefore, accurate and timely classification of blood cells is crucial in clinical practice, playing a key role in timely detection. Traditionally, blood cells in clinical practice are identified manually by skilled experts using microscopes—a repetitive, labor-intensive, and time-consuming process. In recent years, deep learning models have made significant progress in solving real-world classification tasks across various domains. Some researchers have attempted to apply deep learning models to blood cell classification and have shown promising results. However, blood cell images collected from various laboratories and hospitals often exhibit significant domain shifts and data imbalances. Domain shifts can lead to decreased generalization performance and poor performance on unseen datasets, while data imbalances can cause poor performance on minority classes and low classification accuracy.

[0088] In view of this, this embodiment uses staining-aware domain alignment for imbalanced multi-domain blood cell classification. Specifically, a staining-based enhancement method is first introduced to generate domain-transformed samples containing specific domain information of the source domain, while preserving domain-invariant features. Both the original blood cell image and the generated domain-transformed samples are fed into the same feature extraction network (e.g., ResNet50). To encourage the model to capture domain-invariant features, a local alignment constraint is designed to ensure consistency between the original blood cell image and the domain-transformed samples at the feature map level. Furthermore, this embodiment proposes a domain-invariant supervised contrastive learning strategy. By taking the average features of the original blood cell image and the domain-transformed samples as anchors, the influence of domain-specific features can be mitigated, and discriminative feature learning can be promoted. Finally, the training process is decoupled into a representation learning stage and a classification training stage to alleviate the data imbalance problem. Experimental results on four public blood cell datasets and one real dataset demonstrate that this embodiment achieves significantly superior state-of-the-art results and improves classification accuracy.

[0089] This application provides a multi-domain imbalanced blood cell classification method, relating to the field of image recognition technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a multi-domain imbalanced blood cell classification method, but is not limited to the above forms.

[0090] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0091] The embodiments of this application will be explained in detail below with reference to the accompanying drawings:

[0092] Figure 1 This is an optional flowchart of a multi-domain unbalanced blood cell classification method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S101 to S102.

[0093] Step S101: Obtain images of cells to be classified;

[0094] Step S102: Input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result.

[0095] Steps S101 to S102 shown in the embodiments of this application realize blood cell classification and improve classification accuracy.

[0096] In some embodiments, images of cells to be classified can be acquired first, and then input into a blood cell classification model to obtain blood cell classification results, thereby achieving blood cell classification. The accuracy of the blood cell classification results primarily depends on the model precision of the blood cell classification model.

[0097] Figure 2 This is an optional flowchart of a process for obtaining a blood cell classification model provided in the embodiments of this application, which may include, but is not limited to, steps S201 to S205.

[0098] Step S201: Obtain a multi-domain imbalanced blood cell image set, which includes several blood cell images;

[0099] Step S202: Perform staining enhancement processing on the multi-domain unbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image. The domain transformation sample set includes several domain transformation samples.

[0100] Step S203: Perform local alignment processing based on the blood cell image and the corresponding domain transformation sample set, and calculate the local alignment loss;

[0101] Step S204: Based on the blood cell image and the corresponding domain transformation sample set, perform domain-invariant contrastive learning processing and calculate the domain-invariant supervised contrastive loss.

[0102] Step S205: Based on the local alignment loss and the domain-invariant supervised contrast loss, perform multi-stage training on the preset deep learning model to obtain the blood cell classification model.

[0103] Steps S201 to S205, as shown in the embodiments of this application, obtain a blood cell classification model and improve the model accuracy.

[0104] In step S201 of some embodiments, a multi-domain imbalanced blood cell image set can be obtained through a cell database. The multi-domain imbalanced blood cell image set can also be obtained through other methods, and is not limited to these. The multi-domain imbalanced blood cell image set includes several blood cell images, and the number of image samples can be set as the number of blood cell images.

[0105] In some embodiments, step S202 involves performing staining enhancement processing on the multi-domain imbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image. This process may include, but is not limited to, the following steps:

[0106] For each blood cell image in the multi-domain imbalanced blood cell image set, image decomposition is performed to obtain the corresponding density map and staining matrix;

[0107] Based on the number of clusters, multiple coloring matrices are clustered using a pre-defined clustering method to obtain several clusters;

[0108] Set domain labels for the blood cell images corresponding to the staining matrix in each cluster;

[0109] A transformed image set is selected from a multi-domain imbalanced blood cell image set. The transformed image set includes a preset number of transformed images, and the domain labels of each transformed image in the transformed image set are different.

[0110] Select one transformed image from the set of transformed images as the image to be normalized;

[0111] Normalize the density map corresponding to the blood cell image and the density map corresponding to the image to be normalized to obtain a normalized map;

[0112] Calculate the domain transformation sample based on the normalized image, the illumination intensity, and the staining matrix corresponding to the image to be normalized;

[0113] Multiple domain transformation samples are combined to obtain a domain transformation sample set.

[0114] In some embodiments, the staining enhancement process is as follows: Figure 3 As shown in the figure, SNMF represents the Sparse Nonnegative Matrix Factorization method. The role of the staining solution is to color the blood cell image to enhance the visibility of cell structures. The final color effect depends on the specific type and amount of staining solution absorbed by the blood cells. First, image decomposition can be performed on each blood cell image in the multi-domain imbalanced blood cell image set using the Beer-Lambert law to obtain the corresponding density map H and the corresponding staining matrix W. Then, the Sparse Nonnegative Matrix Factorization (SNMF) method is used to optimize the density map H and the staining matrix W. The mathematical expression for image decomposition of the blood cell image is: In the formula, V is the optical density generated by the Beer-Lambert transform, V∈R 3×n X is a blood cell image in RGB space, X∈R 3×n n is the number of pixels, X0 is the illumination intensity (e.g., 255 for an 8-bit image), and W is the coloring matrix, W∈R. 3×r H is a density map, H∈R r×n , where r is the normalized color basis. It's understandable that, assuming a multi-domain imbalanced blood cell image set mini-batch has N samples, each sample x... i Decomposed into coloring matrix W i and density map H iWhere i = 1, 2, ..., N. Then, based on the number of clusters, multiple staining matrices are clustered using a preset clustering method to obtain several clusters. For example, to select samples with significant domain differences, the preset clustering method can be set to the k-means algorithm, the number of clusters can be set to k, and the k-means algorithm can be used to cluster the staining matrix corresponding to each blood cell image in the multi-domain imbalanced blood cell image set to obtain k clusters. Then, domain labels are assigned to the blood cell images corresponding to the staining matrix in each cluster. For example, each cluster generated by k-means has a unique domain label, and for each sample x within a cluster... i Assign the corresponding domain label d to this cluster i ∈{1, 2, ..., k}.

[0115] Then, a transformed image set is selected from the multi-domain imbalanced blood cell image set. This transformed image set includes a predetermined number of transformed images, each with a distinct domain label. For example, when the number of clusters is set to k, excluding the current cluster, a blood cell image can be randomly selected from each of the other clusters with different domain labels (a total of k-1 clusters) as the transformed image, resulting in k-1 transformed images. Each selected transformed image has a corresponding density map and staining matrix. Since there may be scale differences between density maps, additional normalization processing is required. One transformed image can be selected from the transformed image set as the image to be normalized, and the density map H corresponding to the blood cell image is normalized accordingly. i The density map H corresponding to the image to be normalized t Normalization is performed to obtain a normalized graph. The formula for calculating the normalized graph is: GY = Norm(H i H t In the formula, GY is the normalized graph, Norm is the normalization process, and H is the normalized graph. i H is the density map corresponding to the blood cell image. t This is the density map corresponding to the image to be normalized. Then, based on the normalized map, the illumination intensity, and the staining matrix corresponding to the image to be normalized, the domain transformation sample is calculated. The formula for calculating the domain transformation sample is: In the formula, For the domain-transformed sample, X0 is the illumination light intensity, W t This represents the staining matrix corresponding to the image to be normalized. Finally, multiple domain-transformed samples are combined to obtain a domain-transformed sample set. It can be understood that for each blood cell image x... i Each selects k-1 coloring matrices W from different domain labels. i Using k-1 different W i and H i To synthesize k-1 domain transformation samples Specifically, for x i Given k-1 clusters with different domain labels, a coloring matrix W can be randomly selected from each cluster. t To analyze blood cell images x i density map G i Perform a restaining operation. For each blood cell image x in the multi-domain imbalanced blood cell image set mini-batch... i Each generates a corresponding domain transformation sample. Each blood cell image x i Each will generate k-1 domain transformation samples. A total of N(k-1) domain transformation samples are generated.

[0116] In some embodiments, in step S203, local alignment processing is performed based on the blood cell image and the corresponding domain transform sample set, and the local alignment loss is calculated. This may include, but is not limited to, the following steps:

[0117] Feature extraction is performed on blood cell images to obtain blood cell feature maps;

[0118] Feature extraction is performed on each domain transform sample in the domain transform sample set to obtain the corresponding domain transform feature map;

[0119] A nonlinear multilayer perceptron is used to perform feature mapping on the blood cell feature map to obtain the blood cell feature embedding;

[0120] A nonlinear multilayer perceptron is used to perform feature mapping on the domain transform feature map to obtain the domain transform feature embedding;

[0121] Pixel-level alignment loss is calculated based on blood cell feature embedding and domain transformation feature embedding.

[0122] The local alignment loss is calculated based on the number of image samples, the spatial location index of the feature map, and the pixel-level alignment loss.

[0123] In some embodiments, the local alignment process is as follows: Figure 4 As shown, the domain transform samples generated through staining enhancement processing Structure and original blood cell image x i The structures are consistent. To leverage this characteristic, local alignment constraints can be used, combined with a projector, to minimize the difference between the original blood cell image and the domain-transformed sample at the feature map level. Feature extraction can be performed on the blood cell image first to obtain a blood cell feature map, and then feature extraction can be performed on each domain-transformed sample in the domain-transformed sample set to obtain the corresponding domain-transformed feature map. For example, the blood cell feature map F can be extracted from the final layer of the backbone network (ResNet50) (before the global average pooling layer). i,pSum-domain transformation feature map in, D b Let F be the number of output channels of the backbone network, and p be the spatial location index of the feature map. To prevent performance degradation due to representation collapse, a nonlinear multilayer perceptron can be used to optimize the blood cell feature map F. i,p Perform feature mapping to obtain the blood cell feature embedding E i,p And utilize a nonlinear multilayer perceptron to transform the feature map of the domain. Perform feature mapping to obtain the domain transformation feature embedding. Then, based on the blood cell feature embedding and the domain transformation feature embedding, the pixel-level alignment loss is calculated using the pixel-level alignment loss calculation formula, which is as follows: In the formula, For pixel-level alignment loss, E i,p Embedding for blood cell characteristics, For domain transformation feature embedding, k is the number of clusters, and ||·||² is the L2 norm. Finally, based on the number of image samples, the spatial location index of the feature map, and the pixel-level alignment loss, the local alignment loss is calculated. The formula for calculating the local alignment loss is: In the formula, For local alignment loss, N is the number of image samples (i.e., the number of blood cell images in the multi-domain imbalanced blood cell image set), and p is the spatial location index of the feature map.

[0124] In some embodiments, step S204 involves performing domain-invariant contrastive learning processing based on the blood cell image and the corresponding domain-transformed sample set, and calculating the domain-invariant supervised contrastive loss. This process may include, but is not limited to, the following steps:

[0125] Obtain the blood cell feature projection of the blood cell image and the domain transformation feature projection corresponding to each domain transformation sample in the domain transformation sample set;

[0126] L2 normalization was performed on the blood cell characteristic projections;

[0127] L2 normalization is applied to the domain transformation feature projection;

[0128] The average feature is calculated based on the L2-normalized blood cell feature projection and the L2-normalized domain transformation feature projection.

[0129] Calculate positive samples based on class labels;

[0130] The domain-invariant supervised contrast loss is calculated based on the temperature coefficient, average features, and positive samples.

[0131] In some embodiments, the domain-invariant contrastive learning process is as follows: Figure 5As shown, in the presence of domain transfer, the selected anchor point may contain domain-specific features, thus affecting the performance of contrastive learning to some extent. We can first obtain the blood cell feature projection of the blood cell image and the domain transformation feature projection corresponding to each domain transformation sample in the domain transformation sample set. For example, in each mini-batch, after passing through the backbone network (ResNet50) f... b (·) and Projector f p After (·), the original blood cell image x i Sum-domain transformation samples Their feature projections are obtained separately. Then, L2 normalization is performed on both the blood cell feature projection and the domain transform feature projection to utilize the dot product as a distance metric. Finally, based on the L2-normalized blood cell feature projection and the L2-normalized domain transform feature projection, the average feature *a* is calculated. i The average feature is used as an anchor point to obtain anchor points less affected by domain-specific features. Understandably, the anchor point can be obtained by calculating the average of the feature projections between the original blood cell image and the domain-transformed sample. Then, based on the class label, positive samples are calculated, where the formula for calculating positive samples is: In the formula, For positive samples, y i It is x i The class tag, z j Projection of blood cell characteristics The domain transformation features are projected. Finally, based on the temperature coefficient, average features, and positive samples, the domain-invariant supervised contrastive loss is calculated. The formula for calculating the domain-invariant supervised contrastive loss is: In the formula, For domain-invariant supervised contrastive loss, N is the number of image samples, a i For the average feature, z k Let τ be the domain transformation feature projection of the k-th domain transformation sample, where τ is the temperature coefficient and |·| is the number of elements in the set.

[0132] In some embodiments, in step S205, the preset deep learning model is subjected to multi-stage training based on local alignment loss and domain-invariant supervised contrast loss to obtain a blood cell classification model, which may include, but is not limited to, the following steps:

[0133] The loss during the representation learning stage is calculated based on the loss ratio hyperparameter, local alignment loss, and domain-invariant supervised contrast loss.

[0134] The pre-defined deep learning model is trained in the first stage using the representation learning stage loss.

[0135] Calculate the loss during the classification training phase based on the number of categories, hot-coded labels, and model prediction probability distribution;

[0136] The blood cell classification model is obtained by using the classification training stage loss to train the pre-set deep learning model after the first stage of training in the second stage.

[0137] In some embodiments, in multi-stage training, the training process is divided into two stages: a representation learning stage and a classification training stage. This two-stage training approach allows the model to learn features more focused on the minority class, thereby improving minority class performance. The representation learning stage loss can be calculated first based on the loss scaling hyperparameter, local alignment loss, and domain-invariant supervised contrastive loss. The formula for calculating the representation learning stage loss is as follows: In the formula, To characterize the loss during the learning phase, The loss is a domain-invariant supervised contrastive loss, where β is a loss scaling hyperparameter used to control the ratio of the two loss terms. This is the local alignment loss. Then, the representation learning stage loss is used to perform the first stage of training on the pre-defined deep learning model, i.e., training in the representation learning stage. Next, based on the number of categories, hot-coded labels, and the model's predicted probability distribution, the classification training stage loss is calculated. The formula for calculating the classification training stage loss is: In the formula, For the loss during the classification training phase, Let y be the cross-entropy loss function, C be the number of classes, and y be the number of classes. i For hot-coded tags, The model predicts the probability distribution. Finally, the loss from the classification training stage is used to train the pre-set deep learning model after the first stage of training in the second stage, i.e., the training in the classification training stage, to obtain the blood cell classification model.

[0138] In some embodiments, the blood cell classification network model architecture diagram is as follows: Figure 6 As shown, the original blood cell image can first undergo staining enhancement processing to obtain multiple domain-transformed samples. These samples are then simultaneously input into the ResNet50 backbone network with weight sharing. The network output features are then locally aligned, followed by global average pooling, feature flattening, and a projection head. Finally, domain-invariant supervised contrastive learning is combined with this multi-stage training to obtain the blood cell classification model. In the figure, SBA (Stain-baesdaugmentation) represents staining enhancement processing, LA (Local alignment) represents local alignment, GAP (Global average pooling) represents global average pooling, and DISC (Domain-invariant supervised contrastive learning) represents domain-invariant contrastive learning.

[0139] In some embodiments, the method further includes:

[0140] Performance evaluation experiments were conducted on the blood cell classification model;

[0141] The performance evaluation experiment of the blood cell classification model includes the following steps:

[0142] Set the number of clusters, the loss ratio hyperparameter, the learning rate, and the number of image samples;

[0143] The target experimental model is obtained by training the preset experimental model based on the number of iterations, number of clusters, loss ratio hyperparameter, learning rate and number of image samples;

[0144] The experimental dataset is classified using the target experimental model to obtain the performance evaluation experimental results.

[0145] In some embodiments, performance evaluation experiments can be conducted on the blood cell classification model. This embodiment performed extensive experiments on four imbalanced blood cell image datasets: Acevedo-20, LDWBC, Raabin-WBC, and Zheng-18, and also used a private real-world dataset for testing. The number of clusters, loss ratio hyperparameter, learning rate, and number of image samples can be set first. Then, based on the number of iterations, number of clusters, loss ratio hyperparameter, learning rate, and number of image samples, the preset experimental model is trained to obtain the target experimental model. Finally, the target experimental model is used to classify the experimental dataset to obtain the performance evaluation experimental results. For example, this can be performed on a server equipped with two NVIDIA 4090 graphics cards and one Intel i9-10900X processor. The number of clusters k and the loss ratio hyperparameter β are set to 3 and 0.1, respectively, the learning rate is set to 0.00005, and the mini-batch size for each dataset is set to 32. During the representation learning phase, the model training iterated for 4000 steps, and during the classification training phase, the model also iterated for 4000 steps. To improve model training and evaluation, this embodiment performs a five-class classification task involving neutrophils, eosinophils, basophils, monocytes, and lymphocytes. Due to the imbalance of the dataset, the F1-macro score is used as the evaluation metric to reflect the model's classification performance when handling imbalanced data. Additionally, the F1-micro score is reported, providing a comprehensive metric through overall statistics across all classes. The method of this embodiment (hereinafter referred to as SADA) is compared with several state-of-the-art methods, namely ERM, CORAL, Mixup, SelfReg, and BoDA. To ensure a fair comparison, all algorithms use the same backbone network (i.e., a ResNet50 pre-trained on ImageNet).

[0146] For all datasets, a leave-one-out strategy is used for evaluation, with one dataset designated as the target domain and the remaining datasets as the source domain. The average results of three experiments using different comparison methods and the method in this embodiment are as follows: Figure 7 As shown, the method in this embodiment achieves an average F1-micro score of 84.4% and an F1-macro score of 68.6%, exceeding the second-highest performing method by 4.4% and 3.9%, respectively. For each dataset, the method in this embodiment achieves near-best results, except for the Raabin-WBC dataset, where it performs comparably to the Mixup method, achieving an F1-micro score of 87.4% and an F1-macro score of 61.9%. Conversely, the method in this embodiment significantly outperforms the second-highest performing method on the Acevedo-20 dataset, achieving F1-micro and F1-macro scores of 79.9% and 81.6%, respectively, exceeding the second-highest performing method by 9.2% and 8.0%. After training on four public datasets, the method in this embodiment was evaluated on a private real-world dataset. Figure 8 As shown, the method in this embodiment outperforms other comparable methods with an F1-micro score of 85.4% and an F1-macro score of 85.1%, exceeding the second-highest performing method by 2.1% and 1.8%, respectively. The SADA method, specifically designed for classifying imbalanced blood cell datasets, outperforms other methods and demonstrates excellent performance. To evaluate the effectiveness of SADA, ablation experiments were conducted on four public datasets. The method in this embodiment is compared with the following two variants: (1) SADAw / o la, representing SADA without local alignment constraints; and (2) SADAw / sc, representing SADA with domain-invariant supervised contrast loss replaced by supervised contrast loss. Other configurations for these two variants remain the same as the original design to ensure a fair comparison. Evaluation results are listed in Figure 7 Lines 7-8 of the diagram show that the lack of local alignment constraints leads to a certain degree of degradation in the overall model performance. Meanwhile, the domain-invariant supervised contrastive learning loss proposed in this embodiment significantly outperforms supervised contrastive learning loss. To further explore the model's classification performance, the prediction results of the blood cell classification model were visualized using GradCam. The visualization comparison results of the blood cell classification model predictions are shown below. Figure 9 As shown, compared to other methods, the method in this embodiment can more comprehensively and accurately focus on blood cells in the image.

[0147] In some embodiments, the method of this embodiment includes staining enhancement, local alignment, and domain-invariant contrastive learning. Staining enhancement effectively enriches the diversity of the sample domains and further increases the number of samples. Through local alignment constraints, the model can capture domain-invariant features at the feature map level. Domain-invariant contrastive learning utilizes the centroids of the original blood cell images and the domain-transformed samples as anchors to enhance the model's ability to learn discriminative features. Extensive experiments on four public blood cell datasets and one private real-world dataset demonstrate that the method of this embodiment effectively improves classification accuracy.

[0148] The beneficial effects of implementing the embodiments of the present invention include: The embodiments of the present invention first acquire images of cells to be classified, then input these images into a blood cell classification model to obtain blood cell classification results, thereby achieving blood cell classification and improving classification accuracy. Specifically, the blood cell classification model is obtained through the following steps: First, a multi-domain imbalanced blood cell image set is acquired; then, the multi-domain imbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image; next, based on the blood cell image and its corresponding domain transformation sample set, local alignment processing is performed, and the local alignment loss is calculated; then, domain-invariant contrastive learning processing is performed, and the domain-invariant supervised contrastive loss is calculated; finally, based on the local alignment loss and the domain-invariant supervised contrastive loss, a pre-set deep learning model is trained in multiple stages to obtain the blood cell classification model, thereby improving model accuracy.

[0149] like Figure 10 As shown, this embodiment of the invention also provides a multi-domain unbalanced blood cell sorting device, comprising:

[0150] The first module 801 is used to acquire images of cells to be classified.

[0151] The second module 802 is used to input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result;

[0152] The blood cell classification model is obtained through the following steps:

[0153] Acquire a multi-domain imbalanced blood cell image set, which includes several blood cell images;

[0154] A staining enhancement process is performed on a multi-domain unbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image. The domain transformation sample set includes several domain transformation samples.

[0155] Based on the blood cell image and the corresponding domain transformation sample set, local alignment processing is performed, and the local alignment loss is calculated.

[0156] Based on the blood cell image and the corresponding domain transformation sample set, domain-invariant contrastive learning processing is performed, and domain-invariant supervised contrastive loss is calculated.

[0157] Based on local alignment loss and domain-invariant supervised contrast loss, a multi-stage training process is performed on the preset deep learning model to obtain a blood cell classification model.

[0158] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0159] like Figure 11 As shown, embodiments of the present invention also provide a computer device, including:

[0160] At least one processor 901;

[0161] At least one memory 902 is used to store at least one program;

[0162] When at least one program is executed by at least one processor, such that at least one processor achieves Figure 1 The method shown.

[0163] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0164] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figure 1 The method shown.

[0165] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0166] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for classifying multi-domain unbalanced blood cells, characterized in that, Includes the following steps: Acquire images of cells to be classified; Input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result; The blood cell classification model is obtained through the following steps: Acquire a multi-domain imbalanced blood cell image set, which includes several blood cell images; The multi-domain unbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image, wherein the domain transformation sample set includes several domain transformation samples. Based on the blood cell image and the corresponding domain transform sample set, local alignment processing is performed, and local alignment loss is calculated. Based on the blood cell image and the corresponding domain transformation sample set, perform domain-invariant contrastive learning processing and calculate the domain-invariant supervised contrastive loss. Based on the local alignment loss and the domain-invariant supervised contrast loss, the preset deep learning model is subjected to multi-stage training to obtain the blood cell classification model. The step of performing staining enhancement processing on the multi-domain imbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image includes: Image decomposition is performed on each of the blood cell images in the multi-domain imbalanced blood cell image set to obtain the corresponding density map and staining matrix; Based on the number of clusters, multiple coloring matrices are clustered using a preset clustering method to obtain several clusters; Set domain labels for the blood cell images corresponding to the staining matrix in each cluster; A transformed image set is selected from the multi-domain unbalanced blood cell image set. The transformed image set includes a preset number of transformed images, and the domain labels of each transformed image in the transformed image set are different. Select one transformed image from the set of transformed images as the image to be normalized; The density map corresponding to the blood cell image and the density map corresponding to the image to be normalized are normalized to obtain a normalized map; Calculate the domain transformation sample based on the normalized image, the illumination light intensity, and the staining matrix corresponding to the image to be normalized; Multiple domain transformation samples are combined to obtain the domain transformation sample set.

2. The method according to claim 1, characterized in that, The step of performing local alignment processing based on the blood cell image and the corresponding domain transform sample set, and calculating the local alignment loss, includes: Feature extraction is performed on the blood cell image to obtain a blood cell feature map; Feature extraction is performed on each domain transform sample in the domain transform sample set to obtain the corresponding domain transform feature map; The blood cell feature map is mapped using a nonlinear multilayer perceptron to obtain the blood cell feature embedding; The domain transform feature map is mapped using a nonlinear multilayer perceptron to obtain the domain transform feature embedding. Calculate pixel-level alignment loss based on the blood cell feature embedding and the domain transformation feature embedding; The local alignment loss is calculated based on the number of image samples, the spatial location index of the feature map, and the pixel-level alignment loss.

3. The method according to claim 1, characterized in that, The step of performing domain-invariant contrastive learning processing based on the blood cell image and the corresponding domain-transformed sample set, and calculating the domain-invariant supervised contrastive loss, includes: Obtain the blood cell feature projection of the blood cell image and the domain transformation feature projection corresponding to each domain transformation sample in the domain transformation sample set; L2 normalization was performed on the blood cell feature projections; The domain transformation feature projection is L2 normalized; The average feature is calculated based on the L2-normalized blood cell feature projection and the L2-normalized domain transformation feature projection. Calculate positive samples based on class labels; The domain-invariant supervised contrast loss is calculated based on the temperature coefficient, the average feature, and the positive sample.

4. The method according to claim 1, characterized in that, The blood cell classification model is obtained by performing multi-stage training on a preset deep learning model based on the local alignment loss and the domain-invariant supervised contrast loss, including: The loss during the representation learning stage is calculated based on the loss ratio hyperparameter, the local alignment loss, and the domain-invariant supervised contrast loss. The preset deep learning model is trained in the first stage using the representation learning stage loss. Calculate the loss during the classification training phase based on the number of categories, hot-coded labels, and model prediction probability distribution; The blood cell classification model is obtained by using the classification training stage loss to train the preset deep learning model after the first stage of training in the second stage.

5. The method according to claim 1, characterized in that, The method further includes: A performance evaluation experiment was conducted on the blood cell classification model. The performance evaluation experiment of the blood cell classification model includes the following steps: Set the number of clusters, the loss ratio hyperparameter, the learning rate, and the number of image samples; Based on the number of iterations, the number of clusters, the loss ratio hyperparameter, the learning rate, and the number of image samples, the preset experimental model is trained to obtain the target experimental model. The target experimental model is used to classify the experimental dataset to obtain the performance evaluation experimental results.

6. The method according to claim 2, characterized in that, The step of calculating pixel-level alignment loss based on the blood cell feature embedding and the domain transform feature embedding includes: Based on the blood cell feature embedding and the domain transform feature embedding, the pixel-level alignment loss is calculated using the pixel-level alignment loss calculation formula, which is as follows: ； In the formula, For the pixel-level alignment loss, Embedding of the blood cell features, Embedding of the domain transformation features, For the number of clusters, It is an L2 norm.

7. A multi-domain unbalanced blood cell sorting device, characterized in that, include: The first module is used to acquire images of cells to be classified. The second module is used to input the cell image to be classified into the blood cell classification model to obtain the blood cell classification result. The blood cell classification model is obtained through the following steps: Acquire a multi-domain imbalanced blood cell image set, which includes several blood cell images; The multi-domain unbalanced blood cell image set is subjected to staining enhancement processing to obtain a domain transformation sample set corresponding to each blood cell image, wherein the domain transformation sample set includes several domain transformation samples. Based on the blood cell image and the corresponding domain transform sample set, local alignment processing is performed, and local alignment loss is calculated. Based on the blood cell image and the corresponding domain transformation sample set, perform domain-invariant contrastive learning processing and calculate the domain-invariant supervised contrastive loss. Based on the local alignment loss and the domain-invariant supervised contrast loss, the preset deep learning model is subjected to multi-stage training to obtain the blood cell classification model. The step of performing staining enhancement processing on the multi-domain imbalanced blood cell image set to obtain a domain transformation sample set corresponding to each blood cell image includes: Image decomposition is performed on each of the blood cell images in the multi-domain imbalanced blood cell image set to obtain the corresponding density map and staining matrix; Based on the number of clusters, multiple coloring matrices are clustered using a preset clustering method to obtain several clusters; Set domain labels for the blood cell images corresponding to the staining matrix in each cluster; A transformed image set is selected from the multi-domain unbalanced blood cell image set. The transformed image set includes a preset number of transformed images, and the domain labels of each transformed image in the transformed image set are different. Select one transformed image from the set of transformed images as the image to be normalized; The density map corresponding to the blood cell image and the density map corresponding to the image to be normalized are normalized to obtain a normalized map; Calculate the domain transformation sample based on the normalized image, the illumination light intensity, and the staining matrix corresponding to the image to be normalized; Multiple domain transformation samples are combined to obtain the domain transformation sample set.

8. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images