Domain generalization and domain adaptation blood cell classification method based on feature decoupling

Abstract

The present invention specifically discloses a domain generalization and domain adaptation blood cell classification method based on feature decoupling, the method comprising the following steps: S1, building a deep neural network blood cell classification model; S2, constructing a twin network feature decoupling module; S3, preprocessing the source domain blood cell image, and inputting the source domain blood cell image and the preprocessed source domain blood cell image into the twin network feature decoupling module for training; S4, batch normalizing the two outputs of the twin network feature decoupling module, and aligning the style information and content information of the two outputs using KL divergence, and then optimizing the classification model through a predefined classification loss function; S5, predicting and classifying blood cell test samples in the target domain. The present invention adopts a contrastive learning method, which effectively reduces the training time of the network model, improves the classification accuracy and generalization performance of blood cell images, and improves the detection performance of blood cell images under actual complex conditions.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology of artificial intelligence, and in particular to a domain generalization and domain adaptation blood cell classification method based on feature decoupling. Background Art

[0002] Deep convolutional neural networks dominate various vision tasks, including image classification, object detection, and segmentation. However, their training requires a large number of annotated images, which is relatively expensive. Furthermore, these networks rely on the independent and identically distributed assumption for training and test data, a requirement that is difficult to meet in practical applications, resulting in reduced model generalization. Therefore, domain adaptation and domain generalization learning have been proposed to address the performance degradation caused by domain shift. Domain adaptation addresses this issue by reducing the difference in feature representation distribution between the source and target domains. However, since domain adaptation builds models based on single-domain data, while the source and target domains can come from multiple datasets with different distributions, domain adaptation has certain limitations. Domain generalization, on the other hand, enhances model generalization by learning domain-invariant features from multiple sources of data.

[0003] A fundamental idea in domain adaptation and domain generalization is to disentangle domain and semantic information from features and extract domain-invariant content. Yaroslav Gani et al. published "Unsupervised Domain Adaptation by Back Propagation" at the 2015 ICML conference, proposing a domain adaptation method based on a generative adversarial network. This method aims to confuse the network to distinguish between the source and target domains, and also proposes a domain discriminator with a gradient reversal layer to ensure similar feature distributions between the two domains. Saeid Motiian et al. published "Unified Deep Supervised Domain Adaptation and Generalization" at the 2017 ICCV conference, using a Siamese network to encode the source and target domains separately and using a contrastive loss function to align domain and semantic information.

[0004] Although the above-mentioned domain adaptation and domain generalization learning methods based on deep convolutional neural networks have achieved certain results in solving the problem of domain shift, due to the overly complex structure of the constructed network model and the large number of model parameters, the model training process is lengthy and difficult, and it is difficult to maintain good domain adaptation and domain generalization effects on various data sets. At the same time, the relevant principles of domain shift lack a certain degree of explainability. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of complex network model structure, insufficient generalization ability, and poor principle interpretability of domain adaptation and domain generalization methods in current blood cell classification tasks, and to provide a domain generalization and domain adaptation blood cell classification method based on feature decoupling. Its purpose is to use contrastive learning to decouple the style features and content features of blood cell images, clearly explain the working principle of domain shift, and improve the generalization ability from source domain to target domain.

[0006] To solve the above technical problems, the present invention provides a domain generalization and domain adaptation blood cell classification method based on feature decoupling, the method comprising the following steps:

[0007] S1. Build a deep neural network blood cell classification model with VGG16 network as the backbone network;

[0008] S2. Based on the Siamese network, the style and content information of the source domain blood cell images are extracted through the Gram matrix and Squeeze-excitation operation, and then the Siamese network feature decoupling module is constructed;

[0009] S3. Preprocess the source domain blood cell image, and then input the source domain blood cell image and the preprocessed corresponding source domain blood cell image into the twin network feature decoupling module for training;

[0010] S4. Batch normalize the two outputs of the twin network feature decoupling module after training, and use KL divergence to align the style information and content information of the two batch normalized outputs. Then, input the aligned source domain blood cell image into the deep neural network blood cell classification model for training. The predefined classification loss function is optimized based on the stochastic gradient descent method to obtain the trained deep neural network blood cell classification model.

[0011] S5. Use the trained deep neural network blood cell classification model to predict and classify blood cell test samples in the target domain.

[0012] Preferably, the deep neural network blood cell classification model includes a first module, a second module, a third module, a fourth module, a fifth module and a sixth module connected in sequence, wherein the first module includes a first convolutional layer, a feature decoupling submodule, a second convolutional layer and a first maximum pooling layer connected in sequence, the second module includes a third convolutional layer, a fourth convolutional layer and a second maximum pooling layer connected in sequence, the third module includes a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer and a third maximum pooling layer connected in sequence, the fourth module includes an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer and a fourth maximum pooling layer connected in sequence, the fifth module includes an eleventh convolutional layer, a twelfth convolutional layer, a thirteenth convolutional layer and a fifth maximum pooling layer connected in sequence, and the sixth module includes a Flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer and a Softmax function regression prediction category connected in sequence.

[0013] Preferably, the specific implementation of step S2 includes:

[0014] S21, input the same batch of source domain blood cell image feature map I∈R N×C×H×W Perform style instance normalization and then extract the style normalized source domain blood cell image feature map B∈R N×C×H×W , where N represents the number of feature maps of blood cell images in the same batch of source domains, C represents the number of channels, H and W represent the number of spatial dimensions, and the instance normalization operation IN(·) is expressed as:

[0015]

[0016]

[0017]

[0018] Where, I nchw represents the feature map of the nchw-th source domain blood cell image, c is the index of the number of channels, h and w are the spatial dimensions, μ nc is the mean of the feature maps of the same batch of source domain blood cell images, is the standard deviation of the feature maps of the same batch of source domain blood cell images, and ε represents a preset constant;

[0019] S22, subtract the input source domain blood cell image feature map from the corresponding style normalized source domain blood cell image feature map to obtain the remaining source domain blood cell image style coupling feature S D , denoted as S D =IB=I-IN(I);

[0020] S23, extracting domain-independent source domain blood cell image style representation weights based on Gram matrix;

[0021] S24. Use the Gram matrix to decouple the style information of a specific field and calculate the style weight λ that is not related to the specific field. Di and the style weight λ associated with that particular domain Dr , where the style weight λ is independent of the specific domain Di It can be expressed as:

[0022]

[0023]

[0024] The style weight λ associated with this particular domain Dr It can be expressed as:

[0025] λ Dr =softmax(1-λ Di )

[0026] Where G represents the Gram matrix, i and j represent the rows and columns of the Gram matrix respectively, and Softmax represents the normalized exponential function;

[0027] S25. Extracting style information S of the input blood cell feature map D+ ∈R N×C×H×W and content information C D ∈R N×C×H×W , and construct a twin network feature decoupling module based on the style information and content information of the extracted blood cell feature map, where the style information and content information of the blood cell feature map are expressed as follows:

[0028] S D+ =F SE (S D ·λ Dr )

[0029] C D =B+F SE (S D ·λ Di )

[0030] Where, F SE (·) represents the Squeeze operation and the Excitation operation, where the Squeeze operation F sq (·) represents the characteristic graph u of each blood cell nc ∈R C×H×W Perform global average pooling F sq (·), and then obtain a 1×1×C global information data z c , expressed as:

[0031]

[0032] Excitation operation F ex (·) indicates obtaining the weight s of all features in C channels, which can be expressed as:

[0033] s=F ex (z c ,W)=σ(W2δ(W1z c ))

[0034] Among them, W1 and W2 are fully connected layers, and the size of W1 is The size of W2 is r is the scaling factor, δ(·) is the ReLU function, and σ(·) is the sigmoid function;

[0035] F SE (u c ) represents the obtained feature weight s and tensor u nc Multiplication, that is, F SE (u c )=s·u c .

[0036] Preferably, the specific implementation of step S23 is as follows: the input source domain blood cell feature map is vectorized by tiling, and the vectorized feature map source domain blood cells are organized into a Hilbert space based on the Gram matrix, and then the inner product in the Gram matrix is converted into cosine similarity, thereby reducing the norm difference of each channel, and finally using the predefined feature space to capture the style information of each row in the Gram matrix, thereby extracting the domain-independent source domain blood cell image style representation weight.

[0037] Preferably, the style information of each row in the Gram matrix is captured using a predefined feature space and expressed as:

[0038]

[0039] Where i and j represent the rows and columns of the Gram matrix respectively.

[0040] Preferably, the preprocessing of the source domain blood cell image in step S3 specifically includes: first randomly cropping the source domain blood cell image to obtain a blood cell cropped image with a size of 120×120; then horizontally flipping the blood cell cropped image with a probability of 0.5; and finally randomly changing the brightness, contrast, saturation and hue of the blood cell cropped image.

[0041] Preferably, the specific implementation of step S4 includes:

[0042] S41, batch normalize the content information feature map and style information feature map output by the twin network feature decoupling module, and then obtain the corresponding global and local context information based on adaptive average pooling and generate N-dimensional samples x1,…,x N ;

[0043] S42, based on Parzenwindow and Probability density estimation is performed, which is expressed as follows:

[0044]

[0045]

[0046]

[0047]

[0048] Where, and They represent the input source domain blood cell image content information probability density, the input source domain blood cell image style information probability density, the preprocessed source domain blood cell image content information probability density, and the preprocessed source domain blood cell image style information probability density, respectively. It is a window function composed of a Gaussian function with a variance of 0.04, that is, the closer the distance to x, the greater the counting weight. and They represent the content information feature map samples and style information feature map samples of the input source domain blood cell image, and They respectively represent the content information feature map samples and style information feature map samples of the source domain blood cell image after preprocessing;

[0049] S43. Use KL divergence to minimize content information C D The difference between them and the maximum style information S D+ The difference between them is used to obtain the content information loss function and the style information loss function The content information loss function and the style information loss function are constructed as a contrast loss function, where the loss formula of content information is:

[0050]

[0051] The loss formula of style information is:

[0052]

[0053] The contrast loss function is expressed as follows:

[0054]

[0055] S44. Input the content features of the source domain blood cell image after KL divergence processing into the deep neural network blood cell classification model, and define the final regression loss function as the cross entropy function, which is expressed as follows:

[0056]

[0057] Where M is the number of categories, y ic is a sign function, i.e. 1 or 0, when the true label of sample i is the same as the predicted label c, it is 1, otherwise it is 0, p ic is the predicted probability that sample i belongs to category c;

[0058] S45, the contrast loss function L Dual and the cross entropy function L Task Add and use as the predefined classification loss function, that is, L = L Dual +L Task , and then the predefined classification loss function is optimized using the stochastic gradient descent method to obtain a trained deep neural network blood cell classification model.

[0059] Compared with the prior art, the present invention has the following beneficial technical effects:

[0060] 1. Compared with existing domain generalization and domain adaptation classification methods, this paper uses contrastive learning to construct a blood cell classification network model. At the same time, it uses instance normalization and batch normalization to adjust the model's statistics according to the new domain. Without the need for additional parameters, it avoids the adverse effects of network structure complexity on model training and generalization in other methods.

[0061] 2. The present invention decouples the cross-domain features of blood cell images into style features and content features, and uses KL divergence to match cross-domain distributions, effectively explaining the working principle of the present invention in solving the domain shift problem, and significantly improving the classification accuracy and generalization performance of blood cell images, thereby enhancing the image detection performance of blood cells under complex actual conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a domain generalization and domain adaptation blood cell classification method based on feature decoupling of the present invention.

[0063] Figure 2 This is a network structure diagram of the deep neural network blood cell classification model in the present invention;

[0064] Figure 3 This is a structural diagram of the twin network feature decoupling module described in the present invention;

[0065] Figure 4 This is a visualization of source domain data and target domain data using t-SEN in the domain generalization task of an embodiment of the present invention, where (a) is the distribution of source domain data, (b) is the data representation of the target domain by the VGG16 method, and (c) is the data representation of the target domain by this method. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0067] like Figure 1 、 Figure 2 and Figure 3 As shown, a domain generalization and domain adaptation blood cell classification method based on feature decoupling comprises the following steps:

[0068] S1. Build a deep neural network blood cell classification model with VGG16 network as the backbone network;

[0069] In this step, the deep neural network blood cell classification model includes a first module, a second module, a third module, a fourth module, a fifth module and a sixth module connected in sequence, wherein the first module includes a first convolutional layer, a feature decoupling submodule, a second convolutional layer and a first maximum pooling layer connected in sequence, the second module includes a third convolutional layer, a fourth convolutional layer and a second maximum pooling layer connected in sequence, the third module includes a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer and a third maximum pooling layer connected in sequence, the fourth module includes an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer and a fourth maximum pooling layer connected in sequence, the fifth module includes an eleventh convolutional layer, a twelfth convolutional layer, a thirteenth convolutional layer and a fifth maximum pooling layer connected in sequence, and the sixth module includes a Flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer and a Softmax function regression prediction category connected in sequence;

[0070] Among them, the first and second convolutional layers are 64-dimensional convolutional layers with a size of 3×3 and a padding of 1; the third and fourth convolutional layers are 128-dimensional convolutional layers with a size of 3×3 and a padding of 1; the fifth, sixth, and seventh convolutional layers are 256-dimensional convolutional layers with a size of 3×3 and a padding of 1; the eighth, ninth, and tenth convolutional layers are 512-dimensional convolutional layers with a size of 3×3 and a padding of 1; the eleventh, twelfth, and thirteenth convolutional layers are 512-dimensional convolutional layers with a size of 3×3 and a padding of 1; the padding operation is to expand the image matrix with 0 elements on all sides, and each convolutional layer and fully connected layer is followed by a ReLU activation function layer.

[0071] S2. Based on the Siamese network, the style and content information of the source domain blood cell images are extracted through the Gram matrix and Squeeze-excitation operation, and then the Siamese network feature decoupling module is constructed;

[0072] This step specifically includes:

[0073] S21, input the same batch of source domain blood cell image feature map I∈R N×C×H×W Perform style instance normalization and then extract the style normalized source domain blood cell image feature map B∈R N×C×H×W , where N represents the number of feature maps of blood cell images in the same batch of source domains, C represents the number of channels, H and W represent the number of spatial dimensions, and the instance normalization operation IN(·) is expressed as:

[0074]

[0075]

[0076]

[0077] Where, I nchw represents the feature map of the nchw-th source domain blood cell image, c is the index of the number of channels, h and w are the spatial dimensions, μ nc is the mean of the feature maps of the same batch of source domain blood cell images, is the standard deviation of the feature maps of the same batch of source domain blood cell images, and ε represents a preset constant;

[0078] S22, subtract the input source domain blood cell image feature map from the corresponding style normalized source domain blood cell image feature map to obtain the remaining source domain blood cell image style coupling feature S D , denoted as S D =IB=I-IN(I);

[0079] S23. Extracting domain-independent source domain blood cell image style representation weights based on the Gram matrix, specifically: vectorizing the input source domain blood cell feature map by tiling, and forming the vectorized feature map source domain blood cells into a Hilbert space based on the Gram matrix, and then converting the inner product in the Gram matrix into cosine similarity, thereby reducing the norm difference of each channel, each row in the Gram matrix represents a feature map, and finally using a predefined feature space to capture the style information of each row in the Gram matrix, thereby extracting domain-independent source domain blood cell image style representation weights, wherein the style information of each row in the Gram matrix captured by using the predefined feature space is expressed as follows:

[0080]

[0081] Where i and j represent the rows and columns of the Gram matrix, respectively. By extracting the style representation weights of the source domain blood cell images that are independent of the domain, learning is performed, and the extracted style information can be effectively applied to other fields.

[0082] S24. Use the Gram matrix to decouple the style information of a specific field and calculate the style weight λ that is not related to the specific field. Di and the style weight λ associated with that particular domain Dr , where the style weight λ is independent of the specific domain Di It can be expressed as:

[0083]

[0084]

[0085] The style weight λ associated with this particular domain Dr It can be expressed as:

[0086] λ Dr =softmax(1-λ Di )

[0087] Where G represents the Gram matrix, i and j represent the rows and columns of the Gram matrix respectively, and Softmax represents the normalized exponential function;

[0088] S25. Extracting style information S of the input blood cell feature map D+ ∈R N×C×H×W and content information C D ∈R N×C×H×W, and construct a twin network feature decoupling module based on the style information and content information of the extracted blood cell feature map, where the style information and content information of the blood cell feature map are expressed as follows:

[0089] S D+ =F SE (S D ·λ Dr )

[0090] C D =B+F SE (S D ·λ Di )

[0091] Where, F SE (·) represents the Squeeze operation and the Excitation operation, where the Squeeze operation F sq (·) represents the characteristic graph u of each blood cell nc ∈R C×H×W Perform global average pooling F sq (·), and then obtain a 1×1×C global information data z c , expressed as:

[0092]

[0093] Excitation operation F ex (·) indicates obtaining the weight s of all features in C channels, which can be expressed as:

[0094] s=F ex (z c ,W)=σ(W2δ(W1z c ))

[0095] Among them, W1 and W2 are fully connected layers, and the size of W1 is The size of W2 is r is the scaling factor, δ(·) is the ReLU function, and σ(·) is the sigmoid function;

[0096] F SE (u c ) represents the obtained feature weight s and tensor u nc Multiplication, that is, F SE (u c )=s·u c

[0097] S3. Preprocess the source domain blood cell image, and then input the source domain blood cell image and the preprocessed corresponding source domain blood cell image into the twin network feature decoupling module for training;

[0098] This step is as follows: first, randomly crop the source domain blood cell image to obtain a blood cell cropped image with a size of 120×120; then, horizontally flip the blood cell cropped image with a probability of 0.5; finally, randomly change the brightness, contrast, saturation, and hue of the blood cell cropped image.

[0099] S4. Batch normalize the two outputs of the twin network feature decoupling module after training, and use KL divergence to align the style information and content information of the two batch normalized outputs. Then, input the aligned source domain blood cell image into the deep neural network blood cell classification model for training. The predefined classification loss function is optimized based on the stochastic gradient descent method to obtain the trained deep neural network blood cell classification model.

[0100] This step specifically includes:

[0101] S41, batch normalize the content information feature map and style information feature map output by the twin network feature decoupling module, and then obtain the corresponding global and local context information based on adaptive average pooling and generate N-dimensional samples x1,…,x N ;

[0102] S42, based on Parzenwindow and Probability density estimation is performed, which is expressed as follows:

[0103]

[0104]

[0105]

[0106]

[0107] Where, and They represent the input source domain blood cell image content information probability density, the input source domain blood cell image style information probability density, the preprocessed source domain blood cell image content information probability density, and the preprocessed source domain blood cell image style information probability density, respectively. It is a window function composed of a Gaussian function with a variance of 0.04, that is, the closer the distance to x, the greater the counting weight. and They represent the content information feature map samples and style information feature map samples of the input source domain blood cell image, and They respectively represent the content information feature map samples and style information feature map samples of the source domain blood cell image after preprocessing;

[0108] S43. Use KL divergence to minimize content information C D The difference between them and the maximum style information S D+ The difference between them is used to obtain the content information loss function and the style information loss function The content information loss function and the style information loss function are constructed as a contrast loss function, where the loss formula of content information is:

[0109]

[0110] The loss formula of style information is:

[0111]

[0112] The contrast loss function is expressed as follows:

[0113]

[0114] S44. Input the content features of the source domain blood cell image after KL divergence processing into the deep neural network blood cell classification model, and define the final regression loss function as the cross entropy function, which is expressed as follows:

[0115]

[0116] Where M is the number of categories, y ic is a sign function, i.e. 1 or 0, when the true label of sample i is the same as the predicted label c, it is 1, otherwise it is 0, p ic is the predicted probability that sample i belongs to category c;

[0117] S45, the contrast loss function L Dual and the cross entropy function L Task Add and use as the predefined classification loss function, that is, L = L Dual +L Task , and then the predefined classification loss function is optimized using the stochastic gradient descent method to obtain a trained deep neural network blood cell classification model.

[0118] In this embodiment, the minibatch of the stochastic gradient descent method is 128, the momentum parameter is 0.9, and the learning rate is lr=[lr0 / (1+ap) b ] decay, where lr0 is the initial learning rate, set to 0.01, and p increases linearly from 0 to 1. At the same time, in this embodiment, a=10 and b=0.75.

[0119] S5. Use the trained deep neural network blood cell classification model to predict and classify blood cell test samples in the target domain.

[0120] In this embodiment, a contrastive learning method is used to construct a blood cell classification network model. At the same time, instance normalization and batch normalization are used to adjust the statistical data of the model according to the new domain without the need for additional parameters, thereby avoiding the adverse effects of the complexity of the network structure in other methods on model training and model generalization. Moreover, the cross-domain features of blood cell images are decoupled into style features and content features, and the KL divergence is used to match the cross-domain distribution, which effectively explains the working principle of the present invention in solving the domain shift problem, greatly improves the classification accuracy and generalization performance of blood cell images, and enhances the image detection performance of blood cells under actual complex conditions.

[0121] In order to further understand the working principle and technical effects of the present invention, the learning representation of the present invention is visualized using t-SEN to illustrate.

[0122] like Figure 4 As shown, taking the BCISC dataset source domain to the Ukiyo-e style target domain as an example, Figure 4 (a) is the data distribution diagram of the original data set. Figure 4 (b) is the data representation of the target domain by the VGG16 method. Figure 4 (c) is the data representation of the target domain of this method. It can be clearly seen from the figure that the network model with the twin network feature decoupling module added can align the two different fields well and clearly separate the different categories, which greatly improves the classification accuracy and generalization performance of blood cell images and enhances the image detection performance of blood cells under actual complex conditions.

[0123] The above describes in detail the domain-generalized and domain-adaptive blood cell classification method based on feature decoupling provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above examples is only intended to help understand the core concept of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from the principles of the present invention. Such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A domain-generalized and domain-adaptive blood cell classification method based on feature decoupling, characterized by: The method comprises the following steps: S1. Build a deep neural network blood cell classification model with VGG16 network as the backbone network; S2. Based on the twin network, the style information and content information of the source domain blood cell image are extracted through the Gram matrix and Squeeze-excitation operation, and then the twin network feature decoupling module is constructed; S25. The style information S of the input blood cell feature map is extracted. D+ ∈R N×C×H×W and content information C D ∈R N×C×H×W , and construct a twin network feature decoupling module based on the style information and content information of the extracted blood cell feature map, where N represents the number of feature maps of the same batch of source domain blood cell images, C represents the number of channels, H and W represent the number of spatial dimensions, and the style information and content information of the blood cell feature map are respectively expressed as follows: S D+ =F SE (S D ·l Dr ) C D =B+F SE (S D ·λ Di ) Where, F SE (·) represents the Squeeze operation and the Excitation operation, where the Squeeze operation F sq (·) represents the characteristic graph u of each blood cell nc ∈R C×H×W Perform global average pooling F sq (·), and then obtain a 1×1×C global information data z c , expressed as: Excitation operation F ex (·) indicates obtaining the weight s of all features in C channels, which can be expressed as: s=F ex (z c ,W)=σ(W2δ(W1z c )) Among them, W1 and W2 are fully connected layers, and the size of W1 is The size of W2 is r is the scaling factor, δ(·) is the ReLU function, and σ(·) is the sigmoid function; F SE (u c ) represents the obtained feature weight s and tensor u nc Multiplication, that is, F SE (u c )=s·u c ; S3. Preprocess the source domain blood cell image, and then input the source domain blood cell image and the preprocessed corresponding source domain blood cell image into the twin network feature decoupling module for training; S4. Batch normalize the two outputs of the twin network feature decoupling module after training, and use KL divergence to align the style information and content information of the two batch normalized outputs. Then, input the aligned source domain blood cell image into the deep neural network blood cell classification model for training. The predefined classification loss function is optimized based on the stochastic gradient descent method to obtain the trained deep neural network blood cell classification model. S5. Use the trained deep neural network blood cell classification model to predict and classify blood cell test samples in the target domain.

2. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 1, characterized in that: The deep neural network blood cell classification model includes a first module, a second module, a third module, a fourth module, a fifth module and a sixth module connected in sequence, wherein the first module includes a first convolutional layer, a feature decoupling submodule, a second convolutional layer and a first maximum pooling layer connected in sequence, the second module includes a third convolutional layer, a fourth convolutional layer and a second maximum pooling layer connected in sequence, the third module includes a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer and a third maximum pooling layer connected in sequence, the fourth module includes an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer and a fourth maximum pooling layer connected in sequence, the fifth module includes an eleventh convolutional layer, a twelfth convolutional layer, a thirteenth convolutional layer and a fifth maximum pooling layer connected in sequence, and the sixth module includes a Flatten layer, a first fully connected layer, a second fully connected layer, a third fully connected layer and a Softmax function regression prediction category connected in sequence.

3. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 2, characterized in that: The specific implementation of step S2 includes: S21, input the same batch of source domain blood cell image feature map I∈R N×C×H×W Perform style instance normalization and then extract the style normalized source domain blood cell image feature map B∈R N×C×H×W , the instance normalization operation IN(·) is expressed as: Where, I nchw represents the feature map of the nchw-th source domain blood cell image, c is the index of the number of channels, h and w are the spatial dimensions, μ nc is the mean of the feature maps of the same batch of source domain blood cell images, is the standard deviation of the feature maps of the same batch of source domain blood cell images, and ε represents a preset constant; S22, subtract the input source domain blood cell image feature map from the corresponding style normalized source domain blood cell image feature map to obtain the remaining source domain blood cell image style coupling feature S D , denoted as S D =IB=I-IN(I); S23, extracting domain-independent source domain blood cell image style representation weights based on Gram matrix; S24. Use the Gram matrix to decouple the style information of a specific field and calculate the style weight λ that is not related to the specific field. Di and the style weight λ associated with that particular domain Dr , where the style weight λ is independent of the specific domain Di It can be expressed as: The style weight λ associated with this particular domain Dr It can be expressed as: l Dr =softmax(1-λ Di ) Where G represents the Gram matrix, i and j represent the rows and columns of the Gram matrix respectively, and Softmax represents the normalized exponential function.

4. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 3, characterized in that: The specific implementation method of step S23 is as follows: the input source domain blood cell feature map is vectorized by tiling, and the vectorized feature map source domain blood cells are organized into a Hilbert space based on the Gram matrix, and then the inner product in the Gram matrix is converted into cosine similarity, thereby reducing the norm difference of each channel, and finally using the predefined feature space to capture the style information of each row in the Gram matrix, thereby extracting the domain-independent source domain blood cell image style representation weight.

5. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 4, characterized in that: The style information of each row in the Gram matrix is captured using the predefined feature space and expressed as follows: Where i and j represent the rows and columns of the Gram matrix respectively.

6. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 5, characterized in that: The preprocessing of the source domain blood cell image in step S3 specifically includes: first randomly cropping the source domain blood cell image to obtain a blood cell cropped image with a size of 120×120; then horizontally flipping the blood cell cropped image with a probability of 0.5; and finally randomly changing the brightness, contrast, saturation, and hue of the blood cell cropped image.

7. The domain generalization and domain adaptation blood cell classification method based on feature decoupling according to claim 6, characterized in that: The specific implementation of step S4 includes: S41, batch normalize the content information feature map and style information feature map output by the twin network feature decoupling module, and then obtain the corresponding global and local context information based on adaptive average pooling and generate N-dimensional samples x1,…,x N ; S42, based on Parzen window and Probability density estimation is performed, which is expressed as follows: Where, and They represent the input source domain blood cell image content information probability density, the input source domain blood cell image style information probability density, the preprocessed source domain blood cell image content information probability density, and the preprocessed source domain blood cell image style information probability density, respectively. It is a window function composed of a Gaussian function with a variance of 0.04, that is, the closer the distance to x, the greater the counting weight. and They represent the content information feature map samples and style information feature map samples of the input source domain blood cell image, and They respectively represent the content information feature map samples and style information feature map samples of the source domain blood cell image after preprocessing; S43. Use KL divergence to minimize content information C D The difference between them and the maximum style information S D+ The difference between them is used to obtain the content information loss function and the style information loss function The content information loss function and the style information loss function are constructed as a contrast loss function, where the loss formula of content information is: The loss formula of style information is: The contrast loss function is expressed as follows: S44. Input the content features of the source domain blood cell image after KL divergence processing into the deep neural network blood cell classification model, and define the final regression loss function as the cross entropy function, which is expressed as follows: Where M is the number of categories, y ic is a sign function, i.e. 1 or 0, when the true label of sample i is the same as the predicted label c, it is 1, otherwise it is 0, p ic is the predicted probability that sample i belongs to category c; S45, the contrast loss function L Dual and the cross entropy function L Task Add and use as the predefined classification loss function, that is, L = L Dual +L Task , and then the predefined classification loss function is optimized using the stochastic gradient descent method to obtain a trained deep neural network blood cell classification model.

Images