Blood cell detection method based on enhanced Faster R-CNN

Abstract

This invention discloses a blood cell detection method based on enhanced Faster R-CNN, belonging to the fields of medical image analysis and computer vision. The method includes: enhancing global feature modeling capabilities by constructing a hierarchical window attention mechanism, designing a channel aggregation module to strengthen the expression of mid-level semantic information, constructing a high-resolution multi-scale attention feature pyramid network, optimizing cross-scale feature fusion by combining skip connections and channel-space collaborative attention mechanisms, and finally using a normalized Wasserstein distance loss function to improve the model's fault tolerance for locating small target blood cells. This invention effectively solves the technical challenges of drastic scale changes, severe cell overlap, and a high proportion of small targets in blood cell images while ensuring detection accuracy. It is suitable for the development of automated counting, classification, disease screening, and intelligent medical diagnostic systems for clinical blood cells.

Description

Technical Field

[0001] This invention relates to blood cell detection methods, specifically to a blood cell detection method based on enhanced Faster R-CNN, belonging to the field of medical image analysis and computer vision technology. Background Technology

[0002] Clinical testing of blood cells plays a crucial role in medical diagnostics, with their counting and classification results serving as key indicators for screening and diagnosing various diseases such as anemia, infection, and leukemia. Traditional testing methods primarily rely on manual microscopic observation, which suffers from drawbacks such as high subjectivity, low efficiency, and visual fatigue. Furthermore, the accuracy and consistency of the results are highly dependent on the operator's professional experience. While automated methods based on traditional image processing can improve testing efficiency to some extent, their robustness is insufficient when dealing with complex scenarios such as cell overlap, high density, staining differences, and morphological variations, making it difficult to meet the stringent requirements of high-precision clinical diagnosis.

[0003] In recent years, deep learning-based target detection technology has provided a new solution for automated blood cell detection. Convolutional Neural Networks (CNNs), through end-to-end learning, can automatically extract hierarchical features from cell images and complete target localization and classification, significantly improving detection efficiency. This type of method demonstrates the powerful capabilities of deep learning in feature learning and has gradually become a research hotspot in the field of medical image analysis. However, blood cell images have significant inherent characteristics: a wide range of cell scale variations (from tiny platelets to larger white blood cells), severe occlusion and overlap between cells, and a high proportion of small targets (such as platelets and microlymphocytes). These issues mean that existing general-purpose detection networks still suffer from insufficient accuracy and limited generalization ability in handling global contextual dependencies, refined multi-scale feature alignment, and precise localization of small targets.

[0004] As a novel visual backbone network, the Swin Transformer demonstrates superior potential compared to traditional CNNs in efficiently modeling global dependencies through the introduction of a shifting window mechanism and hierarchical design. Its local window attention mechanism effectively reduces computational complexity while maintaining cross-window information interaction capabilities, providing superior feature representation for tasks like blood cells that require global contextual reasoning. However, the Swin Transformer is relatively weak in channel-dimensional feature recalibration, potentially leading to insufficient extraction of discriminative intermediate-order semantic information from blood cells. Furthermore, its computational complexity increases secondarily with image resolution, a problem that needs careful consideration in high-resolution microscopic image processing of blood cells.

[0005] In feature fusion, the Feature Pyramid Network (FPN) has become the standard architecture for solving multi-scale object detection. It effectively improves multi-scale object detection performance by fusing deep semantic information with shallow positional information through a top-down path. However, traditional feature fusion methods often employ simple addition or concatenation operations, failing to adequately model the nonlinear relationships and spatial alignment issues between cross-scale features, and lacking adaptive selection and enhancement mechanisms for key features. When dealing with complex situations such as severe overlap and morphological variations in blood cells, it is difficult to ensure that the feature map focuses on key distinguishing regions such as the cell nucleus and cytoplasm in both channel and spatial dimensions, leading to both feature redundancy and information loss.

[0006] In summary, although existing algorithms have made some progress in blood cell detection tasks, there is still room for improvement in key technical aspects such as global dependency modeling efficiency, multi-scale feature adaptive fusion capability, and robustness in small target localization. Therefore, there is an urgent need for a blood cell detection method that can effectively integrate the global modeling advantages of Swin Transformer with a fine multi-scale feature fusion mechanism to significantly improve detection accuracy and reliability in complex scenarios. Summary of the Invention

[0007] To address the shortcomings of the aforementioned technologies, this invention proposes a blood cell detection method based on enhanced Faster R-CNN. This method integrates the advantages of an improved Swing Transformer backbone network and a high-resolution multi-scale attention feature pyramid network, effectively addressing the challenges of blood cell detection in complex scenarios. This invention enhances the model's ability to model global and mid-level semantic features of blood cells through a hierarchical window attention mechanism and a channel aggregation module. Then, by constructing a channel-space collaborative attention mechanism, it optimizes the cross-scale feature fusion process, strengthens the localization of key regions such as the cell nucleus, and finally employs a regression loss function based on normalized Wasserstein distance to significantly improve the model's fault tolerance for locating small-sized blood cells. This approach effectively improves robustness and generalization ability in complex scenarios while maintaining detection accuracy, thereby meeting the requirements of high accuracy and high reliability in clinical testing.

[0008] To achieve the above objectives, this invention provides a blood cell detection method based on enhanced Faster R-CNN, comprising the following steps:

[0009] Step 1: Preprocess the input blood cell image to obtain a preprocessed image;

[0010] Step 2: Input the preprocessed image into the feature extraction network, and process it through multi-level feature extraction layers and channel aggregation module to obtain enhanced feature map group;

[0011] Step 3: Input the enhanced feature map group into the neck network for processing to obtain multi-scale enhanced feature maps;

[0012] Step 4: Use a region proposal network to process the multi-scale enhanced feature map to generate candidate regions; extract regional features from the candidate regions, and classify and regress the extracted regional features to obtain the detection results.

[0013] Further, step 2 includes:

[0014] The preprocessed image is input into the first feature extraction layer for processing to obtain the first-level feature map;

[0015] The first-level enhanced feature map is input into the second feature extraction layer for processing to obtain the second-level feature map;

[0016] The second-level enhanced feature map is input into the third feature extraction layer for processing to obtain the third-level feature map;

[0017] The third-level enhanced feature map is input into the fourth feature extraction layer for processing to obtain the fourth-level feature map.

[0018] The first-level feature map, the second-level feature map, the third-level feature map, and the fourth-level feature map are respectively input into the channel aggregation module for processing to obtain the corresponding first-level enhanced feature map, the second-level enhanced feature map, the third-level enhanced feature map, and the fourth-level enhanced feature map, which together form an enhanced feature map group.

[0019] Furthermore, the first feature extraction layer includes a block embedding module and a first-level Swing Transformer layer; the second feature extraction layer includes an image block merging downsampling layer and a second-level Swing Transformer layer; the third feature extraction layer includes an image block merging downsampling layer and a third-level Swing Transformer layer; and the fourth feature extraction layer includes an image block merging downsampling layer and a fourth-level Swing Transformer layer.

[0020] Furthermore, the Swin Transformer layer includes: layer normalization, a 7×7 window-based multi-head self-attention mechanism, a shift-window multi-head self-attention mechanism, and a feedforward network.

[0021] Furthermore, the process of the channel aggregation module processing features includes:

[0022] First, the input features are normalized, and then the channel-expanded features are obtained through 1×1 convolution.

[0023] Then, 3×3 depthwise separable convolution and GELU activation function are applied sequentially to the channel-expanded features to obtain the non-linearly processed features;

[0024] Next, channel aggregation is performed on the nonlinearly processed features to obtain the channel-aggregated features. The channel aggregation process includes: channel compression of the nonlinearly processed features through 1×1 convolution, followed by processing through the GELU activation function and subtractor to obtain residual terms; element-wise multiplication of the residual terms using learnable scaling parameters, and element-wise addition of the multiplication result to the nonlinearly processed features to output the channel-aggregated features.

[0025] Finally, the channel-aggregated features are processed by 1×1 convolution and then added element-wise to the input features to obtain the output features.

[0026] Further, step 3 includes:

[0027] The first-level enhanced feature map, the second-level enhanced feature map, the third-level enhanced feature map, and the fourth-level enhanced feature map in the enhanced feature map group are concatenated to obtain the fused feature map;

[0028] Applying a 1×1 convolution to the fused feature map to perform channel dimensionality reduction yields a preliminary fused feature map.

[0029] The initial fused feature map is input into the efficient multi-scale attention mechanism module for feature enhancement, resulting in an attention-enhanced feature map.

[0030] Attention-enhanced feature maps are processed by average pooling operations of different lengths to obtain multi-scale enhanced feature maps.

[0031] Furthermore, the present invention also includes: the total loss function used during training includes a classification loss based on the cross-entropy function and a normalized Wasserstein distance loss function.

[0032] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0033] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] (1) To address the problem that existing deep learning detection models are not capable of global context modeling in complex blood cell image scenarios, this invention improves the structure of the Swin Transformer backbone network, constructs a hierarchical window attention mechanism, enhances the model’s ability to capture long-distance spatial dependencies, and designs a channel aggregation module to strengthen the extraction and expression of mid-level semantic features, effectively alleviates the feature channel redundancy problem, and improves the discrimination accuracy of blood cell morphological features.

[0036] (2) In view of the problems of spatial alignment difficulties, insufficient modeling of nonlinear relationships and lack of attention in key regions in the cross-scale feature fusion process of traditional feature pyramid networks, this invention constructs a high-resolution multi-scale attention feature pyramid network and adopts a skip connection and channel-space collaborative attention mechanism to achieve refined adaptive fusion of multi-scale features, significantly enhance the localization and recognition of key regions such as cell nucleus and cytoplasm, and effectively deal with the feature confusion problem caused by cell overlap and occlusion.

[0037] (3) In response to the challenges of high proportion of small targets (such as platelets and microlymphocytes) in blood cell images, strict requirements for positioning accuracy but poor fault tolerance, this invention introduces a regression loss function based on normalized Wasserstein distance. By modeling with Gaussian distribution, the matching process between the predicted box and the real box is optimized, which improves the robustness and fault tolerance threshold of small target positioning and significantly reduces the false detection and false detection rates caused by slight position deviations. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the network structure of the blood cell detection model of the present invention;

[0039] Figure 2 This is a schematic diagram of the Swin Transformer layer structure of the present invention;

[0040] Figure 3 This is a schematic diagram of the channel aggregation module structure of the present invention;

[0041] Figure 4 This is a schematic diagram of the neck network structure of the present invention;

[0042] Figure 5 This is a schematic diagram of the efficient multi-scale attention mechanism module structure of the present invention;

[0043] Figure 6 This is a schematic diagram of the model reasoning results of the present invention;

[0044] Figure 7 The following is a visualization of the results of the present invention: (a) is the result of the first case, (b) is the result of the second case, (c) is the result of the third case, and (d) is the result of the fourth case. Detailed Implementation

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0046] Example 1:

[0047] This invention provides a blood cell detection method based on enhanced Faster R-CNN, the overall process of which is as follows: Figure 1 As shown, this method first preprocesses the input blood cell microscopic image, including size normalization and pixel value standardization, to adapt to the network input and improve training stability. Then, the preprocessed image is fed into the feature extraction backbone network, where deep semantic feature maps are extracted through multi-layer convolutional operations. Further, multi-scale information is fused, effectively aggregating features at different depths within the backbone network to enhance the model's ability to represent blood cells of different sizes. Based on this, the Region Proposal Network (RPN) slides across the fused feature map, efficiently generating a series of candidate regions that may contain the target. Next, region features are extracted from each candidate region through a region of interest alignment operation, accurately converting it into a fixed-size feature vector. This feature vector is then input to the classification and regression branches, predicting in parallel the specific blood cell category to which the region belongs and the precise bounding box coordinate offset. Finally, the model applies non-maximum suppression post-processing to remove overlapping redundant detection boxes, outputting the final blood cell category and localization results. This method, by systematically integrating the above modules, achieves high-precision and high-efficiency automatic detection and analysis of blood cell images. The following technical solution is adopted:

[0048] Step 1: Obtain blood cell microscopic images and perform image preprocessing.

[0049] This invention employs multi-stage data augmentation and standardized image processing to preprocess the input blood cell microscopic images. The specific implementation steps are as follows:

[0050] Furthermore, the image preprocessing steps include: decoding the input blood cell image into a pixel matrix, obtaining three-channel RGB image data, and then performing data augmentation operations on the image, including a serialization process of random scaling and cropping, multi-scale random adjustment, random horizontal flipping, image normalization, and dimensional rearrangement.

[0051] In this embodiment, the blood cell microscopic image is resized using a random scaling and cropping strategy. A preset cropping size of 400×400, 500×500, and 600×600 pixels is randomly selected, with 384×600 pixels as the final cropping size to ensure the integrity of various cell targets such as red blood cells, white blood cells, and platelets. Subsequently, multi-scale random adjustment is implemented. While maintaining the original aspect ratio, the target size is randomly selected from 11 preset scale combinations (480×1333 to 800×1333 pixels) to enhance the model's robustness to scale changes. Random horizontal flipping is performed with a 50% probability to expand data diversity. Pixel values ​​are standardized using the preset mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] of the ImageNet dataset to eliminate the influence of staining differences from different microscopic devices. A dimensionality rearrangement operation is performed to convert the data format from height-width-channel (HWC) to channel-height-width (CHW) format to meet the requirements of deep learning frameworks. Finally, a batch-level padding mechanism is used to pad the image size to an integer multiple of 32, ensuring the feature map size requirements of each layer of the convolutional neural network and forming a standardized input data stream. This data stream provides multi-scale enhanced and standardized input data for the two-stage object detection framework, enabling efficient object recognition through a cascaded architecture of region proposal networks and detection networks.

[0052] In this embodiment, after acquiring a blood cell microscopic image with dimensions of 1536×2048×3, it is first scaled to 384×600×3 using bilinear interpolation. Then, a 3×3 median filter is applied to eliminate discrete noise points, and horizontal mirroring is performed with a 50% probability to augment the data. After standardization, the image pixel value distribution is normalized to the standard normal range, and finally, it is padded to 384×608×3 by a factor of 32, providing standardized input for subsequent feature extraction.

[0053] Step 2: Input the preprocessed image into the feature extraction network for multi-scale feature extraction.

[0054] The preprocessed image is sequentially input into four feature extraction layers for deep feature extraction, resulting in the first feature map, the second feature map, the third feature map, and the fourth feature map.

[0055] Furthermore, the feature extraction network uses an improved Swin Transformer as its backbone network and implements a four-stage hierarchical feature extraction process, the specific steps of which include:

[0056] The preprocessed image is input into the first feature extraction layer for further processing. The input three-channel RGB image is first processed by a block embedding module, which uses a non-overlapping convolution operation with a kernel size of 4×4 and a stride of 4 to divide the original image into blocks. Each image patch is linearly projected onto [a number of image blocks]. 3D feature space, generating initial feature map X0 is fed into the first-level Swing Transformer layer, which contains two consecutive Swing Transformer blocks, with the structure as follows: Figure 2 As shown. Each Swin Transformer block consists of layer normalization, a 7×7 window-based multi-head self-attention (W-MSA) mechanism, a shifted window multi-head self-attention (SW-MSA) mechanism, and a feedforward network, which alternately performs intra-window attention calculation and cross-window information interaction.

[0057] The final output space size remains unchanged. Channel dimension is First-level enhanced feature map .

[0058] The first-level enhanced feature map X1 is input into the second feature extraction layer for processing. After being processed by an image patch merging and downsampling layer, X1 undergoes 2×2 neighborhood block concatenation and linear transformation, and is then fed into the second-level SwinTransformer layer (containing two Transformer blocks) to output the second-level enhanced feature map. .

[0059] Furthermore, the image patch merging downsampling layer achieves spatial downsampling and channel expansion through concat stitching and linear transformation, as shown in the following formula: ,

[0060] This operation halves the feature map space size. Meanwhile, the number of channels has been expanded to .

[0061] The second feature map is then fed into the third feature extraction layer for processing. The second-level enhanced feature map X2 is further downsampled through image patch merging, halving its spatial size. The number of channels has been expanded to The processed feature map then enters the third-level Swing Transformer layer, which contains six consecutive Swing Transformer blocks. Through multiple rounds of alternating calculations of window attention and shift window attention, the third-level enhanced feature map is output. .

[0062] The third feature map is then fed into the fourth feature extraction layer for processing. The third-level enhanced feature map X3 undergoes a final image patch merging and downsampling process, resulting in its spatial size being halved. The number of channels has been expanded to The system then proceeds to the fourth-level Swing Transformer layer, which contains two consecutive Swing Transformer blocks for the highest-level semantic feature extraction and refinement, outputting the fourth-level enhanced feature map. .

[0063] The four enhanced feature maps are input into the channel aggregation module for further feature enhancement. This is the enhanced feature map output from the four layers of the SwingTransformer. The channel aggregation module is applied again for feature enhancement to address the issues of weak channel feature selection and insufficient extraction of intermediate-order blood cell semantic features in the native Swin Transformer. The module structure is as follows: Figure 3 As shown: The input features are first normalized by a normalization Norm layer, and then the number of channels is expanded to four times the original number using a 1×1 convolution. Spatial context information is then aggregated using a 3×3 depthwise separable convolution, and nonlinearity is introduced by applying the GELU activation function. Channel aggregation is then performed, and a normalization layer is applied again to numerically correct the spatially enhanced features. A 1×1 convolution is used to compress the features to a single channel, and GELU activation is applied to generate adaptive scaling weights representing the importance of each channel. These weights are then element-wise weighted with the original input features to complete adaptive aggregation of the channel dimension. Finally, a 1×1 convolution restores the original number of channels, achieving synergistic enhancement of channel and spatial information. The final enhanced feature map set is output. Their spatial resolutions are respectively , , , The corresponding channel dimensions are C1, C2, C3, and C4.

[0064] The specific implementation process of the channel aggregation module is as follows:

[0065] First, the input features are processed by 1×1 convolution. Processing is performed to obtain the features after channel expansion. The calculation formula is as follows:

[0066] ;

[0067] Here, Conv2D represents a two-dimensional convolutional layer, used to implement feature channel mapping and spatial information extraction; kernel_size is the size of the convolutional kernel window, which determines the spatial neighborhood range covered by the convolution operation. kernel_size=1 corresponds to a 1×1 convolution; out_channels represents the number of channels of the convolutional output features, used to complete channel upscaling, compression, and dimension restoration operations. This represents a fourfold increase in dimensionality for the channel.

[0068] Then, the features after channel expansion By sequentially applying a 3×3 depthwise separable convolution (DWConv2D) and the GELU activation function, the non-linearly processed features are obtained. The calculation formula is as follows:

[0069] ;

[0070] The GELU function is defined as follows: ;

[0071] Where kernel_size=3 corresponds to a 3×3 convolution, and padding=1 indicates that in the feature... Figure 4 Filling the edge with zero pixels and using a 3×3 convolution kernel can ensure that the spatial resolution of the feature map remains unchanged before and after convolution, thus fully preserving the edge contour features of blood cells. For grouped convolution parameters, the total number of feature channels after channel upscaling in this module is: By setting the number of groups equal to the number of input channels, depthwise convolution operations are implemented, and each channel independently extracts the local spatial context, effectively reducing the computational overhead of convolution.

[0072] Next, channel aggregation is performed on the nonlinearly processed features to obtain the channel-aggregated features. The channel aggregation process includes: channel compression of the nonlinearly processed features using a 1×1 convolution, followed by sequential processing with the GELU activation function and a subtractor to obtain residual terms; element-wise multiplication of the residual terms using a learnable scaling parameter; and element-wise addition of the multiplication result to the nonlinearly processed features to output the channel-aggregated features. The channel aggregation operation consists of the following sub-steps:

[0073] The non-linearly processed features are converted using 1×1 convolution. The number of channels is compressed to 1, resulting in the following fused feature map:

[0074] ,

[0075] Next, the adaptive global weight map is obtained by performing GELU nonlinear activation on the single-channel fused feature Z. ,

[0076] The residual term is obtained by subtracting the original spatial augmentation feature Y3 from the adaptive global weight map A element by element using a subtractor. ,

[0077] The scaled residual feature components are obtained by element-wise multiplication of the residual term R with the learnable scaling parameter σ. ,

[0078] Where ⊙ represents element-wise multiplication, and σ is a learnable scaling parameter.

[0079] Channel aggregation final output: ;

[0080] Finally, the channel-aggregated features are processed by a 1×1 convolution and then added element-wise to the input features to obtain the output features, specifically... .

[0081] This invention adds a channel aggregation module after improving the output of each stage of the Swin Transformer backbone network. On the one hand, it compensates for the lack of adaptive modeling of channel dimensions in the original Swin Transformer, purifying multi-scale blood cell discrimination features layer by layer, suppressing channel noise caused by microscopic background and cell overlap, and stabilizing the gradient transfer of the backbone network. On the other hand, it provides high-purity, balanced, multi-level enhanced feature input to the backend high-resolution multi-scale attention feature pyramid network, reducing the semantic gap between deep and shallow features, reducing information confusion during cross-scale fusion, and improving the activation ability of multi-scale collaborative attention on key regions such as cell nuclei and tiny platelets. The Swin Transformer is responsible for capturing the global contextual dependencies of the image, the channel aggregation module completes single-layer feature channel enhancement, and the high-resolution multi-scale attention feature pyramid network realizes multi-scale information interaction. The complementary functions work together to solve the detection difficulties of drastic changes in blood cell scale, cell overlap and occlusion, and sparse features of small targets, amplifying the benefits of feature enhancement layer by layer and improving the overall localization and classification accuracy of the model.

[0082] Step 3: Input the first to fourth feature maps into the neck network for processing to obtain multi-scale enhanced feature maps.

[0083] Furthermore, the neck network processing part constructs a High-Resolution Multi-scale Attention Feature Pyramid Network (HRMA-FPN), with the structure as follows: Figure 4 As shown, the specific steps include:

[0084] The first feature map Second feature map Third feature map Fourth feature map Input to the multi-scale feature fusion module. , , Bilinear interpolation was used to upsample by 2, 4, and 8 times respectively, so that the spatial size was uniformly H / 4×W / 4 and aligned with F1.

[0085] Furthermore, the bilinear interpolation upsampling operation is expressed by the following formula:

[0086] ,

[0087] ,

[0088] ,

[0089] Interpolate is the feature interpolation upsampling function, used to enlarge deep low-resolution features to a uniform spatial size; scale_factor is the spatial scaling factor, set to 2, 4, and 8 respectively to perform upsampling at different magnifications for F2, F3, and F4; This indicates that bilinear interpolation is used to complete pixel reconstruction, smoothly preserving the outline and texture of blood cells and avoiding jagged distortion; , , The images are, in order, equal-resolution feature maps after upsampling F2, F3, and F4.

[0090] Concat the upsampled feature maps along the channel dimension: ,in This is a fused feature map with rich multi-scale information.

[0091] right Channel dimensionality reduction using 1×1 convolution: ,

[0092] Preliminary fused feature map with 256 output channels .

[0093] The initially fused feature map is input into the Efficient Multi-Scale Attention (EMA) module for feature enhancement. The module structure is as follows: Figure 5 As shown, the specific steps are as follows:

[0094] The input feature map is reshaped, splitting the channels into 8 parallel groups, resulting in a grouped feature tensor. Specifically: The feature map shape is from [B, C]out [H / 4, W / 4] becomes [B×8, C out / 8, H / 4, W / 4).

[0095] Global adaptive average pooling (AdaptiveAvgPool2D) operations are performed in parallel along the height and width directions on the grouped feature maps to obtain height-pooled features. The feature map shape is [B×8, C out [ / 8, H / 4, 1] and width pooling features The feature map shape is [B×8, C out [ / 8, 1, W / 4]:

[0096] ,

[0097] ,

[0098] Here, `output_size=(None, 1)` is the output size configuration parameter for adaptive 2D average pooling. The first parameter in the tuple corresponds to the feature height dimension, and the second parameter corresponds to the width dimension. A value of `None` indicates that this dimension remains consistent with the input feature and is not compressed, while a value of `1` indicates that this dimension is globally compressed to a unit length. In the height-direction pooling operation, `output_size=(None, 1)` is used, meaning that the input feature height remains unchanged, and the width dimension is globally pooled and compressed to 1, resulting in the vertical axis global feature. .

[0099] 1×1 convolutions are applied to the height and width pooling features respectively, and a sigmoid activation function is used to generate height attention weights. and width attention weight :

[0100] ,

[0101] ,

[0102] Both are broadcast along the channel dimension to expand to the number of channels C. out / 8, to obtain the expanded weights after channel alignment. and The two are multiplied element-wise with the grouping features through a broadcast mechanism to obtain spatial attention-weighted features. Its shape is [B×8, C] out [ / 8, H / 4, W / 4]: ,

[0103] Simultaneously, 3×3 convolution is performed on the grouped features to generate local features: ,

[0104] The output shape is [B×8, C out Local feature map of [H / 4, W / 4].

[0105] Spatial attention-weighted feature maps Perform a global adaptive average pooling operation to obtain a shape of [B×8, C out Feature map of / 8,1,1] Perform a dimension reshaping operation to obtain the feature map. Its shape is [B×8, C out [ / 8, 1], call the Transpose function to... Perform a dimension transpose, converting the dimensions to [B×8, 1, C]. out [8] Finally, the spatial branch attention weights are obtained through Softmax normalization. :

[0106] ,

[0107] ,

[0108] ,

[0109] Output of local feature branches Applying global adaptive average pooling, we obtain a shape of [B×8, C] out Feature map of / 8,1,1] Similarly, performing dimension reshaping operations yields the feature map as follows: Its shape is [B×8, C] out [8, 1] and transpose it to [B×8, 1, C]. out [8] Finally, the local branch attention weights are obtained through Softmax normalization. :

[0110] ,

[0111] ,

[0112] ,

[0113] Attention weight Compared with the local feature map after dimensional reshaping Performing a matrix multiplication MatMul operation, the local feature map is transformed to dimension [B×8, C] by a reshaping operation. out / 8, (H / 4)×(W / 4)], after calculation, the first attention output is obtained; similarly, the attention weights are... Compared with the reshaped spatial attention-weighted feature map After matrix multiplication, the feature map is also reshaped into [B×8, C out [8, (H / 4)×(W / 4)], to obtain the second attention output; add the two attention outputs element by element to generate a one-dimensional comprehensive attention weight. (Shape [B×8, 1, (H / 4)×(W / 4)]), then the two-dimensional spatial resolution is restored through the Reshape operation to obtain the final comprehensive attention weights. The shape is [B×8, 1, H / 4, W / 4]. The calculation expression corresponding to the above process is:

[0114] ,

[0115] ,

[0116] ,

[0117] ,

[0118] The sigmoid activation function is used to activate the combined attention weights, which are then multiplied with the original grouped feature maps:

[0119] The resulting shape is [B×8, C out The weighted feature map is [B, C]. Finally, it is reconstructed back to [B, C]. out The weighted feature map is added to the original input feature map [H / 4, W / 4], and the final enhanced feature map is output, maintaining the shape of [B, C]. out [H / 4, W / 4]: .

[0120] Based on attention-enhanced feature maps, a multi-scale feature pyramid is reconstructed using average pooling operations of different phase lengths, while maintaining the original resolution output. ; Apply average pooling with a step size of 2 to obtain ; Apply average pooling with a step size of 4 to obtain ; Applying average pooling with a step size of 8 yields 3×3 convolutions are applied to P2 through P5 respectively for feature refinement, keeping the spatial size and channel dimension unchanged, and finally outputting a set of features with a uniform channel dimension C. out The space dimensions are respectively , , , The enhanced multi-scale feature map provides high-quality feature representations for subsequent object detection tasks.

[0121] Step 4: Generate candidate regions based on the multi-scale enhanced feature map.

[0122] A region proposal network is implemented on the P2 to P5 feature pyramid levels output by the neck network, with 15 anchors predefined at each spatial position. The basic sizes of the anchors are 32, 64, 128, 256, and 512 pixels respectively, and each size is matched with three aspect ratios of 0.5, 1.0, and 2.0. The anchor size calculation adopts the geometric mean principle: , , where s is the reference size, and r is the aspect ratio.

[0123] Each anchor takes the feature map position as the center to generate bounding box coordinates [x center -w / 2, y center -h / 2, x center +w / 2, y center +h / 2]. Where x center is the horizontal pixel coordinate of the corresponding anchor center on the feature map, and y center is the vertical pixel coordinate of the corresponding anchor center on the feature map.

[0124] After extracting shared features through a 3×3 convolution, the RPN is subsequently deployed with two dedicated branches in parallel: the classification branch outputs the foreground-background binary classification score for each position through a 1×1 convolution, and uses the sigmoid function to convert the raw score into a probability value:

[0125] , the regression branch outputs bounding box refinement parameters through a 1×1 convolution, and each anchor corresponds to 4 regression values [Δx, Δy, Δw, Δh], which are used to perform spatial transformation on the original anchor.

[0126] In the training phase, a dynamic sample assignment strategy based on the Intersection over Union (IoU) threshold is adopted. An anchor with IoU≥0.7 with any ground truth box is marked as a positive sample, an anchor with IoU≤0.3 is marked as a negative sample, and anchors with 0.3<IoU<0.7 do not participate in training. 256 anchors are sampled for each image, and the ratio of positive and negative samples is strictly controlled at 1:1.

[0127] The present invention adopts the standard Faster R-CNN bounding box regression encoding method, and calculates the four-dimensional offset regression target t of the ground truth target box based on the anchor x , t y , t w , t h . Specifically, t x is the horizontal normalized offset of the ground truth box center relative to the anchor center, t y is the vertical normalized offset of the ground truth box center relative to the anchor center, t w is the logarithmic scaling coefficient of the ground truth box width relative to the anchor box width, th The logarithmic scaling factor of the actual frame height relative to the anchor frame height.

[0128] , ,

[0129] , ,

[0130] in , , , For anchor frame parameters, The horizontal pixel position of the anchor frame center. The vertical pixel position of the anchor frame center. It is the anchor frame reference width. This indicates the anchor frame reference height, where x and y represent the pixel coordinates of the bounding box in the image, and w and h represent the width and height of the box, respectively.

[0131] During the inference phase, approximately 20,000 initial proposals are generated from multi-scale feature maps. After bounding box decoding, they are sorted by foreground score, and the top 6,000 proposals at each level are retained. Non-Maximum Suppression (NMS) is applied, and the IoU threshold is set to 0.5. After global sorting, 1,000 high-quality region proposals are retained as input for subsequent detection heads.

[0132] Step 5: Extract regional features from the candidate regions.

[0133] For the candidate regions output by the region proposal network, a region of interest alignment operation is used to extract fixed-size 7×7 region features from the feature pyramid {P2, P3, P4, P5}. An adaptive hierarchical allocation strategy based on region scale is implemented. , This serves as the baseline level.

[0134] Region of interest alignment is achieved at the sub-pixel level through bilinear interpolation, avoiding quantization errors and outputting a 7×7×256-dimensional region feature map.

[0135] The extracted region features are processed by a two-layer fully connected network: the first fully connected layer maps the 7×7×256 features into a 1024-dimensional vector, and the second fully connected layer further refines it into a 1024-dimensional feature vector. Each layer is followed by a ReLU activation function. The final output 1024-dimensional feature vector integrates rich semantic information and spatial context, providing strong feature support for subsequent classification and regression tasks.

[0136] Step 6: Classify and regress the extracted regional features to obtain the detection results.

[0137] The region feature vectors are processed through two independent fully connected branches.

[0138] The classification branch outputs a K+1 dimensional class score vector (K is the number of target classes, 1 is the background) through a fully connected layer, which is then converted into a probability distribution by a softmax function. This formula ensures that the sum of the probabilities of all classes is 1, where is the original score for the i-th category.

[0139] The classification loss uses the cross-entropy function: Where N is the number of samples, The true label is one-hot encoded.

[0140] The regression branch independently predicts four bounding box fine-tuning parameters for each category, achieving category-specific localization optimization.

[0141] Furthermore, the localization optimization employs the normalized Wasserstein distance loss function:

[0142] , For prediction boxes With real frame Wasserstein distance between them; Set the temperature parameter to 1.5; The weight parameter is set to 8.0.

[0143] The total loss function is: The loss is calculated only for valid samples (label ≥ 0), and the mean reduction strategy is used to stabilize the training process.

[0144] Step 7: Perform non-maximum suppression post-processing on the detection results and output the final detection box.

[0145] The raw detection results output by the network are post-processed with non-maximum suppression: first, low-quality detection hypotheses are filtered out with a confidence threshold of 0.05;

[0146] The NMS algorithm is executed independently for each target category to eliminate duplicate detections on the same object. The core steps of the algorithm include:

[0147] Sort the detection boxes in descending order of confidence level, select the box with the highest score and add it to the result set, calculate its IoU with the remaining boxes, remove overlapping boxes with IoU > 0.5, and repeat this process until all processing is complete.

[0148] In the final output stage, it is ensured that each real object corresponds to only one optimal predicted bounding box. Through the non-maximum suppression algorithm, multiple overlapping detection boxes on the same object are merged, and only the detection result with the highest confidence is retained, forming a one-to-one object-detection box mapping relationship.

[0149] The selected detection box coordinates are mapped back from the network's internal normalized space to the original image coordinate space. The transformation process considers the scaling during image preprocessing to ensure the output coordinates are aligned with the actual image pixels. Each final detection result contains complete information: bounding box coordinates, [x] coordinates in the original image space. min y min x max y max [; Category label, an integer from 0 to K-1; Confidence score, a floating-point number from 0 to 1. After a complete post-processing workflow, a unique corresponding prediction box is generated for each object detected in the image, completing the entire blood cell detection process.]

[0150] To verify the effectiveness of the model described in this invention, experimental verification was conducted. The experiment used an Intel(R) Core(TM) i7-13700K CPU with 32GB of RAM and an NVIDIA GeForce RTX 4060 GPU with 8GB of VRAM. The deep learning framework used was PyTorch 2.7.1, the programming language was Python 3.10, and it was equipped with the CUDA 12.6 acceleration library. The operating system was Ubuntu 22.04. The AdamW optimizer was used for training, with a total of 48 epochs. The initial learning rate was set to 0.0001, and the learning rate was decayed to one-tenth of its original value in epochs 36 and 42, respectively, to adapt to the parameter optimization needs of different training stages. After the model training was completed, the obtained optimal weight file was loaded, and forward inference was performed on an independent validation set. The model processes the input image using forward propagation and outputs a structured list of prediction results, each item corresponding to a detected target instance. Figure 6 This is the model inference result image. The result contains four core fields: image_id, which identifies the source image to which the prediction result belongs, facilitating result organization, visualization, and backtracking; category_id, which is the target category identifier predicted by the model; bbox, which precisely describes the target's location and size in the format [x, y, w, h], and this information comes directly from the regression head output; and score, which is a confidence score between 0 and 1, comprehensively representing the model's confidence in the existence of the target and its classification and localization accuracy, and is a key basis for subsequent score filtering and non-maximum suppression.

[0151] To visually demonstrate the performance of the model described in this invention on the validation set, this embodiment provides a visualization of the detection results. For example... Figure 7As shown, the input image is overlaid with all target instances predicted by the model. Each instance is clearly presented with three visual elements: a colored bounding box precisely delineates the target region, the box color corresponds to the target's category identifier, and the box indicates the corresponding category name and its confidence score. It can be observed that the model proposed in this invention can achieve accurate classification and fine localization of multiple types of blood cells, such as red blood cells and platelets. All detected targets match the correct category labels, and the prediction confidence of various cell types is stably maintained in the range of 0.82 to 1.00. The bounding boxes closely fit the actual cell contours, demonstrating excellent classification and localization accuracy. This invention configures channel aggregation modules at each level of the Swing Transformer backbone network output. Through normalization, channel dimensionality enhancement, depthwise separable convolution, and adaptive channel weight aggregation operations, it strengthens discriminative semantic features such as the cell nucleus and cytoplasm of blood cells. Simultaneously, a high-resolution multi-scale attention feature pyramid network completes cross-scale feature adaptive fusion, providing high-purity, high-discriminative features for subsequent classification and bounding box regression, significantly improving the model's recognition accuracy and prediction confidence.

[0152] The model exhibits strong robustness in complex scenarios such as densely overlapping blood cell images and tiny platelet targets, with no obvious missed or false detections. Figure 7 In images (a) and (b), numerous red blood cells overlap and obscure each other, yet all cells can be detected independently. Tiny platelet dots are also completely identified with confidence levels exceeding 0.95. This achievement is attributed to two innovative design features: the channel aggregation module adaptively suppresses redundant background features in overlapping regions, mitigating feature confusion caused by occlusion; simultaneously, the multi-scale attention structure of the neck network adapts to the scale span of blood cells, and the regression branch employs normalized Wasserstein distance loss to enhance the fault tolerance for small target localization, effectively addressing the shortcomings of existing models such as missed detection of small targets and failure to identify overlapping cells.

[0153] The model can accurately distinguish white blood cells with different karyotypes and morphological characteristics, and can capture microscopic pathological features such as cell nuclei and cytoplasmic granules, meeting the application requirements of automatic classification of clinical blood cells and disease screening. Figure 7 (c) Monocytes and segmented neutrophils can be clearly distinguished. Figure 7 (d) Accurate identification of eosinophils with characteristic granules. The backbone network of this invention adopts a hierarchical window shifting self-attention mechanism, which has the ability to model global long-distance dependencies and can combine the global context of the image to mine subtle morphological differences of cells; coupled with the channel aggregation module to screen key morphology-related channel features, the model can fully learn the pathological features specific to various blood cells, and the detection output results meet the clinical blood testing standards, which has clinical intelligent diagnostic application value.

[0154] Example 2:

[0155] This invention provides an electronic device, including a processor and a memory. The memory stores program instructions for implementing the blood cell detection method based on enhanced Faster R-CNN of Embodiment 1. The processor executes the program instructions stored in the memory to perform blood cell detection.

[0156] The processor can also be called a Central Processing Unit (CPU). A processor may be an integrated circuit chip with signal processing capabilities. A processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0157] The methods described in Embodiment 1 of this invention are applicable to this electronic device embodiment. The specific functions implemented by this electronic device embodiment are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0158] Example 3:

[0159] This embodiment provides a storage medium storing a computer program or instructions capable of implementing the above-described blood cell detection method based on Enhanced Faster R-CNN. When the computer program or instructions are run on a computer, the steps of the above-described blood cell detection method based on Enhanced Faster R-CNN are executed.

[0160] The computer program or instructions can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0161] The methods described in Embodiment 1 of this invention are all applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0162] This invention provides a blood cell detection method based on enhanced Faster R-CNN. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A blood cell detection method based on enhanced Faster R-CNN, characterized in that, Includes the following steps: Step 1: Obtain the preprocessed image; Step 2: Input the preprocessed image into the feature extraction network, and process it through multi-level feature extraction layers and channel aggregation module to obtain enhanced feature map group; Step 3: Input the enhanced feature map group into the neck network for processing to obtain multi-scale enhanced feature maps; Step 4: Use a region proposal network to process the multi-scale enhanced feature map and generate candidate regions; Regional features are extracted from candidate regions, and the extracted regional features are classified and regressed to obtain detection results.

2. The blood cell detection method based on enhanced Faster R-CNN according to claim 1, characterized in that, Step 2 includes: The preprocessed image is input into the first feature extraction layer for processing to obtain the first-level feature map; The first-level enhanced feature map is input into the second feature extraction layer for processing to obtain the second-level feature map; The second-level enhanced feature map is input into the third feature extraction layer for processing to obtain the third-level feature map; The third-level enhanced feature map is input into the fourth feature extraction layer for processing to obtain the fourth-level feature map. The first-level feature map, the second-level feature map, the third-level feature map, and the fourth-level feature map are respectively input into the channel aggregation module for processing to obtain the corresponding first-level enhanced feature map, the second-level enhanced feature map, the third-level enhanced feature map, and the fourth-level enhanced feature map, which together form an enhanced feature map group.

3. The blood cell detection method based on enhanced Faster R-CNN according to claim 2, characterized in that, The first feature extraction layer includes a block embedding module and a first-level Swing Transformer layer; the second feature extraction layer includes an image block merging downsampling layer and a second-level Swing Transformer layer; the third feature extraction layer includes an image block merging downsampling layer and a third-level Swing Transformer layer; and the fourth feature extraction layer includes an image block merging downsampling layer and a fourth-level Swing Transformer layer.

4. The blood cell detection method based on enhanced Faster R-CNN according to claim 3, characterized in that, The Swin Transformer layer includes: layer normalization, a multi-head self-attention mechanism based on a 7×7 window, a shifted window multi-head self-attention mechanism, and a feedforward network.

5. The blood cell detection method based on enhanced Faster R-CNN according to claim 2, characterized in that, The process of feature processing by the channel aggregation module includes: First, the input features are normalized, and then the channel-expanded features are obtained through 1×1 convolution. Then, 3×3 depthwise separable convolution and GELU activation function are applied sequentially to the channel-expanded features to obtain the non-linearly processed features; Next, channel aggregation is performed on the nonlinearly processed features to obtain the channel-aggregated features. The channel aggregation process includes: channel compression of the nonlinearly processed features through 1×1 convolution, followed by processing through the GELU activation function and subtractor to obtain residual terms; element-wise multiplication of the residual terms using learnable scaling parameters, and element-wise addition of the multiplication result to the nonlinearly processed features to output the channel-aggregated features. Finally, the channel-aggregated features are processed by 1×1 convolution and then added element-wise to the input features to obtain the output features.

6. The blood cell detection method based on enhanced Faster R-CNN according to claim 1, characterized in that, Step 3 includes: The first-level enhanced feature map, the second-level enhanced feature map, the third-level enhanced feature map, and the fourth-level enhanced feature map in the enhanced feature map group are concatenated to obtain the fused feature map; Applying a 1×1 convolution to the fused feature map to perform channel dimensionality reduction yields a preliminary fused feature map. The initial fused feature map is input into the efficient multi-scale attention mechanism module for feature enhancement, resulting in an attention-enhanced feature map. Attention-enhanced feature maps are processed by average pooling operations of different lengths to obtain multi-scale enhanced feature maps.

7. The blood cell detection method based on enhanced Faster R-CNN according to claim 1, characterized in that, Also includes: The total loss function used during training includes classification loss based on the cross-entropy function and normalized Wasserstein distance loss function.

8. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the blood cell detection method based on enhanced Faster R-CNN as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The device stores a computer program or instructions that, when run on a computer, execute the steps of the blood cell detection method based on enhanced Faster R-CNN as described in any one of claims 1 to 7.

Images