Cell detection method, model training method and device, and storage medium

Abstract

A cell detection method, a model training method and device, and a storage medium, the cell detection method comprising: acquiring a cell image; inputting the cell image into a pre-trained cell detection model to obtain a cell type and a position in the cell image, wherein the cell detection model comprises a backbone network, an additional feature extraction module, a feature fusion module, and a detection module, the backbone network performs feature extraction on an input image to obtain a first extracted feature map and one or more first intermediate feature maps; the additional feature extraction module performs multi-scale feature extraction on the first extracted feature map to obtain a plurality of additional feature maps; the feature fusion module performs feature fusion on the first intermediate feature maps and the additional feature maps, first from high to low and then from low to high; and the detection module performs target classification and position regression on the fused feature map to obtain the cell type and the position.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of cell detection technology, and particularly to a cell detection method, a model training method and apparatus, and a storage medium. Background Technology

[0002] White blood cell detection and differential counting are core components of clinical hematology analysis, playing a crucial role in the diagnosis of infections, inflammation, and hematological disorders. Clinically, methods primarily include manual microscopy and hematology analyzers. Manual microscopy is the gold standard for white blood cell detection and differential counting, but it suffers from low efficiency and high subjectivity. Hematology analyzers offer simple operation and good repeatability, but their development costs are high, and they cannot observe cell morphology.

[0003] With the development of computer vision and deep learning technologies, image-based automated white blood cell analysis has become a research hotspot. These methods offer advantages such as simplicity, efficiency, and high reproducibility. However, in clinical applications, acquiring high-quality cell images is costly, while images acquired with low-cost imaging devices are often of poor quality, exhibiting issues such as loss of fine intracellular structures, blurred edges, and poor contrast, posing a significant challenge to the model's detection performance. Current detection models show poor robustness to low-quality images captured by low-cost devices and suffer from insufficient multi-scale target detection performance. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] This disclosure provides a cell detection method, including: Acquire cell images; The cell image is input into a pre-trained cell detection model to obtain the cell types and locations in the cell image. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature maps from high to low levels to obtain a first fused feature map, and then perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature maps from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain the cell type and cell location.

[0006] This disclosure also provides a cell detection apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of the cell detection method according to any embodiment of this disclosure based on the instructions stored in the memory.

[0007] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cell detection method described in any embodiment of this disclosure.

[0008] This disclosure also provides a program product including instructions that, when executed by a computer, perform a cell detection method as described in any embodiment of this disclosure.

[0009] This disclosure provides a method for training a cell detection model, including: Obtain a cell image dataset and perform data preprocessing on the cell image dataset; The preprocessed cell image dataset is used to train and evaluate the cell detection model to be trained, resulting in a trained cell detection model. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature maps from high to low levels to obtain a first fused feature map, and then perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature maps from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain cell type and cell location.

[0010] This disclosure also provides a training apparatus for a cell detection model, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of the training method for the cell detection model according to any embodiment of this disclosure based on the instructions stored in the memory.

[0011] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the cell detection model described in any embodiment of this disclosure.

[0012] This disclosure also provides a program product including instructions that, when executed by a computer, perform a training method for a cell detection model as described in any embodiment of this disclosure.

[0013] The cell detection method, model training method, apparatus, and storage medium of this disclosure fuse shallow detail information and high-level semantic information through a bidirectional feature fusion path of the feature fusion module, realizing multi-scale feature interaction. This enhances the model's ability to capture cell features at different scales, helps improve detection accuracy, maximizes cell classification and counting performance under limited image quality, and balances real-time performance and computational complexity, providing accurate and efficient cell detection and classification counting methods for clinical use.

[0014] Other features and advantages of this disclosure will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the disclosure. Other advantages of this disclosure may be realized and obtained by means of the methods described in the description and the accompanying drawings. Attached Figure Description

[0015] The accompanying drawings are used to provide an understanding of the technical solutions of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0016] Figure 1 A schematic flowchart of a cell detection method provided for an exemplary embodiment of this disclosure; Figure 2 A schematic diagram of the structure of a cell detection model provided for an exemplary embodiment of this disclosure; Figure 3 A schematic flowchart illustrating a training method for a cell detection model provided as an exemplary embodiment of this disclosure; Figure 4 A schematic diagram of the structure of a training device for a cell detection model provided as an exemplary embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a cell detection device provided for an exemplary embodiment of the present disclosure. Detailed Implementation

[0017] This disclosure describes several embodiments, but these descriptions are exemplary and not limiting, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0018] This disclosure includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this disclosure may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0019] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that the method or process does not depend on the specific order of steps described herein. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims relating to the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments disclosed herein.

[0020] With the development of computer vision and deep learning technologies, image-based automated cell analysis has become a research hotspot. These methods offer advantages such as simplicity, efficiency, and high reproducibility. Currently, image-based cell classification and counting methods mainly fall into two categories: traditional image methods and deep learning methods.

[0021] Traditional image processing methods acquire cell images and then perform preprocessing such as denoising and region-of-interest segmentation. Following this, features such as color, shape, and texture are extracted, and classifiers like Support Vector Machines (SVM) and Random Forests are used for classification and counting. However, traditional image processing methods are sensitive to factors like lighting and background, resulting in low accuracy in complex backgrounds and varying lighting conditions. Deep learning methods, on the other hand, largely treat cell classification and counting as an object detection task, which can be further subdivided into: (1) Two-stage detection methods first generate candidate regions, and then classify the regions and fine-tune the candidate positions, such as R-CNN, Faster R-CNN, Mask R-CNN, etc. This type of method has higher accuracy in complex scenes, but it has a large amount of computation and is slow, resulting in poor real-time performance of target detection.

[0022] (2) Single-stage detection methods directly predict bounding boxes and categories to achieve end-to-end detection without the need for a candidate region generation stage. Examples include the YOLO series and SSD (Single Shot MultiBox Detector). These methods have a faster detection speed and are suitable for real-time detection, but their accuracy is slightly lower than that of two-stage methods.

[0023] For cell detection tasks, low-level features of object detection models are rich in details such as cell edges and staining textures, but lack semantic information, while high-level features contain semantic information such as the overall morphology of the cell, but lack detailed information. Therefore, how to combine the detailed information of low-level features with the semantic information of high-level features to improve cell detection accuracy while balancing real-time performance and computational cost is a technical problem that urgently needs to be solved in this field.

[0024] like Figure 1 As shown, this disclosure provides a cell detection method, including: Step 101: Obtain cell images; Step 102: Input the cell image into a pre-trained cell detection model to obtain the cell types and locations in the cell image. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature maps from high to low levels to obtain a first fused feature map, and then perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature maps from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain the cell type and cell location.

[0025] The cell detection method provided in this disclosure fuses a first intermediate feature map and an additional feature map from high to low levels using a feature fusion module to obtain a first fused feature map. Then, it fuses the first fused feature map, the first intermediate feature map, and the additional feature map from low to high levels to obtain a second fused feature map. By fusing features at different scales, the model's ability to detect targets at multiple scales is improved. Through cross-level feature interaction, the semantic and spatial information of the features is enriched; that is, semantic information is supplemented for shallow features, and spatial details are restored for deep features. By establishing inter-level connections, the model can simultaneously utilize local details and global context, enhancing its ability to capture features at different cell scales and helping to improve detection accuracy.

[0026] The cell detection model of this disclosure can be used for the detection of various types of cells, such as white blood cells, red blood cells, and cancer cells. In other application scenarios, target detection models with structures similar to the cell detection model of this application can be constructed for the detection of microorganisms or other targets (such as defects in industrial products), and this disclosure does not impose any limitations on this. The following description will use the cell detection model for white blood cell detection as an example.

[0027] In some exemplary embodiments, cell images can be obtained using an optical imaging device. For example, the optical imaging device can be a microscope imaging device incorporating a high-resolution camera.

[0028] In other exemplary embodiments, cell images may also be obtained from public biomedical image databases or collected from the Internet via web crawlers. For example, a public biomedical image database may be the Cancer Imaging Archive (TCIA) database, and this disclosure is not limiting in this regard.

[0029] For example, the cell detection model of this disclosure can be improved based on the Single Shot MultiBox Detector (SSD) model. The SSD model is a one-stage object detection algorithm proposed by Wei Liu et al. at the European Conference on Computer Vision (ECCV 2016). It uses multi-scale feature maps to detect objects of different sizes and uses an anchor box strategy to effectively predict bounding boxes and categories.

[0030] The SSD model can be divided into three parts: the backbone network, the additional feature extraction module, and the detection module. The backbone network is used to extract basic multi-scale features of the image; the backbone network of the original SSD model is a VGG16 network. The additional feature extraction module generates higher-level features by appending convolutional layers after the backbone network, which are then combined with the features extracted by the backbone network. Figure 1 The SSD model serves as a multi-scale feature map for subsequent detection modules. The detection module predicts the class probability and position offset of the prior bounding box on each feature map and uses a non-maximum suppression algorithm to filter out overlapping or incorrect bounding boxes, obtaining the final detection result. Since the SSD model can directly predict bounding boxes and classes to achieve end-to-end detection without requiring a candidate region generation stage, it has a fast detection speed and is suitable for real-time detection.

[0031] In leukocyte detection tasks, models need to simultaneously capture both superficial detail information (such as edges and textures) and high-level semantic information (such as the overall morphology and category features of cells) to achieve accurate multi-scale feature detection. Therefore, the cell detection model in this embodiment adds a feature fusion module to the original SSD model. This module fuses superficial detail information and high-level semantic information through a bidirectional feature fusion path, enabling multi-scale feature interaction and enhancing the model's ability to capture features of leukocytes at different scales, thus improving detection accuracy.

[0032] In some examples, the backbone network of the cell detection model in this disclosure can be a MobileNetV2 network; however, this disclosure is not limiting. In practical applications, the structure of the backbone network can be configured as needed. By replacing the backbone network of the cell detection model from the original VGG16 network to a MobileNetV2 network, this disclosure enables the model to be more lightweight while maintaining detection performance.

[0033] MobileNetV2, proposed by Google, is a lightweight convolutional neural network designed specifically for mobile and resource-constrained devices. It significantly reduces computational cost and parameter count while maintaining high accuracy, and is widely used in mobile vision tasks such as image classification, object detection, and semantic segmentation. MobileNetV2 introduces inverted residuals and linear bottleneck layers, effectively improving feature representation and reducing information loss. Unlike the "compression → convolution → expansion" residual blocks in ResNet, the inverted residuals first expand the number of input channels by a factor of 6 using 1×1 convolutions, then perform 3×3 depthwise separable convolutions in a high-dimensional space to extract rich features, and finally compress back to the original number of channels using 1×1 convolutions. Therefore, the inverted residuals structure is a "expansion → depthwise convolution → compression" structure. This "dimensionality increase followed by dimensionality reduction" design enhances the expressive power of intermediate layers, making it particularly suitable for lightweight models. A linear bottleneck layer refers to using linear activation (i.e., not using ReLU) at low-dimensional inputs and outputs, thus avoiding the loss of feature information caused by ReLU in low-dimensional space. In addition, the MobileNetV2 network significantly reduces the amount of computation and the number of parameters through depthwise separable convolution, while maintaining high accuracy, making it suitable for deployment on embedded devices and mobile devices.

[0034] Table 1 shows the structure of the MobileNetV2 network, where t is the channel expansion factor (default 6x), c is the number of output channels, n is the number of bottleneck repetitions, and s is the stride. MobileNetV2 consists of multiple stacked inverted residual blocks. Referring to Table 1, the MobileNetV2 network includes a first convolutional layer, 17 inverted residual blocks, a second convolutional layer, a global average pooling layer, and a classification layer, all connected sequentially. In Table 1, the 17 inverted residual blocks are divided into 7 inverted residual groups, each corresponding to a different spatial resolution and channel dimension. Each inverted residual block contains two pointwise convolutional layers and one depthwise convolutional layer. Since each inverted residual block contains a linear bottleneck layer, Table 1 uses "bottleneck" to represent the inverted residual block.

[0035] Table 1 In this embodiment of the disclosure, the first intermediate feature map may include: the feature map output by the y-th bottleneck layer and the feature map output by the second convolutional layer (i.e., the convolutional layer before the global average pooling layer), where y is a natural number between 3 and 7. For example, y is 5. However, this disclosure does not limit this. In practical applications, the first intermediate feature map can be selected as the feature map output by which layer in the backbone network, as needed.

[0036] In this embodiment of the disclosure, the feature map output by the y-th (3≤y≤7) bottleneck layer and the feature map output by the second convolutional layer are selected as the first intermediate feature map in order to achieve a balance between detail preservation and semantic discrimination, and to control the amount of computation to meet the goal of overall lightweight deployment.

[0037] MobileNetV2 has numerous intermediate layers, and using them all would increase computational cost. The y-th (3≤y≤7) bottleneck layer is located in the middle to front of the network. Its output feature map has high resolution, rich details, and retains more spatial location information and texture details. In white blood cell images, subtle morphological features such as the segmented nuclei of neutrophils and the granules of eosinophils require high-resolution features to capture. The second convolutional layer (before the global average pooling layer) is the last convolutional layer in the MobileNetV2 backbone network (after the global average pooling layer is the fully connected layer, i.e., the classification layer). Its output feature map has lower resolution but more channels and a larger receptive field, containing rich semantic information and global context. High-level semantic features are more critical when distinguishing white blood cell categories (such as the overall morphological differences between lymphocytes and monocytes). In this embodiment, the feature map output by the convolutional layer before the y-th (3≤y≤7) bottleneck layer and the average pooling layer of MobileNetV2 is selected as the first intermediate feature map. That is, a combination of a shallower layer (rich in details) and a deeper layer (rich in semantics) is chosen, thus adapting to the structural characteristics of the lightweight network.

[0038] In some exemplary embodiments, the additional feature extraction module includes a plurality of standard convolutional layers connected in sequence, and the additional feature map includes the feature map output by each of the plurality of standard convolutional layers.

[0039] For example, the number of standard convolutional layers can be 4 or 5, etc. However, this disclosure does not limit this. In practical use, the value of the number of standard convolutional layers can be set according to actual needs (e.g., the size of the input image, the feature extraction depth of the backbone network, and the feature depth to be extracted, etc.).

[0040] The additional feature extraction module adds several standard convolutional layers (such as 3x3 Conv) after the backbone network to further extract multi-scale features and generate smaller feature maps. These feature maps, together with some feature maps from the backbone network, constitute feature maps of different scales and serve as input to the feature fusion module.

[0041] In some exemplary embodiments, the feature fusion module includes: a channel adjustment module, a first feature fusion module, and a second feature fusion module, wherein: The channel adjustment module is configured to adjust the number of channels of the first intermediate feature map and the additional feature map so that the number of channels of the first intermediate feature map and the additional feature map are unified, and output the adjusted first intermediate feature map and the additional feature map to the first feature fusion module and the second feature fusion module. The first feature fusion module is configured to perform channel attention weighted calculation on at least a portion of the feature maps in the first intermediate feature map and the additional feature map, and perform feature fusion from the high layer to the low layer to obtain a first fused feature map, and input the first fused feature map into the second feature fusion module. The second feature fusion module is configured to perform feature fusion from low to high layers on the first fused feature map, the first intermediate feature map, and the additional feature map to obtain a third fused feature map, and to perform channel attention weighted calculation on at least a portion of the third fused feature map to obtain the second fused feature map.

[0042] The feature fusion module of this disclosure performs multi-scale feature fusion on feature maps of different scales during the feature fusion process, thereby solving problems such as large changes in the scale of the detected target and the difficulty in detecting small targets in the image, improving the model's ability to detect multi-scale targets, and reducing model complexity. Furthermore, the feature fusion module of this disclosure embeds a channel attention mechanism into the feature fusion process. When performing feature fusion from high to low levels, channel attention calculation is placed before feature fusion, which can effectively filter noise in the feature map and allow effective detail information to fully participate in the fusion. When performing feature fusion from low to high levels, channel attention calculation is placed after feature fusion, which can automatically adjust the contribution weights of semantics and details, enhancing key channels.

[0043] In some exemplary embodiments, the method further includes: dividing the first intermediate feature map and the additional feature map into high-level feature maps and low-level feature maps according to the number of convolutional layers passed through.

[0044] This embodiment of the disclosure divides the feature map to be fused into a high-level feature map and a low-level feature map. Subsequently, different fusion methods can be used to fuse the high-level feature map and the low-level feature map, which can further reduce the amount of computation.

[0045] In some exemplary embodiments, the low-level feature maps include the feature map output by the y-th bottleneck layer, the feature map output by the second convolutional layer (i.e., the convolutional layer before the global average pooling layer), and the feature maps output by the first to the Kth convolutional layers among multiple standard convolutional layers. The high-level feature maps include the feature maps output by the (K+1)th to the Nth convolutional layers among multiple standard convolutional layers, where K is a natural number between 1 and N-2, and N is the number of standard convolutional layers and N is greater than 2.

[0046] For example, with Figure 2 Taking the network structure shown as an example, y=5, N=4, K=1, that is, the low-level feature map includes the feature map output by the 5th bottleneck layer. The feature map output by the second convolutional layer (i.e., the convolutional layer before the global average pooling layer) The feature map output by the first convolutional layer in the four standard convolutional layers. The high-level feature maps include the feature maps output from the second to fourth convolutional layers out of the four standard convolutional layers. , and .

[0047] In some exemplary embodiments, the first feature fusion module includes a first high-level feature fusion module, a first channel attention calibration module, and a first low-level feature fusion module; wherein: The first high-level feature fusion module is configured to perform the following operations on the high-level feature map: set the initial feature map of the highest layer as the first fused feature map of the highest layer; starting from the second highest layer, perform upsampling operation by comparing it with the first fused feature map of the next higher layer, and add the upsampled first fused feature map to the initial feature map of the current layer element by element and perform feature enhancement to obtain the first fused feature map of the current layer; The first channel attention calibration module is configured to perform the following operations on the low-level feature map: calculate the channel weights of the original feature map of each layer, and multiply the calculated channel weights element-wise with the original feature map of each layer to obtain the calibrated feature map of each layer. The first low-level feature fusion module is configured to perform the following operations on the low-level feature map: starting from the highest layer in the low-level feature map, it performs an upsampling operation on the first fusion feature map of the next higher layer, and adds the upsampled first fusion feature map to the calibrated feature map of the current layer element by element and performs feature enhancement to obtain the first fusion feature map of the current layer.

[0048] Because low-level feature maps have high resolution and are rich in details such as edges and textures, but contain a lot of noise and weak semantic information, this embodiment adds channel attention to the original features in the low-level feature maps during top-down fusion. This can filter out noise in the low-level feature maps and allow effective detail information to participate in the fusion.

[0049] In some exemplary embodiments, the second feature fusion module includes a second low-level feature fusion module, a second high-level feature fusion module, and a second channel attention calibration module; wherein: The second low-level feature fusion module is configured to perform the following operations on the low-level feature map: add the initial feature map of the lowest layer to the first fused feature map of this layer element by element and perform feature enhancement to obtain the second fused feature map of the lowest layer; starting from the next lowest layer, perform downsampling operation on the second fused feature map of the layer below this layer, and add the downsampled second fused feature map to the initial feature map of this layer and the first fused feature map of this layer element by element and perform feature enhancement to obtain the second fused feature map of this layer; The second high-level feature fusion module is configured to perform the following operations on the high-level feature map: starting from the lowest layer of the high-level feature map, it performs a downsampling operation on the second fusion feature map of the next lower layer, and adds the downsampled second fusion feature map to the initial feature map of the current layer and the first fusion feature map of the current layer element by element and performs feature enhancement to obtain the third fusion feature map of the current layer. The second channel attention calibration module is configured to directly use the third fusion feature map corresponding to the low-level feature map as the second fusion feature map corresponding to the low-level feature map; calculate the channel weights of the third fusion feature map corresponding to the high-level feature map; and multiply the calculated channel weights element-wise with the third fusion feature map to obtain the second fusion feature map corresponding to the high-level feature map.

[0050] Because high-level feature maps have low resolution and are rich in semantic information but lack spatial detail, this embodiment adds channel attention to the fusion result corresponding to the high-level feature map during bottom-up fusion. This can automatically adjust the contribution weights of semantics and details, thereby enhancing key channels.

[0051] Furthermore, both the second low-level feature fusion module and the second high-level feature fusion module in this embodiment of the present disclosure use three-layer aggregation when performing feature fusion. That is, the fusion object includes: the second fusion feature map after downsampling the second fusion feature map one level lower than this layer, the first fusion feature map of this layer, and the initial feature map of this layer, thereby making the fusion information more complete.

[0052] In some exemplary embodiments, feature enhancement is performed, specifically through depthwise separable convolution. However, this disclosure is not limiting in this regard.

[0053] This disclosure utilizes depthwise separable convolution for feature enhancement, significantly reducing the number of parameters and making the model more lightweight while maintaining detection performance. Depthwise separable convolution significantly reduces computation by breaking down the standard convolution operation into two simpler steps: depthwise convolution + pointwise convolution. Taking an input feature map of size... The number of input channels is The number of output channels is Taking convolution operations as an example: The computational cost of standard convolution is: Where K is the kernel size (e.g., 3×3). The computational cost of depthwise separable convolution is: (Pointwise convolution). When using a 3×3 convolution kernel, the computational cost of depthwise separable convolution is about 1 / 8 to 1 / 9 of that of standard convolution, thus significantly reducing the computational burden.

[0054] In this embodiment, channel attention is added only to low-level feature maps during top-down fusion, and only to high-level feature maps during bottom-up fusion. Therefore, this lightweight channel attention design mechanism can significantly reduce computational overhead. Furthermore, in the cell detection model of this embodiment, the hidden layer dimension of the fully connected layer can be set to a small value (e.g., C / 16 or C / 8, where C represents the number of channels in the input feature map, and 16 or 8 is the compression ratio), thereby further reducing the number of parameters. In addition, this disclosure maintains low overall computational cost by combining feature fusion with depthwise separable convolution. Compared to using standard convolutions or complex attention at every layer and every location, this selective, low-overhead design is more suitable for deployment in embedded devices.

[0055] The feature fusion module in this embodiment enriches the semantic and spatial information of features through cross-level feature interaction. Specifically, it supplements semantic information for shallow features and restores spatial details for deep features. By establishing inter-level connections, the model can simultaneously utilize local details and global context. For the white blood cell detection task, the low-level features of the SSD model are rich in details such as cell edges and staining textures, but lack semantic information. High-level features contain semantic information such as the overall cell morphology, but lack detailed information. Therefore, multi-scale feature fusion is necessary.

[0056] Common feature fusion methods include: Feature Pyramid (FPN), which uses top-down unidirectional fusion to upsample high-level semantic features and add them to shallow features, significantly improving small target detection performance. However, the unidirectional information flow results in insufficient semantic information in the lower-level features. Path Aggregation (PANet) adds a bottom-up bidirectional fusion path to FPN to enhance feature transfer and significantly improve small target detection performance. However, the computational cost of dual-path standard convolution is high. Weighted Bidirectional Feature Pyramid (BiFPN) optimizes multi-scale feature interaction through bidirectional cross-scale connections and dynamic weight fusion. Although its node reuse mechanism reduces the number of parameters by about 30% compared to PANet, it faces two limitations in white blood cell detection tasks. First, the dynamic weight mechanism relies on high-quality feature representation. In low-contrast / blurred white blood cell images, weight learning is easily affected by noise. Second, the cross-scale connections constructed by standard convolution still retain a lot of redundant computation, which is not conducive to clinical equipment deployment.

[0057] Therefore, this disclosure presents a lightweight feature fusion method that achieves multi-scale feature interaction through a bidirectional feature fusion path (top-down and bottom-up). Simultaneously, it utilizes depthwise separable convolution (DSConv) instead of standard convolution, significantly reducing the number of parameters and computational cost. The following examples illustrate this approach. Figure 2 Taking the cell detection model shown as an example, the feature fusion process of this disclosure embodiment is introduced. Exemplarily, the feature fusion process includes the following steps: (1) Receive multi-scale feature maps from the backbone network (MobileNetV2) and the additional feature extraction module, assuming there are 6 { , , , , , The dimensions decrease sequentially (e.g., 80×80, 40×40, 20×20, 10×10, 5×5, 1×1), and the number of channels may vary.

[0058] (2) For each feature map Adaptive channel unification is performed using 1×1 convolution, mapping all feature maps to the same number of channels. The resulting feature map is denoted as... The formula is as follows: .

[0059] For a top-down fusion process, fusion proceeds step-by-step from high-level semantic features to low-level detailed features, i.e., from... Begin, gradually downwards and Fusion. Fusion of high-level feature maps. Perform an upsampling operation and compare it with the current layer feature map. Element-wise addition is performed, followed by feature enhancement through depthwise separable convolution to obtain the fused features of this layer. At the same time, some low-level features Lightweight channel attention is used to generate channel weights through global average pooling (GAP) and fully connected layers (MLP), filtering out noise information in shallow features and enhancing effective channels. In this process, high-level semantic information progressively corrects shallow features, achieving semantic information propagation from high-level (low-resolution) to low-level (high-resolution) layers. Simultaneously, depthwise separable convolution (DSConv) is used to eliminate aliasing effects from upsampling (such as jagged edges caused by nearest-neighbor interpolation), as expressed by the following formula: ; ; .

[0060] in, It is a feature that has been merged from the previous level (initially). = Upsample is the nearest neighbor interpolation upsampling.

[0061] For a bottom-up fusion process, the fusion proceeds step-by-step from low-level detailed features to high-level semantic features, that is, from... Begin, gradually move upwards and Fusion. This involves combining low-level feature maps. After downsampling, the feature map is not fused with the current layer. and the feature map obtained by fusion Element-wise addition is performed, followed by feature enhancement through depthwise separable convolution to obtain the fused features of this layer. Simultaneously, lightweight channel attention is added to some of the fused high-level features. This is achieved by connecting an SE module after fusion, which generates channel weights through global average pooling (GAP) and a fully connected layer (MLP), automatically adjusting the contributions of details and semantics. In this process, shallow detail information progressively corrects high-level features, supplementing the details from shallow (high-resolution) to high-level (low-resolution) features. DSConv is also used to smooth out potentially lost local information during downsampling, as expressed by the following formula: ; ; .

[0062] Among them, the initial , It represents the output features of a top-down path. Downsampling can be achieved through depthwise separable convolution or max pooling / average pooling. (Right now ) is the final bidirectional fusion output.

[0063] (3) Output the fused multi-scale features This is used in subsequent detection modules.

[0064] The feature fusion process in this embodiment includes a parallel bidirectional interactive fusion process of top-down and bottom-up, where the two fusion paths coexist and intertwine. In top-down fusion, both the initial features and the top-down fusion result are used simultaneously. In bottom-up fusion, the initial features, the top-down fusion result, and the bottom-up fusion result are used simultaneously. (The last sentence appears to be incomplete and possibly refers to a calculation method.) For example, ,in, The output features are those of a bottom-up path. The output features of the top-down path, The initial features are used, which means that the final fusion result of each layer aggregates information from three directions.

[0065] Furthermore, since low-level features (high resolution) are prone to containing noise, this disclosure, in the top-down fusion process, removes the original features of the low-level layers. Channel attention (SE) is applied first to filter noise before participation in fusion. Since high-level features (low resolution) are rich in semantic information but lack spatial detail, this disclosure, in the bottom-up fusion process, applies channel attention (SE) to filter noise before participating in fusion. Channel attention is applied to automatically adjust the contribution weights of details and semantics. Embodiments of this disclosure embed channel attention into the feature fusion process, thereby dynamically filtering noise and enhancing semantics during feature fusion.

[0066] In some exemplary embodiments, the detection module includes a prediction layer for each second fused feature map, and also includes a non-maximum suppression module, wherein: The prediction layer is configured to output the class probabilities of the predicted prior boxes via the classification head and the position offsets of the predicted prior boxes via the regression head. The nonmaximum suppression module is configured to filter prior boxes to obtain the final detection result.

[0067] In this embodiment of the disclosure, the detection module includes a prediction layer for each second fused feature map, the prediction layer consisting of a classification head and a regression head. The classification head uses convolution to output the class score of each prior box, followed by Softmax normalization to a probability distribution, and the regression head uses convolution to output the position offset of the prior box.

[0068] In this embodiment of the disclosure, the detection module further includes a non-maximum suppression module, which filters the prior boxes using a non-maximum suppression algorithm (NMS) to obtain the final detection result.

[0069] In some exemplary embodiments, to address the problem of poor image quality of white blood cells acquired by optical imaging devices, a deep learning-based super-resolution reconstruction model can be set before the cell detection model in this embodiment. During model training, the super-resolution reconstruction model and the cell detection model are jointly trained, thereby utilizing super-resolution reconstruction technology to improve the quality of the input image of the cell detection model and enhance the detection performance of the cell detection model.

[0070] Clinical deployment requires cell detection models to possess both high accuracy and real-time performance to meet the needs of rapid clinical diagnosis. Simultaneously, the models need to be lightweight to adapt to different hardware environments and resource constraints. This disclosure improves the model structure by replacing the backbone network, designing a lightweight feature fusion module, and introducing contrastive learning. This enhances the model's ability to detect multi-scale cellular features while maintaining detection accuracy, thereby reducing the number of parameters and computational load, thus balancing detection accuracy and speed.

[0071] like Figure 3 As shown in the embodiments of this disclosure, a method for training a cell detection model is also provided, including: Step 301: Obtain the cell image dataset and perform data preprocessing on the cell image dataset; Step 302: Train and evaluate the cell detection model to be trained using the preprocessed cell image dataset to obtain a trained cell detection model. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature map from high to low levels to obtain a first fused feature map, and then perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature map from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain the cell type and cell location.

[0072] In this embodiment, the structure of the specific cell detection model can be referred to the above description, and will not be repeated here.

[0073] The cell detection model of this disclosure can be used for the detection of various types of cells, such as white blood cells, red blood cells, and cancer cells. In other application scenarios, target detection models with structures similar to the cell detection model of this application can be constructed for the detection of microorganisms or other targets (such as defects in industrial products), and this disclosure does not impose any limitations on this. The following description will use the cell detection model for white blood cell detection as an example.

[0074] In some exemplary embodiments, in step 301, the cell image dataset is preprocessed, including at least one of the following operations: data annotation, dataset partitioning, data augmentation, etc.

[0075] In this embodiment of the disclosure, taking white blood cell detection as an example, during data annotation, bounding boxes are marked for various white blood cells (such as neutrophils, lymphocytes, monocytes, eosinophils, basophils, etc.) in the white blood cell image, and the white blood cell category corresponding to each bounding box is recorded. The annotation process can be completed by professional medical personnel to ensure the accuracy of the annotation.

[0076] In this embodiment of the disclosure, when partitioning the dataset, the labeled dataset can be divided into a training set, a validation set, and a test set, which are used for model training, optimization, and final performance evaluation, respectively.

[0077] In this embodiment of the disclosure, various data augmentation operations can be used to increase data diversity and the number of training samples, thereby helping the model to better adapt to different scenarios and improve its generalization ability.

[0078] For example, data augmentation operations may include at least one of the following operations: translation, rotation, scaling, flipping, random cropping, photometric distortion, etc.

[0079] Translation includes horizontal and vertical movement to simulate the different locations of white blood cells; rotation refers to rotating the image at a certain angle to accommodate possible tilting of white blood cells; scaling involves enlarging or reducing the image to enhance the model's adaptability to changes in the distance of white blood cells; flipping includes horizontal, vertical, and biaxial flipping; random cropping involves randomly selecting a region from the image to increase the model's focus on local features; and photometric distortion is a data augmentation method that enhances data diversity by adjusting visual attributes such as color, brightness, and contrast of an image, aiming to improve the model's adaptability to different lighting and color conditions.

[0080] For example, photometric distortion includes one or more of the following types: brightness adjustment, contrast adjustment, hue and saturation perturbation, color jitter, noise injection, etc. Brightness adjustment refers to increasing or decreasing the overall brightness of the image to simulate different lighting scenarios such as daytime, dusk, and nighttime. For example, decreasing image brightness allows the model to learn to identify white blood cells under low-light conditions. Contrast adjustment refers to adjusting the degree of difference between bright and dark areas in the image, enhancing or weakening detail. High contrast can highlight edge features, while low contrast simulates foggy or backlit blurring effects. Hue and saturation perturbation refers to changing the color tendency and vibrancy of the image, simulating the effects of different color temperature light sources (such as the yellowish tint of incandescent bulbs and the bluish tint of LEDs), avoiding misjudgments by the model due to color deviation. Color jitter refers to randomly perturbing the RGB channel values, comprehensively adjusting brightness, contrast, saturation, and hue to generate diverse color variations. Noise injection can add Gaussian noise, salt-and-pepper noise, etc., to simulate sensor noise or transmission interference, improving the model's noise resistance.

[0081] In other examples, data augmentation operations may also include blur enhancement operations, which may include at least one of the following operations: Gaussian blur, motion blur, etc., to address issues such as focus blur and / or motion blur present in real optical imaging devices.

[0082] Gaussian blur is an image smoothing technique based on the Gaussian function. It blurs image details through a weighted average, reducing noise and detail levels. Its core idea is to use a Gaussian function to calculate a convolution kernel (filter kernel), with the highest weight at the center and gradually decreasing towards the edges, thus smoothing the image. When using the Gaussian blur algorithm to simulate focus blur, the degree of blur can be controlled by adjusting the size and standard deviation of the Gaussian kernel.

[0083] Motion blur is a visual blurring effect that simulates the rapid movement of an object within an exposure time. It typically manifests as a linear blur along a specific direction, and its mathematical model is based on the object's displacement within the exposure time. When simulating motion blur using motion blur algorithms, different motion directions and blur lengths can be set.

[0084] In practical applications, the different data augmentation methods described above can be randomly combined and executed to further enhance data diversity. The embodiments of this disclosure significantly improve the generalization ability of the model through the above data augmentation operations.

[0085] In some exemplary embodiments, the preprocessed cell image dataset is used to train and evaluate the cell detection model to be trained, including: The preprocessed cell image dataset was divided into a training set, a validation set, and a test set. The training set is used to train the cell detection model to be trained. The total loss of the model on the validation set is calculated according to the preset loss function. The model parameters are adjusted according to the calculated total loss. The preset loss function includes contrastive learning loss. After training, the model performance is evaluated using a test set.

[0086] In some exemplary embodiments, the loss function of the cell detection model can be the MultiBox Loss multi-task loss function, which is composed of a weighted average of classification loss and regression loss, jointly optimizing the classification and localization tasks.

[0087] The classification loss can be calculated using the Softmax cross-entropy loss, which measures the difference between the predicted class probability and the true label. ; in, .

[0088] in, For category confidence error, This represents the predicted confidence level for the category. Category index (from 0 to C), where 0 represents the background class. Let P be the class probability of the ground truth box (GT box) corresponding to the predicted i-th default box. This indicates whether the i-th default box matches the j-th ground truth box (of category P).

[0089] The localization loss can use Smooth L1 loss to regress the bounding box coordinate offset: ; .

[0090] in, This is due to the error in the search box position. This represents the predicted position of the bounding box corresponding to the prior box. For the position parameters of the actual target, Represents the set of positive samples. The four parameters representing the bounding box are the coordinates of the center point (…). ) and width and height ( ), These are the bounding box parameters predicted for the i-th default box. Let be the bounding box parameters corresponding to the j-th ground truth bounding box matched with positive sample i. To balance the L1 loss function.

[0091] The total loss of the model can be obtained by weighted summing of the classification loss and the localization loss: ; in, This represents the number of positive samples in the prior bounding box. These are the localization loss weights, used to balance the impact of localization loss and classification loss on model training.

[0092] However, when using the MultiBox Loss multi-task loss function mentioned above, the Softmax classification head only constrains correct classification and fails to explicitly constrain the clustering of similar samples and the separation of dissimilar samples. Furthermore, the poor quality of white blood cell images acquired using low-cost imaging equipment results in small differences in visual features between different similar categories of white blood cells. The original detection model struggles to effectively distinguish these similar categories, such as insufficient ability to differentiate between eosinophils and basophils, thus affecting the accuracy of classification and counting.

[0093] To address this issue, this disclosure employs a dual-branch classification head, comprising a conventional Softmax main classification branch and an auxiliary classification branch based on contrastive learning. Contrastive learning applies an attraction loss to features of similar cells and a repulsion loss to features of dissimilar cells, causing similar white blood cells to cluster tightly in the feature space while dissimilar white blood cells (e.g., neutrophils vs. lymphocytes) are spaced further apart. This explicitly optimizes the feature space, enhances the model's ability to distinguish similar categories, and solves the problem of insufficient discrimination ability of the original detection model for similar categories in low-quality images.

[0094] In some exemplary implementations, the contrastive learning loss is calculated using the following methods: The second fused feature map is projected onto the low-dimensional feature embedding space to obtain feature embedding vectors after the projection of multiple prior boxes. Construct contrastive learning samples, which include positive sample pairs and negative sample pairs. Positive sample pairs include feature embedding vectors projected from cell prior boxes of the same class, and negative sample pairs include feature embedding vectors projected from cell prior boxes of different classes. The contrastive learning loss is calculated based on the constructed contrastive learning samples.

[0095] In some exemplary implementations, taking white blood cell detection as an example, the contrastive learning loss can be calculated using the following method: (1) Use 1×1 convolution to convert the input feature map (i.e. Project the vector into a low-dimensional feature embedding space (e.g., 128-dimensional) and output the feature vector f(x).

[0096] (2) Construct contrastive learning samples. Precedent bounding box feature embeddings of the same category of white blood cells constitute positive sample pairs, and need to be filtered by the real label (only positive samples with IoU > 0.5 with the real prior bounding box are selected). Precedent bounding box feature embeddings of different categories of white blood cells constitute negative sample pairs.

[0097] (3) Supervised contrastive learning loss is used as the loss function for the contrastive learning head, calculating the contrastive learning loss of attraction between similar features and repulsion between different features. Real label information is used to constrain similar features to move closer and different features to move further away, improving feature discriminativeness. .

[0098] in, It is the category of the prior bounding box. For prior boxes Feature embedding after projection (after L2 normalization). To and A set of positive samples of the same type (excluding) This means that these samples should be close to each other in the feature space. For all samples (including negative samples), used to distinguish between positive and negative samples, This is the temperature coefficient, which controls the discriminative power of feature similarity. It is usually set to 0.1 (experimental optimization is required).

[0099] In some exemplary implementations, the total model loss (a preset loss function) is a weighted sum of classification loss, regression loss, and contrastive learning loss, wherein the classification loss is cross-entropy loss and the regression loss is smoothed average absolute error loss.

[0100] For example, the total loss function of the entire model is: .

[0101] in, It is the cross-entropy loss of the prior bounding boxes (Softmax branch). It is the position offset of the prior bounding box, the Smooth L1 loss (regression head). It is the contrastive loss (auxiliary branch) of the prior bounding box features. This is the positioning loss weight (usually set to 1, but adjustable). It is the contrastive learning loss weight, used to balance the impact of different loss terms on model training, and is usually set to 0.1 (adjustable parameter).

[0102] In some exemplary implementations, when evaluating model performance using a test set, metrics such as mean AP, accuracy, recall, F1 score, and FPS (Frames Per Second) can be used to evaluate model performance.

[0103] In some exemplary embodiments, the training method further includes: The trained cell detection model is then quantified.

[0104] Depending on the actual application scenario of the cell detection model (such as portable blood cell analyzers, edge computing devices, etc.), after completing the model evaluation and confirming that the model performance meets the expected standards, the model can be quantized to further reduce the model's computational and storage requirements. Specific model quantization methods can be set as needed, and this disclosure does not impose any restrictions on them.

[0105] After quantization, the quantized model is deployed to the pre-specified device and necessary debugging and optimization are performed to ensure that the model can run stably and efficiently in the target environment.

[0106] In this embodiment of the disclosure, taking white blood cell detection as an example, a qualified model can be applied to white blood cell classification and counting tasks in clinical scenarios to achieve rapid and accurate detection and classification of white blood cells.

[0107] For example, the reasoning process of the cell detection model is as follows: The model receives a white blood cell image; through the backbone network and additional feature extraction module, feature maps of different scales are obtained; these feature maps are fused bidirectionally by the feature fusion module to output a second fused feature map; the second fused feature map is input into the detection module for classification and regression prediction, the classification head obtains the class probability of the predicted box, and the regression head obtains the position offset of the predicted box; overlapping predicted boxes are filtered by the non-maximum suppression algorithm to obtain the final detection result (e.g., a white blood cell image contains 2 lymphocytes and 2 centrifugal cells).

[0108] This disclosure also provides a cell detection apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform steps of the cell detection method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0109] like Figure 4As shown, in one example, the cell detection device may include: a first processor 410, a first memory 420, a first bus system 430, and a first transceiver 440, wherein the first processor 410, the first memory 420, and the first transceiver 440 are connected through the first bus system 430, the first memory 420 is used to store instructions, and the first processor 410 is used to execute the instructions stored in the first memory 420 to control the first transceiver 440 to transmit and receive signals. Specifically, the first transceiver 440 can acquire cell images under the control of the first processor 410. The first processor 410 inputs the cell images into a pre-trained cell detection model to obtain the cell types and locations in the cell images. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature maps from high to low levels to obtain a first fused feature map, and perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature maps from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain the cell type and cell location.

[0110] It should be understood that the first processor 410 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0111] The first memory 420 may include read-only memory and random access memory, and provides instructions and data to the first processor 410. A portion of the first memory 420 may also include non-volatile random access memory. For example, the first memory 420 may also store device type information.

[0112] In addition to the data bus, the first bus system 430 may also include a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The general designated all buses as the first bus system 430.

[0113] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the first processor 410 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in the first memory 420. The first processor 410 reads information from the first memory 420 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, further details are omitted here.

[0114] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the cell detection method as described in any embodiment of this disclosure. The cell detection method driven by executing executable instructions is essentially the same as the cell detection method provided in the above embodiments of this disclosure, and will not be described in detail here.

[0115] In some possible implementations, various aspects of the cell detection methods provided in this disclosure may also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps in the cell detection methods according to various exemplary embodiments of this disclosure as described above, for example, the computer device may execute the cell detection methods described in the embodiments of this disclosure.

[0116] This disclosure also provides a training apparatus for a cell detection model, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of a training method for a cell detection model as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0117] like Figure 5As shown, in one example, the training device for the cell detection model may include: a second processor 510, a second memory 520, a second bus system 530, and a second transceiver 540, wherein the second processor 510, the second memory 520, and the second transceiver 540 are connected through the second bus system 530, the second memory 520 is used to store instructions, and the second processor 510 is used to execute the instructions stored in the second memory 520 to control the second transceiver 540 to transmit and receive signals. Specifically, the second transceiver 540, under the control of the second processor 510, acquires a cell image dataset. The second processor 510 preprocesses the cell image dataset and uses the preprocessed cell image dataset to train and evaluate the cell detection model to be trained, thereby obtaining the trained cell detection model. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature map from high to low levels to obtain a first fused feature map, and perform feature fusion on the first fused feature map, the first intermediate feature map, and the additional feature map from low to high levels to obtain a second fused feature map. The detection module is configured to perform target classification and location regression on the second fused feature map to obtain the cell type and cell location.

[0118] It should be understood that the second processor 510 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0119] The second memory 520 may include read-only memory and random access memory, and provides instructions and data to the second processor 510. A portion of the second memory 520 may also include non-volatile random access memory. For example, the second memory 520 may also store device type information.

[0120] In addition to the data bus, the second bus system 530 may also include a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5The general designated all buses as the second bus system 530.

[0121] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the second processor 510 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in the second memory 520. The second processor 510 reads information from the second memory 520 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, further details are omitted here.

[0122] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the cell detection model as described in any embodiment of this disclosure. The training method for the cell detection model driven by executing executable instructions is essentially the same as the training method for the cell detection model provided in the above embodiments of this disclosure, and will not be described in detail here.

[0123] In some possible implementations, various aspects of the training method for the cell detection model provided in this disclosure can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps in the training method for the cell detection model according to various exemplary embodiments of this disclosure described above. For example, the computer device can execute the training method for the cell detection model described in the embodiments of this disclosure.

[0124] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0125] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0126] It should be noted that the above embodiments or implementation methods are merely exemplary and not restrictive. Therefore, this disclosure is not limited to the content specifically shown and described herein. Various modifications, substitutions, or omissions can be made to the form and details of the implementations without departing from the scope of this disclosure.

Claims

1. A cell detection method, characterized in that, include: Acquire cell images; The cell image is input into a pre-trained cell detection model to obtain the types and locations of cells in the cell image. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, to obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature map from high to low to obtain a first fused feature map, and to perform feature fusion on the first fused feature map, the first intermediate feature map and the additional feature map from low to high to obtain a second fused feature map; the detection module is configured to perform target classification and location regression on the second fused feature map to obtain cell type and location.

2. The method according to claim 1, characterized in that, The feature fusion module includes: a channel adjustment module, a first feature fusion module, and a second feature fusion module, wherein: The channel adjustment module is configured to adjust the number of channels of the first intermediate feature map and the additional feature map so that the number of channels of the first intermediate feature map and the additional feature map are unified, and output the adjusted first intermediate feature map and the additional feature map to the first feature fusion module and the second feature fusion module. The first feature fusion module is configured to perform channel attention weighted calculation on at least a portion of the feature maps in the first intermediate feature map and the additional feature map, and perform feature fusion from high layer to low layer to obtain a first fused feature map, and input the first fused feature map into the second feature fusion module; The second feature fusion module is configured to perform feature fusion from low to high layers on the first fused feature map, the first intermediate feature map, and the additional feature map to obtain a third fused feature map, and to perform channel attention weighted calculation on at least a portion of the third fused feature map to obtain the second fused feature map.

3. The method according to claim 2, characterized in that, The method further includes: dividing the first intermediate feature map and the additional feature map into high-level feature maps and low-level feature maps according to the number of convolutional layers passed through; The first feature fusion module includes a first high-level feature fusion module, a first channel attention calibration module, and a first low-level feature fusion module; wherein: The first high-level feature fusion module is configured to perform the following operations on the high-level feature map: set the initial feature map of the highest layer as the first fused feature map of the highest layer; starting from the second highest layer, perform upsampling operation by comparing the first fused feature map of the next higher layer, and add the upsampled first fused feature map to the initial feature map of the current layer element by element and perform feature enhancement to obtain the first fused feature map of the current layer; The first channel attention calibration module is configured to perform the following operation on the low-level feature map: calculate the channel weights of each original feature map, and multiply the calculated channel weights element-wise with each original feature map to obtain the calibrated feature map of each layer; The first low-level feature fusion module is configured to perform the following operations on the low-level feature map: starting from the highest layer of the low-level feature map, perform an upsampling operation on the first fusion feature map one layer higher than the current layer, and add the upsampled first fusion feature map to the calibrated feature map of the current layer element by element and perform feature enhancement to obtain the first fusion feature map of the current layer.

4. The method according to claim 2, characterized in that, The method further includes: dividing the first intermediate feature map and the additional feature map into high-level feature maps and low-level feature maps according to the number of convolutional layers passed through; The second feature fusion module includes a second low-level feature fusion module, a second high-level feature fusion module, and a second channel attention calibration module; wherein: The second low-level feature fusion module is configured to perform the following operations on the low-level feature map: add the initial feature map of the lowest layer to the first fused feature map of the current layer element by element and perform feature enhancement to obtain the second fused feature map of the lowest layer; starting from the next lowest layer, perform downsampling operation on the second fused feature map of the layer below the current layer, and add the downsampled second fused feature map to the initial feature map of the current layer and the first fused feature map of the current layer element by element and perform feature enhancement to obtain the second fused feature map of the current layer; The second high-level feature fusion module is configured to perform the following operations on the high-level feature map: starting from the lowest layer of the high-level feature map, performing a downsampling operation on the second fusion feature map of the next lower layer, and adding the downsampled second fusion feature map element by element with the initial feature map of the current layer and the first fusion feature map of the current layer and performing feature enhancement to obtain the third fusion feature map of the current layer; The second channel attention calibration module is configured to directly use the third fusion feature map corresponding to the low-level feature map as the second fusion feature map corresponding to the low-level feature map; calculate channel weights for the third fusion feature map corresponding to the high-level feature map; and multiply the calculated channel weights element-wise with the third fusion feature map to obtain the second fusion feature map corresponding to the high-level feature map.

5. The method according to claim 3 or 4, characterized in that, The feature enhancement is specifically performed by using depthwise separable convolution.

6. The method according to claim 3 or 4, characterized in that, The backbone network is a MobileNetV2 network, which includes a first convolutional layer, multiple inverted residual blocks, a second convolutional layer, a global average pooling layer, and a classification layer connected in sequence. The first intermediate feature map includes the feature map output by the y-th inverted residual block and the feature map output by the second convolutional layer, where y is a natural number between 3 and 7.

7. The method according to claim 6, characterized in that, The additional feature extraction module includes multiple standard convolutional layers connected in sequence, and the additional feature map includes the feature map output by each of the multiple standard convolutional layers.

8. The method according to claim 7, characterized in that, The low-level feature map includes the feature map output by the y-th inverted residual block, the feature map output by the second convolutional layer, and the feature maps output by the first to the Kth standard convolutional layers among the plurality of standard convolutional layers. The high-level feature map includes the feature maps output by the (K+1)th to the Nth standard convolutional layers among the plurality of standard convolutional layers, where K is a natural number between 1 and N-2, and N is the number of standard convolutional layers and N is greater than 2.

9. The method according to claim 1, characterized in that, The detection module includes a prediction layer for each of the second fused feature maps, and also includes a non-maximum suppression module, wherein: The prediction layer is configured to output the class probabilities of the predicted prior boxes through a classification head and the position offsets of the predicted prior boxes through a regression head. The nonmaximum suppression module is configured to filter the prior boxes to obtain the final detection result.

10. A training method for a cell detection model, characterized in that, include: Obtain a cell image dataset and perform data preprocessing on the cell image dataset; The preprocessed cell image dataset is used to train and evaluate the cell detection model to be trained, resulting in a trained cell detection model. The cell detection model includes a backbone network, an additional feature extraction module, a feature fusion module, and a detection module. The backbone network is configured to extract features from the input image to obtain a first extracted feature map, and during the feature extraction process, obtain one or more first intermediate feature maps. The additional feature extraction module is configured to perform multi-scale feature extraction on the first extracted feature map to obtain multiple additional feature maps. The feature fusion module is configured to perform feature fusion on the first intermediate feature map and the additional feature map from high to low to obtain a first fused feature map, and to perform feature fusion on the first fused feature map, the first intermediate feature map and the additional feature map from low to high to obtain a second fused feature map; the detection module is configured to perform target classification and location regression on the second fused feature map to obtain cell type and location.

11. The method according to claim 10, characterized in that, The process of training and evaluating the cell detection model to be trained using the preprocessed cell image dataset includes: The preprocessed cell image dataset was divided into a training set, a validation set, and a test set. The training set is used to train the cell detection model to be trained. The total loss of the model on the validation set is calculated according to a preset loss function. The model parameters are adjusted according to the calculated total loss. The preset loss function includes contrastive learning loss. The performance of the cell detection model was evaluated using the test set.

12. The method according to claim 11, characterized in that, The contrastive learning loss is calculated using the following method: The second fused feature map is projected onto a low-dimensional feature embedding space to obtain feature embedding vectors after projection of multiple prior boxes. Construct contrastive learning samples, which include positive sample pairs and negative sample pairs. The positive sample pairs include feature embedding vectors after projecting cell prior boxes of the same class, and the negative sample pairs include feature embedding vectors after projecting cell prior boxes of different classes. The contrastive learning loss is calculated based on the constructed contrastive learning samples.

13. The method according to claim 12, characterized in that, The preset loss function is a weighted sum of classification loss, regression loss, and contrastive learning loss, wherein the classification loss is cross-entropy loss, and the regression loss is smoothed average absolute error loss.

14. An electronic device, characterized in that, The device includes a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform the steps of the cell detection method as claimed in any one of claims 1 to 9, or to perform the steps of the cell detection model training method as claimed in any one of claims 10 to 13, based on the instructions stored in the memory.

15. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the cell detection method as described in any one of claims 1 to 9, or the training method for the cell detection model as described in any one of claims 10 to 13.

16. A computer program product, characterized in that, The instructions include, when the computer program product is executed by a computer, performing the cell detection method as described in any one of claims 1 to 9, or the training method for the cell detection model as described in any one of claims 10 to 13.

Images