Real-time sperm detection method and system based on hybrid all-aware enhanced network

Abstract

This invention discloses a real-time sperm detection method based on a hybrid full-sensory enhancement network, comprising the following steps: S1: acquiring sperm images under a microscope and preprocessing them; S2: integrating parallel Mamba aggregation blocks (PMAB) through a Mamba-guided ShuffleNet backbone network (MGS-Net) to simultaneously capture local microscopic features and global morphological context; S3: inputting three scale features from the backbone network into the neck network, which utilizes a hybrid state-space path aggregation network (HSS-PAN) to achieve deep feature fusion and noise suppression; S4: using a detection head to output classification confidence and bounding box parameters to achieve real-time sperm detection. A real-time sperm detection system based on a hybrid full-sensory enhancement network is also disclosed. This invention can be widely applied to real-time sperm screening in clinical assisted reproductive technologies.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical image processing technology, and in particular to a real-time sperm detection method and system based on a hybrid full-sensory enhancement network. Background Technology

[0002] Traditional computer-aided sperm analysis (CASA) systems aim to provide automated quantitative analysis, but their core, based on hand-designed feature extractors, is extremely sensitive to image noise, lighting variations, and sample impurities. In complex environments, they often produce numerous false positives and false negatives, failing to meet the clinical need for precise individual sperm localization. In recent years, deep learning-based target detection technologies have shown great potential in medical imaging. However, sperm detection faces unique challenges: sperm head morphology is often highly similar to cell debris and impurities in the sample, making differentiation difficult based on local morphology alone. The key identification criterion lies in the global structural relationship between the sperm head and tail. Therefore, an ideal detection model must simultaneously capture local detail texture and model long-range spatial dependencies. Mainstream convolutional neural networks (CNNs) are limited by local receptive fields, making it difficult to effectively model global continuity. While the Transformer architecture can capture global information through self-attention mechanisms, its computational complexity increases quadratically with image resolution, making real-time processing difficult on commonly used clinical equipment. Furthermore, its indiscriminate attention mechanism may not effectively focus on key regions.

[0003] Therefore, existing technologies suffer from a contradiction between insufficient global perception capabilities and low computational efficiency, making it difficult to meet the demands of real-time clinical processing while ensuring high accuracy. Clinically, a novel detection paradigm is needed that integrates efficient local feature extraction, selective global context modeling, and linear computational complexity to achieve accurate and real-time sperm detection in complex clinical scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a real-time sperm detection method and system based on a hybrid full-sensory enhancement network, so as to solve the problem that existing sperm detection methods cannot simultaneously achieve high-precision global perception and high computational efficiency, and meet the urgent need for automated and real-time sperm screening in assisted reproductive clinical practice.

[0005] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: to provide a real-time sperm detection method based on a hybrid full-sensory enhancement network, comprising the following steps:

[0006] S1: Acquire sperm images under a microscope and perform preprocessing;

[0007] S2: The preprocessed sperm image is processed by the Mamba-guided ShuffleNet backbone network MGS-Net, and local feature preservation and global context modeling are achieved through the parallel Mamba aggregation block PMAB in MGS-Net;

[0008] S3: Input the multi-scale features extracted by the backbone network MGS-Net into the hybrid state-space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the Ghost Mamba module GMB to enhance long-range dependency modeling through the state-space model.

[0009] S4: Finally, the detection head outputs the classification confidence and bounding box parameters of the sperm target, and the detection accuracy is jointly optimized through a multi-task loss function.

[0010] In a preferred embodiment of the present invention, the MGS-Net backbone network adopts a four-stage hierarchical design, wherein the first stage includes convolutional layers and max pooling layers for preliminary feature extraction; the second, third and fourth stages are core feature enhancement stages, and each stage is composed of multiple parallel Mamba aggregation blocks (PMABs) stacked together.

[0011] Furthermore, the parallel Mamba aggregation block PMAB adopts a parallel dual-branch architecture:

[0012] The first branch is the detail-preserving branch, which sequentially uses 3×3 depthwise convolution, batch normalization, 1×1 pointwise convolution, batch normalization, and ReLU activation function to preserve the microstructural information of the sperm head contour.

[0013] The second branch is the context modeling branch. First, the input features are split into two subspaces along the channel dimension, and then a lightweight adaptive state space block ASSB-S is input into each subspace for processing. Then, the two processed outputs are concatenated by channel, and finally fused by 1×1 pointwise convolution, batch normalization and ReLU activation function.

[0014] The outputs of the detail-preserving branch and the context-modeling branch are fused through channel concatenation to form the final output features of PMAB.

[0015] In a preferred embodiment of the present invention, the lightweight adaptive state space block ASSB-S is composed of an input projection layer, a 2D selective scanning module, and an output projection layer connected in sequence, wherein the input projection layer and the output projection layer are 1×1 standard convolutional layers; the ASSB-S achieves efficient global spatial dependency modeling through the 2D selective scanning module, under the premise of removing local feature calibration and dynamic fusion mechanisms.

[0016] In a preferred embodiment of the present invention, the Hybrid State Space Path Aggregation Network (HSS-PAN) receives the multi-scale feature map output by the MGS-Net backbone network, constructs a feature pyramid containing top-down and bottom-up paths, and integrates the Ghost Mamba module (GMB) in the path for feature enhancement and cross-scale information interaction.

[0017] Furthermore, the Ghost Mamba module GMB processes input features through multiple parallel paths to achieve enhancement, specifically including the following steps:

[0018] First, in the main path, the input features pass through the first Ghost convolutional module, the channel attention SE module, and the second Ghost convolutional module to obtain deep abstract features. At the same time, a local detail compensation path applies a depthwise separable convolutional operation to the input features to preserve spatial detail information. The features output from the above two paths are concatenated in the channel dimension and then input into the full-state adaptive state space block ASSB-L in the global context modeling path to capture long-range dependencies between features.

[0019] Secondly, an identity residual path is set up, which performs 1×1 standard convolution, batch normalization and ReLU activation function operation on the input features to obtain a feature map aligned with the input dimension, which is used to preserve the original feature information and ensure smooth gradient propagation.

[0020] Finally, the features output by the global context modeling path and the features output by the identity residual path are added element by element to fuse global semantics and local details, forming the final output features of the Ghost Mamba module.

[0021] In a preferred embodiment of the present invention, the fully adaptive state space block ASSB-L is a fully functional sequence modeling unit. Its processing flow includes the following steps: the input features are first mapped to dimensions through an input projection layer; then they enter a local calibration block, where the local consistency of the features is enhanced through depthwise convolution and residual connections; the locally calibrated features then enter a global dependency modeling stage, where the features are transformed through 1×1 convolution and depthwise convolution, and after rotational position encoding, they are sent to a 2D selective scanning module to capture contextual information across the entire space; finally, the globally modeled features and the locally calibrated features are adaptively weighted and fused through a dynamic fusion module to form the final output of ASSB-L.

[0022] Furthermore, the 2D selective scanning module scans the feature map along four independent paths: horizontal forward, horizontal backward, vertical forward, and vertical backward. It also uses a discretized selective state space model to process the sequence of each path, capturing contextual information across the entire space with linear complexity.

[0023] In a preferred embodiment of the present invention, the detection head adopts an anchorless design and forms a feature enhancement layer by stacking multiple depth-separable convolutional layers, and finally outputs the sperm target classification confidence score and bounding box coordinate parameters for each spatial location.

[0024] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is: to provide a real-time sperm detection system based on a hybrid full-sensory enhancement network, comprising:

[0025] The image preprocessing module is used to acquire and preprocess sperm images under a microscope;

[0026] The multi-scale feature extraction module is used to process the sperm image preprocessed by the image preprocessing module through the Mamba-guided ShuffleNet backbone network MGS-Net, and to achieve local feature preservation and global context modeling through the parallel Mamba aggregation block PMAB in MGS-Net.

[0027] The feature fusion module is used to input the multi-scale features extracted by the multi-scale feature extraction module into the hybrid state space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the GhostMamba module GMB, which enhances long-range dependency modeling through the state space model.

[0028] The results parsing and output module is used to output the classification confidence and bounding box parameters of sperm targets using the detection head, and to jointly optimize the detection accuracy through a multi-task loss function.

[0029] The beneficial effects of this invention are:

[0030] This invention provides a real-time sperm detection method based on the Mamba-guided Hybrid Full-Perception Enhancement Network (HOPE-Net). This method, through innovative network architecture design, deeply integrates efficient convolutional operations with a state-space model with linear complexity, achieving comprehensive perception of local details and global context of sperm targets.

[0031] (1) This invention is the first to introduce a state-space model into sperm detection tasks. Through a parallel hybrid design of convolutional paths and selected state-space model (SSM) paths, HOPE-Net achieves global modeling capabilities comparable to Transformer without sacrificing the efficiency of local convolutional feature extraction, while maintaining linear growth in computational complexity. This enables the model to achieve high detection accuracy on multiple challenging datasets covering complex impurities, high noise, dense overlap, and multiple sperm types. On the private dataset HSDD, it achieved 98.9% mAP. 50The accuracy achieved mAP of 58.4%, 97.7%, and 90.7% on the public datasets SDTB, SVIA, and VISEM, respectively. 50 Accuracy. Compared to the current dedicated sperm detection network SpermDet, HOPE-Net achieves higher mAP on the four datasets mentioned above. 50 Accuracy improvements of 3.8%, 4.9%, 1.2%, and 8.1% were achieved, fully validating the significant advantages of the proposed method in generalization and robustness. Furthermore, HOPE-Net contains only 1.9M trainable parameters and requires 7.9G FLOPs of computation, compared to SpermDet (17.5M parameters, 50.3G FLOPs), representing an 89.1% reduction in parameters and an 84.3% reduction in computation. Thanks to its lightweight design and the linear computational complexity of the state-space model, this invention achieves a high frame rate of 48.6 FPS for real-time inference on standard GPU hardware (NVIDIA RTX 3090), meeting clinical needs.

[0032] (2) The architecture of this invention directly addresses the core pain points in sperm detection, such as interference from impurities, dense overlap of targets, and morphological variations. Through the parallel Mamba aggregation block in MGS-Net, the model can effectively distinguish morphologically similar sperm heads from impurities, reducing false positives. Through the Ghost Mamba module in HSS-PAN and its integrated all-state adaptive state space block (ASSB-L), the model can accurately separate mutually occluded sperm in high-density samples, reducing missed detections. The model of this invention retains a stronger target response in deep networks and effectively suppresses background noise.

[0033] (3) Comprehensive experiments on public and private datasets from multiple sources and under different imaging conditions show that HOPE-Net has stable performance and strong generalization ability. Its lightweight characteristics make it easy to deploy on routine clinical workstations, significantly reducing the application threshold of automated sperm analysis systems. Attached Figure Description

[0034] Figure 1 This is the overall architecture diagram of the real-time sperm detection method based on a hybrid full-sensory enhancement network of the present invention;

[0035] Figure 2 This is a structural block diagram of the parallel Mamba aggregation block PMAB;

[0036] Figure 3 This is a structural block diagram of the Ghost Mamba module GMB;

[0037] Figure 4 This is a schematic diagram of the structure of the lightweight adaptive state space block and the full-state adaptive state space block;

[0038] Figure 5 This is a schematic diagram of the structure of the 2D selective scanning module; Figure 6 This is a comparison chart of the detection performance of the method described in this invention and existing methods on four datasets;

[0039] Figure 7 This is a visual comparison of the feature maps of the traditional ShuffleNet and the MGS-Net described in this invention at the deep layers (stage 3) of the backbone network;

[0040] Figure 8 This is a visual comparison of the feature maps of the traditional GhostPAN neck network and the HSS-PAN described in this invention in different paths of the feature pyramid;

[0041] Figure 9 This is a block diagram of the real-time sperm detection system based on a hybrid full-sensory enhancement network. Detailed Implementation

[0042] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0043] Please see Figure 1 The embodiments of the present invention include:

[0044] A real-time sperm detection method based on a hybrid full-sensory enhancement network includes the following steps:

[0045] S1: Acquire sperm images under a microscope and perform preprocessing;

[0046] Preprocessing includes normalizing the images and scaling all input images to a fixed resolution (640×640 pixels) for network processing.

[0047] S2: The preprocessed sperm image is processed by the Mamba-guided ShuffleNet backbone network (MGS-Net), and local feature preservation and global context modeling are achieved through the parallel Mamba aggregation block PMAB in MGS-Net;

[0048] The ShuffleNet network, guided by Mamba, is used to extract multi-level, robust feature representations from preprocessed images. As a feature extractor, it employs a hierarchical design. It consists of four stages: Stage 1 is a basic initialization module, composed of a standard convolutional layer and a max-pooling layer, used for fast downsampling and initial capture of edge and texture features; Stages 2-4 are the core of the network, each employing parallel Mamba aggregation blocks (PMAB) to extract multi-level feature representations with rich semantic information.

[0049] The parallel Mamba aggregation block, serving as the core unit of the detection model's backbone network, is specifically optimized for the refined feature extraction of sperm cells, and its structure is as follows: Figure 2 As shown, the PMAB module employs a parallel dual-branch architecture. One branch utilizes depthwise separable convolution for efficient downsampling, focusing on preserving local microstructural details such as the sperm head contour. The other branch utilizes lightweight adaptive state space blocks (ASSB-S) to capture the complete morphology of individual sperm and the global spatial context relationships between sperm through a selective scanning mechanism, achieving linear complexity. The features from both branches are ultimately fused, outputting a feature map that combines local detail with global semantics.

[0050] In the detail-preserving branch, depthwise separable convolutions are used to achieve efficient spatial downsampling while maximizing the preservation of microstructural information. This branch first processes the input features ( ) through 3×3 depth convolution ( This achieves spatial dimension compression, followed by 1×1 pointwise convolution ( Adjusting the channel dimension can be represented as:

[0051] ,

[0052] in For ReLU, This is BatchNorm. This design is optimized for feature extraction of the microstructure of the sperm head contour, ensuring that critical details are not blurred due to overpooling during spatial downsampling.

[0053] In the context modeling branch, a state-space model is used to capture the spatial relationships between sperm and the complete morphological context of each sperm, which is crucial for distinguishing sperm that are occluding each other. This branch first divides the input features into two subspaces along the channel dimension. , Each subspace is processed through an independent lightweight adaptive state space block (ASSB-S). ASSB-S is built upon a structured state space sequence model, its core being the modeling of long-range dependencies with linear complexity. This parallel design significantly reduces computational overhead while maintaining the global receptive field, effectively handling densely distributed sperm. Finally, the two processed features are reconcatenated along the channel dimension and processed through a 1×1 pointwise convolution (…). Feature integration is accomplished through projection, batch normalization, and the ReLU activation function. The complete mathematical expression of this process is:

[0054] ,

[0055] in For ReLU, For BatchNorm, This indicates a channel-level concatenation operation. The outputs of the two branches are fused through channel concatenation to form the final feature representation. Through a targeted dual-branch parallel design, PMAB achieves the optimal balance between local detail preservation and global context modeling in sperm detection tasks, significantly improving detection performance in complex scenarios.

[0056] S3: Input the multi-scale features extracted by the backbone network MGS-Net into the hybrid state-space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the Ghost Mamba module GMB to enhance long-range dependency modeling through the state-space model.

[0057] The Hybrid State Space Path Aggregation Network (HSS-PAN), acting as the neck network, receives feature maps at three scales from the outputs of MGS-Net in stages 2-4. It constructs an enhanced feature pyramid containing both top-down and bottom-up propagation paths to better detect sperm of different sizes and in complex environments. HSS-PAN incorporates standard top-down and bottom-up feature propagation paths, achieving cross-scale information fusion. Its core innovation lies in introducing the GhostMamba module (GMB) for feature enhancement at the feature fusion node. GMB combines lightweight Ghost convolutions, channel attention (SE), and all-state adaptive state space blocks (ASSB-L) to enhance the discriminative power of multi-scale features, particularly excelling at separating individual sperm in dense, overlapping scenes and suppressing complex background noise.

[0058] The Ghost Mamba module, as the core feature processing unit of HSS-PAN, can further enhance the multi-scale fused features with extremely low computational cost. Its structure is as follows: Figure 3 As shown, the GMB backbone employs dual GhostModules and SE attention to progressively extract and focus key features of the sperm head. The first GhostModule is responsible for processing the raw input... Preliminary transformations and channel expansion are performed, employing a divide-and-conquer strategy to balance computational efficiency and feature representation capability. This stage first uses a 1×1 convolutional path (…). Generate basic features Then, through a 3×3 depthwise convolution ( Channel expansion is performed to generate supplementary features. :

[0059] ,

[0060] ,

[0061] in It is ReLU. The first stage output is obtained after concatenation. This design ensures the full expression of subtle sperm features while reducing computational complexity through a decomposition strategy. Subsequently, the SE attention module receives... Enhanced focusing ability on sperm head features through channel attention mechanism. The SE (Search Engine) first uses global average pooling to compress the spatial dimensionality, then generates channel weights to weight the features. The second GhostModule is responsible for further refining and adjusting the channels of the attention-enhanced features. This stage uses the same GhostModule formula, with the input being... The output is Building upon the feature processing of the dual GhostModule and SE modules, the GMB's local detail compensation path introduces Depthwise Separable Convolution (DSC) as a local feature enhancement mechanism. This path first uses a 3×3 depthwise convolution (DWConv) on the original input. Spatial features are extracted, and then the channel dimensions are adjusted using 1×1 pointwise convolutions.

[0062] .

[0063] The local detail compensation path forms a residual connection with the main path, which is specifically used to optimize the detail loss problem during feature transfer in sperm head detection.

[0064] To address the challenges of dense sperm distribution and mutual occlusion during sperm detection, GMB integrates the All-State Adaptive State Space Block (ASSB-L) to capture the global contextual relationships between sperm heads. ASSB-L unfolds the feature map into a sequence along the spatial dimension and utilizes a recursive mechanism to capture long-range dependencies. This module can efficiently model the spatial arrangement patterns of sperm in dense regions while maintaining linear computational complexity, significantly improving robustness to the detection of abnormal or overlapping sperm. Furthermore, GMB introduces a residual enhancement path, with the original input... Dimension adjustment is performed after a standard convolution (Conv):

[0065] .

[0066] This path preserves subtle morphological information that might be filtered out by the main path, and adds it element-wise to the features processed by ASSB-L to obtain the final output. The residual connection and the final output can be represented as:

[0067] .

[0068] The GMB module constructs an efficient feature enhancement unit that balances local detail awareness with global context understanding, enabling accurate detection of dense and occluded sperm targets.

[0069] The ASSB-S and ASSB-L described in steps S2 and S3 improve the accuracy and efficiency of feature representation through the synergistic optimization of local feature calibration and global dependency modeling, with the structure as follows: Figure 4 As shown.

[0070] The ASSB-L first uses a 1×1 convolution to map the input feature z to a preset hidden dimension.

[0071] This achieves channel dimension unification and feature nonlinear transformation. The Local Calibration Module (LCB) performs local calibration on the projected features, constructing local feature interactions through a combination of depthwise convolution and residual connections.

[0072] ,

[0073] in The SiLU activation function is used. The residual scaling factor is learnable, which adaptively adjusts the contribution weights of local features to enhance their local discriminative power. For global dependency modeling, ASSB-L designs a complete state-space model branch. First, it further enhances the correlation of local features through 1×1 convolutions and 3×3 depthwise convolutions, then introduces a Rotation Position Encoding (RoPE) module to dynamically generate position codes.

[0074] ,

[0075] in, Layer Normalization is applied. RoPE achieves precise embedding of positional information through complex domain rotation, adapting to feature maps of different resolutions. Subsequently, the feature input 2D Selective Scan (SS2D) module performs global state space modeling, employing a selective scanning mechanism to capture long-range dependencies. Combined with... Figure 5 The SS2D structure consists of selective scanning branches and gated branches. The main branch passes through a linear projection layer composed of 1×1 convolutions ( ) and 3×3 depthwise convolution ( Perform feature transformation and local enhancement: Then, it enters the core multidimensional selective scanning module. This module performs feature propagation in the spatial dimension through four independent scanning paths: horizontal forward scan, horizontal reverse scan, vertical forward scan, and vertical reverse scan. Each scanning path is processed by an S6 module. The S6 module is a discretized selective state-space system, and for each scanning path... The process of the S6 module processing sequence data can be described as follows:

[0076] ,

[0077] ,

[0078] Among them, for the first One scan path, Indicates at time step The hidden state vector, It is the current input vector. It is the current output vector. It is the output projection matrix. These are learnable skip connection parameters. Parameter matrix. and It is obtained by discretizing the continuous-time parameters:

[0079] ,

[0080] ,

[0081] in, The identity matrix. State matrix. Stability is ensured through logarithmic parameterization:

[0082] ,

[0083] in, These are learnable parameters. The selectivity of the S6 module is reflected in its ability to dynamically generate parameters based on the input. Specifically, the discretization step size... Input matrix and output matrix They are all obtained from input features:

[0084] ,

[0085] ,

[0086] in, It is the first Projection weights of each scan path, It is the time projection weight. The reference parameter representing the discretization step size, It is the time projection bias term. The function formula is The scan outputs are fused by summation and then stabilized by layer normalization.

[0087] .

[0088] The output of the selective scan branch is multiplied element-wise by the gated branch, and finally the feature dimensions are restored through the output projection layer composed of 1×1 convolutions.

[0089] ,

[0090] in, It involves element-wise multiplication. SS2D's multidimensional scanning mechanism ensures comprehensive capture of spatial context, and the state-space model provides content-aware dynamic parameter adjustment, achieving effective modeling of long-range dependencies while maintaining computational efficiency. Local enhancement mechanisms further refine output features through DW and residual connections.

[0091] ,

[0092] .

[0093] Finally, the Dynamic Fusion Module (DFB) achieves dynamic feature fusion through a gating mechanism:

[0094] ,

[0095] in, , and It is a 1×1 convolution.

[0096] The ASSB-S module, as the core feature extraction unit of the backbone network, focuses on efficiently extracting basic visual features, such as... Figure 4 As shown, ASSB-S optimizes computational efficiency by simplifying the structure of ASSB-L, removing LCB, RoPE, and DFB. Meanwhile, ASSB-S retains projective convolution, SS2D modules, and residual connection mechanisms, ensuring that the core capabilities of local calibration and global dependency modeling remain unaffected. By optimizing the module structure and combining it with ASSB-S's MGS-Net, a balance between efficiency and accuracy in feature extraction is achieved, providing high-quality foundational features for subsequent multi-scale aggregation.

[0097] S4: Finally, the detection head outputs the classification confidence and bounding box parameters of the sperm target, and the detection accuracy is jointly optimized through a multi-task loss function.

[0098] HSS-PAN outputs four enhanced multi-scale feature maps. The detection head employs an anchorless design, with each scale feature map followed by a lightweight prediction branch. This branch consists of two stacked depthwise separable convolutional layers, which are then passed through two independent 1×1 convolutional layers to predict the confidence score of the sperm class corresponding to each feature point and the distance of that point from the top, bottom, left, and right boundaries of the target bounding box. The confidence score and bounding box parameters output by the detection head are parsed, and post-processing steps such as non-maximum suppression are applied to obtain the final sperm detection box, which is then overlaid on the original input image to complete real-time detection.

[0099] See Figure 9 This invention also provides a real-time sperm detection system based on a hybrid full-sensory enhancement network, including an image preprocessing module, an MGS-Net backbone network processing (multi-scale feature extraction) module, a hybrid state-space path aggregation network HSS-PAN (feature fusion) module, and a result parsing and output module.

[0100] The image preprocessing module is used to acquire sperm images under a microscope and perform preprocessing.

[0101] The multi-scale feature extraction module is used to process the sperm image preprocessed by the image preprocessing module through the Mamba-guided ShuffleNet backbone network MGS-Net, and to achieve local feature preservation and global context modeling through the parallel Mamba aggregation block PMAB in MGS-Net.

[0102] The feature fusion module is used to input the multi-scale features extracted by the multi-scale feature extraction module into the hybrid state space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the GhostMamba module GMB to enhance long-range dependency modeling through the state space model.

[0103] The result parsing and output module is used to output the classification confidence and bounding box parameters of sperm targets using the detection head, and to jointly optimize the detection accuracy through a multi-task loss function.

[0104] This example illustrates a real-time sperm detection system based on a hybrid full-sensory enhancement network, which can execute a real-time sperm detection method based on a hybrid full-sensory enhancement network provided by this invention. It can perform any combination of the steps in the method example and possesses the corresponding functions and beneficial effects of the method.

[0105] The beneficial effects of the method and system described in this invention will be illustrated by a specific example:

[0106] Step 1: Data Acquisition and Preprocessing

[0107] Image Acquisition: Public datasets (SVIA, SDTB, and VISEM) were used with their provided images and formats. The internal dataset HSDD was imaged under bright-field conditions using an inverted microscope (Olympus, IX73) and a CMOS camera with an objective magnification of 20×. Sperm images were at a resolution of 1024×1024 pixels, divided into 1179 training images and 242 test images. Precise annotation of the datasets was performed manually according to clinical guidelines.

[0108] Labeling: Use the LabelMe tool to label sperm heads (detection task) and generate JSON format label files.

[0109] Step 2: Training and Deployment of the Detection Module

[0110] 1. The backbone network uses the aforementioned MGS-Net, and the neck network uses the aforementioned HSS-PAN network. The input size is 640×640, and the output has four scale detection heads (10×10, 20×20, 40×40, 80×80).

[0111] 2. Loss Function: Classification and regression performance is improved through joint optimization of multi-task losses. The loss function consists of the quality focus loss (…). ), distribution focus loss ( ) and GIOU loss ( )constitute. For classification tasks, it learns both classification confidence and localization quality, using the following formula:

[0112] ,

[0113] in, It is the target IOU score. It is the predicted classification score. It is a regulatory factor. It is the total number of samples. This indicates the Sigmod activation function. Used for bounding box regression, it transforms a continuous regression problem into discrete probability distribution learning:

[0114] ,

[0115] in, These are the actual bounding box coordinates. and These are adjacent discretized coordinate values. and It is the predicted probability distribution. Used for bounding box regression, it solves the gradient vanishing problem of IOU Loss in non-overlapping regions:

[0116] ,

[0117] in, and These are the predicted bounding box and the ground truth bounding box. It includes and The minimum closure rectangle, It's an intersection-union ratio. The total loss is the weighted sum of the losses from the three parts:

[0118] ,

[0119] in, , and These are the weight parameters of the three loss functions, set to 1.0, 0.25, and 2.0 respectively.

[0120] Step 3: Training Parameter Settings

[0121] All networks were implemented in PyTorch and trained on an NVIDIA RTX 3090 using the AdamW optimizer. To reduce computational cost and improve model efficiency, all images were uniformly scaled to 640×640 pixels using bilinear interpolation and normalized to the range [0, 1]. The initial learning rate was 1×10⁻⁶. -3 The weight decays to 5×10 -2 We apply a cosine annealing strategy, with a minimum learning rate set to 1×10⁻⁶. -5 All networks were trained for 200 epochs on four datasets with a batch size of 16.

[0122] Step 4: Definition of Evaluation Indicators

[0123] With IOU thresholds of 0.5 and 0.5–0.95, the calculated mean accuracy (mAP) is used as the primary accuracy metric. AP is derived by calculating the area under the accuracy-range curve at different IOU thresholds, as shown in the following formula:

[0124] ,

[0125] in and These represent precision and recall, respectively. This represents the set of Intersection over Union (IoU) thresholds. A true positive (TP) is defined as a bounding box that is correctly detected and has an IoU higher than the specified threshold. False positives (FP) are detection boxes with an IoU lower than the threshold or duplicate detection boxes, while false negatives (FN) are real objects that are not detected. Floating-point operations per second (FLOPS), frames per second (FPS) using fp32 precision, and the weight parameter size are used to comprehensively measure the model size and inference efficiency.

[0126] The HOPE-Net model was qualitatively compared with a series of advanced detection models on the HSDD, SDTB, SVIA, and VISEM datasets using visualization methods. Figure 6 As shown in the figure. The results demonstrate that the proposed method exhibits significant advantages in addressing the specific challenges of different datasets. In the HSDD dataset, artifact noise is generally bright and visually similar to sperm heads. SpermDet and YOLOv8m misclassify artifacts as sperm targets on HSDD, resulting in false positives. The SDTB dataset contains complex biological backgrounds and a large number of non-sperm cell impurities, posing a severe challenge to the detection task in terms of discriminative power. RT-DETR-X and Libra-RCNN exhibit false negatives on SDTB, while YOLOv8m and EfficientDet overdetect in the face of noise, misidentifying a large number of circular impurities as sperm. HOPE-Net, guided by Mamba, can efficiently model the global context, dynamically suppressing these interfering regions that are similar to local sperm features but have inconsistent global semantics, thus improving detection accuracy. On the SVIA and VISEM datasets, sperm are densely distributed with significant overlap and aggregation, making accurate individual separation difficult. Libra-RCNN and EfficientDet suffer from numerous false negatives, while SpermDet performs poorly in classifying detections and cannot effectively handle densely clustered sperm. HOPE-Net, however, scans the entire image using SSM (Sperm Matrix Synthesis), implicitly understanding the occlusion relationships between instances. Simultaneously, the rich, multi-level features extracted by the CNN ensure accurate boundary localization for each separated instance. This allows HOPE-Net to generate clear, well-separated bounding boxes on dense samples, maintaining the highest accuracy.

[0127] The HOPE-Net was subjected to extensive qualitative comparative experiments on the HSDD, SDTB, SVIA and VISEM datasets to verify its performance, as shown in Table 1. The comparative models can be divided into the following categories: (1) Single-stage object detectors: SSD, RetinaDet, EfficientDet, YOLOv5, YOLOv8, YOLOv9, YOLOv10, YOLOv11, YOLOv12 and their variants with different parameter scales; (2) Two-stage object detectors: Faster-RCNN, Cascade-RCNN, Grid-RCNN, Libra-RCNN, Dynamic-RCNN and Sparse-RCNN; (3) Other types of detectors: DiffusionDet based on the diffusion model, RT-DETR based on CNN and Vision Transformer and their variants with different parameter scales; (4) Sperm detectors: TOD-CNN, ACTIVE and SpermDet.

[0128] On the HSDD dataset, HOPE-Net demonstrates comprehensive superiority over existing methods, especially achieving state-of-the-art (SOTA) performance in balancing accuracy and efficiency. Its mAP... 50 Reaching 98.9%, mAP 50-95 The accuracy rate was 48.2%, representing improvements of 3.8% and 2.3% respectively compared to the best sperm detection model, SpermDet. HOPE-Net also demonstrated significant performance advantages over existing state-of-the-art object detection models, achieving a high mAP (molecular accuracy rate). 50 In terms of performance, HOPE-Net outperforms YOLOv12m by 1.3%, Dynamic-RCNN by 2.1%, and RT-DETR-X by 1.1%. In terms of efficiency, HOPE-Net requires only 1.9M parameters and 7.9G FLOPS, representing reductions of 89.1% and 84.3% respectively compared to SpermDet, while achieving an inference speed of 48.6 FPS, approaching the real-time performance of the YOLO series. This advantage stems from its core backbone, MGS-Net, which uses the Mamba selective scanning mechanism to dynamically plan feature extraction paths to focus on key sperm morphologies, while retaining the lightweight convolutional skeleton of ShuffleNet, achieving a synergistic improvement in both accuracy and efficiency.

[0129] On the SDTB dataset, which presents significant challenges due to its complex backgrounds and tiny sperm, HOPE-Net demonstrates exceptional robustness and accuracy. Its mAP... 50 It reached 58.4%, a 4.9% improvement over SpermDet, which ranked second. mAP 50-95The accuracy reached 26.5%, a 4.2% improvement over Libra-RCNN, which ranked second. Besides accuracy, HOPE-Net also leads in efficiency: compared to Libra-RCNN, it reduces the number of parameters by 95.4%, FLOPS by 91.5%, and inference speed by 5.9 FPS. This performance advantage stems from HOPE-Net's ability to dynamically filter complex background noise through Mamba's selective scanning mechanism and effectively aggregate deep and shallow features through HSS-PAN to cope with background interference and target size variations, enabling the model to accurately locate sperm targets even in cluttered scenes. Even compared to the mAP on the SDTB dataset... 50-95 Compared to the relatively high YOLOv11m, HOPE-Net still improves this metric by 5.6%, while reducing the number of parameters by 90.5% and increasing inference speed by 2.2 FPS, fully demonstrating its suitability for challenging clinical testicular biopsy scenarios.

[0130] On the SVIA dataset, where dense sperm overlap is the main challenge, HOPE-Net maintains a leading position in both accuracy and efficiency. Its mAP... 50 The accuracy reached 97.7%, a 0.9% improvement over the suboptimal model YOLOv8m. mAP 50-95 The accuracy reached 70.2%, a 1.7% improvement over the second-best model, SpermDet. It's worth noting that while HOPE-Net has slightly higher computational complexity compared to EfficientDet, which is characterized by its lightweight design, it exhibits overwhelming advantages in parameter count, inference speed, and detection accuracy. Specifically, HOPE-Net has 1.9M parameters, a 50% reduction compared to EfficientDet's 3.8M, making it more suitable for deployment in resource-constrained clinical instruments; its inference speed reaches 48.6 FPS, a 21.9 FPS improvement over EfficientDet, meeting real-time clinical requirements; and in terms of accuracy, HOPE-Net's mAP is... 50 and mAP 50-95 Compared to EfficientDet, the performance is improved by 38.4% and 44.8% respectively, achieving accurate differentiation and localization of overlapping sperm. This demonstrates that HOPE-Net's PMAB can model the long-range spatial dependence of overlapping sperm through state-space paths and capture local contour details by combining convolutional paths, effectively distinguishing individual instances in dense field of view.

[0131] On the multi-class VISEM dataset, HOPE-Net demonstrates strong adaptability to sperm with different morphologies. In terms of accuracy, HOPE-Net achieves a high mAP. 50-95 The mAP reached 68.3%, a 3.8% improvement over YOLOv10m, which ranked second. 50The accuracy reached 90.7%, a 3.9% improvement over RetinaDet. Among all detection models, TOD-CNN had the fastest inference speed, but it still lagged behind HOPE-Net in balancing accuracy and computational complexity. Specifically, HOPE-Net's mAP... 50 and mAP 50-95 Compared to TOD-CNN, the performance improvements are 43.2% and 38.5% respectively, effectively solving the problems of missed detection and misclassification of sperm clusters and microcephaly in TOD-CNN. Meanwhile, the number of parameters in HOPE-Net is reduced from 32.8M to 1.9M, a reduction of 94.2%, and FLOPS is reduced from 65.2G to 7.9G, a reduction of 87.9%. Even with a slower inference speed, the 48.6 FPS still meets the needs of real-time clinical sperm detection.

[0132] Therefore, the comparison results between the present invention and the latest deep learning real-time inference methods are shown in Table 1.

[0133] Table 1. Comparison with current deep learning real-time detection methods

[0134] The significant performance improvement of HOPE-Net mainly stems from two aspects: 1) The state-space model can model long-range dependencies while maintaining linear complexity, enhancing the model's understanding of global morphology and spatial layout; 2) The state-space module does not simply replace convolution, but is deeply coupled with the lightweight convolutional structure, improving discriminative ability without sacrificing local information such as edges and textures. Regarding the backbone network, while the ShuffleNet lightweight convolutional backbone has lower computational overhead, it suffers from limited local receptive fields and insufficient global semantic integration. For example... Figure 7 As shown in the feature maps, ShuffleNet lost response to a large number of sperm targets in deep stage 3, failing to integrate local and global feature information. MGS-Net, in the feature maps output in stage 3, still maintained clear and high-confidence responses to most sperm targets. This indicates that through the ASSB-S multi-path selective scanning mechanism and global context modeling, it can effectively preserve the features of sperm targets in the deep semantic feature extraction stage, avoiding information decay. Regarding the neck structure, classic FPN, PAN, and their lightweight variants still have two prominent problems: 1) Multi-scale fusion mainly relies on local operations and layer-by-layer propagation, making it difficult to model long-distance cross-scale dependencies, leading to target overlap and missed detections in high-density, strongly occluded scenes; 2) Lack of explicit global noise suppression mechanisms, easily misclassifying cell debris, bubbles, and other impurities as targets in complex backgrounds. Figure 8As shown, GhostPAN exhibits blurred target region responses and severe noise interference in the feature maps of both bottom-up and top-down paths. In contrast, the HSS-PAN proposed in this invention demonstrates more concentrated sperm target confidence and lower responses in the background and impurity regions in the feature maps of both paths. This indicates that HSS-PAN can encode cross-scale local features and global semantics into the feature pyramid through redundant feature reuse of GhostModule and long-range dependency modeling of ASSB-L, thereby enhancing cross-scale consistency representation while maintaining linear complexity and effectively suppressing noise interference and mismatches.

[0135] The HOPE-Net framework proposed in this invention is the first to introduce a state-space model into sperm detection tasks, achieving breakthroughs in both accuracy and efficiency for complex clinical samples. By integrating convolutional operations and a state-space model in a hybrid architecture, HOPE-Net achieves a balance between local detail preservation and global contextual understanding. The proposed backbone network MGS-Net and neck network HSS-PAN synergistically enhance multi-scale feature representation and long-range dependency modeling, laying the foundation for accurate sperm localization in noisy, dense, and overlapping scenarios. Extensive experiments on four challenging datasets (an internal dataset HSDD and three public datasets SDTB, SVIA, and VISEM) demonstrate that HOPE-Net outperforms general-purpose detectors and specialized sperm detectors, achieving real-time sperm detection with the highest accuracy and lowest computational complexity. This work not only provides an efficient and reliable solution for sperm detection in complex clinical environments but also provides strong algorithmic support for the automation and precision of assisted reproductive technologies. The successful implementation of this framework further validates the potential and advantages of state-space models in the detection of dense, small biomedical targets, opening up new technical pathways for research in related fields. This work not only promotes the automation of sperm testing in clinical practice, but also provides embryologists with reliable and efficient tools in assisted reproductive procedures, thereby helping to improve the success rate of assisted reproductive technologies.

[0136] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A real-time sperm detection method based on a hybrid full-sensory enhancement network, characterized in that, Includes the following steps: S1: Acquire sperm images under a microscope and perform preprocessing; S2: The preprocessed sperm image is processed by the Mamba-guided ShuffleNet backbone network MGS-Net to obtain multi-scale features. Local feature preservation and global context modeling are achieved through the parallel Mamba aggregation block PMAB in MGS-Net. S3: Input the multi-scale features extracted by the backbone network MGS-Net into the hybrid state-space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the Ghost Mamba module GMB to enhance long-range dependency modeling through the state-space model. S4: Finally, the detection head outputs the classification confidence and bounding box parameters of the sperm target, and the detection accuracy is jointly optimized through a multi-task loss function.

2. The real-time sperm detection method based on a hybrid full-sensory enhancement network according to claim 1, characterized in that, The backbone network MGS-Net adopts a four-stage hierarchical design. The first stage includes convolutional layers and max pooling layers for preliminary feature extraction. The second, third, and fourth stages are core feature enhancement stages, and each stage consists of multiple parallel Mamba aggregation blocks (PMABs) stacked together.

3. The real-time sperm detection method based on a hybrid full-sensor enhancement network according to claim 2, characterized in that, The parallel Mamba aggregation block PMAB adopts a parallel dual-branch architecture: The first branch is the detail-preserving branch, which sequentially uses 3×3 depthwise convolution, batch normalization, 1×1 pointwise convolution, batch normalization, and ReLU activation function to preserve the microstructural information of the sperm head contour. The second branch is the context modeling branch. First, the input features are split into two subspaces along the channel dimension, and then a lightweight adaptive state space block ASSB-S is input into each subspace for processing. Then, the two processed outputs are concatenated by channel, and finally fused by 1×1 pointwise convolution, batch normalization and ReLU activation function. The outputs of the detail-preserving branch and the context-modeling branch are fused through channel concatenation to form the final output features of PMAB.

4. The real-time sperm detection method based on a hybrid full-sensory enhancement network according to claim 3, characterized in that, The lightweight adaptive state space block ASSB-S is composed of an input projection layer, a 2D selective scanning module, and an output projection layer connected sequentially, wherein the input projection layer and the output projection layer are 1×1 standard convolutional layers; the ASSB-S achieves efficient global spatial dependency modeling through the 2D selective scanning module, under the premise of removing local feature calibration and dynamic fusion mechanisms.

5. The real-time sperm detection method based on a hybrid full-sensor enhancement network according to claim 1, characterized in that, The Hybrid State Space Path Aggregation Network (HSS-PAN) receives multi-scale feature maps output by the backbone network (MGS-Net), constructs a feature pyramid containing top-down and bottom-up paths, and integrates the Ghost Mamba module (GMB) in the path for feature enhancement and cross-scale information interaction.

6. The real-time sperm detection method based on a hybrid full-sensory enhancement network according to claim 5, characterized in that, The Ghost Mamba module GMB processes input features through multiple parallel paths to achieve enhancement. The specific steps include: First, in the main path, the input features pass through the first Ghost convolutional module, the channel attention SE module, and the second Ghost convolutional module to obtain deep abstract features. At the same time, a local detail compensation path applies a depthwise separable convolutional operation to the input features to preserve spatial detail information. The features output from the above two paths are concatenated in the channel dimension and then input into the full-state adaptive state space block ASSB-L in the global context modeling path to capture long-range dependencies between features. Secondly, an identity residual path is set up, which performs 1×1 standard convolution, batch normalization and ReLU activation function operation on the input features to obtain a feature map aligned with the input dimension, which is used to preserve the original feature information and ensure smooth gradient propagation. Finally, the features output by the global context modeling path and the features output by the identity residual path are added element by element to fuse global semantics and local details, forming the final output features of the Ghost Mamba module.

7. The real-time sperm detection method based on a hybrid full-sensor enhancement network according to claim 6, characterized in that, The fully adaptive state space block ASSB-L is a complete sequence modeling unit. Its processing flow includes the following steps: the input features are first mapped to dimensions through the input projection layer; then they enter the local calibration block, where the local consistency of the features is enhanced through depthwise convolution and residual connections; the locally calibrated features then enter the global dependency modeling stage, where the features are transformed through 1×1 convolution and depthwise convolution, and after rotational position encoding, they are sent to the 2D selective scanning module to capture contextual information across the entire space; finally, the globally modeled features and the locally calibrated features are adaptively weighted and fused through the dynamic fusion module to form the final output of ASSB-L.

8. The real-time sperm detection method based on a hybrid full-sensory enhancement network according to claim 4 or 7, characterized in that, The 2D selective scanning module scans the feature map along four independent paths: horizontal forward, horizontal backward, vertical forward, and vertical backward. It then uses a discretized selective state space model to process the sequence of each path, capturing contextual information across the entire space with linear complexity.

9. The real-time sperm detection method based on a hybrid full-sensory enhancement network according to claim 1, characterized in that, The detection head adopts an anchorless design and forms a feature enhancement layer by stacking multiple depth-separable convolutional layers. Finally, it outputs the sperm target classification confidence score and bounding box coordinate parameters for each spatial location.

10. A real-time sperm detection system based on a hybrid full-sensory enhancement network, characterized in that, include: The image preprocessing module is used to acquire and preprocess sperm images under a microscope; The multi-scale feature extraction module is used to process the sperm image preprocessed by the image preprocessing module through the Mamba-guided ShuffleNet backbone network MGS-Net, and to achieve local feature preservation and global context modeling through the parallel Mamba aggregation block PMAB in MGS-Net. The feature fusion module is used to input the multi-scale features extracted by the multi-scale feature extraction module into the hybrid state-space path aggregation network HSS-PAN for deep feature fusion and noise suppression. The HSS-PAN integrates the Ghost Mamba module GMB, which enhances long-range dependency modeling through the state-space model. The results parsing and output module is used to output the classification confidence and bounding box parameters of sperm targets using the detection head, and to jointly optimize the detection accuracy through a multi-task loss function.

Images