Insect sperm detection tracking method and system based on fine-grained subspace perception

Abstract

The invention discloses an insect sperm detection and tracking method and system based on fine-grained subspace perception, and relates to the technical field of insect reproduction prevention and control. Bounding box labeling is performed on an insect sperm image, and a data set is established; training a preset sperm target detection model through the data set; combining a feature map output by a fine-grained subspace attention perception module in the model with an insect sperm image marked by a bounding box to obtain a tracking vector; inputting a preset sperm target tracking model by taking a target bounding box and a tracking vector output by a detection head in the model as inputs, calculating a mahalanobis distance and an appearance distance based on a prediction state and an observation vector, and constructing a comprehensive matching cost through the mahalanobis distance and the appearance distance, and calculating a global optimal matching scheme based on the comprehensive matching cost and a Hungary algorithm, performing trajectory state updating on a successfully matched target, and outputting a tracking result. According to the invention, detection and tracking of insect sperms are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of insect reproductive prevention and control, and more particularly to an insect sperm detection and tracking method and system based on fine-grained subspace perception. Background Art

[0002] Among the existing automatic sperm identification technologies, most of them are aimed at automatically calculating the motility and sperm density of the typical "tadpole-shaped" mammalian sperm, such as the CASA system. Due to its unique morphological structure, insect sperm is long and linear, with a very narrow head or even a head and tail of similar width, and the tail is much longer than the head. It is not "tadpole-shaped" and therefore cannot be identified by existing automatic sperm analysis systems. Only honeybees have developed the SCA CASA method based on observing sperm motility after fluorescent staining. However, this method requires that the sperm head release a strong fluorescent signal after staining, which cannot meet the requirements for some insect sperm heads. In addition, fluorescent staining of sperm in advance increases the complexity of experimental operations and cannot achieve the purpose of express detection. Special fluorescence microscopes and cameras are also required to collect signals, which is costly.

[0003] Locusts are a global agricultural pest. A single locust outbreak can cause massive farmland damage, and a single swarm can consume as much food as 35,000 people. Chemical pesticides are currently the primary means of controlling locust plagues, but the ecological and food security impacts they cause are becoming increasingly significant. Therefore, it is imperative to develop more effective biological control methods. The small brown planthopper (Lagerstroemia striatellum) is another important agricultural pest. It not only sucks rice sap through its mouthparts, causing rice plants to wither and turn yellow due to excessive sap intake, but more importantly, it can also transmit the rice stripe virus, causing diseased plants to frequently wither and stagnate, or to have small, deformed, and unsatisfactory ears. As a result, planthoppers have become a significant pest of rice. Mosquitoes are major vectors of many infectious diseases, particularly malaria, dengue fever, and Zika virus. Research on the reproductive capacity of these pests, particularly sperm detection and tracking, can provide important scientific evidence for population control and disease vector prevention. However, due to the small size, complex morphology, and unpredictable dynamic behavior of insect sperm, their detection and tracking remain a major challenge in scientific research.

[0004] Traditional methods for detecting and tracking insect sperm rely primarily on microscopic observation and manual analysis. These methods typically require highly specialized technicians, consume significant time and effort, are susceptible to subjective factors, and struggle to process large amounts of data. Furthermore, traditional methods are limited in their ability to accurately capture sperm trajectories and dynamic behavior, potentially missing important dynamic information and failing to meet the demands of high-precision, high-throughput research. Therefore, the development of more intelligent, efficient, and accurate detection and tracking methods is an urgent need.

[0005] In recent years, the rapid development of deep learning in computer vision has provided new insights into biological image analysis. Deep learning models have demonstrated exceptional performance in target detection and tracking tasks. These algorithms utilize convolutional neural networks to extract multi-layered features from images, enabling accurate target detection. While deep learning offers a new technical approach for insect sperm detection and tracking, its practical application still faces numerous challenges: insect sperm are small, narrow, and prone to overlapping; and factors such as illumination variations and background noise in the detection environment can interfere with the model's judgment, reducing detection robustness. Currently, no dedicated methods or devices for insect sperm detection and tracking exist.

[0006] Among existing similar technologies, Chinese invention patent publication (announcement) number CN114241006B discloses a sperm tracking and automatic extraction method based on machine learning. The purpose is to address the common problems of low monitoring accuracy and slow monitoring speed in current major sperm monitoring methods, which are not conducive to real-time tracking of multiple sperm targets. The main solution includes obtaining real-time image data of sperm samples; implementing multi-sperm tracking through YOLO-V4, KCF tracker, and Kalman filter; calculating the sperm velocity based on the tracking results and determining the target sperm based on the velocity; using YOLO-V4 to detect the position of the extraction needle in the image; constructing the observation state of the extraction needle controller based on the position of the extraction needle and the tracking position of the sperm; and inputting the observation state into the actor network in the DDPG deep reinforcement learning model to obtain the control quantity. The control quantity output by the actor network is input into the extraction needle controller and converted into a control signal executable by the controller. The controller executes the behavior corresponding to the control quantity according to the control signal.

[0007] For example, Chinese Invention Patent Publication (Announcement) No. CN112580476B discloses a method for sperm identification and multi-target trajectory tracking. The method comprises: selecting a target sperm in the first frame of a sperm image and obtaining its coordinates, using these coordinates as the starting coordinate point of the target sperm's trajectory chain; using a trajectory prediction algorithm to calculate the predicted coordinate point of the target sperm in the i-th frame of the sperm image; setting a search area in the i-th frame of the sperm image using the predicted coordinate point for the search; if multiple sperm are found within the area, a multi-principle matching algorithm is used to perform correlation matching to obtain a best-matching sperm, and the coordinate point of the best-matching sperm is added to the target sperm's trajectory chain; if no sperm is found within the area, the predicted coordinate point is added to the target sperm's trajectory chain. This scheme improves the tracking efficiency of sperm trajectory, adapts to scenarios where sperm appear and disappear briefly, and improves matching accuracy and reduces mismatch rates.

[0008] For example, Chinese invention patent publication (announcement) No. CN118883551B discloses a sperm automatic detection system and its control method, which relate to the field of cell detection technology. The control method of the sperm automatic detection system includes: obtaining large-field image data sent by an upright low-power microscope to confirm the movement characteristics of all sperm in the semen; completing the vitality analysis of all sperm based on the movement characteristics; controlling the electric stage to perform motion compensation so that the target sperm is in the target position of the imaging field of the inverted high-power microscope; obtaining the imaging information sent by the inverted high-power microscope to complete the morphology analysis operation of the target sperm; selecting the next target sperm according to the large-field image data to execute the previous step until all target sperm have completed the morphology analysis operation; controlling the electric stage to perform motion compensation so that the preferred sperm is in the target position of the imaging field of the inverted low-power microscope; the technical effect of this technical solution can ensure the activity of sperm while allowing the sperm to be stably detected in the imaging field of the high-power microscope.

[0009] These methods have limited ability to address the practical challenges of insect sperm, such as their small size, prone to overlap, and susceptibility to background lighting. They lack specific improvements to address these issues, such as sperm overlap and background noise. In particular, the lack of fine-grained feature processing can lead to low sperm detection accuracy and poor sperm tracking continuity.

[0010] Therefore, how to detect and track insect sperm is an urgent problem that those skilled in the art need to solve. Summary of the Invention

[0011] In view of this, the present invention provides an insect sperm detection and tracking method and system based on fine-grained subspace perception, which realizes the detection and tracking of insect sperm.

[0012] In order to achieve the above object, the present invention adopts the following technical solutions:

[0013] A method for detecting and tracking insect sperm based on fine-grained subspace perception, comprising:

[0014] Acquire an insect sperm image, perform bounding box annotation on the insect sperm image, and use the insect sperm image annotated with the bounding box as a data set;

[0015] Training a preset sperm target detection model using the data set;

[0016] Combining the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image annotated with a bounding box to obtain a tracking vector;

[0017] The target bounding box and tracking vector output by the detection head in the sperm target detection model are used as input to the preset sperm target tracking model. The Mahalanobis distance and appearance distance are calculated based on the predicted state and the observation vector. A comprehensive matching cost is constructed using the Mahalanobis distance and appearance distance. The global optimal matching solution is calculated based on the comprehensive matching cost and the Hungarian algorithm. The trajectory state of the successfully matched target is updated and the tracking result is output.

[0018] Preferably, the bounding box annotation of the insect sperm image specifically includes:

[0019] (x min ,y min ,x max ,y max );

[0020] Among them, (x min ,y min ) is the coordinate of the upper left corner of the bounding box, (x max ,y max ) is the coordinate of the lower right corner of the bounding box. For an image with width width and height height, the annotated bounding box satisfies:

[0021] 0≤x min <x max ≤width;

[0022] 0≤y min <y max ≤height;

[0023] Convert the coordinates of the annotation bounding box from absolute to relative:

[0024]

[0025]

[0026] Among them, x min_r 、y min_r 、x max_r 、y max_r are the relative coordinates after conversion, width and height are the width and height of the image respectively.

[0027] Preferably, the preset sperm target detection model specifically includes adding fine-grained subspace perception attention modules with dimensional channel numbers K of 16, 8, and 4 respectively after the second, third, and fourth C2f modules of the original YOLOv8 model.

[0028] Preferably, the fine-grained subspace-aware attention module is specifically:

[0029] Divide the input feature map X into K mutually exclusive subspaces:

[0030] X (k) =X[(k-1)·T:k·T,:,:],k∈{1,2,…,K};

[0031]

[0032] Among them, T = C / K is the number of channels in each subspace, is the feature map of the kth subspace, C is the number of channels of the input feature, H and W are the height and width of the feature map respectively;

[0033] In each subspace, deep convolution and maximum pooling operations are used to extract local salient features:

[0034]

[0035] in, is the depth convolution kernel of the kth subspace, MaxPool 3×3,1 is the maximum pooling operation with a kernel size of 3×3 and a stride of 1; σ(·) represents the ReLU activation function, is the local salient feature extracted from the k-th subspace;

[0036] For each subspace local salient feature Z( k ) Apply point-by-point convolution and Softmax operations to generate corresponding attention weights:

[0037]

[0038] in, is the k-th subspace point-by-point convolution kernel, normalized on the spatial dimension of the feature map, ensuring:

[0039]

[0040] The generated attention weights are used to weight the original features to obtain redistributed features:

[0041] Y (k) =(A (k) ⊙X (k) )+X (k) ;

[0042] Among them: ⊙ represents element-by-element multiplication, which applies the attention map to the original features, and + represents element-by-element addition, which retains the original feature information through residual connections.

[0043] Finally, the redistributed features of all subspaces are concatenated into output features:

[0044]

[0045] Preferably, before inputting the preset sperm target tracking model, the method further includes:

[0046] Assume that in the t-th frame image, the sperm target detection model detects N t mosquito sperm targets, and the bounding box of each target is represented as:

[0047]

[0048] At the same time, obtain its detection confidence

[0049] After the fine-grained subspace-aware attention module, the network outputs the corresponding feature maps at different stages with K=16, K=8, and K=4 as follows: in, Represents the output feature map when K=16;

[0050] For the detected i-th target, the corresponding ROI area is cut out from the original image to perform feature extraction of shallow CNN to obtain a basic feature vector exist The spatial position corresponding to the target ROI is intercepted to obtain the multi-scale target feature vector: The above feature vectors are processed to obtain the final tracking vector used for tracking

[0051]

[0052] Preferably, the preset sperm target tracking model specifically includes:

[0053] Given the Kalman filter state vector of the j-th tracking trajectory at the t-1 frame The predicted state of the tth frame Expressed as:

[0054]

[0055] Among them, A is the state transfer matrix, w is the process noise, and among the targets detected in this frame, each candidate box This will generate an observation vector in Indicates the center coordinates of the candidate box, Represents the width and height of the candidate box, and the formula is used to Convert to

[0056]

[0057] Forecast status With the observation vector First, perform "motion model" screening and calculate the Mahalanobis distance:

[0058]

[0059] Where H is the observation matrix, S is the covariance matrix, if d Mah If the threshold θ is exceeded, the matching pair will be eliminated;

[0060] Among the matching candidates through Mahalanobis distance, let the feature vector saved by tracking trajectory j at the last successful matching be Let the feature vector of the current detection target i be Using cosine similarity metric:

[0061]

[0062] d app The smaller (i, j), the more similar the appearance features are, and the more suitable it is for matching the same target;

[0063] Construct a comprehensive matching cost:

[0064]

[0065] Among them, λ is the trade-off coefficient, and the Hungarian algorithm is used to find the global optimal matching solution between the detection target i and the trajectory j;

[0066] If a track does not successfully match any detection results within a certain number of frames, it is marked as "lost" and a certain amount of appearance feature history is retained. When a new detection occurs, the multi-scale features are compared with the historical "lost" track features:

[0067] d app (i,γ)<θ;

[0068] If the appearance distance is less than the threshold θ, it is determined that the detection i and the lost track γ belong to the same mosquito sperm target, and the tracking ID is restored; for each successfully matched target in the t-th frame, its updated tracking status is obtained and the corresponding target ID to form a complete sequence of mosquito sperm movement trajectories.

[0069] An insect sperm detection and tracking system based on fine-grained subspace perception, including:

[0070] An image processing module is used to obtain an insect sperm image, annotate the insect sperm image with a bounding box, and use the insect sperm image annotated with the bounding box as a data set;

[0071] A target detection module, which trains a preset sperm target detection model using the data set;

[0072] A vector acquisition module combines the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image after bounding box annotation to obtain a tracking vector;

[0073] The target tracking module takes the target bounding box and tracking vector output by the detection head in the sperm target detection model as input, inputs the preset sperm target tracking model, calculates the Mahalanobis distance and appearance distance based on the predicted state and the observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and appearance distance, calculates the global optimal matching solution based on the comprehensive matching cost and the Hungarian algorithm, updates the trajectory state of the successfully matched target, and outputs the tracking result.

[0074] Through the above technical solutions, it can be seen that compared with the existing technology, the present invention discloses a method and system for detecting and tracking insect sperm based on fine-grained subspace perception. Fine-grained subspace division is adopted. By dividing the image space into different subspaces and independently modeling each subspace, the sensitivity to the local features of sperm is greatly improved. This fine-grained operation method can more accurately capture the subtle features of each sperm, and even if the sperm is small in size, it can be clearly identified. Local significant feature extraction focuses on highlighting the highlighted areas in the image, that is, the key areas where the sperm are located, which can effectively reduce the false detection rate of overlapping areas. When faced with overlapping sperm, traditional methods are prone to misjudge multiple sperm as one, or fail to accurately define the boundaries of each sperm. By extracting local significant features, it is possible to more accurately segment and identify overlapping sperm, thereby improving the accuracy of detection.

[0075] The introduction of an attention mechanism significantly improves interference resistance: it can dynamically adjust the weights of each subspace feature. When lighting changes, it automatically focuses on feature areas less affected by lighting, reducing the impact of lighting changes on detection results. For images with complex backgrounds, the attention mechanism allocates more weight to sperm features, suppressing interference from background noise, thereby making detection results more accurate and reliable. In contrast, traditional methods often lack this adaptive adjustment capability, resulting in a significant drop in detection accuracy in complex environments.

[0076] By fusing global and local features and employing a residual mechanism to preserve the overall image characteristics, the system ensures that the overall movement trend of sperm is not lost during tracking. Furthermore, combined with a fine-grained subspace perception module, the system is able to capture the local details of sperm, enabling precise tracking of insect sperm trajectories. Even when sperm undergo complex movements such as rapid turns and acceleration, the system accurately records their trajectories, significantly improving the accuracy and stability of trajectory tracking compared to traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0078] Figure 1 A diagram of the method steps provided by the present invention;

[0079] Figure 2 This is a framework diagram of the sperm target detection algorithm provided by the present invention;

[0080] Figure 3a The locust sperm identification diagram provided by the present invention;

[0081] Figure 3b The mosquito sperm identification diagram provided by the present invention;

[0082] Figure 3c This is a sperm identification diagram of the gray leafhopper provided by the present invention. DETAILED DESCRIPTION

[0083] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0084] The embodiment of the present invention discloses a method for detecting and tracking insect sperm based on fine-grained subspace perception, such as Figure 1 Shown, including:

[0085] Obtain insect sperm images, annotate the insect sperm images with bounding boxes, and use the insect sperm images annotated with bounding boxes as a dataset;

[0086] Train the preset sperm target detection model using the dataset;

[0087] Combine the feature map output by the fine-grained subspace-aware attention module in the sperm target detection model with the original insect sperm image annotated with bounding boxes to obtain the tracking vector;

[0088] The target bounding box and tracking vector output by the detection head in the sperm target detection model are used as input to the preset sperm target tracking model. The Mahalanobis distance and appearance distance are calculated based on the predicted state and the observation vector. The comprehensive matching cost is constructed through the Mahalanobis distance and appearance distance. The global optimal matching solution is calculated based on the comprehensive matching cost and the Hungarian algorithm. The trajectory state of the successfully matched target is updated and the tracking result is output.

[0089] In one specific embodiment, the process also includes setting up a platform to dissect sperm from insect spermatheca using ophthalmic scissors and fine surgical forceps under a stereomicroscope. Image acquisition is performed using the constructed platform, including multi-angle and multi-scene capture of insect sperm to ensure coverage of varying lighting, backgrounds, and motion conditions. Sperm images are captured using a high-resolution microscope and camera, and their motion trajectory is monitored in real time using an image sensor to ensure accurate dynamic data. Finally, the dataset is divided. First, the captured images are annotated to define the bounding box of each insect sperm. The dataset is then divided into training and test sets.

[0090] Locusts: Dissect sperm from the spermatheca of a mated female insect. Cut the spermatheca obliquely about 2-3 mm above the external genitalia at the tail. Use forceps to remove the spermatheca tube and spermatheca, then place it in a 50 μL drop of PBS (containing Ca2+) buffer in a small Petri dish. Remove as much tissue as possible from the spermatheca head and transfer it to another clean drop of PBS. Sperm inside should be squeezed out. Finally, use a pipette to aspirate the sperm suspension and transfer it to a tube for later use. Mosquitoes and planthoppers: Dissect the testes from sexually mature male insects. Use forceps to sever the testes from the 4-5 abdominal segments of a Culex mosquito or the thorax and abdomen of a planthopper. Use forceps to remove a pair of testes from the cut ends. Pulse the testes in a 10 μL drop of PBS (containing Ca2+) buffer to obtain a sperm suspension. Dissection is performed in PBS (containing Ca2+) buffer, and sperm is retrieved from the target tissue.

[0091] Image acquisition involves the following steps: 10 μl of sperm suspension is dripped onto a non-adherent glass slide and gently covered with a 24 mm x 24 mm cover slip, removing as many bubbles as possible. Image acquisition is performed using the constructed platform, with the target sperm field of view at 20 x 10 magnification. Multiple 10-second videos are captured using an E3 ISPM microscope imaging camera. This includes capturing insect sperm from multiple angles and scenes, ensuring coverage of varying lighting, backgrounds, and motion conditions. Sperm images are captured using a high-resolution microscope or camera, and their motion trajectory is monitored in real time using an image sensor to ensure accurate dynamic data. The captured image data is saved as a video file. The dataset is partitioned into training and test sets in an 8:2 ratio. The training set, comprising 80% of the total dataset, is used for model training. The test set, comprising 20% ​​of the total dataset, is used for evaluating model performance. Random sampling is used to partition the dataset to ensure similar sample distributions between the training and test sets to prevent data bias from impacting model performance.

[0092] In a specific embodiment, a picture is saved every 10 frames of the collected video file, and the collected images are annotated to mark the bounding box of each insect sperm. The bounding box can be represented by four parameters:

[0093] (x min ,y min ,x max ,y max );

[0094] Among them, (x min ,y min ) is the coordinate of the upper left corner of the bounding box, (x max ,y max ) is the coordinate of the lower right corner of the bounding box. For an image with width width and height height, the annotated bounding box satisfies:

[0095] 0≤x min <x max ≤width;

[0096] 0≤y min <y max ≤height;

[0097] Convert the coordinates of the annotation bounding box from absolute to relative:

[0098]

[0099] Among them, x min_r 、y min_r 、x max_r 、y max_rare the relative coordinates after conversion, width and height are the width and height of the image respectively.

[0100] In a specific embodiment, Figure 2 As shown, the preset sperm target detection model specifically includes adding fine-grained subspace perception attention modules with dimensional channel numbers K of 16, 8, and 4 in sequence after the second, third, and fourth C2f modules of the Backbone network of the original YOLOv8 model; to realize the detection of insect sperm targets, the fine-grained subspace perception attention module divides the subspace according to the characteristics of the sperm image and generates independent attention weights for each subspace. This method enhances the model's perception ability of detailed information and effectively improves the accuracy of sperm detection. By optimizing the network's attention mechanism, the impact of background noise on the detection results is reduced, and the model's detection ability in complex scenes is improved. Combined with the efficient feature extraction and detection capabilities of YOLOv8, it can quickly identify the presence and location of sperm, especially when sperm overlap and have diverse morphologies, and can still achieve high detection accuracy;

[0101] In a specific embodiment, the fine-grained subspace-aware attention module is specifically:

[0102] Assume that the input feature map is

[0103]

[0104] Among them, C is the number of channels of the input feature, H and W are the height and width of the feature map respectively.

[0105] First, the input feature map X is divided into K mutually exclusive subspaces:

[0106] X (k) =X[(k-1)·T:k·T,:,:],k∈{1,2,…,K};

[0107]

[0108] Among them, T = C / K is the number of channels in each subspace, is the feature map of the kth subspace, C is the number of channels of the input feature, H and W are the height and width of the feature map respectively;

[0109] In each subspace, deep convolution and maximum pooling operations are used to extract local salient features:

[0110]

[0111] in, is the depth convolution kernel of the kth subspace, MaxPool 3×3,1is the maximum pooling operation with a kernel size of 3×3 and a stride of 1; σ(·) represents the ReLU activation function, is the local salient feature extracted from the k-th subspace;

[0112] For each subspace local salient feature Z( k ) Apply point-by-point convolution and Softmax operations to generate corresponding attention weights:

[0113]

[0114] in, is the k-th subspace point-by-point convolution kernel, normalized on the spatial dimension of the feature map, ensuring:

[0115]

[0116] The generated attention weights are used to weight the original features to obtain redistributed features:

[0117] Y (k) =(A (k) ⊙X (k) )+X (k) ;

[0118] Among them: ⊙ represents element-by-element multiplication, which applies the attention map to the original features, and + represents element-by-element addition, which retains the original feature information through residual connections.

[0119] Finally, the redistributed features of all subspaces are concatenated into output features:

[0120]

[0121] The raw insect sperm image is input into the backbone network, first passing through an initial convolutional layer, followed by a C2f layer. This layer performs preliminary convolution, splitting, bottlenecking, and concatenation operations on the input image to extract underlying texture and edge features (initial feature extraction). The C2f layer typically involves feature splitting and reconcatenation between the convolutional branch and the identity branch, capturing more detailed information while maintaining computational efficiency. The fine-grained subspace-aware attention module accurately models the relationship between local and global features of insect sperm by partitioning the subspace and generating corresponding attention maps.

[0122] After the initial feature extraction is completed, the output feature map is input into the fine-grained subspace-aware attention module (K=16). This module divides the subspace using the channel dimension K=16, generates fine-grained attention weights through depthwise convolution, maximum pooling, point convolution, and softmax, and performs weighted distribution. Since the feature map resolution is relatively high and the number of channels is relatively small at this time, choosing K=16 can capture subtle differences in the early features of small targets (insect sperm), thereby strengthening the focus on the edges and textures of small targets. The feature map enhanced by the K=16 attention module is sequentially passed through the C2f layer and the standard convolution layer to achieve further channel fusion and semantic extraction, providing a more robust and detailed insect sperm representation for subsequent deeper networks.

[0123] When the network reaches a medium depth, the output feature map is once again input into the fine-grained subspace perception attention module (K=8). Compared with the previous stage, the current feature map has a lower resolution and an increased number of channels, making it suitable for a K=8 subspace division method. This method can take into account richer channel information and moderate attention refinement while avoiding excessive computational burden. This module further strengthens the network's ability to focus on mid-level features, enabling it to effectively distinguish insect sperm from background noise, overlapping areas, and other difficult scenes. Similarly, the features output by the K=8 attention module are again fed into the C2f layer and the convolutional layer to further refine key semantics and retain certain spatial details.

[0124] At deeper layers, the network feature resolution is further reduced and the number of channels is increased. This is where a fine-grained subspace-aware attention module with K=4 is introduced. By performing a coarser-grained subspace partitioning of the channel dimension, global semantic information and salient areas of small target sperm locations are focused on, further reducing interference from environmental clutter on detection. Entering the SPPF (Spatial Pyramid Pooling-Fast) layer, multi-scale max pooling and convolutional splicing operations are used to integrate global contextual information into the same output feature map, resulting in a feature representation with the richest deep semantics.

[0125] First upsampling and concatenation: The output features of the SPPF layer are upsampled (UpSample) to maintain a similar spatial resolution to that of the previous shallower layer (here corresponding to the output of the fine-grained subspace-aware attention module (K=8) or its subsequent C2f layer). The upsampling result is concatenated (Concat) with the output features of the corresponding layer and fed into the C2f layer for convolutional fusion. This operation integrates deep global semantics with mid-level fine features, improving the localization accuracy and discrimination of small objects.

[0126] Second upsampling and concatenation: After the previous fusion, upsampling is performed again and concatenated with the output of a shallower layer (corresponding to the stage where the fine-grained subspace perception attention module (K=16) is located or its subsequent C2f layer). Continued convolutional fusion of the C2f layer further integrates low-level details (such as edge morphology and texture) with deep semantic information, improving the ability to distinguish insect sperm outlines and overlapping areas.

[0127] Multi-scale detection branch. After completing the above upsampling and feature fusion, after performing some convolution and splicing operations, the network will generate multi-scale feature maps (such as large, medium, and small resolutions), which are fed into the detection head. Each detection head is usually composed of convolution layers, and finally outputs the classification branch and the regression branch.

[0128] During the network training phase, the output of the multi-scale detection branch is jointly optimized using regression loss and classification loss. Since the present invention inserts fine-grained subspace perception attention modules at different depths (K=16, K=8, K=4), during the training process, the network can generate more stable and refined gradient updates for small insect sperm targets and overlapping areas, thereby significantly improving detection accuracy and robustness.

[0129] In a specific embodiment, before inputting the preset sperm target tracking model, the method further includes:

[0130] Assume that in the t-th frame image, YOLOv8 (including attention module) detects a total of N t mosquito sperm targets, and the bounding box of each target is represented as:

[0131]

[0132] At the same time, obtain its detection confidence

[0133] After the fine-grained subspace-aware attention module, the network outputs the corresponding feature maps at different stages with K=16, K=8, and K=4 as follows: in, It represents the output feature map when K=16, and the latter two are the same;

[0134] For the detected i-th target, the corresponding ROI area is cut out from the original image to perform feature extraction of shallow CNN to obtain a basic feature vector exist The spatial position corresponding to the target ROI is intercepted to obtain the multi-scale target feature vector: The above feature vectors are processed to obtain the final tracking vector used for tracking

[0135]

[0136] In a specific embodiment, a multi-scale feature tracking module is introduced to track the vector As input, the DeepSort algorithm model is improved to achieve insect sperm target tracking. The improved DeepSORT model enhances the robustness and continuity of the tracking process by fusing images and features at different scales. This is especially true in scenarios where insect sperm have high speeds and large trajectories, effectively reducing the possibility of target loss or mismatching. The preset sperm target tracking model (improved DeepSORT model) specifically includes:

[0137] Given the Kalman filter state vector of the j-th tracking trajectory at the t-1 frame (including target position, speed, etc.), predicted state of frame t Expressed as:

[0138]

[0139] Among them, A is the state transfer matrix, w is the process noise, and among the targets detected in this frame, each candidate box This will generate an observation vector in Indicates the center coordinates of the candidate box, Represents the width and height of the candidate box, which is used to match the predicted trajectory. Convert to For subsequent Kalman filtering and motion prediction use:

[0140]

[0141] Forecast status With the observation vector First, perform "motion model" screening and calculate the Mahalanobis distance:

[0142]

[0143] Where H is the observation matrix, S is the covariance matrix, if d Mah If the threshold θ is exceeded, the matching pair will be eliminated;

[0144] Among the matching candidates through Mahalanobis distance, let the feature vector (or its historical mean) saved by tracking trajectory j at the last successful matching be Let the feature vector of the current detection target i be Using cosine similarity metric:

[0145]

[0146] dapp The smaller (i, j), the more similar the appearance features are, and the more suitable it is for matching the same target;

[0147] In the present invention, the motion distance and the appearance distance can be weighted or normalized to construct a comprehensive matching cost:

[0148]

[0149] Among them, λ is the trade-off coefficient, and the Hungarian algorithm is used to find the global optimal matching solution between the detection target i and the trajectory j;

[0150] If a track does not successfully match any detection results within a certain number of frames, it is marked as "lost" and a certain amount of appearance feature history is retained. When a new detection occurs, the multi-scale features are compared with the historical "lost" track features:

[0151] d app (i,γ)<θ;

[0152] If the appearance distance is less than the threshold θ, it is determined that the detection i and the lost track γ belong to the same mosquito sperm target, and the tracking ID is restored; for each successfully matched target in the t-th frame, its updated tracking status is obtained and the corresponding target ID to form a complete sequence of mosquito sperm movement trajectories.

[0153] Training the partitioned model mainly includes data preprocessing, model training, and hyperparameter optimization, including the following steps:

[0154] First, the collected training data undergoes necessary preprocessing, including normalizing image pixel values ​​and performing appropriate data augmentation without compromising key insect sperm features. Labeled information is also inspected and standardized to ensure the accuracy and diversity of the training samples. Next, by setting appropriate optimization algorithms and hyperparameters, such as using an Adam-based optimizer, employing separate loss functions for classification and regression, and combining batch size and learning rate decay strategies, the model converges on the training set over multiple epochs and gradually improves accuracy. After training is complete, the corresponding weight files are saved.

[0155] Furthermore, the model testing and application includes the following steps:

[0156] By inferring the trained model on a partitioned test set, key metrics such as detection and tracking accuracy, recall, and multi-target tracking performance (MOTA and MOTP) are evaluated. The results are visualized to detect false detections, missed detections, and mistracking. After verification, the trained model can be deployed in a laboratory or field observation environment to perform real-time detection and tracking analysis of insect sperm images or video streams. This can be applied to a variety of scenarios, such as sperm motility assessment and population distribution monitoring, based on biological needs.

[0157] An insect sperm detection and tracking system based on fine-grained subspace perception, including:

[0158] An image processing module acquires insect sperm images, annotates the insect sperm images with bounding boxes, and uses the insect sperm images annotated with bounding boxes as a data set;

[0159] The target detection module trains the preset sperm target detection model using the dataset;

[0160] The vector acquisition module combines the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image after bounding box annotation to obtain the tracking vector;

[0161] The target tracking module takes the target bounding box and tracking vector output by the detection head in the sperm target detection model as input, inputs the preset sperm target tracking model, calculates the Mahalanobis distance and appearance distance based on the predicted state and the observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and appearance distance, calculates the global optimal matching solution based on the comprehensive matching cost and the Hungarian algorithm, updates the trajectory state of the successfully matched target, and outputs the tracking result.

[0162] The insect sperm detection and tracking method and device based on fine-grained subspace perception of the present invention are highly practical, as shown in the following aspects:

[0163] Operational convenience: The operating procedures of this method and device are simple and clear, and the professional knowledge threshold required of the operator is relatively low. During the detection process, there is no need for complex parameter adjustments and tedious pre-processing steps. Technicians who have received simple training can operate it proficiently, which greatly saves labor costs and time costs. For example, when using this device for sample testing, the operator only needs to place the sample in the designated location according to the established operating procedures and start the detection program. The device can automatically complete a series of operations such as image acquisition, analysis, and result output, greatly improving detection efficiency.

[0164] Equipment compatibility: The device of this invention is compatible with a variety of common laboratory equipment, allowing for flexible integration into diverse experimental environments. Microscopes, image acquisition devices, and data processing terminals all interface well with the device and work collaboratively. This eliminates the need for researchers to purchase extensive new equipment specifically for this detection and tracking method, reducing research costs. Furthermore, this compatibility facilitates data sharing and method dissemination across laboratories, promoting research progress in related fields.

[0165] Wide range of application scenarios: In the field of biological research, it can be used to deeply explore the reproductive biology characteristics of insects, such as sperm development, maturation process, and fertilization mechanism, and provide key data support for understanding the reproductive behavior of insects. In terms of disease prevention and control, it can quickly and accurately detect the sperm characteristics of specific insect populations, which helps to track changes in disease transmission vectors and provide a basis for formulating more effective prevention and control strategies. In addition, in ecological environmental monitoring, through the detection and analysis of insect sperm in different environments, the impact of environmental changes on pest populations can be evaluated, providing a scientific reference for ecological protection. Whether it is basic research or practical application, the method and device of the present invention have demonstrated great potential and practical value.

[0166] Scalability: This method and device possess excellent scalability, enabling functional upgrades and optimizations tailored to diverse research needs and application scenarios. For example, building on existing technologies, artificial intelligence algorithms can be further integrated to enhance the automation and accuracy of detection, or the ability to detect microparticles in other biological samples can be added to broaden its application. This scalability ensures that the method and device of this invention can continue to play a role in evolving scientific research and application needs, maintaining its advanced nature and practicality.

[0167] In a specific embodiment, an Olympus stereo microscope combined with an E3 ISPM microscopic imaging camera platform is used to shoot insect sperm videos under different lighting and background conditions at a resolution of 1536*1024, such as Figure 3a 、 Figure 3b 、 Figure 3c As shown in the figure, the ID represents the automatically identified sperm number; frames are extracted and annotated to form a dataset. During model training, a batch size of 16, a threshold θ of 0.8, an initial learning rate of 1e-4, and a cosine annealing strategy were used for 150 epochs. When constructing the backbone network, fine-grained subspace-aware attention modules with K=16, K=8, and K=4 were inserted at three key depths in YOLOv8, and multi-scale context fusion was performed through the SPPF layer.

[0168] All experiments were performed on an NVIDIA GeForce RTX3090 GPU and an Intel Xeon Platinum8280 CPU, and the programming environment was based on Python 3.8 and PyTorch 1.8.1.

[0169] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0170] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting and tracking insect sperm based on fine-grained subspace perception, characterized in that: include: Acquire an insect sperm image, perform bounding box annotation on the insect sperm image, and use the insect sperm image annotated with the bounding box as a data set; Training a preset sperm target detection model using the data set; Combining the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image annotated with a bounding box to obtain a tracking vector; The target bounding box and tracking vector output by the detection head in the sperm target detection model are used as input to the preset sperm target tracking model. The Mahalanobis distance and appearance distance are calculated based on the predicted state and the observation vector. A comprehensive matching cost is constructed using the Mahalanobis distance and appearance distance. The global optimal matching solution is calculated based on the comprehensive matching cost and the Hungarian algorithm. The trajectory state of the successfully matched target is updated and the tracking result is output.

2. The insect sperm detection and tracking method based on fine-grained subspace perception according to claim 1 is characterized in that: The step of labeling the insect sperm image with a bounding box specifically includes: (x min ,and min ,x max ,and max ); Among them, (x min ,y min ) is the coordinate of the upper left corner of the bounding box, (x max ,y max ) is the coordinate of the lower right corner of the bounding box. For an image with width width and height height, the annotated bounding box satisfies: 0≤x min <x max ≤width; 0≤y min <y max ≤height; Convert the coordinates of the annotation bounding box from absolute to relative: Among them, x min_r 、y min_r 、x max_r 、y max_r are the relative coordinates after conversion, width and height are the width and height of the image respectively.

3. The insect sperm detection and tracking method based on fine-grained subspace perception according to claim 1 is characterized in that: The preset sperm target detection model specifically includes adding fine-grained subspace perception attention modules with dimensional channel numbers K of 16, 8, and 4 respectively after the second, third, and fourth C2f modules of the original YOLOv8 model.

4. The insect sperm detection and tracking method based on fine-grained subspace perception according to claim 3 is characterized in that: The fine-grained subspace-aware attention module is specifically: Divide the input feature map X into K mutually exclusive subspaces: X (k) =X[(k-1)·T:k·T,:,:],k∈{1,2,…,K}; Among them, T = C / K is the number of channels in each subspace, is the feature map of the kth subspace, C is the number of channels of the input feature, H and W are the height and width of the feature map respectively; In each subspace, deep convolution and maximum pooling operations are used to extract local salient features: in, is the depth convolution kernel of the kth subspace, MaxPool 3×3,1 is the maximum pooling operation with a kernel size of 3×3 and a stride of 1; σ(·) represents the ReLU activation function, is the local salient feature extracted from the k-th subspace; For each subspace, the local salient feature Z (k) Apply point-by-point convolution and Softmax operations to generate corresponding attention weights: in, is the k-th subspace point-by-point convolution kernel, normalized on the spatial dimension of the feature map, ensuring: The generated attention weights are used to weight the original features to obtain redistributed features: AND (k) =(A (k) ⊙X (k) )+X (k) ; Among them: ⊙ represents element-by-element multiplication, which applies the attention map to the original features, and + represents element-by-element addition, which preserves the original feature information through residual connections; Finally, the redistributed features of all subspaces are concatenated into output features:

5. The insect sperm detection and tracking method based on fine-grained subspace perception according to claim 1 is characterized in that: Before entering the preset sperm target tracking model, also include: Assume that in the t-th frame image, the sperm target detection model detects N t mosquito sperm targets, and the bounding box of each target is represented as: At the same time, obtain its detection confidence After the fine-grained subspace-aware attention module, the network outputs the corresponding feature maps at different stages with K=16, K=8, and K=4 as follows: in, Represents the output feature map when K=16; For the detected i-th target, the corresponding ROI area is cut out from the original image to perform feature extraction of shallow CNN to obtain a basic feature vector exist The spatial position corresponding to the target ROI is intercepted to obtain the multi-scale target feature vector: The above feature vectors are processed to obtain the final tracking vector used for tracking 6. The insect sperm detection and tracking method based on fine-grained subspace perception according to claim 5 is characterized in that: The preset sperm target tracking model specifically includes: Given the Kalman filter state vector of the j-th tracking trajectory at the t-1 frame The predicted state of the tth frame Expressed as: Among them, A is the state transfer matrix, w is the process noise, and among the targets detected in this frame, each candidate box This will generate an observation vector in Indicates the center coordinates of the candidate box, Represents the width and height of the candidate box, and the formula is used to Convert to Forecast status With the observation vector First, perform "motion model" screening and calculate the Mahalanobis distance: Where H is the observation matrix, S is the covariance matrix, if d Mah If the threshold θ is exceeded, the matching pair will be eliminated; Among the matching candidates through Mahalanobis distance, let the feature vector saved by tracking trajectory j at the last successful matching be Let the feature vector of the current detection target i be Using cosine similarity metric: d app The smaller (i, j), the more similar the appearance features are, and the more suitable it is for matching the same target; Construct a comprehensive matching cost: Among them, λ is the trade-off coefficient, and the Hungarian algorithm is used to find the global optimal matching solution between the detection target i and the trajectory j; If a track does not successfully match any detection results within a certain number of frames, it is marked as "lost" and a certain amount of appearance feature history is retained. When a new detection appears, the multi-scale features are compared with the historical "lost" track features: d app (i,γ)<θ; If the appearance distance is less than the threshold θ, it is determined that the detection i and the lost track γ belong to the same mosquito sperm target, and the tracking ID is restored; for each successfully matched target in the t-th frame, its updated tracking status is obtained and the corresponding target ID to form a complete sequence of mosquito sperm movement trajectories.

7. An insect sperm detection and tracking system based on fine-grained subspace perception, used in the insect sperm detection and tracking method based on fine-grained subspace perception according to any one of claims 1 to 6, characterized in that: include: An image processing module is used to obtain an insect sperm image, annotate the insect sperm image with a bounding box, and use the insect sperm image annotated with the bounding box as a data set; A target detection module, which trains a preset sperm target detection model using the data set; A vector acquisition module combines the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image after bounding box annotation to obtain a tracking vector; The target tracking module takes the target bounding box and tracking vector output by the detection head in the sperm target detection model as input, inputs the preset sperm target tracking model, calculates the Mahalanobis distance and appearance distance based on the predicted state and the observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and appearance distance, calculates the global optimal matching solution based on the comprehensive matching cost and the Hungarian algorithm, updates the trajectory state of the successfully matched target, and outputs the tracking result.

Images