An insect sperm detection and tracking method and system based on fine-grained subspace perception

Abstract

The application discloses an insect sperm detection and tracking method and system based on fine-grained subspace perception, relates to the technical field of insect reproduction prevention and control, and performs boundary box labeling on an insect sperm image to establish a data set; a preset sperm target detection model is trained through the data set; a feature map output by a fine-grained subspace perception attention module in the model is combined with the insect sperm image labeled through the boundary box to obtain a tracking vector; the target boundary box output by a detection head in the model and the tracking vector are taken as inputs, the inputs are input into a preset sperm target tracking model, Mahalanobis distance and appearance distance are calculated based on a predicted state and an observation vector, a comprehensive matching cost is constructed through the Mahalanobis distance and the appearance distance, a global optimal matching scheme is calculated based on the comprehensive matching cost and a Hungarian algorithm, the trajectory state of a successfully matched target is updated, and a tracking result is output. The application realizes detection and tracking of insect sperm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of insect reproduction prevention and control, and more particularly to an insect sperm detection and tracking method and system based on fine-grained subspace perception. BACKGROUND

[0002] In existing sperm automatic recognition technology, most of them are aimed at the typical "tadpole-like" sperm of mammals for automatic calculation of sperm density and motility, such as CASA system. Insect sperm, due to its unique morphological structure, presents long linear shape, head width is very narrow or even head and tail width is close, and tail length is much longer than head, not "tadpole-like", so it cannot be recognized by the existing automatic sperm analysis system. Only honey bees have developed a SCA CASA method for observing sperm motility after fluorescent staining, but this method requires the head of the sperm to release a strong fluorescent signal after staining, which cannot be achieved for some insect sperm heads. In addition, the pre-staining of sperm increases the complexity of experimental operation, and cannot achieve the purpose of rapid detection. Special fluorescent microscopes and cameras are also needed to collect signals, which is costly.

[0003] Locusts are a world-wide agricultural pest, and a locust outbreak can cause massive farmland loss. A locust swarm can consume the food of 35,000 people. Chemical pesticides are currently the main means of controlling locusts, but the ecological and environmental problems and food safety problems caused by them have become increasingly prominent, so it is necessary to find more effective biological control means. The small brown planthopper is another important agricultural pest. It not only sucks the sap of rice plants through its mouthparts, causing the plants to become withered and yellow, but more importantly, it can also transmit rice stripe virus, causing diseased plants to become withered and produce small, deformed and unproductive panicles. Therefore, the planthopper has become an important pest of rice. Mosquitoes are the main transmission media for many infectious diseases, especially in the transmission of diseases such as malaria, dengue fever, and Zika virus. Research on the reproductive capacity of these pests, especially sperm detection and tracking, can provide important scientific basis for population control and disease vector control. However, due to the small size, complex morphology, and unpredictable dynamic behavior of insect sperm, its detection and tracking has always been a difficult problem in scientific research.

[0004] Traditional methods of insect sperm detection and tracking mainly rely on microscope observation and manual analysis. These methods usually require highly skilled technicians to operate, consume a lot of time and effort, and are easily influenced by subjective factors and difficult to achieve large-scale data processing. In addition, the traditional method has limited ability to accurately capture the movement trajectory and dynamic behavior of sperm, which may miss important dynamic information and cannot meet the needs of high-precision and high-throughput research. Therefore, it is urgent to develop a more intelligent, efficient and accurate detection and tracking method.

[0005] In recent years, the rapid development of deep learning in the field of computer vision has provided a new way of thinking for biological image analysis. In particular, in the task of target detection and tracking, deep learning models have shown excellent performance. These algorithms extract multi-level features in images by constructing convolutional neural networks, achieving accurate detection of targets. Although deep learning provides a new technical approach for insect sperm detection and tracking, it still faces many challenges in practical applications: insect sperm is small, narrow and easily overlapped; factors such as light changes and background noise in the detection environment can interfere with the model's judgment and reduce the robustness of detection. In the existing technology, there is no special method and device for insect sperm detection and tracking.

[0006] In the existing similar technology, Chinese invention patent publication (announcement) number CN114241006B discloses a sperm tracking and automatic extraction method based on machine learning. The purpose is to solve the problem that the current main sperm monitoring method generally has low monitoring accuracy and slow monitoring speed, which is not conducive to real-time tracking of multiple sperm targets. The main scheme includes: obtaining real-time image data of sperm samples; realizing multi-sperm tracking through YOLO-V4, KCF tracker and Kalman filter; calculating the speed of sperm according to the tracking result, and determining the target sperm according to the speed; using YOLO-V4 to detect and extract the position of the needle in the image; constructing the observation state of the extraction needle controller according to the position of the extraction needle and the tracking position of the sperm; inputting the observation state into the actor network in the DDPG deep reinforcement learning model to obtain the control amount. The control amount output by the actor network is input into the extraction needle controller, and the control amount is converted into a control signal executable by the controller. The controller executes the behavior corresponding to the control amount according to the control signal.

[0007] For example, Chinese invention patent publication (announcement) number CN112580476B discloses a sperm recognition and multi-target trajectory tracking method, which includes: selecting a target sperm in the first frame of sperm image and obtaining its coordinates, taking the coordinates as the starting coordinate point of the trajectory chain of the target sperm; using a trajectory prediction algorithm to calculate the predicted coordinate point of the target sperm in the i-th frame of sperm image; setting a search area in the i-th frame of sperm image with the predicted coordinate point, if multiple sperms are found in the area, then a multi-principle combination matching algorithm is used to associate and match, a best matching sperm is obtained, and the coordinate point of the best matching sperm is added to the trajectory chain of the target sperm; if no sperm is found in the area, the predicted coordinate point is added to the trajectory chain of the target sperm. This scheme improves the tracking efficiency of sperm motion trajectory, adapts to the scene of temporary loss of sperm, and at the same time improves the matching accuracy and reduces the mismatch rate.

[0008] For example, Chinese invention patent publication (announcement) No. CN118883551B discloses a sperm automatic detection system and a control method thereof, relating to the technical field of cell detection. The control method of the sperm automatic detection system includes: acquiring large field image data sent by an upright low-power objective lens, and confirming the motion characteristics of all sperm in the semen; completing the vitality analysis of all sperm according to the motion characteristics; controlling the motorized stage to perform motion compensation so that the target sperm is at the target position of the imaging field of view of the inverted high-power objective lens; acquiring imaging information sent by the inverted high-power objective lens to complete the morphology analysis operation of the target sperm; selecting the next target sperm according to the large field image data to execute the previous step until all target sperm complete the morphology analysis operation; and controlling the motorized stage to perform motion compensation so that the preferred sperm is located at the target position of the imaging field of view of the inverted low-power objective lens. The technical effect of this technical solution can ensure the activity of sperm while stably detecting sperm in the imaging field of view of the high-power objective lens.

[0009] These methods have limited ability to cope with the actual difficulties of small size and easy overlap of insect sperm, and are easily affected by the detection environment light background factors. There is no specific improvement for the problem of easy overlap and background noise of insect sperm. Especially lacking in fine-grained feature processing operation, which easily leads to low accuracy of insect sperm detection and poor continuity of sperm tracking.

[0010] Therefore, how to realize the detection and tracking of insect sperm is a problem that those skilled in the art need to solve. SUMMARY

[0011] Therefore, the present application provides an insect sperm detection and tracking method and system based on fine-grained subspace perception, which realizes the detection and tracking of insect sperm.

[0012] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0013] An insect sperm detection and tracking method based on fine-grained subspace perception, comprising:

[0014] Acquiring an insect sperm image, performing boundary box labeling on the insect sperm image, and taking the insect sperm image labeled by the boundary box as a data set;

[0015] Training a preset sperm target detection model through the data set;

[0016] Combining the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image labeled by the boundary box to obtain a tracking vector;

[0017] The target bounding box output by the detection head in the sperm target detection model and the tracking vector are taken as input, a preset sperm target tracking model is input, a Mahalanobis distance and an appearance distance are calculated based on a predicted state and an observation vector, a comprehensive matching cost is constructed through the Mahalanobis distance and the appearance distance, a global optimal matching scheme is calculated based on the comprehensive matching cost and the Hungarian algorithm, the trajectory state of a successfully matched target is updated, and a tracking result is output.

[0018] Preferably, the boundary box labeling of the insect sperm image specifically comprises:

[0019] (x min ,y min ,x max ,y max );

[0020] wherein (x min ,y min ) is the coordinate of the upper left corner of the boundary box, (x max ,y max ) is the coordinate of the lower right corner of the boundary box, for an image with a width of width and a height of height, the labeled boundary box satisfies:

[0021] 0≤x min <x max ≤width;

[0022] 0≤y min <y max ≤height;

[0023] Convert the coordinates of the labeled boundary box from absolute coordinates to relative coordinates:

[0024]

[0025]

[0026] wherein x min_r , y min_r , x max_r , y max_r are the converted relative coordinates, and width and height are the width and height of the image, respectively.

[0027] Preferably, the preset sperm target detection model specifically comprises a fine-grained subspace perception attention module with a dimension channel number K of 16, 8 and 4 added in sequence after the second, third and fourth C2f modules of the original YOLOv8 model.

[0028] Preferably, the fine-grained subspace perception attention module specifically comprises:

[0029] The input feature map X is divided into K mutually exclusive subspaces:

[0030] X (k) = X[(k-1)·T:k·T,:,:], k e {1,2,…,K};

[0031]

[0032] where T = C / K is the number of channels of each subspace, is the feature map of the kth subspace, C is the number of channels of the input feature, H and W are the height and width of the feature map, respectively;

[0033] Within each subspace, local salient features are extracted using deep convolution and max pooling operations:

[0034]

[0035] where is the deep convolution kernel of the kth subspace, MaxPool 3×3,1 is the max pooling operation with kernel size 3x3 and stride 1; σ(·) represents the ReLU activation function, is the local salient feature extracted by the kth subspace;

[0036] Point-wise convolution and Softmax operation are applied to each subspace's local salient feature Z( k ) to generate the corresponding attention weight:

[0037]

[0038] where is the point-wise convolution kernel of the kth subspace, normalized in the spatial dimension of the feature map, ensuring:

[0039]

[0040] The generated attention weight is used to weight the original feature to obtain the redistributed feature:

[0041] Y (k) = (A (k) ⊙X (k) )+X (k) ;

[0042] where: ⊙ represents element-wise multiplication, the attention map is applied to the original feature, + represents element-wise addition, and the original feature information is preserved through residual connection.

[0043] Finally, the redistributed features of all subspaces are concatenated into the output feature:

[0044]

[0045] Preferably, before inputting the input preset sperm target tracking model, further comprising:

[0046] Assuming that in the t-th frame image, N t mosquito sperm targets are detected by the sperm target detection model, and the boundary box of each target is represented as:

[0047]

[0048] At the same time, the detection confidence thereof is obtained

[0049] The output of the network at different stages through the fine-grained subspace perception attention module is represented as: Wherein, represents the output feature map when K = 16;

[0050] For the i-th detected target, the ROI region corresponding to the target is intercepted from the original image to extract the feature of the shallow layer CNN, and a basic feature vector is obtained The spatial position corresponding to the target ROI is intercepted in to obtain a multi-scale target feature vector: The above feature vectors are processed to obtain a tracking vector for tracking

[0051]

[0052] Preferably, the preset sperm target tracking model specifically comprises:

[0053] Given the Kalman filter state vector of the j-th tracking trajectory in the t-1 frame The predicted state of the t-th frame is represented as:

[0054]

[0055] Wherein, A is a state transition matrix, and w is a process noise, and in the detected targets in the current frame, each candidate box will generate an observation vector Wherein represents the center point coordinates of the candidate box, represents the width and height of the candidate box, and the formula is used to convert into

[0056]

[0057] ​Predicted state Observation vector First, the "motion model" screening is performed, and the Mahalanobis distance is calculated:

[0058]

[0059] where H is the H observation matrix, S is the covariance matrix, and if d Mah If the distance exceeds the threshold θ, the matching pair is removed.

[0060] Among the matching candidates through the Mahalanobis distance, let the feature vector saved by the tracking trajectory j at the last successful matching be Let the feature vector of the current detection target i be The cosine similarity is measured:

[0061]

[0062] d app The smaller the distance d

[0063] A comprehensive matching cost is constructed:

[0064]

[0065] where λ is a weighting coefficient, and the Hungarian algorithm is used to find the global optimal matching scheme between the detection target i and the trajectory j.

[0066] If a trajectory is not successfully matched to any detection result within several frames, it is marked as "lost" and a certain number of appearance feature histories are retained. When a new detection occurs, the multi-scale feature is compared with the historical "lost" trajectory feature:

[0067] d app (i, γ) < θ.

[0068] If the appearance distance is less than the threshold θ, it is determined that the detection i and the lost trajectory γ belong to the same mosquito sperm target, and the tracking ID is restored. For each successfully matched target in the t-th frame, the updated tracking state of the target is obtained and the corresponding target ID, forming a complete mosquito sperm motion trajectory sequence.

[0069] An insect sperm detection and tracking system based on fine-grained subspace perception, comprising:

[0070] An image processing module acquires an insect sperm image, performs bounding box labeling on the insect sperm image, and the insect sperm image after bounding box labeling is used as a data set.

[0071] A target detection module trains a preset sperm target detection model through the data set.

[0072] a vector acquisition module, which combines the feature map output by the fine-grained subspace-aware attention module in the sperm target detection model with the original insect sperm image labeled by the bounding box to obtain a tracking vector;

[0073] a target tracking module, which inputs the target bounding box output by the detection head in the sperm target detection model and the tracking vector as inputs, inputs a preset sperm target tracking model, calculates a Mahalanobis distance and an appearance distance based on a predicted state and an observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and the appearance distance, calculates a global optimal matching scheme based on the comprehensive matching cost and the Hungarian algorithm, updates a trajectory state of a successfully matched target, and outputs a tracking result.

[0074] Compared with the prior art, the technical solution can provide an insect sperm detection and tracking method and system based on fine-grained subspace awareness. The fine-grained subspace division divides the image space into different subspaces, and each subspace is modeled independently, which greatly improves the sensitivity to local features of sperm. This fine-grained operation mode can more accurately capture the subtle features of each sperm, and even when the sperm size is small, it can still clearly identify it. The local salient feature extraction highlights the highlight area in the image, that is, the key area where the sperm is located, which can effectively reduce the false detection rate of the overlapping area. Traditional methods are prone to misjudging multiple sperms as one or failing to accurately define the boundaries of each sperm when facing overlapping sperms. By extracting local salient features, overlapping sperms can be more accurately segmented and identified, improving the accuracy of detection.

[0075] Meanwhile, the attention mechanism is introduced, which significantly improves the anti-interference ability: it can dynamically adjust the weight of each subspace feature. When the light changes, it can automatically focus on the feature area that is less affected by light, reducing the interference of light changes on the detection result. For images with complex backgrounds, the attention mechanism can allocate more weight to sperm features, suppressing the interference of background noise, so that the detection result is more accurate and reliable. In contrast, traditional methods often lack this adaptive adjustment capability and the detection accuracy will decrease significantly in complex environments.

[0076] Through global and local feature fusion, the residual mechanism is used to retain the overall features of the image, ensuring that the overall movement trend of the sperm will not be lost during tracking. At the same time, combined with the fine-grained subspace awareness module, the local detail features of the sperm can be captured, thereby realizing accurate tracking of the movement trajectory of the insect sperm. Even if the sperm makes rapid turns, accelerates, or other complex situations during movement, its trajectory can still be accurately recorded. Compared with traditional methods, the accuracy and stability of trajectory tracking are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0078] Figure 1 The method steps provided by the present application are shown in the figure;

[0079] Figure 2 The sperm target detection algorithm framework provided by the present application is shown in the figure;

[0080] Figure 3a The locust sperm identification figure provided by the present application is shown in the figure;

[0081] Figure 3b The mosquito sperm identification figure provided by the present application is shown in the figure;

[0082] Figure 3c The small brown planthopper sperm identification figure provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0084] The embodiments of the present application disclose an insect sperm detection and tracking method based on fine-grained subspace perception, as shown in Figure 1 The method comprises the following steps:

[0085] An insect sperm image is acquired, and the insect sperm image is subjected to bounding box labeling. The insect sperm image subjected to bounding box labeling serves as a data set;

[0086] A preset sperm target detection model is trained through the data set;

[0087] The feature map output by the fine-grained subspace perception attention module in the sperm target detection model is combined with the original insect sperm image subjected to bounding box labeling, so as to obtain a tracking vector;

[0088] The target boundary box and the tracking vector output by the sperm target detection model are input into a preset sperm target tracking model, Mahalanobis distance and appearance distance are calculated based on the predicted state and the observation vector, a comprehensive matching cost is constructed through the Mahalanobis distance and the appearance distance, a global optimal matching scheme is calculated based on the comprehensive matching cost and the Hungarian algorithm, the trajectory state of the successfully matched target is updated, and the tracking result is output.

[0089] In one specific embodiment, a platform is also included, and under the stereomicroscope, the sperm is dissected from the insect spermatheca using an ophthalmic scissors and a fine surgical forceps; image acquisition is performed based on the built platform, including multi-angle and multi-scene shooting of insect sperm to ensure coverage of different light, background and motion state conditions. A high-resolution microscope is used to cooperate with a camera to shoot the image of the sperm, and an image sensor is used to monitor the motion trajectory in real time to ensure accurate dynamic data acquisition; the data set is divided, first, the collected images are labeled to mark the boundary box of each insect sperm; then, the data set is divided into a training set and a test set.

[0090] Locusts: sperm is dissected from the spermatheca of mated females, the tail genitalia is cut obliquely about 2-3 mm from the top, the spermatheca tube and spermatheca are pulled out with forceps and placed in a small petri dish with about 50 uL of PBS (containing Ca2+) buffer drops, the tissue wrapped around the head of the spermatheca is removed as much as possible, then transferred to another clean PBS drop, the internal sperm is squeezed in, finally the sperm suspension is sucked up with a pipette and stored in a tube for standby; mosquitoes and planthoppers: dissect the testis from sexually mature male insects, use forceps to pull off the Culex from the 4-5 abdominal segments of the tail, and pull off the planthopper between the thorax and abdomen, then clamp out a pair of testes from the broken end; crush the testes in a clean 10 uL PBS (containing Ca2+) buffer drop to obtain the sperm suspension. Dissection is performed in PBS (containing Ca2+) buffer, and sperm is obtained from the target tissue.

[0091] Collecting images, including the following steps: drop 10ul sperm suspension onto a non-adhesion slide, cover it with a 24mm*24mm cover piece, and try to expel the bubbles. Based on the platform built, collect images, find the target sperm visual field at 20*10 times, connect E3ISPM microscopic imaging camera to shoot multiple 10s videos. Including multi-angle and multi-scene shooting of insect sperm, to ensure coverage of different light, background and motion state. Use high-resolution microscope or camera to shoot sperm images, and monitor the motion trajectory in real time through image sensor to ensure accurate dynamic data; save the collected image data as a video file. Divide the dataset into training set and test set according to the ratio of 8:2; Training set: 80% of the total dataset. Used for model training. Test set: 20% of the total dataset. Used to evaluate the performance of the model. When dividing the dataset, use random sampling to ensure that the sample distribution in the training set and test set is similar, so as to avoid the influence of data bias on the performance of the model.

[0092] In one specific embodiment, save a picture every 10 frames of the collected video file, label each insect sperm boundary box, and the boundary box can be represented by four parameters:

[0093] (x min ,y min ,x max ,y max );

[0094] Where (x min ,y min ) is the coordinate of the upper left corner of the boundary box, and (x max ,y max ) is the coordinate of the lower right corner of the boundary box. For an image with a width of width and a height of height, the labeled boundary box satisfies:

[0095] 0≤x min <x max ≤width;

[0096] 0≤y min <y max ≤height;

[0097] Convert the coordinates of the labeled boundary box from absolute coordinates to relative coordinates:

[0098]

[0099] Where x min_r , y min_r , x max_r , y max_rThe converted relative coordinates, width and height are the width and height of the image, respectively.

[0100] In one specific embodiment, as shown in Figure 2 The preset sperm target detection model specifically includes, after the second, third and fourth C2f modules of the Backbone network of the original YOLOv8 model, fine-grained subspace perception attention modules with dimension channel numbers K of 16, 8 and 4, respectively. The fine-grained subspace perception attention module divides the subspace according to the characteristics of the sperm image and generates independent attention weights for each subspace. This method enhances the model's perception ability in detail information and effectively improves the accuracy of sperm detection. By optimizing the attention mechanism of the network, the influence of background noise on the detection result is reduced, and the detection ability of the model in complex scenes is improved. Combined with the efficient feature extraction and detection ability of YOLOv8, the existence and position of sperm can be quickly identified, especially in the case of sperm overlap and diverse morphology, the detection accuracy can still be high;

[0101] In one specific embodiment, the fine-grained subspace perception attention module is specifically:

[0102] Suppose the input feature map is

[0103]

[0104] Where C is the channel number of the input feature, H and W are the height and width of the feature map, respectively.

[0105] First, the input feature map X is divided into K mutually exclusive subspaces:

[0106] X (k) = X[(k-1)·T:k·T,:,:], k∈{1,2,…,K};

[0107]

[0108] Where T = C / K is the channel number of each subspace, is the feature map of the kth subspace, C is the channel number of the input feature, H and W are the height and width of the feature map, respectively;

[0109] Within each subspace, local salient features are extracted using depth convolution and max pooling operations:

[0110]

[0111] Where is the depth convolution kernel of the kth subspace, MaxPool 3×3,1is a max-pooling operation with kernel size 3x3 and stride 1; σ(·) denotes the ReLU activation function, is the local salient feature extracted by the k-th subspace;

[0112] The local salient feature Zkof each subspace is applied with point-wise convolution and Softmax operation to generate the corresponding attention weight: k

[0113]

[0114] where, is the k-th subspace point-wise convolution kernel, normalized in the spatial dimension of the feature map, ensuring:

[0115]

[0116] The original feature is weighted by the generated attention weight to obtain the re-distributed feature:

[0117] Y (k) = (A (k) ⊙X (k) )+X (k) ;

[0118] where: ⊙ denotes element-wise multiplication, the attention map is applied to the original feature, + denotes element-wise addition, and the original feature information is preserved through the residual connection.

[0119] Finally, the re-distributed features of all subspace are concatenated as the output feature:

[0120]

[0121] The original insect sperm image is input into the backbone network, first passing through an initial convolution layer, and then a C2f layer, which performs preliminary convolution, splitting, bottleneck, and concatenation operations on the input image to extract bottom-level texture and edge features (initial feature extraction). The C2f layer usually includes feature splitting and re-pasting between the convolution branch and the identity branch, which can capture more detailed information while maintaining network computing efficiency. The fine-grained subspace-aware attention module precisely models the local and global feature relationships of insect sperm by dividing the subspace and generating the corresponding attention map.

[0122] ​After the initial feature extraction is completed, the output feature map is input into the fine-grained subspace perception attention module (K=16), which divides the subspace in the form of channel dimension K=16, generates fine-grained attention weights through deep convolution, maximum pooling, point convolution and Softmax, and performs weighted distribution. Since the feature map resolution is relatively high and the channel number is relatively small at this time, selecting K=16 can capture the subtle differences in the early features of small targets (insect sperm), thereby strengthening the attention to the edges, textures, etc. of small targets. The feature map enhanced by the K=16 attention module is sequentially input into the C2f layer and the standard convolution layer to realize further channel fusion and semantic extraction, providing more robust and delicate insect sperm representations for subsequent deeper networks.

[0123] When the network enters a medium depth, the output feature map is again input into the fine-grained subspace perception attention module (K=8). Compared with the previous stage, the current feature map resolution is reduced and the channel number is increased, which is suitable for using K=8 subspace division method, which takes into account more rich channel information and moderate attention precision while avoiding excessive computational burden. This module again strengthens the focusing ability of the network on the middle layer features, enabling it to effectively distinguish insect sperm from background noise, overlapping areas and other difficult scenes. Similarly, the features output by the K=8 attention module are again input into the C2f layer and the convolution layer to further refine key semantics and retain certain spatial details.

[0124] At a deeper level, the network feature resolution is further reduced and the channel number is higher, at which time the fine-grained subspace perception attention module (K=4) is introduced. By performing more coarse-grained subspace division on the channel dimension, focusing on global semantic information and the significant area of small target sperm position, further reducing the interference of environmental clutter on detection. Then enter the SPPF layer (SpatialPyramidPooling-Fast), which integrates global context information into the same output feature map through multi-scale maximum pooling and convolution concatenation operation, obtaining the most rich feature representation of deep layer semantics.

[0125] First upsampling and concatenation. The output feature of the SPPF layer is upsampled (UpSample) once to maintain similar spatial resolution as the previous shallower layer (here corresponding to the output of the fine-grained subspace perception attention module (K=8) or the subsequent C2f layer); the above upsampled result and the corresponding layer output feature are concatenated (Concat) and input into the C2f layer for convolution fusion. This operation can integrate deep global semantics and middle layer fine features, improve the positioning accuracy and discrimination of small targets.

[0126] Second upsampling and splicing. After completing the previous fusion, the network is upsampling again and splicing with the output of the more shallow layer (corresponding to the stage where the fine-grained subspace-aware attention module (K=16) is located or the subsequent C2f layer); continue to use C2f layer convolution fusion to further fuse the bottom layer details (such as edge shape, texture, etc.) and deep semantic information, and improve the ability to distinguish the outline of insect sperm and overlapping areas.

[0127] Multi-scale detection branch. After the above upsampling and feature fusion, the network generates multi-scale feature maps (such as large, medium and small three different resolutions) through some convolution and splicing operations, which are respectively input into the detection head. Each detection head is usually composed of convolution layers, and finally outputs the classification branch and the regression branch

[0128] In the network training stage, the outputs of the multi-scale detection branch are jointly optimized using the regression loss and the classification loss; since the fine-grained subspace-aware attention module (K=16, K=8, K=4) is inserted at different depths in the present application, during the training process, the network can produce more stable and fine gradient updates for small insect sperm targets and overlapping areas, thereby significantly improving the detection accuracy and robustness.

[0129] In one specific embodiment, before inputting the input preset sperm target tracking model, it further includes:

[0130] Suppose that in the t-th frame image, YOLOv8 (containing attention module) detects N t mosquito sperm targets, and the boundary box of each target is represented as:

[0131]

[0132] At the same time, the detection confidence of each target is obtained

[0133] The network with fine-grained subspace-aware attention module at K=16, K=8, and K=4 outputs corresponding feature maps at different stages, which are represented as: wherein, represents the output feature map at K=16, and the latter two are the same;

[0134] For the i-th detected target, the ROI region corresponding to the target is intercepted from the original image to extract the features of the shallow layer CNN, and a basic feature vector is obtained. The spatial position corresponding to the target ROI is intercepted in , and a multi-scale target feature vector is obtained:

[0135]

[0136] In one specific embodiment, a multi-scale feature tracking module is introduced to track the vector As input, the deepsort algorithm model is improved to realize the tracking of insect sperm targets. The improved DeepSORT model enhances the robustness and continuity in the tracking process by fusing images and features of different scales, especially in the scene where the speed of insect sperm is fast and the motion trajectory changes greatly, effectively reducing the situation of losing targets or mismatching. The preset sperm target tracking model (improved DeepSORT model) specifically includes:

[0137] Given the Kalman filter state vector of the jth tracking trajectory at the t-1 frame (including target position, speed, etc.), the predicted state of the t frame is expressed as:

[0138]

[0139] Where A is the state transition matrix, w is the process noise, and in the detected targets of this frame, each candidate box will produce an observation vector Where represents the center point coordinates of the candidate box, represents the width and height of the candidate box, which is used for matching with the predicted trajectory. The formula is used to convert to for subsequent Kalman filtering and motion prediction:

[0140]

[0141] The predicted state and the observation vector are first screened by the "motion model", and the Mahalanobis distance is calculated:

[0142]

[0143] Where H is the observation matrix, S is the covariance matrix, and if d Mah exceeds the threshold θ, the matching pair is removed;

[0144] Among the matching candidates through the Mahalanobis distance, let the feature vector (or its historical mean) saved by the tracking trajectory j at the last successful matching time be Let the feature vector of the current detection target i be The cosine similarity is used to measure:

[0145]

[0146] dapp The smaller (i,j) is, the more similar the appearance feature is, and the more suitable it is to match the same target;

[0147] In the present application, the motion distance and the appearance distance can be weighted or uniformly normalized to construct a comprehensive matching cost:

[0148]

[0149] Wherein, λ is a weighting coefficient, and a global optimal matching scheme is found between the detected target i and the track j by the Hungarian algorithm;

[0150] If a track is not successfully matched to any detection result in several frames, it is marked as "lost" and a certain number of appearance feature history is retained, and when a new detection appears, the multi-scale feature is compared with the historical "lost" track feature:

[0151] d app (i,γ)<θ;

[0152] If the appearance distance is less than the threshold θ, it is determined that the detection i and the lost track γ belong to the same mosquito sperm target, and the tracking ID is recovered; for each successfully matched target of the t-th frame, the updated tracking state of the target is obtained and the corresponding target ID, forming a complete mosquito sperm motion track sequence.

[0153] Based on the divided model for training, mainly includes data preprocessing, model training, hyperparameter optimization, etc., including the following steps:

[0154] First, the collected training data is subjected to necessary preprocessing operations, including normalization of image pixel values and moderate data enhancement without damaging the key features of insect sperm, and the annotation information is also checked and standardized to ensure the accuracy and diversity of the training samples. On this basis, by setting appropriate optimization algorithms and hyperparameters, such as selecting an optimizer based on Adam, using loss functions for classification and regression tasks, and combining batch size, learning rate decay strategy, etc., the model is converged on the training set and gradually improves the accuracy in multiple Epoch iterations. After training, the corresponding weight file is saved.

[0155] Further, model testing and application, including the following steps:

[0156] The accuracy, recall rate and key indicators such as MOTA and MOTP in multi-target tracking of detection and tracking are evaluated by inferring the trained model on the divided test set, and the results are visualized to check the false detection, missed detection and false tracking. After verification, the trained model can finally be deployed in a laboratory or field observation environment to analyze the image or video stream of insect sperm in real time, and combined with biological needs to realize sperm motility evaluation, population distribution monitoring and other application scenarios.

[0157] An insect sperm detection and tracking system based on fine-grained subspace perception, comprising:

[0158] An image processing module acquires an insect sperm image, performs bounding box labeling on the insect sperm image, and the insect sperm image after bounding box labeling is used as a data set;

[0159] A target detection module trains a preset sperm target detection model through the data set;

[0160] A vector acquisition module combines the feature map output by the fine-grained subspace perception attention module in the sperm target detection model with the original insect sperm image after bounding box labeling to obtain a tracking vector;

[0161] A target tracking module inputs the target bounding box output by the detection head in the sperm target detection model and the tracking vector as input, inputs a preset sperm target tracking model, calculates Mahalanobis distance and appearance distance based on the predicted state and the observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and the appearance distance, calculates a global optimal matching scheme based on the comprehensive matching cost and the Hungarian algorithm, updates the trajectory state of the successfully matched target, and outputs a tracking result.

[0162] The insect sperm detection and tracking method and device based on fine-grained subspace perception have strong practicality, which is embodied in the following aspects:

[0163] Convenient operation: the operation process of the method and device is simple and clear, and the professional knowledge threshold requirement of the operator is relatively low. During the detection process, complex parameter adjustment and tedious preprocessing steps are not required, and a simple trained technician can operate skillfully, greatly saving the labor cost and time cost. For example, when using the device to detect samples, the operator only needs to place the sample at the specified position according to the established operation steps, start the detection program, and the device can automatically complete a series of operations such as image acquisition, analysis and result output, greatly improving the detection efficiency.

[0164] Device compatibility: The device of the present application can be compatible with a variety of common laboratory equipment, and can be flexibly integrated into different experimental environments. Whether it is a microscope, image acquisition equipment or data processing terminal, it can realize good docking and collaborative work with the device. This makes researchers do not need to purchase a large number of new equipment specially for using the detection and tracking method, reducing the research cost. At the same time, this compatibility also facilitates data sharing and method promotion between different laboratories, and promotes the research progress in related fields.

[0165] Wide application scenarios: In the field of biological research, it can be used to deeply explore the reproductive biological characteristics of insects, such as sperm production, maturation process and fertilization mechanism, etc., to provide key data support for understanding the reproductive behavior of insects. In terms of disease prevention and control, it can quickly and accurately detect the sperm characteristics of specific insect populations, which helps to track the changes of disease transmission media and provides basis for developing more effective prevention and control strategies. In addition, in ecological environment monitoring, through the detection and analysis of insect sperm in different environments, the influence of environmental changes on pest populations can be evaluated, providing scientific reference for ecological protection. Whether it is basic research or practical application, the method and device of the present application have shown great potential and practical value.

[0166] Extensibility: The method and device have good extensibility and can be upgraded and optimized according to different research needs and application scenarios. For example, on the basis of existing technology, artificial intelligence algorithms can be further integrated to improve the automation and accuracy of detection; or the detection function of other biological samples can be added to expand its application range. This extensibility enables the method and device of the present application to continue to play a role in the continuous development of scientific research and application needs, maintaining its advanced nature and practicality.

[0167] In one specific embodiment, on the Olympus body type microscope combined with E3ISPM micro-imaging camera platform, videos containing insect sperm under different light and background conditions are shot at a resolution of 1536*1024, as shown in Figure 3a 、 Figure 3b 、 Figure 3c After frame extraction and labeling, a data set is formed. In the model training stage, the batch size is selected as 16, the threshold θ is set as 0.8, the initial learning rate is set as 1e-4, and the cosine annealing strategy is combined, and 150 Epochs are iterated. When constructing the backbone network, K=16, K=8 and K=4 fine-grained subspace perception attention modules are inserted at three key depths of YOLOv8, and multi-scale context fusion is performed through SPPF layer.

[0168] All experiments were conducted on an NVIDIA GeForce RTX3090 GPU and an Intel Xeon Platinum 8280 CPU, and the programming environment was based on Python 3.8 and PyTorch 1.8.1.

[0169] The various embodiments in the specification are described in progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between various embodiments can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0170] The above description of disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for insect sperm detection and tracking based on fine-grained subspace-awareness, characterized in that, The method comprises the following steps: An insect sperm image is acquired, and a bounding box is labeled on the insect sperm image, and the insect sperm image after the bounding box labeling is taken as a data set; A preset sperm target detection model is trained through the data set; A feature map output by a fine-grained subspace perception attention module in the sperm target detection model is combined with the original insect sperm image after the bounding box labeling to obtain a tracking vector; A target bounding box output by a detection head in the sperm target detection model and the tracking vector are taken as inputs, and a preset sperm target tracking model is input, Mahalanobis distance and appearance distance are calculated based on a predicted state and an observation vector, a comprehensive matching cost is constructed through the Mahalanobis distance and the appearance distance, a global optimal matching scheme is calculated based on the comprehensive matching cost and a Hungarian algorithm, a trajectory state of a successfully matched target is updated, and a tracking result is output; The preset sperm target detection model specifically comprises a fine-grained subspace perception attention module with a dimension channel number of 16, 8 and 4 successively added after the second, third and fourth C2f modules of the original YOLOv8 model K a fine-grained subspace perception attention module with a dimension channel number of 16, 8 and 4 The fine-grained subspace perception attention module specifically comprises: The input feature map is divided into mutually exclusive subspaces:​ ； ； wherein, is the number of channels of each subspace, is the feature map of the th subspace, is the number of channels of the input feature, and are the height and width of the feature map, respectively; In each subspace, local significant features are extracted by using a deep convolution and a max-pooling operation: ； wherein, is a depth convolution kernel of the th sub-space, is a max-pooling operation with kernel size and stride 1; denotes a ReLU activation function, is a local salient feature extracted by the th sub-space. local salient features for each subspace Applying point-wise convolution and Softmax operation, the corresponding attention weights are generated: ； where, is the th point-wise convolution kernel of the th subspace, normalized over the spatial dimensions of the feature map, ensuring that: ； The original features are weighted by using the generated attention weights to obtain redistributed features: ； wherein: denotes element-wise multiplication, applying the attention map to the original features, denotes element-wise addition, preserving the original feature information through the residual connection; Finally, the redistributed features of all subspaces are spliced into output features: ； Before the input is input into the preset sperm target tracking model, the method further comprises the following steps: Assume that in the t-th frame image, the sperm target detection model detects a total of insect sperm targets, and the boundary box of each target is represented as: ； Simultaneously obtaining its detection confidence ; The network outputs the corresponding feature map at different stages as follows: the fine-grained subspace-aware attention module at K=16, K=8, and K=4: , , ; wherein, represents the output feature map at K=16. For the detected first For each target, the corresponding Region of Interest (ROI) is extracted from the original image and its features are extracted using a shallow CNN to obtain a basic feature vector. ;exist , , Extract the spatial location corresponding to the target ROI from the data to obtain the multi-scale target feature vector: , , The above feature vectors are processed to obtain the final tracking vector used for tracking. : ； The preset sperm target tracking model specifically comprises: Given the first tracking trajectory in the Kalman filter state vector of the first frame prediction state ； wherein, is the state transition matrix, is the process noise, in the detected targets in the current frame, each bounding box will generate an observation vector wherein denotes the center point coordinates of the bounding box, denotes the width and height of the bounding box, which is converted to using the formula : ； on the predicted state and the observation vector First, the Mahalanobis distance is calculated with the "motion model" screening ； wherein observation matrix, is a covariance matrix, if exceeds a threshold the matching pair is rejected; In the matching candidate by Mahalanobis distance, let the tracking trajectory The feature vector saved at the last successful matching is , let the feature vector of the current detection target be , and the cosine similarity is used for measurement: ； The smaller, the more similar the appearance features are, the more suitable for matching the same target; A comprehensive matching cost is constructed: ； wherein, is a trade-off coefficient, the global optimal matching scheme between the detected target and the trajectory is found by the Hungarian algorithm. If a certain trajectory is not successfully matched to any detection result within several frames, the trajectory is marked as "lost" and a certain number of appearance feature histories are reserved, and when a new detection appears, multi-scale features are compared with the historical "lost" trajectory features: ； If the appearance distance is less than a threshold Then determine the detection Same insect sperm target as the lost track And resume tracking the ID; for each successfully matched target in the first Frame, get its updated predicted state And the corresponding target ID, form a complete sequence of insect sperm movement tracks.

2. The method of claim 1, wherein, The bounding box labeling on the insect sperm image specifically comprises: ； wherein, is the coordinate of the top-left corner of the bounding box, is the coordinate of the bottom-right corner of the bounding box, for an image of width and height , the annotated bounding box satisfies: ； ； The coordinates of the labeled bounding box are converted from absolute coordinates to relative coordinates: ； ； ； ； wherein, , , , are the converted relative coordinates, respectively, and are the width and height of the image, respectively.

3. An insect sperm detection and tracking system based on fine-grained subspace perception, for the insect sperm detection and tracking method based on fine-grained subspace perception of any one of claims 1-2, characterized in that, The method comprises the following steps: An image processing module acquires an insect sperm image, labels a bounding box on the insect sperm image, and takes the insect sperm image after the bounding box labeling as a data set; A target detection module trains a preset sperm target detection model through the data set; A vector acquisition module combines a feature map output by a fine-grained subspace perception attention module in the sperm target detection model with an original insect sperm image after a bounding box labeling to obtain a tracking vector; A target tracking module takes a target bounding box output by a detection head in the sperm target detection model and the tracking vector as inputs, inputs a preset sperm target tracking model, calculates Mahalanobis distance and appearance distance based on a predicted state and an observation vector, constructs a comprehensive matching cost through the Mahalanobis distance and the appearance distance, calculates a global optimal matching scheme based on the comprehensive matching cost and a Hungarian algorithm, updates a trajectory state of a successfully matched target, and outputs a tracking result.

Images