Microbial target detection method based on improved YOLOv8s network in microscopic scene

Abstract

The invention discloses a microbial target detection method based on an improved YOLOv8s network in a microscopic scene. The method comprises the following steps: constructing a target image data set; preprocessing the target image data set; extracting image features; fusing image features; and predicting the image. The microbial target detection method based on the improved YOLOv8s network comprises the step of replacing a YOLOv8s backbone network with SMA-YOLOv8s. In an SMA-YOLOv8s network, in a feature extraction part, a space-to-depth conversion mode is adopted to realize lossless downsampling of a low-contrast microbial image and complete feature extraction among different scales, a multi-scale feature fusion module based on a cascade mode is designed at a neck part, and a dynamic weight optimization regression branch loss function is used in a detection head. Therefore, the improved network can effectively extract the fine-grained features of the low-contrast microbiological image, and the capability of representing the context information of an occluded target is enhanced. The recognition and positioning capability of the microbial target in the microscopic scene is enhanced, and compared with a traditional algorithm, the universality is better.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, specifically to the field of target detection, and in particular to a microbial target detection method based on an improved YOLOv8s network in a microscopic scene. Background Art

[0002] As the most basic and abundant life form on Earth, microorganisms play important roles in ecosystems, such as organic matter degradation, nutrient cycling, and the spread of pathogenic microorganisms. Analyzing the movement characteristics of microorganisms can help us understand their ecological adaptability, pathogenic mechanisms, and potential applications in the environment and industry. However, manual analysis of microbial movement characteristics relies on the operator's experience and is subjective and non-reproducible. Therefore, the use of automated analysis methods to quantitatively study microbial movement processes can significantly improve the reproducibility and accuracy of biological research, which is of great significance to research in fields such as life sciences.

[0003] With the rapid development of deep learning and microscopic imaging technologies, deep learning-based multi-target detection has been widely applied in practical tasks such as biomedical image detection and microbial behavior analysis. Deep learning has the ability to automatically extract multi-level features from data, capturing both detailed information and global features during image processing, thereby maintaining excellent detection performance even in complex backgrounds and changing conditions. Compared to traditional manual calibration methods, deep learning possesses stronger generalization capabilities and can adapt to microbial images under different conditions, reducing calibration time, saving labor costs, and improving the accuracy of microbial target detection.

[0004] Although researchers have proposed many excellent multi-target detection algorithms and applied them to various fields, compared to mainstream multi-target detection tasks such as pedestrians and vehicles, microbial images under microscopes have low image contrast and mutual occlusion between objects, which poses challenges to microbial target detection and tracking methods. Therefore, it is necessary to further optimize target detection algorithms to make them suitable for microbial image detection tasks.

[0005] YOLOv8s adopts an end-to-end structure and can simultaneously predict the location and category of the target directly from the input image, eliminating intermediate steps such as candidate region generation. This improves detection efficiency while maintaining high accuracy, meeting the real-time requirements of microbial multi-target detection and tracking methods for microscopic scenes. Therefore, this paper selects YOLOv8s as the basic detection framework. Summary of the Invention

[0006] The purpose of the present invention is to address the problems existing in existing detection of microorganism images and to provide a multi-target detection method for microorganisms in microscopic scenes with low detection cost, high accuracy and good universality.

[0007] To achieve the above objectives, the present invention adopts a microscopic scene-oriented multi-target microbial detection and tracking method, which includes the following steps: Step S1: using a microscopic imaging device to collect microbial images, and annotating the collected microbial images to construct a target image dataset; Step S2: preprocessing the target image dataset and dividing the preprocessed image dataset into a training set and a validation set; Step S3: Image feature extraction, wherein the image feature extraction is to downsample the features using a space-to-depth conversion method, thereby achieving lossless downsampling of low-contrast microbial images and generating multi-scale features; Step S4: Image feature fusion. The image feature fusion is designed for P3 scale features. Multi-scale features are refined through cascade grouping. The scale features are aligned using spatial scale transformation to achieve feature fusion in the depth direction. Step S5: Image prediction. The image prediction is to decouple the target detection head into the regression branch and the classification branch after feature fusion, and add dynamic weights based on the C-IoU loss function of the regression branch to enhance the focus on occluded targets with low confidence, and output the microbial target detection results.

[0008] In the above scheme, step S1 constructs a target image dataset, which includes using a microscopic imaging device to capture images of microorganisms under different staining conditions, different lighting conditions, and different density concentrations, and annotating the captured images with location, category, and ID information.

[0009] In the above scheme, the specific implementation method of step S2 for preprocessing the target image data set includes random horizontal flipping, random vertical flipping, random cropping, random scaling, random rotation, random brightness adjustment and random contrast adjustment. The data preprocessing process also includes corresponding conversion of labels.

[0010] In the above scheme, the image feature extraction in step S3 performs a total of 4 space-to-depth conversion operations on the original microbial image during the detection process to complete feature extraction between different scales and the gradual conversion of fine-grained information to semantic information; the specific process of the space-to-depth conversion method includes dividing the features using a scale with a step size of 2 to achieve downsampling of the length and width scales of the features, splicing the divided features along the channel dimension and using multiple 1x1 convolutions to fit the features along the channel dimension, thereby fitting all the features of the microbial image and completing the downsampling operation.

[0011] In the above scheme, the image feature fusion in step S4 is designed based on the characteristics of small and dense microbial targets under a microscopic perspective, and the P3 layer features are selected as the output end of the multi-scale feature fusion module; in the feature fusion process, the features from different scales are refined by cascade grouping, and the self-attention mechanism is introduced in each group to enhance the expression ability of the features of different scales. The refined features are aligned to the P3 scale size through feature scale transformation, and further through three-dimensional convolution operation and normalization processing, feature fusion is completed in the depth direction.

[0012] In the above scheme, after feature extraction and feature fusion, the image prediction in step S5 outputs the target detection result through a dual-branch decoupling head, which consists of a regression branch and a classification branch. The regression branch is used to locate microbial targets. In the present invention, dynamic weight optimization of the CIoU loss function is used to enhance the attention to occluded targets with low confidence. The classification branch uses the BCE loss function to realize the classification of microbial categories.

[0013] In summary, compared with the prior art, the present invention has the following advantages: 1) downsampling the image by space-to-depth conversion, reducing the loss of fine-grained features while ensuring effective feature downsampling, thereby better capturing the detailed features of the microbial image; 2) constructing a multi-scale feature fusion method at the neck, refining features of different scales by cascade grouping, and fusing multi-scale features by spatial scale transformation, thereby enhancing the expression ability of contextual information; 3) optimizing the regression branch loss function by dynamic weights, thereby enhancing the attention to occluded targets with low confidence; 4) the present invention has a wide range of applicability, high precision and strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of the microbial target detection method based on the improved YOLOv8s network in the microscopic scene in the example;

[0015] Figure 2 Schematic diagram of the improved YOLOv8s network structure in the embodiment;

[0016] Figure 3 Schematic diagram of the multi-scale feature fusion module of the improved YOLOv8s network structure in the embodiment. Specific implementation plan

[0017] In order to make the purpose, technical solution and beneficial effects of the present invention more clear, the present invention is further described in detail below in conjunction with specific embodiments and accompanying drawings. It should be understood that the embodiments and accompanying drawings are only used to illustrate the principles and concepts of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0018] In this embodiment, combined with Figure 1 , provides a microbial multi-target detection and tracking method for microscopic scenes, the method comprising the following steps: Step S1: using a microscopic imaging device to collect microbial images, and annotating the collected microbial images to construct a target image dataset; Step S2: preprocessing the target image dataset and dividing the preprocessed image dataset into a training set and a validation set; Step S3: Image feature extraction, wherein the image feature extraction is to downsample the features using a space-to-depth conversion method, thereby achieving lossless downsampling of low-contrast microbial images and generating multi-scale features; Step S4: Image feature fusion. The image feature fusion is designed for P3 scale features. Multi-scale features are refined through cascade grouping. The scale features are aligned using spatial scale transformation to achieve feature fusion in the depth direction. Step S5: Image prediction. The image prediction is to decouple the target detection head into the regression branch and the classification branch after feature fusion, and add dynamic weights based on the C-IoU loss function of the regression branch to enhance the focus on occluded targets with low confidence, and output the microbial target detection results.

[0019] Furthermore, in this embodiment, the step S1 constructs a target image dataset, which is visualized using a CCD color camera under a Zeiss standard optical microscope, and the collected microbial images are labeled with the category and location information of the microbial targets using Labelme software, and are reviewed.

[0020] Furthermore, in this embodiment, the target image dataset in step S2 is preprocessed, including random horizontal flipping, random vertical flipping, random cropping, random scaling, random rotation, random brightness adjustment and random contrast adjustment, and information conversion of the corresponding image labels. The image dataset is divided into a training set and a validation set in a ratio of 7:3.

[0021] Furthermore, in this embodiment, in the feature extraction of the image in step S3, the downsampling process uses a space-to-depth conversion layer to downsample the feature X, classify the sliced ​​features according to the scale factor, and splice the sub-feature maps along the channel dimension to obtain an intermediate feature map X′. After the space-to-depth conversion layer, the original feature map of size (S, S, C1) is converted to a size of (S / Scale, S / Scale, Scale 2C1) intermediate feature map X′, after the feature conversion layer, the non-stepped convolution layer is used to convert X′ into the final feature map X″ (S / Scale, S / Scale, C2). In the process of microbial target detection, four spatial to depth conversion operations are performed on the original microbial image in sequence, and a multi-scale feature pyramid structure P1 to P5 from high resolution to low resolution is gradually constructed. Among them, the P1 feature layer retains the spatial resolution of the original image and is used to extract the fine edge information and local texture features of the target. On this basis, four SPD transformation operations are performed on the P1 feature map in sequence, and the transformation scale factor is set to r = 2. Each SPD operation reduces the spatial size to half that of the previous layer and expands it along the channel dimension. This method generates feature maps P2 to P5, whose spatial sizes are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image, respectively, and the channel dimension is expanded to 64 times, 128 times, 256 times, and 512 times the original image, respectively. Through this continuous SPD downsampling operation, layer-by-layer feature extraction from high-resolution details to low-resolution semantics is achieved. The constructed multi-scale feature pyramid structure can fully integrate local detail information with global contextual semantics, providing a robust multi-scale feature representation for subsequent microbial target detection and classification tasks.

[0022] Furthermore, in this embodiment, the image feature fusion step S4 Figure 3The network structure of the multi-scale feature fusion module first cascades and groups the multi-scale feature layers. The cascade method gradually refines the feature representation, thereby effectively increasing the depth of the network, and then can capture deeper microbial image features, enhance feature expression capabilities and fusion effectiveness. After extracting information of different scales through the cascade attention mechanism, in order to better capture and fuse multi-level features and improve the utilization of multi-scale information of microorganisms, the extracted multi-scale features are aligned to the P3 scale. Because the P3 layer contains most of the information that is crucial for small target detection, its feature map has high spatial resolution, rich detail information and moderate contextual information. Compared with feature maps of higher levels, the P3 layer retains more image details and can capture small targets more accurately. Among them, the P4 and P5 scale features are aligned to the P3 scale feature map size using the nearest neighbor interpolation method, while P2 is aligned to the P3 scale size using the space-to-depth conversion method to ensure that all scale feature maps have the same resolution as P3. Next, the dimensionality of each feature layer is increased through the unsqueeze method, converting it from a 3D tensor [height, width, channel] to a 4D tensor [depth, height, width, channel]. These 4D feature maps are then concatenated along the depth dimension to form a 3D feature map for subsequent convolution operations. Finally, 3D convolution, normalization, and the SiLU activation function are used to complete scaled sequence feature extraction.

[0023] Furthermore, in this embodiment, after feature extraction and feature fusion, the image prediction in step S5 outputs the target detection result through a dual-branch decoupling head, which consists of a regression branch and a classification branch. The regression branch is used to locate microbial targets. In the present invention, a dynamic weight is used to optimize the CIoU loss function to enhance the focus on occluded targets with low confidence. The classification branch uses the BCE loss function to classify microbial categories.

[0024] Furthermore, in the above scheme, step S5 is used to decode the output results of the model to generate target detection prediction results. The decoding process includes: parsing the coordinate offset, width and height scaling factor and confidence score in the model output, combining the predefined anchor frame, calculating the actual position and size of the detection frame, and normalizing the relevant parameters through the Sigmoid function. Subsequently, the non-maximum suppression (NMS) algorithm is used to screen the candidate detection frames, retain the optimal detection frame, and eliminate redundant prediction results. In the model training stage, a multi-task joint loss function is used to optimize the network. The AdamW optimizer is introduced during the training process, and the cosine annealing learning rate scheduling strategy and gradient clipping technology are combined to improve the stability and convergence of the training process.

[0025] The present invention adopts a single-stage target detection method and improves it with the YOLO network as the basic model, making it suitable for microbial target detection tasks in microscopic image scenes. This method introduces a space-to-depth conversion mechanism in the downsampling process to achieve lossless downsampling of low-contrast microbial images and complete the extraction of multi-scale features at the same time. A specific processing module is designed for the P3 scale feature, and the multi-scale feature expression is further refined through cascade grouping. The scale features are aligned using a spatial scale transformation strategy to achieve feature fusion in the depth direction, thereby improving the detection robustness and accuracy in target occlusion scenarios. In summary, compared with existing methods, the detection algorithm proposed in the present invention has significant advantages in detection accuracy and adaptability.

[0026] The above description is merely a preferred embodiment of the present invention, which is intended to illustrate the technical principles of the present invention and is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications, equivalent replacements, or improvements may be made to the above embodiments without departing from the spirit and essence of the present invention, and all of these should be included in the scope of protection of the present invention.

Claims

1. A microbial target detection method based on an improved YOLOv8s network in a microscopic scene, characterized in that: The method comprises the following steps: Step S1: using a microscopic imaging device to collect microbial images, and annotating the collected microbial images to construct a target image dataset; Step S2: preprocessing the target image dataset and dividing the preprocessed image dataset into a training set and a validation set; Step S3: Image feature extraction, wherein the image feature extraction is to achieve lossless downsampling of low-contrast microbial images and complete feature extraction at different scales by using a space-to-depth conversion method; Step S4: Image feature fusion. The image feature fusion is designed for P3 scale features. Multi-scale features are refined through cascade grouping. Scale features are aligned using spatial scale transformation to achieve feature fusion in depth. Step S5: Image prediction. The image prediction is to decouple the target detection head into the regression branch and the classification branch after feature fusion, and add a dynamic weight ω based on the C-IoU loss function of the regression branch. conf , to enhance the attention to occluded targets with low confidence and output microbial target detection results.

2. The microbial target detection method based on the improved YOLOv8s network in a microscopic scene according to claim 1 is characterized in that: The step S1 constructs a target image dataset, which includes using a microscopic imaging device to capture images of microorganisms under different staining conditions, different lighting conditions, and different density concentrations, and annotating the captured images with location, category, and ID information.

3. The microbial target detection method based on the improved YOLOv8s network in a microscopic scene according to claim 1 is characterized in that: The specific implementation method of step S2 for preprocessing the target image data set includes random horizontal flipping, random vertical flipping, random cropping, random scaling, random rotation, random brightness adjustment and random contrast adjustment. The data preprocessing process also includes corresponding conversion of labels.

4. The microbial target detection method based on the improved YOLOv8s network in a microscopic scene according to claim 1 is characterized in that: In the step S3, the image feature extraction performs a total of four space-to-depth conversion operations on the original microbial image during the detection process to complete feature extraction between different scales and the gradual conversion of fine-grained information to semantic information. The specific process of the space-to-depth conversion method includes dividing the features using a scale with a step size of 2 to achieve downsampling of the length and width scales of the features, splicing the divided features along the channel dimension and using multiple 1x1 convolutions to fit the features along the channel dimension, thereby fitting all the features of the microbial image and completing the downsampling operation.

5. The microbial target detection method based on the improved YOLOv8s network in a microscopic scene according to claim 1 is characterized in that: The image feature fusion in step S4 is designed based on the characteristics of small and dense microbial targets under the microscopic perspective, and the P3 layer feature is selected as the output end of the multi-scale feature fusion module; During the feature fusion process, a cascade grouping approach is used to refine features from different scales. A self-attention mechanism is introduced within each group to enhance the expressiveness of features at different scales. The refined features are aligned to the P3 scale through feature scale transformation, and further feature fusion is completed in the depth direction through three-dimensional convolution operations and normalization.

6. The microbial target detection method based on the improved YOLOv8s network in a microscopic scene according to claim 1 is characterized in that: Step S5: Image prediction After feature extraction and feature fusion, the target detection result is output through a dual-branch decoupling head, which includes a regression branch and a classification branch. The regression branch is used to locate the microbial target. In the present invention, the dynamic weight ω is used. conf The CIoU loss function is optimized to enhance the attention to occluded targets, and the classification branch uses the BCE loss function to achieve the classification of microorganisms.

Images