A method and system for automatic recognition and statistical analysis of pollen grains and pollen tubes

Abstract

This invention relates to the field of image processing technology, specifically disclosing an automatic identification and statistical analysis method and system for pollen grains and pollen tubes. For pollen germination images, this method constructs an improved Mask R-CNN recognition model by combining channel attention and spatial attention mechanisms. The model's output mask image undergoes mask localization and Canny edge refinement processing, followed by skeleton extraction from the edge-refined mask image. This enables automatic calculation and batch output of key indicators such as the number of pollen grains, the number of pollen tubes, pollen tube length, and pollen germination rate in the image. It is widely applicable to plant reproductive biology research, pollen viability evaluation, crop breeding, and other fields.

Description

Technical Field

[0001] This invention belongs to the fields of image recognition and deep learning, and specifically relates to a method and system for automatic identification and statistical analysis of pollen grains and pollen tubes. Background Technology

[0002] Pollen grains are the male reproductive cells of seed plants, and pollen tubes are tubular structures that extend from pollen grains during pollen germination, used to transport sperm cells to the ovule. The germination rate (how many pollen grains successfully develop pollen tubes) and pollen tube length are important indicators of pollen viability and reproductive capacity. Traditionally, researchers typically observe pollen germination in culture media under a microscope, manually counting the number of germinating pollen grains and measuring pollen tube length. This manual method is not only time-consuming and labor-intensive but also easily affected by subjective factors, resulting in poor accuracy and consistency. Especially when processing large numbers of samples, manual detection methods are inefficient and prone to errors. To improve analytical efficiency and accuracy, recent years have seen research applying machine learning and image processing to pollen analysis. For example, some researchers have used the convolutional neural network YOLO to perform target detection on pollen microscopic images, automatically distinguishing between germinating and non-germinating pollen grains, thereby calculating the pollen germination rate; this deep learning-based automated detection method avoids manual counting, significantly improving efficiency and objectivity. Meanwhile, the more advanced instance segmentation algorithm Mask R-CNN is considered a leading technology in the field of object detection and segmentation. Mask R-CNN can generate pixel-level segmentation masks while detecting object positions, making it ideal for accurately segmenting pollen grains and pollen tubes in complex backgrounds. In some studies, Mask R-CNN models have been successfully trained for pollen image analysis, enabling rapid determination of pollen germination rate and pollen tube length. For example, literature reports the use of Mask R-CNN to analyze peony pollen micrographs. To improve the recognition rate of overlapping pollen tubes, the model was trained on artificially synthesized images containing crossed pollen tubes. The final model achieved an average precision of 0.949, and the correlation R² between the detection results of pollen germination rate and average pollen tube length and manual measurements reached 0.909 and 0.958, respectively. Furthermore, the model showed good versatility across other plant varieties. This demonstrates the enormous potential of deep learning-based image recognition technology in pollen viability detection.

[0003] However, existing methods still have some shortcomings. On the one hand, many studies remain at the experimental stage and have not provided complete application systems. For example, while the YOLO-based method mentioned above can automatically determine whether pollen has germinated, it cannot directly measure pollen tube length, nor does it output more in-depth statistical analysis data. On the other hand, even though Mask R-CNN can accurately identify pollen grains and pollen tubes, existing solutions often lack optimization for the unique morphology of pollen tubes, which are long, curved, and prone to overlapping, leading to decreased segmentation accuracy in complex scenes. Furthermore, the diverse shapes of detected pollen tubes present a significant technical challenge in extracting their length information. Thresholding segmentation and morphological methods from traditional image processing are sometimes used for pollen tube extraction, but they are prone to misjudgment when faced with multiple overlapping or connected pollen tubes, making it difficult to accurately distinguish and measure their length. In addition, existing automated analysis tools lack batch processing and result aggregation capabilities for large numbers of images, which is detrimental to the data analysis needs of high-throughput experiments. Therefore, there is an urgent need for an improved technical solution that can combine the high-precision recognition capabilities of deep learning, an optimization module for pollen tube characteristics, and subsequent image processing and statistical analysis modules to achieve automatic identification and comprehensive analysis of pollen grains and pollen tubes. Summary of the Invention

[0004] To address the shortcomings and problems of current methods in the identification and statistical analysis of pollen grains and pollen tubes, this invention provides an automatic identification and statistical analysis method and system for pollen grains and pollen tubes.

[0005] This invention provides a method for automatic identification and statistical analysis of pollen grains and pollen tubes, comprising the following steps: Step 1: Collect pollen culture images to be analyzed and preprocess the images; the pollen culture images are photographs of pollen germination taken under a microscope; Step 2: Introduce channel attention module and spatial attention module to improve the Mask R-CNN network model and build an improved Mask R-CNN instance segmentation model. Input the preprocessed pollen culture image into the improved Mask R-CNN instance segmentation model and output instances containing bounding box coordinates, category and pixel-level mask. Step 3: For instances identified as pollen grains, directly count the number of instances categorized as pollen grains to obtain the pollen grain count. For instances identified as pollen tubes, local image patches are cropped from the original image based on their bounding boxes, and the corresponding mask sub-images are obtained; Canny edge detection is performed on the local image patches to obtain local edge images, and the local edge images are fused with the instance mask; Step 4: Merge all masks belonging to the pollen tube category using a pixel-by-pixel logical OR operation to obtain a binary mask image containing the entire pollen tube region; fuse the binary mask image with the original color image, adjust the fusion transparency parameter, and output a black-background mask image and a color-marked image; Step 5: Convert the black mask image obtained in Step 4 into a grayscale image, and perform denoising and binarization processing to strictly segment the image into the foreground pollen tube region and the background; use a progressive thinning algorithm to extract the central skeleton of the foreground region, count the number of pollen tubes and the number of white pixels in the central skeleton image, and quantitatively calculate the total length and average length of the pollen tubes based on the number of white pixels.

[0006] The above-mentioned automatic identification and statistical analysis method for pollen grains and pollen tubes includes a preprocessing method in step one, which is either median filtering for noise reduction or adjustment of brightness and contrast.

[0007] The above-mentioned automatic identification and statistical analysis method for pollen grains and pollen tubes, the improved Mask R-CNN instance segmentation model mentioned in step two includes: The backbone network unit is used to extract features from the input image and output feature maps at different scales; The feature pyramid network unit is used to connect feature maps of different scales output by the backbone network module to obtain multi-scale feature maps. An attention unit is applied to perform channel attention enhancement and spatial attention enhancement on the multi-scale feature map output by the feature pyramid network module to obtain an attention-enhanced feature map. The region candidate network unit is used to generate a series of candidate boxes from the attention-enhanced feature map. Then, the ROIAlign operation maps the candidate regions onto features of a uniform scale and feeds them into the classification head. Through non-maximum suppression and confidence thresholding, several detection instances containing class, bounding box coordinates and pixel-level masks are obtained.

[0008] The above-described method for automatic identification and statistical analysis of pollen grains and pollen tubes includes a classification head comprising a classification branch, a regression branch, and a mask branch. The classification branch outputs the class probability of pollen grains and pollen tubes in each candidate region; the regression branch outputs the bounding box position; and the mask branch outputs a probability map of each pixel in the candidate region belonging to the corresponding target class.

[0009] The above-mentioned automatic identification and statistical analysis method for pollen grains and pollen tubes also includes, in step three, expanding or refining the fused mask to eliminate false breaks and ensure the connectivity of the pollen tube region.

[0010] In the above-mentioned method for automatic identification and statistical analysis of pollen grains and pollen tubes, after fusing the binary mask image with the original color image in step four, the area covered by the mask is filled with a bright color to highlight it, while the non-masked areas are darkened or set to black, resulting in an intuitive marked image in which the pollen tube area is highlighted and other areas are weakened.

[0011] In the above-mentioned method for automatic identification and statistical analysis of pollen grains and pollen tubes, step five uses a distance-transform-based Euclidean skeleton algorithm to extract the central skeleton of the foreground region.

[0012] In the above-mentioned automatic identification and statistical analysis method for pollen grains and pollen tubes, the total length of the pollen tube in step five is the actual length corresponding to the sum of the pixels of the pollen tube skeleton in the microscope image.

[0013] The above-mentioned method for automatic identification and statistical analysis of pollen grains and pollen tubes includes, in step five, the calculation of pollen germination rate based on the number of pollen tubes and pollen grains.

[0014] This invention also provides an automatic identification and statistical analysis system for pollen grains and pollen tubes, comprising, The image acquisition and preprocessing module is used to acquire pollen culture images and preprocess them. The deep learning recognition module includes a backbone network unit for extracting features from the input image and outputting feature maps at different scales; a feature pyramid network unit for connecting the feature maps at different scales output by the backbone network module to obtain multi-scale feature maps; an application attention unit for performing channel attention enhancement and spatial attention enhancement on the multi-scale feature maps output by the feature pyramid network module to obtain attention-enhanced feature maps; and a region candidate network unit for generating a series of candidate boxes from the attention-enhanced feature maps, then mapping the candidate regions to features of a uniform scale through the ROI Align operation, feeding them into the classification head, and filtering them through non-maximum suppression and confidence thresholding to obtain several detection instances containing category, bounding box coordinates, and pixel-level masks. The mask localization and edge refinement module is used to localize pollen tube instances and perform Canny edge detection on local image blocks to obtain local edge images. The local edge images are then fused with the instance mask to obtain an edge-refined mask image. The mask processing and image enhancement module is used to merge masks belonging to the pollen tube category and fuse them with the original color image to perform black mask enhancement on the original image, and output a black background mask image and a color marker image; The skeleton extraction module is used to extract the central skeleton of the pollen tube from the mask image of the pollen tube. The image result storage module is used to store the black background mask image, the color marker image, and the skeleton image output by the mask processing and image enhancement module; The result calculation and output module is used to count the number of pollen grains based on the pollen grain mask image and to calculate and output the number of pollen tubes, the total length of pollen tubes, and the average length of pollen tubes based on the central skeleton of pollen tubes.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention addresses the unique morphology of pollen tubes—slender, curved, and prone to crossing—by combining an attention mechanism and the Canny edge detection algorithm to optimize the standard Mask R-CNN deep learning model, resulting in an improved Mask R-CNN deep learning model. Based on this improved model, automatic detection and segmentation of pollen grains and pollen tubes are achieved. During the recognition process, image enhancement measures such as median filtering for noise reduction and mask highlighting are incorporated, improving detection accuracy and result readability. It effectively enhances the model's focus on key features and boundary refinement, improving recognition accuracy and robustness. It can reliably identify weak or overlapping pollen tubes, reducing missed and false detections, and ensuring accurate and reliable counting results. Especially in complex backgrounds and pollen tube crossing scenarios, segmentation performance and accuracy are significantly improved.

[0016] This invention extracts the skeleton of the pollen tube mask and obtains the central skeleton that reflects the true path of the pollen tube through a thinning algorithm. The length of the pollen tube is then quantified, which can accurately reflect the curve length of the pollen tube. Even if the tube is curved and tangled, the length of the pollen tube can be accurately calculated, avoiding the errors that may be caused by measuring the straight distance only, and the calculation results are more accurate. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a diagram of pollen skeleton extraction according to the present invention; in the diagram, A is the input image, B is the original output, C is the binary mask of the pollen tube, and D is the pollen tube skeleton. Detailed Implementation

[0018] Existing technologies for the detection and analysis of pollen grains and pollen tubes have the following problems: I. Manual analysis is inefficient and has limited accuracy. Traditional methods relying on manual microscopic counting and measurement are time-consuming and labor-intensive, and prone to statistical errors and subjective biases. With large batches of samples, manual work is difficult to complete in a timely manner, and accuracy is hard to guarantee.

[0019] II. Limitations of Traditional Image Processing Methods. Image processing schemes based on simple threshold segmentation struggle to effectively distinguish pollen grains from pollen tubes in pollen images. This is especially true when pollen tubes intersect or are adjacent to pollen grains, leading to adhesion and unclear identification, resulting in inaccurate counting and length measurements.

[0020] Third, existing automatic detection solutions are incomplete. While some existing solutions utilizing machine learning or deep learning (such as YOLOv5) can automatically identify pollen grains and determine their germination status to a certain extent, they only obtain limited information such as germination rate and cannot directly measure richer indicators such as pollen tube length. Advanced algorithms like Mask R-CNN have been applied in experimental research, but currently lack optimization and improvements for the slender and curved morphology of pollen tubes, resulting in insufficient segmentation accuracy in overlapping scenarios. Furthermore, there is a lack of integrated systems that utilize the identification results for comprehensive analysis.

[0021] Third, the output of results data is insufficient. Existing technologies do not provide a mechanism for automatically summarizing and outputting detection and measurement results. Users often need to manually record the counts and measurement results for each image and then perform statistical summaries, a cumbersome and error-prone process that cannot meet the needs of high-throughput data processing.

[0022] The aforementioned shortcomings have resulted in the lack of an efficient, accurate, and comprehensive automated analysis tool in the field of pollen viability detection. This invention addresses these issues by proposing an automatic identification and statistical analysis method for pollen grain and pollen tube images that integrates improved deep learning instance segmentation, attention mechanisms, edge enhancement algorithms, and image processing techniques, thereby overcoming the deficiencies of existing technologies.

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0024] Example 1: This example provides an automatic identification and statistical analysis method for pollen grains and pollen tubes. This method integrates an improved Mask R-CNN deep learning model with attention mechanisms, Canny edge detection algorithms, and image processing algorithms to achieve accurate detection of pollen grains and pollen tubes in microscope pollen images, extraction of pollen tube length, and automatic calculation and output of key indicators. Figure 1 As shown, the specific content is as follows: Step 1: Image Acquisition and Preprocessing Collect the pollen culture results images to be analyzed into the system's input directory and categorize them into folders according to experimental batches or sample groups. Images can be photographs of pollen germination taken under a microscope, and can be in common formats such as JPEG, PNG, and BMP.

[0025] To improve detection accuracy, images can be selectively enhanced using methods such as median filtering for noise reduction or adjusting brightness and contrast. Preprocessing ensures image quality meets standards, facilitating subsequent analysis by deep learning models.

[0026] Step 2: Improve the Mask R-CNN network model by introducing channel attention and spatial attention modules to construct an improved Mask R-CNN instance segmentation model. Input the preprocessed pollen culture image into the improved Mask R-CNN instance segmentation model, and the output will include bounding box coordinates, category, and pixel-level mask for each instance. This mainly includes: (1) Feature extraction and attention enhancement Feature extraction is performed on the preprocessed input image using a backbone network (such as ResNet-50 or a similar backbone network). A feature pyramid network is then used to connect the features extracted by the backbone network to generate multi-scale feature maps. Simultaneously, a channel attention module is applied to weight the response of each feature channel, enhancing the weights of key channels such as the slender structure of pollen tubes and the outline of pollen grains, while suppressing background noise channels. A spatial attention module is applied to generate a weight map in the spatial dimension of the feature map, making the network more spatially focused on the region where pollen grains / pollen tubes are located, while weakening non-target regions such as culture medium textures and blemishes.

[0027] (2) Candidate region generation and instance segmentation The attention-enhanced feature map is input into the region candidate network to generate a series of candidate boxes. The ROIAlign operation then maps the candidate regions onto features of a uniform scale and feeds them into the classification branch, regression branch, and mask branch. The classification branch outputs the class probability of each candidate region to distinguish between pollen grains and pollen tubes. The regression branch refines the bounding box positions. The mask branch outputs the probability map of each pixel in the candidate region belonging to the corresponding target category.

[0028] By filtering through non-maximum suppression (NMS) and confidence thresholding, several final detection instances are obtained, each containing bounding box coordinates, class labels, and pixel-level masks.

[0029] Step 3: Mask Localization and Edge Refinement For each instance identified as a pollen tube, a local image patch is cropped from the original image based on its bounding box, and the corresponding mask sub-image is obtained; then, Canny edge detection is performed on the local image patch to obtain a local edge image, which contains edge information related to the pollen tube. The local Canny edge is fused with the instance mask. On the one hand, the mask is used to constrain the Canny edge and remove interfering edges that fall outside the mask. On the other hand, the mask is refined and shrunken near the mask boundary according to the edge response so that the mask outline fits the real pollen tube boundary. Optionally, an expansion or refinement operation can be performed on the fused mask to eliminate pseudo-fractures and ensure the connectivity of the pollen tube region.

[0030] Compared to performing Canny edge detection on the entire image during the preprocessing stage, the mask-guided local edge refinement method of this invention has the following advantages: 1. More thorough noise suppression: Global edge detection in the preprocessing stage extracts all textures, bubbles, and blemishes from the culture medium as edges, which can easily interfere with subsequent instance segmentation. In contrast, this invention first uses Mask R-CNN for high-level semantic segmentation, and then performs Canny in the local region of each mask. Most of the background is directly masked by the mask, and noisy edges are basically not included in the refining process.

[0031] 2. Better boundary and semantic consistency: Global Canny obtains pure intensity edges, which cannot distinguish between "pollen tube edges" and "other brightness variations". By fusing "mask + Canny", the mask provides semantic prior, and Canny is only responsible for sub-pixel level correction near the mask edge. This retains the semantic discrimination ability of deep learning and utilizes the fine contour information of traditional edge detection.

[0032] 3. Enhanced segmentation capabilities in overlapping scenarios: In complex scenarios such as pollen tube intersections and entanglements, relying solely on threshold / edge preprocessing can easily cause multiple pollen tubes to stick together into one region. By first segmenting different instances and then refining their respective local masks, each pollen tube is processed independently within its own mask, naturally isolating different instances and significantly reducing adhesion and missegmentation in intersection areas.

[0033] 4. Provides more accurate input for skeleton extraction: The pollen tube mask refined by Canny is closer to the real tubular structure in width and shape, making the center line of the subsequent skeleton extraction smoother and more connected, avoiding extra branches or breaks in the skeleton at the edge "bulge".

[0034] To obtain an improved Mask R-CNN model with good generalization ability, the following implementation method can be adopted: Collect several hundred original pollen images from microscopes, and construct a certain number of synthetic cross pollen tube images as needed to enhance the robustness of the model in cross and overlapping scenarios. We used annotation tools to perform pixel-level instance annotation of pollen grains and pollen tubes, and performed enhancement processing such as random flipping, brightness perturbation, blurring and noise reduction on the data to expand the data volume to thousands of images. The augmented data is divided into training and validation sets. During training, the loss and mAP of the validation set are monitored. When the mAP50 is stable above 0.9 and the validation loss no longer decreases significantly, the optimal weights are selected as the inference model. In practical applications, the system loads the trained model weights (mask_rcnn_pollen_detect.h5) upon startup and enables GPU acceleration as needed to batch infer the microscopic images in the input catalog. It outputs the bounding box, category, and final mask refined by Canny for each instance, providing input for subsequent mask fusion, skeleton extraction, and statistical analysis modules.

[0035] Step 4: Masking and Image Enhancement Since the output mask is a Boolean matrix, it marks the pixel range of each detected target in the image. A mask overlay enhancement step was designed for the pollen tube detection results, specifically: First, all masks belonging to the pollen tube category are merged by a pixel-by-pixel logical OR operation to obtain a binary mask image containing the entire pollen tube region (pollen tube pixel value is 255, background is 0). Then, the binary mask image is fused with the original color image. The areas covered by the mask are filled with a bright color (such as green) for highlighting, while the unmasked areas are darkened or set to black. By adjusting the fusion transparency parameter, a visually intuitive labeled image is obtained, where the pollen tube area is highlighted and other background areas are weakened. This processing is equivalent to applying a black mask enhancement to the original image, focusing attention on the pollen tube area. This step generates two types of image outputs: one is a black-background mask image, where the pollen tube area is white and the background is black, used for subsequent skeleton extraction; the other is a color-labeled image, where colored borders and semi-transparent mask highlights are superimposed on the original image to label the detected pollen grains and pollen tubes, used for visual display of the results. In the color-labeled image, the position of each pollen grain and pollen tube is marked with a rectangular border and labeled with the category name next to it. The mask outline of the pollen tube is drawn with polygonal lines, clearly showing the details of the recognition results.

[0036] Step 5: Extract the skeleton from the masked image of the pollen tube using image morphology analysis, and accurately measure the length of the pollen tube. The steps are as follows: First, the obtained pollen tube black mask image is converted into a grayscale image and then subjected to appropriate noise reduction. In this embodiment, a 3×3 window median filter is used to smooth the image to eliminate minor noise and artifacts.

[0037] Then, the smoothed image is binarized, and a fixed threshold (such as grayscale value 140) is used to strictly segment the image into foreground (pollen tube region) and background.

[0038] Subsequently, a progressive refinement algorithm was used to extract the central skeleton of the foreground region. By counting the number of white pixels in the skeleton image, the total length of the pollen tube was quantitatively obtained.

[0039] As a feasible example, a distance-transform-based Euclidean skeleton algorithm (central axis algorithm) is used to ensure that the obtained skeleton path lies on the central axis of the pollen tube, thus accurately reflecting the length and shape of the tubular structure. The skeleton extraction process shrinks the originally thicker pollen tube mask into connected lines of single-pixel width, fully preserving the direction and extension length of the pollen tube. For example... Figure 2 As shown, in the image obtained by skeleton extraction, the white connected lines represent the central axis of each pollen tube.

[0040] The total length of pollen tubes can be quantitatively determined by counting the number of white pixels in the skeleton image. Typically, a single pixel in a microscope image corresponds to a certain actual length (calibrated according to the microscope magnification, e.g., 760 pixels equals 1 mm), therefore the number of skeleton pixels is directly proportional to the actual pollen tube length. If multiple pollen tubes exist in an image, this invention considers the sum of all pollen tube skeleton pixels as the total pollen tube length of the image. Combined with the previously identified number of pollen tubes, the average length of each pollen tube can be calculated.

[0041] It should be noted that the method for extracting length in this step has significant advantages over simple bounding box measurement: skeleton tracking can accurately adapt to the curvature of pollen tubes. Even if the pollen tube is not straight but meandering, the total number of skeleton pixels can still accurately reflect the length of its curved path and will not underestimate the length due to curvature.

[0042] After completing the pollen grain count, pollen tube count, and pollen tube length extraction for each image, the result calculation and output module automatically calculates the relevant indicators. This invention defines and calculates the following key parameters: Pollen count: The total number of pollen grains detected in the image (including ungerminated and germinated grains).

[0043] Pollen tube count: The number of pollen tubes detected in the image.

[0044] Total pollen tube length: The sum of the number of pixels of all pollen tubes in the image obtained from the skeleton extraction, used to represent the total length of all pollen tubes.

[0045] Average pollen tube length: The total pollen tube length is divided by the number of pollen tubes to obtain the average length of each pollen tube (if there are no pollen tubes in a certain image, the average length is recorded as 0).

[0046] Pollen germination rate: The ratio of the number of pollen tubes to the number of pollen grains multiplied by 100% represents the percentage of pollen grains that have germinated in the image (if the number of pollen grains is 0, the germination rate is recorded as 0).

[0047] The aforementioned metrics are calculated in real-time for each image and stored in the system's data log. After processing each batch (each input folder), the system further summarizes the data from all images within that batch, calculating the average germination rate and average pollen tube length of the samples in that batch. This provides overall statistical metrics for each sample group, facilitating comparative analysis under different treatment conditions or varieties.

[0048] Example 2: This example provides an automatic identification and statistical analysis system for pollen grains and pollen tubes. The system includes: The image acquisition and preprocessing module is used to acquire pollen culture images and preprocess them. The deep learning recognition module includes a backbone network unit for extracting features from the input image and outputting feature maps at different scales; a feature pyramid network unit for connecting the feature maps at different scales output by the backbone network module to obtain multi-scale feature maps; an application attention unit for performing channel attention enhancement and spatial attention enhancement on the multi-scale feature maps output by the feature pyramid network module to obtain attention-enhanced feature maps; and a region candidate network unit for generating a series of candidate boxes from the attention-enhanced feature maps, then mapping the candidate regions to features of a uniform scale through the ROI Align operation, feeding them into the classification head, and filtering them through non-maximum suppression and confidence thresholding to obtain several detection instances containing category, bounding box coordinates, and pixel-level masks. The mask localization and edge refinement module is used to localize pollen tube instances and perform Canny edge detection on local image blocks to obtain local edge images. The local edge images are then fused with the instance mask to obtain an edge-refined mask image. The mask processing and image enhancement module is used to merge masks belonging to the pollen tube category and fuse them with the original color image to perform black mask enhancement on the original image, and output a black background mask image and a color marker image; The skeleton extraction module is used to extract the central skeleton of the pollen tube from the mask image of the pollen tube. The image result storage module is used to store the black background mask image, the color marker image, and the skeleton image output by the mask processing and image enhancement module; The result calculation and output module is used to count the number of pollen grains based on the pollen grain mask image and to calculate and output the number of pollen tubes, the total length of pollen tubes, and the average length of pollen tubes based on the central skeleton of pollen tubes.

[0049] For each input image, the system saves three types of result images: (a) a black-background mask image, used to highlight the pollen tube region; (b) a color-marked image, visually displaying the detection and marking results; and (c) a skeleton image, showing the extracted pollen tube skeleton for length verification. These result images are stored in corresponding subfolders (black, color, skeletonization) in the output directory, with file names consistent with the original images for easy comparison.

[0050] In addition, the system can generate an Excel report file containing detailed data. This report includes at least two worksheets: a "Summary" table listing overall indicators such as average germination rate and average pollen tube length for each sample folder (batch); and a "Log Data" table recording information such as the filename, folder, number of pollen grains, number of pollen tubes, pollen tube length (number of white pixels in the skeleton), average pollen tube length, germination rate (%), and processing time (seconds) for each image. By viewing the Excel spreadsheet, users can easily obtain the statistical results of all experimental samples and clearly compare the differences in pollen germination characteristics under different experimental conditions. The output format of this invention is compatible with commonly used data analysis software, supporting further use of the results for statistical analysis or chart creation. It comprehensively covers the entire process from image reading, target detection, mask processing, length extraction, data calculation to result output, and can be automatically executed by a computer without manual intervention. Anyone skilled in the art can implement this invention by following the steps and module descriptions above, achieving efficient identification and accurate analysis of pollen grains and pollen tubes. The system based on this invention allows for the development of a complete software workflow, enabling batch processing of multiple sets of images and automatically generating intuitive labeled images and Excel data reports from the recognition and analysis results. End-to-end automatic output significantly improves data processing efficiency, allowing users to obtain structured results without manual processing.

[0051] In practical applications, the method of this invention can be implemented through software programs, such as relying on the Python programming environment. Core dependencies include open-source libraries such as OpenCV (for image reading, writing, and basic processing, including Canny edge detection), the Mask R-CNN model framework (an implementation based on TensorFlow with an integrated attention module), NumPy, and Pandas (for numerical computation and tabular data processing). The skeleton extraction algorithm can be implemented using the existing image processing library DIPlib. It is preferred to run on a computer with GPU acceleration to speed up model inference, but with moderate amounts of data, analysis can be completed using a regular CPU. Improving the Mask R-CNN model requires training with pre-annotated image data of pollen grains and pollen tubes to achieve ideal detection performance. The training data should cover as much as possible the diversity of different lighting conditions, backgrounds, and pollen tube morphologies (including length, curvature, and intersections) to improve the model's generalization ability. If applied to new plant species, additional training may be needed to identify different morphologies of pollen grains and pollen tubes. To improve the accuracy of pollen tube length measurement, the conversion relationship between pixels and actual length can be calibrated using a scale image of known scale, converting the number of skeleton pixels into actual length units.

[0052] The above description is merely a preferred embodiment of the present invention and does not limit the invention. In practical applications, the method of the present invention can be used not only for pollen viability determination in plant reproductive biology research, such as comparing the effects of different treatment conditions on pollen germination and screening high-viability pollen varieties, but also for high-throughput pollen quality assessment in agricultural breeding practices, improving breeding screening efficiency. Besides pollen analysis, the concept of this scheme can also be extended to the analysis of other biological images with elongated structures, such as root hair growth measurement and fungal hyphae length statistics, simply by training corresponding models for different objectives. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatic identification and statistical analysis of pollen grains and pollen tubes, characterized in that: Includes the following steps: Step 1: Collect pollen culture images to be analyzed and preprocess the images; the pollen culture images are photographs of pollen germination taken under a microscope; Step 2: Introduce channel attention module and spatial attention module to improve the Mask R-CNN network model and build an improved Mask R-CNN instance segmentation model. Input the preprocessed pollen culture image into the improved Mask R-CNN instance segmentation model and output instances containing bounding box coordinates, category and pixel-level mask. Step 3: For instances identified as pollen grains, directly count the number of instances categorized as pollen grains to obtain the pollen grain count. For instances identified as pollen tubes, local image patches are cropped from the original image based on their bounding boxes, and the corresponding mask sub-images are obtained; Canny edge detection is performed on the local image patches to obtain local edge images, and the local edge images are fused with the instance mask; Step 4: Merge all masks belonging to the pollen tube category using a pixel-by-pixel logical OR operation to obtain a binary mask image containing the entire pollen tube region; fuse the binary mask image with the original color image, adjust the fusion transparency parameter, and output a black-background mask image and a color-marked image; Step 5: Convert the black mask image obtained in Step 4 into a grayscale image, and perform denoising and binarization processing to strictly segment the image into the foreground pollen tube region and the background; use a progressive thinning algorithm to extract the central skeleton of the foreground region, count the number of pollen tubes and the number of white pixels in the central skeleton image, and quantitatively calculate the total length and average length of the pollen tubes based on the number of white pixels.

2. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: The first preprocessing step can be either median filtering for noise reduction or adjusting brightness and contrast.

3. The pollen grain and pollen tube automatic identification and statistical analysis method according to claim 1, characterized in that: The improved Mask R-CNN instance segmentation model described in step two includes: The backbone network unit is used to extract features from the input image and output feature maps at different scales; The feature pyramid network unit is used to connect feature maps of different scales output by the backbone network module to obtain multi-scale feature maps. An attention unit is applied to perform channel attention enhancement and spatial attention enhancement on the multi-scale feature map output by the feature pyramid network module to obtain an attention-enhanced feature map. The Region Candidate Network Unit is used to generate a series of candidate boxes from the attention-enhanced feature map. Then, the ROI Align operation maps the candidate regions onto features of a uniform scale and feeds them into the classification head. Through non-maximum suppression and confidence thresholding, several detection instances containing class, bounding box coordinates and pixel-level masks are obtained.

4. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 3, characterized in that: The classification head includes a classification branch, a regression branch, and a mask branch. The classification branch outputs the class probability of pollen grains and pollen tubes in each candidate region; the regression branch outputs the bounding box position. The mask branch outputs a probability map showing the probability that each pixel in the candidate region belongs to the corresponding target category.

5. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: Step three also includes expanding or refining the fused mask to eliminate false breaks and ensure the connectivity of the pollen tube region.

6. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: In step four, after fusing the binary mask image with the original color image, the area covered by the mask is filled with a bright color to highlight it, while the non-masked areas are darkened or set to black, resulting in an intuitive marked image where the pollen tube area is highlighted and other areas are weakened.

7. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: In step five, the Euclidean skeleton algorithm based on distance transformation is used to extract the central skeleton of the foreground region.

8. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: The total length of the pollen tube mentioned in step five is the actual length corresponding to the sum of the pixels of the pollen tube skeleton in the microscope image.

9. The method for automatic identification and statistical analysis of pollen grains and pollen tubes according to claim 1, characterized in that: Step five also includes calculating the pollen germination rate based on the number of pollen tubes and pollen grains.

10. An automatic identification and statistical analysis system for pollen grains and pollen tubes, characterized in that: include, The image acquisition and preprocessing module is used to acquire pollen culture images and preprocess them. The deep learning recognition module includes a backbone network unit, which is used to extract features from the input image and output feature maps at different scales. The feature pyramid network unit is used to connect feature maps of different scales output by the backbone network module to obtain multi-scale feature maps; the application attention unit is used to perform channel attention enhancement and spatial attention enhancement on the multi-scale feature maps output by the feature pyramid network module to obtain attention-enhanced feature maps. The region candidate network unit is used to generate a series of candidate boxes from the attention-enhanced feature map. Then, the ROI Align operation maps the candidate regions onto features of a uniform scale and feeds them into the classification head. Through non-maximum suppression and confidence thresholding, several detection instances containing category, bounding box coordinates and pixel-level masks are obtained. The mask localization and edge refinement module is used to localize pollen tube instances and perform Canny edge detection on local image blocks to obtain local edge images. The local edge images are then fused with the instance mask to obtain an edge-refined mask image. The mask processing and image enhancement module is used to merge masks belonging to the pollen tube category and fuse them with the original color image to perform black mask enhancement on the original image, and output a black background mask image and a color marker image; The skeleton extraction module is used to extract the central skeleton of the pollen tube from the mask image of the pollen tube. The image result storage module is used to store the black background mask image, the color marker image, and the skeleton image output by the mask processing and image enhancement module; The result calculation and output module is used to count the number of pollen grains based on the pollen grain mask image and to calculate and output the number of pollen tubes, the total length of pollen tubes, and the average length of pollen tubes based on the central skeleton of pollen tubes.

Images