Method for analyzing grape pollen viability and germination rate based on double fluorescence staining and deep learning

Abstract

The application discloses a grape pollen vitality and germination rate analysis method based on double fluorescence staining and deep learning, relates to the technical field of image processing, and aims to overcome the defects of strong artificial dependence of existing pollen vitality and germination rate detection, and comprises the following steps: after grape pollen is dyed with fluorescein diacetate and propidium iodide double fluorescence, the pollen is inoculated into a multi-well plate germination medium, and FITC channel images, TRITC channel images and bright field images of the same field of view are continuously collected; a Mask R-CNN model is used for pollen detection, instance segmentation and initial state recognition of each frame of image; cross-frame target correlation is performed on the results of each frame, multi-modal time sequence image segments are extracted, and a 3D convolutional neural network is input to extract space-time evolution features, so that the final state of each pollen at the terminal time is determined, and the terminal vitality rate and the terminal germination rate are output. The application can realize automatic, high-throughput and high-precision analysis of pollen vitality and germination rate, and has variety adaptability.

Description

Technical Field

[0001] This invention relates to the fields of biotechnology and image processing technology. More specifically, this invention relates to a method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning. Background Technology

[0002] Grape pollen viability and germination rate are key physiological indicators for evaluating grape variety fertility, guiding the configuration of hybrid breeding combinations, and predicting orchard fruit set levels. Accurate and efficient measurement of pollen viability and germination rate is of significant practical importance for grape germplasm resource identification, new variety breeding, and production cultivation management.

[0003] Currently, conventional methods for pollen viability testing mainly fall into two categories: staining and in vitro germination. Staining utilizes the reaction of specific dyes with active substances or membrane structures within pollen cells, and the viability level is assessed by observing the reaction under an optical microscope and manually counting the staining ratio. In vitro germination involves sowing pollen onto a solid or liquid culture medium containing suitable nutrients, culturing it at a constant temperature for a certain period, observing under a microscope whether pollen tubes have extended, and manually counting the number of germinated pollen. Both methods have significant shortcomings in practice. First, whether it's staining judgment or germination observation, current mainstream operations still heavily rely on manual visual judgment and manual counting, which is not only labor-intensive and inefficient, but also susceptible to the influence of observer visual fatigue and inconsistent subjective judgment standards, resulting in a lack of good repeatability and comparability of data between different researchers and even between different batches by the same researcher. Second, although some laboratories have introduced general image processing software for semi-automatic assisted counting, the operation process is cumbersome, has stringent image quality requirements, and is difficult to process large numbers of samples in batches, failing to meet the high-throughput detection needs of modern breeding work.

[0004] Therefore, there is an urgent need to design a technical solution that can overcome the above-mentioned defects. Summary of the Invention

[0005] One objective of this invention is to provide a method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning, aiming to overcome the shortcomings of existing technologies such as strong reliance on manual intervention, susceptibility to misjudgment from static single observations, and difficulty in achieving temporal tracking of single pollen.

[0006] To achieve these objectives and other advantages of the present invention, according to one aspect of the present invention, a method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning is provided, comprising: S1: taking grape pollen, suspending it in buffer solution, adding fluorescein diacetate and propidium iodide, incubating in the dark, centrifuging, washing and resuspending to obtain a stained pollen suspension; S2: adding the stained pollen suspension dropwise to the germination medium of a multi-well plate, placing the multi-well plate on the stage of a fluorescence microscope, and capturing FITC channel images, TRITC channel images and bright field images of the same field of view at set time intervals within a preset time interval to generate a multimodal time-series image dataset; S3: using a Mask... The R-CNN model segments each frame of the multimodal temporal image dataset, locates the detection box for each pollen grain, and identifies the initial state category, which includes dead pollen, live pollen, and germinating pollen. S4: Based on a standard library of polar-to-equatorial axis ratios for grape varieties, the detection boxes are geometrically verified. S5: Cross-frame target association is performed on the geometrically verified detection results to obtain the correspondence of each pollen grain at each collection time point. Based on this, multimodal temporal image segments are extracted and input into a 3D convolutional neural network to extract spatiotemporal evolution features. Based on the spatiotemporal evolution features, the final state of each pollen grain at the endpoint is determined. The final state includes dead pollen, live pollen, and germinating pollen. The proportions of dead pollen, live pollen, and germinating pollen at the endpoint are calculated to account for the total number of pollen grains, and the endpoint viability rate and endpoint germination rate are output.

[0007] Further, in S1, grape pollen was taken and suspended in W5 solution to obtain a pollen suspension. Fluorescein diacetate and propidium iodide were added to make the final concentration of both in the pollen suspension 50-150 μg / mL. The pollen suspension was incubated in the dark at 20-30℃ for 10-30 minutes. After incubation, the pollen suspension was centrifuged, the supernatant was discarded, and W5 solution was added to the precipitate to resuspend it until the volume was equal to the initial volume of the pollen suspension, thus obtaining the stained pollen suspension.

[0008] Further, a germination medium was prepared by dissolving 15% (w / v) sucrose, 100 mg / L boric acid, and 20 mg / L calcium chloride in deionized water. The germination medium was injected into each well of a multi-well plate. The stained pollen suspension was added dropwise to the surface of the germination medium in each well of the multi-well plate, with a dropwise amount of 20-50 μL per well. The multi-well plate was placed on the stage of a fluorescence microscope, and the stage temperature was set and maintained at 25-28℃.

[0009] Further, in S2, after placing the multi-well plate on the stage of the fluorescence microscope, the microscope objective lens is adjusted to a magnification of 10-20x. Within a preset time interval of 0-6 hours, an acquisition interval of 10-15 minutes is set. Under the condition of keeping the stage position unchanged to always capture the same field of view, FITC channel images, TRITC channel images, and bright field images are acquired. After each acquisition, the excitation light source is turned off, so that the multi-well plate is kept in a dark culture state during the interval between two adjacent acquisitions. All acquired FITC channel images, TRITC channel images, and bright field images are arranged in time series to generate a multimodal time-series image dataset.

[0010] Further, S3 includes: S31: For a FITC channel image, a TRITC channel image, and a bright field image at the same time and in the same field of view in the multimodal temporal image dataset, the bright field image, FITC channel image, and TRITC channel image are treated as three independent channels and concatenated along the channel dimension to construct a three-channel composite feature input map. The first channel of the three-channel composite feature input map is the bright field morphology channel, the second channel is the green fluorescence channel, and the third channel is the red fluorescence channel; S32: The three-channel composite feature input map is input into the Mask R-CNN model. The R-CNN model uses ResNet-101 as the backbone network and is supplemented by a feature pyramid network for multi-scale feature extraction. A region proposal network generates candidate detection boxes. The bounding box regression branch and classification branch output the detection box position, initial state category, and corresponding classification confidence score for each pollen grain. Simultaneously, the mask branch outputs an instance segmentation mask for each pollen grain on the three-channel composite feature input map. S33: If the classification confidence score is higher than a preset first confidence threshold, the initial state category output by the classification branch is used as the initial state category determination result for the pollen. If the classification confidence score is lower than or equal to the first confidence threshold, the pollen is marked as pollen to be verified. S34: Output the detection box position, instance segmentation mask, and initial state category determination result for each pollen grain in each frame image.

[0011] Further, S4 includes: S41: Reading a pre-constructed and stored standard library of polar-to-equatorial axis ratios, which contains standard polar-to-equatorial axis ratios and corresponding tolerance ranges for multiple grape varieties; S42: For each detection box in each frame of the image output in step S3, calculating the measured ratio of the polar axis length to the equatorial axis length of the pollen grains within the detection box; S43: Retrieving the corresponding standard polar-to-equatorial axis ratio and tolerance range from the standard library according to the grape variety corresponding to the grape pollen; S44: For each detection box, comparing the measured ratio of the detection box with the standard... The ratio of the polar axis to the equatorial axis is compared: if the measured ratio falls within the allowable deviation range, the detection frame is deemed to have passed the geometric verification, and the detection frame and its corresponding initial state category are retained; if the measured ratio exceeds the allowable deviation range but is within the preset correctable deviation range, an affine transformation is applied to the detection frame to scale the measured ratio of the detection frame to the standard polar axis to equatorial axis ratio, and the original detection frame is replaced with the scaled detection frame, thus the geometric verification of the detection frame is deemed to have passed; if the measured ratio exceeds the correctable deviation range, the geometric verification of the detection frame is deemed to have failed, and the detection frame is marked as a false detection and discarded.

[0012] Furthermore, for the pollen to be verified marked in S33, after the detection frame containing it passes the geometric verification or correction in S44, the average gray values ​​of the green fluorescence channel and the red fluorescence channel are re-extracted based on the verified / corrected detection frame and the ratio is calculated. If the ratio is greater than the first ratio threshold, it is finally determined to be live pollen; if it is less than the second ratio threshold, it is determined to be dead pollen; if it is between the two, its initial state category is corrected to pending. If the detection frame containing it is determined to fail the geometric verification in step S44 and is removed, the pollen to be verified is marked as an invalid target and removed from subsequent analysis.

[0013] Furthermore, in S5, for each detection box in each frame image retained after geometric verification, the centroid coordinates, width, and height of the detection box are extracted as the position and morphological features of the corresponding pollen grain in the frame image. Taking two temporally adjacent frames as a group, for each pollen grain in the previous frame and each pollen grain in the next frame, based on the Euclidean distance between the centroid coordinates of the two detection boxes, the overlap ratio of the two detection boxes, and the contour similarity of the instance segmentation masks within the two detection boxes, it is determined whether the two grains are the same pollen target, thus establishing inter-frame relationships. The correspondence of pollen grains is as follows: Each pollen grain is assigned a unique identification number. After a successful cross-frame match, the pollen grain retains the same identification number at all time points in which it appears. The detection box information and instance segmentation mask at each time point are sequentially assigned to the trajectory sequence corresponding to the identification number. After the trajectory sequence is constructed, the detection box region and instance segmentation mask of each identification number at each acquisition time point are extracted along the time sequence according to the identification number of each pollen grain. They are then combined in chronological order to form the multimodal temporal image fragments of each pollen grain.

[0014] Furthermore, for each pollen grain's multimodal temporal image segment, the FITC channel image, TRITC channel image, and bright field image of each frame arranged in chronological order are cropped using the detection box region of each pollen grain as the spatial cropping range to extract detection box sub-images at each time point. The FITC channel sub-image, TRITC channel sub-image, and bright field sub-image at each time point are then concatenated along the channel dimension to form a single-frame three-channel input block. These single-frame three-channel input blocks from all time points are then stacked along the time dimension to construct a 4D tensor with dimensions of time length × number of channels × height × width, which serves as the spatiotemporal input data for the pollen grain. The 4D tensor is then input into a 3D convolutional neural network. The 3D convolutional neural network sequentially comprises multiple 3D convolutional layers, 3D batch normalization layers, 3D pooling layers, and fully connected layers. Convolutional operations are performed synchronously along the temporal and spatial dimensions to extract the spatiotemporal evolution features of each pollen grain over time in terms of green fluorescence intensity, red fluorescence intensity, and morphological changes. The output is a fixed-length feature vector corresponding to the end time of a preset time interval. The fixed-length feature vector is input into the classification layer and processed by the Softmax activation function to output the probability values ​​of the pollen grain belonging to the three final states of dead pollen, live pollen, and germinating pollen at the end time. The category corresponding to the highest probability value is taken as the final state determination result of the pollen grain at the end time.

[0015] The present invention has at least the following beneficial effects: This invention utilizes the Mask R-CNN deep learning model for automatic instance segmentation and initial state identification of pollen grains in multimodal images. Combined with geometric verification and cross-frame target association, it achieves a fully automated process from image acquisition to viability / germination rate output. This completely eliminates the reliance on manual observation and counting in traditional methods, removing detection biases caused by subjective human factors and enabling high-throughput rapid analysis of large batches of samples. The invention continuously acquires multiple frames of temporal images from the same field of view within a preset time interval and uses a 3D convolutional neural network to simultaneously extract the spatiotemporal evolution features of pollen along both temporal and spatial dimensions to determine the final state. This method fully considers the biological characteristics of individual differences in pollen germination initiation time, capturing the dynamic changes of pollen throughout the germination process. It avoids misjudgments caused by improper observation timing, significantly improving the accuracy and reliability of viability and germination rate evaluation.

[0016] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0017] Figure 1 This is a flowchart of one embodiment of this application. Detailed Implementation

[0018] The present invention will now be described in further detail so that those skilled in the art can implement it based on the description.

[0019] It should be understood that terms such as "having," "comprising," and "including" used in the embodiments of this application do not exclude the presence or addition of one or more other elements or combinations thereof. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of components in a specific posture. If the specific posture changes, the directional indication will also change accordingly. When an element is referred to as "fixed to" or "set on" another element, it can be directly on the other element or may have an intervening element present. When an element is referred to as "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element through an intervening element. Descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features.

[0020] It should be noted that the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0021] In one embodiment, such as Figure 1 As shown, the first step is to prepare the pollen by staining. The grape pollen sample to be tested is placed in a near-neutral buffer solution to uniformly disperse the pollen particles and form a suspension. Two fluorescent dyes, fluorescein diacetate and propidium iodide, are added to this suspension. fluorescein diacetate can permeate intact cell membranes into living cells, where it is hydrolyzed by esterases to produce a green fluorescent product. Propidium iodide, however, cannot permeate intact cell membranes and can only enter damaged dead cells, where it binds to nucleic acids to produce red fluorescence. The differential labeling of the two dyes allows for the differentiation between metabolically active pollen and pollen with damaged membranes. The dye-containing suspension is incubated at room temperature in the dark for a period of time. Then, residual free dye in the supernatant is removed by centrifugation, and the precipitate is resuspended in fresh buffer to obtain a stained pollen suspension with low background fluorescence. The resulting stained pollen suspension is then inoculated into a multi-well plate containing the carbon source and mineral elements required for pollen germination.

[0022] A multi-well plate was stably placed on the stage of a fluorescence microscope. The microscope light source and filter group were adjusted. While keeping the stage position fixed, images were cyclically acquired at fixed time intervals within a preset time range starting from the inoculation time. During each acquisition, the FITC fluorescence channel was switched sequentially to capture green fluorescence signals, and the TRITC fluorescence channel was switched to capture red fluorescence signals. Corresponding transmitted light images were acquired under bright-field lighting, thus obtaining multimodal images containing both fluorescence and morphological information within the same field of view. Images from all acquisition points were arranged chronologically to construct a multimodal time-series image dataset. Each frame in the dataset was input into a pre-trained MaskR-CNN deep learning model for instance segmentation. This model can detect each individual pollen grain in the image, marking its spatial location in the form of a detection box, and outputting the model's initial state category determination result for the target within the detection box. The initial state categories include dead pollen, live pollen, and germinating pollen.

[0023] Since pollen grains of different grape varieties have different morphological characteristics, the ratio of their polar axis length to equatorial axis length is variety-specific. Therefore, a database containing standard ratios and allowable deviation ranges for multiple grape varieties is established in advance. The polar axis and equatorial axis of pollen grains in each detection frame output by the model are measured separately, and the measured ratio is calculated and compared with the standard ratio of the corresponding variety in the database for geometric verification. If the deviation exceeds the allowable range, the detection frame is corrected or removed.

[0024] For the detection results of each frame retained after geometric verification, a target matching algorithm is used to perform cross-frame association processing in the time dimension. That is, based on the spatial proximity and morphological similarity of pollen grains between adjacent frames, the detection results of each frame belonging to the same pollen grain are concatenated to form the trajectory sequence of each pollen individual within the complete observation period. Subsequently, based on the trajectory sequence, the local image regions of each pollen grain at all time points are extracted from the multimodal time-series image dataset, and combined in chronological order to form multimodal time-series image segments of that pollen. These segments are input into a pre-trained three-dimensional convolutional neural network, which can perform convolution operations simultaneously along the time and spatial dimensions to extract the dynamic changes in fluorescence intensity and morphological evolution features of pollen during germination.

[0025] Finally, the three-dimensional convolutional neural network outputs the probability that the pollen belongs to one of three states—dead pollen, live pollen, or germinating pollen—at the final moment based on the extracted spatiotemporal evolution features. The pollen with the highest probability is taken as the final state. The number of pollen grains in the final state of dead pollen, live pollen, and germinating pollen is counted, and the proportion of each of the three states to the total number of pollen grains is calculated to obtain the final viability rate and the final germination rate.

[0026] In existing technologies, pollen viability and germination rate are generally detected by visual counting under a microscope after artificial staining. Operators need to observe each field of view and manually click to count, which not only has low throughput but is also prone to counting errors due to visual fatigue after prolonged operation, resulting in poor consistency of statistical results among different researchers. This embodiment introduces deep learning instance segmentation technology into pollen image analysis, enabling automatic pollen target localization and identification. Furthermore, by constructing a multimodal temporal image dataset and a cross-frame target association strategy, it achieves continuous tracking of the same pollen population throughout the germination process, overcoming the shortcomings of single static observation which is prone to misjudgment due to differences in individual germination initiation times. A geometric verification mechanism based on variety-specific morphological parameters effectively eliminates interference from non-pollen particles, and the extraction of spatiotemporal evolution features by a three-dimensional convolutional neural network fully utilizes temporal information, making the determination of the endpoint state more comprehensive and reliable.

[0027] In one embodiment, W5 solution is used as a buffer suspension medium for pollen staining. W5 solution is an isotonic solution containing sodium chloride, potassium chloride, calcium chloride, and 2-morpholine ethanesulfonic acid. Its osmotic pressure is essentially the same as that within grape pollen cells, maintaining the physiological activity of the pollen during staining. Collected raisin pollen is added to W5 solution and dispersed thoroughly by vortexing or repeated blowing to form a homogeneous pollen suspension. Stock solutions of fluorescein diacetate and propidium iodide are added to this pollen suspension, ensuring a final concentration of both dyes between 50 μg / mL and 150 μg / mL, optionally 50 μg / mL or 100 μg / mL. Too low a concentration results in a weak fluorescence signal that is difficult to distinguish from the background, while too high a concentration may produce non-specific staining or increase background noise. The pollen suspension with added dye was incubated in a constant temperature environment of 20°C to 30°C in the dark for 10, 20, or 30 minutes to allow the dye molecules to fully penetrate the pollen cells and bind to the target substance. After incubation, the centrifuge tube containing the pollen suspension was placed in a centrifuge and centrifuged at an appropriate speed to allow the pollen particles to settle to the bottom of the tube. The supernatant was then carefully aspirated with a pipette to remove any excess dye that failed to bind to the cells. An equal volume of W5 solution was added to the precipitate in the centrifuge tube, and the precipitate was resuspended by repeated pipetting, thus completing the preparation of the stained pollen suspension.

[0028] In one embodiment, during the culture medium preparation stage, sucrose, boric acid, and calcium chloride are weighed out. The amount of sucrose is 15 g per 100 mL of culture medium, i.e., a mass-volume fraction of 15% (w / v). The amount of boric acid is 100 mg / L, and the amount of calcium chloride is 20 mg / L. All three are dissolved in deionized water and stirred until completely dissolved to obtain the germination culture medium. In this medium, sucrose acts as a carbon source and osmotic pressure regulator, providing energy and metabolic substrates for pollen germination and adjusting the osmotic pressure of the medium to match that within the pollen cells. Boric acid promotes the synthesis and elongation of the pollen tube cell wall, playing a crucial role in the germination process. Calcium ions, as a second messenger in signal transduction, participate in regulating vesicle transport and cytoskeleton remodeling during pollen tube apical growth. The prepared germination medium is poured into each well of a multi-well plate. The multi-well plate can be 6-well, 12-well, or 24-well, and the amount of medium poured into each well should be such that the liquid surface is level and does not overflow. Using a micropipette, the stained pollen suspension is added dropwise above the liquid surface of each well. The amount added per well can be set to 20 μL or 50 μL, allowing the pollen grains to settle naturally and distribute evenly on the liquid surface. After placing the multi-well plate on the stage of a fluorescence microscope, the temperature of the stage is set and maintained between 25°C and 28°C using the stage's thermostat. This temperature range is suitable for grape pollen germination.

[0029] In one embodiment, during image acquisition, after adjusting the stage of the fluorescence microscope equipped with a multi-well plate to a horizontal position, the microscope objective magnification is selected to be 10x to 20x. Optionally, a 10x objective can be used to obtain a larger field of view to detect more pollen, or a 20x objective can be used to obtain higher spatial resolution to clearly distinguish the protrusion state of pollen tubes. The preset time interval for the entire image acquisition process is set from the time of pollen inoculation to 6 hours, because the germination process of most grape pollen is basically completed within 6 hours under suitable conditions. Within this time interval, images are acquired every 10 to 15 minutes. Optionally, it can be set to acquire images every 10 minutes to obtain more dense time-series data, or every 15 minutes to reduce data storage pressure. During each acquisition, the position of the stage in the X and Y directions is locked by the microscope's motorized stage locking function to ensure that the field of view of each acquisition is completely consistent in space. Under the automatic control of the microscope's accompanying software, the excitation light of the FITC channel was triggered sequentially to acquire a green fluorescence image, followed by the excitation light of the TRITC channel to acquire a red fluorescence image. Then, the system switched to a bright-field illumination source and acquired a bright-field image. These three images constituted a complete set of data for that moment. After each acquisition, the excitation and bright-field light sources were immediately shut off via software control, ensuring the multi-well plate was completely protected from light during the intervals between acquisitions. This was to prevent photobleaching caused by prolonged illumination, which would reduce the fluorescence signal intensity, and also to avoid phototoxic effects on the normal pollen germination process. All images acquired at each acquisition moment were numbered and archived according to their time sequence, forming a multimodal time-series image dataset for subsequent analysis.

[0030] In one embodiment, for a set of images belonging to the same acquisition time in a multimodal temporal image dataset—namely, a FITC channel image, a TRITC channel image, and a bright-field image—it needs to be integrated into a format suitable for deep learning model input. Specifically, using the channel stitching function of image processing software, the bright-field image is used as the first channel input, which contains morphological information such as the edge contour and internal texture of pollen grains; the FITC channel image is used as the second channel input, which carries a green fluorescence signal reflecting the esterase activity within pollen cells; and the TRITC channel image is used as the third channel input, which carries a red fluorescence signal reflecting the integrity of the pollen cell membrane. These three channels are stitched together along the channel dimension at the matrix level to generate a three-channel composite feature input map, where each pixel position corresponds to the signal value of the same spatial location in three different modalities.

[0031] The three-channel composite feature map is fed into the Mask R-CNN model, whose backbone network uses a ResNet-101 structure. This is a residual network with 101 convolutional layers, capable of effectively extracting high-level image semantic features. Above the backbone network, the model also includes a feature pyramid network, which fuses feature maps of different scales through a top-down lateral connection structure, enabling the model to detect pollen grains of different sizes simultaneously. The region proposal network slides across the multi-scale feature maps output by the feature pyramid to generate candidate detection boxes, and uses non-maximum suppression to filter out the candidate boxes most likely to be pollen targets. For each candidate box, the bounding box regression branch is responsible for adjusting its position and size to obtain more accurate localization, while the classification branch outputs the probability score of the candidate box belonging to three categories: dead pollen, live pollen, and germinating pollen. Simultaneously, the mask branch performs pixel-level segmentation on the feature region corresponding to each candidate box, outputting an instance segmentation mask of the pollen grain in the image. This mask can accurately delineate the outline of the pollen grain and its pollen tube.

[0032] Set a first confidence threshold, which can be set to 0.7, 0.8, or 0.9, etc. If the maximum class probability output by the classification branch is higher than the threshold, the class is directly used as the initial state class determination result; if the maximum class probability is lower than or equal to the threshold, it indicates that the model has doubts about the reliability of the detection result, and the pollen is marked as pollen to be verified, to be further confirmed in subsequent steps with other information.

[0033] The locations of the detection boxes for all pollen in each frame, the instance segmentation mask, and the initial state category determination results are output in the form of structured data for use in subsequent geometric verification steps.

[0034] This embodiment employs the Mask R-CNN model and constructs a three-channel composite input image, making full use of the morphological information of bright field, the activity information of green fluorescence, and the membrane integrity information of red fluorescence. The three complement each other, enabling the model to more accurately distinguish between live pollen, dead pollen, and germinating pollen. The introduction of instance segmentation mask provides pixel-level contour data for subsequent accurate calculation of pollen tube length and morphological changes.

[0035] In one embodiment, a standard library of polar-to-equatorial axis ratios is pre-built and stored. This library is a database file or lookup table containing the standard polar-to-equatorial axis ratios for multiple grape cultivars, along with the permissible deviation range for each cultivar. The data in this standard library is derived from the average and standard deviation obtained by precisely measuring the pollen of a large number of known cultivar purelines under a microscope.

[0036] For each detection box in each frame of the output image, the instance segmentation mask output by Mask R-CNN within that detection box is first read. This mask is a binary image, where pixel regions with a value of 1 correspond to the actual parts of pollen grains. Ellipse fitting is then performed on this mask region, i.e., the minimum bounding ellipse of the region is calculated. The length of the major axis of this ellipse is the polar axis length of the pollen grain, and the length of the minor axis is the equatorial axis length. The ratio of these two is the measured ratio.

[0037] Based on the name of the grape variety to be tested entered by the user, the standard ratio and tolerance range corresponding to the variety are retrieved from the standard library. The tolerance range can be expressed as a multiple of the standard ratio ± the standard deviation, such as the standard ratio ± two times the standard deviation, or the standard ratio ± three times the standard deviation.

[0038] For detection frames whose initial state category is germinating pollen, the ratio of their polar axis to equatorial axis deviates significantly from the standard value for ungerminated pollen due to the extension of their pollen tubes. This is considered a normal germination morphology, and the geometric verification is directly deemed passed. The detection frame, its corresponding initial state category, and the complete instance segmentation mask are retained, and the following ratio comparison is not performed. For detection frames whose initial state category is dead pollen or live pollen, the measured ratio is compared with the standard ratio. If the measured ratio falls within the allowable deviation range, it indicates that the morphological parameters of the detection frame conform to the typical morphological characteristics of the pollen of that variety, and the geometric verification is deemed passed. The detection frame and its corresponding initial state category are retained. If the measured ratio exceeds the allowable deviation range but is still within the preset correctable deviation range, where the correctable deviation range can be 1.5 times or 2 times the allowable deviation range, it indicates that the morphology of the detection frame is distorted, but this can still be corrected after applying an affine transformation to the detection frame. Specifically, the scaling factor between the standard ratio and the measured ratio is calculated. This scaling factor is used as a scaling coefficient. Using the center point of the detection frame as a reference, the width and height of the detection frame are scaled proportionally along the polar axis until the scaled measured ratio equals the standard ratio. The scaled detection frame replaces the original detection frame, and the calibration correction is deemed successful. If the measured ratio exceeds the correctable deviation range, the target within the detection frame differs too much from pollen grains in morphology, and is highly likely to be a false positive target caused by impurities, air bubbles, or overlapping of two or more grains. The geometric calibration is deemed unsuccessful, the detection frame is marked as a false positive, and it is removed from subsequent analyses.

[0039] This embodiment introduces a variety-specific polar-equatorial axis ratio standard library to geometrically constrain and correct the test results, integrating morphological prior knowledge into the automated analysis process. This effectively eliminates false positive targets caused by impurity contamination or particle overlap, and corrects for test frame deformation caused by sample tilt, thereby improving the variety adaptability and spatial positioning accuracy of the test results.

[0040] In one embodiment, after geometric verification is completed, targets previously marked as pollen to be verified need to be re-evaluated based on their fluorescence signals within the verified detection frame. Specifically, for a pollen to be verified, if its detection frame is determined to have passed geometric verification or correction, then based on the spatial position of the verified or corrected detection frame, the gray values ​​of all pixels within the detection frame area are extracted from the FITC channel image and TRITC channel image at the same time, respectively. The average gray value of the green fluorescence channel and the average gray value of the red fluorescence channel are calculated, and then the ratio of the average green gray value to the average red gray value is calculated. This ratio reflects the relative intensity of green fluorescence intensity relative to red fluorescence intensity within the pollen grain. A first ratio threshold and a second ratio threshold are set, wherein the first ratio threshold is greater than the second ratio threshold. Optionally, the first ratio threshold can be set to 2.0 or 2.5, and the second ratio threshold can be set to 0.8 or 1.0. If the calculated ratio is greater than the first ratio threshold, it indicates that green fluorescence is absolutely dominant within the pollen grain, suggesting an intact cell membrane and high esterase activity, thus classifying it as live pollen. If the calculated ratio is less than the second ratio threshold, it indicates that red fluorescence is dominant, suggesting a damaged cell membrane, thus classifying it as dead pollen. If the calculated ratio is between the first and second ratio thresholds, it indicates that the two fluorescence signals are of comparable intensity, making a definitive judgment difficult. The initial state category of the pollen is revised to "pending," meaning that subsequent time-series analysis may require more information from other time points for comprehensive judgment. If the detection frame containing the pollen to be verified fails the geometric check and is removed, the pollen target loses reliable spatial localization information, is marked as an invalid target, and is removed from all subsequent analysis steps, no longer participating in cross-frame association and final statistical calculations.

[0041] This embodiment filters out targets whose classification by the model is uncertain by setting a confidence threshold, and re-extracts the original fluorescence signal intensity and calculates the ratio by using the precise positioning after detection box verification. It then performs secondary discrimination by combining the staining mechanism of dual fluorescent dyes and introduces undetermined categories to avoid premature erroneous judgments, thereby improving the reliability of the classification results.

[0042] In one embodiment, when performing cross-frame target association, for each detection box in each frame image retained after geometric verification, the centroid coordinates of the detection box are calculated. This is achieved by adding half the width of the box to the x-coordinate of its upper-left corner and adding half the height of the box to its y-coordinate. The width and height of the detection box are also recorded. These parameters represent the position and morphological features of the corresponding pollen grains in that frame image. Two temporally adjacent frames are taken as a pair of matching objects. For each pollen grain in the previous frame and each pollen grain in the next frame, three similarity measures are calculated. The first is the Euclidean distance between the centroid coordinates, calculated by taking the square root of the sum of the squares of the differences between the two centroid x and y coordinates. A smaller distance indicates that the two targets are spatially closer. The second is the overlap area ratio of the two detection boxes, calculated by the ratio of the intersection area to the union area of ​​the two detection boxes. A larger ratio indicates a higher degree of spatial overlap between the two targets. The third aspect is the contour similarity of the instance segmentation masks within the two detection boxes. The similarity coefficient between the two binary masks can be calculated using descriptors such as Hu moments or contour moments. Combining these three indicators, if the Euclidean distance is less than a set distance threshold (e.g., 20 pixels), the overlap area ratio is greater than a set ratio threshold (e.g., 0.3), and the mask contour similarity is greater than a set similarity threshold (e.g., 0.7), then the pollen grain in the previous frame and the pollen grain in the next frame are determined to be the same pollen target, thus establishing a matching correspondence between adjacent frames. Each pollen grain is assigned a unique identifier for its first appearance; this identifier can be an incrementing integer. In subsequent cross-frame matching, once a pollen grain with a certain identifier is successfully matched in an adjacent frame, it retains this identifier in all frames it appears in. Simultaneously, the detection box information and instance segmentation mask of the pollen grain in each frame are sequentially assigned to the trajectory sequence corresponding to that identifier in chronological order. After the trajectory sequence is constructed, it indicates that the spatial location, morphological contour, and fluorescence signal of each pollen grain at each acquisition time point have been organized together. Based on each identification number, the image data and instance segmentation mask of the pollen grains within the detection box area corresponding to all collection time points are extracted from the multimodal temporal image dataset. These data fragments arranged in chronological order are then combined into independent multimodal temporal image fragments for each pollen grain.

[0043] This embodiment, by combining three complementary matching criteria—centroid distance, overlap area ratio, and segmentation mask contour similarity—can better adapt to the morphological changes during pollen germination, ensuring that the same pollen maintains identity consistency throughout the complete 6-hour time series, and providing an accurate data foundation for subsequent state determination based on time series features.

[0044] In one embodiment, for each pollen grain's multimodal time-series image segment, the FITC channel image, TRITC channel image, and bright-field image within it are extracted one by one in chronological order. Using the detection bounding box region of the pollen grain in that frame as the spatial cropping range, sub-images within the corresponding regions are cropped from the three images. The three sub-images at the same time point are stitched together along the channel dimension in the order of bright-field, FITC, and TRITC to form a single-frame input block with three channels. All single-frame input blocks corresponding to the acquisition time points are stacked in chronological order along the time dimension to construct a four-dimensional tensor with a size of T×C×H×W, where T represents the time length, equal to the number of effective frames continuously tracked by the pollen grain during the complete 6-hour observation period (e.g., T is 25 when the acquisition interval is 15 minutes), C is the number of channels and is a fixed value of 3, and H and W represent the height and width of the cropped sub-image, respectively, whose values ​​are determined by the size of the detection bounding box, optionally 128 pixels × 128 pixels or 256 pixels × 256 pixels. The four-dimensional tensor is used as the spatiotemporal input data of the pollen grain and input into a pre-trained three-dimensional convolutional neural network.

[0045] The specific structure of the 3D convolutional neural network is as follows: The first layer is a 3D convolutional layer with a kernel size of 3×3×3, that is, the depth along the time dimension is 3, and the width and height along the spatial dimension are both 3. This convolutional layer contains 64 convolutional kernels, and the stride is set to 1 in both the time dimension and the spatial dimension. The input and output sizes are kept constant through a zero-padding strategy. After the first layer, a 3D batch normalization layer is connected to normalize the convolutional output. Then, a 3D max pooling layer is connected with a pooling window size of 1×2×2, that is, no pooling is performed in the time dimension, but downsampling is performed by 2 times in the spatial dimension. Subsequently, a second set of 3D convolutional layers, a 3D batch normalization layer, and a 3D max pooling layer are sequentially set. The second set of 3D convolutional layers has a kernel size of 3×3×3 and 128 kernels, while the third set has a kernel size of 3×3×3 and 256 kernels. Each convolutional layer is followed by a corresponding batch normalization layer and a 3D max pooling layer, with the pooling window set to 1×2×2. After the alternating processing of multiple 3D convolutional and pooling layers, the spatial size of the feature map gradually decreases while the number of channels gradually increases, and the time dimension remains constant at T. The synchronous convolutional operation of the 3D convolutional kernels along the time dimension enables the network to capture the gradual change in fluorescence intensity and morphological evolution trend of the same pollen at different time points. The feature map obtained after multi-layer 3D convolution and pooling is flattened into a one-dimensional vector, which is then input into a fully connected layer. The fully connected layer outputs a fixed-length feature vector, which can be set to 256 or 512 dimensions. This vector is a compact encoding representation of the temporal evolution patterns of green fluorescence intensity, red fluorescence intensity, and polar-equatorial axis size ratio of the pollen grain throughout the 6-hour observation period. This fixed-length feature vector is then input into the final classification layer, which contains three neurons corresponding to dead pollen, live pollen, and germinating pollen. The classification layer uses a Softmax activation function to map the feature vector into three probability values, the sum of which is 1. The category corresponding to the highest probability among the three is taken as the final state of the pollen grain at the endpoint (6 hours). This process is repeated to iterate through the identification numbers of all pollen grains, obtaining the endpoint state of each pollen, thereby statistically obtaining the endpoint viability rate and endpoint germination rate.

[0046] This embodiment uses a three-dimensional convolutional neural network to extract spatiotemporal joint features from complete temporal image segments of each pollen grain. The network's three-dimensional convolutional kernels perform convolution operations simultaneously along both the temporal and spatial dimensions, enabling it to automatically learn the complete evolution pattern of pollen from its initial state to its final state. This allows the determination of the final state to incorporate the multimodal change trends throughout the entire process, resulting in a higher amount of discriminative information compared to methods relying solely on single-frame information from the final state. Furthermore, the extraction of fixed-length feature vectors and the design of the Softmax probability output provide a clear probabilistic interpretation of the determination results, facilitating the setting of reliability evaluation criteria.

[0047] The following is a description of a specific embodiment.

[0048] (1) Experimental group: Pollen from the grape variety to be tested was suspended in W5 solution. Fluorescein diacetate and propidium iodide were added to the pollen suspension to achieve a final concentration of 100 μg / mL for both dyes. The pollen suspension was incubated at 25°C in the dark for 20 min. After incubation, the pollen suspension was centrifuged at 800×g for 5 min, the supernatant was discarded, and the precipitate was resuspended in an equal volume of W5 solution to obtain the stained pollen suspension. Germination medium was prepared by dissolving 15% (w / v) sucrose, 100 mg / L boric acid, and 20 mg / L calcium chloride in deionized water. 300 μL of germination medium was injected into each well of a 24-well plate. 30 μL of the stained pollen suspension was then added dropwise to the surface of the germination medium in each well using a micropipette. The multiwell plate was placed on the stage of a fluorescence microscope, and the stage temperature was maintained at 26°C using the stage heating module. The microscope objective was adjusted to 20x magnification. Starting from the time of pollen inoculation into the culture medium, a preset time interval of 6 hours was set, and images were acquired every 15 minutes. While keeping the stage position unchanged to capture the same field of view, images from the FITC channel, TRITC channel, and brightfield channels were acquired sequentially. After each acquisition, the excitation source was turned off, and the multiwell plate was kept in the dark during the interval between adjacent acquisitions. All acquired images were arranged in time series to generate a multimodal time-series image dataset.

[0049] Brightfield, FITC, and TRITC channel images from the same time point in the dataset are treated as three independent channels and concatenated along the channel dimension to construct a three-channel composite feature input map. The first channel is the brightfield morphology channel, the second channel is the green fluorescence channel, and the third channel is the red fluorescence channel. This input map is fed into the Mask R-CNN model, which uses ResNet-101 as the backbone network and is supplemented by a feature pyramid network for multi-scale feature extraction. Candidate detection boxes are generated through a region proposal network. The bounding box regression branch and classification branch output the detection box position, initial state category, and corresponding classification confidence score for each pollen grain. Simultaneously, the instance segmentation mask is output through a mask branch. A first confidence threshold of 0.8 is set. If the classification confidence score is higher than 0.8, the initial state category output by the classification branch is used as the judgment result; if it is lower than or equal to 0.8, the pollen grain is marked as pollen to be verified.

[0050] Geometric verification is then performed. A pre-constructed standard library of polar-to-equatorial axis ratios is read. This library contains the standard polar-to-equatorial axis ratios and corresponding tolerance ranges for grape varieties such as Kyoho, Summer Black, and Cabernet Sauvignon (the standard deviation is derived from the standard deviation obtained after precise microscopic measurement of a large number of known pollen samples for each variety). For each detection frame output by the model, the measured ratio of the polar axis length to the equatorial axis length of the pollen grains within that frame is calculated. The corresponding standard ratio and tolerance range (standard ratio ± 3 times the standard deviation) are retrieved from the standard library based on the current grape variety. For detection frames whose initial state is germinating pollen, the geometric verification is directly passed, the detection frame and complete segmentation mask are retained, and the following ratio comparison is not performed; for detection frames whose initial state is dead pollen or live pollen, the measured ratio is compared with the standard ratio: if the measured ratio falls within the allowable deviation range, the geometric verification is passed, and the detection frame and initial state category are retained; if the measured ratio exceeds the allowable deviation range but is within 1.5 times the allowable deviation range, an affine transformation is applied to the detection frame to scale the measured ratio to the standard ratio, and the scaled detection frame replaces the original detection frame, and the correction is passed; if the measured ratio exceeds the correctable deviation range, the geometric verification is failed, the detection frame is marked as a false detection and discarded. For pollen samples to be verified, if the detection box passes the geometric verification or correction, the average gray values ​​of the green and red fluorescence channels are re-extracted based on the verified or corrected detection box, and the ratio is calculated. If the ratio is greater than 2.0, it is determined to be live pollen; if it is less than 1.0, it is determined to be dead pollen; if it is between 1.0 and 2.0, it is corrected to be pending. The pending pollen continues to participate in subsequent cross-frame association and 3D convolutional neural network analysis, and its final state is determined by the output of the 3D convolutional neural network. If its detection box is removed, the pollen sample to be verified is marked as an invalid target and removed.

[0051] Next, cross-frame target association is performed. For each detection box in each frame image retained after geometric verification, the centroid coordinates, width, and height of the detection box are extracted. Two temporally adjacent frames are grouped together. For each pollen grain in the previous frame and each pollen grain in the next frame, the Euclidean distance between the centroid coordinates of the two detection boxes, the overlap ratio of the two detection boxes, and the contour similarity of the instance segmentation masks within the two detection boxes are used to determine if the Euclidean distance is less than 20 pixels, the overlap ratio is greater than 0.3, and the mask contour similarity is greater than 0.7. Each pollen grain is assigned a unique identifier. After successful cross-frame matching, the same identifier is maintained across frames. The detection box information and instance segmentation masks at each time point are sequentially assigned to the trajectory sequence corresponding to that identifier. According to the identifier of each pollen grain, the detection box region and instance segmentation mask corresponding to each identifier at each acquisition time point are extracted along the time sequence and combined in chronological order to form the multimodal temporal image fragments for each pollen grain.

[0052] For each pollen grain's multimodal temporal image segment, the detection box region of each pollen grain is used as the spatial cropping range to crop out the detection box sub-images at each time point. The FITC channel sub-image, TRITC channel sub-image, and bright field sub-image at each time point are stitched together along the channel dimension to form a single-frame three-channel input block. Then, the single-frame three-channel input blocks of all time points are stacked along the time dimension to construct a four-dimensional tensor with dimensions T×C×H×W, where T=25 (from the start time to the 6-hour end time, including the start time, a total of 25 acquisition time points), C=3, and H and W are determined by the detection box size (in this experiment, they are uniformly scaled to 128 pixels×128 pixels). The four-dimensional tensor is input into a three-dimensional convolutional neural network, which sequentially includes a three-dimensional convolutional layer (3×3×3 convolutional kernels, 64 convolutional kernels, followed by a ReLU activation function), a three-dimensional batch normalization layer, a three-dimensional max pooling layer (pooling window 1×2×2), a second set of three-dimensional convolutional layers (3×3×3 convolutional kernels, 128 convolutional kernels, followed by a ReLU activation function), a three-dimensional batch normalization layer, a three-dimensional max pooling layer (pooling window 1×2×2), a third set of three-dimensional convolutional layers (3×3×3 convolutional kernels, 256 convolutional kernels, followed by a ReLU activation function), a three-dimensional batch normalization layer, a three-dimensional max pooling layer (pooling window 1×2×2), followed by a flattening layer, a fully connected layer (outputting a 256-dimensional feature vector), and a classification layer (3 neurons, Softmax activation). The network performs convolution operations simultaneously along the temporal and spatial dimensions to extract the spatiotemporal evolution features of each pollen grain. It outputs the probability values ​​of the pollen grain belonging to three states—dead pollen, live pollen, and germinating pollen—at the endpoint. The category corresponding to the highest probability value is taken as the final state determination result for that pollen grain at the endpoint. The network then calculates the proportions of dead pollen, live pollen, and germinating pollen to the total number of pollen grains at the endpoint, outputting the endpoint viability rate and endpoint germination rate.

[0053] (2) Control group 1: The traditional manual staining and microscopic counting method was used. The same stained pollen suspension as the experimental group was inoculated into the same germination medium. After culturing under the same conditions (26℃) for 6 h, the multi-well plate was placed under a fluorescence microscope. Three experienced experimental technicians observed and counted the pollen manually under the same field of view. The number of dead pollen, live pollen and germinating pollen were counted respectively. The average value of the three counts was taken as the final result.

[0054] (3) Control group 2: Static single-frame deep learning analysis method was used. The difference between the control group and the experimental group is that: no temporal image acquisition and cross-frame association were performed. Only single-frame FITC channel image, TRITC channel image and bright field image were acquired at the end of the training (6 h). After constructing the three-channel composite feature input map, it was input into the Mask R-CNN model for segmentation and classification. No geometric verification was performed. The end-time category output by the model was used as the final state determination result. No three-dimensional convolutional neural network was used for temporal feature extraction.

[0055] (4) Control group 3: Temporal analysis without cross-frame association was adopted. The difference between the control group and the experimental group is that although multiple frames of temporal images were collected at the same time interval within 6 hours, and each frame was independently detected and geometrically verified by Mask R-CNN, cross-frame target association was not performed between the frames, the detection results at each time point were not matched for identity, and the final state of each pollen at the end point was directly based on the classification result of Mask R-CNN at that time.

[0056] (5) Control group 4: The complete process without geometric verification is adopted. The difference between the control group and the experimental group is that after the MaskR-CNN model outputs the detection boxes and the initial state category, the geometric verification step of polar axis to equatorial axis ratio is skipped, and all the detection boxes output by the model (including the impurity targets falsely detected by the model) are directly sent to the subsequent cross-frame association and three-dimensional convolutional neural network analysis process.

[0057] (6) Testing and experimentation: Fresh pollen from three grape varieties (Kyoho, Summer Black, and Cabernet Sauvignon) was selected as test samples. Pollen from each variety was collected, stained, and cultured using the same method. Five parallel wells were set up for each variety, with each well representing an independent test sample. All experimental and control groups were conducted under the same environmental conditions: culture temperature 26℃, culture time 6 h, and image acquisition equipment was a fluorescence microscope equipped with a 20x objective lens (for manual counting in control group 1, the same microscope was used for observation at the same magnification).

[0058] (7) Performance parameter testing methods: Viability rate accuracy and germination rate accuracy: For both the experimental and control groups, 100 pollen grains were randomly selected from each well. The number of viable and germinating pollen grains was counted, and the viability and germination rates were calculated. The accuracy of each group was calculated using the results of artificial calibration as a reference standard. The artificial calibration results were obtained by having three experienced technicians jointly interpret each of the 100 pollen grains. The interpretation agreed upon by all three was used as the standard label for that pollen grain, and the viability and germination rates under the standard label were used as the true values. The formula for calculating the viability accuracy is: 1 - |P| detected -Ptrue | / P true ×100%; the germination rate accuracy is calculated similarly. The final accuracy for each test sample is the average of 5 parallel wells.

[0059] Correlation coefficient with manual counting results: The automated analysis results of the experimental group and each control group were compared with the manual counting results of three technicians in control group 1 using Pearson correlation coefficient analysis. Five parallel wells were used for each variety, resulting in 15 data points across the three varieties. The correlation coefficient r between the automated analysis viability rate and the manual counting viability rate was calculated for each group. This index reflects the consistency level between the automated method and the traditional manual method; a value closer to 1 indicates higher consistency.

[0060] Average single-sample analysis time: The computation time (excluding image acquisition time) required from image acquisition completion to the output of final statistical results was recorded for both the experimental and control groups, and the average value of five parallel wells was taken. For manual counting in control group 1, the average time taken by three technicians from the start of observation to the completion of counting was recorded. This indicator reflects the analytical efficiency of each method.

[0061] Inter-variety accuracy standard deviation: The dispersion of the vigor rate accuracy in the experimental and control groups across the three grape varieties was calculated and expressed as standard deviation. This index reflects the stability and adaptability of each method across different varieties; a smaller standard deviation indicates better variety adaptability.

[0062] Table 1 (8) Summary and analysis of experimental results As shown in Table 1, in terms of accuracy, the experimental group achieved a viability rate accuracy of approximately 96.0% and a germination rate accuracy of approximately 94.5%, both significantly better than the four control groups. While control group 1 achieved a manual counting accuracy of approximately 92.5% (viability rate) and approximately 90.0% (germination rate), this was at the cost of being the most time-consuming (approximately 18 min / sample) and significantly affected by human factors. Control group 2, due to using only a single frame image at the endpoint for static analysis, could not utilize the temporal information of fluorescence intensity and morphological changes during pollen germination. Its viability rate accuracy was only approximately 79.0%, and its germination rate accuracy was approximately 74.5%, the lowest among all groups. This indicates that a large number of pollen grains that had not yet germinated at the endpoint but actually had germination potential were misclassified, verifying the necessity of temporal dynamic analysis. Control group 3, lacking cross-frame target association, could not identify the same pollen at different times, resulting in the 3D convolutional neural network receiving temporal image fragments that actually came from different pollen individuals, disrupting the consistency of spatiotemporal features. Its viability rate accuracy was approximately 81.5%, and its germination rate accuracy was approximately 77.0%, slightly better than control group 2 but far lower than the experimental group. Control group 4, lacking geometric verification, had false detection results of impurities, bubbles, and overlapping particles directly included in subsequent analysis. Its viability rate accuracy was approximately 83.0%, and its germination rate accuracy was approximately 79.5%, indicating that even with temporal feature extraction capabilities, erroneous detection boxes in the input data can still severely affect the accuracy of the final statistical results.

[0063] In terms of efficiency, the single-sample analysis time of the experimental group was about 5.5 min, which was slightly higher than that of the control groups 2 to 4 (2.5 min to 4.0 min), but much lower than that of the control group 1 (18 min for manual counting). This shows that the present invention has good analytical efficiency while ensuring high accuracy, and can meet the needs of high-throughput detection.

[0064] Regarding varietal adaptability, the standard deviation of accuracy among varieties in the experimental group was approximately 1.8, the lowest among all groups. This indicates that the present invention, by introducing variety-specific morphological parameters for geometric verification, maintains good stability of the analytical method across different grape varieties. Control group 4, lacking geometric verification, had a standard deviation of approximately 5.5 among varieties, about three times that of the experimental group, directly reflecting the contribution of geometric verification to improving varietal adaptability. Control group 1, with manual counting, had a standard deviation of approximately 2.8 among varieties, which, while better than control groups 2-4, was still higher than the experimental group. This is because differences in pollen morphology among different varieties during manual counting can affect the consistency of interpretation by technicians.

[0065] Based on the above data, this invention demonstrates significant advantages in terms of detection accuracy, consistency with manual methods, analysis efficiency, and variety adaptability. The various technical features (temporal image acquisition, Mask R-CNN segmentation, cross-frame target association, geometric verification, and three-dimensional convolutional neural network temporal feature extraction) work together synergistically to achieve high-precision, high-efficiency, and high-stability automated analysis of grape pollen viability and germination rate.

[0066] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. A method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning, characterized in that, include: S1: Take grape pollen, suspend it in buffer solution, add fluorescein diacetate and propidium iodide, incubate in the dark, centrifuge, wash and resuspend to obtain stained pollen suspension; S2: Add the stained pollen suspension to the germination medium of the multi-well plate, place the multi-well plate on the stage of the fluorescence microscope, and capture FITC channel images, TRITC channel images and bright field images of the same field of view at set time intervals within a preset time interval to generate a multimodal time-series image dataset. S3: The Mask R-CNN model is used to segment each frame of the multimodal temporal image dataset, locate the detection box of each pollen grain and identify the initial state category, which includes dead pollen, live pollen and germinating pollen. S4: Geometric verification of the detection frame is performed based on a standard library of polar-to-equatorial axis ratios for grape varieties; S5: Perform cross-frame target association on the detection results of each frame after geometric verification to obtain the correspondence of each pollen at each collection time point, and extract multimodal temporal image segments accordingly. Input the multimodal temporal image segments into a 3D convolutional neural network to extract spatiotemporal evolution features. Based on the spatiotemporal evolution features, determine the final state of each pollen at the endpoint. The final state includes dead pollen, live pollen, and germinating pollen. Calculate the proportion of dead pollen, live pollen, and germinating pollen at the endpoint to the total number of pollen, and output the endpoint viability rate and endpoint germination rate.

2. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 1, characterized in that, In S1, grape pollen was taken and suspended in W5 solution to obtain a pollen suspension. Fluorescein diacetate and propidium iodide were added to make the final concentration of both in the pollen suspension 50-150 μg / mL. The pollen suspension was incubated in the dark at 20-30℃ for 10-30 minutes. After incubation, the pollen suspension was centrifuged, the supernatant was discarded, and W5 solution was added to the precipitate to resuspend it until the volume was equal to the initial volume of the pollen suspension, thus obtaining the stained pollen suspension.

3. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 1, characterized in that, Prepare germination medium by dissolving 15% (w / v) sucrose, 100 mg / L boric acid, and 20 mg / L calcium chloride in deionized water; inject the germination medium into each well of a multi-well plate; add 20-50 μL of stained pollen suspension to the surface of the germination medium in each well of the multi-well plate; place the multi-well plate on the stage of a fluorescence microscope and set and maintain the stage temperature at 25-28℃.

4. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 1, characterized in that, In S2, after placing the multiwell plate on the stage of the fluorescence microscope, adjust the microscope objective to a magnification of 10-20x. Within a preset time interval of 0-6 hours, acquire images at 10-15 minute intervals, keeping the stage position unchanged to capture the same field of view. Acquire FITC channel images, TRITC channel images, and bright-field images. After each acquisition, turn off the excitation light source to keep the multiwell plate in a dark incubation state during the interval between adjacent acquisitions. Arrange all acquired FITC channel images, TRITC channel images, and bright-field images in time series to generate a multimodal time-series image dataset.

5. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 1, characterized in that, S3 include: S31: For a FITC channel image, a TRITC channel image, and a bright field image at the same time and in the same field of view in a multimodal temporal image dataset, the bright field image, FITC channel image, and TRITC channel image are treated as three independent channels and stitched together along the channel dimension to construct a three-channel composite feature input map. The first channel of the three-channel composite feature input map is the bright field morphology channel, the second channel is the green fluorescence channel, and the third channel is the red fluorescence channel. S32: Input the three-channel composite feature input map into the Mask R-CNN model. The Mask R-CNN model uses ResNet-101 as the backbone network and is supplemented by a feature pyramid network for multi-scale feature extraction. The region proposal network generates candidate detection boxes. The bounding box regression branch and classification branch output the detection box position, initial state category and corresponding classification confidence score of each pollen grain. At the same time, the mask branch outputs the instance segmentation mask of each pollen grain on the three-channel composite feature input map. S33: If the classification confidence score is higher than the preset first confidence threshold, the initial state category output by the classification branch is used as the initial state category determination result of the pollen; if the classification confidence score is lower than or equal to the first confidence threshold, the pollen is marked as pollen to be verified. S34: Output the detection box position of each pollen grain in each frame of the image, the instance segmentation mask, and the initial state category determination result.

6. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 5, characterized in that, S4 include: S41: Read the pre-built and stored standard library of polar axis to equatorial axis ratios. The standard library contains the standard polar axis to equatorial axis ratios and corresponding allowable deviation ranges for multiple grape varieties. S42: For each detection box in each frame of the image output in step S3, calculate the measured ratio of the polar axis length to the equatorial axis length of the pollen grains in the detection box. S43: Based on the grape variety corresponding to the grape pollen, retrieve the corresponding standard polar axis to equatorial axis ratio and allowable deviation range from the standard library; S44: For each detection frame, compare the measured ratio of the detection frame with the standard polar axis to equatorial axis ratio: If the measured ratio falls within the allowable deviation range, the detection frame is deemed to have passed the geometric verification, and the detection frame and its corresponding initial state category are retained; if the measured ratio exceeds the allowable deviation range but is within the preset correctable deviation range, apply an affine transformation to the detection frame to scale the measured ratio of the detection frame to the standard polar axis to equatorial axis ratio, and replace the original detection frame with the scaled detection frame, thus determining that the detection frame geometric verification correction has passed; if the measured ratio exceeds the correctable deviation range, the detection frame geometric verification has failed, the detection frame is marked as a false detection and discarded.

7. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 6, characterized in that, For the pollen to be verified marked in S33, after the detection frame in which it is located passes the geometric check or correction in S44, the average gray values ​​of the green fluorescence channel and the red fluorescence channel are re-extracted and the ratio is calculated based on the checked / corrected detection frame. If the ratio is greater than the first ratio threshold, it is ultimately classified as live pollen; if it is less than the second ratio threshold, it is classified as dead pollen; if it is between the two, its initial state category is revised to pending. If the detection box containing the pollen is determined to fail the geometric verification in step S44 and is removed, the pollen to be verified will be marked as an invalid target and removed from subsequent analysis.

8. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 6, characterized in that, In S5, for each detection box in each frame image retained after geometric verification, the centroid coordinates, width, and height of the detection box are extracted as the position and morphological features of the corresponding pollen grain in the frame image. Two adjacent frames in time sequence are grouped together. For each pollen grain in the previous frame and each pollen grain in the next frame, the Euclidean distance between the centroid coordinates of the two detection boxes, the ratio of the overlapping area of ​​the two detection boxes, and the contour similarity of the instance segmentation mask in the two detection boxes are used to determine whether the two grains are the same pollen target and establish the correspondence between pollen grains in adjacent frames. Each pollen grain is assigned a unique identification number. After a successful cross-frame match, the pollen grain retains the same identification number at all the time points in which it appears. The detection box information and instance segmentation mask at each time point are sequentially assigned to the trajectory sequence corresponding to the identification number. After the trajectory sequence is constructed, the detection box region and instance segmentation mask of all frames corresponding to each identification number at each collection time point are extracted along the time sequence according to the identification number of each pollen. They are then combined in time order to form the multimodal temporal image fragments of each pollen grain.

9. The method for analyzing grape pollen viability and germination rate based on dual fluorescence staining and deep learning as described in claim 8, characterized in that, For each pollen grain's multimodal temporal image segment, the FITC channel image, TRITC channel image, and bright field image of each frame arranged in chronological order are cropped using the detection box region of each pollen grain as the spatial cropping range to extract the detection box sub-images at each time point. The FITC channel sub-image, TRITC channel sub-image, and bright field sub-image at each time point are then stitched together along the channel dimension to form a single-frame three-channel input block. The single-frame three-channel input blocks at all time points are then stacked along the time dimension to construct a 4D tensor with dimensions of time length × number of channels × height × width, which serves as the spatiotemporal input data for pollen grains. A 4D tensor is input into a 3D convolutional neural network. The 3D convolutional neural network contains multiple 3D convolutional layers, 3D batch normalization layers, 3D pooling layers and fully connected layers. Convolution operations are performed synchronously along the time and space dimensions to extract the spatiotemporal evolution features of each pollen grain in terms of green fluorescence intensity, red fluorescence intensity and morphological changes over time. A fixed-length feature vector corresponding to the end time of the preset time interval is output. A fixed-length feature vector is input into the classification layer and processed by the Softmax activation function. The output is the probability value of the pollen grain belonging to the three final states of dead pollen, live pollen, and germinating pollen at the end time. The category corresponding to the highest probability value is taken as the final state determination result of the pollen grain at the end time.

Images