A leaf pore analysis method based on intelligent information compensation and compression

Abstract

The present invention discloses a leaf stoma analysis method based on intelligent information compensation and compression, and the method is used to obtain leaf stomatal phenotypic data. The method includes feature extraction: using a convolutional neural network to extract stomatal texture and semantic features in the stomatal image; feature compensation and compression: by constructing a neck network with progressive spatial and channel feature compensation and compression, the multi-scale features extracted by the backbone network are fused to compensate for the loss of stoma-related features and further compress redundant features. Intelligent detection: the three refined feature maps are introduced into the branch decoupled detection head for prediction to obtain stomatal phenotypic data. This method greatly facilitates researchers to obtain more accurate crop stomatal phenotypic trait data, and plays a positive role in promoting the development of smart agriculture.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and its cross-application with biology, and specifically relates to a leaf stomatal analysis method based on intelligent information compensation and compression. Background Art

[0002] Stomata, primarily located on the surface of leaves, serve as pathways for gas exchange between plants and the external environment. The distribution and number of stomata vary depending on the plant species and are also influenced by environmental factors. Stomata regulate gas exchange between plants and their environment and are an important target for genetically improving water use efficiency. Stomata in the epidermis of dicotyledons consist of a pair of guard cells and an intermediate pore. Stomata act as pathways for air and water vapor during gas metabolism, such as carbon assimilation, respiration, and transpiration. The amount of air and water vapor flowing through them is regulated by the opening and closing of guard cells, and they are physiologically important. Therefore, factors such as stomatal location, density, and aperture play a crucial role in a plant's growth, development, and stress resistance.

[0003] Peanuts are an important cash crop and oilseed in my country. According to statistics, perennial drought has caused 70% of my country's peanut production to suffer varying degrees of drought stress, resulting in a yield reduction of over 20% of the country's total. Therefore, rapidly and accurately acquiring stomatal phenotypic data from images or videos of peanut leaf epidermal cells is crucial for studying and analyzing peanut growth characteristics.

[0004] For the past few decades, counting stomata and their phenotypic traits has primarily relied on manual or semi-automated methods, which consume significant labor and time, and lack high accuracy. Developing intelligent stomatal trait identification technologies is crucial, and relevant reports have emerged in recent years. Among them, Sanyal et al. (Analysis of SEM images of stomata of different tomato cultivars based on morphological features) proposed using microscope image processing techniques, employing a watershed technique to extract features, and then classifying different tomato varieties based on their stomatal structure. With the development of artificial intelligence, an increasing number of researchers have applied convolutional neural networks to process stomatal images to rapidly acquire stomatal feature information. For example, Wang Jingtao (Study on Stomatal Detection in Living Plant Leaves Based on Faster R-CNN) used the Faster R-CNN model to detect stomata in poplar leaves; Song Wenlong et al. used the Mask R-CNN deep semantic segmentation model to detect plant stomata, fitting ellipses to the stomatal sizes to obtain phenotypic information. However, none of these existing methods can obtain or directly obtain the phenotypic information of stomata at one time, and further operations such as segmentation models are required. At the same time, the phenotypic data obtained by these methods are often not accurate enough. Summary of the Invention

[0005] In order to address the deficiencies of the prior art, the present invention aims to provide a leaf stomatal analysis method based on intelligent information compensation and compression. This method can measure the size of leaf epidermal guard cells and automatically obtain phenotypic information such as stomatal location, density, and aperture through high-throughput leaf epidermal images. Stomatal detection is more accurate, greatly facilitating researchers to obtain stomatal phenotypic trait data.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A blade pore analysis method based on intelligent information compensation and compression is implemented based on the following steps:

[0008] S1. A high-throughput leaf epidermal cell image dataset collected using a portable microscope was randomly divided into a training set and a validation set according to a certain ratio. A dual-object rotating box annotation tool was used to annotate each stoma in the training and validation sets.

[0009] S2. Loading data and labels: A convolutional neural network is used as the backbone network to extract stomatal image features. A neck network is constructed to progressively compensate and compress spatial and channel features. The multi-scale features extracted by the backbone network are fused to mitigate the loss of stomatal-related features and further compress redundant features. The three fused feature maps are introduced into the branch-decoupled detection head for prediction and to obtain phenotypic data.

[0010] S3. Training the model: Each round of training is performed on all images in the training set. The model loss function value is calculated at each iteration and then the network parameters are updated by gradient descent. At the same time, after each round of training, the model loss function value is calculated on the validation set, and the network model that minimizes the validation set loss function value so far is saved. This training is repeated until the set number of rounds are completed.

[0011] S4. After loading the trained model, stomatal detection can be performed on high-throughput leaf epidermal cell images to calculate stomatal density and predict the number of open stomata, as well as their length and aperture.

[0012] S5. Threshold adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, stop adjusting the threshold and calculate and obtain the stomatal size and phenotypic information. If not, adjust the stomatal target confidence threshold and the intersection-over-union ratio threshold for non-maximum suppression based on the model recognition situation, and return to step S4 until the desired result is achieved.

[0013] Furthermore, step S1 specifically includes

[0014] S1-1. Acquire high-throughput leaf epidermal cell images using a portable microscope and construct a dataset;

[0015] S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio;

[0016] S1-3. Using the annotation tool X-AnyLabeling, rotated box annotations were performed on the images in the training and validation sets. The annotation information for each peanut stomata includes: the center coordinates, width, height, and rotation angle of the stomatal morphology, and the center coordinates, width, height, and rotation angle of the stomatal aperture box, and the aperture category;

[0017] S1-4. Generate and save the corresponding XML file to obtain the location and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training;

[0018] S1-5. Considering the small size and dense distribution of stomata, the images in the training set were segmented using the DOTA_devkit toolkit. Each image was segmented into multiple 640*640 images, and the corresponding TXT files were segmented so that the segmented images corresponded one-to-one with the TXT tag files.

[0019] Furthermore, step S2 specifically includes:

[0020] S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image;

[0021] S2-2. Construct a ResNet101 convolutional neural network as the backbone network to extract pore image features;

[0022] S2-3. Select the feature maps obtained from the last three layers of the backbone network and input them into the neck network that constructs progressive spatial and channel feature compensation and compression refinement to fuse the multi-scale features extracted by the backbone network, alleviate the loss of compensation and pore-related features, and further compress redundant features. The first step in this stage is to first fuse and refine low-level features and medium-level features, and then add high-level semantic features to participate in global feature interaction, so as to avoid feature redundancy caused by excessive semantic gaps across layers.

[0023] S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head adopts a prediction branch decoupling structure to decouple the head into two prediction branches, namely the target box category and score prediction branch and the target box coordinate information prediction branch. Each bounding box predicts a set of categories, and the category with the largest score is obtained as the category of the bounding box. The score is also the confidence of the box. Finally, the outputs of the two branch decoupling heads are connected as the total output of the model.

[0024] Furthermore, step S3 specifically includes:

[0025] S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s ≥ 1 and q ≥ 1;

[0026] S3-2. Training model: Each round of training is performed on all images in the training set, and the number of images calculated at a time is q; the label assignment method of task alignment is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each stomata image is calculated. total , and update the network parameters using momentum stochastic gradient descent;

[0027] S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total :

[0028]

[0029] Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly.

[0030] Furthermore, step S2-3 specifically includes:

[0031] The last three layers include: the third layer from the bottom, i.e. the low-level features x low , the second to last layer is the high-level feature x high , the last layer is the top feature x top The obtained feature maps are fed into the neck network for progressive spatial and channel feature compensation and compression.

[0032] The low-level feature map is downsampled and input into the high-level feature map compensation compression unit to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and Downsample and input the compensation compression unit with the top-level feature to obtain the refined feature y3;

[0033] Step S2-3 further includes:

[0034] S2-3-1. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit;

[0035] S2-3-2. The information compensation unit includes:

[0036] A1. Adjustment: Before performing feature interaction, it is necessary to adjust the The spatial position information provided adjusts the sampled feature map Y i s , where i≠k′; by learning an offset matrix, each value is interpreted as Y i s The point in The offset value calculated in two-dimensional space between corresponding points in ; the adjusted feature Y i a It can be expressed as:

[0037]

[0038] in, It's Y u s and The union of Δ (·) and f adjust (·) represent the function of learning the offset value from the spatial difference and the function of adjusting the sampled features according to the calculated offset value, respectively, f adjust (·) Implemented using deformable convolution;

[0039] B1. Fusion: Use Y n→k ′ represents the features from the nth layer to the kth layer after alignment and adjustment, and the kth layer fused feature Z k′ for:

[0040]

[0041] Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for W n Each element in Use Sigmoid function to enforce

[0042] C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula:

[0043]

[0044] Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, ⊕ is the addition of matrix elements, flip means flipping the channel head and tail of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, It represents the feature with less information content separated from Z, and W2 is the non-information weight. w Represents the reconstructed features.

[0045] S2-3-3. Redundant compression unit includes:

[0046] A2. Split: After obtaining the spatial refinement features, Z w The channel ratio α is divided into two parts: z αC and z (1-α)C , z CC Indicates the part with the number of channels αC, z(1-α)C Represents the part with (1-α)C channels; C is the total number of channels; each part inputs 1×1 convolution for channel compression;

[0047] B2. Re-extraction: Feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Features after re-extraction process

[0048]

[0049] in, and They are the matrices corresponding to group convolution and point convolution respectively, G is the abbreviation of group convolution, P1 is the abbreviation of the corresponding point convolution; g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, c is the number of channels, h and w are the length and width of the feature map;

[0050] Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process:

[0051]

[0052] in Is the matrix corresponding to another point convolution, P2 is the abbreviation of the corresponding point convolution; ∪ represents cascade,

[0053] C2. Fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final :

[0054] m=1,2;

[0055] β1+β2=1;

[0056] Z final =β1Z1+β2Z2;

[0057] Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z mIndicated as Z1 and Z2 respectively.

[0058] Furthermore, the loss function L in step S3-2 total The calculations specifically include:

[0059]

[0060] Where, L total Represents the loss function value, n positive Indicates the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, w obb represents the loss weight of 1-DIoU, w DFL Represents the loss weight of DFL, w VFL express The loss weight of , DIoU_i represents the distance intersection and union ratio corresponding to the i-th positive sample frame, DFL_i represents the distribution focus loss corresponding to the i-th positive sample frame, L VFL _i represents the variable focus loss of the category corresponding to the i-th prediction box.

[0061] The beneficial effects of the present invention are:

[0062] 1. This method uses ResNet101 as the backbone network to extract stomatal texture and semantic features from stomatal images. The three feature maps, after compensation and compression, are then fed into a branch-decoupled detection head for prediction, yielding stomatal phenotypic data. This greatly facilitates researchers in obtaining crop stomatal phenotypic data. Using this method, researchers can fully automatically acquire phenotypic information such as stomatal location, density, and aperture by monitoring high-throughput leaf epidermal images.

[0063] 2. This invention uses a neck network that progressively compensates and compresses spatial and channel features to fuse the multi-scale features extracted by the backbone network, compensating for the loss of stomatal-related features and further compressing redundant features. This design enables more accurate stomatal detection and phenotypic data.

[0064] 3. The present invention is a lightweight model that can realize real-time detection and identification of plant stomatal videos and external devices. Therefore, the stomatal characteristics of leaves throughout the growth stage can be studied, and the response mechanism of crop stomata to environmental changes can be explored. The study of stomatal characteristics is of great significance and plays a positive role in promoting the development of smart agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of the method of the present invention;

[0066] Figure 2This is a network structure diagram of the present invention;

[0067] Figure 3 This is the structural diagram of the information compensation unit of the present invention

[0068] Figure 4 This is the structure diagram of the redundant compression unit of the present invention

[0069] Figure 5 This is the detection result of a single image of the present invention;

[0070] Figure 6 This is a table of pore data information formed by the batch test results of the present invention. DETAILED DESCRIPTION

[0071] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of application of the present invention.

[0072] like Figure 1-6 As shown, the present invention proposes a leaf pore analysis method based on information intelligent compensation and compression, comprising the following steps:

[0073] S1. A high-throughput leaf epidermal cell image dataset collected using a portable microscope was randomly divided into a training set and a validation set according to a certain ratio. The annotation tool was used to perform dual-target rotating box annotation on each stoma in the training set and the validation set. Specifically:

[0074] S1-1. Acquire high-throughput leaf epidermal cell images using a portable microscope and construct a dataset;

[0075] S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio;

[0076] S1-3. Use the annotation tool X-AnyLabeling to annotate the images of the training set and the validation set with rotating boxes. The annotation information of each peanut stomata includes the coordinates of the center point of the stomata morphology, width, height and rotation angle (expressed as c x 、c y , w, h, θ), the category is stomata; the coordinates of the center point, width, height and rotation angle of the stomata opening frame (i.e. c′ x ,c′ y ,w′,h′,θ), the category is openness;

[0077] S1-4. Generate and save the corresponding XML file to obtain the location and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training;

[0078] S1-5. Considering the small size and dense distribution of stomata, the images in the training set were segmented using the DOTA_devkit toolkit. Each image was segmented into multiple 640*640 images, and the corresponding TXT files were segmented so that the segmented images corresponded one-to-one with the TXT tag files.

[0079] S2. Loading data and labels: Using a convolutional neural network as the backbone network, extract stomatal image features; constructing a neck network that progressively compensates and compresses spatial and channel features to fuse the multi-scale features extracted by the backbone network, alleviating the loss of stomatal-related features and further compressing redundant features; introducing the three refined feature maps into the branch-decoupled detection head for prediction to obtain phenotypic data. This includes:

[0080] S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image;

[0081] S2-2. Construct a ResNet101 convolutional neural network as the backbone network to extract pore image features;

[0082] S2-3. The feature maps obtained from the last three layers of the backbone network are selected and input into the neck network with progressive spatial and channel feature compensation and compression for multi-scale fusion, compensating for the loss of pore-related features and further compressing redundant features. The first step in this stage is to fuse low-level features with medium-level features, and then add high-level semantic features to participate in global feature interaction, in order to avoid feature redundancy caused by excessive semantic gaps across layers.

[0083] S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head adopts a prediction branch decoupling structure to decouple the head into two prediction branches, namely the target box category and score prediction branch and the target box coordinate information prediction branch. Each bounding box predicts a set of categories, and the category with the largest score is obtained as the category of the bounding box. The score is also the confidence of the box. Finally, the outputs of the two branch decoupling heads are connected as the total output of the model.

[0084] S3. Training model: Each round of training is performed on all images in the training set. The model loss function value is calculated at each iteration and then the network parameters are updated by gradient descent. At the same time, after each round of training, the model loss function value is calculated on the validation set and the network model that minimizes the validation set loss function value is saved. This training is repeated until the set number of rounds is completed. Specifically, it includes:

[0085] S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s ≥ 1 and q ≥ 1;

[0086] S3-2. Training model: Each round of training is performed on all images in the training set, with q images calculated at a time. The task aligned assigner (task aligned label assignment method) is used to determine the positive and negative sample boxes in the detection box, and then the loss function value L is calculated for each pore image. total , and update the network parameters using momentum stochastic gradient descent;

[0087] The loss function L in step S3-2 total The calculations specifically include:

[0088]

[0089] Where, L total Represents the loss function value, n positive Indicates the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, w obb represents the loss weight of 1-DIoU, w DFL Represents the loss weight of DFL, w VFL express The loss weight of , DIoU_i represents the distance intersection over union corresponding to the i-th positive sample frame, DFL_i represents the distribution focal loss corresponding to the i-th positive sample frame, L VFL _i represents the varifocal loss of the category corresponding to the i-th prediction box.

[0090] Among them, the positive sample boxes in all detection boxes in the pore image are obtained by using the task aligned assigner method.

[0091] S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total :

[0092]

[0093] Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly.

[0094] Among them, step S2-3 specifically includes:

[0095] The last three layers include: the third layer from the bottom, i.e. the low-level features x low, the second to last layer is the high-level feature x high , the last layer is the top feature x top The obtained feature maps are fed into the neck network for progressive spatial and channel feature compensation and compression refinement.

[0096] The low-level feature map is downsampled and input into the high-level feature map compensation compression unit to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and The refined feature y3 is obtained by downsampling and compensating the compression unit with the top-level feature input. The progressive approach avoids feature redundancy caused by the large semantic gap between layers on the one hand, and alleviates the loss of pore-related features on the other hand, minimizing redundant and irrelevant feature information.

[0097] Step S2-3 further includes:

[0098] S2-3-1. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit;

[0099] S2-3-2. The information compensation unit includes:

[0100] A1. Adjustment: Due to the repeated convolution and pooling operations in the backbone network, the sampled feature map Y i s (i≠k′) and unsampled features There is still a predictable spatial position error between them; therefore, information supplementation by direct element summation or multiplication will introduce erroneous pixel information, resulting in information redundancy; before feature interaction, it is necessary to The spatial position information provided adjusts the sampled feature map Y i s (i≠k′); by learning a bias matrix, each value is interpreted as Y i s The point in The offset value calculated in two-dimensional space between corresponding points in ; the adjusted feature Y i a It can be expressed as:

[0101]

[0102] in, It's Y i s and The union of Δ (·) and f adjust (·) represent the function of learning the offset value and the function of adjusting the sampling features, respectively, f adjust (·) Implemented using deformable convolution;

[0103] B1. Fusion: Different from the information exchange method through cascade or element summation, it assigns spatial weights to low-level features and high-level features, and compensates information in an adaptive weighted manner. This can effectively reduce the semantic gap and the impact of contradictory information between different layers, and avoid redundancy caused by information interaction. n→k′ Represents the aligned and adjusted features from the nth layer to the kth layer, and the fused features Z of the kth layer k′ for:

[0104]

[0105] Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for W n Each element in Use Sigmoid function to enforce

[0106] C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula:

[0107]

[0108] Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, ⊕ is the addition of matrix elements, flip means flipping the channel head and tail of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, It represents the feature with less information content separated from Z, and W2 is the non-information weight. w Represents the reconstructed features.

[0109] S2-3-3. Redundant compression unit includes:

[0110] A2. Split: After obtaining the spatial refinement features, Z w The channel ratio α is divided into two parts: zαC and z (1-α)C , z αC Indicates the part with the number of channels αC, z (1-α)C It represents the part with (1-α)C channels, where C is the total number of channels. Each part is input into 1×1 convolution for channel compression.

[0111] B2. Re-extraction: Feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Characteristics after re-extraction process:

[0112]

[0113] in, and They are the matrices corresponding to group convolution and point convolution respectively, G is the abbreviation of group convolution, P1 is the abbreviation of corresponding point-wise convolution; g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, and c is the number of channels. h and w are the length and width of the feature map;

[0114] Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process:

[0115]

[0116] in, It is the matrix corresponding to another point convolution, r is the channel compression ratio, c is the number of channels, P2 is the abbreviation of the corresponding point-wise convolution; ∪ represents cascade,

[0117] C2. Fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final :

[0118] m=1,2;

[0119] β1+β2=1;

[0120] Z final =β1Z1+β2Z2;

[0121] Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z m Denoted as Z1 and Z2 respectively.

[0122] S4. After loading the trained model, stomatal detection can be performed on high-throughput leaf epidermal cell images to calculate stomatal density and predict the number of open stomata, their length, and their aperture.

[0123] S5. Threshold adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, stop adjusting the threshold and calculate and obtain the stomatal size and phenotypic information. If not, adjust the stomatal target confidence threshold and the intersection-over-union ratio threshold for non-maximum suppression based on the model recognition situation, and return to step S4 until the desired result is achieved.

[0124] This paper designs a high-throughput intelligent analysis method for leaf stomatal phenotyping that integrates spatial and channel-by-channel progressive feature refinement. Using this method, researchers can automatically acquire phenotypic information such as stomatal location, density, and aperture by examining high-throughput leaf epidermal images.

[0125] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A leaf pore analysis method based on intelligent information compensation and compression, characterized by: The method is implemented based on the following steps: S1. A high-throughput leaf epidermal cell image dataset collected using a portable microscope was randomly divided into a training set and a validation set according to a certain ratio. A dual-object rotating box annotation tool was used to annotate each stoma in the training and validation sets. S2. Loading data and labels: A convolutional neural network is used as the backbone network to extract stomatal image features. A neck network is constructed to progressively compensate and compress spatial and channel features. The multi-scale features extracted by the backbone network are fused to mitigate the loss of stomatal-related features and further compress redundant features. The three refined feature maps are introduced into the branch-decoupled detection head for prediction and obtain phenotypic data. S3. Training the model: Each round of training is performed on all images in the training set. The model loss function value is calculated at each iteration and then the network parameters are updated by gradient descent. At the same time, after each round of training, the model loss function value is calculated on the validation set, and the network model that minimizes the validation set loss function value so far is saved. This training is repeated until the set number of rounds are completed. S4. After loading the trained model, stomatal detection can be performed on high-throughput leaf epidermal cell images to calculate stomatal density and predict the number of open stomata, as well as their length and aperture. S5. Threshold Adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, the threshold adjustment is stopped and the stomatal size and phenotypic information are calculated and obtained. If not, the stomatal target confidence threshold and the intersection-over-union ratio threshold for non-maximum suppression are adjusted based on the model recognition results, and the process returns to step S4 until the desired result is achieved. Wherein, step S2 specifically includes: S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image; S2-2. Construct a ResNet101 convolutional neural network as the backbone network to extract pore image features; S2-3. Feature maps from the last three layers of the backbone network are selected and fed into the neck network, which progressively compensates for and compresses spatial and channel features for multi-scale fusion. This compensates for the loss of pore-related features and further compresses redundant features. The first step in this phase is to fuse low-level and mid-level features, then add high-level semantic features for global feature interaction. This avoids feature redundancy caused by large semantic gaps across layers. S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head uses a prediction branch decoupling structure to decouple the head into two prediction branches: a branch for predicting the target box's category and score, and a branch for predicting the target box's coordinate information. Each bounding box predicts a set of categories, and the category with the highest score is taken as the category of the bounding box. This score also serves as the confidence level for the box. Finally, the outputs of the two branch decoupling heads are concatenated to form the total output of the model. Step S2-3 specifically includes: The last three layers include: the third layer from the bottom, i.e. the low-level features x low , the second to last layer is the high-level feature x high , the last layer is the top feature x top ; The low-level feature map is downsampled and input into the high-level feature map compensation compression unit to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and Downsample and input the compensation compression unit with the top-level feature to obtain the refined feature y3.

2. The method according to claim 1, wherein: Step S1 specifically includes: S1-1. Acquire high-throughput leaf epidermal cell images using a portable microscope and construct a dataset; S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio; S1-3. Using the annotation tool X-AnyLabeling, rotated box annotations were performed on the images in the training and validation sets. The annotation information for each peanut stomata includes: the center coordinates, width, height, and rotation angle of the stomatal morphology, and the center coordinates, width, height, and rotation angle of the stomatal aperture box, and the aperture category; S1-4. Generate and save the corresponding XML file to obtain the location and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training; S1-5. Considering the small size and dense distribution of stomata, the DOTA_devkit toolkit was used to segment the images in the training set. Each image was divided into multiple images with a resolution of 640*640, and the corresponding TXT files were split. The segmented images corresponded one-to-one with the TXT tag files.

3. The method according to claim 1, wherein: Step S3 specifically includes: S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s ≥ 1 and q ≥ 1; S3-2. Training model: Each round of training is performed on all images in the training set, and the number of images calculated at a time is q; the label assignment method of task alignment is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each stomata image is calculated. total , and update the network parameters using momentum stochastic gradient descent; S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total : Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly.

4. The method according to claim 3, wherein: Step S2-3 further includes: S2-3-1. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit; S2-3-2. The information compensation unit includes: A1. Adjustment: Before performing feature interaction, it is necessary to adjust the The spatial position information provided adjusts the sampled feature map where i≠k′; by learning a bias matrix, each value is interpreted as The point in The offset value calculated in two-dimensional space between corresponding points in ; the adjusted features It can be expressed as: in, yes and The union of Δ (·) and f adjust (·) represent the function of learning the offset value from the spatial difference and the function of adjusting the sampled features according to the calculated offset value, respectively, f adjust (·) Implemented using deformable convolution; B1. Fusion: Use Y n→k′ Represents the aligned and adjusted features from the nth layer to the kth layer, and the kth layer fused feature Z k′ for: Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for W n Each element in Use Sigmoid function to force C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula: Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, It is the addition of matrix elements, flip means flipping the channel of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, It represents the feature with less information content separated from Z, W2 is the non-information weight; Z w Represents the reconstructed features; S2-3-3. Redundant compression unit includes: A2. Split: After obtaining the spatial refinement features, Z w Divided into z by channel ratio α αC and z (1-α)C Two parts, z αC Indicates the part with the number of channels αC, z (1-α)C Represents the part with (1-α)C channels; C is the total number of channels; each part inputs 1×1 convolution for channel compression; B2. Re-extraction: Feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Characteristics after re-extraction process: in, and They are the matrices corresponding to group convolution and point convolution respectively, G is the abbreviation of group convolution, P1 is the abbreviation of the corresponding point convolution; g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, and c is the number of channels. h and w are the length and width of the feature map; Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process: in, Is the matrix corresponding to another point convolution, P2 is the abbreviation of the corresponding point convolution; ∪ represents cascade, C2. Fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final : Z final =β1Z1+β2Z2; Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z m Indicated as Z1 and Z2 respectively.

5. The method according to claim 4, characterized in that: The loss function L in step S3-2 total The calculations specifically include: Where, L total Represents the loss function value, n positive Indicates the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, w obb represents the loss weight of 1-DIoU, w DFL represents the loss weight of DFL, w VFL express The loss weight of , DIoU_i represents the distance intersection and union ratio corresponding to the i-th positive sample frame, DFL_i represents the distribution focus loss corresponding to the i-th positive sample frame, L VFL _i represents the variable focus loss of the category corresponding to the i-th prediction box.

Images