Intelligent information compensation and compression method for progressive fusion of space and channel characteristics

Abstract

The invention discloses an intelligent information compensation and compression method for progressive fusion of space and channel characteristics. The method is used for obtaining phenotype data of leaf stomata. The method comprises the following steps: feature extraction: extracting pore texture and semantic features in a pore image by using a convolutional neural network; and feature compensation and compression: fusing the multi-scale features extracted by the backbone network by constructing a progressive space and channel feature compensation and compression neck network, compensating the loss of the features related to the pores, and further compressing the redundant features. And intelligent detection: introducing the three refined feature maps into a branch decoupling detection head for prediction to obtain pore phenotype data. The method greatly facilitates researchers to obtain more accurate crop stomatal phenotypic character data, and plays a positive role in promoting the development of smart agriculture.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and its cross-biological applications, and specifically relates to an information intelligent compensation and compression method for progressively fusing spatial and channel features. Background Art

[0002] The stomata of plants are mainly located on the surface of leaves. They are the channels for gas exchange between plants and the external environment. The distribution and number of stomata are related to the plant species and are also affected by environmental factors. Stomata regulate gas exchange between plants and the environment and are an important target for genetic improvement of water use efficiency. The epidermal stomata of dicotyledonous plants are composed of a pair of guard cells and an aperture in the middle. In gas metabolism such as carbon assimilation, respiration, and transpiration, they become pathways for air and water vapor. The amount of air passing through is regulated by the opening and closing of guard cells, which is of great physiological significance. Therefore, factors such as the positioning, density, and opening of stomata play an important role in the growth, development, and stress resistance of plants themselves.

[0003] Peanut is an important cash crop and oil crop in my country. According to statistics, the perennial drought has caused 70% of peanuts in my country to be affected by drought stress to varying degrees, and the reduction in peanut production accounts for more than 20% of the country's total production. Therefore, how to quickly and accurately obtain stomatal phenotype data from peanut leaf epidermal cell images or videos is the premise and key to studying and analyzing peanut growth characteristics.

[0004] In the past few decades, manual or semi-automatic methods have been mainly used to count the number and phenotypic traits of stomata, which consumes a lot of manpower and time costs, and the accuracy is not high. It is particularly important to develop intelligent recognition technology for stomatal traits. In recent years, there have been related reports in this field. Among them, Sanyal et al. (Analysis of SEM images of stomata of different tomato cultivars based on morphological features) proposed to use microscope image processing technology, use watershed technology to extract features, and then classify different types of tomatoes based on stomatal structure. Later, with the development of artificial intelligence, more and more researchers used convolutional neural networks to process stomatal images to quickly obtain stomatal feature information. For example, Wang Jingtao (Study on Stomatal Detection Method of Living Plant Leaves Based on Faster_R-CNN) used the Faster R-CNN model to detect the stomata of poplar leaves; Song Wenlong et al. used the Mask R-CNN deep semantic segmentation model to detect the stomata of plants, and used ellipses to fit the size of the stomata to obtain the phenotypic information of the stomata. However, none of these existing methods can obtain or directly obtain the phenotypic information of stomata at one time, and further operations such as segmentation models are required. At the same time, the phenotypic data obtained by these methods are often not accurate enough. Summary of the invention

[0005] In order to solve the shortcomings of the prior art, the present invention aims to provide an intelligent information compensation and compression method that progressively integrates spatial and channel features. This method can measure the size of leaf epidermal guard cells through high-throughput leaf epidermal images, and automatically obtain phenotypic information such as stomatal location, density and opening, making stomatal detection more accurate, thereby greatly facilitating researchers to obtain stomatal phenotypic trait data.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The information intelligent compensation and compression method of progressively fusing spatial and channel features is implemented based on the following steps:

[0008] S1. The high-throughput leaf epidermal stomatal image dataset collected using a portable microscope was randomly divided into a training set and a validation set according to a certain ratio, and a dual-objective rotating box annotation was performed on each stoma in the training set and the validation set using an annotation tool;

[0009] S2. Loading data and labels: Using the convolutional neural network as the backbone network, extracting the features of the stomatal image; and constructing a neck network with progressive spatial and channel feature compensation and compression to fuse the multi-scale features extracted by the backbone network, alleviate the loss of stomatal-related features and further compress redundant features; introducing the three fused feature maps into the branch decoupled detection head for prediction to obtain phenotypic data;

[0010] S3. Training model: In each round, all images in the training set are trained, and the model loss function value is calculated at each iteration, and then the network parameters are updated by gradient descent. At the same time, after each round of training, the model loss function value is calculated on the validation set, and the network model corresponding to the minimum loss function value of the validation set is saved. Repeat the training until the set number of rounds is completed.

[0011] S4. After loading the trained model, the stomatal detection can be performed on the high-throughput leaf epidermal cell images to calculate the stomatal density; the number of open stomata and their length and opening information can be predicted;

[0012] S5. Threshold adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, stop adjusting the threshold and calculate and obtain the size and phenotypic information of the stoma; if not, adjust the stomatal target confidence threshold and the intersection-over-union ratio threshold of non-maximum suppression according to the model recognition situation, and return to step S4 until the desired result is achieved.

[0013] Furthermore, step S1 specifically includes

[0014] S1-1. Obtain high-throughput leaf epidermal cell images of leaves through a portable microscope and construct a dataset;

[0015] S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio;

[0016] S1-3. Using the annotation tool X-AnyLabeling, the images of the training set and the validation set were annotated with rotating boxes. The annotation information of each peanut stomata includes: the center point coordinates, width, height and rotation angle of the stomatal morphology, the category is stoma; the center point coordinates, width, height and rotation angle of the stomatal aperture box, the category is aperture;

[0017] S1-4. Generate and save the corresponding XML file to obtain the position and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training;

[0018] S1-5. Considering that the stomata targets are small and densely distributed, the toolkit DOTA_devkit is used to segment the images in the training set. Each image is segmented into multiple images with a resolution of 640*640, and the corresponding TXT files are segmented so that the segmented images correspond one to one with the TXT label files.

[0019] Furthermore, step S2 specifically includes:

[0020] S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image;

[0021] S2-2. Construct ResNet101 convolutional neural network as the backbone network to extract pore image features;

[0022] S2-3. Select the feature maps obtained from the last three layers of the backbone network and input them into the neck network that constructs progressive spatial and channel feature compensation and compression refinement to fuse the multi-scale features extracted by the backbone network, alleviate the loss of compensation and pore-related features and further compress redundant features; the first step in this stage is to first fuse and refine the low-level features and the medium-level features, and then add high-level semantic features to participate in the global feature interaction; to avoid feature redundancy caused by excessive semantic intervals across layers.

[0023] S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head adopts a prediction branch decoupling structure to decouple the Head into two prediction branches, namely the category and score prediction branch of the target box and the prediction branch of the coordinate information of the target box. Each bounding box predicts a set of categories, and obtains the category with the largest score as the category of the bounding box. The score is also the confidence of the box. Finally, the outputs of the two branch decoupling heads are connected as the total output of the model.

[0024] Furthermore, step S3 specifically includes:

[0025] S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s≥1, q≥1;

[0026] S3-2. Training model: In each round, all images in the training set are trained, and the number of images calculated at a time is q; the task alignment label assignment method is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each stomata image is calculated. total , and update the network parameters using momentum stochastic gradient descent;

[0027] S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total :

[0028]

[0029] Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly.

[0030] Furthermore, step S3-3 specifically includes:

[0031] S3-3-1. Select the third layer from the backbone network, i.e. the low-level features x low , the second-to-last layer is the high-level feature x high , the first to last layer is the top feature x top ; The obtained feature map is input into the neck network for progressive spatial and channel feature compensation and compression; specifically including:

[0032] The low-level feature map is downsampled and input into the high-level feature map to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and Downsample and input the compensation compression unit with the top-level feature to obtain the refined feature y3;

[0033] S3-3-2. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit;

[0034] S3-3-3. The information compensation unit includes:

[0035] A1. Adjustment: Before performing feature interaction, it is necessary to adjust the The spatial position information provided adjusts the sampled feature map where i ≠ k′; by learning a bias matrix, each value is interpreted as The point in The offset value calculated in two-dimensional space between corresponding points in ; the adjusted features It can be expressed as:

[0036]

[0037] in, yes and The union of Δ(·) and f adjust (·) represent the function of learning the offset value from the spatial difference and the function of adjusting the sampled features according to the calculated offset value, respectively, and f adjust (·) Implemented using deformable convolution;

[0038] B1. Fusion: Use Y n→k′ It represents the features from the nth layer to the kth layer after alignment and adjustment, and the kth layer fused feature Z k′ for:

[0039]

[0040] Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for W n Each element in Use Sigmoid function to force

[0041] C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula:

[0042]

[0043] Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, It is the addition of matrix elements, flip means flipping the channels of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, represents the feature with less information content separated from Z, and W2 is the non-information weight. Z w Represents the reconstructed features.

[0044] S3-3-4. Redundant compression unit includes:

[0045] A2. Split: After obtaining the spatial refinement features, Z w The channel ratio α is divided into two parts: z αC and z (1-α)C , z αC represents the part with the number of channels αC, z (1-α)C Represents the part with (1-α)C channels; C is the total number of channels; each part inputs 1×1 convolution for channel compression;

[0046] B2. Re-extraction: feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Features after re-extraction process

[0047]

[0048] in, and They are the matrices corresponding to group convolution and point convolution respectively. G is the abbreviation of group convolution, and P1 is the abbreviation of the corresponding point convolution. g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, c is the number of channels, and z is αC ∈ h and w are the length and width of the feature map;

[0049] Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process:

[0050]

[0051] in is the matrix corresponding to another point convolution, P2 is the abbreviation of the corresponding point convolution; ∪ represents cascade,

[0052] C2. Fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final :

[0053]

[0054] Z final =β1Z1+β2Z2;

[0055] Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z m Denoted as Z1 and Z2 respectively.

[0056] Furthermore, the loss function L in step S3-2 total The calculations include:

[0057]

[0058] Where, L total represents the loss function value, n positive Represents the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of 1-DIoU, w DFL represents the loss weight of DFL, w VFL express The loss weight, DIoU_i represents the distance intersection and union ratio corresponding to the i-th positive sample box, DFL_i represents the distribution focus loss corresponding to the i-th positive sample box, L VFL _i represents the variable focus loss of the category corresponding to the i-th prediction box.

[0059] The present invention has the following beneficial effects:

[0060] 1. The present invention uses ResNet101 as the backbone network to extract the stomatal texture and semantic features in the stomatal image, and introduces the three feature maps after compensation and compression into the branch decoupled detection head for prediction to obtain the stomatal phenotypic data. This greatly facilitates researchers to obtain crop stomatal phenotypic trait data. Using this method, researchers can automatically obtain phenotypic information such as stomatal location, density and opening by detecting high-throughput leaf epidermal images.

[0061] 2. The present invention fuses the multi-scale features extracted by the backbone network through the neck network with progressive spatial and channel feature compensation and compression, compensates for the loss of stomata-related features and further compresses redundant features. This design can achieve more accurate stomata detection and obtain more accurate phenotypic data.

[0062] 3. The present invention is a lightweight model that can realize real-time detection and identification of plant stomatal videos and external devices. Therefore, the stomatal characteristics of leaves in the entire growth stage can be studied, and the response mechanism of crop stomata to environmental changes can be explored. The study of stomatal characteristics is of great significance and plays a positive role in promoting the development of smart agriculture. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flow chart of the method of the present invention;

[0064] Figure 2 This is a network structure diagram of the present invention;

[0065] Figure 3 This is the structural diagram of the information compensation unit of the present invention

[0066] Figure 4 This is the structure diagram of the redundant compression unit of the present invention

[0067] Figure 5 This is the detection result of a single picture of the present invention;

[0068] Figure 6 This is a pore data information table formed by the batch test results of the present invention. DETAILED DESCRIPTION

[0069] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not intended to limit the scope of use of the present invention.

[0070] like Figure 1-6 As shown, the present invention proposes an information intelligent compensation and compression method for progressively fusing spatial and channel features, comprising the following steps:

[0071] S1. The high-throughput leaf epidermal stomatal image dataset collected using a portable microscope is randomly divided into a training set and a validation set according to a certain ratio, and a dual-target rotating box annotation is performed on each stoma in the training set and the validation set using an annotation tool. Specifically:

[0072] S1-1. Obtain high-throughput leaf epidermal cell images of leaves through a portable microscope and construct a dataset;

[0073] S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio;

[0074] S1-3. Using the annotation tool X-AnyLabeling, the images of the training set and the validation set were annotated with rotating boxes. The annotation information of each peanut stomata included the center point coordinates, width, height and rotation angle (expressed as c x 、c y , w, h, θ), the category is stomata; the center point coordinates, width, height and rotation angle of the stomata opening frame (i.e. c′ x ,c′ y ,w′,h′,θ), the category is openness;

[0075] S1-4. Generate and save the corresponding XML file to obtain the position and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training;

[0076] S1-5. Considering that the stomata targets are small and densely distributed, the toolkit DOTA_devkit is used to segment the images in the training set. Each image is segmented into multiple images with a resolution of 640*640, and the corresponding TXT files are segmented so that the segmented images correspond one to one with the TXT label files.

[0077] S2. Load data and labels: Use the convolutional neural network as the backbone network to extract the features of the stomata image; build a neck network with progressive spatial and channel feature compensation and compression to fuse the multi-scale features extracted by the backbone network, alleviate the loss of features related to stomata and further compress redundant features; introduce the three refined feature maps into the branch decoupled detection head for prediction to obtain phenotypic data. Specifically include:

[0078] S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image;

[0079] S2-2. Construct ResNet101 convolutional neural network as the backbone network to extract pore image features;

[0080] S2-3. Select the feature maps obtained from the last three layers of the backbone network and input them into the neck network with progressive spatial and channel feature compensation and compression for multi-scale fusion to compensate for the loss of pore-related features and further compress redundant features. The first step in this stage is to fuse low-level features with medium-level features, and then add high-level semantic features to participate in global feature interaction to avoid feature redundancy caused by excessive semantic intervals across layers.

[0081] S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head adopts a prediction branch decoupling structure to decouple the Head into two prediction branches, namely the category and score prediction branch of the target box and the prediction branch of the coordinate information of the target box. Each bounding box predicts a set of categories, and obtains the category with the largest score as the category of the bounding box. The score is also the confidence of the box. Finally, the outputs of the two branch decoupling heads are connected as the total output of the model.

[0082] S3. Training model: Train all images in the training set in each round, calculate the model loss function value at each iteration, and then perform gradient descent to update the network parameters; at the same time, after each round of training, calculate the model loss function value on the validation set, and save the network model that minimizes the validation set loss function value so far; repeat the training until the set number of rounds is completed. Specifically include:

[0083] S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s≥1, q≥1;

[0084] S3-2. Training model: In each round, all images in the training set are trained, and the number of images calculated at a time is q; the task aligned assigner is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each pore image is calculated. total, and update the network parameters using momentum stochastic gradient descent;

[0085] The loss function L in step S3-2 yotal The calculations include:

[0086]

[0087] Where, L total represents the loss function value, n positive Represents the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of 1-DIoU, w DFL represents the loss weight of DFL, w VFL express , DIoU_i represents the distance intersection over union corresponding to the i-th positive sample box, DFL_i represents the distribution focal loss corresponding to the i-th positive sample box, L VFL _i represents the varifocal loss of the category corresponding to the i-th prediction box.

[0088] Among them, the positive sample boxes in all detection boxes in the pore image are obtained by adopting the task aligned assigner method.

[0089] S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total :

[0090]

[0091] Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly. Specifically include:

[0092] S3-3-1. Select the third layer from the backbone network, i.e. the low-level features x low , the second-to-last layer is the high-level feature x high , the first to last layer is the top feature x top ; The obtained feature map is input into the neck network for progressive spatial and channel feature compensation and compression refinement; specifically including:

[0093] The low-level feature map is downsampled and input into the high-level feature map to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and The refined feature y3 is obtained by downsampling and compensating the compression unit with the top-level feature input. The progressive method is adopted to avoid feature redundancy caused by the large semantic interval between layers on the one hand, and alleviate the loss of pore-related features on the other hand, thereby minimizing redundant and irrelevant feature information.

[0094] S3-3-2. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit;

[0095] S3-3-3. The information compensation unit includes:

[0096] A1. Adjustment: Due to repeated convolution and pooling operations in the backbone network, the sampled feature maps With unsampled features There is still a foreseeable spatial position error between them; therefore, information supplementation by direct element summation or multiplication will introduce erroneous pixel information and cause information redundancy; before feature interaction, it is necessary to The spatial position information provided adjusts the sampled feature map By learning a bias matrix, each value is interpreted as The point in The offset value calculated in two-dimensional space between corresponding points in ; the adjusted features It can be expressed as:

[0097]

[0098] in, yes and The union of Δ (·) and f adjust (·) represent the function of learning the offset value and adjusting the sampling features, respectively, and f adjust (·) Implemented using deformable convolution;

[0099] B1. Fusion: Different from the information exchange method through cascading or element summation, spatial weights are assigned to low-level features and high-level features, and information is supplemented in an adaptive weighted manner. This can effectively reduce the semantic gap and the impact of contradictory information between different layers and avoid redundancy caused by information interaction.n→k′ It represents the features from the nth layer to the kth layer after alignment and adjustment, and the kth layer fused feature Z k′ for:

[0100]

[0101] Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for W n Each element in Use Sigmoid function to force

[0102] C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula:

[0103]

[0104] Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, It is the addition of matrix elements, flip means flipping the channels of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, represents the feature with less information content separated from Z, and W2 is the non-information weight. Z w Represents the reconstructed features.

[0105] S3-3-4. Redundant compression unit includes:

[0106] A2. Split: After obtaining the spatial refinement features, Z w The channel ratio α is divided into two parts: z αC and z (1-α)C , z αC represents the part with the number of channels αC, z (1-α)C It represents the part with (1-α)C channels; C is the total number of channels. Each part is input into 1×1 convolution for channel compression.

[0107] B2. Re-extraction: feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Characteristics after re-extraction process:

[0108] Z1=M G z αC+M P1 z αC

[0109] in, and They are the matrices corresponding to group convolution and point convolution respectively. G is the abbreviation of group convolution, and P1 is the abbreviation of point-wise convolution. g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, and c is the number of channels. h and w are the length and width of the feature map;

[0110] Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process:

[0111]

[0112] in, is another matrix corresponding to point convolution, r is the channel compression ratio, c is the number of channels, P2 is the abbreviation of the corresponding point convolution (point-wise convolution); ∪ means cascade,

[0113] C2. Fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final :

[0114]

[0115]

[0116] Z final =β1Z1+β2Z2;

[0117] Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z m Represented as Z1 and Z2 respectively.

[0118] S4. After loading the trained model, the stomatal detection can be performed on the high-throughput leaf epidermal cell images to calculate the stomatal density; the number of open stomata and their length and opening information can be predicted;

[0119] S5. Threshold adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, stop adjusting the threshold and calculate and obtain the size and phenotypic information of the stoma; if not, adjust the stomatal target confidence threshold and the intersection-over-union ratio threshold of non-maximum suppression according to the model recognition situation, and return to step S4 until the desired result is achieved.

[0120] The present invention designs a high-throughput leaf stomatal phenotype intelligent analysis method that integrates spatial and channel progressive feature refinement. Using this method, researchers can automatically obtain phenotypic information such as stomatal location, density, and opening by detecting high-throughput leaf epidermal images.

[0121] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. An intelligent information compensation and compression method for progressively integrating spatial and channel features, characterized in that: The method is implemented based on the following steps: S1. The high-throughput leaf epidermal stomatal image dataset collected using a portable microscope was randomly divided into a training set and a validation set according to a certain ratio, and a dual-objective rotating box annotation was performed on each stoma in the training set and the validation set using an annotation tool; S2. Loading data and labels: Using the convolutional neural network as the backbone network, extracting the features of the stomatal image; constructing a neck network with progressive spatial and channel feature compensation and compression, fusing the multi-scale features extracted by the backbone network, alleviating the loss of stomatal-related features and further compressing redundant features; introducing the three refined feature maps into the branch decoupled detection head for prediction to obtain phenotypic data; S3. Training model: In each round, all images in the training set are trained, and the model loss function value is calculated at each iteration, and then the network parameters are updated by gradient descent. At the same time, after each round of training, the model loss function value is calculated on the validation set, and the network model corresponding to the minimum loss function value of the validation set is saved. Repeat the training until the set number of rounds is completed. S4. After loading the trained model, the stomatal detection can be performed on the high-throughput leaf epidermal cell images to calculate the stomatal density; the number of open stomata and their length and opening information can be predicted; S5. Threshold adjustment: If the detection and recognition results of stomatal phenotypic traits are satisfactory, stop adjusting the threshold and calculate and obtain the size and phenotypic information of the stoma; if not, adjust the stomatal target confidence threshold and the intersection-over-union ratio threshold of non-maximum suppression according to the model recognition situation, and return to step S4 until the desired result is achieved.

2. The method for intelligent compensation and compression of information of progressive fusion of spatial and channel features according to claim 1, characterized in that: Step S1 specifically includes: S1-1. Obtain high-throughput leaf epidermal cell images of leaves through a portable microscope and construct a dataset; S1-2. Divide the leaf epidermal cell image data into a training set and a validation set according to a specified ratio; S1-3. Using the annotation tool X-AnyLabeling, the images of the training set and the validation set were annotated with rotating boxes. The annotation information of each peanut stomata includes: the center point coordinates, width, height and rotation angle of the stomatal morphology, the category is stoma; the center point coordinates, width, height and rotation angle of the stomatal aperture box, the category is aperture; S1-4. Generate and save the corresponding XML file to obtain the position and bounding box coordinate information of each pore; then convert the saved XML file into a TXT file in DOTA format for subsequent training; S1-5. Considering that the stomata targets are small and densely distributed, the toolkit DOTA_devkit is used to segment the images in the training set. Each image is segmented into multiple images with a resolution of 640*640, and the corresponding TXT files are segmented so that the segmented images correspond one-to-one to the TXT label files.

3. The information intelligent compensation and compression method of progressively integrating spatial and channel features according to claim 1 is characterized in that: Step S2 specifically includes: S2-1. Read all high-throughput leaf epidermal cell images in the training set and the annotation information of each image; S2-2. Construct ResNet101 convolutional neural network as the backbone network to extract pore image features; S2-3. Select the feature maps obtained from the last three layers of the backbone network and input them into the neck network with progressive spatial and channel feature compensation and compression for multi-scale fusion to compensate for the loss of pore-related features and further compress redundant features. The first step in this stage is to fuse low-level features with medium-level features, and then add high-level semantic features to participate in global feature interaction to avoid feature redundancy caused by excessive semantic intervals across layers. S2-4. The three fused feature maps are introduced into the detection head for prediction. The detection head adopts a prediction branch decoupling structure to decouple the Head into two prediction branches, namely the category and score prediction branch of the target box and the prediction branch of the coordinate information of the target box. Each bounding box predicts a set of categories, and obtains the category with the largest score as the category of the bounding box. The score is also the confidence of the box. Finally, the outputs of the two branch decoupling heads are connected as the total output of the model.

4. The method for intelligent compensation and compression of information of progressive fusion of spatial and channel features according to claim 1, characterized in that: Step S3 specifically includes: S3-1. Set the number of iterations required for training the entire training set to s, and the number of images read in at a time to q, where s≥1, q≥1; S3-2. Training model: In each round, all images in the training set are trained, and the number of images calculated at a time is q; the task alignment label assignment method is used to determine the positive and negative sample frames in the detection frame, and then the loss function value L of each stomata image is calculated. total , and update the network parameters using momentum stochastic gradient descent; S3-3. After each round of training, calculate the average value of the model loss function averageL on the validation set total : Save the network model that minimizes the average value of the validation set loss function so far; repeat step S3-2 until the set number of rounds is completed or the loss function does not decrease significantly.

5. The method for intelligent compensation and compression of information of progressive fusion of spatial and channel features according to claim 4, characterized in that: Step S3-3 specifically includes: S3-3-1. Select the third layer from the backbone network, i.e. the low-level features x low , the second-to-last layer is the high-level feature x high , the first to last layer is the top feature x top ; The obtained feature map is input into the neck network for progressive spatial and channel feature compensation and compression; specifically including: The low-level feature map is downsampled and input into the high-level feature map to obtain a refined feature map. The high-level features are upsampled and input into the low-level feature compensation compression unit to obtain a refined feature map Top-level features and Upsample and Input compensation compression unit to obtain refined feature y1; top-level feature upsampling, feature Downsample and Input the compensation compression unit to obtain the refined feature y2; feature and Downsample and input the compensation compression unit with the top-level feature to obtain the refined feature y3; S3-3-2. The compensation compression unit specifically includes an information compensation unit and a redundant compression unit; S3-3-3. The information compensation unit includes: A1. Adjustment: Before performing feature interaction, it is necessary to adjust the The spatial position information provided adjusts the sampled feature map Y i s , where i≠k′; by learning an offset matrix, each value is interpreted as Y i s The point in The offset value calculated in two-dimensional space between corresponding points in; the adjusted feature Y i a It can be expressed as: in, Yes i s and The union of Δ (·) and f adjust (·) represent the function of learning the offset value from the spatial difference and the function of adjusting the sampled features according to the calculated offset value, respectively, and f adjust (·) Implemented using deformable convolution; B1. Fusion: Use Y n→k′ It represents the aligned and adjusted features from the nth layer to the kth layer, and the fused features Z of the kth layer k′ for: Among them, W n Represents the two-dimensional spatial weight matrix of the n-th layer feature map; for each element in Wn Use Sigmoid function to force W ij n ∈[0, 1]; C1. Reconstruction: For the fused feature Z(Z k′ ), it is necessary to separate the feature channels with large information content in the spatial dimension and reconstruct the feature channels with small information content; the features are reconstructed specifically through the following formula: Among them, GN represents group regularization, w channel is the weight of each channel obtained by the scaling factors in group regularization, is the matrix dot product, It is the addition of matrix elements, flip means flipping the channels of the feature map; Z represents the fused features after the B1 step, represents the information-rich features separated from Z, W1 is the information weight, represents the features with less information content separated from Z, W2 is the non-information weight; Z w represents the reconstructed features; S3-3-4. Redundant compression unit includes: A2. Split: After obtaining the spatial refinement features, Z w Divided into z by channel ratio α αC and z (1-α)C Two parts, z αC represents the part with the number of channels αC, z (1-α)C It represents the part with (1-α)C channels; C is the total number of channels; each part is input into 1×1 convolution for channel compression; B2. Re-extraction: feature z αC The feature Z1 is obtained by the following operation, Z1 represents the feature z αC Characteristics after re-extraction process: in, and They are the matrices corresponding to group convolution and point convolution respectively. G is the abbreviation of group convolution, and P1 is the abbreviation of the corresponding point convolution. g is the number of groups in group convolution, r is the channel compression ratio, k is the kernel size, and c is the number of channels. h and w are the length and width of the feature map; Feature z (1-α)C The feature Z2 is obtained by the following operation, Z2 represents the feature z (1-α)C Characteristics after re-extraction process: in, is the matrix corresponding to another point convolution, P2 is the abbreviation of the corresponding point convolution; ∪ represents cascade, C2. Re-fusion: Finally, the SKNet method is used to fuse the features Z1 and Z2 through the following operations to obtain the output Z of the redundant compression unit. final : Z final =β1Z1+β2Z2; Among them, Pooling(*) represents global average pooling, S m It means that when m takes the values ​​of 1 and 2 respectively, S1 and S2 are obtained, where S1 is the feature statistic corresponding to Z1; S2 is the feature statistic corresponding to Z2; β1 is the weight corresponding to Z1, and β2 is the weight corresponding to Z2; and is an intermediate parameter; when m is 1 and 2, Z m Denoted as Z1 and Z2 respectively.

6. The method for intelligent compensation and compression of information of progressive fusion of spatial and channel features according to claim 5, characterized in that: The loss function L in step S3-2 total The calculations include: Where, L total represents the loss function value, n positive Represents the number of positive sample boxes of stomata images predicted by the model in the training set, n pred represents the number of all sample frames of stomata images predicted by the model in the training set, i represents the sequence number of the sample frame, and w obb represents the loss weight of 1-DIoU, w DFL represents the loss weight of DFL, w VFL express The loss weight, DIoU_i represents the distance intersection and union ratio corresponding to the i-th positive sample box, DFL_i represents the distribution focus loss corresponding to the i-th positive sample box, L VFL _i represents the variable focus loss of the category corresponding to the i-th prediction box.

Images