A method for predicting drought resistance of medicinal crops based on multi-view time-series images

Abstract

The application discloses a kind of based on multi-view time series image medicinal crop drought resistance ability prediction method, belong to image processing technical field;The application adopts multi-view time series acquisition to obtain three groups of time series images of overhead view, side view, front view;Through the improved U-Net model of double-path pooling attention and dynamic up-sampling is fused, plant semantic segmentation is completed;Yellowing stress index and wilting texture index are calculated to generate drought stress thermal map, multi-view fusion features are extracted by spatiotemporal attention and grouping channel attention, and finally drought resistance ability grade is output;The application realizes lossless, dynamic precision prediction of medicinal crop drought resistance, improves early prediction ability and prediction stability, and can be applied to medicinal crop drought resistance variety screening and standardized planting.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for predicting the drought resistance of medicinal crops based on multi-view time-series images. Background Technology

[0002] With the intensification of global climate change, the frequency and intensity of extreme drought events continue to rise, becoming one of the most significant abiotic stress factors restricting the growth, development, and quality stability of medicinal crops. Medicinal crops are the core material foundation for the development of the traditional Chinese medicine industry. Many medicinal materials in my country are mainly produced in arid and semi-arid regions such as the Northwest and Southwest. Drought stress not only causes leaf wilting and plant death, leading to reduced yields, but also significantly interferes with the biosynthesis and accumulation of core active ingredients such as alkaloids, directly reducing the clinical efficacy and market economic value of medicinal materials.

[0003] Traditional assessments of drought resistance in medicinal crops mainly rely on regular manual observation of morphological indicators such as plant height and leaf area, or on the determination of physiological and biochemical parameters such as SOD activity, electrical conductivity, and chlorophyll content through destructive sampling. These methods are time-consuming, labor-intensive, and cannot achieve dynamic tracking throughout the entire lifecycle, making it difficult to meet the needs of large-scale screening of drought-resistant varieties and precise cultivation management of medicinal crops.

[0004] Building upon this foundation, existing researchers have proposed a lightweight CSCA-YOLOv8 model to address the issues of low efficiency, high destructiveness, and difficulty in deploying conventional mung bean drought resistance assessment methods. By introducing StarNet, C2f_Star, CAA, and CGBD modules and adopting SIoU loss, this model achieves high-precision, lightweight detection of mung bean drought resistance. Furthermore, to address the challenges of segmenting individual poplar trees in complex weed backgrounds, low flux in traditional chlorophyll estimation, and difficulties in phenotypic analysis under drought stress, researchers have proposed an AI-driven UAV remote sensing framework. This framework constructs a DS-YOLOv10s-SAM segmentation model and a deep neural network regression model that integrates multimodal features, enabling high-precision, non-destructive estimation of chlorophyll in individual poplar trees and phenotypic analysis under drought stress.

[0005] However, existing technologies still have the following drawbacks:

[0006] 1. Most existing methods rely on single-frame static images for analysis, which cannot capture the continuous temporal changes of medicinal crops under drought stress, from leaf wilting and local yellowing to overall decline, making it difficult to achieve early and accurate drought resistance assessment.

[0007] 2. The encoder of the ordinary U-Net segmentation model is not good at extracting edge details such as leaves in medicinal crop images. At the same time, the decoder is not good at restoring fine structures such as curled edges and dry spots during image restoration, which easily leads to blurred boundaries and loss of details, resulting in inaccurate subsequent phenotypic feature extraction.

[0008] 3. Existing models, after processing time-series images, do not distinguish the differences in the contribution of different feature channels to drought stress, resulting in incomplete assessment of drought resistance and insufficient predictive stability. Summary of the Invention

[0009] To address the problems mentioned in the background art, this invention provides a method for predicting the drought resistance of medicinal crops based on multi-view time-series images. This method solves the problems of existing technologies that mostly rely on single-frame static images for analysis, insufficient encoder edge detail feature extraction capability, inaccurate decoder feature extraction, incomplete drought resistance assessment, and insufficient prediction stability.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A method for predicting the drought resistance of medicinal crops based on multi-view time-series images includes the following steps:

[0012] Step 1: Acquire multi-view time-series images of the drug. The multi-view time-series images include a top view time-series image sequence, a side view time-series image sequence, and a front view time-series image sequence. Each view time-series image sequence consists of single-frame images synchronously acquired at a fixed viewing angle for T consecutive time steps.

[0013] Step 2: Input each single frame image in the temporal image sequence of each view into the pre-trained U-Net segmentation model based on attention and dynamic sampling. The dual-path pooling attention layer of its encoder enhances the plant edge detail features, and the dynamic upsampling module of its decoder restores the feature map spatial resolution, outputting the plant foreground segmentation map corresponding to each frame image.

[0014] Step 3: Use the plant foreground segmentation map of each frame to perform foreground masking on the corresponding single frame image to calculate the drought stress region heat map of each frame; input the drought stress region heat map of each view into the pre-trained spatiotemporal attention model in time series, perform temporal attention calculation and spatial attention calculation in sequence to obtain spatiotemporal fusion features, and then perform grouped channel attention enhancement on the spatiotemporal fusion features to output the final fusion features of the corresponding view;

[0015] Step 4: Concatenate and fuse the three final fusion features corresponding to each view along the channel dimension to obtain the global fusion feature; input the global fusion feature into the pre-trained fully connected layer and process it through the classification function to output the drought resistance level prediction result of the medicinal crop.

[0016] Preferably, image acquisition uses a consistent light source and a white background. Top view, side view, and front view are all used to extract characteristics of the medicinal crop. The top view focuses on characterizing changes in canopy leaf area and color, the side view focuses on characterizing plant height evolution and growth vigor changes, and the front view focuses on characterizing the degree of leaf wilting.

[0017] Preferably, in step two, the encoder of the U-Net segmentation model based on attention and dynamic sampling consists of multiple cascaded coding layers, each of which contains a convolutional block and a dual-path pooling attention layer.

[0018] The computation process of the dual-path pooling attention layer satisfies the following formula: ; ;

[0019] ;

[0020] in, This represents the sigmoid activation function, AP represents average pooling, and MP represents max pooling. This represents the feature map output by the l-th convolutional block. and These represent the attention weights of the outputs of average pooling and max pooling, respectively. This represents the attention-enhanced feature map output by the l-th coding layer.

[0021] Preferably, in step two, the decoder of the U-Net segmentation model based on attention and dynamic sampling includes a multi-level dynamic upsampling module; the processing procedure of the dynamic upsampling module is as follows:

[0022] Preset an initial sampling offset of 2×2 grid centered on each pixel; predict the sampling offset feature map through a 1×1 convolutional layer, and then superimpose the predicted offset with the initial sampling offset after being modulated by a fixed scaling factor to obtain the corrected sampling coordinates; perform bilinear interpolation calculation based on the corrected sampling coordinates to output the upsampled feature map.

[0023] Preferably, in step three, the calculation process for the thermal map of the drought-stressed region is as follows:

[0024] Based on the image after foreground masking, the yellowing stress index and wilting texture index at the pixel level are calculated respectively. The yellowing stress index and wilting texture index are weighted and fused according to preset weights to obtain the original stress features. The original stress features are normalized by min-max and mapped to the [0,1] interval. The background region values ​​are set to 0 to obtain the drought stress region heat map.

[0025] The formula for calculating the yellowing stress index (DYI) is as follows:

[0026] ;

[0027] In the formula, , , These are pixels in the foreground mask image. The grayscale values ​​of the red, green, and blue channels.

[0028] Preferably, the formula for calculating the Wilted Texture Index (WTI) is as follows:

[0029] ;

[0030] In the formula, , pixels Contrast and entropy corresponding to the gray-level co-occurrence matrix of a 3×3 local window. / , / These are the maximum and minimum values ​​of contrast and entropy in the entire foreground mask image, respectively.

[0031] Preferably, in the time attention calculation process in step three, a time-series weighting factor is added to differentiate the attention results at different time steps. The time-series weighting factor increases exponentially with the increase of the time step index, which is used to enhance the distinction between the weak signals in the early stage of drought stress and the drastic changes in the later stage.

[0032] Preferably, the grouped channel attention enhancement process in step three is as follows: the spatiotemporal fusion feature is uniformly divided into g groups of channel group features along the channel dimension; independent channel attention weights are calculated for each group of channel group features; for each group of attention weights, the global arithmetic mean of all its elements is calculated, and the weights above the mean are set to 1, while the weights below the mean are kept at their original values; the processed weights are multiplied by the corresponding channel group features and then added to the residual connection; finally, all group features are concatenated along the channel dimension to obtain the final fusion feature.

[0033] Preferably, the model training step is also included to jointly train the U-Net segmentation model based on attention and dynamic sampling, the spatiotemporal attention model, and the fully connected layer; the total loss function used for training satisfies the following formula:

[0034] ;

[0035] in, , , These are weight hyperparameters; The temporal average of the segmentation Dice loss for all time frames corresponds to the segmentation loss of the U-Net segmentation model based on attention and dynamic sampling; The cross-entropy loss is used for drought resistance level prediction, corresponding to the classification loss of the fully connected layer. This is the inter-frame prediction stability regularization term, and its value is the time-step weighted average of the 1-norm distance between the output features of adjacent time frames.

[0036] Preferably, drought resistance is divided into 5 levels. The level division method is as follows: extract phenotypic features from the plant foreground segmentation image and generate image feature scores normalized to the [0,1] interval, and divide into 5 drought resistance levels with a score interval of 0.2.

[0037] Compared with the prior art, the beneficial effects of this application are as follows:

[0038] 1. A multi-view time-series image modeling method was designed to capture the continuous temporal changes of medicinal crops under drought stress, from leaf wilting, local yellowing, stem shrinkage to overall decline.

[0039] 2. The U-Net segmentation model has been improved, enhancing the ability to extract edge details such as leaves from medicinal crop images, while also improving the ability to restore fine structures such as withered spots on medicinal crops.

[0040] 3. A group channel attention enhancement module was designed to address the problem that global feature processing can easily lead to key stress signals being masked by background noise. This module enhances the ability to capture weak stress signals. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the U-Net segmentation model based on attention and dynamic sampling. Detailed Implementation

[0042] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the scope of the invention.

[0043] A method for predicting the drought resistance of medicinal crops based on multi-view time-series images includes the following steps:

[0044] S1: Data Acquisition;

[0045] 1. Drought stress design;

[0046] Multiple target drugs were selected as seeds and cultivated in petri dishes under identical conditions. After germination, polyethylene glycol (PEG) solutions at concentrations of 0%, 5%, 10%, 15%, and 20% were used to simulate drought stress in the plants. Because PEG molecules cannot penetrate plant cell walls and cell membranes, they do not enter the plant and cause ionic toxicity. Simultaneously, PEG can reduce the external water potential, preventing the plant roots from absorbing water from the environment, thus simulating drought stress.

[0047] 2. Time-series image acquisition;

[0048] To achieve temporal dynamic modeling of the drought stress response of medicinal crops, this invention designs a timed image acquisition process to construct a multi-time-point image sequence. The total number of time frames for image acquisition is set to T, and the time series index is t. At each time step t, images of the target medicinal crop are simultaneously acquired from three fixed perspectives: top view, side view, and front view, thus constructing a multi-view temporal image sequence with three perspectives. The top view temporal image sequence is represented as follows: The focus is on characterizing the canopy leaf area and color variation features of medicinal plants; the side view time-series image sequence is represented as The focus is on characterizing the evolution of plant height and growth potential of medicinal plants; the time-series image sequence of the front view is represented as... It focuses on characterizing the degree of leaf wilting in medicinal plants, among which , , These represent the top view, side view, and front view single-frame images acquired at time step t, respectively. Furthermore, all images were taken from a fixed angle with a consistent light source and against the same white background.

[0049] S2: Semantic segmentation of medicinal crops based on attention and dynamic sampling U-Net;

[0050] Prolonged drought can trigger drastic reshaping of the canopy structure in medicinal crops, leading to leaf wilting, vein curling, and progressive yellowing. By leveraging semantic segmentation algorithms for precise spatial separation of the core structure of medicinal crops, it is possible not only to non-destructively and dynamically track morphological distortions and color variations under drought stress, but also to accurately locate effective phenotypic regions in complex backgrounds. For example... Figure 1 As shown, this invention employs an attention-based and dynamic sampling U-Net model, improving the encoder and decoder of U-Net to achieve refinement and enhancement of the edges of medicinal crops. The single-frame image of the medicinal crop in frame t is input into the attention-based and dynamic sampling U-Net segmentation model, finally obtaining the segmentation feature map. .

[0051] 1. Encoder;

[0052] The U-Net encoder extracts spatial features from images through multiple convolutional layers, transforming the original input image into feature maps with rich semantic information but gradually decreasing resolution. The improved encoder consists of multiple cascaded coding layers, each containing a standard convolutional block followed by a dual-path pooling attention layer. The convolutional block comprises two cascaded 3×3 convolutional layers and corresponding ReLU activation functions. After basic spatial feature extraction through the convolutional blocks, the feature maps are then input to the dual-path pooling attention layer for key feature enhancement. (Top-down time-series image) For example, the image of the medicinal crop in frame t is represented as follows: Where H, W, and C represent the image height, width, and number of channels, respectively. After the [number]th [stage], The feature maps output by the convolutional blocks are ,in This represents the number of channels in the feature map. The formula for each convolution operation is as follows:

[0053] ;

[0054] in, Indicates the first The convolution kernel of the layer, Indicates the first The layer's bias term, ReLU represents the activation function. Indicates the first Feature maps output by the layer.

[0055] To enable the model to better focus on the fine edge features of medicinal crops, a dual-path pooling attention module was designed. This module uses two pooling operations to help the model focus on key details, thereby achieving more comprehensive feature extraction. The calculation process is as follows:

[0056] ;

[0057] ;

[0058] ;

[0059] in, This represents the sigmoid activation function, AP represents average pooling, and MP represents max pooling. and These represent the output weights of average pooling and max pooling, respectively. Indicates the first Attention-enhanced feature maps output from the layer encoding layer.

[0060] 2. Decoder;

[0061] The U-Net decoder restores the spatial resolution of the feature map through upsampling, converting the deep semantic information extracted by the encoder into pixel-level predictions of the same size as the original image. For image segmentation, the quality of the decoder directly determines the sharpness of the segmentation boundaries, the accuracy of contour restoration, and the integrity of the target structure. This is especially crucial in plant image processing, where the ability to restore details such as slender leaves, serrated edges, and withered spots is paramount. In the image segmentation of medicinal crops, traditional upsampling methods often lead to blurred leaf or stem edges and missed detections of yellowed areas. To alleviate these problems, a dynamic upsampling decoder was designed to replace the traditional upsampling method, achieving high-fidelity structural reconstruction and refined edge enhancement through the introduction of a dynamic sampling mechanism.

[0062] The upsampling process first establishes a set of initial sampling offsets, denoted as... These offsets uniformly subdivide each pixel into a 2×2 grid. These offsets are predefined as follows:

[0063] ;

[0064] These represent the relative coordinates of the four initial sampling points centered on each original pixel. This initialization method ensures that the upsampling operation remains consistent with standard bilinear interpolation in the absence of learned displacement.

[0065] For the input feature map First, a 1×1 convolutional layer is used to predict the offset feature map. Subsequently, the predicted offset is modulated by a fixed scaling factor to constrain the sampling displacement range, resulting in the corrected offset Δp′:

[0066] ;

[0067] ;

[0068] ;

[0069] in, Represents the normalized sampling coordinates. Indicates sampling coordinates The four surrounding adjacent pixels, This represents the interpolation weights automatically calculated based on the distances from the sampled coordinates to adjacent pixels. This represents the original feature value at position q in the original low-resolution feature map before upsampling. This represents the output pixel value. Dynamic upsampling provides an adaptive sampling grid with better spatial adaptability, effectively enhancing boundary recovery and detail restoration of medicinal crops. The medicinal crop image is segmented using an attention-based and dynamic sampling U-Net model, with the final output being a foreground segmentation map. .

[0070] S3: Prediction of drought resistance in medicinal crops based on spatiotemporal attention;

[0071] To enhance the model's feature extraction capability from images at different time frames, pixel-level exponential calculations are used to enhance features in drought-stressed regions. The original input image of the medicinal crop is used as an example. A masking process is performed, retaining only the foreground pixels of the plant while setting all background pixels to 0, thus eliminating the interference of the background on the stress calculation:

[0072] ;

[0073] in, To preserve only the ROI image of the plant foreground, The original image of frame t in spatial coordinates Pixel value at that location, Segment the image in spatial coordinates for frame t The pixel value at that location.

[0074] Medicinal crops under drought stress mainly exhibit visual characteristics such as leaf yellowing, wilting, and curling. To enhance the stress characteristics, two pixel-level indices, the yellowing stress index and the wilting texture index, were designed. The yellowing stress index quantifies the degree to which leaves change from healthy green to yellowing and browning, and is the most crucial indicator for discriminating drought stress.

[0075] ;

[0076] in, , , Pixels in the ROI image The grayscale values ​​of the red, green, and blue channels. The yellowing stress index.

[0077] The wilting texture index quantifies the changes in leaf texture roughness caused by wilting and curling, capturing early drought stress without obvious yellowing but only morphological wilting. Grayscale images are obtained by grayscale compression of the ROI images. In pixels Centered on a local window, a 3×3 local window is taken to obtain the local gray-level co-occurrence matrix (GLCM). The contrast of the GLCM is calculated based on the pixel values ​​of the local window. With entropy Then, the wilting texture index is obtained by min-max normalization:

[0078] ;

[0079] in, / , / These represent the maximum and minimum values ​​of contrast and entropy in the entire ROI image, respectively. The wilting texture index.

[0080] The yellowing index and wilting index are weighted and fused, and then normalized to the [0,1] interval using min-max normalization to obtain the final drought stress regional heat map. :

[0081] ;

[0082] ;

[0083] in, , For weight hyperparameters, and , , These are the fused features in the entire ROI image. The maximum and minimum values. The higher the value, the greater the probability that the pixel belongs to a drought-stressed region.

[0084] Each frame of the top-view time-series image is processed using a U-Net model based on attention and dynamic sampling, and the aforementioned pixel-level exponent calculation is performed to obtain the time-series image of the drought stress region. A drought stress region map of all time-series images is then generated. The image is divided into multiple patches, and then temporal and spatial encoding is performed to obtain... First, its temporal attention is calculated. Based on the traditional temporal attention calculation, a time-series weighting factor is added to differentiate the weak signals in the early stage of drought stress and the drastic changes in the later stage, thereby improving the early prediction ability and temporal stability of drought resistance.

[0085] ;

[0086] in, This represents the temporal attention feature, where T represents the time frame number and t represents the time index. For hyperparameters, This means performing attention calculations on all image patches that are in the same spatial location but in different time frames. Then, spatial attention calculations are performed.

[0087] ;

[0088] in, This represents the features of frame t after spatial attention enhancement. Indicates normalization, This means performing spatial attention calculations on all image patches located at different spatial positions within the same time frame. Then, all... ( The spatiotemporal fusion features are obtained by arranging complete sequence features in chronological order and then normalizing them. .

[0089] By employing a channel grouping strategy, the spatiotemporal fusion features are divided into multiple independent semantic subspaces. Channel attention is then calculated separately within each subspace. This approach enables more accurate extraction of fine-grained local stress features while suppressing interference from redundant background information, providing a more discriminative feature representation for subsequent drought resistance level prediction. For spatiotemporal fusion features... First, it is uniformly divided into g groups along the channel dimension to obtain the channel grouping features. , Let g represent the features of the g-th group. For each group of features, independent channel attention is applied to obtain local attention weights, calculated as follows:

[0090] ;

[0091] Where MLP represents a multilayer perceptron. Represents the i-th feature group. This represents the local attention weight.

[0092] Because the weak signals of medicinal crops in the early stages of drought stress are easily masked by background noise, in order to maximize the discriminative power of core phenotypic features in a fine-grained subspace and prevent numerical decay of key signals, the weight tensor is adjusted. Calculate the global arithmetic mean of all its elements, then set the values ​​of elements above the mean to 1, and leave the values ​​of elements below the mean unchanged, thus obtaining the reassigned weights. The above operation is performed on each group of weights to obtain the weight sequence for all channel groups. Multiply the obtained weights by the grouping features. The residual connections are added to obtain the grouped enhancement features, and finally, the features are stitched together along the channel dimension to obtain the final fused features of the view, calculated as follows:

[0093] ;

[0094] ;

[0095] in, Indicates channel splicing. This represents the final fused feature of the view, which includes feature representations of all time frames in the input sequence.

[0096] The time-series images of the front view and side view are also processed using the segmentation and spatiotemporal attention models described above, respectively, to obtain the final fusion features of the front view and the side view. The obtained final fusion features of the top view, the front view, and the side view are then concatenated to obtain the final global fusion feature. This global fusion feature is then passed through a fully connected layer and a softmax function to output the predicted probabilities of different drought resistance levels. The drought resistance level with the highest predicted probability is selected as the drought resistance prediction result for the medicinal crop, thus achieving the prediction of the drought resistance of the medicinal crop.

[0097] This application also includes the loss function and model training steps, specifically:

[0098] U-Net uses the Dice loss function for segmentation:

[0099] ;

[0100] ;

[0101] Where L represents the total number of pixels, T represents the number of time frames, and t represents the time series index. This represents the value of the i-th pixel in the t-th frame of the actual annotation. This represents the predicted value of the i-th pixel in the t-th frame of the image, obtained by applying the output features of the attention-based and dynamic sampling U-Net decoder to the Sigmoid activation function. This indicates the smoothing term.

[0102] Drought resistance prediction employs the cross-entropy loss function to predict the drought resistance level of medicinal crops, as detailed below:

[0103] ;

[0104] Where C represents the number of drought resistance level categories, Indicates the true label, This represents the predicted class probability, obtained by applying the final output features of the prediction to the Sigmoid activation function.

[0105] To maintain consistency among output sequences, an inter-frame prediction stability regularization term was designed:

[0106] ;

[0107] Where T represents the time frame number and t represents the time series index. Denotes the 1-norm. This represents the feature vector output by the spatiotemporal attention model after the t-th frame image is processed, i.e. The t-th feature vector obtained after decomposition in the time dimension This indicates a penalty.

[0108] The overall loss function is as follows:

[0109] ;

[0110] in, , , It's a hyperparameter. It is a segmentation loss. It is classification loss. It is a predictor of stability regularization.

[0111] This application first constructs and trains a drought resistance prediction model for medicinal crops, consisting of an attention-based and dynamic sampling U-Net segmentation model, a spatiotemporal attention model, and a fully connected layer. Historical time-series images of medicinal crops (front view, side view, and top view) are sequentially input into the attention-based and dynamic sampling U-Net segmentation model to obtain segmented time-series images. To construct real labels for supervised learning, key image phenotypic features such as leaf area and yellowing index are extracted from the segmentation results, and a comprehensive image feature score is generated. This score is normalized to [0,1], and a threshold of 0.2 is used to divide the samples into five drought resistance levels, representing their true drought resistance degree and serving as subsequent prediction labels. Subsequently, the yellowing index and wilting index are calculated based on the segmented time-series images to generate a drought stress feature map, which is then input into the spatiotemporal attention model to extract global fusion features and output the drought resistance level prediction probability. During the model training phase, AdamW is used as the optimizer, with an initial learning rate of 0.01, a batch size of 16, and 50 training epochs. During training, the model calculates the overall loss between the predicted probability and the true predicted label, performs backpropagation and parameter updates, and after multiple rounds of iterative training, finally obtains a converged and trained prediction model for the drought resistance of medicinal crops.

[0112] After the model training is completed, the specific process of predicting the drought resistance level of the medicinal crop image to be predicted is as follows: The front view time-series image, side view time-series image, and top view time-series image of the medicinal crop to be predicted, which are acquired in real time, are directly input into the U-Net segmentation model based on attention and dynamic sampling that has been trained above to obtain an accurate plant foreground segmentation image; then, the yellowing index and wilting index are calculated based on the segmented image, and a heat map of the drought stress area is calculated based on the yellowing index and wilting index; finally, the calculation results are input into the spatiotemporal attention model that has been trained, and global features are extracted through the forward propagation of the model. The model will automatically output the final drought resistance level of the medicinal crop to be tested, thereby achieving efficient and lossless dynamic prediction of drought resistance.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting the drought resistance of medicinal crops based on multi-view time-series images, characterized in that, Includes the following steps: Step 1: Acquire multi-view time-series images of the drug. The multi-view time-series images include a top view time-series image sequence, a side view time-series image sequence, and a front view time-series image sequence. Each view time-series image sequence consists of single-frame images synchronously acquired at a fixed viewing angle for T consecutive time steps. Step 2: Input each single frame image in the temporal image sequence of each view into the pre-trained U-Net segmentation model based on attention and dynamic sampling. The dual-path pooling attention layer of its encoder enhances the plant edge detail features, and the dynamic upsampling module of its decoder restores the feature map spatial resolution, outputting the plant foreground segmentation map corresponding to each frame image. Step 3: Use the plant foreground segmentation map of each frame to perform foreground masking on the corresponding single frame image, and calculate the heat map of the drought stress region for each frame. The drought stress region heatmaps of each view are input into a pre-trained spatiotemporal attention model in time series. Temporal attention calculation and spatial attention calculation are performed sequentially to obtain spatiotemporal fusion features. Then, the spatiotemporal fusion features are grouped and channel attention is enhanced to output the final fusion features of the corresponding view. Step 4: Concatenate and fuse the three final fusion features corresponding to each view along the channel dimension to obtain the global fusion feature; The global fusion features are input into a pre-trained fully connected layer and processed by a classification function to output the drought resistance level prediction results of medicinal plants.

2. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, Image acquisition uses a consistent light source and a white background. The top view focuses on characterizing the changes in canopy leaf area and color, the side view focuses on characterizing the evolution of plant height and changes in growth vigor, and the front view focuses on characterizing the degree of leaf wilting.

3. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, In step two, the encoder of the U-Net segmentation model based on attention and dynamic sampling consists of multiple cascaded coding layers, each of which contains a convolutional block and a dual-path pooling attention layer. The computation process of the dual-path pooling attention layer satisfies the following formula: ； ； ； in, This represents the sigmoid activation function, AP represents average pooling, and MP represents max pooling. This represents the feature map output by the l-th convolutional block. and These represent the attention weights of the outputs of average pooling and max pooling, respectively. This represents the attention-enhanced feature map output by the l-th coding layer.

4. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, In step two, the decoder of the U-Net segmentation model based on attention and dynamic sampling includes a multi-level dynamic upsampling module; the processing procedure of the dynamic upsampling module is as follows: Preset an initial sampling offset of 2×2 grid centered on each pixel; predict the sampling offset feature map through a 1×1 convolutional layer, and then superimpose the predicted offset with the initial sampling offset after being modulated by a fixed scaling factor to obtain the corrected sampling coordinates; perform bilinear interpolation calculation based on the corrected sampling coordinates to output the upsampled feature map.

5. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, In step three, the calculation process for the thermal map of the drought-stressed region is as follows: Based on the image after foreground masking, the yellowing stress index and wilting texture index at the pixel level are calculated respectively. The yellowing stress index and wilting texture index are weighted and fused according to preset weights to obtain the original stress features. The original stress features are normalized by min-max and mapped to the [0,1] interval. The background region values ​​are set to 0 to obtain the drought stress region heat map. The formula for calculating the yellowing stress index (DYI) is as follows: ； In the formula, , , These are pixels in the foreground mask image. The grayscale values ​​of the red, green, and blue channels.

6. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 5, characterized in that, The formula for calculating the Wilted Texture Index (WTI) is: ； In the formula, , pixels Contrast and entropy corresponding to the gray-level co-occurrence matrix of a 3×3 local window. / , / These are the maximum and minimum values ​​of contrast and entropy in the entire foreground mask image, respectively.

7. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, In the time attention calculation process in step three, a time-series weighting factor is added to differentiate the attention results at different time steps. The time-series weighting factor increases exponentially with the increase of the time step index, which is used to enhance the distinction between the weak signals in the early stage of drought stress and the drastic changes in the later stage.

8. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, The grouped channel attention enhancement process in step three is as follows: the spatiotemporal fusion features are uniformly divided into g groups of channel grouped features along the channel dimension; Calculate independent channel attention weights for each group of channel features; for each group of attention weights, calculate the global arithmetic mean of all its elements, set weights above the mean to 1, and keep the original values ​​of elements below the mean; multiply the processed weights by the corresponding channel features and add them to the residual connection; finally, concatenate all group features along the channel dimension to obtain the final fused features.

9. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, The method for predicting drought resistance in medicinal crops also includes a model training step, used to jointly train a U-Net segmentation model based on attention and dynamic sampling, a spatiotemporal attention model, and a fully connected layer; the total loss function used for training satisfies the following formula: ； in, , , These are weight hyperparameters; The temporal average of the segmentation Dice loss for all time frames corresponds to the segmentation loss of the U-Net segmentation model based on attention and dynamic sampling; The cross-entropy loss is used for drought resistance level prediction, corresponding to the classification loss of the fully connected layer. The inter-frame prediction stability regularization term is calculated using the feature vectors of adjacent time frames output by the spatiotemporal attention model. Its value is the average value of the 1-norm distance of the output features of adjacent time frames, weighted by time steps, and is used to constrain the temporal consistency of the output of the spatiotemporal attention model.

10. The method for predicting drought resistance of medicinal crops based on multi-view time-series images according to claim 1, characterized in that, Drought resistance is classified into 5 levels. The classification method is as follows: phenotypic features are extracted from the plant foreground segmentation image and image feature scores are generated and normalized to the [0,1] interval. The 5 drought resistance levels are obtained by dividing the scores into 0.2 intervals.

Images