Soybean flowering phase prediction method and system

Abstract

The invention discloses a soybean flowering phase prediction method and system, and belongs to the technical field of biology, and the method comprises the steps: obtaining RGB images and elevation images of each germplasm resource soybean plot shot at multiple time points; according to each elevation image data, determining the soybean plant height of each plot; a plurality of components in the RGB color space factors are selected for linear combination, and the canopy coverage rate is determined; inputting the soybean plant height and the canopy coverage rate into a pre-trained VGG-LSTM model, and outputting a flowering phase prediction result; wherein the VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network which are connected through an attention module; inputting the RGB image into the VGG network, and extracting time sequence features of the color, texture and RGB index of the image; performing saliency feature extraction on a short sequence in the time sequence features by using an attention module, determining the weight of each time step feature in the short sequence, and performing weighted fusion; and inputting the fused features into the LSTM, and outputting a flowering phase prediction result. According to the method, high-precision and high-robustness soybean flowering phase prediction is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image data processing, in particular to a soybean flowering period prediction method and system. BACKGROUND

[0002] Soybean is a short-day crop sensitive to photoperiod, and the maturity period of soybean is one of the important factors affecting the yield and quality of soybean. In the growth and development process of most plants, flowering must be after a certain length of day, otherwise it will be in the vegetative growth state. The photoperiod phenomenon is an effect that the alternation of light period and dark period in the day-night cycle affects the flowering of plants. The same variety in low latitude area has short light time, early flowering, early maturity and low yield; while in high latitude area, the light time is long, late flowering, late maturity and high yield. The response of soybean to photoperiod usually affects the length of the maturity period, thereby affecting the yield, so accurate identification of soybean flowering is of great significance for screening of high-yield germplasm resources.

[0003] The identification of soybean flowering time of traditional soybean germplasm resources is generally recognized by manual digging, which is labor-intensive, time-consuming and low in efficiency. The observation method using unmanned aerial vehicle is widely used in crop growth monitoring because of its low cost, intuition and real-time dynamic collection of crop growth without contact with crops.

[0004] However, since the flowering part of soybean is hidden under the canopy, the RGB sensor carried by the unmanned aerial vehicle cannot directly capture the intuitive information related to flowering, making it difficult to obtain field phenotype data directly related to flowering period in a timely, effective and high-throughput manner. Traditional methods rely on manual observation or single phenotype index, which is not only low in efficiency and strong in subjectivity, but also cannot accurately represent the key state of the transformation of soybean vegetative growth to reproductive growth, thereby cannot support the needs of fine management and breeding of soybean, resulting in insufficient prediction accuracy and poor reliability of soybean flowering time. SUMMARY

[0005] In view of the problems existing in the above field, the present application provides a soybean flowering period prediction method and system, which uses the two phenotype indexes of plant height and canopy coverage that are strongly related to soybean flowering period and can be easily obtained by unmanned aerial vehicle remote sensing to replace the flowering characteristics under the canopy that cannot be directly observed. The plant height and canopy coverage are input into the VGG-LSTM model to realize accurate prediction of the flowering period.

[0006] To solve the above technical problems, the present application discloses a soybean flowering period prediction method, comprising the following steps: obtaining RGB images and elevation images of each germplasm resource soybean plot photographed at multiple time points; According to each high-altitude image data, each pixel elevation point data of each plot field soybean is extracted by DSM, and the soybean plant height of each plot is determined; a plurality of components in the color space factor of RGB are selected for linear combination, and the canopy coverage of each plot field soybean is determined by threshold segmentation method; The soybean plant height and the canopy coverage are input into the pre-trained VGG-LSTM model, the flowering period of soybean is predicted, and the flowering period prediction result is output; The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module. The RGB image is input into the VGG network, the color, texture and RGB index time sequence features of the image are extracted, the attention module is used for significant feature extraction on the short sequence in the time sequence features, the weight of each time step feature in the short sequence is determined and weighted fusion is performed, and the fused features are input into the LSTM, and the flowering period prediction result is output by establishing a nonlinear mapping relationship between the fused features and the flowering period.

[0007] Preferably, the soybean plant height and the canopy coverage are input into the pre-trained VGG-LSTM model, the flowering period of soybean is predicted, and the flowering period prediction result is output, and specifically comprises: The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module. The convolutional neural network VGG comprises an input layer, a convolutional layer and a pooling layer. In the input layer, each germplasm resource soybean plot RGB image photographed at multiple time points in time sequence is input; in the convolutional layer and the pooling layer, the time sequence features of the color, texture and RGB index of the image are gradually extracted through multi-layer convolution and pooling operation; The local time sequence segment in the time sequence features is taken as a short sequence, and is taken as the input of the attention module, the weight of each time step feature in the short sequence is determined through a neural network; each time step feature is weighted with the corresponding weight to obtain the fused features; and the fused features are input into the LSTM network. The LSTM network comprises an input layer, a hidden state updating layer and an output layer. In the hidden state updating layer, the hidden state of the current time step is determined according to the feature of the current time step and the hidden state of the previous time step received by the input layer; and in the output layer, the hidden state of the last time step is used to map the flowering period prediction result through a fully connected layer.

[0008] Preferably, the soybean plant height and the canopy coverage are input into the pre-trained VGG-LSTM model, the flowering period of soybean is predicted, and the flowering period prediction result is output, and specifically comprises: According to each high elevation image data, the DSM is used to extract each pixel elevation point data in each germplasm resource soybean plot block, the highest 1% value of the elevation point is taken as the first mean value, the lowest 1% value of the elevation pixel point is taken as the second mean value, and the difference between the two is taken as the average soybean plant height of each plot.

[0009] Preferably, the acquisition of the RGB image and the high elevation image of each germplasm resource soybean plot taken at multiple time points comprises collecting the RGB image data of the soybean germplasm resource by the unmanned aerial vehicle, reconstructing the three-dimensional model of the RGB image data collected by the unmanned aerial vehicle by using the Pix4D software, and obtaining the DSM image and the orthographic image; the unmanned aerial vehicle is equipped with a high-resolution imaging RGB sensor for field image acquisition, and the collection time is between 11 o'clock in the afternoon and 1 o'clock in the afternoon; the flight height of the unmanned aerial vehicle is 15 m, the heading overlap rate is 70%, and the lateral overlap rate is 80%.

[0010] Preferably, the collected RGB image data is labeled by the global positioning system GPS built in the unmanned aerial vehicle, and the global positioning system GPS comprises black and white ceramic tiles as ground control points GCPs uniformly placed in the field, and a GPS receiver of Zhonghaida RTK is set as a reference station.

[0011] Preferably, the color features include the first moment, the second moment and the third moment, and the texture features are the contrast, the entropy, the energy and the inverse variance of 0°, 45° and 90° determined by the gray level co-occurrence matrix.

[0012] Preferably, the input of the RGB image into the VGG network and the extraction of the color, texture and RGB index time sequence features of the image further comprise: characteristic enhancement, feature augmentation, feature selection and feature conversion; wherein the feature augmentation comprises polynomial feature augmentation, and the feature selection comprises Pearson correlation analysis.

[0013] Preferably, it further comprises a soybean flowering period prediction system, comprising: A data acquisition module acquires the RGB image and the high elevation image of each germplasm resource soybean plot taken at multiple time points; according to each high elevation image data, the DSM is used to extract each pixel elevation point data of the field soybean of each plot, and the soybean plant height of each plot is determined; a plurality of components in the color space factor of RGB are selected for linear combination, and the canopy coverage rate of the field soybean of each plot is determined by threshold segmentation method; A flowering period prediction module is used for inputting the soybean plant height and the canopy coverage rate into a pre-trained VGG-LSTM model to predict the flowering period of soybean and output the flowering period prediction result. The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module; an RGB image is input into the VGG network, and the color, texture and RGB index time sequence features of the image are extracted; the attention module is used for significant feature extraction on a short sequence in the time sequence features, the weight of each time step feature in the short sequence is determined and weighted fusion is performed; and the fused features are input into the LSTM, and the flowering period prediction result is output by establishing a nonlinear mapping relationship between the fused features and the flowering period.

[0014] Compared with the prior art, the present application has the following beneficial effects: The soybean flowering period prediction method provided by the present application captures the time sequence change characteristics (such as the growth rate inflection point) of the soybean plant height as one of the core physiological indicators for judging the arrival of the flowering period, solving the problem of being unable to directly observe the flowering state under the canopy. The time sequence saturation characteristics of the soybean plant height are captured through the canopy coverage, which is used as another key indicator for judging the flowering period, making up for the lack of direct observation of the flowering characteristics. As key indicators affecting the flowering period, the plant height and the canopy coverage describe the soybean growth state from the vertical and horizontal dimensions respectively, avoiding the limitation of a single data source, improving the comprehensiveness of the features, and providing a theoretical basis for model prediction. The VGG-LSTM model constructed by the present application assigns weights to the time sequence features (such as the feature vector of each time step) output by the VGG through the attention module, focusing on the time points (such as the rapid growth period and the flower bud differentiation period) that contribute more to the flowering period prediction. Avoiding the equal treatment of all time steps by the LSTM, reducing noise interference (such as the decline in image quality on rainy days), highlighting the features of the key growth stages, and improving the sensitivity and interpretability of the model. The attention mechanism dynamically adjusts the importance of the features, and adapts to the differences in soybean phenotypes under different varieties and environments. The method realizes high-precision and high-robustness soybean flowering period prediction through multi-source data fusion, deep time sequence feature extraction and attention mechanism, and provides strong technical support for precision planting and variety selection in intelligent agriculture. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The soybean flowering period prediction method provided by the present application is a whole flowchart; Figure 2 The unmanned aerial vehicle provided by the embodiment of the present application is a schematic diagram of working in the field; Figure 3 The cell image acquisition flowchart provided by the embodiment of the present application is a flowchart; Figure 4 The time-based plant height extraction result diagram provided by the embodiment of the present application is a diagram; Figure 5 The time-based canopy extraction result diagram provided by the embodiment of the present application is a diagram; Figure 6A model for more feature extraction by HOG provided by the embodiment of the present application; Figure 7(a) is a feature enhancement flowchart provided by the embodiment of the present application; Figure 7(b) is a feature expansion flowchart provided by the embodiment of the present application; Figure 7(c) is a feature selection flowchart provided by the embodiment of the present application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings of the embodiments of the present application. Figures 1-7(c) The terms described in the present application are only used to describe the specific embodiments, and are not used to limit the present application.

[0017] EMBODIMENT As shown in the method flowchart of the present application, a soybean flowering period prediction method is disclosed, comprising the following steps: Figure 1 S1: Obtain RGB images and elevation images of each germplasm resource soybean plot taken at multiple time points; S2: According to each elevation image data, extract each pixel elevation point data of each plot field soybean through DSM, determine the soybean plant height of each plot, select multiple components in the color space factor of RGB for linear combination, and determine the canopy coverage of each plot field soybean through threshold segmentation method; S3: Input the soybean plant height and canopy coverage into the pre-trained VGG-LSTM model to predict the soybean flowering period, and output the flowering period prediction result; The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module; The RGB image is input into the VGG network to extract the color, texture and RGB index time sequence features of the image; the attention module is used to extract significant features in the short sequence in the time sequence features, determine the weight of each time step feature in the short sequence and perform weighted fusion; the fused features are input into the LSTM to output the flowering period prediction result by establishing a nonlinear mapping relationship between the fused features and the flowering period. The present application uses the plant height and canopy coverage which are strongly related to the flowering period of soybean and are easy to obtain through unmanned aerial vehicle remote sensing as phenotype indicators to replace the flowering characteristics under the canopy which cannot be directly observed. The plant height and canopy coverage are input into the VGG-LSTM model to realize accurate prediction of the flowering period.

[0018] As shown in the schematic diagram of the unmanned aerial vehicle working in the field, as shown in the plot image acquisition flowchart, the unmanned aerial vehicle flies over the plot according to the flight plan, and the RGB image and the elevation image of each plot are obtained.

[0019] Figure 2 As shown in the schematic diagram of the unmanned aerial vehicle working in the field, as shown in the plot image acquisition flowchart, the unmanned aerial vehicle flies over the plot according to the flight plan, and the RGB image and the elevation image of each plot are obtained. Figure 3 ​​Figure 3 The flow in the image data of the plot is collected. 2500 soybean germplasm resources are sown at the same time, and after germination, a UAV is equipped with a high-resolution imaging RGB sensor to obtain field image data. The collection time is between 11am and 1pm, the flight height of the UAV is 15m, the heading overlap rate is 70%, and the lateral overlap rate is 80%. The pictures obtained are marked by the global positioning system (GPS) built-in the UAV, and in order to correct the self-generated GPS measurement value, ground control point (GCP) black and white ceramic tiles are uniformly placed in the field, and a GPS receiver (Zhonghai RTK) is set as a reference station.

[0020] The single-point positioning result obtained by the UAV is compared with the reference station coordinates, and the real-time differential correction value is solved. The RGB picture data collected by the UAV is used for three-dimensional model reconstruction by Pix4D software. The obtained DSM image and orthographic image. Through programming, the cutting plot program is written to batch cut the synthesized ground plot, and the RGB and elevation image of each germplasm resource soybean plot is obtained.

[0021] As shown in Figure 4 The time height extraction result figure provided by the embodiment of the present application. Using DSM to extract each pixel elevation point data in each resource plot block, the highest 1% value of the elevation point is calculated as the mean value, and then the lowest 1% value of the elevation pixel point is subtracted as the mean value, and the difference between the two is taken as the average soybean plant height of each plot. Linear regression analysis is performed on the plant height extracted by high-throughput unmanned aerial vehicle and the plant height extracted by high-throughput unmanned aerial vehicle. The accuracy of high-throughput extraction.

[0022] The present application calculates the average soybean plant height of each plot through the elevation image of the unmanned aerial vehicle, captures the time sequence change characteristics (such as the inflection point of the growth rate) of the plant height, and uses it as one of the core physiological indicators for judging the coming of the flowering period. The problem of not being able to directly observe the flowering state under the canopy is solved.

[0023] Soybean plant height directly reflects the development stage of soybean plant (such as vegetative growth period, reproductive growth period), which is a core index for predicting flowering period. For example, the LSTM model can capture the growth rate mutation point through the time sequence data (such as weekly growth) of soybean plant height, and locate the starting time of the flowering period.

[0024] The canopy coverage rate refers to the ratio of the green part of the crop in the unit area to the total area on the field.

[0025] The present application extracts the canopy coverage rate by linear combination of RGB image color components + threshold segmentation method, captures its time sequence saturation characteristics, and uses it as another key indicator for judging the flowering period, which makes up for the lack of direct observation of flowering characteristics.

[0026] Canopy cover can indirectly reflect resource utilization efficiency (such as light interception rate), and its dynamic changes (such as a decline in cover rate) may indicate the cessation of vegetative growth and the onset of reproductive growth. For example, attention mechanisms can focus on periods when canopy cover growth stagnates, assigning higher weight to features during those periods to improve prediction accuracy.

[0027] In other words, soybean plant height reflects vertical growth, while canopy coverage reflects horizontal expansion; together, they constitute the soybean plant's "space-occupying capacity." When resources are limited (such as in low-fertility soil), soybean plants may prioritize increasing plant height to compete for sunlight, resulting in lower canopy coverage. However, when resources are abundant, both increase simultaneously, forming a "dwarf and sturdy" or "high-density" canopy structure, which affects the flowering process.

[0028] In practical applications, crop coverage calculation is simplified to the number of pixels occupied by the target crop in an image divided by the total number of pixels in the image. Therefore, the prerequisite for calculating crop canopy coverage is obtaining the number of pixels occupied by the crop, and the key lies in segmenting the crop from a complex background. Image segmentation divides an image into two parts: a foreground region and a background region, where the foreground region is the region of interest. When drones collect images of soybean fields, the soybean canopy is green, which is significantly different from the soil background color. Color features in the color space are used to separate the target crop from the background region. Threshold segmentation based on RGB color factors has 256 brightness levels in the R, G, and B channels, and the values ​​of the three channels are all in the range of [0, 255]. However, extracting the canopy by taking only one component is not ideal, as the crop in the image is mixed with the soil background. By using two or three components in the RGB color space to linearly combine and obtain color factors, these color factors can enhance the contrast between the soybean canopy and the background region in the image. Combined with the threshold segmentation method to find the segmentation threshold, the soybean canopy can be separated from the background.

[0029] like Figure 5 The image shown is a temporal canopy extraction result. Feature extraction was performed on the acquired image. Image features include color (first moment, second moment, and third moment) and texture (contrast at 0°, 45°, and 90°, entropy (eng), energy (agm), and inverse variance (idm) calculated using the gray-level co-occurrence matrix (GLMC).

[0030] RGB index feature extraction, such as Figure 6 As shown, a soybean flowering identification model is established by combining plant height and canopy coverage features with HOG for further feature extraction. The currently selected algorithms are: Random Forest (RF); Fully Connected Neural Network (MLP); and Gradient Boosting Decision Tree (XGBoost).

[0031] The logic regression algorithm is used to determine whether the image plot of the date is in flower (2500 soybean germplasm resources do not flower on June 15, and the image features of the materials on June 15 are used as the reference, combined with the image features collected by the unmanned aerial vehicle on the following dates), for example: on June 29, the image features combined with the reference are used to determine that 300 soybean materials have already flowered (June 29 is the initial flowering period of the 300 soybean materials), and then on July 1, the image features combined with the reference determine that 500 soybean materials have flowered, and finally the judgment of whether the soybean materials of all unmanned aerial vehicle collection dates have flowered is completed. Finally, the regression analysis of the machine learning soybean material date and the hand-picked soybean material flowering period is carried out.

[0032] As shown in FIG. 7(a), a feature enhancement flowchart provided for the embodiment includes feature enhancement, feature construction, feature selection, and feature conversion. Among them, the feature enhancement includes data cleaning, standardization and normalization processing on the data; as shown in FIG. 7(b), the purpose of feature construction is to use existing features to construct new features, so that the model learns from them; as shown in FIG. 7(c), the feature selection process performs Pearson correlation analysis, eliminates features with high noise, performs statistical feature selection, and performs Pearson correlation coefficient hypothesis verification; the feature conversion adopts principal component analysis (PCA), and the main idea is to map n-dimensional features to k-dimensional features. The k-dimensional features are new orthogonal features also known as principal components, which are k-dimensional features reconstructed on the basis of the original n-dimensional features.

[0033] The constructed VGG-LSTM model includes a convolutional neural network VGG and an LSTM network connected through an attention module.

[0034] The RGB image is input into the VGG network to extract the color, texture and RGB index time sequence features of the image; the attention module is used to extract significant features from the short sequence in the time sequence features, determine the weight of each time step feature in the short sequence and perform weighted fusion; the fused features are input into the LSTM to output the flowering period prediction result by establishing a nonlinear mapping relationship between the fused features and the flowering period.

[0035] Specifically, the VGG-LSTM model includes a convolutional neural network VGG and an LSTM network connected through an attention module, wherein: The convolutional neural network VGG includes an input layer, a convolutional layer and a pooling layer; In the input layer, the RGB images of each germplasm resource soybean plot taken at multiple time points are input in time sequence; in the convolutional layer and the pooling layer, the time sequence features of the color, texture and RGB index of the image are gradually extracted through multi-layer convolution and pooling operations; The local time sequence segment in the time sequence feature is taken as a short sequence, taken as an input of the attention module, and the weight of each time step feature in the short sequence is determined through a neural network; each time step feature is weighted with the corresponding weight to obtain a fused feature; and the fused feature is input into an LSTM network; The LSTM network comprises an input layer, a hidden state updating layer and an output layer. In the hidden state updating layer, the hidden state of the current time step is determined according to the feature of the current time step received by the input layer and the hidden state of the previous time step; and in the output layer, the hidden state of the last time step is used to map an output flowering period prediction result through a fully connected layer.

[0036] A single plant height or canopy coverage index is easily disturbed by environmental factors such as temperature, water and fertilizer, resulting in a deviation in the judgment of the flowering period. In the present application, the time sequence growth characteristics of plant height and the time sequence saturation characteristics of canopy coverage are input into a pre-trained VGG-LSTM model to realize accurate prediction through the following mechanism:

[0037] The VGG network extracts the color, texture and other detail features of the RGB image to assist in correcting the calculation accuracy of the canopy coverage.

[0038] The attention module weights and fuses the time sequence short sequence features of the plant height and the canopy coverage, highlights the key time step features (such as the inflection point of the plant height growth rate and the peak value of the canopy coverage) that are strongly related to the flowering period. The LSTM network mines the coupling rules of the time sequence changes of the two indexes, establishes a nonlinear mapping relationship between the phenotype characteristics and the flowering period, and finally outputs an accurate flowering period prediction result.

[0039] Based on the obtained image features and time sequence, a hybrid deep learning algorithm network (VGG-LSTM) model is constructed to predict the flowering time. Time sequence is generally defined according to the order of occurrence time, and time sequence prediction has important application value. Time sequence data has time sequence correlation, and time sequence features have different importance in the time dimension. Long short-term memory network (LSTM) has a certain ability to mine long-distance time sequence data. However, training a single LSTM may cause instability and gradient disappearance, and cannot capture very long-term interdependence. VGG has the ability to abstract high-dimensional features from short sequence features. Combined with the attention module, the influence of important time sequence features in the model is enhanced, and the influence of non-important features in the model is reduced. Convolutional neural network VGG mainly includes convolutional layers and pooling layers.

[0040] The pooling layer retains strong features and removes weak features of the convolutional layer, reduces the number of parameters, and prevents overfitting.

[0041] The application extracts original image features through VGG, and transmits the processed time sequence features into an LSTM network for long sequence prediction. The attention mechanism is used to extract significant features of short sequences. The soft attention mechanism is used to perform weighted processing by using the weight values trained by the neural network. The LSTM recurrent neural network trains the VGG feature extraction results in combination with the feature fusion and attention mechanism module. The whole process is end-to-end, and finally the flowering time of soybeans is predicted through time sequence images.

[0042] The method provided by the application describes the growth state of soybeans from the vertical and horizontal dimensions respectively by the obtained plant height and canopy coverage, avoids the limitation of a single data source, improves the comprehensiveness of features, and provides a theoretical basis for model prediction. The VGG-LSTM model constructed by the application assigns weights to the time sequence features (such as feature vectors at each time step) output by VGG through the attention module, focuses on time points (such as the rapid growth period and the flower bud differentiation period) that contribute more to the flowering prediction, avoids equal processing of all time steps by LSTM, reduces noise interference (such as the decline in image quality on rainy days), highlights the features of key growth stages, and improves the sensitivity and interpretability of the model. The attention mechanism dynamically adjusts the importance of features to adapt to the phenotypic differences of soybeans in different varieties and environments.

[0043] The VGG-LSTM model can automatically learn the complex nonlinear relationship (such as interaction and threshold effect) between plant height, canopy coverage and flowering period, and avoid the limitations of traditional statistical models (such as linear regression). For example, the model may find that the flowering probability significantly increases when the plant height is greater than 50 cm and the canopy coverage is greater than 80%.

[0044] The application further provides a soybean flowering period prediction system, which comprises: The data acquisition module acquires RGB images and elevation images of each germplasm resource soybean plot taken at multiple time points, extracts the elevation point data of each pixel of the field soybeans in each plot from each elevation image data through DSM to determine the soybean plant height of each plot, and selects multiple components in the color space factor of RGB for linear combination and determines the canopy coverage of the field soybeans in each plot through threshold segmentation. The flowering period prediction module is used for inputting the soybean plant height and canopy coverage into the pre-trained VGG-LSTM model to predict the flowering period of soybeans and output the flowering period prediction result. The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module; the RGB image is input into the VGG network, and the color, texture and time sequence features of the RGB index of the image are extracted; the attention module is used to extract significant features from the short sequence in the time sequence features, determine the weight of each time step feature in the short sequence and perform weighted fusion; the fused features are input into the LSTM, and the nonlinear mapping relationship between the fused features and the flowering period is established, and the flowering period prediction result is output.

[0045] The method realizes high-precision and high-robustness soybean flowering period prediction through multi-source data fusion, deep time sequence feature extraction and attention mechanism, and provides strong technical support for precision planting and variety breeding in intelligent agriculture.

[0046] The application utilizes the unmanned aerial vehicle platform to model the flowering traits of soybeans through machine learning and deep learning algorithms, can reduce labor costs, and accurately performs high-throughput identification on the flowering traits of soybean field germplasm resources.

[0047] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

[0048] In addition, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs, unless otherwise specified. All documents mentioned in the specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of any conflict with any incorporated document, the content of the specification shall prevail.

Claims

1. A method for predicting the flowering period of soybean, characterized by, The method comprises the following steps: Obtain RGB images and elevation images of each germplasm resource soybean plot at multiple time points; According to each elevation image data, extract the elevation point data of each pixel of the field soybean in each plot by DSM, and determine the soybean plant height of each plot; Select multiple components in the color space factor of RGB for linear combination, and determine the canopy coverage of the field soybean in each plot by threshold segmentation method; Input the soybean plant height and canopy coverage into the pre-trained VGG-LSTM model to predict the flowering period of soybean and output the flowering period prediction result; The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module; The RGB image is input into the VGG network to extract the time sequence features of color, texture and RGB index of the image; the attention module is used to extract significant features from the short sequence in the time sequence features, determine the weight of each time step feature in the short sequence and perform weighted fusion; the fused features are input into the LSTM to output the flowering period prediction result by establishing a nonlinear mapping relationship between the fused features and the flowering period.

2. The method of claim 1, wherein, The VGG-LSTM model comprises a convolutional neural network VGG and an LSTM network connected through an attention module, wherein: The convolutional neural network VGG comprises an input layer, a convolutional layer and a pooling layer; In the input layer, the RGB images of each germplasm resource soybean plot at multiple time points are input in time sequence; in the convolutional layer and the pooling layer, the time sequence features of color, texture and RGB index of the image are gradually extracted through multi-layer convolution and pooling operation; Local time sequence segments in the time sequence features are taken as short sequences and input into the attention module, and the weight of each time step feature in the short sequence is determined through a neural network; each time step feature is weighted with the corresponding weight to obtain fused features; the fused features are input into the LSTM network; The LSTM network comprises an input layer, a hidden state updating layer and an output layer; In the hidden state updating layer, the hidden state of the current time step is determined according to the feature of the current time step received by the input layer and the hidden state of the previous time step; in the output layer, the hidden state of the last time step is used to map the flowering period prediction result through a fully connected layer. According to each elevation image data, extract the elevation point data of each pixel in each germplasm resource soybean plot block by DSM, take the highest value of 1% of the elevation points as the first average value, take the lowest value of 1% of the elevation points as the second average value, and take the difference between the two as the average soybean plant height of each plot.

3. The method of claim 1, wherein, ​ ​ 4. The method of claim 1, wherein, The acquisition of the RGB image and the elevation image of each germplasm resource soybean plot at multiple time points comprises collecting RGB image data of the soybean germplasm resource by a UAV, and reconstructing a three-dimensional model of the UAV-collected RGB image data by using Pix4D software to obtain a DSM image and an orthographic image; the UAV is equipped with a high-resolution imaging RGB sensor for field image acquisition, and the collection time is between 11 am and 1 pm; the flight height of the UAV is 15 m, the heading overlap rate is 70%, and the lateral overlap rate is 80%.

5. The method of claim 4, wherein the method is characterized by, The collected RGB image data is labeled by a global positioning system (GPS) built in the UAV, wherein the global positioning system (GPS) comprises uniformly placed ground control points (GCPs) in the field, which are black and white ceramic tiles, and a GPS receiver of Zhonghai RTK is set as a reference station.

6. The method of claim 1, wherein, The color features comprise first moments, second moments and third moments, and the texture features are determined by a gray level co-occurrence matrix at 0°, 45° and 90° contrast, entropy, energy and inverse variance.

7. The method of claim 1, wherein, The RGB image is input into a VGG network to extract the color, texture and RGB index time sequence features of the image, and the method further comprises: performing feature enhancement, feature augmentation, feature selection and feature conversion on the time sequence features, wherein the feature augmentation comprises polynomial feature augmentation, and the feature selection comprises Pearson correlation analysis.

8. A soybean flowering prediction system, characterized by, The method comprises: a data acquisition module that acquires RGB images and elevation images of each soybean plot of a germplasm resource at multiple time points; according to each elevation image data, extracts each pixel elevation point data of the field soybean in each plot by DSM, and determines the soybean plant height of each plot; selecting multiple components in the color space factor of RGB for linear combination, and determining the canopy coverage of the field soybean in each plot by threshold segmentation; a flowering period prediction module for inputting the soybean plant height and the canopy coverage into a pre-trained VGG-LSTM model to predict the flowering period of the soybean and output a flowering period prediction result; The VGG-LSTM model comprises a convolutional neural network (VGG) and an LSTM network connected by an attention module; the RGB image is input into the VGG network to extract the color, texture and RGB index time sequence features of the image; the attention module is used to extract significant features from short sequences in the time sequence features, determine the weight of each time step feature in the short sequences and perform weighted fusion; the fused features are input into the LSTM to output the flowering period prediction result by establishing a nonlinear mapping relationship between the fused features and the flowering period.

Images