Cotton yield prediction method, device, equipment and medium

Abstract

The invention provides a cotton yield prediction method and device, equipment and a medium, and relates to the field of cotton yield prediction, and the method comprises the steps: obtaining an RGB image and a multispectral image of to-be-predicted cotton in a flowering and boll-setting period; based on the RGB image and the multispectral image, determining the growth state of the to-be-predicted cotton and agricultural condition parameters of a cotton field where the to-be-predicted cotton is planted; inputting the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain predicted chlorophyll of the to-be-predicted cotton in a flowering and boll setting period; inputting the growth state and the predicted chlorophyll into a pre-trained cotton yield prediction model to obtain the predicted yield of the to-be-predicted cotton; wherein the cotton yield prediction model is obtained by training chlorophyll detection values and growth states of different varieties of cotton in historical flowering and boll-forming periods and yields of corresponding cotton in historical harvesting periods. The cotton chlorophyll content can be accurately predicted, the cotton yield can be efficiently predicted, and support is provided for cotton germplasm screening and accurate breeding.

Description

Technical Field

[0001] This application relates to the field of cotton yield prediction. Specifically, it relates to a method, device, equipment, and medium for predicting cotton yield. Background Art

[0002] As an important cash crop, improving the yield and quality of cotton is of great significance for ensuring the stable supply of the global textile industry and promoting the development of the agricultural economy. Chlorophyll, as the core pigment of cotton photosynthesis, its content directly reflects the nutritional status, health level, and growth vitality of cotton plants, and is closely related to the final cotton yield. Therefore, accurately monitoring the chlorophyll content of cotton and predicting the yield not only provides a scientific basis for cotton planting management, promotes the development of precision agriculture, but also helps to address challenges such as climate change and resource shortages, and achieve sustainable agricultural production.

[0003] Traditional methods for monitoring chlorophyll content rely on manual sampling and laboratory analysis, which have the defects of being time-consuming, low in efficiency, and difficult to conduct large-scale and real-time monitoring, and cannot meet the requirements of modern precision agriculture for high-throughput and high-precision monitoring. Most existing cotton yield prediction methods rely on historical data and empirical models, lacking real-time monitoring and dynamic analysis of key physiological indicators, resulting in low prediction accuracy and difficulty in accurately reflecting the changes in actual yield. Although unmanned aerial vehicle (UAV) remote sensing technology has become an important tool for agricultural monitoring due to its advantages such as high efficiency, non-destructiveness, and flexibility, how to effectively integrate multi-source remote sensing data (such as multispectral and RGB data) to achieve accurate monitoring of chlorophyll content and accurate prediction of yield remains a technical problem to be solved urgently.

[0004] In the prior art, multispectral data and RGB data each have their own advantages and disadvantages: multispectral data has rich spectral information, but its spatial resolution is low; RGB data has high spatial resolution, but lacks red edge and near-infrared bands. Therefore, how to effectively fuse multi-source data and improve the accuracy of chlorophyll content and yield prediction is the current research difficulty. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method, device, equipment, and medium for predicting cotton yield, which solves the above problems existing in the prior art, can achieve accurate prediction of cotton chlorophyll content and efficient prediction of cotton yield, and provides support for cotton germplasm screening and precision breeding.

[0006] In a first aspect, the present invention provides a method for predicting cotton yield, the method comprising:

[0007] Obtain the RGB image and multispectral image of the cotton to be predicted during the flowering and boll-forming period;

[0008] Determining the growth status of the cotton to be predicted and agricultural parameters of a cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image;

[0009] Inputting the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain a predicted chlorophyll value of the cotton to be predicted at the flowering and boll stage; wherein the chlorophyll prediction model is trained using chlorophyll detection values of different varieties of cotton at historical flowering and boll stages and historical agricultural condition parameters of cotton fields planted with the corresponding cotton at historical flowering and boll stages;

[0010] The growth status and the predicted chlorophyll are input into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted; wherein the cotton yield prediction model is trained using the chlorophyll detection values and growth status of different varieties of cotton during the historical flowering and boll period and the yield of the corresponding cotton during the historical harvest period.

[0011] In an optional embodiment, after acquiring the RGB image and the multispectral image of cotton at the flowering and boll stage, the method further includes:

[0012] The RGB image and the multispectral image are preprocessed respectively to obtain an RGB digital orthoimage and a multispectral digital orthoimage.

[0013] In an optional embodiment, the agricultural condition parameters include: vegetation index and color index;

[0014] The vegetation index includes: chlorophyll index, soil adjusted vegetation index, normalized difference red edge index, green-red vegetation index and vegetation red edge index;

[0015] The color index includes: vegetation extraction color index, excess green index and visible difference vegetation index;

[0016] The multispectral image includes spectral data of cotton in different wavelength bands.

[0017] In an optional embodiment, determining agricultural parameters of a cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image includes:

[0018] For any vegetation index to be calculated, match the target band corresponding to the vegetation index to be calculated from the configured comparison table of different vegetation indices and different bands;

[0019] Extract the spectral data of the target band from the multispectral digital orthoimage to obtain the target spectral data;

[0020] The vegetation index to be calculated is calculated based on the target spectral data.

[0021] In an alternative embodiment, based on the RGB image and the multispectral image, determining the agricultural condition parameters of the cotton field where the cotton to be predicted is planted further includes:

[0022] Input the RGB digital orthophoto image and the multispectral digital orthophoto image into a pre-trained cotton canopy segmentation and vectorization model to obtain the cotton canopy vector boundary;

[0023] Calculate the color index according to the pixels within the cotton canopy vector boundary.

[0024] In an alternative embodiment, the growth state includes: the plant type and leaf area index of the cotton plant;

[0025] Based on the RGB image and the multispectral image, determining the growth state of the cotton to be predicted includes:

[0026] Input the RGB digital orthophoto image and the multispectral digital orthophoto image into a pre-trained cotton plant classification model to obtain the plant type of the cotton to be predicted;

[0027] Input the RGB digital orthophoto image and the multispectral digital orthophoto image into a pre-trained leaf area multi-modal estimation model to obtain the leaf area index of the cotton to be predicted.

[0028] In an alternative embodiment, the cotton yield prediction model is constructed using the multiple linear regression algorithm based on the correlation relationship between the growth state, chlorophyll change data, chlorophyll spatial distribution data of different varieties of cotton during the historical flowering and boll-setting period, and the yield of the corresponding different varieties of cotton during the historical harvest period.

[0029] In a second aspect, the present invention provides a device for predicting cotton yield, the device includes:

[0030] An acquisition unit for acquiring the RGB image and the multispectral image of the cotton to be predicted during the flowering and boll-setting period;

[0031] A determination unit for determining the growth state of the cotton to be predicted and the agricultural condition parameters of the cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image;

[0032] A first prediction unit for inputting the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain the predicted chlorophyll of the cotton to be predicted during the flowering and boll-setting period; wherein, the chlorophyll prediction model is trained using the chlorophyll detection values of different varieties of cotton during the historical flowering and boll-setting period and the historical agricultural condition parameters of the cotton fields where the corresponding cotton is planted during the historical flowering and boll-setting period;

[0033] A second prediction unit, configured to input the growth state and the predicted chlorophyll into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted; wherein, the cotton yield prediction model is trained by using the chlorophyll detection values, growth states of different varieties of cotton during the historical flowering and boll-setting periods, and the yields of the corresponding cotton at the historical harvest periods.

[0034] In a third aspect, the present invention provides an electronic device, which includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0035] The memory is used to store a computer program;

[0036] The processor is configured to implement the method according to any one of the foregoing embodiments when executing the program stored on the memory.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program implements the method according to any one of the foregoing embodiments when executed by a processor.

[0038] The present application realizes the efficient and non-destructive monitoring of the chlorophyll content of cotton, significantly improving the monitoring efficiency; the present application combines the advantages of multi-spectral and RGB data to improve the accuracy of chlorophyll content and yield prediction; the present application deeply understands the cotton growth dynamics by predicting the chlorophyll content, providing a scientific basis for precision agriculture; the present application can play an important role in the process of obtaining and analyzing the phenotypic traits of large-scale populations in the field at multiple growth stages.

[0039] The present application can realize the efficient prediction of cotton yield, providing support for cotton germplasm screening and precision breeding. Description of the Drawings

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a flowchart of a method for predicting the yield of cotton provided by an embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of using a drone to collect red, green, and blue (RGB) data and multi-spectral data and further process and extract color indices and vegetation indices provided by an embodiment of the present application;

[0043] Figure 3 A schematic diagram provided by an embodiment of the present application for screening important features of cotton at different growth stages using the hierarchical segmentation method;

[0044] Figure 4 A comparison diagram provided by an embodiment of the present application for comparing the predicted leaf chlorophyll content (LCC) with the measured leaf chlorophyll content using different models and data sources;

[0045] Figure 5 A comparison diagram provided by an embodiment of the present application for the comparison of the measured cotton yield and the predicted yield obtained from remote sensing data at different growth stages;

[0046] Figure 6 A comparison diagram provided by an embodiment of the present application for the cluster analysis of 419 cotton germplasms at different growth stages and the dynamic change of leaf chlorophyll content (LCC);

[0047] Figure 7 A schematic structural diagram of a cotton yield prediction device provided by an embodiment of the present application;

[0048] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0049] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0050] The cotton yield prediction method provided by the embodiments of this application can be applied to a server or a terminal with strong computing power. The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The terminal can be a user equipment (UE) such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a personal digital assistant (PDA), a tablet computer (PAD), a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing devices connected to a wireless modem, a mobile station (MS), a mobile terminal, etc. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, which are not limited in this application.

[0051] The preferred embodiments of this application will be described below in conjunction with the accompanying drawings of the specification. It should be understood that the preferred embodiments described here are only used to illustrate and explain this application and are not used to limit this application. And without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0052] Figure 1 It is a schematic flowchart of a cotton yield prediction method provided by the embodiments of this application. As Figure 1 shown, the method may include:

[0053] S110. Obtain the RGB image and the multispectral image of the cotton to be predicted during the flowering and boll-forming period, and based on the RGB image and the multispectral image, determine the growth state of the cotton to be predicted and the agricultural situation parameters of the cotton field where the cotton to be predicted is planted.

[0054] In the embodiments of this application, the flowering and boll-forming period refers to the period from the cotton flowering to the beginning of boll opening, usually occurring from early July to the end of August or early September, lasting about 45 - 60 days; the RGB image and the multispectral image of the cotton field where the cotton is planted can be collected at a certain moment on any day during the flowering and boll-forming period; or the RGB image and the multispectral image of the cotton field where the cotton is planted can be collected at multiple same or different moments on multiple different dates during the flowering and boll-forming period.

[0055] In the embodiments of this application, the RGB image and the multispectral image are collected by an image acquisition device set on a drone; the multispectral image includes spectral data of the cotton in different bands.

[0056] In the embodiments of the present application, after obtaining the RGB image and the multispectral image of the cotton to be predicted during the flowering and boll-setting period, it is necessary to preprocess the RGB image and the multispectral image to obtain the RGB digital orthophoto image and the multispectral digital orthophoto image; wherein, the preprocessing methods include at least one of the following: radiometric calibration, geometric correction, image alignment, camera alignment optimization, dense point cloud generation, depth map creation, generation of the RGB digital orthophoto image model, and generation of the multispectral digital orthophoto image model.

[0057] In the embodiments of the present application, the growth state of the cotton includes: the plant type, size, and leaf area index of the cotton plant; the agricultural situation parameters include: vegetation index, color index, water stress index, nitrogen content, and incidence of pests and diseases; specifically, the vegetation index includes: chlorophyll index (LCI), soil-adjusted vegetation index (SAVI), normalized difference red edge index (NDRE), green-red vegetation index (GRVI), and vegetation red edge index (VREI); the color index includes: vegetation extraction color index (CIVE), excess green index (ExG), and visible difference vegetation index (VDVI).

[0058] In the embodiments of the present application, the above-mentioned vegetation index and color index of the present application are indices of chlorophyll sensitivity obtained by analyzing historical data; the determination methods for the vegetation index to be calculated and the color index to be calculated include:

[0059] Obtain the RGB image data and multispectral image data of different varieties of cotton planted in different regions of the cotton field at different growth stages by a drone; collect the measured chlorophyll values of different varieties of cotton in each region of the cotton field by a chlorophyll analyzer respectively; wherein, the time period between the collection time corresponding to the drone and the collection time corresponding to the chlorophyll analyzer is less than the specified time period.

[0060] Preprocess the RGB image data and the multispectral image data; analyze the texture features and color features of the preprocessed RGB image, use the gray-level co-occurrence matrix (GLCM) or other texture analysis methods to extract the texture information of the preprocessed RGB image, and further enhance the features related to the chlorophyll content.

[0061] Based on color features and texture features, color indices highly correlated with chlorophyll content are screened out; the color indices include, but are not limited to, the Normalized Green Red Difference Index (NGRDI), the Green Vegetation Index (GVI), etc.; the spectral features of the preprocessed multispectral image data are analyzed, and by calculating the reflectance and absorbance of different bands, spectral features related to chlorophyll content are extracted; based on the spectral features, vegetation indices highly correlated with chlorophyll content are screened out; the features of the RGB image data and the multispectral image data are further screened and optimized using a hierarchical segmentation method to exclude interference information, and the correlation between the screened color indices (CIs) and vegetation indices (VIs) and chlorophyll content is evaluated to ensure the high correlation and stability of the selected features.

[0062] In the embodiment of the present application, the method for determining the vegetation index to be calculated and the color index to be calculated further includes:

[0063] Input the RGB digital orthophoto into the cotton canopy segmentation and vectorization model, and automatically perform image classification and divide the cotton canopy range based on the RGB digital orthophoto through the cotton canopy segmentation and vectorization model to obtain the cotton canopy vector boundaries of each cotton variety at different growth stages;

[0064] According to the cotton canopy vector boundaries of each cotton variety at different growth stages, the mean values of the color indices corresponding to different cotton varieties are extracted;

[0065] Based on the multispectral digital orthophoto, multiple vegetation indices sensitive to chlorophyll are calculated, and according to the cotton canopy vector boundaries, the mean values of the vegetation indices corresponding to different cotton varieties are extracted;

[0066] Based on the RGB image data and the multispectral image data, the hierarchical segmentation method is used to screen out the optimal vegetation indices and color indices highly correlated with cotton chlorophyll; 10 vegetation indices are extracted from the multispectral image, and 8 color indices are extracted from the RGB image;

[0067] The features of the RGB image and the multispectral image are further screened and optimized using hierarchical segmentation technology to exclude interference information, and the correlation between the screened color indices and vegetation indices and chlorophyll content is evaluated to ensure the high correlation and stability of the selected features, and 5 vegetation indices (LCI, SAVI, NDRE, GRVI, VREI) and 3 color indices (CIVE, ExG, VDVI) highly correlated with chlorophyll content are screened out.

[0068] In the embodiment of the present application, based on the RGB image and the multispectral image, the agronomic parameters of the cotton field where the cotton to be predicted is planted are determined, including:

[0069] For any vegetation index to be calculated, match the target band corresponding to the vegetation index to be calculated from the configured comparison tables of different vegetation indices and different bands; extract the spectral data of the target band from the multispectral digital orthophoto image to obtain the target spectral data; calculate the vegetation index to be calculated based on the target spectral data.

[0070] Input the RGB digital orthophoto image and the multispectral digital orthophoto image into a pre-trained cotton canopy segmentation and vectorization model to obtain the cotton canopy vector boundary; calculate the color index according to the pixels within the cotton canopy vector boundary.

[0071] In the embodiments of the present application, different vegetation indices do correspond to different bands or band combinations; each vegetation index evaluates different characteristics of vegetation using the reflectance differences between specific bands (such as red light, near-infrared light, green light, etc.); the multispectral digital orthophoto image includes spectral data of the blue light band, green light band, red light band, red edge band, and near-infrared band; the chlorophyll index corresponds to the red edge band (RedEdge) and the near-infrared band (NIR); the soil-adjusted vegetation index corresponds to the red light band (Red) and the near-infrared band (NIR); the normalized difference red edge index corresponds to the red edge band (RedEdge) and the near-infrared band (NIR); the green-red vegetation index corresponds to the green light band (Green) and the near-infrared band (NIR); the vegetation red edge index corresponds to the red edge band (RedEdge) and the near-infrared band (NIR).

[0072] In the embodiments of the present application, the cotton canopy segmentation and vectorization model includes:

[0073] An input layer for inputting the RGB digital orthophoto image and the multispectral orthophoto image;

[0074] A preprocessing layer for preprocessing the RGB digital orthophoto image and the multispectral orthophoto image;

[0075] An encoder for extracting the features of the preprocessed RGB digital orthophoto image and the multispectral orthophoto image to obtain a first feature map;

[0076] A bottleneck layer for compressing the features of the first feature map to obtain a second feature map;

[0077] A decoder for generating a segmentation mask according to the second feature map and the first feature map;

[0078] A boundary extraction and vectorization layer for extracting the boundary from the segmentation mask and vectorizing it to obtain the cotton canopy vector boundary.

[0079] In the embodiments of the present application, the method for predicting cotton yield further includes:

[0080] Obtain the thermal infrared image of the cotton to be predicted and the air temperature in the area where the cotton field for planting the cotton to be predicted is located; preprocess the thermal infrared image to obtain the infrared temperature image of the cotton leaves; extract the surface temperature of the cotton leaves from the infrared temperature image of the cotton leaves; take the difference between the air temperature and the surface temperature of the cotton leaves as the water stress index;

[0081] Input the calculated vegetation index and color index into the pre-trained nitrogen prediction model to predict the nitrogen content of the cotton to be predicted; input the calculated vegetation index and color index into the pre-trained pest and disease prediction model to predict the incidence of pests and diseases of the cotton to be predicted.

[0082] In the embodiments of the present application, the nitrogen prediction model can be a regression model (such as linear regression, random forest, support vector machine, etc.), which is trained using the nitrogen detection values (obtained through cotton leaf analysis) of different varieties of cotton during the historical flowering and boll-setting period and the corresponding vegetation index and color index; the pest and disease prediction model is a machine learning model (such as random forest, support vector machine, convolutional neural network, etc.), which is trained using the probabilities of different varieties of cotton planted in the cotton field for planting the cotton to be predicted without being affected by pests and diseases and the probabilities of being affected by pests and diseases and the corresponding vegetation index and color index during the historical time period.

[0083] In the embodiments of the present application, based on the RGB image and the multispectral image, determine the growth status of the cotton to be predicted, including:

[0084] Input the RGB digital orthophoto image and the multispectral digital orthophoto image into the pre-trained cotton plant classification model to obtain the plant type of the cotton to be predicted; input the RGB digital orthophoto image and the multispectral digital orthophoto image into the pre-trained leaf area multi-modal estimation model to obtain the leaf area index of the cotton to be predicted.

[0085] In the embodiments of the present application, the leaf area multi-modal estimation model includes:

[0086] An input layer for inputting the RGB digital orthophoto image (H×W×3) and the multispectral digital orthophoto image (H×W×N), where N is the number of multispectral bands;

[0087] A data preprocessing layer for performing convolution, batch normalization, and max pooling on the RGB digital orthophoto image (H×W×3) and the multispectral digital orthophoto image (H×W×N) to extract a preliminary feature map; calculating various vegetation indices to obtain a vegetation index map;

[0088] A multi-modal feature fusion layer, which is used to perform weighted fusion on feature maps of different modalities using a channel attention mechanism; fuse the RGB and multi-spectral feature maps through Concatenate or Add operations to obtain a fused feature map (H / 2×W / 2×128);

[0089] A feature pyramid network, which is used to construct a feature pyramid network, extract features at different scales, and enhance the ability to capture multi-scale information; use a top-down path and lateral connections to fuse features at different levels to obtain a multi-scale feature map set;

[0090] A Transformer encoder-decoder, which is used to perform global feature extraction on the multi-scale feature map set; gradually restore the spatial resolution, and combine local context information to output a high-resolution feature map;

[0091] A global pooling layer, which is used to reduce the dimension of the high-resolution feature map to obtain a feature vector;

[0092] A regression layer, which is used to perform feature transformation on the feature vector to obtain the leaf area index.

[0093] In the embodiments of the present application, the size of the plant can be predicted through RGB digital orthophoto images and multi-spectral digital orthophoto images, or geometric images of cotton can be collected and obtained by using geometric relationships and depth estimation.

[0094] S120: Input the agricultural situation parameters into a pre-trained chlorophyll prediction model to obtain the predicted chlorophyll of the cotton to be predicted during the flowering and boll-setting period, and input the growth state and the predicted chlorophyll into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted.

[0095] In the embodiments of the present application, the chlorophyll prediction model is trained using the chlorophyll detection values of different varieties of cotton during the historical flowering and boll-setting period (obtained by actually measuring the chlorophyll of cotton leaves) and the historical agricultural situation parameters of the cotton fields where the corresponding cotton is planted during the historical flowering and boll-setting period.

[0096] In the embodiments of the present application, the chlorophyll prediction model is one or more of machine learning models such as random forest (RF), support vector machine (SVM), partial least squares regression (PLSR), and ridge regression (RR).

[0097] In the embodiments of the present application, the cotton yield prediction model is trained using the chlorophyll detection values, growth states of different varieties of cotton during the historical flowering and boll-setting period, and the yields of the corresponding cotton during the historical harvest period.

[0098] In the embodiments of the present application, the cotton yield prediction model is constructed by using the multiple linear regression algorithm based on the correlation relationship between the growth states, chlorophyll change data, chlorophyll spatial distribution data of different cotton varieties during the historical flowering and boll-setting periods, and the yields of the corresponding different cotton varieties during the historical harvest period.

[0099] In the embodiments of the present application, the method for obtaining the correlation relationship between the growth states, chlorophyll change data, chlorophyll spatial distribution data of different cotton varieties during the historical flowering and boll-setting periods, and the yields of the corresponding different cotton varieties during the historical harvest period includes:

[0100] Based on the chlorophyll detection model, perform K-means clustering analysis and GIS spatial analysis. Divide 419 cotton varieties into 3 clusters based on the chlorophyll dynamic changes, analyze the chlorophyll dynamics and yield performance of different clusters, obtain the chlorophyll change data and chlorophyll spatial distribution data of different cotton varieties, and determine the correlation relationship between the chlorophyll change data, chlorophyll spatial distribution data and the yields of different cotton varieties.

[0101] In an embodiment of the present application, the corresponding RGB images and multispectral data can be time series data, and the corresponding data are all data within at least one time period of each day during the flowering and boll-setting period. All subsequent analyses are performed on the time series data.

[0102] In another embodiment of the present application, according to the cotton yield prediction result, generate cotton water and fertilizer control instructions to effectively control the water and fertilizer during the subsequent growth period of cotton, thereby increasing the cotton yield.

[0103] In another embodiment of the present application, multiple regions are set in the cotton field. Predict the cotton yields of different regions, and screen the cotton in the regions where the predicted yield is greater than the configured yield threshold as germplasm cotton, so as to provide support for cotton germplasm screening and precision breeding.

[0104] In an embodiment of the present application, as Figure 2 shown, the method for predicting the cotton yield includes:

[0105] 1. Data collection and preprocessing:

[0106] Fly a drone equipped with an RGB sensor and a multispectral sensor over the cotton field during different key growth periods of the cotton in the field to obtain RGB image data and multispectral image data of the cotton field;

[0107] Use a chlorophyll analyzer to collect the chlorophyll content data measured on-site for different cotton varieties in the specified field areas respectively; the time difference between the collection time corresponding to the drone and the collection time corresponding to the chlorophyll analyzer is less than the specified time period.

[0108] 2. Feature extraction:

[0109] Input the RGB digital orthophoto in the RGB data into the image automatic recognition and classification model through the image classification code. Based on the RGB digital orthophoto, automatically classify the image through the image automatic recognition and classification model and divide the cotton canopy range. Based on the image classification results and the cotton canopy range division results, extract the mean values of the color indices corresponding to different varieties of cotton at different growth stages by extracting the cotton canopy vector boundaries of multi-variety cotton at different growth stages;

[0110] Calculate multiple vegetation indices sensitive to chlorophyll based on the multispectral digital orthophoto in the multispectral data, and extract the mean values of the vegetation indices corresponding to different varieties of cotton according to the cotton canopy vector boundary;

[0111] Based on the RGB image data and multispectral image data, use the hierarchical segmentation method to screen the optimal vegetation indices and color indices highly correlated with cotton chlorophyll; extract 10 vegetation indices from the multispectral image and 8 color indices from the RGB image.

[0112] 3. Feature screening:

[0113] Use the hierarchical segmentation technology for the features of the RGB image and multispectral image to further screen and optimize the features, exclude interference information, evaluate the correlation between the screened color indices and vegetation indices and chlorophyll content, ensure the high correlation and stability of the selected features, and screen out 5 vegetation indices (LCI, SAVI, NDRE, GRVI, VREI) and 3 color indices (CIVE, ExG, VDVI) highly correlated with chlorophyll content.

[0114] 4. Chlorophyll content prediction:

[0115] Based on the RGB digital orthophoto, multispectral digital orthophoto, the mean values of the optimal color indices, optimal vegetation indices, and the chlorophyll content data measured in the field for different varieties of cotton, use machine learning algorithms to construct and optimize the chlorophyll detection models for different varieties of cotton at different growth stages by adjusting the initial model parameters, and obtain the optimal chlorophyll detection model; use machine learning models such as random forest (RF), support vector machine (SVM), partial least squares regression (PLSR), and ridge regression (RR), based on multispectral data, RGB data, and their fusion data, to predict the cotton chlorophyll content; evaluate the model performance through the validation set and select the optimal model (random forest model, R 2 = 0.827, RMSE = 2.168).

[0116] 5. Yield prediction and variety classification:

[0117] During the cotton harvesting period, cotton in the corresponding areas was harvested respectively, and its yield was calculated. Based on the dynamic change data of chlorophyll content and combined with the characteristics of the key growth stages, a cotton yield prediction model was constructed using the multiple linear regression algorithm. The cotton yield was predicted through the cotton yield prediction model to obtain the predicted yield value; through multi-stage chlorophyll content data fusion, the accuracy of yield prediction was improved (R 2 = 0.723).

[0118] Based on the optimal chlorophyll detection model, K-means clustering analysis and GIS spatial analysis were carried out. Based on the dynamic change of chlorophyll, 419 cotton varieties were divided into 3 clusters, and the chlorophyll dynamics and yield performance of different clusters were analyzed; the chlorophyll change law data and chlorophyll spatial distribution law data of different cotton varieties were obtained, and the correlation between the chlorophyll change law data, chlorophyll spatial distribution law data and the yields of different cotton varieties was determined.

[0119] The following further illustrates the present application with specific experiments.

[0120] 1. Plant 419 cotton germplasms in the experimental fields of a certain city in a certain province. Each variety is planted in a 4.5-meter × 5-meter experimental plot. Use DJI Mavic 3E (RGB sensor) and DJI Mavic 3M (multispectral sensor) for image acquisition. Through drone remote sensing technology combined with a real-time kinematic (RTK) module, RGB and multispectral data of the infected and non-infected areas of cotton at different key growth stages (the key growth stages include the budding stage, flowering stage, boll-setting stage, and boll-opening stage) are obtained in the field. The user needs to determine the drone field scanning range, and the present application sets the flight route according to the size of the user's plot to ensure a 70% side overlap rate and an 80% forward overlap rate. Through drone remote sensing technology combined with a real-time kinematic (RTK) module, RGB and multispectral data of the infected and non-infected areas of cotton at different key growth stages (the key growth stages include the flower bud stage, flowering stage, full-bloom stage, boll stage, and full-boll stage) are obtained in the field. The user needs to determine the drone field scanning range, and the present application sets the flight route according to the size of the user's plot to ensure a 70% side overlap rate and an 80% forward overlap rate.

[0121] 2. After the UAV completes the image acquisition of the experimental plots, the chlorophyll content of the leaves is measured manually. For each plot, 12 cotton plants are selected to measure the chlorophyll content of their canopy leaves, and the average value is taken as the chlorophyll content of the leaves of this germplasm on that day. It should be noted that the time difference between the acquisition time corresponding to the UAV and the acquisition time corresponding to the chlorophyll analyzer is less than the specified time period. At the maturity stage of cotton, all cotton plants in each experimental plot are harvested manually. The weight of the cotton is weighed using an electronic balance, and the yield is calculated in grams per square meter (g / m 2 ), and then converted to kilograms per hectare (kg / hm 2 ).

[0122] 3. The RGB digital orthophoto in the RGB data is input into the image automatic recognition and classification model through the image classification code. Based on the RGB digital orthophoto, the image is automatically classified by the image automatic recognition and classification model, and the cotton canopy range is divided. Based on the image classification result and the cotton canopy range division result, the mean values of the color indices corresponding to different varieties of cotton are extracted at different growth stages of multi-variety cotton by extracting the cotton canopy vector boundary;

[0123] Based on the multi-spectral digital orthophoto in the multi-spectral data, multiple vegetation indices sensitive to chlorophyll are calculated, and the mean values of the vegetation indices corresponding to different varieties of cotton are extracted according to the cotton canopy vector boundary;

[0124] Based on the RGB image data and the multi-spectral image data, the hierarchical segmentation method is used to screen the optimal vegetation indices and color indices highly correlated with cotton chlorophyll; 10 vegetation indices are extracted from the multi-spectral image, and 8 color indices are extracted from the RGB image.

[0125] 4. As Figure 3 shown, using the hierarchical segmentation technology for the features of the RGB image and the multi-spectral image, the features are further screened and optimized, the interference information is excluded, and the correlation between the screened color indices and vegetation indices and the chlorophyll content is evaluated to ensure the high correlation and stability of the selected features. Five vegetation indices (LCI, SAVI, NDRE, GRVI, VREI) and three color indices (CIVE, ExG, VDVI) highly correlated with the chlorophyll content are screened out; Figure 3 The vegetation index (VI) and canopy index (CI) are screened in; the percentage increase in mean squared error (%IncMSE) value is used to evaluate the importance of each index. Among them, the higher the %IncMSE value, the stronger the prediction ability for the leaf chlorophyll content (LCC); Figure 3 (A) represents the seedling stage, Figure 3 (B) represents the flowering stage, Figure 3 (C) represents the boll stage, Figure 3 (D) represents the boll opening stage.

[0126] 5. As Figure 4 shown, using machine learning models such as random forest (RF), support vector machine (SVM), partial least squares regression (PLSR), and ridge regression (RR), based on multispectral data, RGB data, and their fusion data, by adjusting the model parameters, construct and optimize the detection models of chlorophyll in different cotton varieties at different growth stages, and detect the accuracy of the models by calculating the coefficient of determination (R 2 ), root mean square error (RMSE), and mean absolute error (MAE). Obtain the optimal chlorophyll detection model. The higher the R 2 value and the lower the RMSE and MAE values, the higher the accuracy and precision of the model in estimating the SPAD value, realizing the improvement of the accuracy of the chlorophyll prediction model throughout the growth period through machine learning algorithms. Predict the chlorophyll content of cotton. Evaluate the model performance through the validation set and select the optimal model (random forest model, R 2 = 0.827, RMSE = 2.168);

[0127] Figure 4 The 1:1 scatter plot in

[0128] shows the relationship between the leaf chlorophyll content predicted by various machine learning models and the measured values. The models in the figure include partial least squares regression (PLSR), random forest (RF), ridge regression (RR), and support vector machine (SVM), and these models are applied to red, green, blue (RGB) data, multispectral (MS) data, and the fused red, green, blue (RGB) and multispectral (MS) data.

[0129] 6. As Figure 5 shown, harvest the cotton in the corresponding areas during the cotton harvest period and calculate its yield. Based on the dynamic change data of chlorophyll content, combined with the characteristics of the key growth stages, use the multiple linear regression algorithm to construct a cotton yield prediction model, and improve the accuracy of yield prediction through the fusion of multi-stage chlorophyll content data (R 2 = 0.723);

[0130] Figure 5A in it represents the cotton yield predicted by the leaf chlorophyll content (LCC) at the seedling stage; Figure 5 B in it represents the cotton yield predicted by the leaf chlorophyll content (LCC) at the flowering stage; Figure 5 C in it represents the cotton yield predicted by the leaf chlorophyll content (LCC) at the boll stage; Figure 5 D in it represents the cotton yield predicted by the leaf chlorophyll content (LCC) at the boll-opening stage; Figure 5 E in it represents the cotton yield predicted by the leaf chlorophyll content (LCC) in the comprehensive situation of multiple stages; Figure 5 The black dashed line in it represents the ideal 1:1 correspondence between the measured yield and the predicted yield; the shaded area represents the 95% confidence interval of the regression line.

[0131] 7. As Figure 6 shown, based on the optimal chlorophyll detection model, K-means clustering analysis and GIS spatial analysis are carried out. Based on the dynamic changes of chlorophyll, 419 cotton varieties are divided into 3 clusters, and the chlorophyll dynamics and yield performance of different clusters are analyzed; the data of the chlorophyll change law and the chlorophyll spatial distribution law of different cotton varieties are obtained, and the correlation between the chlorophyll change law data, the chlorophyll spatial distribution law data and the yields of different cotton varieties is determined. Analyzing the chlorophyll dynamics and yield performance of different clusters provides a reference for breeding and cultivation management.

[0132] Figure 6 A in it represents the sum of squares corresponding to different k values, and the k value corresponding to the place where the slope change is not obvious is selected as the optimal clustering number (k = 3); Figure 6 B in it represents the Gap statistic corresponding to different k values, and the k value corresponding to the place where the Gap value is the largest is selected as the optimal clustering number (k = 3); Figure 6 C in it represents the K-means clustering results of 419 cotton germplasms, and different colors represent different clustering categories; Figure 6 D in it represents the dynamic change of the cotton leaf chlorophyll content over time in the three clustering categories obtained based on K-means clustering; Figure 6 E in it represents the cotton yield in the three clustering categories obtained based on K-means clustering.

[0133] This application provides a method for dynamically monitoring cotton chlorophyll and predicting yield based on the fusion of unmanned aerial vehicle multispectral and RGB data. By using the hierarchical segmentation technology to screen the key features in the multi-source data and combining with the machine learning model, it helps users to realize that after measuring the chlorophyll data of the cotton canopy in the field, only need to adopt the chlorophyll detection model constructed based on the unmanned aerial vehicle high-throughput data and machine learning algorithm provided by this application to further realize the efficient prediction of the yield, providing support for cotton germplasm screening and precision breeding.

[0134] Correspondingly, an embodiment of the present application further provides a prediction device for cotton yield, as Figure 7 shown. The prediction device for cotton yield includes:

[0135] An acquisition unit 710, configured to acquire the RGB image and multispectral image of the cotton to be predicted during the flowering and boll-forming period;

[0136] A determination unit 720, configured to determine the growth state of the cotton to be predicted and the agricultural condition parameters of the cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image;

[0137] A first prediction unit 730, configured to input the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain the predicted chlorophyll of the cotton to be predicted during the flowering and boll-forming period; wherein, the chlorophyll prediction model is trained using the chlorophyll detection values of different varieties of cotton during the historical flowering and boll-forming period and the historical agricultural condition parameters of the cotton fields where the corresponding cotton is planted during the historical flowering and boll-forming period;

[0138] A second prediction unit 740, configured to input the growth state and the predicted chlorophyll into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted; wherein, the cotton yield prediction model is trained using the chlorophyll detection values, growth states of different varieties of cotton during the historical flowering and boll-forming period, and the yields of the corresponding cotton during the historical harvest period.

[0139] The functions of the functional units of the prediction device for cotton yield provided in the above embodiments of the present application can be implemented by the above method steps. Therefore, the specific working processes and beneficial effects of each unit in the prediction device for cotton yield provided in the embodiments of the present application will not be repeated here.

[0140] An embodiment of the present application further provides an electronic device, as Figure 8 shown, including a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840.

[0141] The memory 830 is used to store a computer program;

[0142] When the processor 810 is configured to execute the program stored in the memory 830, the following steps are implemented:

[0143] Acquire the RGB image and multispectral image of the cotton to be predicted during the flowering and boll-forming period;

[0144] Based on the RGB image and the multispectral image, determine the growth state of the cotton to be predicted and the agricultural condition parameters of the cotton field where the cotton to be predicted is planted;

[0145] Input the agricultural condition parameters into the pre-trained chlorophyll prediction model to obtain the predicted chlorophyll of the cotton to be predicted during the flowering and boll-setting period. Among them, the chlorophyll prediction model is trained using the chlorophyll detection values of different cotton varieties during the historical flowering and boll-setting periods and the historical agricultural condition parameters of the cotton fields where the corresponding cotton is planted during the historical flowering and boll-setting periods.

[0146] Input the growth status and the predicted chlorophyll into the pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted. Among them, the cotton yield prediction model is trained using the chlorophyll detection values, growth status of different cotton varieties during the historical flowering and boll-setting periods, and the yields of the corresponding cotton during the historical harvest periods.

[0147] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0148] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0149] The memory can include a Random Access Memory (RAM), and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located far from the aforementioned processor.

[0150] The above-mentioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0151] Since the implementation manners and beneficial effects of each component of the electronic device in the above embodiments for solving problems can be referred to Figure 1It is implemented by the steps in the embodiments shown. Therefore, the specific working process and beneficial effects of the electronic device provided in the embodiments of this application will not be elaborated here.

[0152] In another embodiment provided by this application, a computer-readable storage medium is further provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the cotton yield prediction method described in any one of the above embodiments.

[0153] In another embodiment provided by this application, a computer program product containing instructions is further provided. When it runs on a computer, it causes the computer to execute the cotton yield prediction method described in any one of the above embodiments.

[0154] Those skilled in the art should understand that the embodiments in the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, the embodiments in this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments in this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] The embodiments in the embodiments of this application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products in the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0156] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device realizes the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks. Figure 1 One process or a plurality of processes and / or blocks Figure 1 Steps for implementing the functions specified in one block or a plurality of blocks.

[0158] Although the preferred embodiments in the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0159] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the embodiments of the present application without departing from the spirit and scope of the embodiments in the embodiments of the present application. Thus, if these modifications and variations of the embodiments in the embodiments of the present application fall within the scope of the claims of the embodiments of the present application and their equivalent technologies, the embodiments in the embodiments of the present application are also intended to include these changes and modifications.

Claims

1. A method for predicting cotton yield, characterized in that, The method comprises: Obtain RGB images and multispectral images of the cotton to be predicted at the flowering and boll stage; Determining the growth status of the cotton to be predicted and agricultural parameters of a cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image; Inputting the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain a predicted chlorophyll value of the cotton to be predicted at the flowering and boll stage; wherein the chlorophyll prediction model is trained using chlorophyll detection values of different varieties of cotton at historical flowering and boll stages and historical agricultural condition parameters of cotton fields planted with the corresponding cotton at historical flowering and boll stages; The growth status and the predicted chlorophyll are input into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted; wherein the cotton yield prediction model is trained using the chlorophyll detection values and growth status of different varieties of cotton during the historical flowering and boll period and the yield of the corresponding cotton during the historical harvest period.

2. The method according to claim 1, wherein After acquiring the RGB image and the multispectral image of the cotton at the flowering and boll stage, the method further includes: The RGB image and the multispectral image are preprocessed respectively to obtain an RGB digital orthoimage and a multispectral digital orthoimage.

3. The method according to claim 2, wherein The agricultural parameters include: vegetation index and color index; The vegetation index includes: chlorophyll index, soil adjusted vegetation index, normalized difference red edge index, green-red vegetation index and vegetation red edge index; The color index includes: vegetation extraction color index, excess green index and visible difference vegetation index; The multispectral image includes spectral data of cotton in different wavelength bands.

4. The method according to claim 3, wherein Determining agricultural parameters of a cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image includes: For any vegetation index to be calculated, match the target band corresponding to the vegetation index to be calculated from the configured comparison table of different vegetation indices and different bands; Extract the spectral data of the target band from the multispectral digital orthoimage to obtain the target spectral data; The vegetation index to be calculated is calculated based on the target spectral data.

5. The method according to claim 4, wherein Determining agricultural parameters of a cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image further includes: Inputting the RGB digital orthoimage and the multispectral digital orthoimage into a pre-trained cotton canopy segmentation and vectorization model to obtain a cotton canopy vector boundary; The color index is calculated based on the pixels within the cotton canopy vector boundary.

6. The method according to claim 2, wherein The growth status includes: plant type and leaf area index of the cotton plant; Determining the growth status of the cotton to be predicted based on the RGB image and the multispectral image includes: Inputting the RGB digital orthoimage and the multispectral digital orthoimage into a pre-trained cotton plant classification model to obtain the cotton plant type to be predicted; The RGB digital orthoimage and the multispectral digital orthoimage are input into a pre-trained leaf area multimodal estimation model to obtain the leaf area index of the cotton to be predicted.

7. The method according to claim 1, wherein The cotton yield prediction model is constructed by using the multiple linear regression algorithm based on the correlation relationship between the growth states, chlorophyll change data, chlorophyll spatial distribution data of different cotton varieties during the historical flowering and boll-forming periods, and the yields of the corresponding different cotton varieties during the historical harvest periods.

8. A prediction device for cotton yield, characterized in that, The device includes: An acquisition unit, configured to acquire the RGB image and the multispectral image of the cotton to be predicted during the flowering and boll-forming periods; A determination unit, configured to determine the growth state of the cotton to be predicted and the agricultural condition parameters of the cotton field where the cotton to be predicted is planted based on the RGB image and the multispectral image; A first prediction unit, configured to input the agricultural condition parameters into a pre-trained chlorophyll prediction model to obtain the predicted chlorophyll of the cotton to be predicted during the flowering and boll-forming periods; wherein, the chlorophyll prediction model is trained by using the chlorophyll detection values of different cotton varieties during the historical flowering and boll-forming periods and the historical agricultural condition parameters of the cotton fields where the corresponding cotton is planted during the historical flowering and boll-forming periods; A second prediction unit, configured to input the growth state and the predicted chlorophyll into a pre-trained cotton yield prediction model to obtain the predicted yield of the cotton to be predicted; wherein, the cotton yield prediction model is trained by using the chlorophyll detection values, growth states of different cotton varieties during the historical flowering and boll-forming periods, and the yields of the corresponding cotton during the historical harvest periods.

9. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used for storing a computer program; The processor is configured to implement the method according to any one of claims 1-7 when executing the program stored on the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1-7 is implemented.

Images