Crop growth prediction method, device, equipment and medium

Abstract

The present application provides a crop growth prediction method, device, equipment and medium, wherein the method includes: determining a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm; fusing the original three-dimensional point cloud data set, the lossless two-dimensional image data set and meteorological factor data, and inverting the growth parameters of different crops in combination with their spatiotemporal specificity to obtain an optimal crop growth parameter multidimensional deep learning model, so as to evaluate the growth parameter information of different crops and obtain the spatiotemporal distribution of the growth parameters of different crops. The present application determines a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm to obtain more accurate features, and then constructs an optimal crop growth parameter multidimensional deep learning model through the original three-dimensional point cloud data set, the lossless two-dimensional image data set and meteorological factor data to obtain the spatiotemporal distribution of the growth parameters of different crops, thereby improving the accuracy of the crop growth parameter prediction results.

Description

Technical Field

[0001] The present application relates to the field of agricultural technology, and in particular to a crop growth prediction method, device, equipment and medium. Background Art

[0002] Ensuring crop yields is the foundation of people's livelihood and the most important research topic for mankind. Crop growth refers to the conditions and trends of crop growth. That is, by predicting crop growth, we can timely understand the growth status, pests and diseases, or nutritional status of crops, thereby guiding people to take corresponding management measures to ensure the normal growth of crops.

[0003] At present, the analysis of field crop data is generally based on a single image after stitching, and the original image set with high overlap collected by drones is not fully utilized. Due to the influence of natural wind and environmental differences at different shooting times, stitched images will produce phenomena such as leaf tip loss and leaf ghosting, which will affect the accuracy of growth parameter prediction. Moreover, after acquiring the images, the prediction results of crop growth parameters based on the images are not accurate enough. Summary of the invention

[0004] The present application provides a crop growth prediction method, device, equipment and medium, aiming to achieve the purpose of improving the prediction accuracy of crop growth parameters.

[0005] In a first aspect, the present application provides a crop growth prediction method, comprising:

[0006] Determine the lossless 2D image dataset of the original 3D point cloud dataset based on the image backtracking algorithm;

[0007] The original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data are integrated to construct a multi-dimensional deep learning model of each original crop growth parameter;

[0008] The optimal crop growth parameter multidimensional deep learning model is obtained by inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops;

[0009] The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0010] In one embodiment, the inversion of each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops to obtain an optimal crop growth parameter multidimensional deep learning model includes:

[0011] Inverting the spatiotemporal specificity of growth parameters of different crops through the multidimensional deep learning models of the original crop growth parameters to obtain growth parameters of different crops after different sowing times;

[0012] According to the growth parameters of the different crops after different sowing times, determining the respective matching degree values ​​of the actual biomass value of the multidimensional deep learning model of the original crop growth parameters and the model calculated value;

[0013] Based on each of the matching degree values, the optimal crop growth parameter multidimensional deep learning model among the original crop growth parameter multidimensional deep learning models is determined.

[0014] The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops, including:

[0015] By combining the optimal crop growth parameter multidimensional deep learning model with the lossless two-dimensional image data set of different crops, feature extraction is performed on the single-flight data of the entire growth stage of different crops to obtain multi-phase feature information of different crops in different growth stages;

[0016] The temporal and spatial distribution of growth parameters of different crops are obtained by evaluating the multi-phase characteristic information of different crops in different growth stages.

[0017] The step of evaluating the multi-phase characteristic information of different crops at different growth stages to obtain the spatiotemporal distribution of growth parameters of different crops includes:

[0018] Determine the multi-phase characteristic information of the different crops in different growth stages as input features at different time points of the output layer for calculation to obtain final growth parameters of the different crops;

[0019] The position information of the preset area where different crops are located is determined, and according to the final growth parameters of the different crops and the position information thereof, the spatiotemporal distribution of the growth parameters of the different crops in the entire preset area is determined.

[0020] The method of determining a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm includes:

[0021] Acquire imaging parameters of the aerial camera, and acquire an image retrieval matrix based on the imaging parameters;

[0022] Based on the imaging parameters and the image tracing matrix, a lossless two-dimensional image dataset of the three-dimensional point cloud dataset is acquired.

[0023] The step of acquiring a lossless two-dimensional image data set of the three-dimensional point cloud data set based on the imaging parameters and the image tracing matrix includes:

[0024] Based on the imaging parameters and the image tracing matrix, acquiring an initial lossless two-dimensional image data set;

[0025] Inputting the initial lossless two-dimensional image data set into a pre-built image optimization model to obtain a lossless two-dimensional image data set output by the image optimization model;

[0026] The image optimization model is determined based on the color information of the lossless two-dimensional image data set and the optimal hyperparameters corresponding to the color information.

[0027] In a second aspect, the present application further provides a crop growth prediction device, comprising:

[0028] An image acquisition module, used for determining a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm;

[0029] A model building module, used to fuse the original three-dimensional point cloud data set, the lossless two-dimensional image data set and meteorological factor data to build a multi-dimensional deep learning model of each original crop growth parameter;

[0030] A model determination module is used to perform inversion by combining the original crop growth parameter multidimensional deep learning models with the spatiotemporal specificity of growth parameters of different crops to obtain an optimal crop growth parameter multidimensional deep learning model;

[0031] The result prediction module is used to evaluate the growth parameter information of different crops through the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0032] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the crop growth prediction method of the first aspect is implemented.

[0033] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the crop growth prediction method of the first aspect.

[0034] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the crop growth prediction method of the first aspect.

[0035] The crop growth prediction method, device, equipment and medium provided in the present application, in the process of crop growth prediction, determine the lossless two-dimensional image data set of the original three-dimensional point cloud data set according to the image backtracking algorithm to obtain more accurate features, and then construct an optimal crop growth parameter multidimensional deep learning model through the original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data to obtain the spatiotemporal distribution of growth parameters of different crops, thereby improving the accuracy of the crop growth parameter prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 It is a flow chart of the crop growth prediction method provided by the present application;

[0038] Figure 2 It is a structural schematic diagram of the image acquisition device provided by the present application;

[0039] Figure 3 It is a schematic diagram of the deepGPD model process provided by this application;

[0040] Figure 4 It is a structural schematic diagram of the crop growth prediction device provided by the present application;

[0041] Figure 5 It is a schematic diagram of the structure of the electronic device provided by this application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] Combine the following Figures 1 to 5 The present invention describes the crop growth prediction method, device, equipment and medium. Figure 1 It is a flow chart of the crop growth prediction method provided by the present application;

[0044] Figure 2 It is a structural schematic diagram of the image acquisition device provided by the present application; Figure 3It is a schematic diagram of the deepGPD model process provided by this application; Figure 4 It is a structural schematic diagram of the crop growth prediction device provided by the present application; Figure 5 It is a schematic diagram of the structure of the electronic device provided by this application.

[0045] The embodiments of the present application provide an embodiment of a crop growth prediction method. It should be noted that although a logical order is shown in the flow chart, under certain data, the steps shown or described may be completed in an order different from that shown here.

[0046] The present application provides a crop growth prediction method, wherein the execution subject may be a terminal, such as a computer, a vehicle-mounted terminal, etc.

[0047] Reference Figure 1 , Figure 1 : is a flow chart of the crop growth prediction method provided by the present application. The crop growth prediction method provided by the embodiment of the present application includes:

[0048] Step 10: determine a lossless two-dimensional image dataset of the original three-dimensional point cloud dataset based on an image backtracking algorithm.

[0049] It can be understood as obtaining images of field crops through aerial photography with a camera, then extracting three-dimensional point clouds from the aerial images, and obtaining a three-dimensional point cloud dataset of field crops.

[0050] For example, Figure 2 is a schematic diagram of the structure of the image acquisition device provided by the present application, such as Figure 2 As shown, a parallel acquisition device for digital and multispectral cameras adapted to UAVs is set up. The acquisition device is made of 2 mm matte carbon fiber board, and is equipped with a multispectral shock-absorbing mounting area and a global positioning system (GPS) mounting area, while not blocking the obstacle sensing detector of the UAV. The device realizes the simultaneous acquisition of two sensors, digital camera and multispectral camera, and can obtain RGB digital images and multispectral images in one flight, effectively improving the data acquisition throughput of field experiments.

[0051] The aerial flight route can be planned through open source software, and then the 3D spatial position of the ground control points evenly distributed in the experimental field can be measured by real-time differential positioning (RTK). The aerial image sequence is obtained, processed and 3D reconstructed using image processing software to generate a 3D point cloud data set and spliced ​​image of the entire experimental field. That is, after simultaneously acquiring digital RGB images and multispectral images of field crops through the parallel acquisition device of the drone digital camera and the multispectral camera, the aerial image sequence is sequentially reconstructed into 3D point cloud, digital elevation model and digital orthophoto using conventional image processing software to obtain a 3D point cloud data set of the entire experimental field and a spliced ​​image of the experimental field.

[0052] The four vertices of the entire experimental field in the point cloud are manually extracted. Based on the field experiment layout, the point cloud is further divided into individual plots using field plot segmentation technology, and the geographical location of each plot is labeled and numbered. Finally, the target plot point cloud is screened and saved to obtain the target 3D point cloud dataset (3D point cloud dataset of field crops).

[0053] After obtaining the three-dimensional point cloud dataset of field crops, the three-dimensional point cloud dataset is processed by an image backtracking algorithm, and a high-quality lossless two-dimensional image is segmented by positioning the three-dimensional point cloud dataset. That is, by obtaining the positioning of the three-dimensional point cloud dataset, the image backtracking algorithm is used to determine the aerial image (lossless two-dimensional image dataset) corresponding to the positioning of the three-dimensional point cloud dataset in the aerial route, and multiple lossless two-dimensional image datasets at different angles can be obtained through the three-dimensional point cloud dataset, as described in steps S101 to S102.

[0054] Further, the description of step S101 to step S102 is as follows:

[0055] Step S101, obtaining imaging parameters of an aerial camera, and obtaining an image tracing matrix based on the imaging parameters;

[0056] Step S102: acquiring a lossless two-dimensional image dataset of the three-dimensional point cloud dataset based on the imaging parameters and the image tracing matrix.

[0057] It can be understood that this embodiment first reads the internal and external parameter files of the camera to interpret the parameters of the camera imaging. The internal parameters of the camera are parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera. The external parameters of the camera are parameters in the world coordinate system, such as the position and rotation direction of the camera. They include: the accuracy of the pixel in the two-dimensional direction (dx, dy), the focal length (f), the position of the optical center (cx, cy), the 3×3 rotation matrix (R) and the 3×1 translation matrix (T).

[0058] The image retrieval matrix involves four coordinate systems: world coordinate system, camera coordinate system, pixel coordinate system and image coordinate system.

[0059] The world coordinate system is the absolute coordinate system of the objective three-dimensional world. This reference coordinate system is needed to describe the position of the digital camera, and it is used to describe the position of any other objects placed in this three-dimensional environment, and its coordinate values ​​are expressed as (Xw, Yw, Zw).

[0060] The camera coordinate system is also called the optical center coordinate system. The optical center of the camera is used as the coordinate origin. The X-axis and Y-axis are parallel to the X-axis and Y-axis of the image coordinate system. The optical axis of the camera is the Z-axis, and its coordinate value is expressed as (Xc, Yc, Zc).

[0061] The image coordinate system takes the center of the image plane as the coordinate origin, and the X-axis and Y-axis are parallel to the two vertical sides of the image plane respectively, and their coordinate values ​​are represented by (x, y).

[0062] The pixel coordinate system is an image coordinate system with pixels as the unit, with the upper left corner vertex of the image plane as the origin, and the X-axis and Y-axis are parallel to the X-axis and Y-axis of the image coordinate system respectively, and its coordinate value is represented by (u, v). Among them, the unit of the world coordinate system and the camera coordinate system is m, the unit of the pixel coordinate system is pixel, and the unit of the image coordinate system is mm.

[0063] After setting the world coordinate system, camera coordinate system, pixel coordinate system and image coordinate system, the four coordinate systems are combined with the camera imaging parameters to obtain the image tracing matrix. Then, the lossless two-dimensional image data set corresponding to the three-dimensional point cloud data set can be obtained through the imaging parameters and the image tracing matrix, as described in steps S1021 to S1022.

[0064] Specifically, after the actual position of the three-dimensional point cloud dataset is determined through actual measurement or other means, it is marked as the first position coordinate. From the world coordinate system to the camera coordinate system, only rotation and translation are required to obtain the second position coordinate in the camera coordinate system. Then, through the principle of similar triangles, the second position coordinate is converted into the third position coordinate in the image plane coordinate system. The three position coordinates are translated to obtain the fourth position coordinate of the three-dimensional point cloud dataset in the pixel coordinate system.

[0065] After the fourth position coordinates in the pixel coordinate system are obtained, the lossless image data set corresponding to the three-dimensional point cloud data set can be accurately obtained according to the coordinates.

[0066] For example, let P be an object in the world coordinate system, Pw be its coordinates in the world coordinate system, and Pc be its coordinates in the camera coordinate system. Use a rotation matrix R and a translation vector t to achieve the conversion between Pc and Pw. R is a 3×3 rotation matrix, and t is a 3×1 translation vector. The form of the homogeneous coordinates calculated above is as follows:

[0067]

[0068] Where Xc, Yc, and Zc are the position coordinates in the camera coordinate system, and Xw, Yw, and Zw are the position coordinates in the world coordinate system.

[0069] The principle of similar triangles is used to realize the conversion from the camera coordinate system to the image coordinate system. The coordinates of the point P in the three-dimensional space in the camera coordinate system are Pc. The coordinates of the imaging point p in the image coordinate system are p. The conversion formula of the image coordinate system is as follows:

[0070]

[0071] Where Xc, Yc, and Zc are the position coordinates in the camera coordinate system, f is the distance from the camera focus to the imaging plane, and x and y are the position coordinates in the image coordinate system.

[0072] Since the image coordinate system takes the center of the image as its origin, and the origin of the pixel coordinate system is usually the upper left corner of the image, it is necessary to translate the coordinates of the midpoint of the image plane coordinate system to the upper left corner to achieve conversion with the pixel coordinate system. The formula is as follows:

[0073]

[0074] Where u and v are the position coordinates in the pixel coordinate system, dx and dy are the accuracy of the pixel in the x and y directions, and cx and cy are the positions of the optical center (image center).

[0075] That is, after determining the first position coordinates, the fourth position coordinates can be obtained by the camera backtracking matrix formula, and the camera backtracking matrix formula is as follows:

[0076]

[0077] Where u and v are the position coordinates in the pixel coordinate system, Xw, Yw, and Zw are the position coordinates in the world coordinate system, f is the distance from the focus of the camera to the imaging plane, dx and dy are the accuracy of the pixel in the x and y directions, and cx and cy are the positions of the optical center (image center).

[0078] The present application obtains the actual coordinate position of the three-dimensional point cloud dataset of field crops, obtains the position coordinates in a high-precision pixel coordinate system through the image tracing matrix, and accurately obtains the lossless image dataset corresponding to the three-dimensional point cloud dataset, which is beneficial to improving the accuracy of crop growth parameter prediction.

[0079] Further, the description of step S1021 to step S1022 is as follows:

[0080] Step S1021, acquiring an initial lossless two-dimensional image data set based on the imaging parameters and the image tracing matrix;

[0081] Step S1022, inputting the initial lossless two-dimensional image data set into a pre-built image optimization model to obtain a lossless two-dimensional image data set output by the image optimization model;

[0082] The image optimization model is determined based on the color information of the lossless two-dimensional image data set and the optimal hyperparameters corresponding to the color information.

[0083] It can be understood that an initial lossless two-dimensional image data set of the area to be predicted can be obtained according to the camera imaging parameters and the image traceback matrix, and the initial lossless two-dimensional image data set contains redundant background information.

[0084] The initial lossless two-dimensional image data set containing complete information is then input into a pre-built image optimization model to obtain a lossless two-dimensional image data set containing only crop information output by the image optimization model.

[0085] The image optimization model can be a support vector machine model, which is used to perform support vector machine classification modeling based on the color information of the lossless two-dimensional image data set, and then the classification coefficient of the support vector machine model is tested to obtain the optimal hyperparameters of the support vector machine model, and the image optimization model is determined by the optimal hyperparameters. The parameters that minimize the sum of squared errors of the detection values ​​of the image optimization model are selected, and the image background is eliminated to obtain the target lossless two-dimensional image data set containing only crop information.

[0086] This application optimizes the acquired lossless two-dimensional image data set and removes the background that does not contain crop information, which can further improve the accuracy of feature extraction.

[0087] Step 20, fusing the original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data to construct a multi-dimensional deep learning model of each original crop growth parameter;

[0088] Step S30, inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops to obtain an optimal crop growth parameter multidimensional deep learning model.

[0089] The original 3D point cloud dataset, lossless 2D image dataset and meteorological factor data are fused to construct a multidimensional deep learning model of each original crop growth parameter. Specifically, an embedding layer is used to encode the features of the 3D point cloud dataset and meteorological factor data, and any point cloud is represented as a set of discrete codes, each of which corresponds to a clear local structure (dVAE).

[0090] After converting the text data of meteorological factors (CNN-non-static) into a feature vector of fixed dimension, a convolution kernel is used to obtain the feature representation of all the above information.

[0091] In the backbone network, depthwise separable convolution (DS) and group convolution (GC) are used to extract feature maps. Compared with conventional convolution operations, the number of parameters and computational cost are relatively low. Group channel-wise convolution (GCW) is also added to allow convolution to slide in the channel dimension, which can better preserve the information exchange between channels.

[0092] Since the deep feature graph has very rich semantic information, the Path Aggregation Network (PAN) is used to aggregate the features between the deep and shallow layers, increase the shallow semantic information, ensure the integrity and diversity of the features, and improve the prediction accuracy. In order to further solve the gradient disappearance and gradient explosion problems in the long sequence training process, the LSTM model is used in the output layer to extract the phenotypic data features at different times throughout the growth period, control the transmission state through the gating state, focus on the information related to yield, emphasize important features and suppress unnecessary features, which helps to screen out the target features and improve the prediction accuracy of the model.

[0093] Furthermore, the optimal crop growth parameter multidimensional deep learning model is obtained by inverting the original crop growth parameter multidimensional deep learning model in combination with the spatiotemporal specificity of the growth parameters of different crops, as described in steps S301 to S303, wherein the optimal crop growth parameter multidimensional deep learning model is named deepGPD model, i.e. deeplearning model for growth parameter combining 2D and 3D. The optimal crop growth parameter multidimensional deep learning model can integrate features such as image features of different scales and spatial depth and angle information of three-dimensional point clouds, and therefore has higher resolution and accuracy.

[0094] Further, refer to Figure 3 , Figure 3 It is a flowchart of the deepGPD model provided in this application. Specifically, the deepGPD model is a crop growth parameter prediction model that integrates the improved YOLOV5 and LSTM. The yield of crops can be expressed as the result of the combined action of major physiological processes such as the interception of solar radiation by the crop canopy and the conversion of solar radiation into biomass. The canopy structure characteristics are the main genetic characteristics that affect the efficiency of solar radiation interception. Climate factors are one of the important factors affecting the efficiency of light energy utilization, especially light and CO2 concentration, because they affect carbon fixation and respiration. In order to improve the accuracy of yield estimation, improvements are made in the input layer, integrating two-dimensional drone images, three-dimensional point clouds and meteorological factors. Feature extraction is performed on the single flight data of the entire growth stage, and multi-phase feature information collected in different growth periods is obtained, which is used as the input features of the output layer at different time points to estimate the final growth parameters.

[0095] The Focus structure is used to extract numerical feature vectors from the two-dimensional image dataset. First, the input image (2n*2n*3) is cropped through the Slice operation, and then a n*n*12 feature map is output through Concat splicing. Finally, the Conv layer is used to obtain the feature vector of the image. For the three-dimensional point cloud and meteorological data, the embedding layer (Embeddinglayer) is used to encode the features, and any point cloud is represented as a set of discrete codes, each of which corresponds to a clear local structure (dVAE); the meteorological text data (CNN-non-static, CNS) is converted into a fixed-dimensional feature vector. Then the concat layer is used to concatenate all the above input features.

[0096] The depthwise separable groupchannel-wise convolution module (DGCW) is used in the backbone network to extract some common feature representations. This module is composed of depthwise separable convolution (DS), group convolution (GC) and group channel-wise convolution (GCW). Depthwise separable convolution (DS) and group convolution (GC) are used to extract feature maps. Compared with conventional convolution, the number of parameters and computational cost are relatively low. Group channel convolution (GCW) is added to make the convolution slide on the channel dimension, which can better retain the information exchange between channels.

[0097] The Neck network adopts the FPN+PAN structure. FPN is used to pass down the strong semantic features of the high-level layers to enhance the entire pyramid. The Path Aggregation Network (PAN) is used to pass up the strong positioning features of the low-level layers, which is conducive to aggregating the features between the deep and shallow layers, further improving the diversity and robustness of the features, and thus improving the accuracy of the prediction. Within the FPN+PAN structure, the Deep Separable Grouped Channel Convolution Module (DGCW) is used to reduce the computational cost while strengthening the information exchange between channels.

[0098] The phenotypic data characteristics (X t , X t+1 ,···,X t+n ) is input to LSTM, and the output layer of each layer corresponds to h t ,h t+1 ,···,h t+n The input layer at time T includes the phenotypic features X at the same time t and the output h at the previous time point t-1 , discard features irrelevant to the output through the forget gate, obtain the forget gate result through a σ function, obtain the new information to be updated through the σ and tanh functions, and update the LSTM state at that time point by combining the forget gate result and the new information to be updated. Use the tanh function to process the LSTM state at time T, and then merge it with the forget gate result to obtain the output result h at time T t In summary, using LSTM as the output layer can combine phenotypic features at different time points, use the gating state to control the transmission state, focus on information related to yield, emphasize important features and suppress unnecessary features, which helps to screen out target features and improve the prediction accuracy of the model.

[0099] Furthermore, the optimal crop growth parameter multidimensional deep learning model is a crop growth parameter prediction model based on the improved YOLOV5 and LSTM. The optimal crop growth parameter multidimensional deep learning model is based on the input layer, backbone network and output layer of the improved YOLOV5 network. The input layer integrates the lossless two-dimensional image data set, the three-dimensional point cloud data set and the meteorological factor data. The embedding layer is used to encode the features of the three-dimensional point cloud data set and the meteorological factor data. The backbone network uses depthwise separable convolution (DS), group convolution (GC) and group channel-wise convolution (GCW), which have relatively low parameter number and computational cost, and can better retain the information exchange between channels. The output layer integrates the LSTM model to extract the phenotypic data features at different times throughout the growth period, which helps to screen out the target features and improve the prediction accuracy of the model.

[0100] Further, the description of step S301 to step S303 is as follows:

[0101] Step S301, inverting the spatiotemporal specificity of growth parameters of different crops through the multidimensional deep learning model of each original crop growth parameter to obtain growth parameters of different crops after different sowing times;

[0102] Step S302, determining the respective matching degree values ​​of the actual biomass value of each of the original crop growth parameter multidimensional deep learning models and the model calculated value according to the growth parameters of the different crops after different sowing times;

[0103] Step S303: determining the optimal crop growth parameter multidimensional deep learning model among the original crop growth parameter multidimensional deep learning models based on the respective matching degree values.

[0104] Specifically, the temporal and spatial specificity of the growth parameters of different crops are inverted by the multidimensional deep learning model of each original crop growth parameter to obtain the growth parameters of different crops after different sowing times, that is, the growth parameters of different crops after different sowing times include but are not limited to the coefficient of determination (R 2), root mean square error (RMSE) and relative root mean square error (rRMSE) were used to test leaf area index and biomass. Furthermore, according to the growth parameters of different crops after different sowing times, the degree of agreement between the actual value of biomass of the multidimensional deep learning model of each original crop growth parameter and the calculated value of the model was determined, among which the determination coefficient R 2 The larger the value, the smaller the corresponding root mean square error RMSE and relative root mean square error rRMSE, and the higher the estimation accuracy of the model. 2 The larger the value, the smaller the root mean square error RMSE and the relative root mean square error rRMSE. 2 The larger the value, the smaller the corresponding root mean square error RMSE and relative root mean square error rRMSE. The target degree of fit value among each degree of fit value is determined, and the original crop growth parameter multidimensional deep learning model corresponding to the target degree of fit value is determined as the optimal crop growth parameter multidimensional deep learning model among each original crop growth parameter multidimensional deep learning model.

[0105] Step S40, evaluating the growth parameter information of different crops through the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0106] Furthermore, the growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops, specifically: the optimal crop growth parameter multidimensional deep learning model is combined with the lossless two-dimensional image data set of different crops to extract features from the single flight data of the entire growth stage of different crops to obtain the multi-phase feature information of different crops in different growth periods. Then, the multi-phase feature information of different crops in different growth periods is evaluated to obtain the spatiotemporal distribution of the growth parameters of different crops, as described in steps S401 to S402.

[0107] Furthermore, crop yield can be expressed as the result of the combined effects of major physiological processes such as crop canopy interception of solar radiation and conversion of solar radiation into biomass. Canopy structural characteristics are the main genetic traits that affect solar radiation interception efficiency. Climate factors are one of the important factors affecting light energy utilization efficiency, especially light and CO2 concentration, because they affect carbon fixation and respiration.

[0108] Further, the description of step S401 to step S402 is as follows:

[0109] Step S401, using the optimal crop growth parameter multidimensional deep learning model combined with the lossless two-dimensional image data set of different crops, feature extraction is performed on the single flight data of the entire growth stage of different crops to obtain multi-phase feature information of different crops in different growth stages;

[0110] Step S402, evaluating the multi-phase characteristic information of the different crops in different growth stages to obtain the spatiotemporal distribution of growth parameters of the different crops.

[0111] Specifically, the multi-dimensional deep learning model of optimal crop growth parameters is combined with a lossless two-dimensional image data set of different crops to extract features from single-flight data of the entire growth stage of different crops, and obtain multi-phase feature information of different crops in different growth periods. Further, the multi-phase feature information of different crops in different growth periods is determined as the input features of the output layer at different time points for calculation to obtain the final growth parameters of different crops. Further, the location information of the preset area where the different crops are located is determined, and based on the final growth parameters of the different crops and their location information, the spatiotemporal distribution of the growth parameters of different crops in the entire preset area is determined, as described in steps S4021 to S4022, wherein the preset area can be a certain field area.

[0112] Further, the description of step S4021 to step S4022 is as follows:

[0113] Step S4021, determining the multi-phase characteristic information of the different crops in different growth stages as the input features of the output layer at different time points for calculation to obtain the final growth parameters of the different crops;

[0114] Step S4022, determining the location information of the preset area where different crops are located, and determining the spatiotemporal distribution of the growth parameters of the different crops in the entire preset area according to the final growth parameters of the different crops and their location information.

[0115] Specifically, the multi-phase characteristic information of different crops in different growth periods is determined as the input features of the output layer at different time points for calculation to obtain the final growth parameters of different crops. Furthermore, the field locations of different crops are determined, and the spatiotemporal distribution of the growth parameters of different crops in the entire field is determined based on the final growth parameters of different crops and their field locations.

[0116] The crop growth prediction method provided in the embodiment of the present application, during the process of crop growth prediction, determines the lossless two-dimensional image dataset of the original three-dimensional point cloud dataset according to the image backtracking algorithm to obtain more accurate features, and then constructs an optimal crop growth parameter multidimensional deep learning model through the original three-dimensional point cloud dataset, the lossless two-dimensional image dataset and the meteorological factor data to obtain the spatiotemporal distribution of growth parameters of different crops, thereby improving the accuracy of the crop growth parameter prediction results.

[0117] The crop growth prediction device provided in the present application is described below. The crop growth prediction device described below and the crop growth prediction method described above can be referenced to each other.

[0118] The present application provides a crop growth prediction device. Figure 4 It is a structural schematic diagram of the crop growth prediction device provided in this application.

[0119] like Figure 4 As shown, Figure 4 : is a schematic diagram of the structure of the crop growth prediction device provided by the present application, and the crop growth prediction device comprises:

[0120] An image acquisition module 401 is used to determine a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm;

[0121] A model building module 402 is used to fuse the original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data to build a multi-dimensional deep learning model of each original crop growth parameter;

[0122] A model determination module 403 is used to perform inversion by combining the original crop growth parameter multidimensional deep learning models with the spatiotemporal specificity of growth parameters of different crops to obtain an optimal crop growth parameter multidimensional deep learning model;

[0123] The result prediction module 404 is used to evaluate the growth parameter information of different crops through the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0124] Furthermore, the model determination module 403 is also used for:

[0125] Inverting the spatiotemporal specificity of growth parameters of different crops through the multidimensional deep learning models of the original crop growth parameters to obtain growth parameters of different crops after different sowing times;

[0126] According to the growth parameters of the different crops after different sowing times, determining the respective matching degree values ​​of the actual biomass value of the multidimensional deep learning model of the original crop growth parameters and the model calculated value;

[0127] Based on each of the matching degree values, the optimal crop growth parameter multidimensional deep learning model among the original crop growth parameter multidimensional deep learning models is determined.

[0128] Furthermore, the result prediction module 404 is also used for:

[0129] By combining the optimal crop growth parameter multidimensional deep learning model with the lossless two-dimensional image data set of different crops, feature extraction is performed on the single-flight data of the entire growth stage of different crops to obtain multi-phase feature information of different crops in different growth stages;

[0130] The temporal and spatial distribution of growth parameters of different crops are obtained by evaluating the multi-phase characteristic information of different crops in different growth stages.

[0131] Furthermore, the result prediction module 404 is also used for:

[0132] Determine the multi-phase characteristic information of the different crops in different growth stages as input features at different time points of the output layer for calculation to obtain final growth parameters of the different crops;

[0133] The position information of the preset area where different crops are located is determined, and according to the final growth parameters of the different crops and the position information thereof, the spatiotemporal distribution of the growth parameters of the different crops in the entire preset area is determined.

[0134] Furthermore, the image acquisition module 401 is also used for:

[0135] Acquire imaging parameters of the aerial camera, and acquire an image retrieval matrix based on the imaging parameters;

[0136] Based on the imaging parameters and the image tracing matrix, a lossless two-dimensional image dataset of the three-dimensional point cloud dataset is acquired.

[0137] Furthermore, the image acquisition module 401 is also used for:

[0138] Based on the imaging parameters and the image tracing matrix, acquiring an initial lossless two-dimensional image data set;

[0139] Inputting the initial lossless two-dimensional image data set into a pre-built image optimization model to obtain a lossless two-dimensional image data set output by the image optimization model;

[0140] The image optimization model is determined based on the color information of the lossless two-dimensional image data set and the optimal hyperparameters corresponding to the color information.

[0141] The specific embodiments of the crop growth prediction device provided in the present application are basically the same as the embodiments of the above-mentioned crop growth prediction method, and will not be described in detail here.

[0142] Figure 5 An example of a physical structure diagram of an electronic device is shown below: Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the crop growth prediction method provided by the above-mentioned method embodiments, and the method may include:

[0143] Determine the lossless 2D image dataset of the original 3D point cloud dataset based on the image backtracking algorithm;

[0144] The original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data are integrated to construct a multi-dimensional deep learning model of each original crop growth parameter;

[0145] The optimal crop growth parameter multidimensional deep learning model is obtained by inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops;

[0146] The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0147] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0148] On the other hand, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the crop growth prediction method provided by the above-mentioned method embodiments, the method for example comprising:

[0149] Determine the lossless 2D image dataset of the original 3D point cloud dataset based on the image backtracking algorithm;

[0150] The original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data are integrated to construct a multi-dimensional deep learning model of each original crop growth parameter;

[0151] The optimal crop growth parameter multidimensional deep learning model is obtained by inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops;

[0152] The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0153] In another aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the crop growth prediction method provided by the above-mentioned method embodiments is implemented, and the method, for example, includes:

[0154] Determine the lossless 2D image dataset of the original 3D point cloud dataset based on the image backtracking algorithm;

[0155] The original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data are integrated to construct a multi-dimensional deep learning model of each original crop growth parameter;

[0156] The optimal crop growth parameter multidimensional deep learning model is obtained by inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops;

[0157] The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops.

[0158] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0159] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A crop growth prediction method, characterized in that: include: Determine the lossless 2D image dataset of the original 3D point cloud dataset based on the image backtracking algorithm; The original three-dimensional point cloud data set, the lossless two-dimensional image data set and the meteorological factor data are integrated to construct a multi-dimensional deep learning model of each original crop growth parameter; The optimal crop growth parameter multidimensional deep learning model is obtained by inverting each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops; The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops; The inversion of each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops to obtain the optimal crop growth parameter multidimensional deep learning model includes: Inverting the spatiotemporal specificity of growth parameters of different crops through the multidimensional deep learning models of the original crop growth parameters to obtain growth parameters of different crops after different sowing times; According to the growth parameters of the different crops after different sowing times, determining the respective matching degree values ​​of the actual biomass value of the multidimensional deep learning model of the original crop growth parameters and the model calculated value; Based on each of the matching degree values, the optimal crop growth parameter multidimensional deep learning model among the original crop growth parameter multidimensional deep learning models is determined.

2. The crop growth prediction method according to claim 1, characterized in that: The growth parameter information of different crops is evaluated by the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops, including: By combining the optimal crop growth parameter multidimensional deep learning model with the lossless two-dimensional image data set of different crops, feature extraction is performed on the single-flight data of the entire growth stage of different crops to obtain multi-phase feature information of different crops in different growth stages; The temporal and spatial distribution of growth parameters of different crops are obtained by evaluating the multi-phase characteristic information of different crops in different growth stages.

3. The crop growth prediction method according to claim 2, characterized in that: The step of evaluating the multi-phase characteristic information of different crops at different growth stages to obtain the spatiotemporal distribution of growth parameters of different crops includes: Determine the multi-phase characteristic information of the different crops in different growth stages as input features at different time points of the output layer for calculation to obtain final growth parameters of the different crops; The position information of the preset area where different crops are located is determined, and according to the final growth parameters of the different crops and the position information thereof, the spatiotemporal distribution of the growth parameters of the different crops in the entire preset area is determined.

4. The crop growth prediction method according to claim 1, characterized in that: The method of determining a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm includes: Acquire imaging parameters of the aerial camera, and acquire an image retrieval matrix based on the imaging parameters; Based on the imaging parameters and the image tracing matrix, a lossless two-dimensional image dataset of the three-dimensional point cloud dataset is acquired.

5. The crop growth prediction method according to claim 4, characterized in that: The step of acquiring a lossless two-dimensional image data set of the three-dimensional point cloud data set based on the imaging parameters and the image tracing matrix includes: Based on the imaging parameters and the image tracing matrix, acquiring an initial lossless two-dimensional image data set; Inputting the initial lossless two-dimensional image data set into a pre-built image optimization model to obtain a lossless two-dimensional image data set output by the image optimization model; The image optimization model is determined based on the color information of the lossless two-dimensional image data set and the optimal hyperparameters corresponding to the color information.

6. A crop growth prediction device, characterized in that: include: An image acquisition module, used for determining a lossless two-dimensional image data set of an original three-dimensional point cloud data set based on an image backtracking algorithm; A model building module, used to fuse the original three-dimensional point cloud data set, the lossless two-dimensional image data set and meteorological factor data to build a multi-dimensional deep learning model of each original crop growth parameter; A model determination module is used to perform inversion by combining the original crop growth parameter multidimensional deep learning models with the spatiotemporal specificity of growth parameters of different crops to obtain an optimal crop growth parameter multidimensional deep learning model; A result prediction module, used to evaluate the growth parameter information of different crops through the optimal crop growth parameter multidimensional deep learning model to obtain the spatiotemporal distribution of the growth parameters of different crops; The inversion of each of the original crop growth parameter multidimensional deep learning models in combination with the spatiotemporal specificity of growth parameters of different crops to obtain the optimal crop growth parameter multidimensional deep learning model includes: Inverting the spatiotemporal specificity of growth parameters of different crops through the multidimensional deep learning models of the original crop growth parameters to obtain growth parameters of different crops after different sowing times; According to the growth parameters of the different crops after different sowing times, determining the respective matching degree values ​​of the actual biomass value of the multidimensional deep learning model of the original crop growth parameters and the model calculated value; Based on each of the matching degree values, the optimal crop growth parameter multidimensional deep learning model among the original crop growth parameter multidimensional deep learning models is determined.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the crop growth prediction method as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the crop growth prediction method according to any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the crop growth prediction method according to any one of claims 1 to 5 is implemented.

Images