Agricultural product yield prediction method based on self-attention mechanism neural network
Through the self-attention mechanism neural network combined with multi-head self-attention and convolutional neural network, the problems of accuracy and efficiency in agricultural product output prediction are solved, and efficient and accurate agricultural product output prediction is achieved.
Patent Information
- Application Number
- CN202510277170.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-10
- Publication Date
- 2025-07-25
AI Technical Summary
The prior art has problems such as insufficient prediction accuracy, low computational efficiency, poor model stability and insufficient adaptability in agricultural product yield prediction, especially when dealing with complex and nonlinear data relationships, it is difficult to capture long-term trends and seasonal changes.
A self-attention mechanism neural network is adopted, combined with a multi-head self-attention mechanism and a convolutional neural network, and through an embedding layer, a multi-head self-attention mechanism, a convolutional neural network and a regression loss function, an agricultural product yield prediction model is constructed, the long-term dependence relationship of the time series is captured and local features are extracted, and the dimensionality reduction process is mapped to the prediction value of future time points.
It improves the accuracy of agricultural product yield prediction and the fitting degree of model, reduces calculation complexity and training costs, improves training efficiency and prediction accuracy, and achieves efficient agricultural product yield prediction.
Smart Images
Figure CN120373514A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big data processing, and in particular to a method for predicting agricultural product yields based on a neural network with self-attention mechanism. Background Art
[0002] The yields of agricultural products are easily affected by factors such as weather and planting areas, and have great uncertainties. In the prior art, methods of conducting several predicted surveys at different growth stages by region and by crop are adopted to obtain the predicted yields. However, such measurement methods require a large amount of manpower input and time costs, and it is also difficult to conduct real-time supervision on the yields of each region.
[0003] With the development of science and technology, machine learning methods have been applied to the prediction of commodity yields. By using machine learning methods to predict commodity yields, complex and non-linear data relationships can be efficiently processed, and the complex interactions between commodity yields and various factors (such as meteorological conditions, market demand, and economic indicators) can be captured, thereby improving the accuracy of prediction. For example, by establishing a multiple linear regression model to analyze the fluctuations in grain yields; however, the regression model based on machine learning has poor stability and is easily affected by the complexity and uncertainty of data, resulting in the predicted values deviating from the actual values. Moreover, the machine learning model has a complex structure and low computational efficiency. However, there are many deficiencies in the prediction of commodity yields by traditional models at present. First, the performance of the model highly depends on the quality and quantity of the training data. If the data has noise or missing values, the prediction results are often affected. Second, the model performs excellently during the training process, but has poor generalization ability during the application process, resulting in a decrease in the prediction accuracy. Especially for time series data, the model often performs poorly in capturing long-term trends and seasonal changes and fails to effectively utilize time dependence. Finally, the adaptability of deep learning models in a dynamic environment is also a key issue. Traditional models may not be able to respond to external changes in a timely manner, resulting in inaccurate predictions.
[0004] Therefore, a method for predicting agricultural product yields that takes into account both prediction accuracy and computational efficiency is needed. Summary of the Invention
[0005] In view of this, the present invention provides a method for predicting agricultural product yields based on a neural network with self-attention mechanism, which makes the data more accurate and the fitting degree of the model higher through the self-attention mechanism.
[0006] To this end, the present invention provides the following technical solutions:
[0007] A method for predicting agricultural product yields based on a neural network with self-attention mechanism, comprising:
[0008] Constructing an agricultural product yield prediction model;
[0009] Collect agricultural product yield data and preprocess it to construct a training dataset;
[0010] Pre-train an agricultural product yield prediction model based on the training dataset;
[0011] Use the pre-trained agricultural product yield prediction model to obtain the agricultural product yield.
[0012] Furthermore, the agricultural product yield prediction model includes:
[0013] Convert the input data into a feature matrix through an embedding layer;
[0014] Map the feature matrix into key vectors, value vectors, and query vectors through a multi-head self-attention mechanism to capture the long-term dependencies of the time series;
[0015] Extract local features through a convolutional neural network and perform dimensionality reduction through pooling;
[0016] Map the feature map after dimensionality reduction into predicted values for future time points;
[0017] And optimize the model through a regression loss function.
[0018] Furthermore, the multi-head self-attention mechanism includes:
[0019] The number of heads of the multi-head attention mechanism is 8;
[0020] And introduce positional encoding to preserve the time order information.
[0021] Furthermore, the extraction of local features and dimensionality reduction through pooling includes:
[0022] The first convolutional layer uses 6 3×3 convolutional kernels to extract local features and outputs an initial feature map;
[0023] The first pooling layer compresses the initial feature map through max pooling;
[0024] The second convolutional layer uses 6 3×3 convolutional kernels to extract features and outputs an intermediate feature map;
[0025] The second pooling layer compresses the intermediate feature map to obtain the final feature map.
[0026] Furthermore, mapping the feature map after dimensionality reduction into predicted values for future time points includes:
[0027] Map the final feature map output by the convolutional neural network into predicted values for future time points through a fully connected layer.
[0028] Furthermore, the loss function includes: mean absolute error.
[0029] Advantages and positive effects of the present invention:
[0030] Through the self-attention mechanism, the data of the present invention is more accurate and the fitting degree of the model is relatively high, and there will be no overfitting and underfitting phenomena; in the present invention, the multi-head self-attention mechanism and the convolutional neural network are combined. The multi-head self-attention mechanism can capture global context information by calculating the relationships between different positions. After combining with the convolutional neural network, the model not only retains the local feature extraction ability of the convolutional neural network, but also supplements global information through the multi-head self-attention mechanism, so as to understand the data more comprehensively and improve the accuracy of model prediction. At the same time, the redundant parameters of the traditional complex network architecture are reduced, the computational complexity and training cost are lowered, and the training efficiency and convergence speed of the model are improved. Therefore, the method of the present invention takes into account both the prediction accuracy and the model operation efficiency. Description of the drawings
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0032] Figure 1 It is the overall structure diagram of the agricultural product yield prediction model in the embodiment of the present invention;
[0033] Figure 2 It is the training data set in the embodiment of the present invention;
[0034] Figure 3 It is the operation flow chart of the multi-head attention mechanism of the agricultural product yield prediction model in the embodiment of the present invention;
[0035] Figure 4 It is the operation flow chart of the convolutional neural network of the agricultural product yield prediction model in the embodiment of the present invention;
[0036] Figure 5 It is the schematic diagram of the prediction error of the agricultural product yield prediction model in the embodiment of the present invention;
[0037] Figure 6 It is the schematic diagram of the training accuracy during the training of the agricultural product yield prediction model in the embodiment of the present invention. Detailed implementation manners
[0038] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0040] The present invention provides an agricultural product yield prediction method based on a self-attention mechanism neural network. By using a machine learning model based on the self-attention mechanism to predict the yield of bulk commodities, and by combining the multi-head attention in the self-attention mechanism with a convolutional neural network, the self-attention mechanism model is used to process and analyze data, making the model both universal and improving the accuracy of data prediction.
[0041] The self-attention mechanism deep learning model has many advantages, making it perform well in multiple fields. It can process the entire input sequence in parallel, greatly improving the training speed, and effectively captures long-range dependencies through the self-attention mechanism, overcoming the limitations of traditional RNNs in processing long sequences. Its flexible input-output structure and encoder-decoder architecture make the model suitable for various tasks, such as translation, text generation, and image processing.
[0042] Specifically, the time-series data of agricultural product yields is input into the neural network through a feed-forward neural network. The self-attention mechanism is used to extract relevant trend information and features from the input data, and finally the predicted value of future yields is output through a linear activation neuron. The self-attention mechanism deep learning model introduced in the present invention not only increases the ability to extract data features but also consolidates the stability of model prediction. After training, the prediction accuracy of this model reaches more than 90%, which is very suitable for yield predictions of agricultural products and other products that are greatly affected by external factors.
[0043] Combined with Figure 1The process of the method of the present invention is described as follows:
[0044] S1. Collect agricultural product data and perform preprocessing; input the preprocessed agricultural product data into the model;
[0045] S2. Build the neural network structure of the agricultural product prediction model and train the neural network.
[0046] S3. Parallelly input the data through the multi-head attention mechanism, perform feature recognition on the output 34*34 two-dimensional image through the convolutional neural network, and finally input the predicted yield.
[0047] The method of the present invention is further described by the following specific embodiments:
[0048] Data collection and preprocessing, self-attention neural network construction, self-attention neural network training, and yield prediction using the trained self-attention neural network.
[0049] Specifically, the neural network used in this embodiment is mainly the multi-head attention mechanism of the transformer model and the convolutional neural network. The transformer model is a deep learning model for natural language processing tasks. It overcomes some limitations of traditional sequence models, such as RNN (Recurrent Neural Network) and LSTM (Long Short-Term Memory) in processing long sequence data, especially in parallel computing and long-distance dependence modeling.
[0050] After preprocessing the data, it is converted into key, value, and query vectors and input into the multi-head attention mechanism, and then processed by the convolutional neural network to finally obtain the predicted value of the next year's yield of agricultural products. This model requires a large amount of data for training to ensure the stability of the model.
[0051] 1) Data collection and preprocessing, constructing a training data set;
[0052] In this embodiment, the yield data of 4,947 soybean and corn production areas from 1988 to 2023 are collected. The location of each area is represented by longitude and latitude. The area of each production area is 50x50 km, and the unit of yield data is tons / hectare. Among them, the x-axis is longitude, the y-axis is latitude, and the z-axis is the year. If the data is sliced horizontally by year, the yield map of bulk agricultural products in a certain year can be obtained.
[0053] Therefore, the yield data of agricultural products in 4,946 areas are obtained, and these data are converted into a two-dimensional structure, and the result is as Figure 2As shown. A total of 4946 lines of data were obtained, and the output of each line of data corresponds to a region. Each region can be represented by a combination of longitude and latitude. At the same time, each line of data contains 36 columns of data, which represent the agricultural product output data for a total of 36 years from 1998 to 2023.
[0054] Use the data from 1988 to 2022 as the training data X. At the same time, extract the output data in 2023 as the training data label Y. Therefore, the dimension of the training data X is 4946×35. The dimension of the data label is 4946×1.
[0055] 2) Construct a self-attention neural network agricultural product output prediction model;
[0056] In this embodiment, the multi-head attention mechanism is combined with the convolutional neural network to improve the accuracy of model prediction.
[0057] Specifically, first set the number of multi-head attentions to 8, which corresponds to 8 groups of different W q , W k and W v ; W q , W k , W v represent trainable parameter matrices. Setting multiple different parameter matrices is because it enhances the representation ability of the model and captures different subspace information. Use the 4946 one-dimensional data preprocessed for 35 years as X n imported, X n is multiplied by W q , W k , W v matrices respectively, and the products obtained are the query (Query), key (Key), and value (Value) vectors. The query vector refers to the range of the query, and the self-prompt is the feature vector of subjective consciousness. The key vector refers to the item to be compared, and the non-self-prompt is the prominent feature information vector of the object. The value vector represents the feature vector of the object itself and usually appears in pairs with the key vector. After normalizing the obtained different query, key, and value vectors through the Softmax function, the weight coefficient Z is obtained. Each weight coefficient Z is greater than 0 and less than 1, and the sum of the values of the weight coefficients is 1.
[0058] Next, connect all the obtained weight coefficients Z together and multiply by the matrix W0 to perform a linear transformation for dimensionality reduction to obtain Figure 3 the two-dimensional image digital feature with a dimension of 34*34 in. The two-dimensional image digital feature of 34*34 captures all the important information of the weight coefficient Z, including the trend that the output level of this region has a gradually increasing trend from the output in previous years, and convert this feature into a two-dimensional image.
[0059] Since the output of the multi-head attention mechanism is two-dimensional image digital features with a dimension of 34*34, and the convolutional neural network has strong image recognition ability and can better capture the digital features in the image, so in this embodiment, the convolutional neural network is used to process the two-dimensional image.
[0060] The two-dimensional image digital features with a dimension of 34*34 output by the multi-head attention mechanism are converted into three-dimensional data of 34*34*1 through the reshape function (reconstruction function);
[0061] The three-dimensional data is input into the convolutional neural network to obtain the predicted value of agricultural product yield;
[0062] Specifically, the three-dimensional data is input into the convolutional layer. The first layer is the convolutional layer. For example, Figure 4 There are a total of 6 convolutional kernels, and the size of each convolutional kernel is specified as 3*3, which means that a 3*3 matrix captures the two-dimensional image data features, can magnify the change trend of each group of data, and thus can make more accurate predictions. The feature map output from the first convolutional layer is 34*34*6.
[0063] The 34*34*6 feature map then enters the max-average pooling layer. By capturing the maximum weight coefficient, it can better highlight the data characteristics, and the 34*34*6 feature map is compressed to a 17*17*6 feature map.
[0064] The third layer is also a convolutional layer, and the number and size of the convolutional kernels are the same as those of the first layer, further refining the features to increase the prediction accuracy. The feature map output after the third convolution is 17*17*12.
[0065] The fourth layer is still the max-average pooling layer to compress the feature map, and the final output is an 8*8*12 feature map.
[0066] The fifth and sixth layers are two fully connected layers, aiming to perform multiple dimensionality reductions on the output feature map to ensure that the final output is a one-dimensional data. The one-dimensional data output by the fifth fully connected layer is 128, and the last fully connected layer adds a linear function to output a one-dimensional data, which is the predicted yield data for 2023. The goodness of fit R is obtained by regressing the predicted 2023 yield with the actual 2023 yield. 2 Using R 2 To judge the accuracy of the model prediction: the closer it is to 1, the greater the overlap between the model prediction value and the actual value, and the more accurate the prediction. Conversely, the closer it is to 0, the smaller the overlap between the model prediction value and the actual value, and the less accurate the prediction.
[0067] 3) Train the self-attention neural network agricultural product yield prediction model;
[0068] Use 4,946 pieces of 1×35 time series data in the training data X to train a self-attention neural network so that it can predict the agricultural product output in 2023. Each piece of training data represents the time series data of the output in a certain region from 1988 to 2022, and its corresponding data label is the output data of that region in 2023. If the training accuracy of this model reaches more than 90%, then this neural network has the ability to predict the output of the next year, and the prediction accuracy can reach more than 90%.
[0069] Specifically, during the training process, Adam is used as the optimization algorithm to optimize the model, the mean absolute error is used as the loss function, the output of the previous 35 years is used as the data, and the output of the last year is used as the data label, and then they are put into the neural network to start training; the number of training times is set to 20, and the batch size is set to 5, that is, training 20 times, and each time 5 samples are used as a group for training. At the same time, in order to evaluate the performance of the model, the validation set ratio is set to 0.1, that is, 90% of the data is used as training data, and 10% of the data is used as validation data for model evaluation.
[0070] Figure 5 is the prediction error of the neural network. During the first 5 rounds of training, both the training error and the test error are relatively large. There is a small fluctuation during the 5th round of training, and then the error gradually decreases to 0.4. After 20 rounds of training, the training error drops to about 0.2, and the test error stabilizes at about 0.3. Figure 6 are the training accuracy and test accuracy during training. During the first 5 rounds of training, the accuracy is relatively low; after 20 stages of training, the training accuracy gradually increases to 95%, and the test accuracy basically stabilizes at 92%.
[0071] Through comparative experiments, it is verified that the method of the present invention is completely different from the traditional transformer model in data processing, and taking the traditional transformer model as the control group, it can be clearly seen that the prediction accuracy of the present invention is higher than that of the traditional transformer model: the accuracy of the transformer model is 0.8319367509022295; the accuracy of the method of the present invention is: 0.9623512402677887.
[0072] The specific training data is shown in Table 1. After training, the accuracy of the model of the present invention reaches more than 95%. Therefore, the degree of fit between the predicted value of the next year's corn and soybean output level output by the trained self-attention neural network agricultural product output prediction model and the actual value is very high, indicating that the method of the present invention has high data feature capture ability, which greatly improves the usability of the predicted value.
[0073] Table 1
[0074] Number of training rounds Training error Training accuracy Test error Test accuracy 1 1.509173257 0.547851852 0.703564234 0.831166667 2 0.590505749 0.843185185 0.398227773 0.898166667 3 0.422766301 0.884722222 0.329913737 0.909333333 4 0.353594485 0.899666667 0.271886317 0.923166667 5 0.315588128 0.910333333 0.252434665 0.927166667 6 0.292026466 0.915111111 0.253874879 0.929166667 7 0.274671156 0.920092593 0.223573253 0.934666667 8 0.262491555 0.923962963 0.220129146 0.9345 9 0.253391318 0.925833333 0.214039695 0.9375 10 0.244605096 0.929018519 0.20441847 0.937333333 11 0.237689587 0.930833333 0.206044807 0.938166667 12 0.231385597 0.932722222 0.210141642 0.938666667 13 0.227029874 0.933481481 0.202860791 0.9375 14 0.222746578 0.935148148 0.192643067 0.942333333 15 0.21808334 0.93687037 0.190366698 0.945333333 16 0.21435387 0.936962963 0.182843432 0.945833333 17 0.210433878 0.938648148 0.185066249 0.944833333 18 0.207114764 0.94 0.179817399 0.945 19 0.204245694 0.940148148 0.183388193 0.9485 20 0.201077945 0.941037037 0.181470261 0.9455
[0075] 4) Use the trained self-attention neural network agricultural product yield prediction model to predict the agricultural product yield;
[0076] By inputting the yield levels of bulk commodities in a certain or certain regions over 35 years, the yield value of the commodity in the next year can be obtained, where the unit of the annual agricultural product yield value input is tons / hectare.
[0077] Use the corn and soybean yield values of 4946 regions from 1989 to 2023 to predict the corn and soybean yield levels in each region in 2024. The yield values of corn and soybeans in these regions each year are drawn in the form of satellite images using the matplotlib library as input quantities, and are input into the model. Through the processing of the self-attention mechanism, the yield values in 2024 are output in the form of satellite images. Based on the predicted yield values of agricultural products (corn and soybeans) in each region in 2024, the total yield values of corn and soybeans in the world in 2024 can also be obtained. The following is the calculation process of the total yield value. Let the corn and soybean output in each region in 2024 be Pi (i = 1, 2,..., 4946) with the unit of tons / hectare. From unit conversion, it can be obtained that:
[0078] 2500km 2 = 2500 × 100 = 250000 hectares
[0079] Calculate the total output of each region and sum them up to obtain the total output P of the world (world) It is:
[0080]
[0081] Calculate the total yield value of corn and soybeans in the world in 2024 according to the above formula, with the unit of tons.
[0082] The present invention solves the influence of the complexity and uncertainty of data on the accuracy of prediction results; the complexity and uncertainty of data include: factors such as supply and demand relationships, geopolitics, climate change, and market sentiment are usually highly dynamic and uncertain, increasing the complexity of prediction; the available data may have missing values, noise, and inconsistencies, affecting the training and prediction capabilities of the model. In addition, there may be problems of inconsistent formats and time spans between different data sources.
[0083] Therefore, the method of the present invention realizes the prediction of the output of bulk agricultural products without manual intervention in complex environments; for the high-precision prediction of the future output of bulk commodities, the data of the previous 35 years are used to train the output of the last year, and the final fitting degree reaches 90%, which provides a strong guarantee for the prediction of the output of bulk commodities; and automatically generates the annual output prediction maps of soybeans and corn in various regions of the world.
[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An agricultural product yield prediction method based on a neural network with self-attention mechanism, characterized in that, Including: Construct an agricultural product yield prediction model; Collect agricultural product yield data and preprocess it to construct a training dataset; Pre-train the agricultural product yield prediction model based on the training dataset; Use the pre-trained agricultural product yield prediction model to obtain the agricultural product yield.
2. The agricultural product yield prediction method based on a neural network with self-attention mechanism according to claim 1, wherein The agricultural product yield prediction model includes: Convert the input data into a feature matrix through an embedding layer; Map the feature matrix into a key vector, a value vector, and a query vector through a multi-head self-attention mechanism to capture the long-term dependencies of the time series; Extract local features through a convolutional neural network and perform dimensionality reduction through pooling; Map the feature after dimensionality reduction into the predicted value of the future time point; And optimize the model through a regression loss function.
3. The agricultural product yield prediction method based on the self-attention mechanism neural network according to claim 2, wherein The multi-head self-attention mechanism includes: The number of heads of the multi-head attention mechanism is 8; And introduce positional encoding to retain the time order information.
4. The agricultural product yield prediction method based on a neural network with self-attention mechanism according to claim 2, characterized in that, The extracting local features and performing dimensionality reduction through pooling includes: The first convolutional layer uses 6 3×3 convolutional kernels to extract local features and outputs an initial feature map; The first pooling layer compresses the initial feature map through max pooling; The second convolutional layer uses 6 3×3 convolutional kernels to extract features and outputs an intermediate feature map; The second pooling layer compresses the intermediate feature map to obtain the final feature map.
5. The agricultural product yield prediction method based on a self-attention mechanism neural network according to claim 4, wherein Mapping the feature after dimensionality reduction into the predicted value of the future time point includes: Map the final feature map output by the convolutional neural network into the predicted value of the future time point through a fully connected layer.
6. The agricultural product yield prediction method based on a neural network with self-attention mechanism according to claim 1, wherein The loss function includes: mean absolute error.