Multi-mode agricultural pest and disease prediction method

Abstract

The invention provides a multi-modal agricultural disease and pest prediction method. The method comprises the following steps: acquiring image data and audio data by using a sensor, respectively preprocessing the image data and the audio data, extracting semantic features of the image data by using a CNN model, extracting MFCC features of the audio data by using a Mel system, and splicing and fusing the semantic features and the MFCC features to obtain fused features; the fusion features are input into a BiLSTM model according to time steps, an output result is obtained through training, the output result is input into a full connection layer for classification, probability distribution of all classes is obtained, a pest and disease damage prediction model is obtained, and a final model is obtained after verification set testing and parameter adjustment and used for prediction. Through feature extraction and fusion of the image and audio data, the BiLSTM model is used to carry out classification prediction on the fusion features, automatic prediction of agricultural diseases and insect pests is realized, and the prediction accuracy is higher on the basis of reducing manpower and time costs.

Description

Technical Field

[0001] The present invention relates to the field of agricultural information technology, and in particular to a multi-modal agricultural pest prediction method. Background Art

[0002] In traditional agricultural production, pest control relies on manual observation and statistics. With the advancement of various types of sensor technologies and the reduction of costs, it is feasible to deploy sensors on a large scale in agricultural production to collect corresponding data. For example, sound sensors can capture sound signals emitted by insects, while image sensors can obtain image information such as the shape, color, and texture of crops. By arranging multiple types of sensors in agricultural production scenes, a large amount of raw data information can be obtained, but if these raw data information are processed manually, it will cost an inestimable amount of labor and time. How to quickly process the data information collected by sensors and accurately predict the pest and disease targets pointed to in the data information is a difficult problem that needs to be solved urgently.

[0003] In response to the above problems, the model built based on deep learning technology can well complete the task of agricultural pest prediction. However, in actual agricultural production, there are not only a wide variety of crops and pests, but also a complex and changeable production environment. Simply applying deep learning technology cannot accurately identify various pests. At the same time, the occurrence of pests and diseases is random and uncertain, and it is difficult to collect the complete process of pest and disease occurrence and the corresponding sound and image data at the set time and place. In addition, how to effectively fuse the different types of data features collected by multiple types of sensors while avoiding excessive computational complexity is also a difficulty in the application of deep learning technology in pest and disease prediction. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a high-accuracy multimodal agricultural pest prediction method.

[0005] Technical solution: A multimodal agricultural pest prediction method, comprising the following steps: (1) Deploy image sensors and audio sensors in the farmland to collect image data and audio data, annotate the image data and audio data respectively, and randomly divide them into training set and validation set according to the set ratio; (2) After preprocessing the image data and audio data in the training set, the CNN model is used to extract the semantic features of the image data, and the Mel system is used to extract the MFCC features of the audio data; (3) Concatenate and fuse the semantic features of the image data and the MFCC features of the audio data to obtain fused features; (4) Input the fused features into the BiLSTM model according to the time step, obtain the output result of the last time step after training, and input the output result into the fully connected layer for classification to obtain the probability distribution of each category and obtain the pest and disease prediction model; (5) Use the validation set to test the pest and disease prediction model obtained in step (4), calculate the classification accuracy, and compare the accuracy with the set threshold. If it is lower than the set threshold, adjust the model parameters of the pest and disease prediction model and repeat step (4) until the accuracy is greater than or equal to the set threshold.

[0006] Specifically, in step (1), the image sensor includes a high-resolution camera, a multi-spectral camera and an infrared thermal imaging camera, and the audio sensor includes a microphone array and an ultrasonic sensor.

[0007] Specifically, in step (2), the preprocessing includes: performing image compression, Gaussian denoising and image enhancement on the image data in sequence; and performing pre-emphasis, frame division and windowing on the audio data.

[0008] Specifically, in step (2), the semantic features of image data extracted using the CNN model include: Input the image data into the CNN model, set the height, width and number of channels of the input image, set the height, width, number of channels, stride and padding of the convolution kernel in the convolution layer, output the height, width and number of channels of the feature map obtained by the convolution operation, and set the activation function; Set the pooling window size and step size in the two-dimensional maximum pooling layer, and output the height, width, and number of channels of the feature map obtained by the pooling operation; Repeat the above steps to perform the set rounds of convolution and pooling operations; In the fully connected layer, the feature map is mapped to a feature vector of fixed dimension to obtain the semantic features of the image data.

[0009] Specifically, in step (2), the MFCC features of the audio data extracted using the Mel system include: Input the audio data into the Mel system, perform fast Fourier transform on each frame of audio signal, convert the time domain signal into the frequency domain signal, and calculate the amplitude spectrum and power spectrum of the spectrum; The number of filters in the Mel filter bank is set, the energy output after the Mel filter bank is calculated, and the energy output is logarithmically operated. The energy output after the logarithmic operation is subjected to discrete cosine transform to obtain the MFCC feature.

[0010] Specifically, step (3) includes the following sub-steps: (31) Mapping the semantic features and MFCC features to the query, key, and value spaces, respectively, to obtain the query vector, key vector, and value vector of the semantic features and the query vector, key vector, and value vector of the MFCC features, respectively; (32) Calculate the cross-modal attention score of semantic features on MFCC features and the MFCC feature representation under semantic feature attention; (33) Calculate the cross-modal attention score of MFCC features on semantic features and the semantic feature representation under MFCC feature attention; (34) The MFCC feature representation under the attention of semantic features and the semantic feature representation under the attention of MFCC features are concatenated and fused to obtain fused features.

[0011] Specifically, in step (3), the cross-modal attention score of the semantic feature on the MFCC feature is: , Where: is the cross-modal attention score of semantic features on MFCC features, a refers to MFCC features, i refers to semantic features, and softmax is a normalized exponential function. is the query vector of semantic features, is the key vector of MFCC features, T is the transposition operation, is the dimension of the key vector; The MFCC feature representation under semantic feature attention is: , Where: It is the MFCC feature representation under the semantic feature attention. is the value vector of MFCC features; The cross-modal attention score of MFCC features on semantic features is: , Where: is the cross-modal attention score of MFCC features on semantic features, is the query vector of MFCC features, is the key vector of semantic features; , Where: It is the semantic feature representation under the attention of MFCC features. is the value vector of semantic features; The fusion features are: , Where: To fusion features, is the concatenation function.

[0012] Specifically, in step (4), the probability distribution of each category is: , Where: is the probability of the kth category, is the output of the BiLSTM model after the average pooling operation, C is the total number of categories, O is the output of the fully connected layer, j=1,2,...,C, , is the weight matrix, is the bias term.

[0013] Specifically, in step (4), the Adam optimizer is used to update the parameters during the BiLSTM model training process, and the multi-classification cross entropy loss function is used to calculate the loss.

[0014] Specifically, in step (5), adjusting the model parameters of the pest prediction model includes: Adjust the number of layers of the BiLSTM model; adjust the size of the hidden layer of the BiLSTM model; set a Dropout layer before the classification layer of the BiLSTM model and adjust the random dropout probability; adjust the learning rate of the Adam optimizer.

[0015] Beneficial effects: Compared with the prior art, the significant effects of the present invention are: the present invention uses the CNN model to extract image features, uses the Mel system to extract audio features, and performs multimodal fusion of image features and audio features, and uses the BiLSTM model to input multimodal features into the sequence, performs sequence data processing and captures long-term dependencies, and realizes automatic recognition and prediction of pest information. The CNN model is used to extract farmland image features. Since the CNN model has the characteristics of translation invariance, as long as the features related to pests and diseases are detected by the convolution kernel, they can be recognized regardless of their position in the image, making the model more robust; MFCC can extract the spectrum envelope information of the audio signal. In the prediction of pests and diseases, the sounds emitted by different pests will have obvious differences in the spectrum envelope, and MFCC can highlight these differences. The attention mechanism enables the model to automatically learn and focus on the most valuable parts of the two modalities for pest and disease prediction, and assign different weights to sound and image features according to the specific content of the input data, so that the model can adaptively adjust the degree of attention to sound and image features. The BiLSTM model can fuse sequence data of different modes step by step, and it has the ability to learn long-term dependencies, which can avoid the problem of gradient vanishing or gradient exploding in long sequence data. The BiLSTM model can also remember past information and use it for current predictions, thereby better capturing the dynamic changes in pest sound behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the process of the present invention.

[0017] Figure 2 It is a schematic diagram of the architecture of the CNN model of the present invention. DETAILED DESCRIPTION

[0018] A preferred embodiment of the present invention is further described below in conjunction with the accompanying drawings.

[0019] Example 1

[0020] See also Figure 1 As shown, this embodiment provides a multimodal agricultural pest prediction method, and the specific steps are as follows: Step 1: Deploy image sensors and audio sensors in the farmland to collect image data and audio data, label the image data and audio data respectively, and randomly divide them into training set and validation set according to the set ratio.

[0021] In this embodiment, the image sensor includes a high-resolution camera, a multi-spectral camera, and an infrared thermal imaging camera, and the audio sensor includes a microphone array and an ultrasonic sensor.

[0022] In specific implementation scenarios, high-resolution cameras can be mounted on drones to conduct large-scale inspections in farmland and take photos of pests and diseases from a variety of different angles and distances; multispectral cameras can simultaneously obtain image information of multiple spectral bands. Different crops and pests and diseases have unique reflection characteristics in different spectral bands; infrared thermal imaging cameras are used to detect the temperature distribution on the surface of objects. When crops are attacked by pests and diseases, their physiological state will change, resulting in changes in local temperature, and infrared thermal imaging cameras can capture these temperature differences. An array composed of multiple microphones can collect sound signals in a farmland environment and locate the location of pests through information such as the direction of sound propagation and time difference. Some types of pests generate ultrasonic signals during flight or movement. Ultrasonic sensors can specifically detect these ultrasonic waves and have high sensitivity and anti-interference capabilities.

[0023] In this embodiment, the feasibility of the present invention is demonstrated by taking the public image dataset IP102 and the corresponding audio dataset that have completed data annotation as examples. The IP102 dataset is a large-scale benchmark dataset for pest identification, which contains 102 pest categories and a total of 75,000 images. The audio dataset contains audio data corresponding to the pest categories in the IP102 dataset. The above two datasets are randomly divided into a training set and a validation set according to a certain ratio.

[0024] Step 2: After preprocessing the image data and audio data respectively, the CNN model is used to extract the semantic features of the image data, and the Mel system is used to extract the MFCC features of the audio data.

[0025] In this embodiment, a typical preprocessing process for image data includes the following steps: (1) For image data, input the image path. For each image, specify the compression quality parameter, such as quality=50, to compress the image.

[0026] (2) Perform Gaussian denoising on the image. Use the Gaussian function to calculate the weight of the filter and perform weighted averaging on the image. This can effectively remove Gaussian noise and retain the details of the image edges.

[0027] The formula for the Gaussian function is:

[0028] Where: represents the position of the pixel in the filter, is the standard deviation, which determines the width of the Gaussian distribution.

[0029] (3) Linearly map the grayscale value range in the original image to ,Using image enhancement technology based on contrast stretching, the contrast of the image can be ,enhanced and the difference between the target and the background in the image can be ,highlighted.

[0030] The linear mapping formula is:

[0031] Where: is the original gray value, is the gray value after transformation, a and b represent the upper and lower limits of the gray value of the original image respectively.

[0032] In this embodiment, a typical preprocessing process for audio data includes the following steps: (1) Use pre-emphasis technology to enhance high-frequency components. Pre-emphasis is achieved through a first-order high-pass filter, and the formula is:

[0033] Where: is the nth sample after pre-emphasis processing, is the th sample of the original audio signal, a is the pre-emphasis coefficient, and its value is between 0.9 and 1.0.

[0034] (2) Perform frame processing on the audio signal. The formula is:

[0035] Where: is the audio signal after frame processing, k is the frame index, M is the frame shift, , is the frame length.

[0036] (3) Perform windowing on the audio signal to reduce spectrum leakage. Using the Hamming window, the formula is:

[0037] Where: is the Hamming window function, and the audio signal after windowing the kth frame is .

[0038] After completing data preprocessing, perform the following operations on the image data: (1) Please refer to Figure 2 As shown in the figure, the overall architecture of the CNN convolutional neural network includes 1 input layer, 3 groups of 2D convolutional layers-pooling layers of different sizes and 1 fully connected layer. The image data is input into the convolutional neural network, and the height H, width W and number of channels C of the input image X are set.

[0039] (2) In the two-dimensional convolution layer, set the convolution kernel K height h, width w, number of channels C, step size s, padding p, use n convolution kernels, and output the height of the feature map Y ,width , number of channels The specific calculation formula of feature map Y is:

[0040] Where: is the bias term, i represents the index in the height direction, j represents the index in the width direction, and k represents the channel number index.

[0041] (3) Set the activation function ReLU. For each element y in the convolutional layer output Y, the activation function formula is: .

[0042] (4) In the 2D max pooling layer, set the pooling window size , step length , the height of the output feature map Z ,width , number of channels The specific calculation formula of the feature map Z is:

[0043] (5) Repeat the above steps (2)-(4) and perform multiple rounds of convolution and pooling operations to gradually extract richer semantic features.

[0044] (6) In the fully connected layer, the feature map is mapped to a feature vector of fixed dimension to obtain the semantic features of the image data. The calculation formula is:

[0045] Where: is the semantic feature of the image data, is the weight matrix, is the bias term.

[0046] Perform the following operations on audio data: (1) Input the audio data into the Mel system, perform fast Fourier transform on each frame of audio signal, and convert the time domain signal into the frequency domain signal. The Fourier transform formula is:

[0047] Where: k is the discrete frequency in the frequency domain, and N is the number of sampling points in the time domain.

[0048] (2) Calculate the amplitude spectrum of the frequency spectrum:

[0049] Where: yes The real part of yes The imaginary part of . Calculate the power spectrum: .

[0050] (3) Set up the Mel filter bank and calculate the energy output after passing through the Mel filter bank:

[0051] Where: M is the number of filters, is the mth filter.

[0052] (4) Perform logarithmic operation on energy output: .

[0053] (5) Perform discrete cosine transform (DCT) on the energy output after logarithmic operation to obtain MFCC features The DCT formula is:

[0054] Where: is the normalization coefficient, and N is the number of MFCCs to be extracted.

[0055] Step 3: Concatenate and fuse the semantic features of the image data and the MFCC features of the audio data to obtain fused features.

[0056] (1) Semantic features With MFCC features Mapped to query, key and value spaces respectively, and obtain query vectors of semantic features respectively , key vector Sum value vector And the query vector, key vector and value vector of MFCC features.

[0057] (2) Calculate the cross-modal attention score of semantic features on MFCC features and the MFCC feature representation under semantic feature attention; the cross-modal attention score of semantic features on MFCC features is:

[0058] Where: is the cross-modal attention score of semantic features on MFCC features, a refers to MFCC features, i refers to semantic features, and softmax is a normalized exponential function. is the query vector of semantic features, is the key vector of MFCC features, T is the transposition operation, is the dimension of the key vector.

[0059] The MFCC feature representation under semantic feature attention is:

[0060] Where: It is the MFCC feature representation under the attention of semantic features. is the value vector of MFCC features.

[0061] (3) Calculate the cross-modal attention score of MFCC features on semantic features and the semantic feature representation under MFCC feature attention; the cross-modal attention score of MFCC features on semantic features is:

[0062] Where: is the cross-modal attention score of MFCC features on semantic features, is the query vector of MFCC features, is the key vector of the semantic feature.

[0063]

[0064] Where: It is the semantic feature representation under the attention of MFCC features. is the value vector of semantic features.

[0065] (4) The MFCC feature representation under semantic feature attention and the semantic feature representation under MFCC feature attention are concatenated and fused to obtain the fused feature.

[0066] The fusion features are:

[0067] Where: To fusion features, is the concatenation function.

[0068] Step 4: Input the fused features into the BiLSTM (bidirectional long short-term memory neural network) model according to the time step. After training, the output result of the last time step is obtained, and the output result is input into the fully connected layer for classification. The classification targets of multimodal data include pests and diseases that often occur in different regions, such as rice leaf roller, rice planthopper, rice leaf miner, rice gall midge, etc. Finally, the probability distribution of each category is obtained, and the pest and disease prediction model is obtained.

[0069] (1) In the BiLSTM model, the model considers both forward and reverse sequence information. For the forward LSTM, the calculation is performed from the beginning to the end of the sequence in chronological order to obtain a series of forward hidden layer states: For the reverse LSTM, the calculation is performed from the end of the sequence to the beginning to obtain the reverse hidden layer state , the order of its input sequence is reversed. At each time step t, the output of BiLSTM is the forward hidden layer state With the reverse hidden layer state splicing, that is In this way, BiLSTM can simultaneously utilize the past and future information of the sequence at each time step.

[0070] Using the above splicing and fusion method, the image feature vector and the audio feature vector are spliced ​​according to the time step Here, i is used to refer to image features, a is used to refer to audio features, and t is the time step. This setting is used in the following steps.

[0071] (2) The BiLSTM layer consists of two LSTM layers, one for processing the forward sequence and the other for processing the reverse sequence. Taking the forward calculation as an example, the cell state information that needs to be retained at the previous moment is calculated through the forget gate:

[0072] Where: is the weight matrix of the forget gate, is the bias term, is the activation function, is the state of the forward hidden layer at the previous moment.

[0073] (3) The input gate controls the new input information that can be updated to the cell state at the current moment, which consists of two parts:

[0074] Where: is the weight matrix of the input gate, is the bias term, is the weight matrix used to generate candidate cell states, is the bias term, is the activation function, is a candidate cell state.

[0075] (4) Combine the forget gate and the input gate to update the cell state:

[0076] Where: is the current cell state, Represents the Hadamard product.

[0077] (5) Determine the current cell state through the output gate How much information is output at the current moment:

[0078] Where: is the weight matrix of the output gate, is the bias term, is the forward hidden layer state, Output gate status.

[0079] (6) After obtaining the forward output through the above steps, the reverse hidden layer state is obtained through reverse calculation , then the output of BiLSTM is , perform an average pooling operation on the output:

[0080] Where: T is the total number of time steps.

[0081] (7) In the fully connected layer, set the weight matrix , bias term , the output formula of the fully connected layer is: . The probability distribution of each category is generated by the softmax function: , Where: is the probability of the kth category, is the output of the BiLSTM model after the average pooling operation, C is the total number of categories, O is the output of the fully connected layer, j=1,2,...,C.

[0082] Calculate the prediction result: .

[0083] Through the above steps, the pest and disease prediction model is obtained.

[0084] In the above training process, the Adam optimizer is used to update the parameters: For the parameter θ, let the gradient be:

[0085] Where: L is the loss function, and t represents the tth iteration.

[0086] Compute first moment estimates :

[0087] Compute second moment estimates :

[0088] Where: , is a hyperparameter.

[0089] right , To make a deviation correction:

[0090]

[0091] Last updated parameters:

[0092] Where: is the learning rate, , A very small number used to prevent the denominator from being 0.

[0093] During the training process, the multi-classification cross entropy loss function is used to calculate the loss, and the formula is:

[0094] Where: C is the number of categories, is the real category, is the predicted category.

[0095] Step 5: Use the validation set to test the pest and disease prediction model obtained in step 4, calculate the classification accuracy, and compare the accuracy with the set threshold. If it is lower than the set threshold, adjust the model parameters of the pest and disease prediction model and repeat step 4 until the accuracy is greater than or equal to the set threshold.

[0096] Model parameter adjustments include: (1) Adjust the number of layers of the BiLSTM model.

[0097] (2) Adjust the size of the hidden layer of the BiLSTM model.

[0098] (3) Set the Dropout layer before the classification layer of the BiLSTM model and adjust the random dropout probability p , the discarded neurons have an output of 0 and their weights are not updated during the back propagation process. The neurons that are not discarded will be Scale by the ratio of .

[0099] Its output can be expressed as: .

[0100] (4) Adjust the learning rate of the Adam optimizer .

[0101] After completing the above steps, save the model whose accuracy reaches the set threshold, call the model in the next training, and update it again to improve the model accuracy under more and more multimodal data training.

Claims

1. A multimodal agricultural pest prediction method, characterized in that: The following steps are involved: S1. Deploy image sensors and audio sensors in the farmland to collect image data and audio data, annotate the image data and audio data respectively, and randomly divide them into training set and validation set according to the set ratio; S2, after preprocessing the image data and audio data in the training set, the semantic features of the image data are extracted using the CNN model, and the MFCC features of the audio data are extracted using the Mel system; S3, concatenating and fusing the semantic features of the image data and the MFCC features of the audio data to obtain fused features; S4. Input the fused features into the BiLSTM model according to the time step, obtain the output result of the last time step after training, and input the output result into the fully connected layer for classification, obtain the probability distribution of each category, and obtain the pest and disease prediction model; S5. Use the validation set to test the pest prediction model obtained in step S4, calculate the classification accuracy, and compare the accuracy with the set threshold. If it is lower than the set threshold, adjust the model parameters of the pest prediction model and repeat step S4 until the accuracy is greater than or equal to the set threshold.

2. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In step S1, the image sensor includes a high-resolution camera, a multi-spectral camera and an infrared thermal imaging camera, and the audio sensor includes a microphone array and an ultrasonic sensor.

3. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In the step S2, the preprocessing includes: performing image compression, Gaussian denoising and image enhancement on the image data in sequence; and performing pre-emphasis, frame division and windowing on the audio data.

4. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In step S2, extracting semantic features of image data using a CNN model includes: Input the image data into the CNN model, set the height, width and number of channels of the input image, set the height, width, number of channels, stride and padding of the convolution kernel in the convolution layer, output the height, width and number of channels of the feature map obtained by the convolution operation, and set the activation function; Set the pooling window size and stride in the two-dimensional maximum pooling layer, and output the height, width, and number of channels of the feature map obtained by the pooling operation; Repeat the above steps to perform the set rounds of convolution and pooling operations; In the fully connected layer, the feature map is mapped to a feature vector of fixed dimension to obtain the semantic features of the image data.

5. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In step S2, extracting MFCC features of audio data using the Mel system includes: Input the audio data into the Mel system, perform fast Fourier transform on each frame of audio signal, convert the time domain signal into the frequency domain signal, and calculate the amplitude spectrum and power spectrum of the spectrum; The number of filters in the Mel filter bank is set, the energy output after the Mel filter bank is calculated, and the energy output is logarithmically operated. The energy output after the logarithmic operation is subjected to discrete cosine transform to obtain the MFCC feature.

6. The multimodal agricultural pest prediction method according to claim 1, characterized in that: The step S3 comprises the following sub-steps: S31, mapping the semantic features and the MFCC features to the query, key and value spaces respectively, and obtaining the query vector, key vector and value vector of the semantic features and the query vector, key vector and value vector of the MFCC features respectively; S32, calculating the cross-modal attention score of the semantic feature on the MFCC feature and the MFCC feature representation under the attention of the semantic feature; S33, calculating the cross-modal attention score of the MFCC feature on the semantic feature and the semantic feature representation under the MFCC feature attention; S34, concatenating and fusing the MFCC feature representation under the semantic feature focus and the semantic feature representation under the MFCC feature focus to obtain a fused feature.

7. The multimodal agricultural pest prediction method according to claim 6, characterized in that: In step S3, the cross-modal attention score of the semantic feature to the MFCC feature is: ， Where: is the cross-modal attention score of semantic features on MFCC features, a refers to MFCC features, i refers to semantic features, and softmax is a normalized exponential function. is the query vector of semantic features, is the key vector of MFCC features, T is the transposition operation, is the dimension of the key vector; The MFCC feature representation under semantic feature attention is: ， Where: It is the MFCC feature representation under the attention of semantic features. is the value vector of MFCC features; The cross-modal attention score of MFCC features on semantic features is: ， Where: is the cross-modal attention score of MFCC features on semantic features, is the query vector of MFCC features, is the key vector of semantic features; ， Where: It is the semantic feature representation under the attention of MFCC features. is the value vector of semantic features; The fusion features are: ， Where: To fusion features, is the concatenation function.

8. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In step S4, the probability distribution of each category is: ， Where: is the probability of the kth category, is the output of the BiLSTM model after the average pooling operation, C is the total number of categories, O is the output of the fully connected layer, j=1,2,...,C, , is the weight matrix, is the bias term.

9. The multimodal agricultural pest prediction method according to claim 1, characterized in that: In step S4, the Adam optimizer is used to update parameters during the BiLSTM model training process, and the multi-classification cross entropy loss function is used to calculate the loss.

10. The multimodal agricultural pest prediction method according to claim 9, characterized in that: In step S5, adjusting the model parameters of the pest prediction model includes: Adjust the number of layers of the BiLSTM model; adjust the size of the hidden layer of the BiLSTM model; set a Dropout layer before the classification layer of the BiLSTM model and adjust the random dropout probability; adjust the learning rate of the Adam optimizer.

Images