A method and system for identifying crop diseases and pests based on dual-dimensional fusion decision.

Abstract

This invention provides a method and system for identifying crop pests and diseases based on a two-dimensional fusion decision, relating to the field of crop pest and disease identification. The method includes preprocessing acquired images of pest biological features and images of pest damage symptoms to obtain corresponding three-dimensional arrays as input features; extracting corresponding feature maps from the input features using a pre-defined two-dimensional fusion decision model; flattening the corresponding feature maps using pooling operations to obtain feature vectors for the pest biological feature images and the pest damage symptoms images, respectively; fusing the feature vectors of the pest biological feature images and the pest damage symptoms images to obtain the final output; and training the two-dimensional fusion decision model using a loss function to obtain the final output identification result. This invention effectively addresses the difficulty of identifying closely related and similar species of pests.

Description

Technical Field

[0001] This invention relates to the field of crop pest and disease identification technology, specifically to a crop pest and disease identification method and system based on dual-dimensional fusion decision. Background Technology

[0002] Currently, due to a combination of factors such as improved agricultural production levels, changes in farming systems, and abnormal natural climate, crop pests and diseases are showing trends of unpredictable occurrence cycles, increased variety of pests, larger affected areas, faster frequency, and longer control periods. Furthermore, the invasion of invasive pests has significantly increased the difficulty and cost of overall pest and disease management. Currently, pest monitoring mainly relies on manual identification, which is extremely difficult when identifying closely related species. Farmers are prone to errors in identification or delays, leading to direct economic losses. Utilizing modern technology for timely and accurate identification of crop pests and diseases, and preventing problems before they occur, is an urgent issue that needs to be addressed.

[0003] Agricultural professionals say that identifying pests in the field based solely on their biological characteristics is often insufficient; a comprehensive judgment is usually necessary, considering both the pest's damage patterns and the overall impact. This is because while closely related or similar pest species may share similar characteristics, their damage patterns can differ. Furthermore, this approach is inspired by multimodal thinking—learning a model from data across multiple modalities to address complex problems. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for identifying crop pests and diseases based on dual-dimensional fusion decision-making, effectively addressing the difficulty in identifying closely related and similar species of pests.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] Firstly, a method for identifying crop diseases and pests based on dual-dimensional fusion decision is provided, the method comprising the following steps:

[0009] The preprocessing step involves preprocessing the acquired images of pest biological features and images of pest damage symptoms to obtain corresponding three-dimensional arrays as input features.

[0010] In the feature extraction step, the corresponding feature map is extracted from the input features using a preset two-dimensional fusion decision model. The corresponding feature map is then flattened using pooling operations to obtain feature vectors for the pest biological feature image and the pest damage feature image, respectively.

[0011] The feature fusion step involves fusing the feature vectors of the pest biological feature image and the pest damage feature image to obtain the final output.

[0012] The model training and result output steps involve using a loss function to train the two-dimensional fusion decision model and obtaining the final output recognition result.

[0013] Preferably, the feature extraction step specifically includes:

[0014] The backbone network CNN is used to extract the corresponding feature map F from the input features. F F D ∈R H×W×C Here, H, W, and C represent the height, width, and number of channels of the feature map, respectively. The specific values ​​of the two feature maps need to be calculated based on the backbone network used. Pooling operations are used to flatten the feature map into a one-dimensional vector. As shown in the following formula:

[0015]

[0016] Where Pool(·) represents the pooling function, Flatten(·) represents the flattening function, and F is the feature map. This represents the feature vector of the i-th image in the k-th category. The elements are used to obtain the feature vectors of the biological feature image of the pest and the feature vector of the pest's harmful symptoms, respectively.

[0017] Preferably, the feature fusion step includes:

[0018] Generate bias terms Using the generated bias term as input, we obtain the output of the next layer, as shown in the following equation:

[0019]

[0020]

[0021] in, As a bias term, W i1 W' i1 These are the weights of the corresponding elements, and activate(·) is the activation function. 1O This is the output of the next layer.

[0022] Preferably, the feature fusion step further includes: obtaining the calculation result of the layer preceding the output layer. and bias The final output This can be expressed as:

[0023]

[0024]

[0025] Among them, W i1 W' i1 All are the weights of the output elements, and f(·) is the log-odds function.

[0026] Preferably, the formula for training the two-dimensional fusion decision model using the loss function is as follows:

[0027]

[0028]

[0029] Where, α t The quality adjustment factor 1-p is used to adjust the ratio between positive and negative sample losses. t γ The loss function is used to adjust the contribution of easily classified samples. When γ = 0, the loss function is equivalent to the log loss function; when γ > 0, the loss function can focus on samples that are difficult to classify.

[0030] Preferably, the final output recognition result is obtained as follows:

[0031] During training, a batch contains 24 sets of data. The data is input into a two-dimensional fusion decision model to calculate the difference between different samples and to feed back the error loss. Gradient optimization is then completed and the data is imported into a softmax classifier to obtain the final output recognition result.

[0032] Secondly, a crop disease and pest identification system based on dual-dimensional fusion decision is provided, the system comprising the following modules:

[0033] The preprocessing module is used to preprocess the acquired images of pest biological features and images of pest damage symptoms to obtain the corresponding three-dimensional array as input features.

[0034] The feature extraction module is used to extract the corresponding feature map from the input features using a preset two-dimensional fusion decision model, and flatten the corresponding feature map by pooling operation to obtain the feature vectors of the pest biological feature image and the pest damage feature image, respectively.

[0035] The feature fusion module is used to fuse the feature vectors of pest biological feature images and pest damage feature images to obtain the final output.

[0036] The model training and result output module is used to train the two-dimensional fusion decision model using a loss function to obtain the final output recognition result.

[0037] Preferably, the model training and result output module includes:

[0038] The model training module is used to train the two-dimensional fusion decision model using the focus loss function.

[0039] The results output module is used during the training process. A batch contains 24 sets of data. The data is input into the two-dimensional fusion decision model to calculate the difference between different samples and feed back the error loss. Gradient optimization is completed and the data is imported into the softmax classifier to obtain the final output recognition result.

[0040] Thirdly, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described.

[0041] Fourthly, a computing device is provided, comprising:

[0042] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described.

[0043] (III) Beneficial Effects

[0044] This invention presents a method and system for identifying crop pests and diseases based on dual-dimensional fusion decision-making. Utilizing multimodal thinking, it comprehensively judges the pest category from the biological characteristics and the characteristics of the pest's damage, effectively improving the identification accuracy and robustness. Furthermore, the backbone network can be replaced according to different specific pest identification tasks to achieve optimal performance. Therefore, this model has stronger generalization performance and is more scalable and practical. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the process of the crop disease and pest identification method based on dual-dimensional fusion decision of the present invention;

[0046] Figure 2 This is a block diagram of the dual-dimensional fusion decision model of the present invention;

[0047] Figure 3 This is a data distribution diagram of the pest-damage status dataset in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0049] Example

[0050] like Figure 1-2 As shown, one embodiment of the present invention provides a method for identifying crop diseases and pests based on dual-dimensional fusion decision, the method comprising the following steps:

[0051] The preprocessing step involves preprocessing the acquired images of pest biological features and images of pest damage characteristics to obtain corresponding three-dimensional arrays (e.g., a size of [229,229,3] as input features);

[0052] In the feature extraction step, the corresponding feature map is extracted from the input features using a preset two-dimensional fusion decision model. The corresponding feature map is then flattened using pooling operations to obtain feature vectors for the pest biological feature image and the pest damage feature image, respectively.

[0053] The feature fusion step involves fusing the feature vectors of the pest biological feature image and the pest damage feature image to obtain the final output.

[0054] The model training and result output steps involve using a loss function to train the two-dimensional fusion decision model and obtaining the final output recognition result.

[0055] Furthermore, the feature extraction steps specifically include:

[0056] The backbone network CNN is used to extract the corresponding feature map F from the input features. F F D ∈R H×W×C Here, H, W, and C represent the height, width, and number of channels of the feature map, respectively. The specific values ​​of the two feature maps need to be calculated based on the backbone network used. Pooling operations are used to flatten the feature map into a one-dimensional vector. As shown in the following formula:

[0057]

[0058] Where Pool(·) represents the pooling function, Flatten(·) represents the flattening function, and F is the feature map. This represents the feature vector of the i-th image in the k-th category. The elements are used to obtain the feature vectors of the biological feature image of the pest and the feature vector of the pest's harmful symptoms, respectively.

[0059] Furthermore, the feature fusion steps include:

[0060] Generate bias terms Using the generated bias term as input, we obtain the output of the next layer, as shown in the following equation:

[0061]

[0062]

[0063] in, As a bias term, W i1 W' i1 These are the weights of the corresponding elements, and activate(·) is the activation function. 1O This is the output of the next layer.

[0064] Furthermore, the feature fusion step also includes: obtaining the calculation results of the layer preceding the output layer. and bias and The final output This can be expressed as:

[0065]

[0066]

[0067] Among them, W i1 W' i1 All are the weights of the output elements, and f(·) is the log-odds function.

[0068] Furthermore, in addition to model improvements, to address the data imbalance problem in practical applications, this invention does not use the original cross-entropy loss function, but instead employs a focus loss function. This loss function is suitable for training models with a large difference in the number of positive and negative samples. Its core idea is to ignore easily classified samples, i.e., samples with high confidence in their classification results, to focus more on difficult-to-classify samples, thereby reducing the proportion of easily classified samples in the loss function and mitigating the impact of class imbalance. The formula for training the two-dimensional fusion decision model using the loss function is as follows:

[0069]

[0070]

[0071] Where, α t The quality adjustment factor 1-p is used to adjust the ratio between positive and negative sample losses. t γThe loss function is used to adjust the contribution of easily classified samples. When γ = 0, the loss function is equivalent to the log loss function; when γ > 0, the loss function can focus on samples that are difficult to classify.

[0072] Furthermore, the final output recognition result is as follows:

[0073] During training, a batch contains 24 sets of data. The data is input into a two-dimensional fusion decision model to calculate the difference between different samples and to feed back the error loss. Gradient optimization is then completed and the data is imported into a softmax classifier to obtain the final output recognition result.

[0074] Another embodiment of the present invention provides a crop disease and pest identification system based on dual-dimensional fusion decision, the system comprising the following modules:

[0075] The preprocessing module is used to preprocess the acquired images of pest biological features and images of pest damage symptoms to obtain the corresponding three-dimensional array as input features.

[0076] The feature extraction module is used to extract the corresponding feature map from the input features using a preset two-dimensional fusion decision model, and flatten the corresponding feature map by pooling operation to obtain the feature vectors of the pest biological feature image and the pest damage feature image, respectively.

[0077] The feature fusion module is used to fuse the feature vectors of pest biological feature images and pest damage feature images to obtain the final output.

[0078] The model training and result output module is used to train the two-dimensional fusion decision model using a loss function to obtain the final output recognition result.

[0079] Furthermore, the model training and result output module includes:

[0080] The model training module is used to train the two-dimensional fusion decision model using the focus loss function.

[0081] The results output module is used during the training process. A batch contains 24 sets of data. The data is input into the two-dimensional fusion decision model to calculate the difference between different samples and feed back the error loss. Gradient optimization is completed and the data is imported into the softmax classifier to obtain the final output recognition result.

[0082] Please see Figure 3The dataset used in this invention's experiments is a self-built pest-damage dataset, established in collaboration with the Chinese Academy of Agricultural Sciences. This dataset contains 27 types of pests, totaling 54,712 images. It focuses on nine major invasive pests in China in recent years, including their closely related and similar species. The overall distribution exhibits a natural long-tail distribution, accurately reflecting the data distribution, which also poses a challenge to model performance. Furthermore, in this dataset, 17 types of pest images have a one-to-one correspondence with damage images, while the remaining 10 types of images are all real photographs, but the pests and damage images do not correspond.

[0083] The hardware operating system for this experiment was Ubuntu 20.04, the programming language was Python 3.8, and the model was built entirely using the PyTorch 1.11.0 deep learning framework. During the overall training process, there were 80 iterations. Specifically, the dataset was divided into training, validation, and test sets in a 7:2:1 ratio. The experiment used Top-1 accuracy, Top-5 accuracy, recall, and precision as evaluation metrics and consisted of three parts: 1. Using images corresponding to 17 biological features and their corresponding damage patterns from the Pest-Damage17 dataset as the data source, different backbone networks were used to verify the effectiveness of the model framework. 2. Using all 27 image categories from the Pest-Damage27 dataset as the data source, different backbone networks were used to verify the generalization performance of the model framework. 3. For models with excellent performance, the training loss function was replaced to verify the effectiveness of the loss function optimization.

[0084] Table 1 shows the crop disease and pest identification results using different backbone networks on the Pest-Damage17 dataset.

[0085]

[0086]

[0087] Table 2 shows the crop disease and pest identification results for different backbone networks on the Pest-Damage27 dataset.

[0088]

[0089]

[0090] Table 3 shows the results of the effectiveness validation of the loss function on the Pest-Damage27 dataset.

[0091]

[0092] Table 1 compares the recognition results of three backbone networks (EfficientNetV2, ResNet101, InceptionV3) on the Pest-Damage17 dataset. The table shows that the two-dimensional fusion model performs better. This demonstrates that the two-dimensional fusion model can effectively extract the inline information between images, thereby improving the model's recognition accuracy.

[0093] Table 2 compares the recognition results of the three backbone networks (EfficientNetV2, ResNet101, InceptionV3) on the Pest-Damage27 dataset. This dataset contains 10 categories of pest images that do not correspond to images of the damage. However, the table shows that this does not affect the recognition accuracy of the two-dimensional fusion model. It also shows that the model can effectively extract the inline information between images. Even if the information does not conform to reality, it is enough to demonstrate the effectiveness of the model.

[0094] Table 3 shows the effectiveness validation results of the two-dimensional fusion model of the three backbone networks with different loss functions on the Pest-Damage27 dataset. It can be seen that due to the imbalance of data in the dataset, using FocalLoss as the loss function has a certain performance improvement compared with the traditional cross-entropy loss.

[0095] Embodiments of this application may be provided as methods or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application may be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0096] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0097] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0098] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0099] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for identifying crop diseases and pests based on dual-dimensional fusion decision-making, characterized in that, The method includes the following steps: The preprocessing step involves preprocessing the acquired images of pest biological features and images of pest damage symptoms to obtain corresponding three-dimensional arrays as input features. In the feature extraction step, the corresponding feature map is extracted from the input features using a preset two-dimensional fusion decision model. The corresponding feature map is then flattened using pooling operations to obtain feature vectors for the pest biological feature image and the pest damage feature image, respectively. The feature extraction steps specifically include: Each backbone network (CNN) is used to extract the corresponding feature maps from the input features. ,in, and These represent the height, width, and number of channels of the feature map, respectively. The specific values ​​of the two feature maps need to be calculated based on the backbone network used. Pooling operations are used to flatten the feature map into a one-dimensional vector. As shown in the following formula: in, Represents the pooling function. Represents the flattening function. For feature maps, Indicates the first The first of the categories Feature vectors of an image The elements are used to obtain the feature vectors of the pest biological feature image and the pest damage feature image, respectively. The feature fusion step involves fusing the feature vectors of the pest biological feature image and the pest damage feature image to obtain the final output. The feature fusion step includes: Generate bias terms The generated bias term is used as input to obtain the output of the next layer, as shown in the following equation: in, For bias terms, These are all weights of the corresponding elements. For activation function, This is the output of the next layer; Obtain the calculation results of the layer preceding the output layer. , and bias and ( The final output will be... Represented as: in, These are all weights of the output elements. It is a logarithmic probability function; The model training and result output steps involve training the two-dimensional fusion decision model using a loss function to obtain the final output recognition result. The formula for training the two-dimensional fusion decision model using the loss function is as follows: in, The quality control factor is used to adjust the ratio between positive and negative sample losses. Used to adjust the contribution of the loss function to easily classified samples, when When the loss function is equal to the logarithmic loss function, when... At this time, the loss function can focus on samples that are difficult to classify.

2. The method for identifying crop diseases and pests based on dual-dimensional fusion decision according to claim 1, characterized in that: The final output recognition result is specifically as follows: During training, a batch contains 24 sets of data. The data is input into a two-dimensional fusion decision model to calculate the difference between different samples and to feed back the error loss. Gradient optimization is then completed and the data is imported into a softmax classifier to obtain the final output recognition result.

3. A crop disease and pest identification system based on dual-dimensional fusion decision, characterized in that, The system is used to implement the crop pest and disease identification method based on dual-dimensional fusion decision as described in any one of claims 1-2, and the system includes the following modules: The preprocessing module is used to preprocess the acquired images of pest biological features and images of pest damage symptoms to obtain the corresponding three-dimensional array as input features. The feature extraction module is used to extract the corresponding feature map from the input features using a preset two-dimensional fusion decision model, and flatten the corresponding feature map by pooling operation to obtain the feature vectors of the pest biological feature image and the pest damage feature image, respectively. The feature fusion module is used to fuse the feature vectors of pest biological feature images and pest damage feature images to obtain the final output. The model training and result output module is used to train the two-dimensional fusion decision model using a loss function to obtain the final output recognition result.

4. The crop disease and pest identification system based on dual-dimensional fusion decision according to claim 3, characterized in that: The model training and result output module includes: The model training module is used to train the two-dimensional fusion decision model using the focus loss function. The results output module is used during training. A batch contains 24 sets of data. The data is input into the two-dimensional fusion decision model to calculate the difference between different samples and feed back the error loss. Gradient optimization is completed and the data is imported into the softmax classifier to obtain the final output recognition result.

5. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1-2.

6. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1-2.

Images