Method, device, equipment and medium for constructing plant disease and insect pest identification model

By combining lightweight networks and self-attention mechanisms to build an initial model and lightweight processing of the intermediate model, the problems of low plant pest recognition accuracy and high resource consumption in the prior art are solved, and the mobile device and embedded system are realized Efficient recognition effect on the device.

CN118521842BActive Publication Date: 2024-11-26NANJING AGRICULTURAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411002067.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-26
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

The existing plant pest identification technology has problems such as low recognition accuracy, limited feature extraction capabilities, and weak generalization capabilities, and it is difficult to operate efficiently on mobile devices and embedded devices, which affects the timely identification and diagnosis of plant pests and diseases.

Method used

By combining lightweight networks and self-attention mechanisms to build initial models, we can improve the efficient extraction of image features and long-distance dependency information capture capabilities, and lighten the training intermediate models to ensure that they are on mobile devices and embedded devices efficient operation reduces computing resource consumption.

Benefits of technology

It improves the recognition accuracy and recognition speed of plant pests, enhances the application capabilities of the model in resource-constrained environments, and achieves more accurate and efficient pest identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118521842B_ABST
    Figure CN118521842B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment and medium for constructing a plant pest and disease identification model. Among them, the method constructs an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; uses a sample data set to train the initial model to obtain an intermediate model; lightweight processes the intermediate model to obtain a plant pest and disease identification model, so as to identify pests and diseases on the plant image to be identified based on the plant pest and disease identification model. This technical solution, by combining a lightweight network and a self-attention mechanism to build an initial model, enables the model to efficiently extract image features while improving the ability to capture long-distance dependent information in the image; and then lightweight processes the trained intermediate model to ensure the efficient operation of the model on mobile devices and embedded devices, while reducing the consumption of computing resources, thereby improving the recognition accuracy and recognition speed of plant pests and diseases.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for constructing a plant disease and insect pest recognition model. Background Art

[0002] Studies have shown that global warming has increased the number of generations of pests, making pest control more difficult; and with the development of global economic integration, the probability of invasion of foreign pests has also increased. Although the use of chemical pesticides can control plant diseases and insect pests, due to the wide variety of diseases and insect pests, it is easy to misdiagnose by relying solely on the naked eye and experience, making it impossible to diagnose and treat plant diseases and insect pests in a timely and effective manner.

[0003] With the interactive development of agriculture and modern information technology, my country's agriculture has gradually become modernized, intelligent and digital. In recent years, the use of computer vision technology to identify plant diseases and insect pests has been widely used. For example, by calculating the leaf chromaticity value, a multi-layer BP neural network model is established to realize plant disease and insect pest identification. Although good recognition results have been achieved, the extracted features are not able to fully or well represent the plant disease and insect pest information, which has an adverse effect on the recognition accuracy. For another example, the identification of plant diseases and insect pests based on the Dise-EfficientNetV2 model can achieve miniaturization and rapid deployment of the model, but there are problems such as loss of recognition accuracy, limited feature extraction ability, and weak generalization ability.

[0004] Therefore, how to provide a technical solution that can efficiently and accurately identify plant diseases and insect pests is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The present application provides a method, apparatus, device and medium for constructing a plant disease and insect pest recognition model. An initial model is built by combining a lightweight network and a self-attention mechanism so that the model can efficiently extract image features while improving the ability to capture long-distance dependent information in the image; the trained intermediate model is then lightweighted to ensure efficient operation of the model on mobile devices and embedded devices, while reducing the consumption of computing resources, thereby improving the recognition accuracy and speed of plant diseases and insect pests.

[0006] According to one aspect of the present application, a method for constructing a plant disease and insect pest identification model is provided, the method comprising:

[0007] An initial model is constructed based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type;

[0008] The initial model is trained using a sample data set to obtain an intermediate model; wherein the sample data set includes at least one type of plant image to be trained;

[0009] The intermediate model is lightweighted to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be recognized based on the plant pest and disease recognition model.

[0010] According to another aspect of the present application, a device for constructing a plant disease and insect pest identification model is provided, the device comprising:

[0011] A model building module, used to build an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type;

[0012] A model training module, used to train the initial model using a sample data set to obtain an intermediate model; wherein the sample data set includes at least one type of plant images to be trained;

[0013] The lightweight processing module is used to perform lightweight processing on the intermediate model to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be recognized based on the plant pest and disease recognition model.

[0014] According to another aspect of the present application, an electronic device is provided, the device comprising:

[0015] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for constructing a plant disease and pest identification model as described in any embodiment of the present application.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for constructing a plant disease and pest identification model described in any embodiment of the present application when executed.

[0017] The technical solution provided by the present application is to construct an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; to train the initial model using a sample data set to obtain an intermediate model; to perform lightweight processing on the intermediate model to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be identified based on the plant pest and disease recognition model. This technical solution builds an initial model by combining a lightweight network and a self-attention mechanism, so that the model can efficiently extract image features while improving the ability to capture long-distance dependent information in the image; and then lightweight processing is performed on the trained intermediate model to ensure the efficient operation of the model on mobile devices and embedded devices, while reducing the consumption of computing resources, thereby improving the recognition accuracy and recognition speed of plant pests and diseases.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flow chart of a method for constructing a plant disease and insect pest identification model provided in Example 1 of the present application;

[0021] Figure 2 A schematic diagram of the architecture of a lightweight network provided in Example 2 of the present application;

[0022] Figure 3 A schematic diagram of the architecture of a mobile reverse bottleneck convolutional network provided in Example 2 of the present application;

[0023] Figure 4 A schematic diagram of the structure of a device for constructing a plant disease and insect pest identification model provided in Example 3 of the present application;

[0024] Figure 5 It is a structural schematic diagram of a device for implementing a method for constructing a plant disease and insect pest identification model in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1 This is a flow chart of a method for constructing a plant pest identification model provided in the first embodiment of the present application. This embodiment can be applied to the case of identifying pest types in tree or crop images. The method can be executed by a device for constructing a plant pest identification model. The device for constructing a plant pest identification model can be implemented in the form of hardware and / or software. The device for constructing a plant pest identification model can be configured in a device with data processing capabilities. Figure 1 As shown, the method includes:

[0029] S110, constructing an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier, wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type.

[0030] A lightweight network can be a neural network with a smaller model size and lower computational complexity under the premise of ensuring certain performance, and is used to quickly and efficiently extract features of the input image. For example, the purpose of lightweight can be achieved by using fewer convolution kernels, shallower network layers, or smaller convolution windows. Common lightweight networks include MobileNet, ShuffleNet, ESP Net, EfficientNet, and ChannelNets, etc. This embodiment of the present application does not limit this and can be determined according to actual needs.

[0031] The self-attention mechanism can adjust the importance of the feature by calculating the similarity between each position in the first feature and other positions, thereby capturing the correlation between different positions. In this application, the first feature can be first passed through three different fully connected layers or The convolutional layer converts into query, key and value vectors; then generates attention weights based on the query, key and value vectors; finally, outputs the second feature based on the attention weights.

[0032] Specifically, the query, key, and value vectors can be determined using the following formula:

[0033] ; ; ;

[0034] In the formula, represents the query vector, represents the key vector, represents a value vector, Represents the first feature, represents the weight matrix of the query vector, represents the weight matrix of the key vector, A weight matrix representing a vector of values.

[0035] Furthermore, the second feature can be determined by the following formula:

[0036] ;

[0037] In the formula, Represents the second feature, Represents the key vector The dimension of Indicates that the attention score is normalized.

[0038] The feature fusion layer can be used to fuse features from multiple sources or different types to obtain more comprehensive and representative information, and improve the performance and expression ability of the model. Common feature fusion methods include addition, multiplication, splicing, etc., which are not limited in the embodiments of this application and can be determined according to specific tasks and data characteristics.

[0039] Exemplarily, the first feature and the second feature may be combined by adding them together. Specifically, the third feature may be determined by the following formula:

[0040] ;

[0041] In the formula, Indicates the third feature.

[0042] The classifier can map the third feature to the corresponding category label by learning the differences between different categories. Common classifiers include support vector machines, decision trees, neural networks, etc., which are not limited in the embodiments of the present application and can be determined based on data characteristics, model performance requirements, model complexity, etc.

[0043] In this application, a model architecture can be built based on a lightweight network, a self-attention mechanism, a feature fusion layer, and a classifier, and initial values ​​are assigned to the model parameters to construct an initial model.

[0044] It should be noted that before the lightweight network processes the input image, the input image can be preprocessed to improve the accuracy of subsequent image feature extraction and image recognition. Common preprocessing methods include image enhancement, filtering and denoising, resizing, geometric transformation, grayscale and normalization.

[0045] S120, using a sample data set to train the initial model to obtain an intermediate model, wherein the sample data set includes at least one type of plant images to be trained.

[0046] The sample dataset may be a dataset consisting of a large number of different types of plant pest and disease images, including images of a variety of plants in healthy and infected states, which is used to train a model to identify and distinguish various pests and diseases.

[0047] In this application, the sample data set is used to train the initial model, and the model can compare the predicted results of the pest and disease types of the training plant images with the actual pest and disease types, and the model parameters are continuously adjusted according to the comparison results so that the model can learn the laws and patterns in the data. For example, the model parameters can be updated and trained using the stochastic gradient descent method, the mini-batch gradient descent method, etc.

[0048] Optionally, the initial model is trained using a sample data set to obtain an intermediate model, including: determining at least two sub-datasets from the sample data set; wherein the sub-datasets are plant image sets for the same pest and disease identification task, and the sub-datasets are divided into training sub-datasets and verification sub-datasets; the initial model is iteratively trained using the training sub-datasets to obtain meta-knowledge; based on the meta-knowledge, the performance of the trained model is verified using the verification sub-dataset until preset conditions are met to obtain an intermediate model.

[0049] The sub-dataset may be a plant image set for the same pest identification task. Specifically, the sub-dataset may be a data set of different pest types of the same plant, a data set of different development stages of the same pest type, a data set under different lighting conditions or shooting angles, or a data set of the same pest of different plants. This embodiment of the application does not limit this, and it can be determined according to the sample data set and actual needs.

[0050] In the present application, the initial model can be trained by using multiple related but different sub-datasets so that the model can learn how to extract effective information from plant disease and insect pest images and improve the generalization ability of the model.

[0051] Meta-knowledge can be the learning methods, strategies, patterns or rules about learning itself obtained during the iterative training of the initial model using training sub-datasets, which can help the model better learn and adapt to new tasks.

[0052] The validation sub-dataset can be used to verify the performance of the model meta-learning process. In the validation sub-dataset, the number of samples is generally small, but not too small. Therefore, a sample number threshold can be set for the validation sub-dataset. If it is less than the sample number threshold, continue to collect sample images for the validation sub-dataset. If it is not less than the sample number threshold, proceed to the next validation process.

[0053] Specifically, the validation sub-dataset can be input into the trained model to quickly fine-tune the model to optimize the response to new pest and disease identification tasks; and the model's performance parameters such as recognition accuracy and response time can be tested to evaluate its performance in practical applications; if the performance parameters do not meet expectations, the training sub-dataset can be used again to iteratively train the model until expectations are met and an intermediate model is output.

[0054] Optionally, based on the meta-knowledge, the trained model is trained using the verification sub-dataset until a preset condition is met to obtain an intermediate model, including: based on the meta-knowledge, the performance of the trained model is verified using the verification sub-dataset to determine the performance parameters of the model in the current training round; the model learning rate is adjusted according to the performance parameters, and the next training round is performed based on the adjusted model learning rate until the preset condition is met to obtain an intermediate model.

[0055] Performance parameters can be used to evaluate the prediction accuracy and prediction speed of the model, for example, they can be precision, recall, F1 value and response time.

[0056] The model learning rate can be used to determine the magnitude of the model parameter update during iterative training. If the model learning rate is too high, the model may not converge or oscillate, otherwise the training process will be very slow.

[0057] In this application, in order to avoid the above two situations, the model learning rate of the next training round is determined according to the performance parameters of the model in each training round, that is, the model learning rate is dynamically adjusted so that the model can converge more stably during the training process and improve the generalization ability of the model. For example, when the performance of the model on the validation sub-dataset stops improving, the model learning rate can be reduced to encourage the model to explore the parameter space more carefully and avoid overfitting.

[0058] S130, performing lightweight processing on the intermediate model to obtain a plant disease and insect pest recognition model, so as to perform disease and insect pest recognition on the plant image to be recognized based on the plant disease and insect pest recognition model.

[0059] Lightweight processing refers to a processing method that reduces the amount of computation and / or the number of parameters of a model, so that the model can be more easily applied in resource-constrained environments, such as mobile devices, embedded systems, etc., while maintaining certain performance. Common lightweight processing methods include pruning, quantization, and knowledge distillation. The embodiments of this application do not limit the lightweight processing methods, and one or more of these methods can be selected according to actual needs to perform lightweight processing on the intermediate model.

[0060] Optionally, the intermediate model is lightweighted to obtain a plant disease and pest identification model, including: pruning the intermediate model to obtain a pruned model; and performing knowledge distillation on the pruned model to obtain a plant disease and pest identification model.

[0061] Pruning refers to a method of reducing the size of a model by identifying and removing redundant parameters in the model. In a deep neural network, there are usually a large number of parameters, which may not be necessary during the training and reasoning process of the model, and some are even redundant. Therefore, by reducing the number of model parameters while ensuring model performance, the use of computing resources can be reduced, and the efficiency and reasoning speed of the model can be improved.

[0062] Specifically, the pruning process may include the following steps: 1) Parameter importance evaluation: First, the importance of the weights or neurons in the model is evaluated to determine the weights or neurons that contribute less to the model performance or are redundant. 2) Parameter deletion: Based on the evaluation results, weights or neurons that are considered to be unimportant or redundant are deleted. 3) Parameter pruning: After deleting the parameters, the model is retrained to correct the model performance after the parameters are deleted. In the process of retraining the model, the model learning rate can be adjusted or regularization methods can be used to avoid overfitting. 4) Fine-tuning and retraining: Fine-tune and retrain the pruned model to further improve the performance and ensure the stability of the model.

[0063] Knowledge distillation refers to transferring the knowledge of a large and complex model to another model with a simple structure through model compression.

[0064] Specifically, knowledge distillation may include the following steps: 1) Determine the teacher model and the student model: the teacher model is the intermediate model, and the student model may be a model with a similar structure to the intermediate model but relatively simple, which may be determined specifically according to the structure of the intermediate model. 2) Train the teacher model: Use a preset data set to train the intermediate model to achieve a higher performance level. 3) Transfer learning: Obtain the output of the intermediate model, which may generally be the output result of the Softmax function in the intermediate model or the output result of the Logits layer before the Softmax function. Such output results contain rich information about labels other than the correct label, which are usually called soft labels; Use a preset data set with soft labels and true labels to train the student model, that is, while learning the true label, the student model also tries to learn the soft label output of the intermediate model, thereby improving the behavior and generalization ability of the intermediate model. 4) Evaluate the performance of the student model on an independent test data set to confirm whether it has successfully learned the knowledge of the teacher model and has good generalization ability. If so, it is determined as a plant disease and insect pest recognition model.

[0065] In the present application, the intermediate model may be first subjected to full pruning, and then the model after full pruning may be subjected to knowledge distillation to obtain a plant pest and disease identification model. The intermediate model may also be subjected to multiple partial pruning and knowledge distillation processes to obtain a plant pest and disease identification model. The present application embodiment does not limit this, and it may be determined according to actual needs.

[0066] Optionally, the intermediate model is pruned to obtain a pruned model, including: determining at least one local model to be processed in the intermediate model; performing pruning on each of the local models to be processed in turn to obtain a local pruned model; accordingly, the pruned model is subjected to knowledge distillation to obtain a plant disease and pest identification model, including: performing knowledge distillation on each of the local pruned models in turn to obtain a plant disease and pest identification model.

[0067] The local model to be processed may be a specific local area in the intermediate model, such as certain layers or certain neuron connections.

[0068] In this application, each local model to be processed can be locally pruned in turn to obtain a local pruned model, and then each local pruned model that has completed the pruning process can be subjected to knowledge distillation in turn to obtain a plant pest and disease recognition model. For example, if it is found through analysis that some local models to be processed have obvious room for optimization, such as some layers or modules have too many parameters and contribute relatively little to the overall performance, the degree of pruning can be relatively improved.

[0069] Through local pruning and local knowledge distillation, the performance degradation caused by global pruning can be avoided, local redundancy can be reduced in a targeted manner, and the complexity and computational complexity of the model can be reduced to a certain extent, while maintaining the performance of the model in key areas as much as possible.

[0070] The embodiment of the present application provides a method for constructing a plant pest and disease recognition model, which is based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier to construct an initial model; a sample data set is used to train the initial model to obtain an intermediate model; the intermediate model is lightweight processed to obtain a plant pest and disease recognition model, so as to identify pests and diseases on the plant image to be identified based on the plant pest and disease recognition model. This technical solution, by combining a lightweight network and a self-attention mechanism to build an initial model, enables the model to efficiently extract image features while improving the ability to capture long-distance dependent information in the image; and then lightweight processes the trained intermediate model to ensure the efficient operation of the model on mobile devices and embedded devices, while reducing the consumption of computing resources, thereby improving the recognition accuracy and recognition speed of plant pests and diseases.

[0071] Embodiment 2

[0072] Figure 2 The schematic diagram of the architecture of a lightweight network provided in the second embodiment of the present application is shown in FIG. This embodiment is optimized based on the above embodiment, and specifically, the lightweight network is an improved network based on the EfficientNet architecture. Figure 2As shown, the lightweight network includes:

[0073] The initial convolutional layer 210 is used to extract features from the input image.

[0074] The initial convolution layer is the first layer of the lightweight network, which can be used to perform preliminary feature extraction on the input image. Specifically, a full convolution operation can be used to convolve the input image with the convolution kernel to obtain a set of feature maps that can capture low-level features in the input image.

[0075] The moving reverse bottleneck convolutional network 220 is used to extract features from the features obtained by the initial convolutional layer; wherein the moving reverse bottleneck convolutional network includes a standard moving reverse bottleneck convolutional layer and a fused moving reverse bottleneck convolutional layer.

[0076] The standard mobile inverted bottleneck convolution (MBConv) layer can be a structure that combines depthwise separable convolution and 1x1 convolution to efficiently extract features and reduce the amount of computation.

[0077] The fused mobile inverted bottleneck convolutionspecialized (Fused-MBConvS) layer can be used to simplify the traditional MBConv structure to reduce computational costs and improve sensitivity to plant pest and disease characteristics. In addition, based on the traditional Fused-MBConv, it adds special optimization of plant pest and disease image features, such as sensitivity to spots and rot.

[0078] Since plant pest and disease images usually contain complex backgrounds and irregular disease features, this places higher demands on the model's feature extraction capabilities. Therefore, the Fused-MBConvS layer is used to enhance the model's ability to recognize these specific features, so that the model can more accurately identify the type of pest and disease, thereby improving the model's overall recognition efficiency and accuracy, especially in low resolution or uneven lighting conditions.

[0079] It should be noted that the embodiment of the present application does not limit the number and order of MBConv and Fused-MBConvS, which can be set according to model requirements.

[0080] Optionally, the mobile reverse bottleneck convolutional network 220, Figure 3 A schematic diagram of the architecture of a mobile reverse bottleneck convolutional network provided in Example 2 of the present application is shown in FIG. Figure 3 As shown, the network includes:

[0081] The first fused mobile reverse bottleneck convolution layer 221 is used to extract features from the features obtained by the initial convolution layer.

[0082] This layer is used to perform preliminary feature extraction on the input image data in the early stage of the model, and to specifically optimize the basic features in plant disease images (such as spots, rot, etc.). By introducing specific image processing techniques such as edge detection and texture analysis, it helps to enhance the model's ability to recognize preliminary disease features while reducing computational costs.

[0083] The first standard moving reverse bottleneck convolution layer 222 is used to extract features obtained by the first fused moving reverse bottleneck convolution layer.

[0084] This layer is used to further refine the features passed down from the first fused mobile reverse bottleneck convolution layer 221. The combination of depthwise separable convolution and 1x1 convolution can effectively improve the computational efficiency while maintaining the richness of features.

[0085] The second fused moving reverse bottleneck convolution layer 223 is used to extract features from the features obtained by the first standard moving reverse bottleneck convolution layer.

[0086] This layer is used to further refine the features extracted from the first standard moving reverse bottleneck convolution layer 222, and enhance the model's sensitivity to slightly complex disease features. In terms of structure, a deeper feature recognition strategy can be adopted to enhance the recognition of specific disease features in the image, helping the model maintain high performance at a lower computational complexity.

[0087] The second standard moving reverse bottleneck convolution layer 224 is used to extract features from the features obtained by the second fused moving reverse bottleneck convolution layer.

[0088] This layer is used to further enhance the learning ability of the model during the feature extraction process, optimize feature representation, adapt to more complex image scenes, and increase the depth and complexity of the model to process and optimize more advanced image features.

[0089] The third fused moving backward bottleneck convolution layer 225 is used to extract features from the features obtained by the second standard moving backward bottleneck convolution layer.

[0090] This layer is used to complete advanced optimization and identification of plant pest and disease characteristics, so as to process more detailed pest and disease characteristics and ensure that the model can accurately identify pests and diseases even under conditions of uneven lighting or low resolution.

[0091] The third standard moving backward bottleneck convolution layer 226 is used to extract features from the features obtained by the third fused moving backward bottleneck convolution layer.

[0092] This layer is used to complete the final stage of feature extraction of the mobile reverse bottleneck convolutional network, preparing for accurate prediction of the fully connected layer and the output layer. By integrating and optimizing the features extracted and processed by the previous layers, it ensures that the output features are the most suitable for classification.

[0093] The beneficial effect of the above technical solution is that, through the alternating use of three Fused-MBConvS and three MBConv to gradually extract and optimize features, each layer focuses on extracting and refining features at different levels from the image. This progressive feature extraction is conducive to the model to respond more flexibly and effectively when facing different image challenges; secondly, the combination of the Fused-MBConvS layer and the MBConv layer enables the model to reduce irrelevant computational burden while maintaining high efficiency; thirdly, the Fused-MBConvS layer focuses on optimizing specific types of image features, such as plant disease and pest features, while the MBConv layer processes various image features more widely, which can ensure that the model has good performance in different environments; finally, through a hierarchical structural design, the model can achieve high-precision output at a lower complexity, and the design of each layer is intended to further refine the output of the previous layer. This coarse-to-fine layer-by-layer optimization process helps to improve the accuracy of the model's final prediction and the overall efficiency of the model.

[0094] The compression excitation layer 230 is used to compress and excite the features obtained by the mobile reverse bottleneck convolutional network.

[0095] The Squeeze-and-Excitation (SE block) layer can be used to adjust the relationship between channels and enhance the model's ability to learn important features.

[0096] The compression process can be performed by aggregating the spatial dimensions of the feature maps to produce channel descriptors, resulting in a globally distributed embedding, allowing the network to use information from the global receptive field.

[0097] The excitation processing can take the form of a self-gating mechanism that takes the embedding as input and produces a set of per-channel modulation weights that are applied to the feature map to generate the output of the SE block.

[0098] The average pooling layer 240 is used to perform average pooling processing on the features obtained by the compression excitation layer.

[0099] The average pooling layer can convert the feature map into a more concise form to reduce the number of parameters in the final fully connected layer. Specifically, each channel of the feature map can be averaged to obtain an output of a fixed size.

[0100] The fully connected layer 250 is used to integrate the features obtained by the average pooling layer to obtain the first feature.

[0101] In the fully connected layer, each neuron in the previous layer is connected to all neurons in this layer, which is used to integrate and classify the features extracted by the previous layer. Specifically, the features are mapped to different categories or output spaces by linearly combining and nonlinearly transforming the input features.

[0102] In summary, the first feature output by the lightweight network can be expressed by the following formula:

[0103] ;

[0104] In the formula, represents the input image, Represents the first feature, Represents the nonlinear activation function Sigmoid, , represents the convolution operation of a 1x1 convolution kernel, represents a nonlinear activation function, , represents a depth-wise separable convolution operation, Represents batch normalization.

[0105] The embodiment of the present application provides a lightweight network architecture, which includes an initial convolution layer, a mobile reverse bottleneck convolution network, a compression excitation layer, an average pooling layer, and a fully connected layer. By replacing the standard mobile reverse bottleneck convolution layer in the lightweight network EfficientNet with a mobile reverse bottleneck convolution network including a standard mobile reverse bottleneck convolution layer and a fused mobile reverse bottleneck convolution layer, the computational cost of the model is reduced, and the sensitivity to specific image features is effectively improved, thereby improving the model's recognition accuracy and recognition speed for plant diseases and insect pests.

[0106] Embodiment 3

[0107] Figure 4 This is a schematic diagram of a structure of a device for constructing a plant pest identification model provided in Example 3 of the present application. Figure 4 As shown, the device comprises:

[0108] A model building module 310 is used to build an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type;

[0109] A model training module 320 is used to train the initial model using a sample data set to obtain an intermediate model; wherein the sample data set includes at least one type of plant image to be trained;

[0110] The lightweight processing module 330 is used to perform lightweight processing on the intermediate model to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be recognized based on the plant pest and disease recognition model.

[0111] An embodiment of the present application provides a device for constructing a plant disease and pest identification model. The device builds an initial model by combining a lightweight network and a self-attention mechanism, so that the model can efficiently extract image features while improving the ability to capture long-distance dependent information in the image; and then lightweight processing is performed on the trained intermediate model to ensure the efficient operation of the model on mobile devices and embedded devices, while reducing the consumption of computing resources, thereby improving the recognition accuracy and speed of plant diseases and pests.

[0112] Furthermore, the lightweight network includes:

[0113] The initial convolutional layer is used to extract features from the input image;

[0114] A moving reverse bottleneck convolutional network, used for extracting features obtained by the initial convolutional layer; wherein the moving reverse bottleneck convolutional network includes a standard moving reverse bottleneck convolutional layer and a fused moving reverse bottleneck convolutional layer;

[0115] The compression excitation layer is used to compress and excite the features obtained by the mobile reverse bottleneck convolutional network;

[0116] The average pooling layer is used to perform average pooling processing on the features obtained from the compression excitation layer;

[0117] The fully connected layer is used to integrate the features obtained by the average pooling layer to obtain the first feature.

[0118] Furthermore, the mobile reverse bottleneck convolutional network includes:

[0119] The first fused mobile reverse bottleneck convolution layer is used to extract features from the features obtained by the initial convolution layer;

[0120] A first standard moving reverse bottleneck convolution layer, used for extracting features from the features obtained by the first fused moving reverse bottleneck convolution layer;

[0121] The second fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the first standard moving reverse bottleneck convolution layer;

[0122] A second standard moving reverse bottleneck convolution layer, used for extracting features obtained by the second fused moving reverse bottleneck convolution layer;

[0123] The third fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the second standard moving reverse bottleneck convolution layer;

[0124] The third standard moving reverse bottleneck convolution layer is used to extract features obtained by the third fused moving reverse bottleneck convolution layer.

[0125] Furthermore, the model training module 320 includes:

[0126] A data set acquisition unit, used to determine at least two sub-data sets from the sample data set; wherein the sub-data sets are plant image sets for the same pest and disease identification task, and the sub-data sets are divided into a training sub-data set and a verification sub-data set;

[0127] A model training unit, configured to iteratively train the initial model using the training sub-dataset to obtain meta-knowledge;

[0128] The model verification unit is used to verify the performance of the trained model based on the meta-knowledge using the verification sub-dataset until the preset conditions are met to obtain an intermediate model.

[0129] Furthermore, the model verification unit includes:

[0130] A performance parameter determination subunit, configured to perform performance verification on the trained model using the verification subdata set based on the meta-knowledge, and determine the performance parameters of the model in the current training round;

[0131] The learning rate adjustment subunit is used to adjust the model learning rate according to the performance parameter, and perform the next training round based on the adjusted model learning rate until the preset conditions are met to obtain an intermediate model.

[0132] Furthermore, the lightweight processing module 330 includes:

[0133] A model pruning unit, used for pruning the intermediate model to obtain a pruned model;

[0134] The model distillation unit is used to perform knowledge distillation processing on the pruning model to obtain a plant disease and insect pest recognition model.

[0135] Furthermore, the model pruning unit includes:

[0136] A local model to be processed determining subunit, used to determine at least one local model to be processed in the intermediate model;

[0137] A local pruning subunit is used to perform pruning processing on each of the local models to be processed in sequence to obtain a local pruning model;

[0138] Accordingly, the model distillation unit comprises:

[0139] The local distillation subunit is used to perform knowledge distillation processing on each of the local pruning models in turn to obtain a plant disease and pest recognition model.

[0140] A device for constructing a plant disease and pest identification model provided in an embodiment of the present application can execute a method for constructing a plant disease and pest identification model provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

[0141] Embodiment 4

[0142] Figure 5 A schematic diagram of a device 10 that can be used to implement an embodiment of the present application is shown. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0143] like Figure 5 As shown, the device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0144] A number of components in the device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0145] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for constructing a plant pest and disease identification model.

[0146] In some embodiments, the method for constructing a plant pest identification model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for constructing a plant pest identification model described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for constructing a plant pest identification model in any other appropriate manner (e.g., by means of firmware).

[0147] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor, the functions / operations specified in the flow charts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0149] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein may be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0151] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0152] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0153] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.

[0154] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for constructing a plant disease and insect pest identification model, characterized in that: The method comprises: An initial model is constructed based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type; The initial model is trained using a sample data set to obtain an intermediate model; wherein the sample data set includes at least one type of plant image to be trained; Lightweighting the intermediate model to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be recognized based on the plant pest and disease recognition model; Wherein, the lightweight network includes: The initial convolutional layer is used to extract features from the input image; A moving reverse bottleneck convolutional network, used for extracting features obtained by the initial convolutional layer; wherein the moving reverse bottleneck convolutional network includes a standard moving reverse bottleneck convolutional layer and a fused moving reverse bottleneck convolutional layer; The compression excitation layer is used to compress and excite the features obtained by the mobile reverse bottleneck convolutional network; The average pooling layer is used to perform average pooling processing on the features obtained from the compression excitation layer; The fully connected layer is used to integrate the features obtained by the average pooling layer to obtain the first feature; The intermediate model is lightweighted to obtain a plant pest and disease identification model, including: Determining at least one local model to be processed in the intermediate model; For each of the local models to be processed, pruning is performed in sequence to obtain a local pruned model; Performing knowledge distillation processing on each of the local pruning models in turn to obtain a plant disease and insect pest recognition model; Wherein, the mobile reverse bottleneck convolutional network comprises: The first fused mobile reverse bottleneck convolution layer is used to extract features from the features obtained by the initial convolution layer; A first standard moving reverse bottleneck convolution layer, used for extracting features from the features obtained by the first fused moving reverse bottleneck convolution layer; The second fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the first standard moving reverse bottleneck convolution layer; A second standard moving reverse bottleneck convolution layer, used for extracting features from the features obtained by the second fused moving reverse bottleneck convolution layer; The third fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the second standard moving reverse bottleneck convolution layer; The third standard moving reverse bottleneck convolution layer is used to extract features obtained by the third fused moving reverse bottleneck convolution layer.

2. The method according to claim 1, characterized in that The initial model is trained using a sample data set to obtain an intermediate model, including: Determine at least two sub-datasets from the sample data set; wherein the sub-datasets are plant image sets for the same pest and disease identification task, and the sub-datasets are divided into a training sub-dataset and a verification sub-dataset; Iteratively training the initial model using the training sub-dataset to obtain meta-knowledge; Based on the meta-knowledge, the performance of the trained model is verified using the verification sub-dataset until the preset conditions are met to obtain an intermediate model.

3. The method according to claim 2, characterized in that Based on the meta-knowledge, the trained model is trained using the verification sub-dataset until a preset condition is met to obtain an intermediate model, including: Based on the meta-knowledge, the performance of the trained model is verified using the verification sub-dataset to determine the performance parameters of the model in the current training round; The model learning rate is adjusted according to the performance parameter, and the next training round is performed based on the adjusted model learning rate until a preset condition is met to obtain an intermediate model.

4. A device for constructing a plant disease and insect pest identification model, characterized in that: The device comprises: A model building module, used to build an initial model based on a lightweight network, a self-attention mechanism, a feature fusion layer and a classifier; wherein the lightweight network is used to extract a first feature of an input image, the self-attention mechanism is used to perform correlation processing on the first feature to obtain a second feature, the feature fusion layer is used to fuse the first feature and the second feature to obtain a third feature, and the classifier is used to classify the third feature to obtain a pest type; A model training module, used to train the initial model using a sample data set to obtain an intermediate model; wherein the sample data set includes at least one type of plant images to be trained; A lightweight processing module, used for performing lightweight processing on the intermediate model to obtain a plant pest and disease recognition model, so as to perform pest and disease recognition on the plant image to be recognized based on the plant pest and disease recognition model; Wherein, the lightweight network includes: The initial convolutional layer is used to extract features from the input image; A moving reverse bottleneck convolutional network, used for extracting features obtained by the initial convolutional layer; wherein the moving reverse bottleneck convolutional network includes a standard moving reverse bottleneck convolutional layer and a fused moving reverse bottleneck convolutional layer; The compression excitation layer is used to compress and excite the features obtained by the mobile reverse bottleneck convolutional network; The average pooling layer is used to perform average pooling processing on the features obtained from the compression excitation layer; The fully connected layer is used to integrate the features obtained by the average pooling layer to obtain the first feature; Wherein, the lightweight processing module includes: A local model to be processed determining subunit, used to determine at least one local model to be processed in the intermediate model; A local pruning subunit is used to perform pruning processing on each of the local models to be processed in sequence to obtain a local pruning model; A local distillation subunit is used to perform knowledge distillation processing on each of the local pruning models in turn to obtain a plant disease and insect pest recognition model; Wherein, the mobile reverse bottleneck convolutional network comprises: The first fused mobile reverse bottleneck convolution layer is used to extract features from the features obtained by the initial convolution layer; A first standard moving reverse bottleneck convolution layer, used for extracting features from the features obtained by the first fused moving reverse bottleneck convolution layer; The second fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the first standard moving reverse bottleneck convolution layer; A second standard moving reverse bottleneck convolution layer, used for extracting features from the features obtained by the second fused moving reverse bottleneck convolution layer; The third fused moving reverse bottleneck convolution layer is used to extract features from the features obtained by the second standard moving reverse bottleneck convolution layer; The third standard moving reverse bottleneck convolution layer is used to extract features obtained by the third fused moving reverse bottleneck convolution layer.

5. An electronic device, characterized in that: The device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for constructing a plant disease and pest identification model according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for constructing a plant disease and insect pest identification model according to any one of claims 1 to 3 when executed.