A method for identifying leaf diseases and insect pests of Acer truncatum based on improved feature fusion

Abstract

The present invention relates to a method for identifying plant diseases and insect pests on leaves of Acer truncatum based on improved feature fusion, and belongs to the field of plant disease and insect pest detection. The method comprises the following steps: S1. Acquire Acer truncatum leaf images and construct a data set, wherein the data set comprises a training set and a test set; S2. Data preprocessing; S3. Construct a plant disease and insect pest identification model based on improved feature fusion, wherein the model comprises: a multi-attribute feature extraction module, a multi-attribute feature fusion module, and a disease and insect pest identification module; S4. Use images in the training set to train the plant disease and insect pest identification model based on improved feature fusion, and optimize it through a loss function to obtain a trained model; S5. The plant disease and insect pest identification result of the plant disease and insect pest is obtained by using the trained model for the plant disease and insect pest images in the test set. The present invention makes full use of multi-attribute fusion feature information and the prediction differences of each classifier to improve the accuracy of the prediction result.

Description

Technical Field

[0001] The invention belongs to the field of plant disease and insect pest detection, and in particular relates to a method for identifying disease and insect pests on Acer truncatum leaves based on improved feature fusion. Background Art

[0002] Acer truncatum is an adaptable ornamental plant that likes temperate climates, is cold-resistant and drought-resistant. It grows in North China, Northeast my country, Northwest China and other regions, and has a wide distribution range. Because its leaves turn golden or red in autumn, it is often used for landscaping or garden beautification, and has extremely high ornamental value. In addition, it also has certain economic value (Acer truncatum wood is tough and can be used to make furniture, handicrafts and building materials; gum can be used to make industrial adhesives) and medicinal value (Acer truncatum bark and leaves have the effects of clearing heat and detoxifying, reducing swelling and relieving pain; seeds are rich in neuraminic acid, which plays an important role in the development and repair of the brain and nervous system; neuraminic acid can help repair the skin barrier and delay aging). However, during the cultivation and planting process of Acer truncatum, its leaves are affected by a variety of diseases and pests, common diseases and pests include powdery mildew and brown spot disease. These diseases and pests usually cause abnormal phenotypes in maple leaves, such as abnormal leaf color (normal leaf color is green, golden yellow or red. Abnormal leaves may have chlorotic spots, purple-brown spots or white mold, etc.), abnormal leaf shape (normal leaves are palm-shaped, five-lobed or seven-lobed. Abnormal leaves may be curled, twisted and other deformities) and abnormal leaf edges (normal leaf edges are serrated. Abnormal leaf edges may appear dry and irregular). Therefore, the health of maple can be judged by identifying diseases and pests on its leaves. With the rapid development of computer vision and machine learning technology and their combination in the agricultural field, more and more intelligent recognition technologies are being used to identify diseases and pests on maple leaves, but these technologies are not detailed enough in feature extraction and utilization, the model generalization is poor, and the recognition accuracy and robustness need to be improved.

[0003] In order to overcome the shortcomings of existing intelligent recognition technology, the present invention aims to provide a method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion. The multi-attribute features of Acer truncatum leaves (leaf color features, leaf shape features, leaf edge features) are extracted by different methods, and the high-dimensional overall features of the leaves are extracted to calculate the similarity between each individual feature and the similarity as the basis for weight distribution of multi-attribute feature fusion, so as to obtain multi-attribute fusion information with high robustness and good completeness for the identification of diseases and insect pests on Acer truncatum leaves. Summary of the invention

[0004] In order to improve the accuracy of identifying diseases and insect pests on Acer truncatum leaves, the present invention provides a method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0006] In a first aspect, a method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion comprises the following steps:

[0007] S1. Obtaining an image of acer truncatum leaves and constructing a data set, the data set comprising a training set and a test set;

[0008] S2. Data preprocessing;

[0009] S3. Constructing a maple leaf pest and disease identification model based on improved feature fusion, the model comprising: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a pest and disease identification module;

[0010] S4. The maple leaf pest and disease recognition model based on improved feature fusion is trained using images in the training set, and optimized by a loss function to obtain a trained model;

[0011] S5. The images of Acer truncatum leaves in the test set are subjected to the trained model to obtain the results of Acer truncatum leaf disease and insect pest recognition.

[0012] Furthermore, the step S1 is specifically as follows:

[0013] S11. Data acquisition, using a smart camera to shoot images of Acer truncatum leaves to obtain original images of Acer truncatum leaves, wherein the Acer truncatum leaf images include normal leaf images, leaf images suffering from powdery mildew, leaf images suffering from brown spot disease, and leaf images suffering from lacquer spot disease, and the Acer truncatum leaf images are shot under different lighting and angles;

[0014] S12. Data labeling, manually labeling the original image of the maple leaf, labeling each image with corresponding disease information, and obtaining a labeled image, wherein the disease information includes normal, powdery mildew, brown spot, and lacquer spot;

[0015] S13. Dataset construction and division: construct the labeled images into an overall data set , and divided into training set and training set in a ratio of 3:1. and test set The training set and the test set both include normal leaf images and diseased leaf images.

[0016] Furthermore, in step S2, the Acer truncatum leaf image is converted into a JPEG format with a size of 224×224, and data enhancement is performed on the image to obtain a processed image, wherein the data enhancement includes rotation, horizontal flipping, vertical flipping and brightness adjustment.

[0017] Further, in step S4, the Acer truncatum leaf images in the training set are subjected to a multi-attribute feature extraction module to obtain leaf color features, leaf shape features, leaf edge features, and high-dimensional comprehensive features, specifically:

[0018] The multi-attribute feature extraction module includes a leaf color feature extractor, a leaf shape feature extractor, a leaf edge feature extractor and a high-dimensional comprehensive feature extraction module;

[0019] Leaf color feature extractor: Use the color moments method to extract the statistical characteristics of the color distribution in the maple leaves and obtain the leaf color features , the formula is as follows:

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] in, represents the multi-attribute feature extraction module, represents the mapping function corresponding to the leaf color feature extractor, represents the trainable parameters for extracting leaf color features, , , They represent the mean as the first-order moment, the variance as the second-order moment, and the skewness as the third-order moment. Indicates color channels: red R, green G, blue B, Indicates pixels in the color channel The value of is the total number of pixels, Indicates color channels The mean of Indicates color channels The variance of Indicates color channels The skewness of

[0025] Leaf shape feature extractor: The Canny edge detection algorithm is used to extract the leaf contour information of the maple tree. The leaf contour information of the maple tree includes the perimeter and area. The leaf contour shape factor is obtained through the perimeter and area. Then the perimeter, area and shape factor of the leaf contour are summarized into a shape feature vector to obtain the leaf shape feature. , the formula is as follows:

[0026] ,

[0027] ,

[0028] in, represents the blade profile shape factor, Indicates area, Indicates the circumference, represents the mapping function corresponding to the leaf shape feature extractor, represents the trainable parameters when extracting leaf shape features;

[0029] Leaf edge feature extractor: The second-order derivative of the Laplacian operator is used to extract the edge features of the maple leaves to obtain the leaf edge features. , the formula is as follows:

[0030] ,

[0031] in, represents the mapping function corresponding to the leaf edge feature extractor, represents the trainable parameters for extracting leaf edge features, represents the grayscale representation of the input image, and Represent the second-order derivatives of the image in the x-direction and the y-direction respectively;

[0032] High-dimensional comprehensive feature extraction module: The high-dimensional comprehensive feature extraction module is a ResNet neural network. The image of the maple leaf is subjected to feature extraction by the high-dimensional comprehensive feature extraction module to obtain high-dimensional comprehensive features. , the formula is as follows:

[0033] ,

[0034] ,

[0035] in, Represents the mapping function corresponding to the high-dimensional comprehensive feature extraction module, Represents the trainable parameters when extracting high-dimensional comprehensive features, represents the output of the hth layer, represents the image after h layers of convolution and pooling operations, represents the ReLU activation function, represents the operation in the residual network, represents the weight of the h-th layer, represents the bias of the hth layer.

[0036] Furthermore, in step S4, the leaf color feature, leaf shape feature and leaf edge feature are subjected to a multi-attribute feature fusion module to obtain a multi-attribute fusion feature , specifically:

[0037] Using the high-dimensional comprehensive features, calculate each single attribute feature The cosine similarity between the high-dimensional comprehensive features is normalized to obtain the weight coefficient for fusion , and then obtain the multi-attribute fusion features , the formula is as follows:

[0038] ,

[0039] ,

[0040] in, Representation characteristics The weight when performing multi-attribute fusion, Representation characteristics With features The cosine similarity between represents the bias parameter in the linear transformation process, Represents the activation function in the linear transformation process.

[0041] Furthermore, in step S4, the multi-attribute fusion features are passed through the pest and disease identification module to obtain the final prediction result, which is specifically:

[0042] In the pest and disease identification module, a decision fusion method is used to fuse the results of multiple classifiers through weighted voting. , n represents the number of classifiers, the classifiers include support vector machine SVM, multi-layer perceptron MLP, random forest, K-nearest neighbor K-NN, long short-term memory network LSTM, etc., to obtain the final prediction result The formula is as follows:

[0043] ,

[0044] in, Indicates The mapping function of a classifier, Indicates The trainable parameters of the classifier.

[0045] Furthermore, the loss function of step S4 includes multi-attribute feature extraction loss and classification loss, specifically:

[0046] Multi-attribute feature extraction loss , the calculation formula is as follows:

[0047] ,

[0048] ,

[0049] ,

[0050] ,

[0051] in, , , They represent the weight coefficients of leaf color features, leaf shape features, and leaf edge features, respectively. , and Respectively represent the loss of extracting leaf color features, leaf shape features and leaf edge features, Indicates The leaf color characteristics of the samples, Indicates the true color feature label of the sample; Indicates The leaf shape characteristics of the samples, Indicates the true shape feature label of the sample; Indicates The true edge label of pixels, Indicates The predicted edge probability of pixels;

[0052] Classification Loss , the calculation formula is as follows:

[0053] ,

[0054] ,

[0055] in, They represent the trainable parameters of the initial multi-attribute feature extraction module, the multi-attribute feature fusion module, the classifiers of the pest and disease identification module, and the entire pest and disease identification module. Indicates The true data labels of samples, Indicates The classifier in The predicted value on samples, Indicates The weight coefficient of the classifier, represents the number of classifiers, represents the number of samples, Represents the optimized trainable parameters of the multi-attribute feature extraction module, multi-attribute feature fusion module, each classifier of the pest and disease identification module, and the entire pest and disease identification module.

[0056] In the second aspect, a device for identifying diseases and insect pests of Acer truncatum leaves based on improved feature fusion includes:

[0057] Data acquisition unit: used to acquire the image of Acer truncatum leaves and construct a data set;

[0058] Preprocessing unit: used for preprocessing the image data of Acer truncatum leaves;

[0059] Model building unit: used to build a maple leaf disease and insect pest recognition model based on improved feature fusion, the model includes: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a disease and insect pest recognition module;

[0060] Model training unit: used for training the Acer truncatum leaf disease and insect pest recognition model based on improved feature fusion by using images in the training set, and optimizing it by loss function to obtain a trained model;

[0061] Pest and disease identification unit: used to use the trained model to identify pests and diseases on the Acer truncatum leaf images in the test set and obtain pest and disease identification results.

[0062] In a third aspect, an electronic device includes: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a method for identifying maple leaf diseases and pests based on improved feature fusion is performed.

[0063] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, executes a method for identifying maple leaf diseases and insect pests based on improved feature fusion.

[0064] The advantages of the present invention are:

[0065] The present invention adopts a dual fusion mechanism of improved feature fusion + decision fusion based on cosine similarity, which more reasonably allocates weights to each attribute feature, so that the fused feature has higher quality (contains more comprehensive information); it makes full use of multi-attribute fusion feature information and prediction differences of each classifier, so that the prediction results are significantly improved. In the feature fusion stage, considering the differences in the importance of different features, the existing weight distribution methods are either uniformly distributed or distributed using automatic training such as neural networks, and the distribution method is not reasonable enough. Therefore, a more reasonable weight distribution method based on cosine similarity calculation is proposed in this module, which makes full use of the high-dimensional comprehensive features extracted above, calculates the cosine similarity (cos) between each single attribute feature and the high-dimensional comprehensive feature and normalizes it, and uses it as the weight coefficient of the feature during fusion. The fusion features obtained in this way contain more comprehensive leaf information. Different classifiers have different focuses on the use of fusion features during classification, and the prediction results obtained are also different. Therefore, in order to make full use of the fused multi-attribute features, decision fusion is used in the pest and disease recognition module to fuse the results of multiple classifiers (including support vector machine SVM, multi-layer perceptron MLP, random forest, long short-term memory network LSTM, etc.) through a weighted voting mechanism. , and get the final prediction result ,In the weighted voting stage, the weights are allocated according to the accuracy of each classifier. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0067] Figure 1 is a flow chart of the method of the present invention;

[0068] Figure 2 The maple leaf image in the data set of the present invention;

[0069] Figure 3 This is a comparison chart of the results of different number of classifiers of the method of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Example 1

[0072] In this embodiment, Figure 1As shown, the present invention provides a method for identifying maple leaf diseases and pests based on improved feature fusion, and the specific steps include: obtaining maple leaf images and constructing a data set, the data set including a training set and a test set; data preprocessing; constructing a maple leaf disease and pest identification model based on improved feature fusion, the model including: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a disease and pest identification module; using images in the training set to train the maple leaf disease and pest identification model based on improved feature fusion, and optimizing it through a loss function to obtain a trained model; the maple leaf images in the test set are subjected to the trained model to obtain maple leaf disease and pest identification results.

[0073] Specifically, data acquisition, such as Figure 2 The figure shows the maple leaf image obtained by the present invention. We use a smart camera to shoot the maple leaf image to obtain the original maple leaf image, which includes a normal leaf image, a leaf image with powdery mildew, a leaf image with brown spot disease, and a leaf image with paint spot disease. The maple leaf image is taken under different lighting and angles; data annotation, manually annotating the original maple leaf image, annotating each image with corresponding disease information, and obtaining an annotated image, wherein the disease information includes normal, powdery mildew, brown spot disease, and paint spot disease; data set construction and division, constructing the annotated images into an overall data set , and divided into training set and training set in a ratio of 3:1. and test set The training set and the test set both include normal leaf images and diseased leaf images.

[0074] Specifically, the maple leaf image is converted into a JPEG format with a size of 224×224, and data enhancement is performed on the image to obtain a processed image, wherein the data enhancement includes rotation, horizontal flipping, vertical flipping and brightness adjustment.

[0075] Specifically, the Acer truncatum leaf images in the training set are subjected to a multi-attribute feature extraction module to obtain leaf color features, leaf shape features, leaf edge features, and high-dimensional comprehensive features, including:

[0076] The multi-attribute feature extraction module includes a leaf color feature extractor, a leaf shape feature extractor, a leaf edge feature extractor and a high-dimensional comprehensive feature extraction module;

[0077] Leaf color feature extractor: Use the color moments method to extract the statistical characteristics of the color distribution in the maple leaves and obtain the leaf color features , the formula is as follows:

[0078] ,

[0079] ,

[0080] ,

[0081] ,

[0082] in, represents the multi-attribute feature extraction module, represents the mapping function corresponding to the leaf color feature extractor, represents the trainable parameters for extracting leaf color features, , , They represent the mean as the first-order moment, the variance as the second-order moment, and the skewness as the third-order moment. Indicates color channels: red R, green G, blue B, Indicates pixels in the color channel The value of is the total number of pixels, Indicates color channels The mean of Indicates color channels The variance of Indicates color channels The skewness of

[0083] Leaf shape feature extractor: The Canny edge detection algorithm is used to extract the leaf contour information of the maple tree. The leaf contour information of the maple tree includes the perimeter and area. The leaf contour shape factor is obtained through the perimeter and area. Then the perimeter, area and shape factor of the leaf contour are summarized into a shape feature vector to obtain the leaf shape feature. , the formula is as follows:

[0084] ,

[0085] ,

[0086] in, represents the blade profile shape factor, Indicates area, Indicates the circumference, represents the mapping function corresponding to the leaf shape feature extractor, represents the trainable parameters when extracting leaf shape features;

[0087] Leaf edge feature extractor: The second-order derivative of the Laplacian operator Laplacian is used to extract the edge features of the maple leaves and obtain the leaf edge features. , the formula is as follows:

[0088] ,

[0089] in, represents the mapping function corresponding to the leaf edge feature extractor, represents the trainable parameters for extracting leaf edge features, represents the grayscale representation of the input image, and Represent the second-order derivatives of the image in the x-direction and the y-direction respectively;

[0090] High-dimensional comprehensive feature extraction module: The high-dimensional comprehensive feature extraction module is a ResNet neural network. The image of the maple leaf is subjected to feature extraction by the high-dimensional comprehensive feature extraction module to obtain high-dimensional comprehensive features. , the formula is as follows:

[0091] ,

[0092] ,

[0093] in, Represents the mapping function corresponding to the high-dimensional comprehensive feature extraction module, Represents the trainable parameters when extracting high-dimensional comprehensive features, represents the output of the hth layer, represents the image after h layers of convolution and pooling operations, represents the ReLU activation function, represents the operation in the residual network, represents the weight of the hth layer, represents the bias of the hth layer.

[0094] Specifically, the leaf color feature, leaf shape feature and leaf edge feature are subjected to a multi-attribute feature fusion module to obtain a multi-attribute fusion feature When performing feature fusion, we take into account the differences in the importance of different features, and the existing weight distribution methods either use uniform distribution or use neural network automatic distribution, which is extremely unreasonable. Therefore, in this module, we make full use of the high-dimensional comprehensive features extracted above to calculate each single attribute feature. The cosine similarity between the high-dimensional comprehensive features is normalized to obtain the weight coefficient for fusion , and then obtain the multi-attribute fusion features , the formula is as follows:

[0095] ,

[0096] ,

[0097] in, Representation characteristics The weight when performing multi-attribute fusion, Representation characteristics With features The cosine similarity between represents the bias parameter in the linear transformation process, Represents the activation function in the linear transformation process.

[0098] Specifically, the multi-attribute fusion features are used in the pest and disease identification module to obtain the final prediction results. Considering that the fusion features contain more comprehensive leaf information, and different classifiers have different focuses on the use of fusion features during classification, the conclusions obtained are also different. Therefore, in order to make full use of the fused multi-attribute features, a decision fusion method is adopted in the pest and disease identification module to fuse the results of multiple classifiers through weighted voting. , n represents the number of classifiers, the classifiers include support vector machine SVM, multi-layer perceptron MLP, random forest, K-nearest neighbor K-NN, long short-term memory network LSTM, etc., to obtain the final prediction result The formula is as follows:

[0099] ,

[0100] in, Indicates The mapping function of a classifier, Indicates The trainable parameters of the classifier.

[0101] Specifically, the loss function includes multi-attribute feature extraction loss and classification loss:

[0102] Multi-attribute feature extraction loss , the calculation formula is as follows:

[0103] ,

[0104] ,

[0105] ,

[0106] ,

[0107] in, , , They represent the weight coefficients of leaf color features, leaf shape features, and leaf edge features, respectively. , and Respectively represent the loss of extracting leaf color features, leaf shape features and leaf edge features, Indicates The leaf color characteristics of the samples, Indicates the true color feature label of the sample; Indicates The leaf shape characteristics of the samples, Indicates the true shape feature label of the sample; Indicates The true edge label of pixels, Indicates The predicted edge probability of pixels;

[0108] Classification Loss , the calculation formula is as follows:

[0109] ,

[0110] ,

[0111] in, They represent the trainable parameters of the initial multi-attribute feature extraction module, the multi-attribute feature fusion module, the classifiers of the pest and disease identification module, and the entire pest and disease identification module. Indicates The true data labels of samples, Indicates The classifier in The predicted value on samples, Indicates The weight coefficient of the classifier, represents the number of classifiers, represents the number of samples, Represents the optimized trainable parameters of the multi-attribute feature extraction module, multi-attribute feature fusion module, each classifier of the pest and disease identification module, and the entire pest and disease identification module.

[0112] Example 2

[0113] In this example, we conducted an experimental comparison, and the parameters in the verification process were set as follows: the initial learning rate was set to 10 -3, the learning rate decay cycle is set to 0.1, the training batch size is set to 64, and the Adam optimizer is used for 100 training cycles. Table 1 shows the experimental comparison data between the method of the present invention and the existing method, wherein RF represents the random forest algorithm, SVM represents the support vector machine algorithm, MLP represents the multi-layer perceptron, LSTM represents the long short-term memory network, CNN represents the ResNet convolutional neural network, and PFRA represents the maple leaf disease and insect pest identification method based on improved feature fusion proposed by the present invention. It can be seen from the table that compared with the original method, the maple leaf disease and insect pest identification method based on improved feature fusion proposed by the present invention has achieved significant improvements in accuracy, recall rate, F1 value and other aspects, which illustrates the effectiveness of the method of the present invention.

[0114] Table 1 Comparison of experimental data between the method of the present invention and the prior art

[0115]

[0116] In this embodiment, Figure 3 As shown in the figure, the effect of the number of classifiers on the accuracy of the maple leaf pest identification module is shown. The horizontal axis represents the number of classifiers and the vertical axis represents the accuracy. It can be seen that with the increase of the number of classifiers in the decision fusion stage, the accuracy continues to improve, and the effect is best when the number of classifiers is 5.

[0117] In this embodiment, we also conducted an ablation experiment. Table 2 shows the results of the ablation experiment of the present invention, where w\o cos represents a feature fusion module without cosine similarity, w\o df represents a feature fusion module without a decision fusion module, and w\ cos&df represents a feature fusion module with cosine similarity and a decision fusion module. By comparing w\o cos and w\ cos&df, it can be seen that the detection accuracy is significantly reduced when feature fusion based on cosine similarity is not performed. This is because the quality of the high-dimensional comprehensive features extracted by the neural network is poor; by comparing w\o df and w\ cos&df, it can be found that the result of a single classifier for recognition is far less accurate than that of multiple classifiers for decision fusion. This is because it is difficult for a single classifier to make full use of the fused multi-attribute information. The ablation experiment fully proves the effectiveness of the method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion proposed by this technology.

[0118] Table 2 Ablation experimental results of the method of the present invention

[0119]

[0120] Example 3

[0121] This embodiment provides a device for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion, comprising:

[0122] Data acquisition unit: used to acquire the image of Acer truncatum leaves and construct a data set;

[0123] Preprocessing unit: used for preprocessing the image data of Acer truncatum leaves;

[0124] Model building unit: used to build a maple leaf disease and insect pest recognition model based on improved feature fusion, the model includes: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a disease and insect pest recognition module;

[0125] Model training unit: used for training the Acer truncatum leaf disease and insect pest recognition model based on improved feature fusion by using images in the training set, and optimizing it by loss function to obtain a trained model;

[0126] Pest and disease identification unit: used to use the trained model to identify pests and diseases on the Acer truncatum leaf images in the test set and obtain pest and disease identification results.

[0127] Example 4

[0128] This embodiment provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, a method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion is executed;

[0129] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] Example 5

[0131] The present embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for identifying diseases and insect pests on Acer truncatum leaves based on improved feature fusion is executed.

[0132] Storage media include: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store programs.

[0133] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying leaf diseases and insect pests of Acer truncatum based on improved feature fusion, characterized in that: The following steps are involved: S1. Obtaining an image of acer truncatum leaves and constructing a data set, the data set comprising a training set and a test set; S2. Data preprocessing; S3. Constructing a maple leaf pest and disease identification model based on improved feature fusion, the model comprising: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a pest and disease identification module; S4. The maple leaf pest and disease recognition model based on improved feature fusion is trained using images in the training set, and optimized by a loss function to obtain a trained model; The specific process is as follows: The image of the maple leaves in the training set is subjected to a multi-attribute feature extraction module to obtain leaf color features, leaf shape features, leaf edge features and high-dimensional comprehensive features; the multi-attribute feature extraction module includes a leaf color feature extractor, a leaf shape feature extractor, a leaf edge feature extractor and a high-dimensional comprehensive feature extraction module, specifically: Leaf color feature extractor: Use the color moments method to extract the statistical characteristics of the color distribution in the maple leaves and obtain the leaf color features , the formula is as follows: ， ， ， ， in, represents the multi-attribute feature extraction module, represents the mapping function corresponding to the leaf color feature extractor, represents the trainable parameters for extracting leaf color features, , , They represent the mean as the first-order moment, the variance as the second-order moment, and the skewness as the third-order moment. Indicates color channels: red R, green G, blue B, Indicates pixels in the color channel The value of is the total number of pixels, Indicates color channels The mean of Indicates color channels The variance of Indicates color channels The skewness of Leaf shape feature extractor: The Canny edge detection algorithm is used to extract the leaf contour information of the maple tree. The leaf contour information of the maple tree includes the perimeter and area. The leaf contour shape factor is obtained through the perimeter and area. Then the perimeter, area and shape factor of the leaf contour are summarized into a shape feature vector to obtain the leaf shape feature. , the formula is as follows: ， ， in, represents the blade profile shape factor, Indicates area, Indicates the circumference, represents the mapping function corresponding to the leaf shape feature extractor, represents the trainable parameters when extracting leaf shape features; Leaf edge feature extractor: The second-order derivative of the Laplacian operator is used to extract the edge features of the maple leaves to obtain the leaf edge features. , the formula is as follows: ， in, represents the mapping function corresponding to the leaf edge feature extractor, represents the trainable parameters for extracting leaf edge features, represents the grayscale representation of the input image, and Represent the second-order derivatives of the image in the x-direction and the y-direction respectively; High-dimensional comprehensive feature extraction module: The high-dimensional comprehensive feature extraction module is a ResNet neural network. The image of the maple leaf is subjected to feature extraction by the high-dimensional comprehensive feature extraction module to obtain high-dimensional comprehensive features. , the formula is as follows: ， ， in, Represents the mapping function corresponding to the high-dimensional comprehensive feature extraction module, Represents the trainable parameters when extracting high-dimensional comprehensive features, represents the output of the hth layer, represents the image after h layers of convolution and pooling operations, represents the ReLU activation function, represents the operation in the residual network, represents the weight of the h-th layer, represents the bias of the hth layer; The leaf color features, leaf shape features and leaf edge features are fused by a multi-attribute feature fusion module to obtain a multi-attribute fusion feature. , specifically: Using the high-dimensional comprehensive features, calculate each single attribute feature The cosine similarity between the high-dimensional comprehensive features is normalized to obtain the weight coefficient for fusion , and then obtain the multi-attribute fusion feature , the formula is as follows: ， ， in, Representation characteristics The weight when performing multi-attribute fusion, Representation characteristics With features The cosine similarity between represents the bias parameter in the linear transformation process, Represents the activation function in the linear transformation process; The multi-attribute fusion features are passed through the pest and disease identification module to obtain the final prediction result, which is specifically: In the pest and disease identification module, a decision fusion method is used to fuse the results of multiple classifiers through weighted voting. , n represents the number of classifiers, the classifiers include support vector machine SVM, multi-layer perceptron MLP, random forest, K-nearest neighbor K-NN, long short-term memory network LSTM, etc., to obtain the final prediction result , the formula is as follows: ， in, Indicates The mapping function of a classifier, Indicates The trainable parameters of the classifier; The loss function includes multi-attribute feature extraction loss and classification loss, specifically: Multi-attribute feature extraction loss , the calculation formula is as follows: ， ， ， ， in, , , They represent the weight coefficients of leaf color features, leaf shape features, and leaf edge features, respectively. , and Respectively represent the loss of extracting leaf color features, leaf shape features and leaf edge features, Indicates The leaf color characteristics of the samples, Indicates the true color feature label of the sample; Indicates The leaf shape characteristics of the samples, Indicates the true shape feature label of the sample; Indicates The true edge label of pixels, Indicates The predicted edge probability of pixels; Classification Loss , the calculation formula is as follows: ， ， in, They represent the trainable parameters of the initial multi-attribute feature extraction module, the multi-attribute feature fusion module, the classifiers of the pest and disease identification module, and the entire pest and disease identification module. Indicates The true data labels of samples, Indicates The classifier in The predicted value on samples, Indicates The weight coefficient of the classifier, represents the number of classifiers, represents the number of samples, Represents the optimized trainable parameters of the multi-attribute feature extraction module, multi-attribute feature fusion module, each classifier of the pest and disease identification module, and the entire pest and disease identification module; S5. The images of Acer truncatum leaves in the test set are subjected to the trained model to obtain the results of Acer truncatum leaf disease and insect pest recognition.

2. The method for identifying leaf diseases and insect pests of Acer truncatum based on improved feature fusion according to claim 1, characterized in that: The step S1 is specifically as follows: S11. Data acquisition, using a smart camera to shoot images of Acer truncatum leaves to obtain original images of Acer truncatum leaves, wherein the Acer truncatum leaf images include normal leaf images, leaf images suffering from powdery mildew, leaf images suffering from brown spot disease, and leaf images suffering from lacquer spot disease, and the Acer truncatum leaf images are shot under different lighting and angles; S12. Data labeling, manually labeling the original image of the maple leaf, labeling each image with corresponding disease information, and obtaining a labeled image, wherein the disease information includes normal, powdery mildew, brown spot, and lacquer spot; S13. Dataset construction and division: construct the labeled images into an overall data set , and divided into training sets at a ratio of 3:1 and test set The training set and the test set both include normal leaf images and diseased leaf images.

3. The method for identifying leaf diseases and insect pests of Acer truncatum based on improved feature fusion according to claim 1, characterized in that: In the step S2, the Acer truncatum leaf image is converted into a JPEG format with a size of 224×224, and data enhancement is performed on the image to obtain a processed image, wherein the data enhancement includes rotation, horizontal flipping, vertical flipping and brightness adjustment.

4. A device using the method for identifying plant diseases and insect pests on Acer truncatum leaves based on improved feature fusion as described in claims 1-3, characterized in that: include: Data acquisition unit: used to acquire the image of Acer truncatum leaves and construct a data set; Preprocessing unit: used for preprocessing the image data of Acer truncatum leaves; Model building unit: used to build a maple leaf disease and insect pest recognition model based on improved feature fusion, the model includes: a multi-attribute feature extraction module, a multi-attribute feature fusion module and a disease and insect pest recognition module; Model training unit: used for training the Acer truncatum leaf disease and insect pest recognition model based on improved feature fusion by using images in the training set, and optimizing it by loss function to obtain a trained model; Pest and disease identification unit: used to use the trained model to identify pests and diseases on the maple leaf images in the test set and obtain the pest and disease identification results.

5. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, a method for identifying maple leaf diseases and insect pests based on improved feature fusion as described in any one of claims 1 to 3 is executed.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying plant diseases and insect pests on leaves of Acer truncatum based on improved feature fusion according to any one of claims 1 to 3 is executed.

Images