Municipal greening nursery stock pest and disease identification method and system based on Internet of Things

Abstract

The invention relates to the technical field of intelligent recognition, and particularly discloses a municipal greening nursery stock disease and pest recognition method and system based on the Internet of Things, and the method comprises the steps: carrying out the leaf surface state feature extraction of a nursery stock leaf image collected by a camera based on a computer vision technology and an artificial intelligence technology; and leaf surface semantic feature information focused on the pest and disease damage area is obtained, so that automatic identification of pest and disease damage types is realized. Therefore, manpower resources can be saved, and the efficiency and accuracy of seedling disease and pest detection are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent identification technology, and more specifically, to a municipal greening seedling pest and disease identification method and system based on the Internet of Things. Background Art

[0002] Municipal greening plays an important role in the urban environment. It not only beautifies the urban landscape, but also provides ecosystem services such as air purification, water source protection and biodiversity maintenance. As an important part of the urban environment, the prevention and control of seedling pests and diseases is of great significance to maintaining the ecological balance of the city. In urban greening, pests and diseases may cause tree death and poor growth, affect the landscape effect, and may even affect the health of citizens. Therefore, pest and disease control is one of the important tasks in urban greening maintenance.

[0003] However, the traditional method of pest control for municipal greening seedlings mainly relies on regular manual inspections of the health of greening plants. This inspection method requires a lot of manpower, is not only inefficient, but also may not be able to detect early signs of pests and diseases in time due to the periodicity and limitations of manual inspections.

[0004] Therefore, a method and system for identifying pests and diseases of municipal greening seedlings based on the Internet of Things is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a municipal greening seedling pest identification method and system based on the Internet of Things, which extracts leaf surface state features from seedling leaf images captured by a camera based on computer vision technology and artificial intelligence technology, and obtains leaf surface semantic feature information focused on the pest area, thereby realizing automatic identification of pest types. In this way, human resources can be saved and the efficiency and accuracy of seedling pest detection can be improved.

[0006] Accordingly, according to one aspect of the present application, a method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things is provided, which includes:

[0007] Acquire the seedling leaf image collected by the camera;

[0008] Extracting leaf surface state features from the seedling leaf image to obtain a pest-focused seedling leaf surface state feature map;

[0009] Extracting leaf surface semantic features of the pest-focused seedling leaf surface state feature map to obtain a pest-focused seedling leaf surface semantic feature map;

[0010] Performing feature fusion on the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature map;

[0011] Based on the semantically transferred pest and disease focused multi-scale feature map of the seedling leaf surface, the type of pest and disease is determined.

[0012] In the above-mentioned municipal greening seedling disease and insect pest identification method based on the Internet of Things, the seedling leaf image is subjected to leaf surface state feature extraction to obtain a disease and insect pest focused seedling leaf surface state feature map, including: passing the seedling leaf image through a leaf surface state feature extractor based on a first convolutional neural network model to obtain a seedling leaf surface state feature map; passing the seedling leaf surface state feature map through a region of interest focusing attention layer to obtain the disease and insect pest focused seedling leaf surface state feature map.

[0013] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the seedling leaf surface state feature map is passed through the region of interest focusing attention layer to obtain the pest and disease focused seedling leaf surface state feature map, including: using the pooling layer to encode each channel of the seedling leaf surface state feature map along the horizontal coordinate and the vertical coordinate respectively to obtain a first direction perception feature map and a second direction perception feature map; splicing the first direction perception feature map and the second direction perception feature map to obtain a spliced ​​feature map; performing 1×1 convolution processing on the spliced ​​feature map to obtain a coordinate information feature map; and performing a multi-dimensional convolution operation on the coordinate information feature map. The rows are expanded to obtain a first coordinate attention weight feature matrix and a second coordinate attention weight feature matrix; the first coordinate attention weight feature matrix is ​​used to weight each feature matrix in the X-axis direction of the seedling leaf surface state feature map to obtain a first weighted feature map; the second coordinate attention weight feature matrix is ​​used to weight each feature matrix in the Y-axis direction of the seedling leaf surface state feature map to obtain a second weighted feature map; the seedling leaf surface state feature map, the first weighted feature map and the second weighted feature map are fused through a residual module to obtain the pest and disease focused seedling leaf surface state feature map.

[0014] In the above-mentioned municipal greening seedling disease and pest identification method based on the Internet of Things, the coordinate information feature map is expanded to obtain a first coordinate attention weight feature matrix and a second coordinate attention weight feature matrix, including: decomposing the coordinate information feature map along the spatial dimension to obtain a first decomposition feature map and a second decomposition feature map; performing 1×1 convolution processing on the first decomposition feature map and the second decomposition feature map respectively to obtain the first coordinate attention weight feature matrix and the second coordinate attention weight feature matrix.

[0015] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the leaf surface semantic features of the pest and disease focused seedling leaf surface state feature map are extracted to obtain the pest and disease focused seedling leaf surface semantic feature map, including: passing the pest and disease focused seedling leaf surface state feature map through a seedling leaf surface semantic feature extractor based on a second convolutional neural network model to obtain the pest and disease focused seedling leaf surface semantic feature map.

[0016] In the above-mentioned municipal greening seedling disease and pest identification method based on the Internet of Things, the seedling leaf surface semantic feature extractor based on the second convolutional neural network model includes an input layer, a convolution layer, an activation layer, a maximum pooling layer and an output layer; wherein the convolution layer uses multiple 3x3 convolution kernels with a step size of 1 and a padding of 1; the maximum pooling layer uses a 2x2 pooling window with a step size of 2.

[0017] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the surface state feature map of the pest and disease focused seedling leaves and the semantic feature map of the pest and disease focused seedling leaves are feature fused to obtain a semantically transferred pest and disease focused seedling leaf surface multi-scale feature map, including: using a semantic information efficient transmission module to fuse the surface state feature map of the pest and disease focused seedling leaves and the semantic feature map of the pest and disease focused seedling leaves to obtain the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map.

[0018] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, a semantic information efficient transmission module is used to fuse the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain the semantic transmission pest and disease focused seedling leaf surface multi-scale feature map, including: upsampling the pest and disease focused seedling leaf surface semantic feature map to obtain a resolution reconstruction feature map; performing point convolution, batch normalization and ReLU-based non-activation function on the global mean feature vector obtained by global mean pooling of the resolution reconstruction feature map. Operation to obtain a global semantic vector; perform point convolution, batch normalization and ReLU-based inactivation function operations on the resolution reconstruction feature map to obtain a local semantic vector; perform point addition on the global semantic vector and the local semantic vector to obtain a semantic weight vector; use the semantic weight vector as the weight vector to weight the surface state feature map of the leaves of the pest-focused seedlings to obtain a semantic joint feature map; fuse the surface state feature map of the leaves of the pest-focused seedlings with the semantic joint feature map to obtain the semantically transmitted pest-focused seedling leaf surface multi-scale feature map.

[0019] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the type of pest and disease is determined based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map, including: performing feature map expansion on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map to obtain the semantically transferred pest and disease focused seedling leaf surface multi-scale feature vector; performing feature distribution correction on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature vector to obtain the corrected semantically transferred pest and disease focused seedling leaf surface multi-scale feature vector; passing the corrected semantically transferred pest and disease focused seedling leaf surface multi-scale feature vector through a classifier to obtain a classification result, and the classification result is used to represent the pest and disease type label.

[0020] According to another aspect of the present application, a municipal greening seedling pest identification system based on the Internet of Things is provided, which includes:

[0021] The seedling leaf monitoring module is used to obtain the seedling leaf images collected by the camera;

[0022] A leaf surface state feature extraction module is used to extract leaf surface state features from the seedling leaf image to obtain a pest and disease focused seedling leaf surface state feature map;

[0023] A leaf surface semantic feature extraction module is used to extract leaf surface semantic features of the pest-focused seedling leaf surface state feature map to obtain a pest-focused seedling leaf surface semantic feature map;

[0024] A semantic information transfer module is used to perform feature fusion on the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature map;

[0025] The pest and disease type analysis module is used to determine the pest and disease type based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map.

[0026] Compared with the prior art, the municipal greening seedling pest identification method and system based on the Internet of Things provided by this application is based on computer vision technology and artificial intelligence technology, extracts leaf surface state features from seedling leaf images collected by cameras, and obtains leaf surface semantic feature information focused on the pest area, thereby realizing automatic identification of pest types. In this way, human resources can be saved and the efficiency and accuracy of seedling pest detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 The present invention is a flowchart of a municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application.

[0029] Figure 2 Schematic diagram of the architecture of the municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application.

[0030] Figure 3 The present invention is a flowchart of extracting leaf surface state features from the seedling leaf image in the municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application to obtain a leaf surface state feature map of the pest and disease focused seedlings.

[0031] Figure 4 This is a flowchart of the method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to an embodiment of the present application, in which the surface state characteristic map of the seedling leaves is focused on the attention layer through the region of interest to obtain the surface state characteristic map of the seedling leaves focused on pests and diseases.

[0032] Figure 5 A flowchart of using a semantic information efficient transmission module in the method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to an embodiment of the present application to fuse the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain the semantically transmitted pest and disease focused seedling leaf surface multi-scale feature map.

[0033] Figure 6 This is a flowchart of determining the type of pests and diseases based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map in the municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application.

[0034] Figure 7 It is a block diagram of a municipal greening seedling pest and disease identification system based on the Internet of Things according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] To facilitate the understanding of the embodiments of the present invention, the specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the specific embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0036] Figure 1 The present invention is a flowchart of a municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a municipal greening seedling pest identification method based on the Internet of Things according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the municipal greening seedling pest and disease identification method based on the Internet of Things according to the embodiment of the present application, the steps include: S110, acquiring a seedling leaf image collected by a camera; S120, performing leaf surface state feature extraction on the seedling leaf image to obtain a pest and disease focused seedling leaf surface state feature map; S130, extracting leaf surface semantic features of the pest and disease focused seedling leaf surface state feature map to obtain a pest and disease focused seedling leaf surface semantic feature map; S140, performing feature fusion on the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain a semantic transfer pest and disease focused seedling leaf surface multi-scale feature map; S150, determining the pest and disease type based on the semantic transfer pest and disease focused seedling leaf surface multi-scale feature map.

[0037] As mentioned in the above background technology, pests and diseases may have a negative impact on the health of trees and the landscape effect of urban greening. For example, pests such as aphids and mites can suck the sap of trees, causing damage to trees and affecting their normal growth. Trees attacked by pests and diseases may experience phenomena such as leaf fall, yellowing, and branch death, which will reduce the quality of urban greening landscape and affect the beauty and livability of the city. In addition, some insects can spread diseases, such as ticks spreading Lyme disease, which may affect the health of citizens. Therefore, pest and disease control is one of the important tasks of urban greening maintenance.

[0038] However, the traditional method of pest control for municipal greening seedlings mainly relies on regular manual inspections of the health of greening plants. This inspection method requires a large number of professionals to regularly inspect large areas of greening plants, which is not only inefficient, but also difficult to cover every corner of the city and large areas of greening plants due to human resource limitations, which may lead to the neglect or delay of pests and diseases in some areas or plant groups.

[0039] In response to the above technical problems, the technical concept of this application is to extract the leaf surface state features of the seedling leaf images captured by the camera based on computer vision technology and artificial intelligence technology, and obtain the leaf surface semantic feature information focused on the pest and disease area, so as to realize the automatic identification of the pest and disease type. In this way, human resources can be saved and the efficiency and accuracy of seedling pest and disease detection can be improved.

[0040] In the above-mentioned municipal greening seedling pest identification method based on the Internet of Things, the step S110 obtains the seedling leaf image collected by the camera. It should be understood that the seedling leaf image contains the visual information of the seedling leaf. Further using computer vision technology to analyze and process the image, it is possible to obtain the characteristic information of the leaf surface state, and it is possible to realize the automatic identification of pests and diseases. In addition, the method of using the camera to collect the seedling leaf image is not invasive to the seedling itself, and will not cause harm or interference to the plant. Compared with the traditional manual inspection method, such as sampling or cutting leaves, the camera does not need to contact the plant to collect the image, which can better protect the integrity of the seedling. In addition, the camera can realize the continuous monitoring of large-scale seedlings, is not limited by time and space, and can obtain a large number of leaf images in a short time to realize the comprehensive monitoring and identification of seedling pests and diseases, and improve the detection efficiency.

[0041] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the step S120 extracts the leaf surface state features of the seedling leaf image to obtain a seedling leaf surface state feature map focused on pests and diseases. It should be understood that pests and diseases will show different characteristics and symptoms on the surface of seedling leaves, and these characteristics and symptoms can be used to distinguish different types of pests and diseases. Therefore, in the technical solution of the present application, image processing technology is further used to extract the seedling leaf surface state features focused on the pest and disease area from the seedling leaf image for analyzing the type of pest and disease.

[0042] Figure 3 The flowchart is a method for extracting leaf surface state features from the seedling leaf image to obtain a leaf surface state feature map of the pest-focused seedling leaf in the municipal greening seedling pest identification method based on the Internet of Things according to an embodiment of the present application. Figure 3 As shown, the step S120 includes: S121, passing the seedling leaf image through a leaf surface state feature extractor based on a first convolutional neural network model to obtain a seedling leaf surface state feature map; S122, passing the seedling leaf surface state feature map through a region of interest focusing attention layer to obtain the pest and disease focused seedling leaf surface state feature map.

[0043] Specifically, the step S121, the seedling leaf image is passed through a leaf surface state feature extractor based on a first convolutional neural network model to obtain a seedling leaf surface state feature map. It should be understood that a convolutional neural network (CNN) is a deep learning model, which is widely used in the field of computer vision because of its outstanding performance in image processing tasks. The convolution layer is one of the core components of a convolutional neural network, which performs a convolution operation on an input image using a convolution kernel, and locally perceives the input image to detect key feature information of the input image. Moreover, when performing feature extraction, the convolutional neural network can gradually abstract higher-level features through multi-layer convolution and pooling operations. In the technical solution of the present application, the seedling leaf image is processed by a leaf surface state feature extractor based on a first convolutional neural network model, and the key features of the leaf surface, such as information such as the texture, shape and color of the leaf, can be automatically learned and extracted using the characteristics of the convolutional neural network, and the leaf changes and abnormal features caused by pests and diseases are captured, so as to better represent the state and characteristics of the leaf and improve the accuracy of pest and disease identification. Moreover, the original seedling leaf image has a high dimension and contains a large amount of pixel information. By using the leaf surface state feature extractor based on the first convolutional neural network model for feature extraction, the original image can be reduced to a lower dimension, thereby reducing the complexity of subsequent processing.

[0044] Specifically, in step S122, the surface state feature map of the seedling leaves is passed through the region of interest focusing attention layer to obtain the pest-focused seedling leaf surface state feature map. It should be understood that the region of interest focusing attention layer can guide the network to pay more attention to the regions of interest and extract features related to these regions. In the technical solution of the present application, it is considered that there may be some interference information in the seedling leaf image that is not related to pests and diseases, such as irrelevant background. During convolution, the iterative accumulation of the background will form a large amount of redundant information, thereby submerging part of the target and resulting in low detection accuracy. The region of interest focusing attention layer can help the network better locate the area where pests and diseases are located, more accurately determine the location and range of pests and diseases, and help the network better capture more fine-grained features related to pests and diseases, and improve the recognition performance of pests and diseases. Specifically, by using the region of interest focusing attention layer, the position information of the input seedling leaf surface state feature map is coordinate encoded, which can not only capture cross-channel information, but also capture direction perception and position perception information, thereby reducing attention to irrelevant interference information and increasing the network's attention to pests and diseases, so that the model can more accurately locate and identify pests and diseases, thereby improving the accuracy of pest and disease type identification.

[0045] Figure 4The present invention is a flowchart of the method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to an embodiment of the present application, in which the seedling leaf surface state characteristic map is focused on the attention layer through the region of interest to obtain the pest-focused seedling leaf surface state characteristic map. Figure 4 As shown, the step S122 includes: S1221, using a pooling layer to encode each channel of the seedling leaf surface state feature map along the horizontal coordinate and the vertical coordinate respectively to obtain a first direction perception feature map and a second direction perception feature map; S1222, splicing the first direction perception feature map and the second direction perception feature map to obtain a spliced ​​feature map; S1223, performing a 1×1 convolution process on the spliced ​​feature map to obtain a coordinate information feature map; S1224, expanding the coordinate information feature map to obtain a first coordinate attention weight feature matrix and a second coordinate attention feature matrix. weight feature matrix; S1225, using the first coordinate attention weight feature matrix, weighting each feature matrix in the X-axis direction of the seedling leaf surface state feature map to obtain a first weighted feature map; S1226, using the second coordinate attention weight feature matrix, weighting each feature matrix in the Y-axis direction of the seedling leaf surface state feature map to obtain a second weighted feature map; S1227, using a residual module to fuse the seedling leaf surface state feature map, the first weighted feature map and the second weighted feature map to obtain the pest and disease focused seedling leaf surface state feature map.

[0046] More specifically, the step S1224 expands the coordinate information feature map to obtain a first coordinate attention weight feature matrix and a second coordinate attention weight feature matrix, including: decomposing the coordinate information feature map along the spatial dimension to obtain a first decomposition feature map and a second decomposition feature map; performing 1×1 convolution processing on the first decomposition feature map and the second decomposition feature map respectively to obtain the first coordinate attention weight feature matrix and the second coordinate attention weight feature matrix.

[0047] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the step S130 extracts the leaf surface semantic features of the leaf surface state feature map of the leaf of the pest-focused seedling to obtain the leaf surface semantic feature map of the leaf of the pest-focused seedling. In a specific example of the present application, the encoding method of extracting the leaf surface semantic features of the leaf surface state feature map of the leaf of the pest-focused seedling to obtain the leaf surface semantic feature map of the leaf of the pest-focused seedling is to pass the leaf surface state feature map of the leaf of the pest-focused seedling through a seedling leaf surface semantic feature extractor based on a second convolutional neural network model to obtain the leaf surface semantic feature map of the leaf of the pest-focused seedling. In the technical solution of the present application, the second convolutional neural network model has a deeper network structure, can learn and extract higher-level semantic features, and has a stronger feature extraction and representation capability. By inputting the pest-focused seedling leaf surface state feature map into the seedling leaf surface semantic feature extractor based on the second convolutional neural network model, more abstract semantic feature information can be gradually extracted through the stacking of multiple convolutional layers and pooling layers, which can better capture the key features related to pests and diseases, and help enhance the understanding and distinction capabilities of pests and diseases. More specifically, the seedling leaf surface semantic feature extractor based on the second convolutional neural network model includes an input layer, a convolutional layer, an activation layer, a maximum pooling layer, and an output layer; wherein the convolutional layer uses multiple 3x3 convolution kernels with a step size of 1 and a padding of 1; the maximum pooling layer uses a 2x2 pooling window with a step size of 2.

[0048] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the step S140 performs feature fusion on the surface state feature map of the pest and disease focused seedling leaf and the semantic feature map of the pest and disease focused seedling leaf to obtain a semantically transferred pest and disease focused seedling leaf surface multi-scale feature map. In a specific example of the present application, the feature fusion of the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map is to obtain a semantically transferred pest and disease focused seedling leaf surface multi-scale feature map encoding method is to use the semantic information efficient transmission module to fuse the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map. It should be understood that the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map represent different levels of feature information, respectively. In order to comprehensively utilize the feature information at different levels and improve the performance and accuracy of pest and disease identification, the two are further feature fused. The semantic information efficient transmission module is used to establish effective connections between feature maps of different scales, realize feature transmission and fusion, propagate low-level feature information to high-level, and propagate high-level contextual semantic information to low-level. That is, the surface detail information in the pest-focused seedling leaf surface state feature map is interacted and fused with the deep semantic information in the pest-focused seedling leaf surface semantic feature map, so as to obtain a more comprehensive and accurate semantically transmitted pest-focused seedling leaf surface multi-scale feature map.

[0049] Figure 5 A flowchart of using a semantic information efficient transmission module in the method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to an embodiment of the present application to fuse the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain the semantic transmission pest and disease focused seedling leaf surface multi-scale feature map. Figure 5As shown, the step S140 includes: S141, upsampling the pest-focused seedling leaf surface semantic feature map to obtain a resolution reconstruction feature map; S142, performing point convolution, batch normalization and ReLU-based inactivation function operations on the global mean feature vector obtained after global mean pooling of the resolution reconstruction feature map to obtain a global semantic vector; S143, performing point convolution, batch normalization and ReLU-based inactivation function operations on the resolution reconstruction feature map to obtain a local semantic vector; S144, performing point addition on the global semantic vector and the local semantic vector to obtain a semantic weight vector; S145, using the semantic weight vector as the weight vector, weighting the pest-focused seedling leaf surface state feature map to obtain a semantic joint feature map; S146, fusing the pest-focused seedling leaf surface state feature map with the semantic joint feature map to obtain the semantically transferred pest-focused seedling leaf surface multi-scale feature map.

[0050] In the above-mentioned municipal greening seedling pest and disease identification method based on the Internet of Things, the step S150 determines the type of pest and disease based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map. It should be understood that the type of pest and disease can usually be judged by the color, spots, leaf deformity, insects present on the leaves and other features of the seedling leaves. In the technical solution of the present application, the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map integrates the surface detail information and deep semantic information of the seedling leaves, and has a comprehensive characterization capability for the type of pest and disease. Therefore, the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map is further analyzed by a machine learning algorithm to determine the type of pest and disease.

[0051] Figure 6 This is a flow chart of determining the type of pests and diseases based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map in the municipal greening seedling pest and disease identification method based on the Internet of Things according to an embodiment of the present application. Figure 6 As shown, the step S150 includes: S151, performing feature map expansion on the semantically transferred pests and diseases focused seedling leaf surface multi-scale feature map to obtain a semantically transferred pests and diseases focused seedling leaf surface multi-scale feature vector; S152, performing feature distribution correction on the semantically transferred pests and diseases focused seedling leaf surface multi-scale feature vector to obtain a corrected semantically transferred pests and diseases focused seedling leaf surface multi-scale feature vector; S153, passing the corrected semantically transferred pests and diseases focused seedling leaf surface multi-scale feature vector through a classifier to obtain a classification result, and the classification result is used to represent the pest and disease type label.

[0052] Specifically, in steps S151 and S152, feature map expansion is performed on the semantically transferred pest-focused seedling leaf surface multi-scale feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature vector, and feature distribution correction is performed on the semantically transferred pest-focused seedling leaf surface multi-scale feature vector to obtain a corrected semantically transferred pest-focused seedling leaf surface multi-scale feature vector. In the technical solution of the present application, the pest-focused seedling leaf surface state feature map expresses shallow image semantic features enhanced by the spatial distribution of local feature regions of interest of the seedling leaf image, while the pest-focused seedling leaf surface semantic feature map expresses deep image semantic features based on the shallow image semantic feature expression enhanced by the spatial distribution of local features of the pest-focused seedling leaf surface state feature map. Therefore, there will be differences in the expression of image semantic features between the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map. In the technical scheme of the present application, taking into account the differences in image semantic feature expression between the surface state feature map of leaves of pests and diseases focused seedlings and the surface semantic feature map of leaves of pests and diseases focused seedlings, it is expected that when using the semantic information efficient transmission module to fuse the surface state feature map of leaves of pests and diseases focused seedlings and the surface semantic feature map of leaves of pests and diseases focused seedlings, the differences in high-dimensional geometric changes of the surface state feature map of leaves of pests and diseases focused seedlings and the surface semantic feature map of leaves of pests and diseases focused seedlings based on feature distribution directions corresponding to different geometric transformations in the high-dimensional feature space caused by the differences in image semantic feature expression can be suppressed, thereby improving the fusion effect.

[0053] Based on this, in the technical solution of the present application, the image semantics of the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map are further fused and corrected by the following fusion correction formula to obtain a corrected feature vector, wherein the fusion correction formula is:

[0054]

[0055] V1 is the first feature vector obtained after the surface state feature map of the plant leaf focusing on pests and diseases is expanded, V2 is the second feature vector obtained after the surface semantic feature map of the plant leaf focusing on pests and diseases is expanded, and v 1i 、v 2i and v ci are the eigenvalues ​​of the first eigenvector V1, the second eigenvector V2 and the corrected eigenvector, respectively, ||·||1 and ||·||2 are the 1-norm and 2-norm of the eigenvectors, respectively, L is the length of the eigenvector, and α is a weight hyperparameter.

[0056] Specifically, in order to improve the perception and cognitive ability of features for spatial transformations corresponding to different feature distributions during fusion, the first feature vector V1 after the surface state feature map of the pest-focused seedling leaf is expanded and the second feature vector V2 after the surface semantic feature map of the pest-focused seedling leaf is expanded is used to control the rotation of the feature distribution of the feature vector in different directions from the vector scale dimension, so as to retain the rotational invariance of the fused feature through relative rotational unwrapping, thereby avoiding the high-dimensional geometric change difference of the feature distribution caused by the geometric transformation in the high-dimensional feature space during feature fusion. In this way, when passing through the classifier, the corrected feature vector is fused with the semantically transferred pest-focused seedling leaf surface multi-scale feature vector obtained after the semantically transferred pest-focused seedling leaf surface multi-scale feature map is expanded, so as to improve the fusion expression effect of the semantically transferred pest-focused seedling leaf surface multi-scale feature vector, thereby improving the accuracy of the classification result obtained by the classifier.

[0057] Specifically, the step S153 is to pass the corrected semantically transferred pest-focused seedling leaf surface multi-scale feature vector through a classifier to obtain a classification result, and the classification result is used to represent the pest type label. A person of ordinary skill in the art should know that a classifier is a machine learning model that builds a classification model by learning the relationship between the features of the input data and the category label, and classifies the new unlabeled data according to this model, thereby giving a corresponding classification result. In the technical solution of the present application, by inputting the corrected semantically transferred pest-focused seedling leaf surface multi-scale feature vector into the classifier, the feature learning ability and classification mapping ability of the classifier are utilized to associate and map the pest features in the corrected semantically transferred pest-focused seedling leaf surface multi-scale feature vector with the corresponding pest type, and then output an accurate pest type label to achieve automated pest identification, thereby improving the efficiency and accuracy of seedling pest detection.

[0058] In summary, the municipal greening seedling pest identification method based on the Internet of Things according to the embodiment of the present application is explained, which is based on computer vision technology and artificial intelligence technology, extracts leaf surface state features from the seedling leaf image captured by the camera, and obtains leaf surface semantic feature information focused on the pest area, thereby realizing automatic identification of pest types. In this way, human resources can be saved and the efficiency and accuracy of seedling pest detection can be improved.

[0059] Figure 7 FIG. 1 is a block diagram of a municipal greening seedling pest identification system based on the Internet of Things according to an embodiment of the present application. Figure 7As shown, according to the embodiment of the present application, the municipal greening seedling pest and disease identification system 100 based on the Internet of Things includes: a seedling leaf monitoring module 110, which is used to obtain a seedling leaf image collected by a camera; a leaf surface state feature extraction module 120, which is used to extract leaf surface state features of the seedling leaf image to obtain a pest and disease focused seedling leaf surface state feature map; a leaf surface semantic feature extraction module 130, which is used to extract leaf surface semantic features of the pest and disease focused seedling leaf surface state feature map to obtain a pest and disease focused seedling leaf surface semantic feature map; a semantic information transfer module 140, which is used to perform feature fusion on the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain a semantically transferred pest and disease focused seedling leaf surface multi-scale feature map; a pest and disease type analysis module 150, which is used to determine the pest and disease type based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map.

[0060] Here, those skilled in the art can understand that the specific operations of each module in the above-mentioned municipal greening seedling pest identification system based on the Internet of Things have been referred to above. Figures 1 to 6 The description of the method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things has been introduced in detail, and therefore, its repeated description will be omitted.

[0061] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0062] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0063] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0064] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things, characterized in that: include: Acquire the seedling leaf image collected by the camera; Extracting leaf surface state features from the seedling leaf image to obtain a pest-focused seedling leaf surface state feature map; Extracting leaf surface semantic features of the pest-focused seedling leaf surface state feature map to obtain a pest-focused seedling leaf surface semantic feature map; Performing feature fusion on the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature map; Based on the semantically transferred pest and disease focused multi-scale feature map of the seedling leaf surface, the type of pest and disease is determined.

2. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 1 is characterized in that: Extracting leaf surface state features from the seedling leaf image to obtain a pest-focused seedling leaf surface state feature map includes: The seedling leaf image is passed through a leaf surface state feature extractor based on a first convolutional neural network model to obtain a seedling leaf surface state feature map; The seedling leaf surface state characteristic map is focused through the region of interest attention layer to obtain the pest and disease focused seedling leaf surface state characteristic map.

3. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 2 is characterized in that: The seedling leaf surface state characteristic map is focused on the attention layer of the region of interest to obtain the pest-focused seedling leaf surface state characteristic map, including: Using a pooling layer to encode each channel of the seedling leaf surface state feature map along the horizontal coordinate and the vertical coordinate respectively to obtain a first direction perception feature map and a second direction perception feature map; splicing the first direction-aware feature map and the second direction-aware feature map to obtain a spliced ​​feature map; Performing a 1×1 convolution process on the spliced ​​feature map to obtain a coordinate information feature map; Expanding the coordinate information feature map to obtain a first coordinate attention weight feature matrix and a second coordinate attention weight feature matrix; Using the first coordinate attention weight feature matrix, weighting each feature matrix in the X-axis direction of the seedling leaf surface state feature map to obtain a first weighted feature map; Using the second coordinate attention weight feature matrix, weighting each feature matrix in the Y-axis direction of the seedling leaf surface state feature map to obtain a second weighted feature map; The seedling leaf surface state feature map, the first weighted feature map and the second weighted feature map are fused through a residual module to obtain the pest-focused seedling leaf surface state feature map.

4. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 3 is characterized in that: The coordinate information feature map is expanded to obtain a first coordinate attention weight feature matrix and a second coordinate attention weight feature matrix, including: Decomposing the coordinate information feature map along the spatial dimension to obtain a first decomposition feature map and a second decomposition feature map; The first decomposition feature map and the second decomposition feature map are respectively subjected to 1×1 convolution processing to obtain the first coordinate attention weight feature matrix and the second coordinate attention weight feature matrix.

5. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 4 is characterized in that: Extracting leaf surface semantic features of the pest-focused seedling leaf surface state feature map to obtain a pest-focused seedling leaf surface semantic feature map, including: The pest-focused seedling leaf surface state feature map is passed through a seedling leaf surface semantic feature extractor based on a second convolutional neural network model to obtain the pest-focused seedling leaf surface semantic feature map.

6. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 5 is characterized in that: The seedling leaf surface semantic feature extractor based on the second convolutional neural network model includes an input layer, a convolution layer, an activation layer, a maximum pooling layer and an output layer; wherein the convolution layer uses multiple 3x3 convolution kernels with a step size of 1 and a padding of 1; the maximum pooling layer uses a 2x2 pooling window with a step size of 2.

7. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 6 is characterized in that: The feature fusion of the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map is performed to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature map, including: A semantic information efficient transmission module is used to fuse the pest and disease focused seedling leaf surface state feature map and the pest and disease focused seedling leaf surface semantic feature map to obtain the semantically transmitted pest and disease focused seedling leaf surface multi-scale feature map.

8. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 7, characterized in that: The semantic information efficient transmission module is used to fuse the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map to obtain the semantically transmitted pest-focused seedling leaf surface multi-scale feature map, including: Upsampling the pest-focused seedling leaf surface semantic feature map to obtain a resolution reconstruction feature map; Performing point convolution, batch normalization and ReLU-based inactivation function operations on a global mean feature vector obtained by global mean pooling of the feature map reconstructed at the resolution to obtain a global semantic vector; Performing point convolution, batch normalization and ReLU-based inactivation function operations on the resolution reconstructed feature map to obtain a local semantic vector; Perform a dotted sum on the global semantic vector and the local semantic vector to obtain a semantic weight vector; Taking the semantic weight vector as the weight vector, weighted processing is performed on the pest-focused seedling leaf surface state feature map to obtain a semantic joint feature map; The pest and disease focused seedling leaf surface state feature map is fused with the semantic joint feature map to obtain the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map.

9. The method for identifying pests and diseases of municipal greening seedlings based on the Internet of Things according to claim 8, characterized in that: Based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map, the pest and disease type is determined, including: Expanding the semantically transferred pest-focused seedling leaf surface multi-scale feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature vector; Performing feature distribution correction on the semantically transferred pest-and-disease focused seedling leaf surface multi-scale feature vector to obtain a corrected semantically transferred pest-and-disease focused seedling leaf surface multi-scale feature vector; The corrected semantically transferred pest and disease focused seedling leaf surface multi-scale feature vector is passed through a classifier to obtain a classification result, and the classification result is used to represent the pest and disease type label.

10. A municipal greening seedling pest identification system based on the Internet of Things, characterized in that: include: The seedling leaf monitoring module is used to obtain the seedling leaf images collected by the camera; A leaf surface state feature extraction module is used to extract leaf surface state features from the seedling leaf image to obtain a pest and disease focused seedling leaf surface state feature map; A leaf surface semantic feature extraction module is used to extract leaf surface semantic features of the pest-focused seedling leaf surface state feature map to obtain a pest-focused seedling leaf surface semantic feature map; A semantic information transfer module is used to perform feature fusion on the pest-focused seedling leaf surface state feature map and the pest-focused seedling leaf surface semantic feature map to obtain a semantically transferred pest-focused seedling leaf surface multi-scale feature map; The pest and disease type analysis module is used to determine the pest and disease type based on the semantically transferred pest and disease focused seedling leaf surface multi-scale feature map.

Images