Wheat disease detection method based on deep learning

Abstract

The invention relates to the field of wheat disease detection, in particular to a wheat disease detection method based on deep learning. According to the method, foreground target extraction and multi-level feature extraction are performed on a to-be-detected wheat image by adopting an image processing technology based on deep learning so as to capture wheat foreground shallow layer features and deep layer features, and global interactive modeling is performed on the wheat foreground shallow layer features and the deep layer features by using a Transform architecture; the contextual information expression ability of the features is improved, and whether the wheat has diseases or not is intelligently identified through bidirectional attention joint perception of the features and the features. Therefore, the accuracy of wheat disease detection can be effectively improved, and the robustness of the model in a complex scene is enhanced.

Description

Technical Field

[0001] The present application relates to the field of wheat disease detection, and specifically to a wheat disease detection method based on deep learning. Background Art

[0002] As one of the three major grain crops in the world, wheat plays a pivotal role in my country's agricultural production. Its planting area and output account for more than half of my country's total grain production, which is of great significance for ensuring national food security and increasing farmers' income. However, wheat faces the threat of various diseases during its growth, among which wheat stripe rust, fusarium head blight and powdery mildew are particularly prominent, which have a serious impact on wheat yield and quality.

[0003] Traditional wheat disease detection methods mainly rely on manual experience, and judge whether there is disease by observing leaf color, morphological changes and damage symptoms in the field. This method is not only inefficient and labor-intensive, but also has large errors in disease identification accuracy and classification due to differences in subjective judgment of investigators.

[0004] The invention patent with publication number CN108021894A proposes a wheat disease detection method based on deep strategy learning, which collects a large number of wheat disease images and focuses on the diseased part of the image to form a training sample. Then, a deep neural network is used to build and train a wheat disease image detection model based on the constructed state strategy, action strategy and reward strategy, so that it learns which actions to take in a given state can get the maximum reward (that is, the disease is detected most accurately). Finally, the wheat image to be detected is input into the trained model for disease detection. This scheme reduces the search range required for disease detection through deep strategy learning, thereby effectively improving the detection efficiency. However, the prior art mainly focuses on the learning of local disease features, while ignoring the relationship between disease features and the overall morphology of wheat, which may lead to insufficient detection accuracy and robustness in complex scenarios.

[0005] Therefore, an optimized deep learning-based wheat disease detection method is expected. Summary of the invention

[0006] This application is made in consideration of the above problems. One purpose of this application is to provide a wheat disease detection method based on deep learning.

[0007] The embodiment of the present application provides a wheat disease detection method based on deep learning, which includes:

[0008] Obtain the wheat image to be detected collected by the camera;

[0009] Performing foreground target multi-level feature extraction on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map;

[0010] Performing global context interaction on the wheat foreground shallow feature map and the wheat foreground deep feature map respectively to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map;

[0011] Performing fine-grained joint perception on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map;

[0012] Based on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map, it is determined whether the wheat has disease.

[0013] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, a multi-level feature extraction of a foreground target is performed on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map, including:

[0014] Extracting the foreground area of ​​the foreground portion of the wheat image to be detected to obtain the foreground portion of the wheat image to be detected;

[0015] The wheat foreground part image to be detected is input into a wheat multi-scale image feature extractor based on a dilute pyramid network to obtain the wheat foreground shallow feature map and the wheat foreground deep feature map.

[0016] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, extracting the foreground area of ​​the foreground image of the wheat to be detected to obtain the foreground image of the wheat to be detected includes:

[0017] The wheat image to be detected is input into a foreground target detection module based on the Yolo network to obtain the foreground part image of the wheat to be detected.

[0018] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, the wheat foreground shallow feature map and the wheat foreground deep feature map are respectively subjected to global context interaction to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map, including:

[0019] The wheat foreground shallow feature map and the wheat foreground deep feature map are respectively input into a fine-grained global interaction module based on a Transformer structure to obtain the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map.

[0020] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are fine-grained jointly perceived to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including:

[0021] The wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are input into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map.

[0022] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are input into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including:

[0023] Performing feature fine-grained deconstruction on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a set of wheat foreground shallow channel dimension local feature matrices and a set of wheat foreground deep channel dimension local feature matrices;

[0024] Using each wheat foreground shallow channel dimension local feature matrix in the set of wheat foreground shallow channel dimension local feature matrices as a query feature matrix, using the set of wheat foreground deep channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on the first converter structure to obtain a set of one-way global attention optimized wheat foreground shallow channel dimension local feature matrices;

[0025] Using each wheat foreground deep channel dimension local feature matrix in the set of wheat foreground deep channel dimension local feature matrices as a query feature matrix, using the set of wheat foreground shallow channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a second converter structure to obtain a set of one-way global attention optimized wheat foreground deep channel dimension local feature matrices;

[0026] The set of local feature matrices of shallow channel dimension of wheat foreground optimized by one-way global attention and the set of local feature matrices of deep channel dimension of wheat foreground optimized by one-way global attention are respectively coupled along the channel dimension to obtain a shallow feature map of wheat foreground optimized by one-way global interaction and a deep feature map of wheat foreground optimized by one-way global interaction;

[0027] The position-weighted sum between the one-way global interactive optimization wheat foreground shallow feature map and the one-way global interactive optimization wheat foreground deep feature map is calculated to obtain the wheat foreground shallow-deep fine-grained interactive significant fusion feature map.

[0028] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, each wheat foreground shallow channel dimension local feature matrix in the set of the wheat foreground shallow channel dimension local feature matrix is ​​used as a query feature matrix, and the set of the wheat foreground deep channel dimension local feature matrix is ​​used as a set of key feature matrices, and the query feature matrix and the set of the key feature matrix are input into a unidirectional global attention interaction module based on a first converter structure to obtain a set of unidirectional global attention optimized wheat foreground shallow channel dimension local feature matrices, including:

[0029] Calculate the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices respectively, and then divide the matrix product by the square root of the scale of the key feature matrix and pass it through a softmax function to obtain a set of attention weight matrices;

[0030] Each attention weight matrix in the set of attention weight matrices is multiplied with the query feature matrix respectively, and then the multiplication results are summed by position point to obtain the one-way global attention optimized wheat foreground shallow channel dimension local feature matrix.

[0031] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, determining whether wheat has a disease based on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map includes:

[0032] The wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is a disease.

[0033] For example, according to the wheat disease detection method based on deep learning of an embodiment of the present application, the wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is a disease, including:

[0034] Expanding the wheat foreground shallow-deep fine-grained interactive significant fusion feature map into a classification feature vector according to a row vector or a column vector;

[0035] Performing full-connection encoding on the classification feature vector using the fully-connected layer of the classifier-based disease detection module to obtain an encoded classification feature vector;

[0036] The encoded classification feature vector is input into the Softmax classification function of the classifier-based disease detection module to obtain the detection result.

[0037] According to the wheat disease detection method based on deep learning of the embodiment of the present application, it uses image processing technology based on deep learning to perform foreground target extraction and multi-level feature extraction on the wheat image to be detected, so as to capture the shallow and deep features of the wheat foreground, and use the Transformer architecture to perform global interactive modeling on the shallow and deep features of the wheat foreground, so as to improve the context information expression ability of the features, and then perform two-way attention joint perception on the two, so as to intelligently identify whether the wheat has diseases. In this way, the accuracy of wheat disease detection can be effectively improved, and the robustness of the model in complex scenarios can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0039] Figure 1 A schematic diagram of the application architecture of the wheat disease detection method based on deep learning in an embodiment of the present application is shown;

[0040] Figure 2 A flowchart of a wheat disease detection method based on deep learning in an embodiment of the present application is shown;

[0041] Figure 3 A flowchart of sub-step S520 of the wheat disease detection method based on deep learning in an embodiment of the present application is shown;

[0042] Figure 4 A flowchart of sub-step S540 of the wheat disease detection method based on deep learning in an embodiment of the present application is shown;

[0043] Figure 5 A schematic diagram of the structure of a wheat disease detection system based on deep learning in an embodiment of the present application is shown; and

[0044] Figure 6 An application scenario diagram of the wheat disease detection method based on deep learning in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0046] The terms used in this specification are those common terms currently widely used in the art in consideration of the functions of the present application, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected, and in this case, their detailed meanings will be described in the detailed description of the present application. Therefore, the terms used in the specification should not be understood as simple names, but rather as a general description based on the meaning of the terms and the present application.

[0047] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0048] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0049] Figure 1 A schematic diagram of the application architecture of a wheat disease detection method based on deep learning in an embodiment of the present application is shown, including a server 100 and a terminal device 200.

[0050] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or a virtual private network. In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0051] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.

[0052] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In an embodiment of the present application, the memory 120 may be used to store the program of the wheat disease detection method based on deep learning in an embodiment of the present application.

[0053] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the deep learning-based wheat disease detection methods in the embodiments of the present application according to the obtained program instructions.

[0054] In addition, the application architecture diagram in the embodiment of the present application is intended to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems.

[0055] The following is a non-restrictive explanation of the deep learning-based wheat disease detection method provided according to at least one embodiment of the present application through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present application.

[0056] In response to the above technical problems, the technical concept of this application is: to use deep learning-based image processing technology to extract foreground targets and multi-level features of wheat images to be detected, so as to capture the shallow and deep features of wheat foreground, and use the Transformer architecture to perform global interaction modeling on the shallow and deep features of wheat foreground, so as to improve the contextual information expression ability of the features, and then to perform two-way attention joint perception on the two, so as to intelligently identify whether wheat has diseases. In this way, the accuracy of wheat disease detection can be effectively improved, and the robustness of the model in complex scenarios can be enhanced.

[0057] Based on this, Figure 2 The flowchart of the wheat disease detection method based on deep learning in the embodiment of the present application is shown. For example, the wheat disease detection method based on deep learning can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 2As shown, according to the deep learning-based wheat disease detection method of the embodiment of the present application, the method includes the following steps: S510, obtaining a wheat image to be detected collected by a camera; S520, performing multi-level feature extraction of foreground targets on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map; S530, performing global context interaction on the wheat foreground shallow feature map and the wheat foreground deep feature map respectively to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map; S540, performing fine-grained joint perception on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map; S550, determining whether the wheat has a disease based on the wheat foreground shallow-deep fine-grained interaction significant fusion feature map.

[0058] Specifically, in the technical solution of the present application, first, an image of wheat to be detected captured by a camera is obtained. Then, considering that the image of wheat to be detected captured by the camera usually also contains irrelevant background noise and non-target areas, and wheat diseases usually occur in specific parts of wheat plants, such as leaves, stems, etc. Therefore, in order to reduce image background interference, reduce the amount of data for subsequent processing, and improve overall detection efficiency, the present application uses a foreground target detection module based on the Yolo network to extract foreground targets from the wheat image to be detected, so as to utilize the powerful target detection capability of the Yolo network to efficiently identify and extract the wheat foreground part in the image, remove most of the background information irrelevant to disease detection, and obtain the image of the foreground part of the wheat to be detected, so that subsequent disease detection can focus on the wheat foreground target area to improve detection accuracy and efficiency.

[0059] Next, in order to more comprehensively extract the disease-related information in the foreground image of the wheat to be detected, the present application adopts the Atrous Pyramid Network (ASPP) as a wheat multi-scale image feature extractor to perform multi-scale feature extraction on the foreground image of the wheat to be detected. The Atrous Pyramid Network can capture disease features of different scales at the same time by setting convolution kernels with different expansion rates, fully understand the characteristic manifestations of the disease at different scales, and thus extract the wheat foreground shallow feature map and the wheat foreground deep feature map. Among them, the shallow features of the wheat foreground shallow feature map mainly focus on the local details of the disease, such as the shape and color of the lesions, while the wheat foreground deep feature map focuses on the semantic information and overall structure of the image, revealing the relationship between the disease and the overall growth state of wheat, as well as the position and distribution of the disease in the global image.

[0060] Accordingly, in step S520, Figure 3As shown, multi-level feature extraction of foreground targets is performed on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map, including: S521, extracting the foreground area of ​​the wheat foreground partial image to be detected to obtain the wheat foreground partial image to be detected; S522, inputting the wheat foreground partial image to be detected into a wheat multi-scale image feature extractor based on a dilute pyramid network to obtain the wheat foreground shallow feature map and the wheat foreground deep feature map.

[0061] Among them, in step S521, the foreground area of ​​the foreground part image of the wheat to be detected is extracted to obtain the foreground part image of the wheat to be detected, including: inputting the wheat image to be detected into a foreground target detection module based on the Yolo network to obtain the foreground part image of the wheat to be detected.

[0062] Then, the wheat foreground shallow feature map and the wheat foreground deep feature map are input into a fine-grained global interaction module based on a Transformer structure. It should be understood that considering the association between disease features and the overall morphology of wheat is crucial to improving detection accuracy, and convolutional neural networks have advantages in processing local features, but have limited modeling capabilities for global context information. Therefore, the present application further adopts a fine-grained global interaction module based on a Transformer structure to process the wheat foreground shallow feature map and the wheat foreground deep feature map respectively, and utilizes the advantages of the Transformer structure in capturing long-distance dependencies to further strengthen the global context information in the wheat foreground shallow feature map and the wheat foreground deep feature map, so as to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map. Specifically, the fine-grained global interaction module splits the feature map into multiple sub-regions, regards it as a sequence of local features, and uses a self-attention mechanism to perform global interaction modeling on it, so as to effectively capture the correlation between features at different positions in the feature map and enhance the global context information expression capability of the feature map.

[0063] Correspondingly, in step S530, global context interaction is performed on the wheat foreground shallow feature map and the wheat foreground deep feature map respectively to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map, including: inputting the wheat foreground shallow feature map and the wheat foreground deep feature map into a fine-grained global interaction module based on a Transformer structure respectively to obtain the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map.

[0064] Furthermore, in order to comprehensively utilize the multi-level information in the shallow fine-grained global interaction feature map of the wheat foreground and the deep fine-grained global interaction feature map of the wheat foreground, the present application introduces a bidirectional global attention joint perception module based on fine-grained deconstruction to process the shallow fine-grained global interaction feature map of the wheat foreground and the deep fine-grained global interaction feature map of the wheat foreground. Through the bidirectional attention mechanism, the dependency relationship between the shallow features and the deep features is captured, so as to achieve deep fusion of shallow and deep features. Specifically, the module first performs fine-grained deconstruction on the shallow fine-grained global interaction feature map of the wheat foreground and the deep fine-grained global interaction feature map of the wheat foreground, and decomposes each feature map along the channel dimension into a set of channel dimension local feature matrices, so as to analyze the local information in the feature map in more detail. Next, the multi-head attention mechanism in the Transformer structure is used to measure the correlation between the two channel-dimensional local feature matrix sets, strengthen the mutual relationship between the multi-level features, and more comprehensively capture the global bidirectional semantic association information between the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map by performing one-way semantic interaction between each channel-dimensional local feature matrix and all channel-dimensional local feature matrices in the other set. Then, the attention-optimized local feature matrices are recombined along the channel dimension to form complete wheat foreground shallow features and deep features, and the two are fused by weighted sum by position to synthesize the complementary information of the two and generate a wheat foreground shallow-deep fine-grained interaction salient fusion feature map containing multi-level interaction features.

[0065] Correspondingly, in step S540, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are fine-grained jointly perceived to obtain a wheat foreground shallow-deep fine-grained interactive significant fusion feature map, including: inputting the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interactive significant fusion feature map.

[0066] Specifically, Figure 4As shown, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are input into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including: S541, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are subjected to feature fine-grained deconstruction to obtain a set of wheat foreground shallow channel dimension local feature matrices and a set of wheat foreground deep channel dimension local feature matrices; S542, each wheat foreground shallow channel dimension local feature matrix in the set of wheat foreground shallow channel dimension local feature matrices is used as a query feature matrix, and the set of wheat foreground deep channel dimension local feature matrices is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a unidirectional global attention interaction module based on the first converter structure to obtain a set of unidirectional global attention optimized wheat foreground shallow channel dimension local feature matrices; S543, Using each wheat foreground deep channel dimension local feature matrix in the set of wheat foreground deep channel dimension local feature matrices as a query feature matrix, and using the set of wheat foreground shallow channel dimension local feature matrices as a key feature matrix set, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a second converter structure to obtain a set of one-way global attention optimized wheat foreground deep channel dimension local feature matrices; S544, performing feature coupling along the channel dimension on the set of one-way global attention optimized wheat foreground shallow channel dimension local feature matrices and the set of one-way global attention optimized wheat foreground deep channel dimension local feature matrices respectively to obtain a one-way global interaction optimized wheat foreground shallow feature map and a one-way global interaction optimized wheat foreground deep feature map; S545, calculating the position-weighted sum between the one-way global interaction optimized wheat foreground shallow feature map and the one-way global interaction optimized wheat foreground deep feature map to obtain the wheat foreground shallow-deep fine-grained interactive significant fusion feature map.

[0067] Among them, in step S542, each local feature matrix of the shallow channel dimension of the wheat foreground in the set of local feature matrices of the shallow channel dimension of the wheat foreground is used as a query feature matrix, and the set of local feature matrices of the deep channel dimension of the wheat foreground is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on the first converter structure to obtain a set of local feature matrices of the shallow channel dimension of the wheat foreground optimized by one-way global attention, including: respectively calculating the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices, and then dividing the matrix product by the square root of the scale of the key feature matrix and passing it through a softmax function to obtain a set of attention weight matrices; multiplying each attention weight matrix in the set of attention weight matrices with the query feature matrix respectively, and then summing the multiplication results by position points to obtain the one-way global attention optimized local feature matrix of the shallow channel dimension of the wheat foreground.

[0068] In one example, the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are input into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including: processing the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map with the following attention joint perception formula to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map, wherein the attention joint perception formula is:

[0069] Decouple(X1)={M 11 ,M 12 ,...,M 1i ,...,M 1n}

[0070] Decouple(X2)={M 21 ,M 22 ,...,M 2j ,...,M 2n}

[0071]

[0072] X1′=Aggregate{M′ 11 ,M′ 12 ,...,M′ 1i ,...,M′ 1n}

[0073] X2′=Aggregate{M′ 21 ,M′22 ,...,M′ 2j ,...,M′ 2n}

[0074] F=aF1′+bF2′

[0075] Wherein, X1 represents the shallow fine-grained global interactive feature map of the wheat foreground, X2 represents the deep fine-grained global interactive feature map of the wheat foreground, Decouple(·) represents the deconstruction process, M 11 、M 12 、M 1i and M 1n Respectively represent the first, second, i-th and n-th wheat foreground shallow channel dimension local feature matrices of the wheat foreground shallow fine-grained global interactive feature map along the channel dimension, where n is the number of channels of the wheat foreground shallow fine-grained global interactive feature map, M 21 、M 22 、M 2j 、M 2n They respectively represent the first, second, j-th and n-th wheat foreground deep channel dimension local feature matrices of the wheat foreground deep fine-grained global interactive feature map along the channel dimension, represents a matrix multiplication operation, S represents the scale of the local feature matrix of the wheat foreground deep channel dimension, and the scale of the local feature matrix of the wheat foreground deep channel dimension is the same as the local feature matrix of the wheat foreground shallow channel dimension, (·) T represents the transpose of the matrix, softmax is the normalized exponential function, M 11 '、M 12 '、M 1i ' and M 1n 'respectively represent the first, second, i-th and n-th unidirectional global attention optimization wheat foreground shallow channel dimension local feature matrix in the set of the unidirectional global attention optimization wheat foreground shallow channel dimension local feature matrix, M 21 '、M 22 '、M 2j ' and M 2n 'respectively represent the first, second, j-th and n-th one-way global attention optimization of wheat foreground deep channel dimension local feature matrices in the set of the one-way global attention optimization of wheat foreground deep channel dimension local feature matrices, Aggregate{·} represents feature aggregation processing, X1' represents the one-way global interactive optimization of wheat foreground shallow feature map, X2' represents the one-way global interactive optimization of wheat foreground deep feature map, a and b are different weight coefficients, and F represents the wheat foreground shallow-deep fine-grained interactive significant fusion feature map.

[0076] Finally, the wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is a disease.

[0077] Accordingly, in step S550, based on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map, it is determined whether the wheat has disease, including: inputting the wheat foreground shallow-deep fine-grained interactive significant fusion feature map into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is disease.

[0078] Specifically, the wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is a disease, including: expanding the wheat foreground shallow-deep fine-grained interactive significant fusion feature map into a classification feature vector according to a row vector or a column vector; using the fully connected layer of the classifier-based disease detection module to fully connect encode the classification feature vector to obtain an encoded classification feature vector; inputting the encoded classification feature vector into the Softmax classification function of the classifier-based disease detection module to obtain the detection result.

[0079] It should be understood that the role of the classifier is to use the given categories and known training data to learn classification rules, and then classify (or predict) unknown data. Logistic regression and SVM are often used to solve binary classification problems. For multi-class classification problems, logistic regression or SVM can also be used, but multiple binary classifications are required to form a multi-classification. However, this is prone to errors and is not efficient. Commonly used multi-classification methods include the Softmax classification function.

[0080] In a preferred embodiment, the wheat foreground shallow-deep fine-grained interactively significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, including: calculating the sum of the absolute values ​​of each eigenvalue of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map to obtain a first wheat foreground shallow-deep fine-grained interactively significant fusion and modulation value, and calculating the square root of the sum of the squares of each eigenvalue of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map to obtain a second wheat foreground shallow-deep fine-grained interactively significant fusion and modulation value; after dot-wise subtraction of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map and the second wheat foreground shallow-deep fine-grained interactively significant fusion and modulation value, dot-wise multiplication is performed with the number of eigenvalues ​​of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map and the inverse of the first wheat foreground shallow-deep fine-grained interactively significant fusion and modulation value, and taking the inverse of each eigenvalue to obtain the first wheat foreground shallow-deep fine-grained interactively significant fusion and modulation value. The method comprises the following steps: generating a phase transition feature map of shallow-deep fine-grained interactive significant fusion of wheat foreground; performing dot-wise subtraction on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map and the first wheat foreground shallow-deep fine-grained interactive significant fusion and modulation value, respectively performing dot-wise multiplication with the square root of the number of eigenvalues ​​of the wheat foreground shallow-deep fine-grained interactive significant fusion feature map and the inverse of the second wheat foreground shallow-deep fine-grained interactive significant fusion and modulation value, and taking the inverse of each eigenvalue to obtain a second wheat foreground shallow-deep fine-grained interactive significant fusion phase transition feature map; performing dot-wise subtraction on the first wheat foreground shallow-deep fine-grained interactive significant fusion phase transition feature map and the dot product vector of the second wheat foreground shallow-deep fine-grained interactive significant fusion phase transition feature map and the weighted hyperparameter to obtain an optimized wheat foreground shallow-deep fine-grained interactive significant fusion feature map; and inputting the optimized wheat foreground shallow-deep fine-grained interactive significant fusion feature map into the classifier-based disease detection module to obtain the detection result.

[0081] Here, the optimization process of the wheat foreground shallow-deep fine-grained interactive significant fusion feature map, denoted as F, is expressed as follows:

[0082]

[0083] f i ∈F∈R W×H×C

[0084] Among them, F is the shallow-deep fine-grained interactive significant fusion feature map of the wheat foreground, f iare the eigenvalues ​​of the shallow-deep fine-grained interactive significant fusion feature map of the wheat foreground, |·| represents the absolute value calculation, α is the first wheat foreground shallow-deep fine-grained interactive significant fusion and modulation value, β is the second wheat foreground shallow-deep fine-grained interactive significant fusion and modulation value, represents point subtraction, ⊙ represents point multiplication, n represents the number of eigenvalues ​​of the shallow-deep fine-grained interactive significant fusion feature map of the wheat foreground, [·] ⊙-1 Represents the inverse of each eigenvalue in the calculated feature map, F1 is the first wheat foreground shallow-deep fine-grained interactive significant fusion phase transition feature map, F2 represents the second wheat foreground shallow-deep fine-grained interactive significant fusion phase transition feature map, ω represents the weighted hyperparameter, F' is the optimized wheat foreground shallow-deep fine-grained interactive significant fusion feature map, W represents the width of the feature map, H represents the height of the feature map, and C represents the number of channels of the feature map.

[0085] That is, considering that the shallow fine-grained global interaction feature map of the wheat foreground and the deep fine-grained global interaction feature map of the wheat foreground respectively represent the image shallow pixel granularity semantic association features of the wheat foreground part image to be detected and the image deep pixel granularity semantic association features of the wheat foreground part image to be detected, when they are passed through the bidirectional global attention joint perception module based on fine-grained deconstruction, the spatial fine-grained image semantic feature distribution difference based on the semantic distribution of the source image will also have the global attention difference of the local spatial distribution deconstruction. Therefore, it is expected to improve the detail semantic joint perception aggregation expression effect based on the local-global bidirectional image semantic feature spatial distribution difference of the wheat foreground shallow-deep fine-grained interaction significant fusion feature map.

[0086] Therefore, in the preferred example, the difference of the different and modulation representations of the feature values ​​of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map relative to the overall feature set of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map is used as semantic change intensity information, and a phase-like transformation corresponding to the position-based intensity modulation is performed through different and modulation representations, so that the aggregation enhancement of semantic change phase perception can enhance the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the set scale equalization of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map, thereby improving the perception effect of the aggregated semantics of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map on the detailed semantic changes, thereby improving the expression effect of the wheat foreground shallow-deep fine-grained interactively significant fusion feature map, and improving the accuracy of the detection result obtained by inputting it into the classifier-based disease detection module.

[0087] Based on the above embodiments, see Figure 5As shown, it is a structural diagram of a wheat disease detection system 800 based on deep learning in an embodiment of the present application. The wheat disease detection system 800 based on deep learning includes: an image acquisition module 810, which is used to obtain a wheat image to be detected acquired by a camera; a multi-level feature extraction module 820, which is used to perform foreground target multi-level feature extraction on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map; a global context interaction module 830, which is used to perform global context interaction on the wheat foreground shallow feature map and the wheat foreground deep feature map respectively to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map; a fine-grained joint perception module 840, which is used to perform fine-grained joint perception on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map; and a disease analysis module 850, which is used to determine whether wheat has a disease based on the wheat foreground shallow-deep fine-grained interaction significant fusion feature map.

[0088] Here, those skilled in the art will appreciate that the specific functions and operations of each module in the above-mentioned wheat disease detection system 800 based on deep learning have been described in detail above. Figures 2 to 4 The description of the deep learning-based wheat disease detection method has been introduced in detail, and therefore, its repeated description will be omitted.

[0089] Figure 6 FIG. 1 is an application scenario diagram of a wheat disease detection method based on deep learning according to an embodiment of the present application. Figure 6 As shown, in this application scenario, first, the wheat image to be detected collected by the camera is obtained (for example, Figure 6 D), then, the wheat image to be detected is input to a server (for example, Figure 6 In S) shown in , the server is capable of using the wheat disease detection algorithm based on deep learning to process the wheat image to be detected to obtain a detection result indicating whether a disease exists.

[0090] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present application. In some possible implementations, the electronic device in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the wheat disease detection method based on deep learning in the above embodiments when executing the program.

[0091] For example, in the case of electronic equipment Figure 1Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.

[0092] The embodiments of the present application also provide a computer-readable storage medium having computer-executable instructions stored thereon. When the computer-executable instructions are executed by a processor, the wheat disease detection method based on deep learning according to the embodiments of the present application described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.

[0093] The embodiment of the present application also provides a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the computer device executes the wheat disease detection method based on deep learning according to the embodiment of the present application.

[0094] Those skilled in the art will appreciate that the contents disclosed in this application may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0095] In addition, although the present application makes various references to certain units in the system according to embodiments of the present application, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0096] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present application is not limited to any particular form of combination of hardware and software.

[0097] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0098] The above is an explanation of the present application and should not be considered as limiting thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application.

Claims

1. A wheat disease detection method based on deep learning, characterized in that: include: Obtain the wheat image to be detected collected by the camera; Performing foreground target multi-level feature extraction on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map; Performing global context interaction on the wheat foreground shallow feature map and the wheat foreground deep feature map respectively to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map; Performing fine-grained joint perception on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map; Based on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map, it is determined whether the wheat has disease.

2. The wheat disease detection method based on deep learning according to claim 1, characterized in that: Performing multi-level feature extraction of foreground targets on the wheat image to be detected to obtain a wheat foreground shallow feature map and a wheat foreground deep feature map, including: Extracting the foreground area of ​​the foreground portion of the wheat image to be detected to obtain the foreground portion of the wheat image to be detected; The wheat foreground part image to be detected is input into a wheat multi-scale image feature extractor based on a dilute pyramid network to obtain the wheat foreground shallow feature map and the wheat foreground deep feature map.

3. The wheat disease detection method based on deep learning according to claim 2, characterized in that: Extracting the foreground area of ​​the foreground portion of the wheat image to be detected to obtain the foreground portion of the wheat image to be detected, comprising: The wheat image to be detected is input into a foreground target detection module based on the Yolo network to obtain the foreground part image of the wheat to be detected.

4. The wheat disease detection method based on deep learning according to claim 3 is characterized in that: The wheat foreground shallow feature map and the wheat foreground deep feature map are respectively subjected to global context interaction to obtain a wheat foreground shallow fine-grained global interaction feature map and a wheat foreground deep fine-grained global interaction feature map, including: The wheat foreground shallow feature map and the wheat foreground deep feature map are respectively input into a fine-grained global interaction module based on a Transformer structure to obtain the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map.

5. The wheat disease detection method based on deep learning according to claim 4, characterized in that: Fine-grained joint perception is performed on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including: The wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map are input into a bidirectional global attention joint perception module based on fine-grained deconstruction to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map.

6. The wheat disease detection method based on deep learning according to claim 5, characterized in that: Inputting the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map into a fine-grained deconstruction-based bidirectional global attention joint perception module to obtain the wheat foreground shallow-deep fine-grained interaction significant fusion feature map, including: Performing feature fine-grained deconstruction on the wheat foreground shallow fine-grained global interaction feature map and the wheat foreground deep fine-grained global interaction feature map to obtain a set of wheat foreground shallow channel dimension local feature matrices and a set of wheat foreground deep channel dimension local feature matrices; Using each wheat foreground shallow channel dimension local feature matrix in the set of wheat foreground shallow channel dimension local feature matrices as a query feature matrix, using the set of wheat foreground deep channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on the first converter structure to obtain a set of one-way global attention optimized wheat foreground shallow channel dimension local feature matrices; Using each wheat foreground deep channel dimension local feature matrix in the set of wheat foreground deep channel dimension local feature matrices as a query feature matrix, using the set of wheat foreground shallow channel dimension local feature matrices as a set of key feature matrices, inputting the query feature matrix and the set of key feature matrices into a one-way global attention interaction module based on a second converter structure to obtain a set of one-way global attention optimized wheat foreground deep channel dimension local feature matrices; The set of local feature matrices of shallow channel dimension of wheat foreground optimized by one-way global attention and the set of local feature matrices of deep channel dimension of wheat foreground optimized by one-way global attention are respectively coupled along the channel dimension to obtain a shallow feature map of wheat foreground optimized by one-way global interaction and a deep feature map of wheat foreground optimized by one-way global interaction; The position-weighted sum between the one-way global interactive optimization wheat foreground shallow feature map and the one-way global interactive optimization wheat foreground deep feature map is calculated to obtain the wheat foreground shallow-deep fine-grained interactive significant fusion feature map.

7. The wheat disease detection method based on deep learning according to claim 6, characterized in that: Each wheat foreground shallow channel dimension local feature matrix in the set of wheat foreground shallow channel dimension local feature matrices is used as a query feature matrix, and the set of wheat foreground deep channel dimension local feature matrices is used as a set of key feature matrices, and the query feature matrix and the set of key feature matrices are input into a one-way global attention interaction module based on the first converter structure to obtain a set of one-way global attention optimized wheat foreground shallow channel dimension local feature matrices, including: Calculate the matrix product of the query feature matrix and the transposed matrix of each key feature matrix in the set of key feature matrices respectively, and then divide the matrix product by the square root of the scale of the key feature matrix and pass it through a softmax function to obtain a set of attention weight matrices; Each attention weight matrix in the set of attention weight matrices is multiplied with the query feature matrix respectively, and then the multiplication results are summed by position point to obtain the one-way global attention optimized wheat foreground shallow channel dimension local feature matrix.

8. The wheat disease detection method based on deep learning according to claim 7, characterized in that: Based on the wheat foreground shallow-deep fine-grained interactive significant fusion feature map, determine whether the wheat has disease, including: The wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, and the detection result is used to indicate whether there is a disease.

9. The wheat disease detection method based on deep learning according to claim 8, characterized in that: The wheat foreground shallow-deep fine-grained interactive significant fusion feature map is input into a classifier-based disease detection module to obtain a detection result, which is used to indicate whether there is a disease, including: Expanding the wheat foreground shallow-deep fine-grained interactive significant fusion feature map into a classification feature vector according to a row vector or a column vector; Performing full-connection encoding on the classification feature vector using the fully-connected layer of the classifier-based disease detection module to obtain an encoded classification feature vector; The encoded classification feature vector is input into the Softmax classification function of the classifier-based disease detection module to obtain the detection result.

Images