Monitoring Method for Citrus Huanglongbing Based on UAV Multispectral

Abstract

The present application discloses a method for monitoring citrus huanglongbing based on drone multispectral. It acquires multispectral images of the detected citrus leaves through a multispectral camera carried by a drone, extracts reference multispectral images of citrus leaves labeled as healthy from a database, and uses image monitoring analysis and processing techniques based on deep learning to extract image features and perform differential calculations on the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves. Based on this, it automatically discriminates whether the detected citrus leaves have huanglongbing according to the multi-dimensional significant differential features between the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves. In this way, it can use the multispectral camera carried by the drone to collect data in real time and quickly, and more accurately identify the huanglongbing characteristics of citrus leaves, so as to take timely measures to prevent the spread of the disease.

Description

Technical Field

[0001] This application relates to the field of intelligent monitoring, and more specifically, to a method for monitoring citrus huanglongbing based on drone multispectral technology. Background Art

[0002] Citrus huanglongbing is considered the "cancer" of citrus, caused by Gram-negative anaerobic bacteria. This disease can lead to yellowing of citrus leaves, poor fruit development, and ultimately the death of the trees, seriously affecting citrus yield and quality. Once a citrus orchard is infected, the disease will spread rapidly, causing huge economic losses to fruit farmers. Therefore, by monitoring citrus, the infection of huanglongbing can be detected early, and corresponding measures can be taken at the initial stage of the disease to prevent the rapid spread of the disease.

[0003] However, due to cost and time limitations, it is difficult to achieve rapid random inspections of large areas of citrus huanglongbing by traditional methods, resulting in multiple outbreaks of the disease and difficult to control. In addition, the field symptom diagnosis of huanglongbing relies on observing manifestations such as leaf yellowing, but these symptoms may be similar to nutrient deficiencies or other diseases, prone to misdiagnosis.

[0004] Therefore, a method for monitoring citrus huanglongbing based on drone multispectral technology is desired. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. An embodiment of this application provides a method for monitoring citrus huanglongbing based on drone multispectral technology, which collects multispectral images of the detected citrus leaves by a multispectral camera carried by a drone, extracts reference multispectral images of citrus leaves labeled as healthy from a database, and uses image feature extraction and differential calculation techniques for image monitoring based on deep learning to perform image feature extraction and differential calculation on the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves, so as to automatically determine whether the detected citrus leaves have huanglongbing based on the multi-dimensional significant differential features between the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves. In this way, it is possible to use the multispectral camera carried by the drone to collect data in real time and quickly, and more accurately identify the huanglongbing characteristics of citrus leaves, so as to take timely measures to prevent the spread of the disease.

[0006] According to one aspect of this application, a method for monitoring citrus huanglongbing based on drone multispectral technology is provided, which includes:

[0007] Obtaining multispectral images of the detected citrus leaves collected by a multispectral camera carried by a drone;

[0008] Extracting reference multispectral images of citrus leaves labeled as healthy from a database;

[0009] Perform multi - spectral multi - scale feature extraction on the reference multi - spectral image of the citrus leaves labeled as healthy and the multi - spectral image of the detected citrus leaves to obtain a shallow - layer feature map for citrus leaf detection, a middle - layer feature map for citrus leaf detection, a deep - layer feature map for citrus leaf detection, a shallow - layer feature map for citrus leaf reference, a middle - layer feature map for citrus leaf reference, and a deep - layer feature map for citrus leaf reference;

[0010] Calculate the differential feature maps between the shallow - layer feature map for citrus leaf detection and the shallow - layer feature map for citrus leaf reference, between the middle - layer feature map for citrus leaf detection and the middle - layer feature map for citrus leaf reference, and between the deep - layer feature map for citrus leaf detection and the deep - layer feature map for citrus leaf reference to obtain a shallow - layer differential feature map for citrus leaves, a middle - layer differential feature map for citrus leaves, and a semantic differential feature map for citrus leaves;

[0011] Pass the shallow - layer differential feature map for citrus leaves, the middle - layer differential feature map for citrus leaves, and the semantic differential feature map for citrus leaves through a gated screening adaptive attention module respectively to obtain an adaptively enhanced shallow - layer differential feature map for citrus leaves, an adaptively enhanced middle - layer differential feature map for citrus leaves, and an adaptively enhanced semantic differential feature map for citrus leaves;

[0012] Fuse the adaptively enhanced shallow - layer differential feature map for citrus leaves, the adaptively enhanced middle - layer differential feature map for citrus leaves, and the adaptively enhanced semantic differential feature map for citrus leaves to obtain a globally multi - dimensional significant differential feature map for citrus leaves as the globally multi - dimensional significant differential feature of citrus leaves;

[0013] Based on the globally multi - dimensional significant differential feature of citrus leaves, obtain a monitoring result, and the monitoring result is used to indicate whether the detected citrus leaves have huanglongbing.

[0014] In the above - mentioned method for monitoring citrus huanglongbing based on unmanned aerial vehicle multi - spectrum, performing multi - spectral multi - scale feature extraction on the reference multi - spectral image of the citrus leaves labeled as healthy and the multi - spectral image of the detected citrus leaves to obtain a shallow - layer feature map for citrus leaf detection, a middle - layer feature map for citrus leaf detection, a deep - layer feature map for citrus leaf detection, a shallow - layer feature map for citrus leaf reference, a middle - layer feature map for citrus leaf reference, and a deep - layer feature map for citrus leaf reference includes: inputting the reference multi - spectral image of the citrus leaves labeled as healthy and the multi - spectral image of the detected citrus leaves into a multi - spectral feature extractor based on a dilated convolutional neural network model to obtain the shallow - layer feature map for citrus leaf detection, the middle - layer feature map for citrus leaf detection, the deep - layer feature map for citrus leaf detection, the shallow - layer feature map for citrus leaf reference, the middle - layer feature map for citrus leaf reference, and the deep - layer feature map for citrus leaf reference.

[0015] In the above-mentioned citrus huanglongbing monitoring method based on drone multispectral, the shallow-layer differential feature map of the citrus leaf, the middle-layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf are respectively passed through a gated screening adaptive attention module to obtain an adaptively enhanced shallow-layer differential feature map of the citrus leaf, an adaptively enhanced middle-layer differential feature map of the citrus leaf, and an adaptively enhanced semantic differential feature map of the citrus leaf, including: performing multiple pooling processes on each feature matrix of the shallow-layer differential feature map of the citrus leaf along the channel dimension to obtain a global maximum pooling feature vector of the shallow-layer differential of the citrus leaf, a global mean pooling feature vector of the shallow-layer differential of the citrus leaf, and a global random value pooling feature vector of the shallow-layer differential of the citrus leaf; performing weighted fusion on the global maximum pooling feature vector of the shallow-layer differential of the citrus leaf, the global mean pooling feature vector of the shallow-layer differential of the citrus leaf, and the global random value pooling feature vector of the shallow-layer differential of the citrus leaf to obtain a global representation vector of the shallow-layer differential of the citrus leaf; performing per-channel semantic feature interaction and feature activation based on a fully connected layer on the global representation vector of the shallow-layer differential of the citrus leaf to obtain a per-channel semantic feature vector of the shallow-layer differential of the citrus leaf; inputting the per-channel semantic feature vector of the shallow-layer differential of the citrus leaf into a gated unit to obtain a gated screening per-channel semantic feature vector of the shallow-layer differential of the citrus leaf; performing normalization processing on the gated screening per-channel semantic feature vector of the shallow-layer differential of the citrus leaf to obtain a gated screening per-channel semantic weight feature vector of the shallow-layer differential of the citrus leaf; and using the gated screening per-channel semantic weight feature vector of the shallow-layer differential of the citrus leaf as a weight feature vector, calculating its per-channel product with the shallow-layer differential feature map of the citrus leaf to obtain the adaptively enhanced shallow-layer differential feature map of the citrus leaf.

[0016] In the above-mentioned citrus huanglongbing monitoring method based on drone multispectral, performing multiple pooling processes on each feature matrix of the shallow-layer differential feature map of the citrus leaf along the channel dimension to obtain a global maximum pooling feature vector of the shallow-layer differential of the citrus leaf, a global mean pooling feature vector of the shallow-layer differential of the citrus leaf, and a global random value pooling feature vector of the shallow-layer differential of the citrus leaf, including: performing global pooling processing based on the maximum value, global pooling processing based on the average value, and global pooling processing based on the random value on each feature matrix of the shallow-layer differential feature map of the citrus leaf along the channel dimension to obtain the global maximum pooling feature vector of the shallow-layer differential of the citrus leaf, the global mean pooling feature vector of the shallow-layer differential of the citrus leaf, and the global random value pooling feature vector of the shallow-layer differential of the citrus leaf.

[0017] In the above-mentioned citrus huanglongbing monitoring method based on unmanned aerial vehicle multispectral, weighted fusion is performed on the citrus leaf shallow differential global maximum pooling feature vector, the citrus leaf shallow differential global mean pooling feature vector, and the citrus leaf shallow differential global random value pooling feature vector to obtain a citrus leaf shallow differential global representation vector, including: multiplying the citrus leaf shallow differential global maximum pooling feature vector, the citrus leaf shallow differential global mean pooling feature vector, and the citrus leaf shallow differential global random value pooling feature vector with corresponding adjustment parameters respectively by position to obtain a citrus leaf shallow differential global maximum modulation pooling feature vector, a citrus leaf shallow differential global mean modulation pooling feature vector, and a citrus leaf shallow differential global random value modulation pooling feature vector; and adding the citrus leaf shallow differential global maximum modulation pooling feature vector, the citrus leaf shallow differential global mean modulation pooling feature vector, and the citrus leaf shallow differential global random value modulation pooling feature vector by position to obtain the citrus leaf shallow differential global representation vector.

[0018] In the above-mentioned citrus huanglongbing monitoring method based on unmanned aerial vehicle multispectral, per-channel semantic feature interaction and feature activation based on a fully connected layer are performed on the citrus leaf shallow differential global representation vector to obtain a citrus leaf shallow differential channel semantic feature vector, including: calculating the matrix multiplication of the citrus leaf shallow differential global representation vector and a weight matrix and then adding a bias vector by vector to obtain a bias-adjusted citrus leaf shallow differential global representation vector; and performing activation processing on the bias-adjusted citrus leaf shallow differential global representation vector through an activation function to obtain the citrus leaf shallow differential channel semantic feature vector.

[0019] In the above-mentioned citrus huanglongbing monitoring method based on unmanned aerial vehicle multispectral, the citrus leaf shallow differential channel semantic feature vector is input into a gating unit to obtain a gated and screened citrus leaf shallow differential channel semantic feature vector, including: in response to each position feature value in the citrus leaf shallow differential channel semantic feature vector being greater than or equal to a predetermined threshold, the position feature value takes the original value, otherwise it is zero, to obtain the gated and screened citrus leaf shallow differential channel semantic feature vector.

[0020] In the above-mentioned citrus huanglongbing monitoring method based on unmanned aerial vehicle multispectral, based on the citrus leaf global multi-dimensional significant differential features, a monitoring result is obtained, including: passing the citrus leaf global multi-dimensional significant differential feature map through a huanglongbing monitoring module based on a classifier to obtain the monitoring result.

[0021] In the above-mentioned citrus huanglongbing monitoring method based on drone multispectral, it further includes a training step: for training the multispectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the citrus huanglongbing monitoring module based on the classifier.

[0022] In the above citrus huanglongbing monitoring method based on drone multispectral, the training step includes: obtaining training data, where the training data includes training multispectral images of the citrus leaves to be detected collected by a multispectral camera carried by a drone, extracting reference training multispectral images of citrus leaves labeled as healthy from a database, and the true monitoring result, where the true monitoring result is the true value of whether the citrus leaves to be detected have huanglongbing; inputting the reference training multispectral images of citrus leaves labeled as healthy and the training multispectral images of the citrus leaves to be detected into the multispectral feature extractor based on the dilated convolutional neural network model to obtain a training citrus leaf detection shallow feature map, a training citrus leaf detection middle feature map, a training citrus leaf detection deep feature map, a training citrus leaf reference shallow feature map, a training citrus leaf reference middle feature map, and a training citrus leaf reference deep feature map; calculating the differential feature map between the training citrus leaf detection shallow feature map and the training citrus leaf reference shallow feature map, the differential feature map between the training citrus leaf detection middle feature map and the training citrus leaf reference middle feature map, and the differential feature map between the training citrus leaf detection deep feature map and the training citrus leaf reference deep feature map to obtain a training citrus leaf shallow differential feature map, a training citrus leaf middle differential feature map, and a training citrus leaf semantic differential feature map; passing the training citrus leaf shallow differential feature map, the training citrus leaf middle differential feature map, and the training citrus leaf semantic differential feature map through the gated screening adaptive attention module respectively to obtain a training adaptive enhanced citrus leaf shallow differential feature map, a training adaptive enhanced citrus leaf middle differential feature map, and a training adaptive enhanced citrus leaf semantic differential feature map; fusing the training adaptive enhanced citrus leaf shallow differential feature map, the training adaptive enhanced citrus leaf middle differential feature map, and the training adaptive enhanced citrus leaf semantic differential feature map to obtain a training citrus leaf global multi-dimensional significant differential feature map; passing the training citrus leaf global multi-dimensional significant differential feature map through the huanglongbing monitoring module based on the classifier to obtain a classification loss function value; calculating a citrus leaf global multi-dimensional significant differential loss function value based on the training citrus leaf global multi-dimensional significant differential feature map; and using the weighted sum of the classification loss function value and the citrus leaf global multi-dimensional significant differential loss function value as the final loss function value, and training the multispectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the huanglongbing monitoring module based on the classifier through backpropagation of gradient descent.

[0023] Compared with the prior art, a citrus huanglongbing monitoring method based on drone multispectral provided by the present application acquires multispectral images of the detected citrus leaves through a multispectral camera carried by a drone, extracts reference multispectral images of citrus leaves labeled as healthy from a database, and uses image monitoring analysis and processing techniques based on deep learning to extract image features and perform differential calculations on the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves, so as to automatically determine whether the detected citrus leaves have huanglongbing according to the multi-dimensional significant differential features between the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves. In this way, data can be collected in real time and quickly by using the multispectral camera carried by the drone, and the huanglongbing characteristics of the citrus leaves can be identified more accurately, so as to take timely measures to prevent the spread of the disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 It is a flowchart of a citrus huanglongbing monitoring method based on drone multispectral according to an embodiment of the present application.

[0026] Figure 2 It is a schematic diagram of the architecture of a citrus huanglongbing monitoring method based on drone multispectral according to an embodiment of the present application.

[0027] Figure 3 It is a flowchart of obtaining an adaptively enhanced shallow differential feature map of citrus leaves, an adaptively enhanced middle differential feature map of citrus leaves, and an adaptively enhanced semantic differential feature map of citrus leaves by respectively passing the shallow differential feature map of citrus leaves, the middle differential feature map of citrus leaves, and the semantic differential feature map of citrus leaves through a gated screening adaptive attention module in a citrus huanglongbing monitoring method based on drone multispectral according to an embodiment of the present application.

[0028] Figure 4 It is a flowchart of training a multispectral feature extractor based on a dilated convolutional neural network model, a gated screening adaptive attention module, and a huanglongbing monitoring module based on a classifier in a citrus huanglongbing monitoring method based on drone multispectral according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not used to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0031] In the description of the embodiments of the present disclosure, the term "including" and its like terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0032] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0033] Huanglongbing of citrus is considered the "cancer" of citrus and is caused by Gram-negative anaerobic bacteria. This disease can cause yellowing of citrus leaves, poor fruit development, and ultimately the death of the trees, seriously affecting the yield and quality of citrus. Once a citrus orchard is infected, the disease will spread rapidly, bringing huge economic losses to fruit farmers. Therefore, by monitoring citrus, the infection of Huanglongbing can be detected early, so that corresponding measures can be taken at the initial stage of the disease to prevent the rapid spread of the disease.

[0034] However, due to cost and time limitations, it is difficult to achieve rapid random inspections of Huanglongbing of citrus on a large scale by traditional methods, resulting in multiple outbreaks of the disease and difficult to control. In addition, the field symptom diagnosis of Huanglongbing depends on observing manifestations such as leaf yellowing, but these symptoms may be similar to nutrient deficiencies or other diseases, and it is easy to cause misdiagnosis.

[0035] It should be understood that unmanned aerial vehicle (UAV) multispectral imaging is a technology that uses sensors carried by UAVs to capture spectral information of different wavelengths. This technology can obtain spectral data beyond the visible light, such as infrared and near-infrared, and has many benefits and advantages in the detection of citrus huanglongbing. Specifically, it can provide large-scale monitoring, which is crucial for timely detection and control of the spread of the disease. In addition, UAV multispectral technology can capture subtle changes in the spectral reflectance of citrus, which may be related to the health status of the plants, thereby improving the accuracy of disease detection.

[0036] Based on this, the present application proposes a method for monitoring citrus huanglongbing based on UAV multispectral. It acquires the multispectral images of the citrus leaves to be detected by a multispectral camera carried by a UAV, extracts the reference multispectral images of healthy citrus leaves from a database, and uses image feature extraction and differential calculation techniques for image monitoring based on deep learning to perform image feature extraction and differential calculation on the multispectral images of the citrus leaves to be detected and the reference multispectral images of the citrus leaves. Thus, it automatically determines whether the citrus leaves to be detected have huanglongbing according to the multi-dimensional significant differential features between the multispectral images of the citrus leaves to be detected and the reference multispectral images of the citrus leaves. In this way, it can use the multispectral camera carried by the UAV to collect data in real time and quickly, and more accurately identify the huanglongbing characteristics of citrus leaves, so as to take timely measures to prevent the spread of the disease.

[0037] Figure 1 FIG. is a flowchart of a method for monitoring citrus huanglongbing based on UAV multispectral according to an embodiment of the present application. Figure 2 FIG. is a schematic structural diagram of a method for monitoring citrus huanglongbing based on UAV multispectral according to an embodiment of the present application. As Figure 1 and Figure 2As shown in the figure, the citrus huanglongbing monitoring method based on drone multispectral according to the embodiment of the present application includes: S110, obtaining a multispectral image of the detected citrus leaves collected by a multispectral camera carried by a drone; S120, extracting a reference multispectral image of citrus leaves marked as healthy from a database; S130, performing multispectral multi-scale feature extraction on the reference multispectral image of the citrus leaves marked as healthy and the multispectral image of the detected citrus leaves to obtain a citrus leaf detection shallow feature map, a citrus leaf detection middle feature map, a citrus leaf detection deep feature map, a citrus leaf reference shallow feature map, a citrus leaf reference middle feature map, and a citrus leaf reference deep feature map; S140, calculating the differential feature map between the citrus leaf detection shallow feature map and the citrus leaf reference shallow feature map, the differential feature map between the citrus leaf detection middle feature map and the citrus leaf reference middle feature map, and the differential feature map between the citrus leaf detection deep feature map and the citrus leaf reference deep feature map to obtain a citrus leaf shallow differential feature map, a citrus leaf middle differential feature map, and a citrus leaf semantic differential feature map; S150, respectively passing the citrus leaf shallow differential feature map, the citrus leaf middle differential feature map, and the citrus leaf semantic differential feature map through a gated screening adaptive attention module to obtain an adaptive enhanced citrus leaf shallow differential feature map, an adaptive enhanced citrus leaf middle differential feature map, and an adaptive enhanced citrus leaf semantic differential feature map; S160, fusing the adaptive enhanced citrus leaf shallow differential feature map, the adaptive enhanced citrus leaf middle differential feature map, and the adaptive enhanced citrus leaf semantic differential feature map to obtain a citrus leaf global multi-dimensional significant differential feature map as the citrus leaf global multi-dimensional significant difference feature; and, S170, obtaining a monitoring result based on the citrus leaf global multi-dimensional significant difference feature, where the monitoring result is used to indicate whether the detected citrus leaves have huanglongbing.

[0038] In step S110, a multispectral image of the detected citrus leaves collected by a multispectral camera carried by a drone is obtained. It should be understood that the multispectral image of the detected citrus leaves is collected in real time by a multispectral camera carried by a drone during a specific flight mission, reflecting the reflection characteristics of the current citrus leaves in multiple spectral bands. Therefore, in the technical solution of the present application, obtaining a multispectral image of the detected citrus leaves collected by a multispectral camera carried by a drone can reveal the physiological and biochemical characteristics of plants that cannot be directly observed by the human eye and identify the spectral characteristics of possible huanglongbing symptoms.

[0039] In step S120, a reference multispectral image of a citrus leaf labeled as healthy is extracted from the database. Correspondingly, considering that the reference multispectral image of the citrus leaf labeled as healthy is extracted from the database and is a multispectral image of a citrus leaf that has been confirmed by experts to be in a healthy state. Based on this, in order to use these images as a reference standard to accurately compare and analyze with the detected citrus leaf images, in the technical solution of this application, a reference multispectral image of a citrus leaf labeled as healthy is extracted from the database.

[0040] In step S130, perform multi-spectral multi-scale feature extraction on the multi-spectral image of the citrus leaf labeled as healthy and the multi-spectral image of the citrus leaf to be detected to obtain a shallow feature map for citrus leaf detection, a middle feature map for citrus leaf detection, a deep feature map for citrus leaf detection, a shallow feature map for citrus leaf reference, a middle feature map for citrus leaf reference, and a deep feature map for citrus leaf reference. Specifically, in the embodiment of the present application, performing multi-spectral multi-scale feature extraction on the multi-spectral image of the citrus leaf labeled as healthy and the multi-spectral image of the citrus leaf to be detected to obtain a shallow feature map for citrus leaf detection, a middle feature map for citrus leaf detection, a deep feature map for citrus leaf detection, a shallow feature map for citrus leaf reference, a middle feature map for citrus leaf reference, and a deep feature map for citrus leaf reference includes: inputting the multi-spectral image of the citrus leaf labeled as healthy and the multi-spectral image of the citrus leaf to be detected into a multi-spectral feature extractor based on a dilated convolutional neural network model to obtain the shallow feature map for citrus leaf detection, the middle feature map for citrus leaf detection, the deep feature map for citrus leaf detection, the shallow feature map for citrus leaf reference, the middle feature map for citrus leaf reference, and the deep feature map for citrus leaf reference. It should be understood that considering that the multi-spectral image of the citrus leaf labeled as healthy and the multi-spectral image of the citrus leaf to be detected both reflect the feature information of citrus leaves at different scales. For example, at a smaller scale, the image contains details about the surface texture of the leaf; at a medium scale, the overall shape and edge features of the leaf can be observed; at a larger scale, the internal structure of the leaf can be provided. And dilated convolution usually has a deep structure and can capture the multi-scale features of the image, that is, gradually extract more and more abstract features from the shallow layer to the deep layer, which is crucial for understanding the spectral characteristics of citrus leaves at different levels. Based on this, in the technical solution of the present application, the multi-spectral image of the citrus leaf labeled as healthy and the multi-spectral image of the citrus leaf to be detected are input into a multi-spectral feature extractor based on a dilated convolutional neural network model to respectively capture and extract the feature information of the detected and reference citrus leaves at different scales, so as to obtain a shallow feature map for citrus leaf detection, a middle feature map for citrus leaf detection, a deep feature map for citrus leaf detection, a shallow feature map for citrus leaf reference, a middle feature map for citrus leaf reference, and a deep feature map for citrus leaf reference.

[0041] In step S140, calculate the differential feature maps between the citrus leaf detection shallow feature map and the citrus leaf reference shallow feature map, the differential feature maps between the citrus leaf detection middle feature map and the citrus leaf reference middle feature map, and the differential feature maps between the citrus leaf detection deep feature map and the citrus leaf reference deep feature map to obtain the citrus leaf shallow differential feature map, the citrus leaf middle differential feature map, and the citrus leaf semantic differential feature map. Correspondingly, in order to highlight the differences between the multi-spectral image of the detected citrus leaf and the citrus leaf reference multi-spectral image, so as to better distinguish whether huanglongbing exists. Further, in the technical solution of the present application, calculate the differential feature maps between the citrus leaf detection shallow feature map and the citrus leaf reference shallow feature map, the differential feature maps between the citrus leaf detection middle feature map and the citrus leaf reference middle feature map, and the differential feature maps between the citrus leaf detection deep feature map and the citrus leaf reference deep feature map to obtain the citrus leaf shallow differential feature map, the citrus leaf middle differential feature map, and the citrus leaf semantic differential feature map. That is, by calculating the differential features between the detected citrus leaf and the healthy reference citrus leaf, the abnormal changes caused by huanglongbing infection can be highlighted. These abnormal changes may manifest as differences in color, texture, shape, etc., to help enhance the distinguishability between different categories and make the differences between the huanglongbing-infected area and the healthy area more obvious.

[0042] In step S150, the shallow-layer differential feature map of the citrus leaf, the middle-layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf are respectively passed through a gated screening adaptive attention module to obtain an adaptively enhanced shallow-layer differential feature map of the citrus leaf, an adaptively enhanced middle-layer differential feature map of the citrus leaf, and an adaptively enhanced semantic differential feature map of the citrus leaf. It should be understood that considering that each differential feature map in the shallow-layer differential feature map of the citrus leaf, the middle-layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf shows different citrus leaf feature information in each channel, but some of these different feature information in each channel are of important influence on the detection of huanglongbing disease, and some are irrelevant noise information. Based on this, in the technical solution of the present application, the shallow-layer differential feature map of the citrus leaf, the middle-layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf are respectively passed through a gated screening adaptive attention module to obtain an adaptively enhanced shallow-layer differential feature map of the citrus leaf, an adaptively enhanced middle-layer differential feature map of the citrus leaf, and an adaptively enhanced semantic differential feature map of the citrus leaf. Specifically, the gated screening adaptive attention module first extracts key information at different scales and levels in the differential feature map by performing pooling processing and weighted fusion on the differential feature map in various ways, so as to convert the channel features of the differential feature map into a global channel representation vector. Then, per-channel semantic interaction and feature activation processing are performed on the channel representation vector to enable the citrus leaf information between different channels to be communicated and fused, enhance the correlation and synergy between the citrus leaf features of each channel, and obtain a channel semantic feature vector. Then, the weight vector obtained after masking and normalizing the channel semantic feature vector in sequence is used to perform feature weighting on the differential feature map along the channel dimension, so as to finely adjust and screen important citrus leaf features, and at the same time reduce the influence of irrelevant or noise features on the final citrus leaf feature representation, thereby enhancing the contribution of these citrus leaf features in the final feature representation, and further improving the detection accuracy of citrus huanglongbing.

[0043] Figure 3 A flowchart of passing the shallow-layer differential feature map of the citrus leaf, the middle-layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf through a gated screening adaptive attention module respectively to obtain an adaptively enhanced shallow-layer differential feature map of the citrus leaf, an adaptively enhanced middle-layer differential feature map of the citrus leaf, and an adaptively enhanced semantic differential feature map of the citrus leaf in the method for monitoring citrus huanglongbing based on unmanned aerial vehicle multispectral according to an embodiment of the present application. Specifically, in an embodiment of the present application, as Figure 3As shown, the shallow - layer differential feature map of the citrus leaf, the middle - layer differential feature map of the citrus leaf, and the semantic differential feature map of the citrus leaf are respectively passed through a gated screening adaptive attention module to obtain an adaptively enhanced shallow - layer differential feature map of the citrus leaf, an adaptively enhanced middle - layer differential feature map of the citrus leaf, and an adaptively enhanced semantic differential feature map of the citrus leaf, including: S210, performing pooling processing on each feature matrix of the shallow - layer differential feature map of the citrus leaf along the channel dimension in multiple ways to obtain a global maximum pooling feature vector of the shallow - layer differential of the citrus leaf, a global average pooling feature vector of the shallow - layer differential of the citrus leaf, and a global random - value pooling feature vector of the shallow - layer differential of the citrus leaf; S220, performing weighted fusion on the global maximum pooling feature vector of the shallow - layer differential of the citrus leaf, the global average pooling feature vector of the shallow - layer differential of the citrus leaf, and the global random - value pooling feature vector of the shallow - layer differential of the citrus leaf to obtain a global representation vector of the shallow - layer differential of the citrus leaf; S230, performing per - channel semantic feature interaction and feature activation based on a fully - connected layer on the global representation vector of the shallow - layer differential of the citrus leaf to obtain a per - channel semantic feature vector of the shallow - layer differential of the citrus leaf; S240, inputting the per - channel semantic feature vector of the shallow - layer differential of the citrus leaf into a gated unit to obtain a gated - screened per - channel semantic feature vector of the shallow - layer differential of the citrus leaf; S250, performing normalization processing on the gated - screened per - channel semantic feature vector of the shallow - layer differential of the citrus leaf to obtain a gated - screened per - channel semantic weight feature vector of the shallow - layer differential of the citrus leaf; and, S260, using the gated - screened per - channel semantic weight feature vector of the shallow - layer differential of the citrus leaf as a weight feature vector, calculating the per - channel product of it and the shallow - layer differential feature map of the citrus leaf to obtain the adaptively enhanced shallow - layer differential feature map of the citrus leaf. In particular, the processing methods of the middle - layer differential feature map of the citrus leaf and the semantic differential feature map of the citrus leaf are the same as those of the shallow - layer differential feature map of the citrus leaf.

[0044] More specifically, in the embodiment of the present application, performing pooling processing on each feature matrix of the shallow - layer differential feature map of the citrus leaf along the channel dimension in multiple ways to obtain a global maximum pooling feature vector of the shallow - layer differential of the citrus leaf, a global average pooling feature vector of the shallow - layer differential of the citrus leaf, and a global random - value pooling feature vector of the shallow - layer differential of the citrus leaf includes: performing global pooling processing based on the maximum value, global pooling processing based on the average value, and global pooling processing based on random values on each feature matrix of the shallow - layer differential feature map of the citrus leaf along the channel dimension to obtain the global maximum pooling feature vector of the shallow - layer differential of the citrus leaf, the global average pooling feature vector of the shallow - layer differential of the citrus leaf, and the global random - value pooling feature vector of the shallow - layer differential of the citrus leaf.

[0045] More specifically, in the embodiments of the present application, weighted fusion is performed on the citrus leaf shallow-layer differential global maximum pooling feature vector, the citrus leaf shallow-layer differential global mean pooling feature vector, and the citrus leaf shallow-layer differential global random value pooling feature vector to obtain a citrus leaf shallow-layer differential global representation vector, including: multiplying the citrus leaf shallow-layer differential global maximum pooling feature vector, the citrus leaf shallow-layer differential global mean pooling feature vector, and the citrus leaf shallow-layer differential global random value pooling feature vector with corresponding adjustment parameters respectively by element-wise multiplication to obtain a citrus leaf shallow-layer differential global maximum modulation pooling feature vector, a citrus leaf shallow-layer differential global mean modulation pooling feature vector, and a citrus leaf shallow-layer differential global random value modulation pooling feature vector; and, adding the citrus leaf shallow-layer differential global maximum modulation pooling feature vector, the citrus leaf shallow-layer differential global mean modulation pooling feature vector, and the citrus leaf shallow-layer differential global random value modulation pooling feature vector by element-wise addition to obtain the citrus leaf shallow-layer differential global representation vector.

[0046] More specifically, in the embodiments of the present application, per-channel semantic feature interaction and feature activation based on a fully connected layer are performed on the citrus leaf shallow-layer differential global representation vector to obtain a citrus leaf shallow-layer differential channel semantic feature vector, including: calculating the matrix multiplication of the citrus leaf shallow-layer differential global representation vector and a weight matrix and then adding a bias vector by vector addition to obtain a bias-adjusted citrus leaf shallow-layer differential global representation vector; and, passing the bias-adjusted citrus leaf shallow-layer differential global representation vector through an activation function for activation processing to obtain the citrus leaf shallow-layer differential channel semantic feature vector.

[0047] More specifically, in the embodiments of the present application, the citrus leaf shallow-layer differential channel semantic feature vector is input into a gating unit to obtain a gated and screened citrus leaf shallow-layer differential channel semantic feature vector, including: in response to each position feature value in the citrus leaf shallow-layer differential channel semantic feature vector being greater than or equal to a predetermined threshold, the position feature value takes the original value, otherwise it is zero, to obtain the gated and screened citrus leaf shallow-layer differential channel semantic feature vector.

[0048] In the embodiments of the present application, specifically, the citrus leaf shallow-layer differential feature map is passed through a gated and screened adaptive attention module to obtain an adaptively enhanced citrus leaf shallow-layer differential feature map, including: using the gated and screened adaptive attention module to process the citrus leaf shallow-layer differential feature map with the following adaptive screening formula to obtain the adaptively enhanced citrus leaf shallow-layer differential feature map; where the adaptive screening formula is:

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Among them, represents the shallow differential feature map of the citrus leaf, , and are respectively the global pooling processing based on the average value, the global pooling processing based on the maximum value, and the global pooling processing based on the random value for each feature matrix of the feature map along the channel dimension, , and are respectively the corresponding adjustment parameters, represents the shallow differential global representation vector of the citrus leaf, is the weight matrix, is the bias vector, represents the activation processing, represents the shallow differential channel semantic feature vector of the citrus leaf, represents the masking processing, represents the gated screening of the shallow differential channel semantic feature vector of the citrus leaf, represents the number of eigenvalues in the gated screening of the shallow differential channel semantic feature vector of the citrus leaf, and respectively represent the th and th eigenvalues in the gated screening of the shallow differential channel semantic feature vector of the citrus leaf, is the th eigenvalue in the gated screening of the shallow differential channel semantic weight feature vector of the citrus leaf, is the gated screening of the shallow differential channel semantic weight feature vector of the citrus leaf, represents the adaptive reinforcement of the shallow differential feature map of the citrus leaf, represents the weighted multiplication along the channel dimension. In particular, the encoding methods of the middle differential feature map of the citrus leaf and the semantic differential feature map of the citrus leaf are the same as those of the shallow differential feature map of the citrus leaf.

[0055] In step S160, the adaptive enhanced citrus leaf shallow differential feature map, the adaptive enhanced citrus leaf middle - layer differential feature map, and the adaptive enhanced citrus leaf semantic differential feature map are fused to obtain a citrus leaf global multi - dimensional significant differential feature map as the citrus leaf global multi - dimensional significant differential feature. Correspondingly, considering that the adaptive enhanced citrus leaf shallow differential feature map, the adaptive enhanced citrus leaf middle - layer differential feature map, and the adaptive enhanced citrus leaf semantic differential feature map respectively represent distinguishable enhanced feature information obtained after adaptive enhancement of different degrees of citrus leaves. Therefore, in order to comprehensively utilize these different levels of citrus leaf information and make the final feature representation more comprehensive and rich, in the technical solution of this application, the adaptive enhanced citrus leaf shallow differential feature map, the adaptive enhanced citrus leaf middle - layer differential feature map, and the adaptive enhanced citrus leaf semantic differential feature map are fused to obtain a citrus leaf global multi - dimensional significant differential feature map. That is to say, by fusing features of different levels, the feature representation ability and expression ability can be improved. Since the differential features of the adaptive enhanced citrus leaves at each level have their unique contributions, the obtained citrus leaf global multi - dimensional significant differential feature map can better describe the relationship between the multi - dimensional hierarchical features of citrus leaves.

[0056] In step S170, based on the citrus leaf global multi - dimensional significant differential feature, a monitoring result is obtained, and the monitoring result is used to indicate whether the detected citrus leaf has huanglongbing. Specifically, in the embodiment of this application, based on the citrus leaf global multi - dimensional significant differential feature, obtaining the monitoring result includes: passing the citrus leaf global multi - dimensional significant differential feature map through a huanglongbing monitoring module based on a classifier to obtain the monitoring result. That is, using the citrus leaf global multi - dimensional significant differential feature obtained by fusing the adaptive enhanced citrus leaf shallow differential feature map, the adaptive enhanced citrus leaf middle - layer differential feature map, and the adaptive enhanced citrus leaf semantic differential feature map for classification processing, so as to automatically determine whether the detected citrus leaf has huanglongbing. By this method, data can be collected in real - time and quickly using a multi - spectral camera carried by a drone, and the huanglongbing characteristics of citrus leaves can be identified more accurately, so as to take timely measures to prevent the spread of the disease.

[0057] It is worth mentioning that those of ordinary skill in the art should know that before applying a deep neural network model for inference, the deep neural network model needs to be trained first so that the deep neural network can implement a specific function.

[0058] Specifically, in the embodiment of the present application, it further includes a training step: for training the multi-spectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the huanglongbing monitoring module based on the classifier.

[0059] Figure 4 It is a flowchart for training the multi-spectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the huanglongbing monitoring module based on the classifier in the citrus huanglongbing monitoring method based on unmanned aerial vehicle multi-spectral according to the embodiment of the present application. As Figure 4As shown, the training steps include: S310, obtaining training data, where the training data includes training multispectral images of the citrus leaves to be detected collected by a multispectral camera carried by a drone, extracting reference training multispectral images of citrus leaves labeled as healthy from a database, and real monitoring results, where the real monitoring result is the true value of whether the citrus leaves to be detected have huanglongbing; S320, inputting the reference training multispectral images of citrus leaves labeled as healthy and the training multispectral images of the citrus leaves to be detected into the multispectral feature extractor based on the dilated convolutional neural network model to obtain a training citrus leaf detection shallow feature map, a training citrus leaf detection middle feature map, a training citrus leaf detection deep feature map, a training citrus leaf reference shallow feature map, a training citrus leaf reference middle feature map, and a training citrus leaf reference deep feature map; S330, calculating the differential feature map between the training citrus leaf detection shallow feature map and the training citrus leaf reference shallow feature map, the differential feature map between the training citrus leaf detection middle feature map and the training citrus leaf reference middle feature map, and the differential feature map between the training citrus leaf detection deep feature map and the training citrus leaf reference deep feature map to obtain a training citrus leaf shallow differential feature map, a training citrus leaf middle differential feature map, and a training citrus leaf semantic differential feature map; S340, respectively passing the training citrus leaf shallow differential feature map, the training citrus leaf middle differential feature map, and the training citrus leaf semantic differential feature map through the gated screening adaptive attention module to obtain a training adaptive enhanced citrus leaf shallow differential feature map, a training adaptive enhanced citrus leaf middle differential feature map, and a training adaptive enhanced citrus leaf semantic differential feature map; S350, fusing the training adaptive enhanced citrus leaf shallow differential feature map, the training adaptive enhanced citrus leaf middle differential feature map, and the training adaptive enhanced citrus leaf semantic differential feature map to obtain a training citrus leaf global multi-dimensional significant differential feature map; S360, passing the training citrus leaf global multi-dimensional significant differential feature map through the huanglongbing monitoring module based on the classifier to obtain a classification loss function value; S370, calculating a citrus leaf global multi-dimensional significant differential loss function value based on the training citrus leaf global multi-dimensional significant differential feature map; and S380, using the weighted sum of the classification loss function value and the citrus leaf global multi-dimensional significant differential loss function value as the final loss function value, and training the multispectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the huanglongbing monitoring module based on the classifier through backpropagation of gradient descent.

[0060] In the embodiments of the present application, the training citrus leaf shallow differential feature map, the training citrus leaf middle differential feature map, and the training citrus leaf semantic differential feature map respectively represent the cross-scale and cross-depth image differential semantic features of the reference training multispectral image of citrus leaves labeled as healthy and the training multispectral image of the detected citrus leaves. Therefore, the difference in their cross-scale and cross-depth image differential semantic features will be further enlarged due to the difference in the gated screening adaptive attention weights, making the fused training citrus leaf global multi-dimensional significant differential feature map also have the complexity of image semantic distribution knowledge corresponding to the difference in cross-scale and cross-depth image differential semantic features. Therefore, it is expected to enhance the accuracy of the classification result for the complex feature distribution expression characteristics of the training citrus leaf global multi-dimensional significant differential feature map.

[0061] Based on this, the present application further introduces a loss function for promoting classification understanding of the complex feature expression of the training citrus leaf global multi-dimensional significant differential feature map. Based on the training citrus leaf global multi-dimensional significant differential feature map, the citrus leaf global multi-dimensional significant differential loss function value is calculated. Among them, by first calculating the mean and variance between the feature values at any two positions of the training citrus leaf global multi-dimensional significant differential feature vector obtained after unfolding the training citrus leaf global multi-dimensional significant differential feature map to obtain the mean weight matrix and the variance weight matrix, and then multiplying the training citrus leaf global multi-dimensional significant differential feature vector with the mean weight matrix and the variance weight matrix respectively in a query-like manner to obtain the mean feature vector and the variance feature vector, and then calculating the norm of the self-correlation matrix of the obtained mean feature vector and variance feature vector, and subtracting the product of the norm of the self-correlation matrix of the training citrus leaf global multi-dimensional significant differential feature vector and the weight as a hyperparameter to obtain the citrus leaf global multi-dimensional significant differential loss function value.

[0062] For example, it is expressed as:

[0063]

[0064] Among them, represents the training citrus leaf global multi-dimensional significant differential feature vector, is the mean weight matrix, is the variance weight matrix, represents the transpose of the vector, represents the matrix multiplication operation, is the weight as a hyperparameter, represents the norm of the matrix, represents the citrus leaf global multi-dimensional significant differential loss function value. And as described above:

[0065]

[0066] Among them, represents the th eigenvalue of the globally multi-dimensional significant differential feature vector of the training citrus leaf, represents the th eigenvalue of the globally multi-dimensional significant differential feature vector of the training citrus leaf, represents the eigenvalue at the position of the mean weight matrix, represents the eigenvalue at the position of the variance weight matrix.

[0067] That is, on the basis of determining the overall distributed structure bottleneck of the training citrus leaf globally multi-dimensional significant differential feature vector through the multi-dimensional interaction of the detail group of the training citrus leaf globally multi-dimensional significant differential feature vector, the globally multi-dimensional significant differential loss function value of the citrus leaf is used to associatively simulate the correlation between the structural details and the macro-structural feature representation behavior of the training citrus leaf globally multi-dimensional significant differential feature vector through the low-rank structured inference of the training citrus leaf globally multi-dimensional significant differential feature vector. In this way, by training a classifier with the globally multi-dimensional significant differential loss function value of the citrus leaf, the structural detail dependence with respect to the macro complexity can be modeled in the classification scenario, thereby promoting the understanding of the macro complex feature expression based on the structural details of the training citrus leaf globally multi-dimensional significant differential feature map and improving the accuracy of the classification result.

[0068] In summary, the method for monitoring citrus huanglongbing based on drone multispectral of the embodiments of the present application is elucidated. It collects the multispectral images of the detected citrus leaves by the multispectral camera carried by the drone, extracts the reference multispectral images of the citrus leaves labeled as healthy from the database, and uses the analysis and processing technology of deep learning-based image monitoring to perform image feature extraction and differential calculation on the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves, so as to automatically determine whether the detected citrus leaves have huanglongbing according to the multi-dimensional significant differential features between the multispectral images of the detected citrus leaves and the reference multispectral images of the citrus leaves. By this means, the data can be collected in real time and quickly by the multispectral camera carried by the drone, and the huanglongbing characteristics of the citrus leaves can be identified more accurately, so as to take timely measures to prevent the spread of the disease.

[0069] The foregoing are merely examples of the principles of the present disclosure, and those skilled in the art can make various modifications without departing from the scope of the present disclosure. The above embodiments are presented for illustrative purposes rather than for limitation. The present disclosure can also take many forms other than those explicitly described herein. Therefore, it should be emphasized that the present disclosure is not limited to the explicitly disclosed methods, systems, and devices, but is intended to include variations and modifications within the spirit of the appended claims.

Claims

1. A method for monitoring citrus huanglongbing based on drone multispectral, characterized in that, Including: Obtain the multispectral image of the citrus leaf to be detected collected by the multispectral camera carried by the drone; Extract the reference multispectral image of the citrus leaf labeled as healthy from the database; Perform multispectral multi-scale feature extraction on the reference multispectral image of the citrus leaf labeled as healthy and the multispectral image of the citrus leaf to be detected to obtain the shallow feature map for citrus leaf detection, the middle feature map for citrus leaf detection, the deep feature map for citrus leaf detection, the shallow feature map for citrus leaf reference, the middle feature map for citrus leaf reference, and the deep feature map for citrus leaf reference; Calculate the differential feature map between the shallow feature map for citrus leaf detection and the shallow feature map for citrus leaf reference, the differential feature map between the middle feature map for citrus leaf detection and the middle feature map for citrus leaf reference, and the differential feature map between the deep feature map for citrus leaf detection and the deep feature map for citrus leaf reference to obtain the shallow differential feature map for citrus leaf, the middle differential feature map for citrus leaf, and the semantic differential feature map for citrus leaf; Pass the shallow differential feature map for citrus leaf, the middle differential feature map for citrus leaf, and the semantic differential feature map for citrus leaf through the gated screening adaptive attention module respectively to obtain the adaptively enhanced shallow differential feature map for citrus leaf, the adaptively enhanced middle differential feature map for citrus leaf, and the adaptively enhanced semantic differential feature map for citrus leaf; Fuse the adaptively enhanced shallow differential feature map for citrus leaf, the adaptively enhanced middle differential feature map for citrus leaf, and the adaptively enhanced semantic differential feature map for citrus leaf to obtain the global multi-dimensional significant differential feature map for citrus leaf as the global multi-dimensional significant differential feature of citrus leaf; Based on the global multi-dimensional significant differential feature of citrus leaf, obtain the monitoring result, and the monitoring result is used to indicate whether the citrus leaf to be detected has huanglongbing; Perform multispectral multi-scale feature extraction on the reference multispectral image of the citrus leaf labeled as healthy and the multispectral image of the citrus leaf to be detected to obtain the shallow feature map for citrus leaf detection, the middle feature map for citrus leaf detection, the deep feature map for citrus leaf detection, the shallow feature map for citrus leaf reference, the middle feature map for citrus leaf reference, and the deep feature map for citrus leaf reference, including: input the reference multispectral image of the citrus leaf labeled as healthy and the multispectral image of the citrus leaf to be detected into the multispectral feature extractor based on the dilated convolutional neural network model to obtain the shallow feature map for citrus leaf detection, the middle feature map for citrus leaf detection, the deep feature map for citrus leaf detection, the shallow feature map for citrus leaf reference, the middle feature map for citrus leaf reference, and the deep feature map for citrus leaf reference; Pass the shallow differential feature map for citrus leaf, the middle differential feature map for citrus leaf, and the semantic differential feature map for citrus leaf through the gated screening adaptive attention module respectively to obtain the adaptively enhanced shallow differential feature map for citrus leaf, the adaptively enhanced middle differential feature map for citrus leaf, and the adaptively enhanced semantic differential feature map for citrus leaf, including: Perform pooling processing on each feature matrix of the shallow differential feature map of the citrus leaf along the channel dimension in multiple ways to obtain the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf; Perform weighted fusion on the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf to obtain the global representation vector of the shallow differential of the citrus leaf; Perform per-channel semantic feature interaction and feature activation based on the fully connected layer on the global representation vector of the shallow differential of the citrus leaf to obtain the per-channel semantic feature vector of the shallow differential of the citrus leaf; Input the per-channel semantic feature vector of the shallow differential of the citrus leaf into the gating unit to obtain the gated and screened per-channel semantic feature vector of the shallow differential of the citrus leaf; Perform normalization processing on the gated and screened per-channel semantic feature vector of the shallow differential of the citrus leaf to obtain the gated and screened per-channel semantic weight feature vector of the shallow differential of the citrus leaf; Use the gated and screened per-channel semantic weight feature vector of the shallow differential of the citrus leaf as the weight feature vector, and calculate the per-channel product of it and the shallow differential feature map of the citrus leaf to obtain the adaptively enhanced shallow differential feature map of the citrus leaf.

2. The citrus huanglongbing monitoring method based on drone multispectral according to claim 1, characterized in that, Perform pooling processing on each feature matrix of the shallow differential feature map of the citrus leaf along the channel dimension in multiple ways to obtain the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf, including: performing global pooling processing based on the maximum value, global pooling processing based on the average value, and global pooling processing based on the random value on each feature matrix of the shallow differential feature map of the citrus leaf along the channel dimension to obtain the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf.

3. The citrus huanglongbing monitoring method based on drone multi-spectral according to claim 2, wherein Perform weighted fusion on the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf to obtain the global representation vector of the shallow differential of the citrus leaf, including: Multiply the global maximum pooling feature vector of the shallow differential of the citrus leaf, the global mean pooling feature vector of the shallow differential of the citrus leaf, and the global random value pooling feature vector of the shallow differential of the citrus leaf with the corresponding adjustment parameters respectively in a position-wise manner to obtain the globally maximum-modulated pooling feature vector of the shallow differential of the citrus leaf, the globally mean-modulated pooling feature vector of the shallow differential of the citrus leaf, and the globally random-value-modulated pooling feature vector of the shallow differential of the citrus leaf; Add the citrus leaf shallow differential global maximum modulation pooling feature vector, the citrus leaf shallow differential global mean modulation pooling feature vector, and the citrus leaf shallow differential global random value modulation pooling feature vector by position to obtain the citrus leaf shallow differential global representation vector.

4. The citrus huanglongbing monitoring method based on drone multispectral according to claim 3, characterized in that Perform per-channel semantic feature interaction and feature activation based on a fully connected layer on the citrus leaf shallow differential global representation vector to obtain the citrus leaf shallow differential channel semantic feature vector, including: Calculate the matrix multiplication of the citrus leaf shallow differential global representation vector and the weight matrix and then perform vector addition with the bias vector to obtain the bias-adjusted citrus leaf shallow differential global representation vector; Activate the bias-adjusted citrus leaf shallow differential global representation vector through the sigmoid activation function to obtain the citrus leaf shallow differential channel semantic feature vector.

5. The citrus huanglongbing monitoring method based on drone multispectral according to claim 4, characterized in that Input the citrus leaf shallow differential channel semantic feature vector into a gated unit to obtain the gated-filtered citrus leaf shallow differential channel semantic feature vector, including: in response to each position feature value in the citrus leaf shallow differential channel semantic feature vector being greater than or equal to a predetermined threshold, the position feature value takes the original value, otherwise it is zero, to obtain the gated-filtered citrus leaf shallow differential channel semantic feature vector.

6. The citrus huanglongbing monitoring method based on multi-spectral of unmanned aerial vehicle according to claim 5, wherein, Based on the citrus leaf global multi-dimensional significant differential features, obtain a monitoring result, including: passing the citrus leaf global multi-dimensional significant differential feature map through a huanglongbing monitoring module based on a classifier to obtain the monitoring result.

7. The citrus huanglongbing monitoring method based on drone multispectral according to claim 6, characterized in that Also includes a training step: for training the multi-spectral feature extractor based on the dilated convolutional neural network model, the gated-filtered adaptive attention module, and the huanglongbing monitoring module based on the classifier.

8. The citrus huanglongbing monitoring method based on drone multispectral according to claim 7, characterized in that, The training step includes: Obtain training data, where the training data includes training multi-spectral images of the detected citrus leaves collected by a multi-spectral camera carried by a drone, extract reference training multi-spectral images of citrus leaves labeled as healthy from a database, and a true monitoring result, where the true monitoring result is the true value of whether the detected citrus leaves have huanglongbing; Input the reference training multi-spectral images of citrus leaves labeled as healthy and the training multi-spectral images of the detected citrus leaves into the multi-spectral feature extractor based on the dilated convolutional neural network model to obtain a training citrus leaf detection shallow feature map, a training citrus leaf detection middle feature map, a training citrus leaf detection deep feature map, a training citrus leaf reference shallow feature map, a training citrus leaf reference middle feature map, and a training citrus leaf reference deep feature map; Calculate the differential feature maps between the training citrus leaf detection shallow feature map and the training citrus leaf reference shallow feature map, the differential feature maps between the training citrus leaf detection middle feature map and the training citrus leaf reference middle feature map, and the differential feature maps between the training citrus leaf detection deep feature map and the training citrus leaf reference deep feature map to obtain the training citrus leaf shallow differential feature map, the training citrus leaf middle differential feature map, and the training citrus leaf semantic differential feature map; Pass the training citrus leaf shallow differential feature map, the training citrus leaf middle differential feature map, and the training citrus leaf semantic differential feature map through the gated screening adaptive attention module respectively to obtain the training adaptive enhanced citrus leaf shallow differential feature map, the training adaptive enhanced citrus leaf middle differential feature map, and the training adaptive enhanced citrus leaf semantic differential feature map; Fuse the training adaptive enhanced citrus leaf shallow differential feature map, the training adaptive enhanced citrus leaf middle differential feature map, and the training adaptive enhanced citrus leaf semantic differential feature map to obtain the training citrus leaf global multi-dimensional significant differential feature map; Pass the training citrus leaf global multi-dimensional significant differential feature map through the Huanglongbing monitoring module based on the classifier to obtain the classification loss function value; Based on the training citrus leaf global multi-dimensional significant differential feature map, calculate the citrus leaf global multi-dimensional significant differential loss function value; Use the weighted sum of the classification loss function value and the citrus leaf global multi-dimensional significant differential loss function value as the final loss function value, and train the multi-spectral feature extractor based on the dilated convolutional neural network model, the gated screening adaptive attention module, and the Huanglongbing monitoring module based on the classifier through backpropagation of gradient descent.

Images