Remote sensing monitoring method and device for crop diseases and insect pests and electronic equipment

Abstract

The invention provides a crop disease and insect pest remote sensing monitoring method and device and electronic equipment, and the method comprises the steps: carrying out the image collection of farmland crops through remote sensing equipment, and carrying out the preprocessing, and obtaining a visible light subimage and a vegetation index subimage; constructing a double-flow deep network based on cross-modal spectrogram interaction; the double-flow deep network comprises a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectrogram feature interaction module, a cross-modal spectrogram feature fusion module and a double-task processing module; and inputting the visible light subimage and the vegetation index subimage into the double-flow depth network to obtain a crop disease and insect pest distribution diagram in the area range of the farmland crops. According to the method and the device, self-adaptive interaction and self-adaptive fusion can be carried out on the cross-modal interaction characteristics extracted from different deep network levels, so that various signs of diseases and insect pests are more comprehensively captured, and the problem that cross-modal information interaction monitoring is difficult to carry out in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of agricultural remote sensing monitoring technology, and in particular to a method, device and electronic equipment for remote sensing monitoring of crop diseases and pests. Background Technology

[0002] Traditional remote sensing monitoring methods for crop diseases and pests mainly acquire visible light and spectral images from UAV or satellite remote sensing imagery, extract color, texture, and vegetation index features that characterize diseases and pests, and input these features into machine learning models for qualitative or quantitative monitoring of diseases and pests. These methods rely on manual feature extraction engineering and all use point source data, neglecting spatial correlation and the information complementarity between different modalities.

[0003] In recent years, deep learning algorithms have been widely used in remote sensing monitoring of crop diseases and pests due to their reduced reliance on prior knowledge and effective feature extraction. While existing methods and systems have avoided the complexities of manual feature extraction to some extent and have fused spectral and image information, they still fail to consider the information interaction between cross-modal features and are mostly single-task models, which cannot meet the actual needs of multiple scenarios or personnel.

[0004] Chinese invention patent CN120599500A discloses an intelligent monitoring method for cotton aphid infestation based on multi-source remote sensing fusion data. While it integrates multispectral and visible light images, it still relies on traditional vegetation index point source data for pest monitoring, failing to consider the spatial correlation of pest occurrence in farmland areas and only capable of monitoring the severity of aphid infestation. Another Chinese invention patent, CN120599500A, discloses an intelligent pest monitoring system based on UAV multimodal data fusion. It primarily utilizes the YOLO algorithm for feature extraction from UAV multimodal data and performs decision-level fusion, improving the spatial correlation of features. However, it still fails to achieve effective cross-modal feature interaction, and the accuracy and effectiveness of feature extraction need improvement. Furthermore, it can only classify pests and diseases.

[0005] There is currently no effective solution to the problem that existing technologies struggle to monitor cross-modal information interaction. Summary of the Invention

[0006] This invention provides a method, device, and electronic equipment for remote sensing monitoring of crop diseases and pests, which addresses the shortcomings of existing related technologies in conducting cross-modal information interaction monitoring and enables qualitative and quantitative monitoring of crop diseases and pests via remote sensing.

[0007] In a first aspect, the present invention provides a remote sensing monitoring method for crop diseases and pests, comprising: Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images. A two-stream deep network based on cross-modal spectral graph interaction is constructed; the two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module; The visible photon image and the vegetation index sub-image are input into the dual-stream depth network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located.

[0008] According to the present invention, a remote sensing monitoring method for crop diseases and pests is provided, which acquires images of farmland crops using remote sensing equipment and performs preprocessing to obtain visible photon images and vegetation index sub-images, including: Visible light and multispectral images of the farmland crops were collected using drones or satellite remote sensing equipment. Geometric correction, radiometric correction, vegetation index calculation, and segmented output are performed on the visible light image and the multispectral image to obtain the visible photon image and the vegetation index sub-image.

[0009] According to the remote sensing monitoring method for crop diseases and pests provided by the present invention, the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork each include the same number of residual modules; The residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork are respectively connected to a cross-modal spectral image feature interaction module; any cross-modal spectral image feature interaction module and the residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork connected to it constitute an interaction layer; the dual-stream deep network includes multiple interaction layers to realize adaptive interaction of semantic information and spatial information; The cross-modal spectrogram feature interaction module is connected to a cross-modal spectrogram feature fusion module, which is used to perform feature fusion on the semantic and spatial information transmitted at different interaction levels.

[0010] According to the remote sensing monitoring method for crop diseases and pests provided by the present invention, the visible photon image and the vegetation index sub-image are input into the dual-stream depth network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located, including: The visible photon image and the vegetation index image are respectively input into the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork to obtain the visible light image depth features and the vegetation index image depth features. The visible light image depth features and the vegetation index image depth features are input into the cross-modal spectral map feature interaction module to obtain a cross-modal spectral map interaction feature map; The cross-modal spectral map interaction feature map is input into the cross-modal spectral map feature fusion module to obtain the cross-modal spectral map fusion feature map; The cross-modal spectral map fusion feature map is input into the dual-task processing module to construct a dual-task loss function, update the model parameters of the dual-stream deep network, and output a crop disease and pest distribution map within the area where the farmland crops are located.

[0011] According to a remote sensing monitoring method for crop diseases and pests provided by the present invention, the depth features of the visible light image and the depth features of the vegetation index image are input into the cross-modal spectral map feature interaction module to obtain a cross-modal spectral map interaction feature map, including: Channel attention weights are applied to the visible light image depth features and the vegetation index image depth features respectively to obtain visible light image channel weighted features and vegetation index image channel weighted features; The weighted features of the visible light image channels are transposed and multiplied with the weighted features of the vegetation index image channels to obtain a cross-modal semantic feature map. The visible light image depth features and the vegetation index image depth features are multiplied by the cross-modal semantic feature map, and then cross-modal feature interaction in the channel dimension is performed through the softmax activation function to obtain the visible light image semantic interaction feature map and the vegetation index image semantic interaction feature map. Spatial attention weights are applied to the semantic interaction feature map of the visible light image and the semantic interaction feature map of the vegetation index image, respectively, to obtain the spatially weighted features of the visible light image and the spatially weighted features of the vegetation index image. The spatial weighted features of the visible light image and the spatial weighted features of the vegetation index image are added together to obtain a cross-modal spatial attention feature map; The semantic interaction feature map of the visible light image and the semantic interaction feature map of the vegetation index image are multiplied by the cross-modal spatial attention feature map, and then cross-modal feature interaction in the spatial dimension is performed through the softmax activation function to obtain the semantic-spatial interaction feature map of the visible light and vegetation index images. The semantic-spatial interaction feature maps are spliced ​​along the channel direction to obtain a cross-modal spectral interaction feature map.

[0012] According to a remote sensing monitoring method for crop diseases and pests provided by the present invention, the cross-modal spectral map interaction feature map is input into the cross-modal spectral map feature fusion module to obtain a cross-modal spectral map fusion feature map, including: Adaptive average pooling is performed on the cross-modal spectral interaction feature maps of each interaction level input, and weight factors for different cross-modal spectral interaction feature maps are generated. The cross-modal spectral interaction feature maps of different interaction levels are weighted and fused using the weighting factors to obtain the cross-modal spectral fusion feature map.

[0013] According to a remote sensing monitoring method for crop diseases and pests provided by the present invention, the dual-task processing module includes a classifier and a regressor, wherein the classifier and the regressor are each composed of the same number of multilayer perceptrons. The dual-task loss function includes a classification loss function and a regression loss function.

[0014] According to the remote sensing monitoring method for crop diseases and pests provided by the present invention, the crop disease and pest distribution map includes a distribution map of disease and pest occurrence categories and a distribution map of disease and pest indexes output in sections for the area where the farmland crops are located.

[0015] Secondly, the present invention also provides a remote sensing monitoring device for crop diseases and pests, comprising: The acquisition module is used to acquire images of farmland crops through remote sensing equipment and perform preprocessing to obtain visible photon images and vegetation index sub-images. A construction module is used to build a two-stream deep network based on cross-modal spectral graph interaction; the two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module; The monitoring module is used to input the visible photon image and the vegetation index sub-image into the dual-stream depth network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located.

[0016] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the remote sensing monitoring method for crop diseases and pests as described in the first aspect above.

[0017] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing monitoring method for crop diseases and pests as described in the first aspect above.

[0018] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the remote sensing monitoring method for crop diseases and pests as described in the first aspect above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. Traditional methods typically only perform simple stitching or decision-level fusion of visible light and vegetation index image features, failing to deeply explore the intrinsic correlations between modalities. This invention, through a cross-modal spectral feature interaction module, achieves information exchange between visible light image features and vegetation index image features in both channel and spatial dimensions, promoting the feature expression of crop diseases and pests. It refines spectral features using spatial regions, suppresses spatial noise unrelated to the spectrum, and focuses on the occurrence of diseases and pests in spatial regions using spectral features. This refined interaction mechanism allows the model to fully utilize the rich texture and shape information of visible light images and the crop physiological state information reflected by the vegetation index, forming information complementarity and enhancing the model's perceptual capabilities.

[0020] 2. This invention employs a cross-modal spectral feature fusion module to adaptively weight and fuse cross-modal interaction features extracted from different network depths. This module automatically learns and assigns higher weights to more informative feature layers, effectively combining shallow detail information with deep semantic information. This multi-scale fusion strategy enables the model to more comprehensively capture various signs of pests and diseases, and is particularly valuable for monitoring early and mild pest and disease outbreaks.

[0021] 3. Most existing technologies can only complete a single task in classifying the "presence" (qualitative) or assessing the "severity" (quantitative) of pests and diseases, failing to meet the comprehensive understanding of pests and diseases required in agricultural production. This invention innovatively introduces a dual-task processing module, using parallel classifiers and regressors to simultaneously output pest and disease category labels and severity indices within the same model. This not only simplifies the monitoring process and improves efficiency, but more importantly, the qualitative and quantitative tasks mutually promote and constrain each other during feature learning. This ensures that the features learned by the model possess both inter-class discriminative power and reflect the continuity of intra-class severity, thus achieving superior accuracy and robustness compared to independent models in both tasks.

[0022] 4. This invention ultimately yields a thematic atlas containing distribution maps of pest and disease occurrence categories and pest and disease index distribution maps. These distribution maps intuitively and clearly demonstrate the spatial distribution differences of crop pests and diseases in farmland. Farmers or agricultural technicians can use this information to formulate precise control measures such as spraying and fertilization, achieving the goals of reducing pesticide use while increasing efficiency and ensuring the yield and quality of agricultural products. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the remote sensing monitoring method for crop diseases and pests provided by the present invention; Figure 2 This is a schematic diagram of the structure of the two-stream deep network in an embodiment of the present invention; Figure 3 This is a structural block diagram of the remote sensing monitoring device for crop diseases and pests provided by the present invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] This invention provides a remote sensing monitoring method for crop diseases and pests. Figure 1 This is a flowchart of the remote sensing monitoring method for crop diseases and pests provided by the present invention, such as... Figure 1 As shown, the method includes the following steps: Step S101: Collect images of farmland crops using remote sensing equipment and perform preprocessing to obtain visible photon images and vegetation index sub-images. Step S102: Construct a two-stream deep network based on cross-modal spectral graph interaction; the two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module; Step S103: Input the visible photon image and the vegetation index sub-image into the dual-stream depth network to obtain the distribution map of crop diseases and pests within the area where the farmland crops are located.

[0027] In this method, firstly, images of farmland crops are acquired and preprocessed using remote sensing equipment to obtain visible photon images and vegetation index sub-images. Then, a two-stream deep network based on cross-modal spectral image interaction is constructed. The visible photon images and vegetation index sub-images are input into the two-stream deep network for processing. The cross-modal spectral image feature interaction module achieves information exchange between the visible photon images and vegetation index sub-images in both channel and spatial dimensions, promoting the feature expression of crop diseases and pests. Furthermore, the cross-modal spectral image feature fusion module can interactively and adaptively fuse cross-modal interaction features extracted from different network depth levels, thereby more comprehensively capturing various signs of diseases and pests, solving the problem that existing related technologies struggle to perform cross-modal information interaction monitoring.

[0028] In some embodiments, step S101 involves acquiring images of farmland crops using remote sensing equipment and preprocessing them to obtain visible photon images and vegetation index sub-images. This includes: acquiring visible light images and multispectral images of farmland crops using drones or satellite remote sensing equipment; performing geometric correction, radiometric correction, vegetation index calculation, and segmented output on the visible light images and multispectral images to obtain visible photon images and vegetation index sub-images.

[0029] For example, the visible light image bands include green (G) at 560±16 nm, red (R) at 650±16 nm, red edge (RE) at 730±16 nm, and near-infrared (NIR) at 860±26 nm. Vegetation index calculations include, but are not limited to, the Difference Vegetation Index (DVI), Normalized Difference Vegetation Index (NDIV), Optimized Soil Adjusted Vegetation Index (OSAVI), Simplified Canopy Chlorophyll Content Index (SCCCI), Modified Nonlinear Index (MNLI), and Triangle Vegetation Index (TVI), calculated using the following formulas:

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] in, Indicates the differential vegetation index, Indicates the spectral reflectance in the near-infrared band. Indicates the spectral reflectance in the red band. Represents the normalized difference vegetation index. This indicates that optimizing the soil adjusts the vegetation index. This represents a simplified canopy chlorophyll content index. Indicates the spectral reflectance in the red-edge band. This indicates a correction for the nonlinear exponent. Indicates the triangular vegetation index, This represents the spectral reflectance in the green band.

[0036] In some embodiments, the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork each include the same number of residual modules; the residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork are respectively connected to a cross-modal spectrogram feature interaction module; any of the cross-modal spectrogram feature interaction modules and the residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork connected to it constitute an interaction layer; the two-stream deep network includes multiple interaction layers for adaptive interaction of semantic and spatial information; the cross-modal spectrogram feature interaction module is connected to a cross-modal spectrogram feature fusion module, which is used to perform feature fusion on the semantic and spatial information transmitted by the different interaction layers. Further, the interaction layers can be divided into low-level layers and high-level layers, with low-level layers focusing more on spatial information interaction and high-level layers focusing more on semantic information interaction.

[0037] Traditional two-stream deep networks lack cross-modal interaction and dual-task processing. This embodiment designs a cross-modal spectral feature interaction module and a cross-modal spectral feature fusion module based on the feature differences between visible light images and vegetation index images. Furthermore, a dual-task processing module is designed to address the needs of qualitative and quantitative monitoring of crop diseases and pests.

[0038] Figure 2 This is a schematic diagram of the structure of a two-stream deep network in an embodiment of the present invention, as shown below. Figure 2 As shown, based on this, a two-stream deep network based on cross-modal spectral graph interaction includes at least: Convolutional block 1 and convolutional block 2 are used to normalize the size of the visible light image and the vegetation index image, respectively, so as to input the subsequent residual block to extract depth features; Residual blocks 3, 6, 9, and 12 are used to extract depth features from visible light images at different levels, respectively. Residual blocks 5, 8, 11, and 14 are used to extract depth features from vegetation index images at different levels, respectively. Cross-modal feature interaction modules 4, 7, 10, and 13 are used to extract cross-modal interaction features between the visible light image depth features and the vegetation index image depth features obtained from residual blocks 3 and 5, residual blocks 6 and 8, residual blocks 9 and 11, and residual blocks 12 and 14, respectively. The cross-modal feature fusion module 15 is used to adaptively fuse the cross-modal interaction features extracted by the cross-modal feature interaction modules 4, 7, 10, and 13 at various levels. The dual-task processing module 16 is used to further analyze the cross-modal fusion features extracted by the cross-modal feature fusion module 15, and simultaneously output the category and index of crop diseases and pests.

[0039] For example, refer to Figure 2 Residual block 3, cross-modal feature interaction module 4, and residual block 5 constitute an interaction layer, denoted as the first layer. Residual block 6, cross-modal feature interaction module 7, and residual block 8 constitute an interaction layer, denoted as the second layer, and so on, including a third and fourth layer. In this embodiment, the first and second layers are lower-level layers, focusing more on realizing the interaction of spatial information, while the third and fourth layers are higher-level layers, focusing more on realizing the interaction of semantic information. Furthermore, all four layers are connected to the cross-modal feature fusion module 15, which fuses semantic and spatial information.

[0040] In this embodiment, the dual-stream deep network enriches the feature representation of crop diseases and pests through multi-level cross-modal feature interaction and fusion, further improving the model's feature extraction and fusion capabilities, thereby enhancing the model's monitoring accuracy.

[0041] Based on this, in step S103, the visible photon image and the vegetation index sub-image are input into the dual-stream deep network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located. This includes: inputting the visible photon image and the vegetation index sub-image into the visible light image feature extraction sub-network and the vegetation index image feature extraction sub-network, respectively, to obtain the visible light image depth features and the vegetation index image depth features; inputting the visible light image depth features and the vegetation index image depth features into the cross-modal spectral map feature interaction module to obtain the cross-modal spectral map interaction feature map; inputting the cross-modal spectral map interaction feature map into the cross-modal spectral map feature fusion module to obtain the cross-modal spectral map fusion feature map; inputting the cross-modal spectral map fusion feature map into the dual-task processing module to construct the dual-task loss function, update the model parameters of the dual-stream deep network, and output the distribution map of crop diseases and pests within the area where the farmland crops are located.

[0042] For example, a 3-channel RGB visible light sub-image and a 6-channel vegetation index sub-image (DVI, NDVI, OSAVI, SCCCI, MNLI, and TVI vegetation index images stacked along the channel direction) are respectively input into the visible light image feature extraction sub-network and the vegetation index image feature extraction sub-network. The visible light image feature extraction sub-network and the vegetation index image feature extraction sub-network each include four residual modules to achieve depth feature extraction from the visible light image and the vegetation index image.

[0043] Specifically, the depth features of the visible light image and the depth features of the vegetation index image are input into the cross-modal spectral map feature interaction module to obtain a cross-modal spectral map interaction feature map. This includes: applying channel attention weights to the depth features of the visible light image and the depth features of the vegetation index image respectively to obtain the channel-weighted features of the visible light image and the channel-weighted features of the vegetation index image. The calculation formula is as follows:

[0044] in, Indicates channel weighting characteristics, This represents the Sigmod activation function. Represents convolution. This indicates max pooling. Indicates average pooling. Represents the input feature map; The channel-weighted features of the visible light image are transposed and multiplied with the channel-weighted features of the vegetation index image to obtain a cross-modal semantic feature map. The depth features of the visible light image and the depth features of the vegetation index image are then multiplied with the cross-modal semantic feature map, and the channels are interacted using a softmax activation function to obtain the visible light image semantic interaction feature map and the vegetation index image semantic interaction feature map. Spatial attention weights are then applied to the visible light image semantic interaction feature map and the vegetation index image semantic interaction feature map respectively to obtain the spatially weighted visible light image. The image spatial weighted features of the visible light image and the vegetation index image are summed to obtain a cross-modal spatial attention feature map. The semantic interaction feature map of the visible light image and the semantic interaction feature map of the vegetation index image are multiplied by the cross-modal spatial attention feature map, and then subjected to a softmax activation function to perform cross-modal feature interaction in the spatial dimension, resulting in a semantic-spatial interaction feature map of the visible light and vegetation index images. The semantic-spatial interaction feature maps are then concatenated along the channel direction to obtain a cross-modal spectral interaction feature map.

[0045] The cross-modal spectral map interaction feature map is input into the cross-modal spectral map feature fusion module to obtain the cross-modal spectral map fusion feature map. This includes: performing adaptive average pooling on all input cross-modal spectral map interaction feature maps at each interaction level, and generating weight factors for different cross-modal spectral map interaction feature maps; and using the weight factors of different interaction levels to perform weighted fusion of the cross-modal spectral map interaction feature maps to obtain the cross-modal spectral map fusion feature map. The calculation formula is as follows:

[0046]

[0047]

[0048] in, and This represents the feature map obtained after adaptive average pooling. Indicates adaptive average pooling. Represents the cross-modal spectral graph interaction feature map. Indicates the weighting factor. This represents the cross-modal spectral fusion feature map. This indicates cascading fusion.

[0049] In some embodiments, the dual-task processing module includes a classifier and a regressor, each composed of the same number of multilayer perceptrons; the dual-task loss function includes a classification loss function and a regression loss function.

[0050] For example, the formula for calculating the classification loss function is as follows:

[0051] in, Represents the classification loss function. Indicates category label as y m The predicted probability, This indicates adjustable hyperparameters. Indicates the category label index. Indicates the number of category labels.

[0052] The formula for calculating the regression loss function is as follows:

[0053] in, Represents the regression loss function. Represents the true value of the sample. Indicates the sample predicted value. This indicates adjustable hyperparameters. Indicates the sample index. Indicates the number of samples.

[0054] The formula for calculating the model parameters update of a two-stream deep network is as follows:

[0055] in, and These represent the number of iterations respectively. He Shi Model parameter variables, Indicates the parameters of the historical model. This indicates adjustable hyperparameters.

[0056] In some embodiments, the crop disease and pest distribution map includes a disease and pest occurrence category distribution map and a disease and pest index distribution map output in sections for the area where the farmland crops are located.

[0057] In the above process, the complementary information between RGB and vegetation index data was deeply explored through cross-modal feature interaction, which enhanced the feature representation; through cross-modal feature fusion, the information transmission of spatial and semantic features at different levels was enhanced, and multimodal features representing the occurrence of pests and diseases were adaptively fused, which further improved the model accuracy; in addition, the qualitative and quantitative monitoring of crop pests and diseases was realized by using a dual-task processing module, which met the needs of different personnel and task scenarios and can provide precise guidance for agricultural production.

[0058] To verify the effectiveness of the above method, the following simulation experiments were conducted. The experiments were performed on a computer running Windows 11 64-bit, equipped with an Intel(R) Core(TM) Ultra 7 265K CPU and an NVIDIA RTX 5070 Ti GPU. Five dual-stream deep networks were selected for comparison: Dual-CNN, Dual-ResNet50, CNN-ResNet50, CNN-ResNet18, and Dual-ResNet18. To ensure consistency, all models were trained for 300 epochs with a batch size of 32. An adaptive moment estimation with a weight decay optimizer and a learning rate scheduler were used, with an initial learning rate of 0.001 and a weight decay factor of 0.0001. A unified set of qualitative and quantitative evaluation metrics was used: accuracy (Accuracy, Acc), F1 score (F1), Kappa coefficient (Kappa), coefficient of determination (R2), root mean square error of prediction (RMSEP), and residual prediction deviation (RPD). The dual-stream deep network based on cross-modal spectral graph interaction, denoted as CMDBNet, was proposed in this embodiment and evaluated against five other dual-stream deep networks on a self-built UAV remote sensing dataset of cotton aphid infestation. The experimental results are shown in Table 1.

[0059] Table 1 Experimental results of networks with different two-stream depths

[0060] As can be seen from Table 1, the proposed method, a two-stream deep network based on cross-modal spectral graph interaction (CMDBNet), outperforms other mainstream two-stream deep networks in all metrics, demonstrating its superiority.

[0061] This invention provides a remote sensing monitoring device for crop diseases and pests. The following is a description of the remote sensing monitoring device for crop diseases and pests provided by this invention. The remote sensing monitoring device for crop diseases and pests described below can be referred to in correspondence with the remote sensing monitoring method for crop diseases and pests described above. Figure 3 This is a structural block diagram of the remote sensing monitoring device for crop diseases and pests provided by the present invention, as shown below. Figure 3 As shown, the device includes: The acquisition module 301 is used to acquire images of farmland crops through remote sensing equipment and perform preprocessing to obtain visible photon images and vegetation index sub-images. Module 302 is used to construct a two-stream deep network based on cross-modal spectral graph interaction. The two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module. The monitoring module 303 is used to input visible photon images and vegetation index sub-images into a two-stream depth network to obtain a distribution map of crop diseases and pests within the area where farmland crops are located.

[0062] In operation, this device first acquires and preprocesses images of farmland crops using the acquisition module 301, obtaining visible photon images and vegetation index sub-images. Then, the construction module 302 constructs a two-stream deep network based on cross-modal spectral image interaction. The monitoring module 303 inputs the visible photon images and vegetation index sub-images into the two-stream deep network for processing. The cross-modal spectral image feature interaction module within this network achieves information exchange between the visible photon images and the vegetation index sub-images in both channel and spatial dimensions, promoting the feature expression of crop diseases and pests. Furthermore, the cross-modal spectral image feature fusion module can adaptively weight and fuse cross-modal interaction features extracted from different network depth levels, thereby more comprehensively capturing various signs of diseases and pests, solving the problem that existing related technologies struggle to perform cross-modal information interaction monitoring.

[0063] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other via the communication bus 404. The processor 401 can call logical instructions in the memory 403 to execute a remote sensing monitoring method for crop diseases and pests, the method including: Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images. A two-stream deep network based on cross-modal spectral graph interaction is constructed. The two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module. By inputting visible photon images and vegetation index sub-images into a two-stream deep network, a distribution map of crop diseases and pests within the area where farmland crops are located is obtained.

[0064] Furthermore, the logical instructions in the aforementioned memory 403 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the crop disease and pest remote sensing monitoring method provided by the above methods, the method comprising: Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images. A two-stream deep network based on cross-modal spectral graph interaction is constructed. The two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module. By inputting visible photon images and vegetation index sub-images into a two-stream deep network, a distribution map of crop diseases and pests within the area where farmland crops are located is obtained.

[0066] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing monitoring method for crop diseases and pests provided by the methods described above, the method comprising: Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images. A two-stream deep network based on cross-modal spectral graph interaction is constructed. The two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module. By inputting visible photon images and vegetation index sub-images into a two-stream deep network, a distribution map of crop diseases and pests within the area where farmland crops are located is obtained.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing monitoring method for crop diseases and pests, characterized in that, include: Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images. A two-stream deep network based on cross-modal spectral graph interaction is constructed; the two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module; The visible photon image and the vegetation index sub-image are input into the dual-stream depth network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located.

2. The remote sensing monitoring method for crop diseases and pests according to claim 1, characterized in that, Images of farmland crops are acquired using remote sensing equipment and preprocessed to obtain visible photon images and vegetation index sub-images, including: Visible light and multispectral images of the farmland crops were collected using drones or satellite remote sensing equipment. Geometric correction, radiometric correction, vegetation index calculation, and segmented output are performed on the visible light image and the multispectral image to obtain the visible photon image and the vegetation index sub-image.

3. The remote sensing monitoring method for crop diseases and pests according to claim 1, characterized in that, The visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork each include the same number of residual modules; The residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork are respectively connected to a cross-modal spectral feature interaction module; Any of the cross-modal spectral feature interaction modules and the residual modules of the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork connected thereto constitute an interaction layer; the dual-stream deep network includes multiple interaction layers to achieve adaptive interaction of semantic and spatial information; The cross-modal spectrogram feature interaction module is connected to a cross-modal spectrogram feature fusion module, which is used to perform feature fusion on the semantic and spatial information transmitted at different interaction levels.

4. The remote sensing monitoring method for crop diseases and pests according to claim 3, characterized in that, The visible photon image and the vegetation index sub-image are input into the dual-stream depth network to obtain a crop pest and disease distribution map within the area where the farmland crops are located, including: The visible photon image and the vegetation index image are respectively input into the visible light image feature extraction subnetwork and the vegetation index image feature extraction subnetwork to obtain the visible light image depth features and the vegetation index image depth features. The visible light image depth features and the vegetation index image depth features are input into the cross-modal spectral map feature interaction module to obtain a cross-modal spectral map interaction feature map; The cross-modal spectral map interaction feature map is input into the cross-modal spectral map feature fusion module to obtain the cross-modal spectral map fusion feature map; The cross-modal spectral map fusion feature map is input into the dual-task processing module to construct a dual-task loss function, update the model parameters of the dual-stream deep network, and output a crop disease and pest distribution map within the area where the farmland crops are located.

5. The remote sensing monitoring method for crop diseases and pests according to claim 4, characterized in that, The visible light image depth features and the vegetation index image depth features are input into the cross-modal spectral map feature interaction module to obtain a cross-modal spectral map interaction feature map, including: Channel attention weights are applied to the visible light image depth features and the vegetation index image depth features respectively to obtain visible light image channel weighted features and vegetation index image channel weighted features; The weighted features of the visible light image channels are transposed and multiplied with the weighted features of the vegetation index image channels to obtain a cross-modal semantic feature map. The visible light image depth features and the vegetation index image depth features are multiplied by the cross-modal semantic feature map, and then cross-modal feature interaction in the channel dimension is performed through the softmax activation function to obtain the visible light image semantic interaction feature map and the vegetation index image semantic interaction feature map. Spatial attention weights are applied to the semantic interaction feature map of the visible light image and the semantic interaction feature map of the vegetation index image, respectively, to obtain the spatially weighted features of the visible light image and the spatially weighted features of the vegetation index image. The spatial weighted features of the visible light image and the spatial weighted features of the vegetation index image are added together to obtain a cross-modal spatial attention feature map; The semantic interaction feature map of the visible light image and the semantic interaction feature map of the vegetation index image are multiplied by the cross-modal spatial attention feature map, and then cross-modal feature interaction in the spatial dimension is performed through the softmax activation function to obtain the semantic-spatial interaction feature map of the visible light and vegetation index images. The semantic-spatial interaction feature maps are spliced ​​along the channel direction to obtain a cross-modal spectral interaction feature map.

6. The remote sensing monitoring method for crop diseases and pests according to claim 4, characterized in that, The cross-modal spectral map interaction feature map is input into the cross-modal spectral map feature fusion module to obtain a cross-modal spectral map fusion feature map, including: Adaptive average pooling is performed on the cross-modal spectral interaction feature maps of each interaction level input, and weight factors for different cross-modal spectral interaction feature maps are generated. The cross-modal spectral interaction feature maps of different interaction levels are weighted and fused using the weighting factors to obtain the cross-modal spectral fusion feature map.

7. The remote sensing monitoring method for crop diseases and pests according to claim 4, characterized in that, The dual-task processing module includes a classifier and a regressor, and the classifier and the regressor are each composed of the same number of multilayer perceptrons. The dual-task loss function includes a classification loss function and a regression loss function.

8. The remote sensing monitoring method for crop diseases and pests according to claim 1, characterized in that, The crop disease and pest distribution map includes a disease and pest occurrence category distribution map and a disease and pest index distribution map output in sections for the area where the farmland crops are located.

9. A remote sensing monitoring device for crop diseases and pests, characterized in that, include: The acquisition module is used to acquire images of farmland crops through remote sensing equipment and perform preprocessing to obtain visible photon images and vegetation index sub-images. A construction module is used to build a two-stream deep network based on cross-modal spectral graph interaction; the two-stream deep network includes a visible light image feature extraction sub-network, a vegetation index image feature extraction sub-network, a cross-modal spectral graph feature interaction module, a cross-modal spectral graph feature fusion module, and a dual-task processing module; The monitoring module is used to input the visible photon image and the vegetation index sub-image into the dual-stream depth network to obtain a distribution map of crop diseases and pests within the area where the farmland crops are located.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the remote sensing monitoring method for crop diseases and pests as described in any one of claims 1 to 8.

Images