Bursaphelenchus xylophilus disease monitoring method and system based on remote sensing satellite

Abstract

The invention provides a pine wood nematode disease monitoring method and system based on a remote sensing satellite, and relates to the technical field of image data processing. The method comprises the following steps: firstly, acquiring and preprocessing optical, radar and thermal infrared remote sensing data of a forest region; then labeling the pine wood nematode disease infected wood sample in the optical image, and cutting all data sources into image tiles which are spatially aligned; then extracting physical features from each image tile, calculating a multi-modal infected index, and fusing the index into the optical image tiles at the same spatial position to construct a sample data set containing multiple features and labels for training and verifying a deep learning monitoring model; and finally, carrying out disease monitoring by using the trained model, and outputting a pine wood nematode disease monitoring result. And various remote sensing data are cooperatively fused as input features of the deep learning model, so that the monitoring reliability is greatly improved, and false alarms are avoided. Meanwhile, early-stage infected trees are found in time, and the monitoring timeliness is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and more particularly to a pine wilt disease monitoring method and system based on remote sensing satellites. BACKGROUND

[0002] Pine wilt disease, also known as pine wilt, is a common disease and pest of trees, and is a devastating pest with the characteristics of fast spread and difficult prevention and control. It has caused disasters in many areas. Early detection of pine wilt disease and removal of the affected trees can prevent the spread of the disease and greatly reduce the damage to the local ecological environment and economic losses.

[0003] Traditional monitoring methods for pine wilt disease are mostly manual monitoring, but due to the complex topography and uneven distribution of pine wilt disease outbreak sites, manual monitoring cannot fundamentally solve the problem. In order to alleviate the pressure of manual monitoring, researchers at home and abroad have developed various automatic monitoring methods. According to the monitoring methods, they can be divided into two categories: generating Lab color space images and feature images from satellite ortho-fused images, and comprehensively judging the range of pests and diseases through segmentation algorithms and vegetation feature thresholds. This method is still irreplaceable in delineating key epidemic areas, assessing disaster spread trends, and planning ground survey routes at large scales. However, this method mainly relies on color images and may overlook the great value of the near-infrared band. For example, healthy vegetation has high reflectivity in the near-infrared band, but it decreases significantly after being infected with pine wilt disease. Many vegetation indices are based on this principle and use the near-infrared band to monitor the extent of tree pests and diseases. The color space image method often cannot detect that a tree has been infected with pine wilt disease in time, and the monitoring timeliness is poor. Moreover, it cannot effectively distinguish pine wilt disease from other factors that cause tree yellowing, resulting in a high false positive rate.

[0004] With the continuous development of satellite sensors, intelligent extraction of pest and disease areas based on satellite images, aerial images, and unmanned aerial vehicle data has emerged. However, satellite data intelligent extraction can only obtain the approximate area of pest and disease trees, and cannot accurately determine the severity of tree pests and diseases. Although the aerial image and unmanned aerial vehicle data intelligent extraction method has high resolution, it has the problems of high cost and small range of single image. SUMMARY

[0005] To solve the problem of poor monitoring timeliness and high false positive rate in current pine wilt disease monitoring methods, the present application provides a pine wilt disease monitoring method and system based on remote sensing satellites, which can improve the monitoring timeliness while ensuring the monitoring efficiency, accurately determine the cause of tree yellowing, and reduce the false positive rate.

[0006] In order to achieve the above technical effects, the technical solutions of the present application are as follows: In a first aspect, the present application provides a pine wood nematode disease monitoring method based on remote sensing satellites, comprising the following steps: Obtain remote sensing satellite data of trees in a certain forest area, preprocess the remote sensing satellite data, and the remote sensing satellite data includes optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of trees; Label the pine wood nematode disease sample in the preprocessed optical image remote sensing satellite data with a sample infection degree label; Cut the labeled optical image remote sensing satellite data to obtain optical image tiles and label image tiles, process the radar remote sensing satellite data and the thermal infrared remote sensing satellite data based on the optical image tiles, and cut out radar remote sensing satellite data tiles and thermal infrared remote sensing satellite data tiles at the same spatial position corresponding to each optical image tile; Extract the physical characteristics of each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile at the same spatial position, calculate the multi-modal infection index based on the physical characteristics of the optical image, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile; Fuse the multi-modal infection index into the corresponding optical image tile to obtain multi-feature optical image tiles, combine the label image tiles, construct a tile sample dataset, divide the tile sample dataset into a training tile sample set, a verification tile sample set and a test tile set; Construct a pine wood nematode disease monitoring model, train the pine wood nematode disease monitoring model using the training tile sample set and the verification tile sample set, test and evaluate the pine wood nematode disease monitoring model using the test tile sample set, and obtain the trained pine wood nematode disease monitoring model; Use the trained pine wood nematode disease monitoring model to extract pine wood nematode disease samples from the remote sensing image data to be detected, and obtain the pine wood nematode disease monitoring result based on the pine wood nematode disease sample extraction result.

[0007] In the technical solution, firstly, optical, radar and thermal infrared remote sensing data of a forest area are acquired and preprocessed; then, pine wood nematode disease epidemic wood samples in the optical image are labeled, and all data sources are cut into spatially aligned image tiles; then, physical features are extracted from each image tile, a multi-modal infected index is calculated, the index is fused back into the optical image tile at the same spatial position, to construct a sample data set containing multiple features and labels, for training and verifying a deep learning monitoring model; finally, the trained model is used to monitor the disease of new remote sensing images, and the final pine wood nematode disease monitoring result is output. The method in the scheme cooperatively fuses multiple remote sensing data as input features of the deep learning model, greatly improves the reliability of the monitoring, and avoids false positives. At the same time, the early infected trees are found in time, and the timeliness of the monitoring is improved.

[0008] Preferably, the process of preprocessing the remote sensing satellite data comprises: unifying the optical image remote sensing satellite data, the radar remote sensing satellite data and the thermal infrared remote sensing satellite data into the same geographic coordinate system, and unifying the spatial resolution of the optical image remote sensing satellite data, the radar remote sensing satellite data and the thermal infrared remote sensing satellite data; uniformly lightening and colorizing the remote sensing image data by using the spatial reference template of the remote sensing satellite; the process is: performing multi-scale feature statistics on the optical image remote sensing satellite data, and constructing an original feature value space and a target feature value space, respectively; performing color mode conversion on each pixel of the optical image remote sensing satellite data, and separating the brightness information in the optical image remote sensing satellite data; based on the original feature value space and the target feature value space, performing Wallis transformation processing on the brightness information of each pixel of the optical image remote sensing satellite data, so that the feature value of the brightness information of each pixel reaches the target feature value, to obtain the preprocessed optical image remote sensing satellite data.

[0009] Preferably, before the preprocessed optical image remote sensing satellite data is labeled by using the sample label, the method further comprises: delineating the seed points of the pine wood nematode disease epidemic wood in the optical image remote sensing satellite data by using the gesture dragging method, then performing multi-scale segmentation around the seed points by using the Graph cut algorithm, and establishing a weighted graph of the similarity between each pixel point around the seed point and the seed point, to calculate a feature difference threshold, and obtain the range label of the pine wood nematode disease epidemic wood, the range label draws the geographical range of the pine wood nematode disease epidemic wood, the sample label labels the infected degree of the pine wood nematode disease epidemic wood based on the range label, and the infected degree comprises: healthy, early stage of disease, middle stage of disease and death.

[0010] Preferably, the process of extracting the physical features of each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile is: Radiometric calibration is performed on each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile to obtain backscattering coefficients; Atmospheric and topographic disturbances and geometric distortions are removed from the data, and atmospheric correction is performed on the optical image remote sensing satellite data and the thermal infrared remote sensing satellite data to obtain surface reflectance; ground temperature data is obtained based on radiometric brightness and surface reflectance.

[0011] Preferably, the multi-modal infected index is calculated by the expression:

[0012] wherein, represents a tree vigor factor, represents a temperature stress factor, represents a structural change factor, represents a weight coefficient of the tree vigor factor, represents a weight coefficient of the temperature stress factor, represents a weight coefficient of the structural change factor; For quantifying the decrease in physiological activity of vegetation caused by chlorophyll degradation and cell structure destruction, the expression is:

[0013] wherein, represents surface reflectance in the near-infrared band, represents surface reflectance in the red edge band; For quantifying the abnormal warming of the tree crown caused by blockage of water transport tissues and weakening of transpiration, the expression is:

[0014] wherein, represents ground temperature data, represents historical average temperature data at the same location; For quantifying the dramatic changes in the physical structure and dielectric constant of the forest canopy caused by needle wilting, water loss and tree death, the expression is:

[0015] wherein, represents backscattering coefficients in radar remote sensing satellite data at the current time period t, represents Backscattering coefficient under the radar remote sensing satellite data of the period.

[0016] Preferably, the fusing of the calculated multi-modal infected index into the corresponding optical image tile comprises: stacking the multi-modal infected index as a new feature channel with the original multiple spectral channels of the optical image tile to form a multi-feature optical image tile with increased number of feature channels.

[0017] Preferably, the pine wood nematode epidemic wood monitoring model is based on an encoder-decoder structure, and an MPT network is designed, wherein the encoder part is designed as a "CNN + Transformer backbone network", the "CNN + Transformer backbone network" comprises a CNN part and a Transformer part, the Transformer part adopts a local parallel mode, and the decoder part adopts an UPerHead composed of a detection head of a pyramid pooling module PPM and a top-down feature pyramid network FPN decoder.

[0018] Preferably, before the training, the training tile sample set is further subjected to radiation enhancement processing and resolution enhancement processing, wherein the radiation enhancement processing comprises brightness, hue, saturation, contrast, noise disturbance and image blur, and the resolution enhancement processing comprises random scaling, random horizontal flipping and random vertical flipping; the pine wood nematode epidemic wood monitoring model is trained by using the training tile sample set subjected to the radiation enhancement processing and the resolution enhancement processing.

[0019] Preferably, the process of obtaining the pine wood nematode monitoring result comprises: obtaining remote sensing satellite data to be detected, performing pest and disease epidemic wood extraction on the remote sensing satellite data to be detected by using the trained pine wood nematode epidemic wood monitoring model to obtain a grid probability map of a pest and disease range; combining the image characteristics of the remote sensing satellite data and the grid probability map of the pest and disease range to remove pseudo-plot spots in the pest and disease area of the remote sensing satellite data to obtain a grid result; performing result vectorization based on the grid result, and performing primary and secondary analysis, small face removal and vector thinning and smoothing processing to obtain a final pest and disease plot spot containing pest and diseases in different stages of onset as a pine wood nematode monitoring result; The pseudo-plot spot removal comprises: removing pseudo-plot spots in the pest and disease area of the remote sensing satellite data by using morphological pseudo-removal and similarity pseudo-removal. The morphological de-pseudo utilizes morphological characteristic factors to remove pseudo spots, the morphological characteristic factors include spot area, length and compactness, threshold values of the morphological characteristic factors are set according to the features of the pine wilt disease, and the target greater than the threshold value is removed; the similarity de-pseudo utilizes spectral characteristic factors and texture characteristic factors to remove pseudo spots, remote sensing satellite data to be detected and a grid probability map of the range of the disease and insect pests are loaded, spectral characteristic factor threshold values and texture characteristic factor threshold values of the pine wilt disease of each disease and insect pest degree are calculated according to the features of the pine wilt disease in the remote sensing satellite data, and the spot different from the spectral characteristic factor threshold values and the texture characteristic factor threshold values is removed.

[0020] In a second aspect, the application further provides a pine wilt disease monitoring system based on remote sensing satellites, the system comprising: A remote sensing data acquisition and preprocessing module is configured to acquire remote sensing satellite data of forest trees in a certain forest area, and to preprocess the remote sensing satellite data, wherein the remote sensing satellite data comprises optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of the forest trees. A sample label annotation module is configured to annotate pine wilt disease samples in the preprocessed optical image remote sensing satellite data by using sample infection degree labels. A remote sensing data cropping module is configured to crop the annotated optical image remote sensing satellite data to obtain optical image tiles and label image tiles, and to process the radar remote sensing satellite data and the thermal infrared remote sensing satellite data based on the optical image tiles, wherein the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles are cropped at the same spatial positions corresponding to each optical image tile. A multi-modal infection index calculation module is configured to extract physical features of the optical image tiles, the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles at the same spatial positions, and to calculate multi-modal infection indexes based on the physical features of the optical image tiles, the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles. A tile sample dataset construction module is configured to fuse the calculated multi-modal infection indexes into corresponding optical image tiles to obtain multi-feature optical image tiles, to construct a tile sample dataset in combination with the label image tiles, and to divide the tile sample dataset into a training tile sample set, a verification tile sample set and a test tile set. A pine wilt disease monitoring model training module is configured to construct a pine wilt disease monitoring model, to train the pine wilt disease monitoring model by using the training tile sample set and the verification tile sample set, to test and evaluate the pine wilt disease monitoring model by using the test tile sample set, and to obtain a trained pine wilt disease monitoring model. The pine wood nematode disease pest forest monitoring module is used for extracting pine wood nematode disease pest forests from remote sensing image data to be detected by using a trained pine wood nematode disease forest monitoring model, and obtaining a pine wood nematode disease monitoring result based on the pine wood nematode disease pest forest extraction result.

[0021] Compared with the prior art, the pine wood nematode disease monitoring method and system based on remote sensing satellites have the following beneficial effects: The pine wood nematode disease monitoring method and system based on remote sensing satellites are provided, which first acquires and pre-processes optical, radar and thermal infrared remote sensing data of a forest area; then, pine wood nematode disease forest samples in the optical image are labeled, and all data sources are cut into spatially aligned image tiles; then, physical features are extracted from each image tile, and a multi-modal infection index is calculated, which is fused back into the optical image tile at the same spatial position to construct a sample data set containing multiple features and labels, which is used to train and verify a deep learning monitoring model; finally, the trained model is used for disease monitoring of new remote sensing images, and the final pine wood nematode disease monitoring result is output. The method in the scheme cooperatively fuses multiple remote sensing data as input features of the deep learning model, greatly improves the reliability of the monitoring, and avoids false positives. At the same time, early infected trees are discovered in time, and the timeliness of the monitoring is improved.

[0022] Meanwhile, in the pine wood nematode disease forest monitoring scenario, a pine wood nematode disease forest monitoring system covering the whole business process is constructed based on deep learning technology, which provides sample production, sample management, model training iteration, model testing, model management, model prediction and other capabilities, and realizes the whole process of pine wood nematode disease forest monitoring model training and application. The sample production module includes remote sensing data acquisition and preprocessing, sample label labeling, remote sensing data cutting and other functions, which can realize fast labeling and constrained cutting of samples, provide more effective and higher precision sample data for model training, and improve the model convergence speed. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 Fig. 1 shows a flowchart of a pine wood nematode disease monitoring method based on remote sensing satellites according to an embodiment of the present application; Figure 2 Fig. 2 shows a composition diagram of optical image sample data according to an embodiment of the present application; Figure 3 Fig. 3 shows a backbone network structure diagram of a pine wood nematode disease forest monitoring model according to an embodiment of the present application; Figure 4 Fig. 4 shows a training flowchart of a pine wood nematode disease forest monitoring model according to an embodiment of the present application; Figure 5An information flow chart of a pine wood nematode disease monitoring method based on remote sensing satellites according to an embodiment of the present application is shown in FIG. 1. Figure 6 A structural schematic diagram of a pine wood nematode disease monitoring system based on remote sensing satellites according to an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0024] The accompanying drawings are only used for illustrative purposes and should not be construed as limiting the present patent; In order to better illustrate the present embodiment, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the actual size; It is understandable for those skilled in the art that some well-known descriptions in the accompanying drawings may be omitted.

[0025] The technical solutions of the present application will be further described below in combination with the accompanying drawings and embodiments.

[0026] The positional relationship described in the accompanying drawings is only used for illustrative purposes and should not be construed as limiting the present patent; Embodiment 1 The present embodiment proposes a pine wood nematode disease monitoring method based on remote sensing satellites, and a flowchart of the method is shown in FIG. 1, which comprises the following steps: Figure 1 S1. Obtain remote sensing satellite data of trees in a certain forest area, and pre-process the remote sensing satellite data, wherein the remote sensing satellite data comprises optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of the trees; S2. Label pine wood nematode disease infected sample in the pre-processed optical image remote sensing satellite data with sample infected degree label; S3. Cut the labeled optical image remote sensing satellite data to obtain optical image tiles and label image tiles, and based on the optical image tiles, process the radar remote sensing satellite data and the thermal infrared remote sensing satellite data respectively, and cut out radar remote sensing satellite data tiles and thermal infrared remote sensing satellite data tiles at the same spatial position corresponding to each optical image tile; S4. Extract physical features of each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile at the same spatial position, and calculate multi-modal infected index based on the physical features of the optical image tile, the radar remote sensing satellite data tile and the thermal infrared remote sensing satellite data tile; S5. Fuse the multi-modal infected index into the corresponding optical image tile to obtain multi-feature optical image tile, combine the label image tile, construct tile sample data set, and divide the tile sample data set into training tile sample set, verification tile sample set and test tile set; ​S6. Constructing the pine wood nematode epidemic forest monitoring model, training the pine wood nematode epidemic forest monitoring model by using the training tile sample set and the verification tile sample set, testing and evaluating the pine wood nematode epidemic forest monitoring model by using the test tile sample set, and obtaining the trained pine wood nematode epidemic forest monitoring model; S7. Monitoring the pine wood nematode epidemic forest by using the trained pine wood nematode epidemic forest monitoring model on the remote sensing image data to be detected, and obtaining the pine wood nematode monitoring result based on the pine wood nematode epidemic forest extraction result.

[0027] In the embodiment, first, optical, radar and thermal infrared remote sensing data of a forest area are acquired and preprocessed; then, pine wood nematode epidemic forest samples in the optical image are labeled, and all data sources are cut into spatially aligned image tiles; then, physical features are extracted from each image tile, and a multi-modal infection index is calculated, which is fused back into the optical image tile at the same spatial location to construct a sample data set containing multiple features and labels for training and verifying a deep learning monitoring model; finally, the trained model is used to monitor the disease of new remote sensing images, and the final pine wood nematode monitoring result is output. The method in the scheme cooperatively fuses multiple remote sensing data as input features of the deep learning model, greatly improves the reliability of the monitoring, and avoids false positives. At the same time, early infected trees are discovered in time, and the timeliness of the monitoring is improved.

[0028] Embodiment 2 In the embodiment, the process of preprocessing the remote sensing satellite data includes: unifying the optical image remote sensing satellite data, the radar remote sensing satellite data and the thermal infrared remote sensing satellite data to the same geographic coordinate system, and unifying the spatial resolution of the optical image remote sensing satellite data, the radar remote sensing satellite data and the thermal infrared remote sensing satellite data; uniformly lightening and colorizing the remote sensing image data by using the spatial reference template of the remote sensing satellite; the process is: performing multi-scale feature statistics on the optical image remote sensing satellite data, and constructing an original feature value space and a target feature value space respectively; performing color mode conversion on each pixel of the optical image remote sensing satellite data, and separating the brightness information in the optical image remote sensing satellite data; based on the original feature value space and the target feature value space, performing Wallis transformation processing on the brightness information of each pixel of the optical image remote sensing satellite data, so that the brightness information feature value of each pixel reaches the target feature value, and obtaining the preprocessed optical image remote sensing satellite data.

[0029] Specifically, the optical remote sensing image data is image data collected by Beijing-03N satellite, and the resolution of the image data collected by Beijing-03N satellite is 0.3. The radar remote sensing satellite data is radar remote sensing data collected by Nuwa constellation commercial radar remote sensing satellite, and the thermal infrared remote sensing satellite data is thermal infrared remote sensing data collected by Gaofen-5 satellite.

[0030] Specifically, the use of the spatial reference template of the remote sensing satellite for uniform light and color of the remote sensing image data can make the remote sensing image data have uniform tone, moderate contrast, close to natural true color, and rich spectral information. The spatial reference template refers to the element class or layer to be used as a template for setting the value of the spatial reference.

[0031] In addition, the remote sensing image data is subjected to band processing before uniform light and color.

[0032] Specifically, the Wallis transformation processing is processed by a Wallis filter. The Wallis filter is a linear transformation for local images, which makes the mean value and variance of different images or different positions of the image tend to be approximate values, that is, the contrast of the image is increased in the area with small contrast, and the contrast is reduced in the area with large contrast. The mean value of the image represents the tone and brightness, and the variance represents the range of pixel value change. The transition of different images or different positions of the image should be continuous, with consistent tone, brightness and contrast, and consistent gray dynamic change range. The desired mean value and variance should be consistent. The principle of the Wallis filter is to reduce the difference between the mean value and the variance of the image to achieve the consistency of the image color. The gray value change formula of the image after Wallis transformation processing is:

[0033] wherein, is the original image gray value; is the gray value of the result image after Wallis transformation; is the local mean value of the original image; is the local variance of the original image; is the target value of the local mean value of the result image; is the target value of the local variance of the result image; is the expansion constant of the image variance; is the brightness coefficient of the image.

[0034] In the embodiment, before the optical remote sensing image data preprocessed by the sample label is labeled, the method further comprises: The pine wilt disease seed point is drawn in the optical image remote sensing satellite data by using a gesture dragging manner, a multi-scale segmentation is realized around the seed point by using a Graph cut algorithm, a weighting graph of the similarity of each pixel point around the seed point to the seed point is established, a feature difference threshold is calculated, and a range label of the pine wilt disease is obtained.

[0035] Specifically, the sample label is based on the range label and labels the infection degree of the pine wilt disease, and the infection degree includes healthy, early stage of disease, middle stage of disease and death. The label of the healthy disease area is set to 1, the label of the early stage of disease is set to 2, the label of the middle stage of disease is set to 3, the label of the death disease area is set to 4, and the label of the background area is set to 0. Based on the sample scheme, a sample labeling project is created, and the four types of pine wilt diseases including healthy, early stage of disease, middle stage of disease and death are labeled. When labeling the pine wilt disease in the optical image remote sensing satellite data, the seed point of the pine wilt disease is drawn in the optical image remote sensing satellite data by using a gesture dragging manner, a multi-scale segmentation is realized around the seed point by using a Graph cut algorithm, a weighting graph of the similarity of each pixel point around the seed point to the seed point is established, a feature difference threshold is calculated, and a range label of the pine wilt disease is obtained.

[0036] In the embodiment, before the physical features of the optical image tile, the radar remote sensing satellite data tile and the thermal infrared remote sensing satellite data tile at the same spatial position of each block are extracted, the labeled optical image remote sensing satellite data is cut to obtain the optical image tile and the label image tile, and the radar remote sensing satellite data and the thermal infrared remote sensing satellite data are processed based on the optical image tile. The radar remote sensing satellite data tile and the thermal infrared remote sensing satellite data tile are cut at the same spatial position corresponding to each optical image tile; Specifically, the cutting parameters of the annotated optical image remote sensing satellite data include sample name and number, tile size, overlap size, label effective proportion, image effective proportion, stretching effect, etc. The tile size is set to 512x512 pixels, the overlap size is 128 pixels, the label effective proportion is 0.00001, the image effective proportion is set to 0.5, the stretching effect is set to percentage truncation, and the maximum and minimum truncation values are set to 0.002. By setting the sample name and number, the uniqueness of the tile sample can be ensured; by setting the tile overlap size, the sample diversity, sample quantity and model robustness can be increased; by setting the label effective proportion, negative samples can be filtered and the model convergence speed can be improved; by setting the image effective proportion, the effectiveness of the sample can be improved and the model training efficiency can be improved; the stretching effect can enhance the visual effect of the sample, reduce the learning difficulty and improve the training efficiency.

[0037] In the embodiment, the process of extracting the physical characteristics of each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile is as follows: Radiometric calibration is performed on each optical image tile, radar remote sensing satellite data tile and thermal infrared remote sensing satellite data tile to obtain the backscattering coefficient. Atmospheric and terrain interference and geometric deformation caused by data are removed, atmospheric correction is performed on the optical image remote sensing satellite data and the thermal infrared remote sensing satellite data to obtain the ground reflectance, and the ground temperature data is obtained based on the radiance and the ground reflectance.

[0038] In the embodiment, the multi-modal infected index is calculated according to the following expression:

[0039] wherein, represents a tree vigor factor, represents a temperature stress factor, represents a structure change factor, represents a weight coefficient of the tree vigor factor, represents a weight coefficient of the temperature stress factor, represents a weight coefficient of the structure change factor. is used to quantify the decrease in physiological activity of vegetation caused by chlorophyll degradation and cell structure damage, and the expression is as follows:

[0040] wherein, represents the ground reflectance in the near-infrared band, represents the ground reflectance in the red edge band. For quantifying the abnormal temperature rise of the tree crown caused by the moisture transport tissue blockage and the weakened transpiration, the expression is:

[0041] wherein, represents the ground temperature data, represents the historical average temperature data at the same place; For quantifying the dramatic change of the forest crown physical structure and dielectric constant caused by the needle wilting, water loss and tree death, the expression is:

[0042] wherein, represents the backscattering coefficient under the radar remote sensing satellite data at the current t period, represents the backscattering coefficient under the radar remote sensing satellite data at the t period.

[0043] In the embodiment, the fusion of the calculated multi-modal infected index into the corresponding optical image tile includes: stacking the multi-modal infected index as a new feature channel with the original multiple spectral channels of the optical image tile to form a multi-feature optical image tile with increased feature channel number.

[0044] In the embodiment, before the construction of the pine wilt disease epidemic forest monitoring model, the multi-modal infected index is further fused into the corresponding optical image tile to obtain a multi-feature optical image tile, and the tile sample data set is constructed in combination with the label image tile, and the tile sample data set is divided into a training tile sample set, a verification tile sample set and a test tile set. Specifically, it further includes setting a sample effective range to constrain the sample annotation area before sample annotation, and exporting the results after annotation to obtain pine wilt disease optical image samples organized in a specific way. The composition diagram of the optical image sample data is shown in Figure 2 Each optical image sample data is a folder named after the name of the optical image data used for annotation, and the specific content includes an optical image sample metadata file, Beijing No. 3 N satellite remote sensing image data, pine wilt disease label range vector, effective range data, and pine wilt disease classification scheme.

[0045] The optical image sample meta file records the image sample metadata information, including scene type, classification scheme, optical image path and resolution, acquisition time, basic attribute information, sample label vector data path and name, sample effective range vector data path and name, etc. The pine wilt disease classification scheme file records the sample category number, label value, category name, etc. of healthy, early stage of disease, medium stage of disease and death.​

[0046] Specifically, the present embodiment is divided into a training tile sample set, a validation tile sample set and a test tile set in a ratio of The validation tile sample set contains typical sample types, including early-stage pine wilt disease epidemic wood types, mid-stage pine wilt disease epidemic wood types and dead pine wilt disease epidemic wood types, and the image data corresponding to the image tiles in the validation tile sample set should cover remote sensing image data obtained under different seasons and light conditions. Moreover, on the remote sensing image, the early-stage and mid-stage pine wilt disease epidemic wood types are easy to be confused with trees in the flowering period, and the characteristics of the dead pine wilt disease epidemic wood types are easy to be confused with bare land, causing false extraction. Therefore, usually in the process of making tile samples, positive and negative samples are distinguished, and the image area of the ground object that is easy to confuse is cut into negative samples.

[0047] In the present embodiment, the pine wilt disease epidemic wood monitoring model is based on an encoder-decoder structure, and an MPT network is designed, wherein the encoder part is designed as a "backbone network of CNN+Transformer", the "backbone network of CNN+Transformer" includes a CNN part and a Transformer part, the Transformer part adopts a local parallel mode, and the decoder part adopts a UPerHead composed of a detection head of a pyramid pooling module PPM and a top-down feature pyramid network FPN decoder.

[0048] Specifically, the backbone network structure diagram of the pine wilt disease epidemic wood monitoring model is as shown in Figure 3 The backbone network of the pine wilt disease epidemic wood monitoring model of the present embodiment includes a dimension reduction sampling module, a first encoder architecture, a second encoder architecture and a third encoder architecture connected in sequence. The first encoder architecture, the second encoder architecture and the third encoder architecture each include a parallel multi-head attention module and a multi-scale feature aggregation module. The parallel multi-head attention module mainly adopts a PTB (parallel transformer block) module, and the multi-scale feature extraction module mainly adopts a pyramid convolution module PRM (Pyramid Reduction Module). Specifically, in the parallel multi-head attention module, an HxW image is input into a pyramid convolution module to obtain three feature maps with sizes of HxW, H / 2xW / 2 and H / 4xW / 4, the three feature maps are respectively input into a multi-layer attention mechanism layer based on a sliding window to obtain output feature maps, and the three output feature maps obtained, the input HxW image and the three feature maps with sizes of HxW, H / 2xW / 2 and H / 4xW / 4 obtained by convolution of the pyramid convolution module are input into a feature fusion of a multi-layer perception for feature fusion.

[0049] In the training, the pine wilt disease monitoring model is input into the pine wilt disease monitoring model, a binary cross-entropy loss function is used, training parameters are set for iterative training, and when the training reaches the iteration stop condition, the trained pine wilt disease monitoring model is obtained, which is packaged. The contents of the set training parameters include learning rate, learning rate decay, and batch size. The initial learning rate is set to 0.001, the learning rate decay is set to 0.000001, and the batch size is set to 16.

[0050] Specifically, the training flowchart of the pine wilt disease monitoring model is as shown in Figure 4 The training process of the pine wilt disease monitoring model is as follows: the model starts from input data, transforms the data through layers with initial weights level by level, and finally outputs the predicted value; the predicted value and the true data are jointly input into the loss function to calculate the error degree of the current model; the accuracy is judged by the error degree, if the accuracy does not meet the standard, the optimizer triggers back propagation, and the weights of each layer are adjusted along the network in reverse to reduce the error; if the accuracy meets the standard, the training is ended.

[0051] In this embodiment, before training, the training tile sample set is also subjected to radiation enhancement processing and resolution enhancement processing, wherein the radiation enhancement processing includes brightness, hue, saturation, contrast, noise disturbance and image blur, and the resolution enhancement processing includes random scaling, random horizontal flipping and random vertical flipping; the training tile sample set obtained after the radiation enhancement processing and the resolution enhancement processing is used to train the pine wilt disease monitoring model.

[0052] In this embodiment, the process of obtaining the pine wilt disease monitoring result is as follows: The remote sensing satellite data to be detected is obtained, and the trained pine wilt disease monitoring model is used to monitor the disease and pest forest of the remote sensing satellite data to be detected to obtain a grid probability map of the disease and pest range; Combined with the image characteristics of the remote sensing satellite data and the grid probability map of the disease and pest range, the disease and pest area of the remote sensing satellite data is subjected to pseudo-plot removal to obtain a grid result; Based on the grid result, the achievement vectorization is performed, and the primary and secondary analysis, small face removal and vector thinning and smoothing processing are performed to obtain the final disease and pest plot containing different disease stages, which is taken as the pine wilt disease monitoring result; The pseudo-plot removal includes: removing the pseudo-plot of the disease and pest area of the remote sensing satellite data by using morphological pseudo-removal and similarity pseudo-removal. Morphological despoofing utilizes morphological feature factors to remove false patches, including patch area, narrow length, and compactness. Thresholds for morphological feature factors are set based on the ground features of pine wilt-infected trees, and targets exceeding the threshold are removed. Similarity despoofing utilizes spectral and texture feature factors to remove false patches. The remote sensing satellite data to be detected and the raster probability map of the disease and pest range are loaded. The spectral and texture feature factor thresholds for pine wilt-infected trees of different disease and pest severity levels are calculated using the ground features of healthy, early-stage, mid-stage, and dead pine wilt-infected trees from the remote sensing satellite data. Patches that differ significantly from the spectral and texture feature factor thresholds are removed.

[0053] Specifically, the majority / minority analysis is primarily used to remove small patches, thereby improving the accuracy of the patch results. Removing small patches refers to eliminating patches with an area smaller than a set threshold. In the vector thinning and smoothing process, considering the tendency of traditional thinning algorithms to oversimplify key inflection points, the node density is first dynamically adjusted based on the curvature of vector arcs to remove unnecessary points and simplify the vector data while preserving the basic characteristics and diffusion features of the disease. Then, data points are fitted based on Bézier curves and anisotropic diffusion models, generating a smooth curve while retaining the sharpness of patch edges. This vector thinning and smoothing process improves the display effect of the vector results data.

[0054] Specifically, in this embodiment, the information flow diagram of this method is as follows: Figure 5 As shown, Figure 5 In this process, optical imagery remote sensing satellite data acquired via Beijing-3N satellite was first processed through banding and homogenization. Then, sample labels were used to annotate the optical remote sensing data of pine wilt-infected trees, forming pine wilt-infected tree samples. The annotated optical imagery remote sensing satellite data was cropped, and the tile size was set to 512×512 pixels. Combined with the labeled image tiles, a tile sample dataset was constructed. A pine wilt-infected tree monitoring model was then built. The model was trained using training and validation tile sample sets, and tested and evaluated using a test tile sample set, resulting in a trained pine wilt-infected tree monitoring model. The trained pine wilt-infected tree monitoring model was then used to monitor pine wilt-infected trees in the remote sensing imagery data to be detected. After pseudo-patch removal and raster vectorization post-processing, disease and pest patches at different disease stages were obtained.

[0055] Example 3 This embodiment proposes a pine wilt disease monitoring system based on remote sensing satellites. In this embodiment, the system is used to implement a method for monitoring pine wilt disease based on remote sensing satellites. The structural schematic diagram is shown below. Figure 6 As shown, it includes: The remote sensing data acquisition and preprocessing module is used for acquiring remote sensing satellite data of trees in a certain tree area, and preprocessing the remote sensing satellite data, wherein the remote sensing satellite data comprises optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of the trees; The sample label annotation module is used for annotating pine wood nematode disease epidemic tree samples in the preprocessed optical image remote sensing satellite data by using sample infection degree labels. The remote sensing data cropping module is used for cropping the annotated optical image remote sensing satellite data to obtain optical image tiles and label image tiles, and processing the radar remote sensing satellite data and the thermal infrared remote sensing satellite data based on the optical image tiles, wherein the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles are cropped at the same spatial positions corresponding to each optical image tile. The multi-modal infection index calculation module is used for extracting physical features of the optical image tiles, the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles at the same spatial positions, and calculating multi-modal infection indexes based on the physical features of the optical image tiles, the radar remote sensing satellite data tiles and the thermal infrared remote sensing satellite data tiles. The tile sample dataset construction module is used for fusing the calculated multi-modal infection indexes into corresponding optical image tiles to obtain multi-feature optical image tiles, and constructing a tile sample dataset by combining the label image tiles, and dividing the tile sample dataset into a training tile sample set, a verification tile sample set and a test tile set. The pine wood nematode disease epidemic tree monitoring model training module is used for constructing a pine wood nematode disease epidemic tree monitoring model, training the pine wood nematode disease epidemic tree monitoring model by using the training tile sample set and the verification tile sample set, testing and evaluating the pine wood nematode disease epidemic tree monitoring model by using the test tile sample set, and obtaining a trained pine wood nematode disease epidemic tree monitoring model. The pine wood nematode disease epidemic tree monitoring module is used for extracting pine wood nematode disease epidemic trees from to-be-detected remote sensing image data by using the trained pine wood nematode disease epidemic tree monitoring model, and acquiring pine wood nematode disease monitoring results based on the pine wood nematode disease epidemic tree extraction results.

[0056] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments are not required to be exhausted. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A method for monitoring pine wilt disease based on remote sensing satellites, characterized in that, Includes the following steps: The remote sensing satellite data of trees in a certain forest area is acquired and preprocessed. The remote sensing satellite data includes optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of trees. Using infection level labels to label pine wilt disease-infected wood samples from preprocessed optical imagery remote sensing satellite data; After cropping and labeling the optical image remote sensing satellite data, optical image tiles and labeled image tiles are obtained. Based on the optical image tiles, radar remote sensing satellite data and thermal infrared remote sensing satellite data are processed separately. At the same spatial position corresponding to each optical image tile, radar remote sensing satellite data tiles and thermal infrared remote sensing satellite data tiles are cropped. The physical features of optical image tiles, radar remote sensing satellite data tiles, and thermal infrared remote sensing satellite data tiles at the same spatial location are extracted. Based on the physical features of optical image tiles, radar remote sensing satellite data tiles, and thermal infrared remote sensing satellite data tiles, the multimodal infection index is calculated. Multimodal infection indices are fused into corresponding optical image tiles to obtain multi-feature optical image tiles. Combined with labeled image tiles, a tile sample dataset is constructed, which is then divided into a training tile sample set, a validation tile sample set, and a test tile set. A monitoring model for pine wilt disease-infected trees was constructed. The model was trained using training and validation tile sample sets, and tested and evaluated using test tile sample sets to obtain the trained monitoring model for pine wilt disease-infected trees. The trained pine wilt disease-infected wood monitoring model was used to monitor pine wilt disease-infected wood in the remote sensing image data to be detected. Based on the extraction results of pine wilt disease-infected wood, the monitoring results of pine wilt disease were obtained.

2. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 1, characterized in that, The process of preprocessing remote sensing satellite data includes: Unify optical image remote sensing satellite data, radar remote sensing satellite data, and thermal infrared remote sensing satellite data into the same geographic coordinate system, and unify the spatial resolution of optical image remote sensing satellite data, radar remote sensing satellite data, and thermal infrared remote sensing satellite data; The process involves using a spatial reference template from a remote sensing satellite to homogenize the light and color of remote sensing image data. Multi-scale feature statistics are performed on optical image remote sensing satellite data to construct the original feature value space and the target feature value space, respectively. Color mode conversion is performed on each pixel of the optical image remote sensing satellite data to separate the brightness information in the optical image remote sensing satellite data; Based on the original feature space and the target feature space, the brightness information of each pixel in the optical image remote sensing satellite data is processed by Wallis transform so that the brightness information feature value of each pixel reaches the target feature value, thus obtaining the preprocessed optical image remote sensing satellite data.

3. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 2, characterized in that, Before labeling the preprocessed optical imagery remote sensing satellite data with sample labels, the process also includes: Using gesture-guided mapping, seed points of pine wilt-infected trees were delineated in optical imagery remote sensing satellite data. Then, a graph cut algorithm was used to perform multi-scale segmentation around the seed points, and a weighted graph of the similarity between the seed point and each surrounding pixel was constructed. Feature difference thresholds were calculated to obtain range labels for the pine wilt-infected trees. These range labels delineated the geographical extent of the trees. Sample labels, based on these range labels, indicated the degree of infection in the pine wilt-infected trees, including healthy, early-stage disease, mid-stage disease, and dead trees.

4. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 1, characterized in that, The process of extracting the physical features of each optical image tile, radar remote sensing satellite data tile, and thermal infrared remote sensing satellite data tile is as follows: Radiometric calibration was performed on each optical image tile, radar remote sensing satellite data tile, and thermal infrared remote sensing satellite data tile to obtain the backscattering coefficient; To obtain the surface reflectance, atmospheric correction is performed on optical image remote sensing satellite data and thermal infrared remote sensing satellite data to remove interference and geometric distortion caused by the atmosphere and topography. Ground temperature data are obtained based on radiance and surface reflectance.

5. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 4, characterized in that, The multimodal infection index The calculation expression is: in, Indicating tree vitality factors, Indicates the temperature stress factor. Indicates the structural change factor. The weighting coefficients represent the tree vitality factor. The weighting coefficients representing the temperature stress factor. The weighting coefficients representing structural change factors; The expression used to quantify the decline in vegetation physiological activity caused by chlorophyll degradation and cell structure damage is: in, Represents the surface reflectance in the near-infrared band. Indicates the surface reflectance in the red-edge band; The expression used to quantify abnormal canopy warming caused by blockage of water transport tissues and reduced transpiration is: in, Represents ground temperature data. This represents the historical average temperature data for the same location; The expression used to quantify the drastic changes in the physical structure and dielectric constant of the forest canopy caused by needle wilting, water loss, and tree death is: in, This represents the backscattering coefficient under radar remote sensing satellite data at time t. express Backscattering coefficients under radar remote sensing satellite data of the period.

6. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 5, characterized in that, The step of fusing the calculated multimodal infection index into the corresponding optical image tile includes: using the multimodal infection index as a new feature channel and stacking it with the original multiple spectral channels of the optical image tile to form a multi-feature optical image tile with an increased number of feature channels.

7. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 1, characterized in that, The pine wilt disease monitoring model is based on an encoder-decoder structure and uses an MPT network. The encoder part is designed as a "CNN+Transformer backbone network". The "CNN+Transformer backbone network" includes a CNN part and a Transformer part. The Transformer part adopts a local parallel approach. The decoder part adopts UPerHead and consists of a detection head of a pyramid pooling module PPM and a top-down feature pyramid network FPN decoder.

8. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 7, characterized in that, Before training, the training tile sample set is subjected to radiometric enhancement and resolution enhancement processing. Radiometric enhancement processing includes brightness, hue, saturation, contrast, noise perturbation, and image blurring, while resolution enhancement processing includes random scaling, random horizontal flipping, and random vertical flipping. The training tile sample set obtained after radiometric enhancement and resolution enhancement processing is used to train the pine wilt disease infestation monitoring model.

9. The method for monitoring pine wilt disease based on remote sensing satellites according to claim 7, characterized in that, The process for obtaining monitoring results of pine wilt disease is as follows: Acquire remote sensing satellite data to be detected, and use the trained pine wilt disease-infected wood monitoring model to extract diseased and infected wood from the remote sensing satellite data to be detected, and obtain a raster probability map of the disease and pest range. By combining the image characteristics of remote sensing satellite data and the raster probability map of the extent of pests and diseases, false patches are removed from the pest and disease areas in the remote sensing satellite data to obtain raster results; The results were vectorized based on the raster results, and then subjected to primary and secondary analysis, facet removal, and vector thinning and smoothing to obtain the final disease and pest patches containing different disease stages, which served as the monitoring results of pine wilt disease. The pseudo-spot removal includes: using morphological despoofing and similarity despoofing to remove pseudo-spots from pest and disease areas in remote sensing satellite data; Morphological despoofing utilizes morphological feature factors to remove false patches, including patch area, narrow length, and compactness. Thresholds for morphological feature factors are set based on the ground features of pine wilt-infected trees, and targets exceeding the threshold are removed. Similarity despoofing utilizes spectral and texture feature factors to remove false patches. The remote sensing satellite data to be detected and the raster probability map of the disease and pest range are loaded. The spectral and texture feature factor thresholds for pine wilt-infected trees of different disease and pest severity levels are calculated using the ground features of healthy, early-stage, mid-stage, and dead pine wilt-infected trees from the remote sensing satellite data. Patches that differ significantly from the spectral and texture feature factor thresholds are removed.

10. A monitoring system for pine wilt disease based on remote sensing satellites, characterized in that, The system is used to implement the method according to any one of claims 1 to 9, comprising: The remote sensing data acquisition and preprocessing module is used to acquire remote sensing satellite data of trees in a certain forest area and preprocess the remote sensing satellite data. The remote sensing satellite data includes optical image remote sensing satellite data, radar remote sensing satellite data and thermal infrared remote sensing satellite data of trees. The sample labeling module is used to label pine wilt disease-infected wood samples in preprocessed optical image remote sensing satellite data using the infection degree of the samples. The remote sensing data cropping module is used to crop the labeled optical image remote sensing satellite data to obtain optical image tiles and labeled image tiles. Based on the optical image tiles, the radar remote sensing satellite data and thermal infrared remote sensing satellite data are processed separately. At the same spatial position corresponding to each optical image tile, radar remote sensing satellite data tiles and thermal infrared remote sensing satellite data tiles are cropped. The multimodal infection index calculation module is used to extract the physical features of each optical image tile, radar remote sensing satellite data tile, and thermal infrared remote sensing satellite data tile at the same spatial location, and calculate the multimodal infection index based on the physical features of the optical image, radar remote sensing satellite data tile, and thermal infrared remote sensing satellite data tile. The tile sample dataset construction module is used to fuse the calculated multimodal infection index into the corresponding optical image tiles to obtain multi-feature optical image tiles. Combined with the labeled image tiles, a tile sample dataset is constructed, which is divided into a training tile sample set, a validation tile sample set, and a test tile set. The training module for the monitoring model of pine wilt disease-infected trees is used to construct a monitoring model of pine wilt disease-infected trees. The monitoring model of pine wilt disease-infected trees is trained using training tile sample sets and validation tile sample sets, and tested and evaluated using test tile sample sets to obtain the trained monitoring model of pine wilt disease-infected trees. The pine wilt disease and pest-infected wood monitoring module is used to extract pine wilt disease and pest-infected wood from remote sensing image data to be detected using a trained pine wilt disease and pest-infected wood monitoring model, and to obtain pine wilt disease monitoring results based on the extraction results.

Images