A remote sensing monitoring and recognition system based on high-resolution reconstruction of remote sensing images

Abstract

The present application provides a remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images, wherein the acquisition unit is used to acquire remote sensing images of sensors in the same key monitoring area for crop diseases and insect pests, the preprocessing unit is used to preprocess the acquired remote sensing images of the monitoring area, the reconstruction unit is used to perform high-resolution reconstruction on the input low-resolution images without paired high-resolution images, the extraction unit is used to extract on the high-resolution reconstructed images to obtain the key monitoring crop planting range in the monitoring area, the identification unit is used to identify the crop diseases and insect pests using a preset disease and insect pest identification model, and obtain the regional monitoring crop diseases and insect pests, and the early warning display unit is used to perform early warning display of crop diseases and insect pests in the current planting area using a preset disease and insect pest expert guidance system. The present application realizes the timely acquisition and accurate monitoring, identification, and early warning of the spatial distribution and development trend of crop diseases and insect pests in crop fields.

Description

Technical Field

[0001] The present application relates to the technical field of pattern recognition of image processing, and in particular to a remote sensing monitoring and recognition system based on high-resolution reconstruction of remote sensing images. Background Art

[0002] Compared with traditional field survey technology and UAV remote sensing monitoring methods, using high spatial resolution remote sensing satellite images to identify the status of crop pests and diseases can efficiently and objectively monitor the occurrence and development of crop pests and diseases on a large scale, which can effectively overcome the defects of traditional methods. However, there is an irreconcilable contradiction between the temporal and spatial resolution of commonly used remote sensing satellite images. The return visit period of satellites carrying high spatial resolution sensors is relatively long, usually more than half a month, and sometimes they cannot obtain available images due to bad weather and cloud cover, which easily misses the best control period of crop pests and diseases. Although the images collected by sensors carried by satellites with short return visits have high temporal resolution, their spatial resolution is very low, which cannot accurately extract the crop planting range and the affected range of pests and diseases; or the temporal and spatial resolution of remote sensing images varies greatly due to the differences in the types of sensors carried by different satellites. Therefore, researchers hope to fuse the spatiotemporal data of satellite images from two sensors to obtain high temporal resolution and high spatial resolution impact sequences.

[0003] However, existing spatiotemporal fusion methods have the following problems: (1) Existing spatiotemporal fusion methods usually reconstruct high-resolution images under the assumption that image changes can be directly transmitted from one sensor to another. This assumption does not take into account the differences in the ability of different sensors to represent changes, which will lead to spectral and spatial distortion of the reconstructed image. (2) Existing spatiotemporal fusion methods usually use linear weighting methods to directly fuse temporal features. Linear weighting methods do not fully consider the change characteristics of all pixels and have limited image feature representation capabilities. Therefore, the existing technology does not have a good high-temporal resolution and high-spatial resolution remote sensing satellite image monitoring and recognition system for crop planting area monitoring, recognition, and early warning. Summary of the invention

[0004] The present application provides a remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images, which can realize timely acquisition and accurate monitoring, identification and early warning of the spatial distribution and development trends of crop diseases and insect pests in crop fields, and realize wider coverage, large-scale, multi-regional, visual and intelligent crop planting area monitoring with higher efficiency and lower cost.

[0005] This application is implemented through the following technical solutions:

[0006] A remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images, characterized in that it includes:

[0007] A collection unit is used to collect remote sensing images of sensors in the same key monitoring area for crop diseases and insect pests;

[0008] A preprocessing unit is used to perform sensor calibration, atmosphere calibration, solar altitude calibration, terrain calibration, geometric calibration and coordinate alignment preprocessing on the collected remote sensing images of the monitoring area to obtain images after preprocessing;

[0009] A reconstruction unit is used to receive the pre-processed image, and use a remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion to perform high-resolution reconstruction on the input low-resolution image without a paired high-resolution image, so as to obtain a high-resolution reconstructed image paired with the low-resolution image;

[0010] An extraction unit is used to extract the high-resolution reconstructed image to obtain the key monitoring crop planting range in the monitoring area;

[0011] An identification unit is used to identify the crop pests and diseases according to the extracted spectral information of the key monitored crop planting range and the corresponding range remote sensing image using a preset pest and disease identification model to obtain the crop pest and disease conditions of the monitored area;

[0012] The early warning display unit analyzes and evaluates the crop disease and insect pest situation monitored in the crop planting area according to the identification unit, and uses the preset disease and insect pest expert guidance system to perform graded and classified early warning display of crop diseases and insect pests in the current planting area.

[0013] Optionally, the high-resolution reconstruction of the input low-resolution image without paired high-resolution image using the remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion comprises the following steps:

[0014] Acquire a first phase low-resolution image, a second phase low-resolution image, and a third phase low-resolution image, wherein the first phase low-resolution image has a paired first phase high-resolution image of the same region and the same phase, and the third phase low-resolution image has a paired third phase high-resolution image of the same region and the same phase;

[0015] Inputting the first phase low-resolution image and the second phase low-resolution image into a super-resolution convolutional neural network to obtain a first two-phase transition residual image, and inputting the second phase low-resolution image and the third phase low-resolution image into a super-resolution convolutional neural network to obtain a second three-phase transition residual image;

[0016] Input the first two-phase transition residual image into the bias feature extraction convolutional neural network to obtain the first two-phase sensor bias, and input the second three-phase transition residual image into the bias feature extraction convolutional neural network to obtain the second three-phase sensor bias;

[0017] The first phase high-resolution image, the first two-phase transition residual image and the first two-phase sensor deviation are added to obtain a second phase forward transition high-resolution image, and the third phase high-resolution image, the second three-phase transition residual image and the second three-phase sensor deviation are added to obtain a second phase backward transition high-resolution image;

[0018] The second phase forward transition high-resolution image and the second phase backward transition high-resolution image are input into the time fusion convolutional neural network to obtain the second phase high-resolution reconstructed image.

[0019] Optionally, the step of inputting the first phase low-resolution image and the second phase low-resolution image into a super-resolution convolutional neural network to obtain a first two-phase transition residual image, and inputting the second phase low-resolution image and the third phase low-resolution image into a super-resolution convolutional neural network to obtain a second three-phase transition residual image is specifically:

[0020] Use the loss function formula

[0021] loss sr (θ sr )=MSE(ΔL ij ,SRCNN(ΔM ij ,θ sr ))

[0022] Train the super-resolution convolutional neural network, where SRCNN() represents the mapping function of the super-resolution convolutional neural network, θ sr Represents the training weight parameter of the mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔM ij =M j -M i is the j-th phase low-resolution image M in the training set j and the low-resolution image M at the i-th phase i The difference, Where ||·|| represents the Euclidean modulus length, m is the resolution of the X and Y image length, and n is the resolution of the X and Y image width;

[0023] Calculate the image difference between the low-resolution image of the first phase and the low-resolution image of the second phase, and input the image difference into the trained super-resolution convolutional neural network to obtain the transition residual image between the first and second phases;

[0024] The image difference between the second phase low-resolution image and the third phase low-resolution image is calculated, and the image difference is input into the trained super-resolution convolutional neural network to obtain the second and third phase transition residual image.

[0025] Optionally, the step of inputting the first two-phase transition residual image into a bias feature extraction convolutional neural network to obtain a first two-phase sensor deviation, and inputting the second three-phase transition residual image into a bias feature extraction convolutional neural network to obtain a second three-phase sensor deviation is specifically:

[0026] Using loss function

[0027] loss bia (θ bia )=MSE(ΔL ij -ΔF ij ,BIACNN(ΔF ij ,θ bia ))

[0028] Train the bias feature extraction convolutional neural network, where BIACNN() represents the mapping function of the bias feature extraction convolutional neural network, θ bia Represents the training weight parameter of the bias feature extraction convolutional neural network mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔF ij is the residual image of the transition between the i-th and j-th phases;

[0029] Inputting the first two-phase transition residual image into the trained bias feature extraction convolutional neural network to obtain the first two-phase sensor deviation;

[0030] The second three-phase transition residual image is input into the trained bias feature extraction convolutional neural network to obtain the second three-phase sensor deviation.

[0031] Optionally, the biased feature extraction convolutional neural network includes an input layer, three convolutional hidden layers and an output layer, and the three convolutional hidden layers correspond to feature extraction operations, nonlinear mapping operations and reconstruction operations, respectively.

[0032] Optionally, the step of inputting the second phase forward transition high-resolution image and the second phase backward transition high-resolution image into a time fusion convolutional neural network to obtain the second phase high-resolution reconstructed image is specifically as follows:

[0033] The Bicubic interpolation method is used to upsample the first phase low-resolution image, the second phase low-resolution image and the third phase low-resolution image to obtain the first phase upsampled image, the second phase upsampled image and the third phase upsampled image, wherein the upsampling ratio is the same as the sampling ratio of the image after high-resolution reconstruction;

[0034] Subtract the first time phase upsampled image from the second time phase upsampled image to obtain the first two time phase upsampled residual image, and subtract the second time phase upsampled image from the third time phase upsampled image to obtain the second three time phase upsampled residual image;

[0035] Using loss function

[0036] loss ti (θ ti )=MSE(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,θ ti ))-SSIM(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,θ ti ))

[0037] Train the time fusion convolutional neural network, where TICNN() represents the mapping function of the time fusion convolutional neural network, θ ti represents the training weight parameter of the mapping function, i, j, k represents the i, j, kth phase in the training set, L j is the high-resolution image at the jth phase, L ij is the forward transition high-resolution image of the jth phase from the ith phase, L kj is the high-resolution image of the backward transition from the kth phase to the jth phase, ΔC ij =C j -C i is the residual image sampled at the i,j time phase, ΔC jk =C j -C k is the upsampled residual image at the j,kth time phase, C i ,C j ,C kare respectively the upsampled images of the i, j, and k phases obtained by upsampling the low-resolution images of the i, j, and k phases, is the image structure similarity function, where μ x ,μ y Respectively represent the mean of all elements in image X, Y, σ x ,σ y Respectively represent the standard deviation of all elements in image X, Y, σ xy Represents the covariance of the elements in the image X, Y. c1 and c2 are two small constants to prevent the denominator from being 0.

[0038] The second phase forward transition high-resolution image, the second phase backward transition high-resolution image, the first two-phase upsampled residual image and the second three-phase upsampled residual image are input into the trained time fusion convolutional neural network to obtain the second phase high-resolution reconstructed image.

[0039] Optionally, the time-fusion convolutional neural network includes an input layer, three convolutional hidden layers and an output layer;

[0040] The input layer is used to stack the first two-phase upsampled residual image, the second three-phase upsampled residual image, the second phase forward transition high-resolution image and the second phase backward transition high-resolution image to form a four-channel data input model;

[0041] The three hidden layers correspond to feature extraction operation, nonlinear mapping operation and weight extraction operation respectively, and the hidden layer corresponding to weight extraction outputs a 1×s×t tensor W, where s and t are the length and width resolutions of the high-resolution image;

[0042] The output layer output formula is:

[0043]

[0044] In the formula, For the high-resolution reconstruction of the second phase, L 12 It is the high-resolution image of the second phase forward transition, L 32 is the high-resolution image of the backward transition in the second phase, Ι represents a 1×s×t tensor whose elements are all 1, W is a 1×s×t tensor output by the previous layer, and “*” represents the multiplication of elements at corresponding positions.

[0045] Optionally, the warning display unit performs classified warning display of the degree of damage caused by pests and diseases in the crop planting area through text labels or color labels.

[0046] Beneficial effects of the present invention:

[0047] The remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images of the present invention fully considers the differences between different sensors in its remote sensing image super-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion, and learns the errors of two sensors by training a bias feature convolutional neural network, thereby improving the accuracy of high-resolution reconstruction of remote sensing images. It also fully considers that remote sensing images are time series data, and different time length changes have different influences on reconstructed date images. On the one hand, by training a time fusion convolutional neural network, the influence of image differences caused by time changes at each pixel point on the reconstructed date image is learned, and this is used as the time fusion weight of the pixel point, which is more expressive and more accurate than the traditional method of directly using residual image values ​​to calculate weights with linear functions. On the other hand, in addition to the classic MSE loss function, the SSIM structural similarity function is also added to the model loss function, and the closer the SSIM value is to 1, the higher the structural similarity of the two images is, which can ensure that the optimized model not only has a small least squares loss, but also maintains a good structural similarity with the real picture, and has a better visual effect. The remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images designed on this basis can, on the one hand, overcome the shortcomings of traditional field survey technology of pests and diseases and UAV remote sensing monitoring methods, such as high cost, low coverage and low efficiency; on the other hand, it can overcome the shortcomings of insufficient temporal or spatial resolution of single satellite sensor images, and insufficient spectral, spatial distortion and temporal feature fusion of traditional spatiotemporal fusion algorithms; as a whole, it realizes the timely acquisition and accurate monitoring, identification and early warning of the spatial distribution and development trends of crop diseases and pests in crop fields, and realizes wider coverage, large-scale, multi-regional, visual and intelligent crop planting area monitoring with higher efficiency and lower cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] in:

[0050] Figure 1 This is an architecture diagram of a remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images in one embodiment;

[0051] Figure 2 A flowchart of performing high-resolution reconstruction on an input low-resolution image without a paired high-resolution image in one embodiment;

[0052] Figure 3A schematic diagram of a logic framework for performing high-resolution reconstruction on an input low-resolution image without a paired high-resolution image in one embodiment;

[0053] Figure 4 A flowchart of obtaining a high-resolution reconstructed image by using a temporal fusion convolutional neural network in one embodiment;

[0054] Figure 5 A schematic diagram of a temporal fusion convolutional neural network structure in one embodiment;

[0055] Figure 6 This is a diagram showing the actual monitoring effect of a remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images in one embodiment;

[0056] Figure 7 It is a high-resolution image sequence obtained based on high-resolution reconstruction of remote sensing images in an embodiment. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] It should be noted that the terms "include", "comprises" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In the claims, specification and drawings of the present application, relational terms such as "first" and "second" are merely used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] refer to Figure 1 As shown, in one embodiment, a remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images is proposed, comprising:

[0062] A collection unit is used to collect remote sensing images of sensors in the same key monitoring area for crop diseases and insect pests;

[0063] A preprocessing unit is used to perform sensor calibration, atmosphere calibration, solar altitude calibration, terrain calibration, geometric calibration and coordinate alignment preprocessing on the collected remote sensing images of the monitoring area to obtain images after preprocessing;

[0064] A reconstruction unit is used to receive the pre-processed image, and use a remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion to perform high-resolution reconstruction on the input low-resolution image without a paired high-resolution image, so as to obtain a high-resolution reconstructed image paired with the low-resolution image;

[0065] An extraction unit, used to extract the key monitoring crop planting range in the monitoring area by using a preset crop range extraction model on the high-resolution reconstructed image;

[0066] An identification unit is used to identify crop pests and diseases based on the extracted spectral information of the key monitoring crop planting range and the corresponding range remote sensing image using a preset pest and disease identification model to obtain the regional monitoring crop pest and disease situation;

[0067] The early warning display unit analyzes and evaluates the crop disease and insect pest situation monitored in the crop planting area according to the identification unit, and uses the preset disease and insect pest expert guidance system to perform graded and classified early warning display of crop diseases and insect pests in the current planting area.

[0068] Specifically, the working principle and workflow of each structural unit are described in detail as follows.

[0069] First, the acquisition unit obtains remote sensing images of two sensors in the same key monitoring area of ​​crop pests and diseases, wherein the image of sensor one is a low temporal and high spatial resolution (hereinafter referred to as high resolution) image, and the image sequence of sensor two is a paired high temporal and low spatial resolution (hereinafter referred to as low resolution) image sequence, and the paired high and low resolution image sequences constitute a training sample set, which will be used to train the neural network designed by the present invention. In addition to the low-resolution images with paired high-resolution images, there are still a large number of high-resolution images without paired images in the low-resolution image sequence, and the reconstruction unit of the system will reconstruct paired high-resolution images for them; the acquisition unit sends the high-resolution and low-resolution remote sensing image sequences to the preprocessing unit.

[0070] The preprocessing unit performs sensor calibration, atmosphere calibration, solar altitude calibration, terrain calibration, geometric calibration and coordinate alignment preprocessing on the collected low-resolution image sequence and the paired high-resolution image sequence of the monitoring area to obtain the remote sensing image after preprocessing. The preprocessing unit sends the remote sensing image after preprocessing to the reconstruction unit.

[0071] The reconstruction unit inputs the preprocessed remote sensing image into the reconstruction unit, uses a remote sensing image super-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion to perform high-resolution reconstruction on the image to be identified, obtains a high-resolution reconstructed image of the image, and the reconstruction unit sends the high-resolution reconstructed image of the image to be identified to the extraction unit.

[0072] The extraction unit uses the image feature extraction and segmentation algorithm in the field of image processing technology to extract the key monitoring crop planting range based on the preset crop planting range extraction model on the high-resolution reconstructed image, and obtains the key monitoring crop planting range of the region. The extraction unit sends the monitoring crop planting range extraction result to the recognition unit. The preset crop planting range extraction model here is a trained crop range extraction model, which inputs remote sensing images and crop categories, and outputs crop planting ranges to facilitate the next step of pest and disease identification.

[0073] The recognition unit, based on the extracted spectral information of the key monitored crop planting range and the remote sensing image of the corresponding range, uses a pattern recognition algorithm based on image processing and a preset pest and disease recognition model to identify the crop pest and disease situation, and obtains the key monitored crop pest and disease situation in the region. The preset pest and disease recognition model here is a trained recognition model, which inputs the crop planting range and the spectral information of each pixel in the corresponding range, and outputs whether the spectrum is normal, whether there are pests and diseases, and what kind of pests and diseases it may be.

[0074] The early warning display unit analyzes and evaluates the crop pest and disease monitoring situation in the crop planting area according to the identification unit, and uses the preset pest and disease expert guidance system to classify and classify crop pests and diseases in the current planting area. For example, according to the "Regulations on the Prevention and Control of Crop Pests and Diseases", crop pests and diseases are divided into three categories according to the characteristics of crop pests and diseases and the degree of harm to agricultural production: Class I crop pests and diseases, Class II crop pests and diseases, and Class III crop pests and diseases. Therefore, early warnings can be given based on the classification identification of pests and diseases in the crop planting area, such as: text identifications such as Class I, Class II, Class III, and No Pests and Diseases, or color identifications for the classification of the degree of pest and disease damage in the crop planting area can be used for early warning, such as: red areas, yellow areas, orange areas, and green areas correspond to Class I, Class II, Class III, and No Pests and Diseases, respectively, so as to monitor the crop planting area in a large-scale, multi-regional, visual, and intelligent manner. The preset pest and disease expert guidance system here classifies pests and diseases according to the proportion of abnormal pixels output by the pest and disease recognition model.

[0075] In one embodiment, the warning display unit performs classified warning display of the degree of damage caused by pests and diseases in the crop planting area through text labels or color labels.

[0076] refer to Figure 2 , Figure 3 As shown, in one embodiment, a remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion is used to reconstruct a low-resolution image without a paired high-resolution image that is input, specifically comprising:

[0077] Step 100, obtaining a first phase low-resolution image M1, a second phase low-resolution image M2, and a third phase low-resolution image M3, wherein the first phase low-resolution image M1 has a paired first phase high-resolution image L1 of the same region and the same phase, and the third phase low-resolution image M3 has a paired third phase high-resolution image L3 of the same region and the same phase.

[0078] It should be noted that the embodiment of the present invention first obtains remote sensing images of two sensors for a specific planting structure area, wherein the image of sensor one is a high-resolution image, and the image sequence is L1, L2, L3..., and the image of sensor two is a paired low-resolution image sequence M1, M2, M3..., and the paired high- and low-resolution image sequences constitute a training sample set, which will be used to train the neural network designed by the present invention.

[0079] In addition to the low-resolution images with paired high-resolution images, there are still a large number of images in the low-resolution image sequence that have no paired high-resolution images. The method of the present invention will reconstruct paired high-resolution images for them. In order to clearly and briefly introduce the embodiments of the present invention, in the following embodiments, the reconstruction part only uses three low-resolution image sequences M1, M2, M3 with different phases and two paired high-resolution image sequences L1, L3 to reconstruct the high spatial resolution image paired with M2. Take the example of a remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion. By analogy, a high-spatial resolution image without pairing in a low-resolution image sequence can be reconstructed, thereby obtaining a high-temporal and high-spatial resolution image sequence. In addition, in order to understand the meaning of various physical quantities consistently, the training set parameter representation of each step of the neural network training phase and the parameter representation in the prediction, recognition, and reconstruction steps based on the neural network all use the same parameter representation method. This is a brief description method that can be understood by ordinary technicians in this field.

[0080] Step 200: Input the first phase low-resolution image M1 and the second phase low-resolution image M2 into a super-resolution convolutional neural network to obtain a first-phase transition residual image ΔF 12 , the second phase low-resolution image M2 and the third phase low-resolution image M3 are input into the super-resolution convolutional neural network to obtain the second and third phase transition residual image ΔF 32 .

[0081] In one embodiment, step 200 specifically further includes:

[0082] Step 201, using the loss function formula

[0083] loss sr (θ sr )=MSE(ΔL ij ,SRCNN(ΔM ij ,θ sr ))

[0084] Train the super-resolution convolutional neural network, where SRCNN() represents the mapping function of the super-resolution convolutional neural network, θ sr Represents the training weight parameter of the mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔM ij =M j -M i is the j-th phase low-resolution image M in the training setj and the low-resolution image M at the i-th phase i The difference, Where ||·|| represents the Euclidean modulus length, m is the resolution of the X and Y image length, and n is the resolution of the X and Y image width;

[0085] Step 202: Calculate the image difference ΔM between the first phase low-resolution image M1 and the second phase low-resolution image M2 12 =M2-M1, the image difference ΔM 12 Input the trained super-resolution convolutional neural network to obtain the first and second phase transition residual image ΔF 12 =SRCNN(ΔM 12 ,θ sr ), where θ sr Represents the weight parameters of the trained super-resolution convolutional neural network; calculates the image difference ΔM between the second phase low-resolution image M2 and the third phase low-resolution image M3 32 =M3-M2, the image difference ΔM 32 Input the trained super-resolution convolutional neural network to obtain the second and third phase transition residual image ΔF 32 =SRCNN(ΔM 32 ,θ sr ).

[0086] It should be noted that the super-resolution convolutional neural network (SRCNN) used in this embodiment belongs to the prior art. For details, see Dong, C., et al. "Image Super-Resolution Using Deep Convolutional Networks." IEEE Trans Pattern Anal Mach Intell 38.2 (2016): 295-307.

[0087] Step 300: convert the first phase transition residual image ΔF 12 Input bias feature extraction convolutional neural network to obtain the first and second phase sensor deviation B 12 , the second three-phase transition residual image ΔF 32 Input bias feature extraction convolutional neural network to obtain the second and third phase sensor deviation B 32 .

[0088] In one embodiment, step 300 specifically further includes:

[0089] Step 301, using loss function

[0090] loss bia (θ bia )=MSE(ΔL ij -ΔF ij ,BIACNN(ΔF ij ,θ bia ))

[0091] Train the bias feature extraction convolutional neural network, where BIACNN() represents the mapping function of the bias feature extraction convolutional neural network, θ bia Represents the training weight parameter of the bias feature extraction convolutional neural network mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔF ij is the residual image of the transition between the i-th and j-th phases;

[0092] Step 302: convert the first phase transition residual image ΔF 12 Input the trained bias feature extraction convolutional neural network to obtain the first and second phase sensor deviation ΔB 12 =BIACNN(ΔF 12 ,θ bia ), where θ bia represents the weight parameter of the trained bias feature extraction convolutional neural network; the second and third phase transition residual image ΔF 32 Input the trained bias feature extraction convolutional neural network to obtain the second and third phase sensor deviation ΔB 32 =BIACNN(ΔF 32 ,θ bia ).

[0093] It should be noted that during the training of the bias feature extraction convolutional neural network, ΔF ij The result is calculated by using the trained super-resolution convolutional neural network according to step 200 and step 202.

[0094] In one embodiment, the biased feature extraction convolutional neural network includes an input layer, three convolutional hidden layers and an output layer, and the three convolutional hidden layers correspond to feature extraction operations, nonlinear mapping operations and reconstruction operations, respectively.

[0095] Step 400: The first phase high-resolution image L1 and the first two phase transition residual image ΔF 12 and the first and second phase sensor deviation B 12 Add together to obtain the high-resolution image L of the second phase forward transition 12, the third phase high resolution image L3, the second three phase transition residual image ΔF 32 and the second three-phase sensor deviation B 32 Add together to obtain the high-resolution image L of the second phase backward transition 32 .

[0096] Step 500: forward the high-resolution image L of the second phase 12 and the second phase backward transition high-resolution image L 32 Input the time fusion convolutional neural network to obtain the high-resolution reconstructed image of the second phase

[0097] refer to Figure 4 As shown, in one embodiment, step 500 specifically further includes:

[0098] Step 501, using Bicubic interpolation method to upsample the first phase low-resolution image M1, the second phase low-resolution image M2 and the third phase low-resolution image M3 to obtain the first phase upsampled image C1, the second phase upsampled image C2 and the third phase upsampled image C3, wherein the upsampling ratio is the same as the sampling ratio of the image after high-resolution reconstruction;

[0099] Step 502: Subtract the first phase upsampled image C1 from the second phase upsampled image C2 to obtain a first phase upsampled residual image ΔC. 12 = C2-C1, subtract the second phase upsampled image C2 from the third phase upsampled image C3 to obtain the second three phase upsampled residual image ΔC 32 =C2-C1;

[0100] Step 503, using loss function

[0101] loss ti (θ ti )=MSE(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,θ ti ))-SSIM(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,θ ti ))

[0102] Train the time-fusion convolutional neural network, where TICNN() represents the mapping function of the time-fusion convolutional neural network, θ tirepresents the training weight parameter of the mapping function, i, j, k represents the i, j, kth phase in the training set, L j is the high-resolution image at the jth phase, L ij is the forward transition high-resolution image of the jth phase from the ith phase, L kj is the high-resolution image of the backward transition from the kth phase to the jth phase, ΔC ij =C j -C i is the residual image sampled at the i,j time phase, ΔC jk =C j -C k is the upsampled residual image at the j,kth time phase, C i ,C j ,C k are respectively the upsampled images of the i, j, and k phases obtained by upsampling the low-resolution images of the i, j, and k phases, is the image structure similarity function, where μ x ,μ y Respectively represent the mean of all elements in image X, Y, σ x ,σ y Respectively represent the standard deviation of all elements in image X, Y, σ xy Represents the covariance of the elements in the image X, Y. c1 and c2 are two small constants to prevent the denominator from being 0.

[0103] Step 504: forward the high-resolution image L of the second phase 12 , the second phase backward transition high-resolution image L 32 , the first two phase upsampling residual image ΔC 12 and the residual image ΔC upsampled from the second and third phases 32 Input the trained time fusion convolutional neural network to obtain the high-resolution reconstructed image of the second phase

[0104] It should be noted that in the training process of the time-fusion convolutional neural network, L ij , L kj Calculated according to step 400.

[0105] It should be noted that in the above description of a high-resolution reconstruction method for remote sensing images, only one, two, and three phases are used. However, in the training of three convolutional neural networks, any three images in the set can form a combination of three phases, and i, j, and k correspond to the one, two, and three phases, respectively.

[0106] refer to Figure 5 As shown, in one embodiment, the time-fusion convolutional neural network includes an input layer, three convolutional hidden layers, and an output layer;

[0107] The input layer is used to stack the first two-phase upsampled residual image, the second three-phase upsampled residual image, the second phase forward transition high-resolution image and the second phase backward transition high-resolution image to form a four-channel data input model;

[0108] The three hidden layers correspond to feature extraction operations, nonlinear mapping operations, and weight extraction operations, respectively. The hidden layer corresponding to weight extraction outputs a 1×s×t tensor W, where s and t are the length and width resolutions of the high-resolution image.

[0109] The output formula of the output layer is

[0110]

[0111] In the formula, L is the high-resolution reconstruction image of the second phase. 12 It is the high-resolution image of the second phase forward transition, L 32 is the high-resolution image of the backward transition in the second phase, Ι represents a 1×s×t tensor whose elements are all 1, W is a 1×s×t tensor output by the previous layer, and “*” represents the multiplication of elements at corresponding positions.

[0112] refer to Figure 6 As shown, The high-resolution image paired with M2 is reconstructed based on the low-resolution image sequence M1, M2, M3 and the high-resolution image sequence L1, L3 acquired by two different sensors. It not only ensures the same resolution as the high-resolution image sequence, but also maintains the temporal change information of the low-resolution time series image. Similarly, the high-spatial resolution image without pairing in the low-resolution image sequence can be reconstructed, thereby obtaining a high temporal and high-spatial resolution image sequence.

[0113] refer to Figure 7 As shown, All of them are reconstructed images, which together with L1 and L5 constitute a complete high-resolution image sequence.

[0114] It can be understood from the above embodiments that the advantages and beneficial technical effects of the present invention are as follows:

[0115] The remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images of the present invention, the remote sensing image super-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion fully considers the differences between different sensors, and learns the errors of two sensors by training the bias feature convolutional neural network, so as to improve the accuracy of high-resolution reconstruction of remote sensing images; it also fully considers that the remote sensing image is a time series data, and the influence of different time length changes on the reconstructed date image is different. On the one hand, by training the time fusion convolutional neural network, the influence of the image difference caused by the time change at each pixel point on the reconstructed date image is learned, and this is used as the time fusion weight of the pixel point, which is more expressive and accurate than the traditional method of directly using the residual image value to calculate the weight with a linear function; on the other hand, in addition to the classic MSE loss function, the SSIM structural similarity is also added to the model loss function. The closer the SSIM value is to 1, the higher the structural similarity of the two images is, which can ensure that the optimized model not only has a small least squares loss, but also maintains a good structural similarity with the real picture, and has a better visual effect. On this basis, the remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images designed can overcome the shortcomings of traditional field survey technology for pests and diseases and UAV remote sensing monitoring methods, such as high cost, low coverage, and low efficiency. On the other hand, it can overcome the shortcomings of insufficient time or spatial resolution of single satellite sensor images, and insufficient spectral, spatial distortion, and time feature fusion of traditional spatiotemporal fusion algorithms. As a whole, it realizes the timely acquisition, accurate monitoring, identification, and early warning of the spatial distribution and development trends of crop pests and diseases in crop fields, and realizes wider coverage, large-scale, multi-regional, visualized, and intelligent crop planting area monitoring, with higher efficiency and lower cost.

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images, characterized in that: include: A collection unit is used to collect remote sensing images of sensors in the same key monitoring area for crop diseases and insect pests; A preprocessing unit is used to perform sensor calibration, atmosphere calibration, solar altitude calibration, terrain calibration, geometric calibration and coordinate alignment preprocessing on the collected remote sensing images of the monitoring area to obtain images after preprocessing; A reconstruction unit is used to receive the pre-processed image, and use a remote sensing image high-resolution reconstruction method based on sensor error correction and spatiotemporal data fusion to perform high-resolution reconstruction on the input low-resolution image without a paired high-resolution image, so as to obtain a high-resolution reconstructed image paired with the low-resolution image; An extraction unit is used to extract the high-resolution reconstructed image to obtain the key monitoring crop planting range in the monitoring area; An identification unit is used to identify the crop pests and diseases according to the extracted spectral information of the key monitored crop planting range and the corresponding range remote sensing image using a preset pest and disease identification model to obtain the crop pest and disease conditions of the monitored area; The early warning display unit analyzes and evaluates the crop pest and disease situation in the crop planting area based on the identification unit, and uses the preset pest and disease expert guidance system to classify and classify the crop pest and disease in the current planting area. The method for high-resolution reconstruction of remote sensing images based on sensor error correction and spatiotemporal data fusion is used to reconstruct high-resolution images of input low-resolution images without paired high-resolution images, and includes the following steps: Acquire a first phase low-resolution image, a second phase low-resolution image, and a third phase low-resolution image, wherein the first phase low-resolution image has a paired first phase high-resolution image of the same region and the same phase, and the third phase low-resolution image has a paired third phase high-resolution image of the same region and the same phase; Inputting the first phase low-resolution image and the second phase low-resolution image into a super-resolution convolutional neural network to obtain a first two-phase transition residual image, and inputting the second phase low-resolution image and the third phase low-resolution image into a super-resolution convolutional neural network to obtain a second three-phase transition residual image; Input the first two-phase transition residual image into the bias feature extraction convolutional neural network to obtain the first two-phase sensor bias, and input the second three-phase transition residual image into the bias feature extraction convolutional neural network to obtain the second three-phase sensor bias; The first phase high-resolution image, the first two-phase transition residual image and the first two-phase sensor deviation are added to obtain a second phase forward transition high-resolution image, and the third phase high-resolution image, the second three-phase transition residual image and the second three-phase sensor deviation are added to obtain a second phase backward transition high-resolution image; Inputting the second phase forward transition high-resolution image and the second phase backward transition high-resolution image into a time fusion convolutional neural network to obtain a second phase high-resolution reconstructed image; The step of inputting the first phase low-resolution image and the second phase low-resolution image into a super-resolution convolutional neural network to obtain a first two-phase transition residual image, and inputting the second phase low-resolution image and the third phase low-resolution image into a super-resolution convolutional neural network to obtain a second three-phase transition residual image is specifically: Use the loss function formula loss sr (i sr )=MSE(ΔL ij ,SRCNN(ΔM ij ,i sr )) Train the super-resolution convolutional neural network, where SRCNN() represents the mapping function of the super-resolution convolutional neural network, θ sr Represents the training weight parameter of the mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔM ij =M j -M i is the j-th phase low-resolution image M in the training set j and the low-resolution image M at the i-th phase i The difference, Where ||·|| represents the Euclidean modulus length, m is the resolution of the X and Y image length, and n is the resolution of the X and Y image width; Calculate the image difference between the low-resolution image of the first phase and the low-resolution image of the second phase, and input the image difference into the trained super-resolution convolutional neural network to obtain the transition residual image between the first and second phases; Calculating the image difference between the second phase low-resolution image and the third phase low-resolution image, and inputting the image difference into the trained super-resolution convolutional neural network to obtain the second and third phase transition residual image; The step of inputting the first two-phase transition residual image into the bias feature extraction convolutional neural network to obtain the first two-phase sensor deviation, and inputting the second three-phase transition residual image into the bias feature extraction convolutional neural network to obtain the second three-phase sensor deviation is specifically: Using loss function loss bia (i bia )=MSE(ΔL ij -ΔF ij ,BIACNN(ΔF ij ,i bia )) Train the bias feature extraction convolutional neural network, where BIACNN() represents the mapping function of the bias feature extraction convolutional neural network, θ bia Represents the training weight parameter of the bias feature extraction convolutional neural network mapping function, ΔL ij =L j -L i is the j-th phase high-resolution image L in the training set j and the high-resolution image L at the i-th phase i The difference, ΔF ij is the residual image of the transition between the i-th and j-th phases; Inputting the first two-phase transition residual image into the trained bias feature extraction convolutional neural network to obtain the first two-phase sensor deviation; Inputting the second three-phase transition residual image into the trained bias feature extraction convolutional neural network to obtain the second three-phase sensor deviation; The step of inputting the second phase forward transition high-resolution image and the second phase backward transition high-resolution image into the time fusion convolutional neural network to obtain the second phase high-resolution reconstructed image is specifically as follows: The Bicubic interpolation method is used to upsample the first phase low-resolution image, the second phase low-resolution image and the third phase low-resolution image to obtain the first phase upsampled image, the second phase upsampled image and the third phase upsampled image, wherein the upsampling ratio is the same as the sampling ratio of the image after high-resolution reconstruction; Subtract the first time phase upsampled image from the second time phase upsampled image to obtain the first two time phase upsampled residual image, and subtract the second time phase upsampled image from the third time phase upsampled image to obtain the second three time phase upsampled residual image; Using loss function loss ti (i ti )=MSE(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,i ti ))-SSIM(L j ,TICNN(ΔC ij ,ΔC jk ,L ij ,L kj ,i ti )) Train the time fusion convolutional neural network, where TICNN() represents the mapping function of the time fusion convolutional neural network, θ ti represents the training weight parameter of the mapping function, i, j, k represents the i, j, kth phase in the training set, L j is the high-resolution image at the jth phase, L ij is the forward transition high-resolution image of the jth phase from the ith phase, L kj is the high-resolution image of the backward transition from the kth phase to the jth phase, ΔC ij =C j -C i is the residual image sampled at the i,j time phase, ΔC jk =C j -C k is the upsampled residual image at the j,kth time phase, C i ,C j ,C k are respectively the upsampled images of the i, j, and k phases obtained by upsampling the low-resolution images of the i, j, and k phases, is the image structure similarity function, where μ x ,μ y Respectively represent the mean of all elements in image X, Y, σ x ,σ y Respectively represent the standard deviation of all elements in image X, Y, σ xy Represents the covariance of the elements in the image X, Y. c1 and c2 are two small constants to prevent the denominator from being 0. The second phase forward transition high-resolution image, the second phase backward transition high-resolution image, the first two-phase upsampled residual image and the second three-phase upsampled residual image are input into the trained time fusion convolutional neural network to obtain the second phase high-resolution reconstructed image.

2. A remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images as claimed in claim 1, characterized in that: The bias feature extraction convolutional neural network includes an input layer, three convolution hidden layers and an output layer, and the three convolution hidden layers correspond to feature extraction operations, nonlinear mapping operations and reconstruction operations respectively.

3. A remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images as claimed in claim 1, characterized in that: The time-fusion convolutional neural network includes an input layer, three convolutional hidden layers and an output layer; The input layer is used to stack the first two-phase upsampled residual image, the second three-phase upsampled residual image, the second phase forward transition high-resolution image and the second phase backward transition high-resolution image to form a four-channel data input model; The three convolutional hidden layers correspond to feature extraction operations, nonlinear mapping operations, and weight extraction operations, respectively. The hidden layer corresponding to weight extraction outputs a 1×s×t tensor W, where s and t are the length and width resolutions of the high-resolution image. The output layer output formula is: In the formula, For the high-resolution reconstruction of the second phase, L 12 It is the high-resolution image of the second phase forward transition, L 32 is the high-resolution image of the second phase backward transition, Ι represents a 1×s×t tensor with all elements equal to 1, W is a 1×s×t tensor output by the previous layer, and "*" represents the multiplication of elements at corresponding positions.

4. A remote sensing monitoring and identification system based on high-resolution reconstruction of remote sensing images as described in claim 1, characterized in that the early warning display unit classifies and displays the degree of damage caused by pests and diseases in crop planting areas through text labels or color labels.

Images