A forest land change detection method based on unmanned aerial vehicle and satellite image

Abstract

The application discloses a forest land change detection method based on a UAV and a satellite image, and comprises the following steps: step one: taking one image from a satellite image data set, obtaining UAV image data according to the longitude and latitude of the satellite image, and generating a corresponding UAV pseudo-satellite image; step two: performing seasonal color enhancement processing on the satellite image data according to the pseudo-satellite image generated by the UAV image data, so that the satellite image data is closer to the UAV pseudo-satellite image; step three: directly obtaining the image difference between the UAV pseudo-satellite image and the satellite image after the enhancement processing by using an improved Siam-HRNet deep learning algorithm; and step four: judging whether there is unprocessed satellite image data in the satellite image data set, and if yes, returning to step one; if not, ending. The method disclosed by the application extracts a UAV orthographic video to generate pseudo-satellite data, and compares the pseudo-satellite data with satellite image data, so as to generate forest land change data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of forest protection, in particular to a forest land change detection method based on unmanned aerial vehicles and satellite images. BACKGROUND

[0002] It is particularly important to find out the current situation of forest resources through annual forest land change surveys and make scientific and reasonable planning and protection. Since 2017, the annual forest land change survey has been implemented nationwide. Due to the short start, different technical means in different regions and forestry bureaus have caused certain difficulties in the management and development of forest resources.

[0003] Remote sensing satellite images are one of the main survey methods and one of the most important survey methods for current forest land change surveys. Staff can better understand the change rules of local forest land by analyzing remote sensing images of the same forest land at different time periods, and become the core means of change enforcement. However, due to weather reasons such as clouds, fog, and rain, and the limitation of satellite costs, there are problems of unclear remote sensing images and untimely satellite image updates, especially the free forest satellite image slices are not updated for several years at a time, which makes it difficult to detect the need for forest land change information in a timely manner.

[0004] Unmanned aerial vehicle videos can well fill this gap. It can be targeted for flight shooting to get the most timely first-hand data, but the latest data of unmanned aerial vehicles lacks the historical nature of satellite data, so it is best to combine the two methods to provide a comparison method that can solve the great difference between the two types of images.

[0005] Related materials:

[0006] LAB color space, https: / / baike.baidu.com / item / Lab%E8%89%B2%E5BD%A9%E7%A9%BA%E9%97%B4 / 6833664.

[0007] HRNet, https: / / arxiv.org / abs / 1902.09212.

[0008] OCRNet, https: / / arxiv.org / abs / 1909.11065. SUMMARY

[0009] In view of at least one defect of the prior art, the purpose of the present application is to provide a forest land change detection method based on unmanned aerial vehicles and satellite images, which extracts unmanned aerial vehicle orthographic videos to generate pseudo-satellite image data, and compares the pseudo-satellite image data with satellite image data to generate forest land change data.

[0010] In order to achieve the above object, the present application adopts the following technical scheme: a forest land change detection method based on unmanned aerial vehicle and satellite image, comprising the following steps:

[0011] Step one: take an image in the satellite image data set, and obtain the corresponding unmanned aerial vehicle image data according to the latitude and longitude of the satellite image to generate an unmanned aerial vehicle satellite image;

[0012] Step two: according to the satellite image generated by the unmanned aerial vehicle image data, the satellite image is subjected to seasonal color enhancement processing, so that it is closer to the unmanned aerial vehicle satellite image;

[0013] Step three: using the improved Siam-HRNet deep learning algorithm, the image difference between the unmanned aerial vehicle satellite image and the enhanced satellite image is directly obtained;

[0014] Step four: judge whether there are untreated satellite images in the satellite image data set, if yes, go to step one; if no, end.

[0015] Step one includes:

[0016] Step 1.1: take an image in the satellite image data set of the maximum level, denoted as IMG1; the resolution of its satellite image is set to K×K, and the latitude and longitude of its upper left corner and lower right corner are set as <x1, y1, x2, y2>;

[0017] Step 1.2: find all images within the latitude and longitude range of <x1, y1, x2, y2> from the flight log and the corresponding video of the unmanned aerial vehicle, and perform orthographic correction and image stitching to splice them into a large orthographic geographic information image;

[0018] Step 1.3: from the orthographic geographic information image generated in step 1.2, crop a new unmanned aerial vehicle image according to the GPS position <x1, y1, x2, y2>, and scale it to K×K size; thus, the unmanned aerial vehicle video generates a satellite image, denoted as IMG2.

[0019] Step two includes:

[0020] Step 2.1: convert IMG1 and IMG2 into LAB space images to obtain new image data, denoted as IMG1_1 and IMG2_1;

[0021] Step 2.2: zero mean and unit variance of IMG1_1 generated in step 2.1 to obtain normalized image IMG1_2;

[0022] Step 2.3: then multiply each channel of the normalized IMG1_2 by the variance of each channel of the IMG2_1 image, and add the mean value of each channel of the IMG2_1 image to obtain the image IMG1_3;

[0023] Step 2.4: convert the new generated image IMG1_3 from LAB color space back to RGB color space; save as IMG1_4.

[0024] Step three comprises:

[0025] The improved Siam-HRNet discriminant network provides the capability of end-to-end image comparison and change detection;

[0026] The flow of the network model comprises: 1) input the front and rear period comparison images IMG1_4 and IMG2 into two structure same and weight shared twin network branches respectively for feature extraction; considering that high resolution remote sensing image change detection has the requirement of rich feature and semantic information extraction and utilization capability, HRNet which keeps high resolution representation in the whole feature extraction process is adopted as the feature extraction backbone network; and the pixel attention mechanism is introduced, the feature map obtained through the backbone network is coarsely segmented, then the target region representation is obtained by integrating the pixel feature representation, then the relationship between the pixels and the target region is calculated, and finally the context information enhanced pixel representation is obtained; 2) FCN is adopted as the semantic segmentation head to obtain the multi-classification semantic segmentation result of the front and rear period images; 3) the difference between the semantic segmentation results of the front and rear period remote sensing images obtained above is taken and the absolute value is sent into the difference discriminant network also composed of FCN for binary change detection to obtain the final change detection result image.

[0027] The main features of the improved Siam-HRNet discriminant network are:

[0028] The twin network is introduced on the basis of HRNet, and the common weight is used for pixel-level data discrimination;

[0029] The attention mechanism is added in the FCN network, and the pixel feature representation is enhanced by the pixel semantic information, so that the subsequent semantic segmentation accuracy is improved.

[0030] Significant effect: the application provides a forest land change detection method based on unmanned aerial vehicle and satellite image, which extracts the unmanned aerial vehicle orthographic video to generate pseudo-satellite image data, and compares the pseudo-satellite image data with the satellite image data to generate forest land change data. BRIEF DESCRIPTION OF DRAWINGS

[0031] Fig. 1 is a structural diagram of the application;

[0032] Fig. 2 is a structure diagram of the improved Siam-HRNet detection model. DETAILED DESCRIPTION

[0033] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0034] As Figs. 1-2 shown, the application discloses a method for detecting forest land change by comparing unmanned aerial vehicle (UAV) and satellite image tile data (referred to as satellite image). It can use UAV video to fill the defect of satellite image update not in time, and can use current UAV cruising video to compare historical satellite image in time to obtain current clear forest land change record. The method comprises the following steps: (1) obtaining the longitude and latitude information of the upper left corner and the lower right corner of a satellite image with a resolution of KxK; (2) finding all longitude and latitude images within the satellite image range from the flight log and the corresponding video of the UAV, and projecting the obtained UAV images according to the flight log; (3) image stitching the corrected images to form a complete image containing the corresponding longitude and latitude; (4) re-clipping the correct satellite image comparison data from the generated UAV image according to the satellite image longitude and latitude, and scaling the satellite image to the same size; (5) reading the image color data of the UAV, adding a seasonal filter to the satellite image data according to the mean and variance of the data, and keeping the consistency of the comparison data as much as possible; (6) sending the processed UAV image and the satellite image into a deep learning detector based on an improved Siam-HRNet network for detection, and outputting forest land change image data. The application can simultaneously use original satellite image data and timely UAV data to solve the problem of satellite image data update not in time, so as to obtain first-hand forest land change information.

[0035] In view of various defects of the prior art, the purpose of the application is to provide a method for generating pseudo-satellite image data by extracting UAV orthographic video, and comparing the pseudo-satellite image data with satellite image data to generate forest land change data. It can use UAV data to achieve the effect that can only be achieved by expensive satellite image data, and achieve the goal of timely forest land change survey without large investment.

[0036] In order to achieve the above purpose, the application adopts the following technical scheme: a forest land change detection method based on UAV and satellite image, which comprises the following steps:

[0037] Step one: generating corresponding UAV image data according to satellite image longitude and latitude. It comprises the following sub-steps:

[0038] Step 1.1: taking a piece of image from the largest level of available satellite image data as IMG1. The largest level is generally not less than 18 levels, and is generally between 20-22 levels, the resolution of the satellite image is set to KxK (generally 512x512 or 256x256), and the longitude and latitude of the upper left corner and the lower right corner are taken as <x1, y1, x2, y2>.

[0039] Step 1.2 Find all images in the latitude and longitude range of <x1, y1, x2, y2> from the flight log and corresponding video of the UAV, and orthorectify and mosaic them into a large orthographic geographic information image.

[0040] Step 1.3 Crop a new UAV image according to the GPS position <x1, y1, x2, y2> from the orthographic geographic information image generated in step 1.2, and scale it to KxK size. At this point, we have generated a simulated satellite image using UAV video, denoted as IMG2.

[0041] Step Two: Perform seasonal color enhancement on satellite image data according to the simulated satellite image generated by the UAV, making it closer to the UAV simulated satellite image. It contains the following steps:

[0042] Step 2.1 Convert both IMG1 and IMG2 to LAB space images to obtain new image data, denoted as IMG1_1 and IMG2_1. LAB space is an orthogonal space, and color changes on it are less likely to cause color problems.

[0043] Step 2.2 Normalize IMG1_1 generated in step 2.1 to zero mean and unit variance to obtain normalized image IMG1_2;

[0044] Step 2.3 Then multiply each channel of the normalized IMG1_2 by the variance of each channel of the IMG2_1 image, and add the mean of each channel of the IMG2_1 image to obtain the image IMG1_3.

[0045] Step 2.4 Convert the new generated image IMG1_3 from LAB space to RGB color space and save it as IMG1 again.

[0046] At this point, we have completed the color migration of satellite images to UAV images. Generally speaking, the color of UAV images will be better than that of satellite images, providing better recognition ability. At the same time, due to seasonal reasons (especially summer, autumn, and winter), color migration can introduce seasonal features. Experiments show that color migration of satellite image data can better reduce contrast errors and improve contrast accuracy.

[0047] Step Three: Use the improved Siam-HRNet deep learning algorithm to directly obtain the image difference between the UAV and the satellite.

[0048] The improved Siam-HRNet discriminant network provides end-to-end image comparison and change detection capabilities, and its network structure is as shown in Fig. 2

[0049] ​The main process of its network model includes: 1) the front and rear period contrast image IMG1 and IMG2 are respectively input into two structure same and weight shared twin network branches for feature extraction. Considering that high resolution remote sensing image change detection has the requirement of rich feature and semantic information extraction and utilization ability, the research adopts HRNet which keeps high resolution representation in the whole feature extraction process as the feature extraction backbone network. And the pixel attention mechanism is introduced, the feature map obtained through the backbone network is roughly segmented, then the target area representation is obtained by integrating the pixel feature representation, then the relationship between the pixels and the target area is calculated, and finally the pixel representation enhanced by the context information is obtained. 2) FCN is used as the semantic segmentation head to obtain the multi-class semantic segmentation results of the front and rear period images. 3) The difference between the above obtained semantic segmentation results of the front and rear period remote sensing images is taken and the absolute value is sent into the difference discrimination network composed of FCN to obtain the final change detection result image.

[0050] The main features of the improved Siam-HRNet are:

[0051] The twin network is introduced based on HRNet, and the pixel-level data discrimination is performed on the shared weight.

[0052] The attention mechanism is added in the FCN network, and the pixel feature representation is enhanced by the pixel semantic information to improve the subsequent semantic segmentation accuracy.

[0053] Step four: loop processing until there is no satellite data that needs to be processed in the entire data set. Otherwise, go to step one.

[0054] The complete flow chart is as shown in Fig. 1 .

[0055] Technical protection points:

[0056] A forest change detection method based on unmanned aerial vehicle and satellite image, characterized in that it comprises the following steps: first, locating the unmanned aerial vehicle data from the satellite image data, and taking out the unmanned aerial vehicle data in the same area, and transforming it into the size of the satellite image; next, according to the season data of the unmanned aerial vehicle, performing 3D LUT filter processing on the satellite image data, so as to make the color of the satellite image data consistent with the color of the unmanned aerial vehicle data as much as possible; finally, using the improved Siam-HRNet network to detect the forest change of the unmanned aerial vehicle data and the satellite image data. Thus, the forest change data with good effect can be obtained.

[0057] Technical protection range summary:

[0058] A forest change detection method based on unmanned aerial vehicle and satellite image, characterized in that it comprises:

[0059] Feature one, simultaneously using unmanned aerial vehicle video and satellite image for forest change detection;

[0060] Feature two, after splicing the UAV data, the data is converted into satellite image data, and pixel-level comparison is performed on the scale of satellite image data.

[0061] Feature three, in order to improve the visual effect and detection effect, the color data of the UAV is introduced, the color migration processing is performed on the satellite image data, the satellite image data is added with the corresponding seasonal characteristics of the UAV, and the contrast detection effect is better improved.

[0062] Feature four, an improved Siam-HRNet network is used to learn and detect the corresponding IoU characteristics, in the above processing, a fast and good contrast result can be obtained, and end-to-end data output is realized.

[0063] Finally, it should be noted that: the above enumeration is only a specific embodiment of the present application, of course, the person skilled in the art can modify and change the present application, if the modification and change belong to the scope of the claims of the present application and the equivalent technology, it should be considered that the present application is within the protection scope.

Claims

1. A forest land change detection method based on unmanned aerial vehicle and satellite images, characterized in that, Comprise the following steps: Step one: take a piece of image in the aerial photograph data set, according to the latitude and longitude of the aerial photograph image, obtain the unmanned aerial vehicle image data to generate the corresponding unmanned aerial vehicle aerial photograph image; Step two: according to the aerial photograph image generated by the unmanned aerial vehicle image data, the seasonal color enhancement processing is carried out on the aerial photograph image, so that it is more close to the unmanned aerial vehicle aerial photograph image; Step three: using the improved Siam-HRNet deep learning algorithm, the image difference between the unmanned aerial vehicle aerial photograph image and the enhanced aerial photograph image is directly obtained; Step four: judge whether there is untreated aerial photograph image in the aerial photograph data set, if yes, go to step one; If not, end; Said step three includes: The improved Siam-HRNet discriminant network provides end-to-end image comparison and change detection capability; The main features of the improved Siam-HRNet discriminant network are: Introduce twin network based on HRNet, share weight for pixel level data discrimination; Add attention mechanism in FCN network, enhance pixel feature representation with pixel semantic information, to improve the accuracy of subsequent semantic segmentation; The network model process includes: 1) the front and rear period contrast image IMG1_4 and IMG2 are input into two twin network branches with the same structure and weight sharing for feature extraction; considering the requirement of high resolution remote sensing image change detection for more rich feature and semantic information extraction and utilization ability, HRNet with high resolution representation throughout the feature extraction process is adopted as the feature extraction backbone network; and the pixel attention mechanism is introduced, the feature map obtained through the backbone network is coarsely segmented, then the target region representation is obtained by integrating the pixel feature representation, and then the relationship between the pixels and the target region is calculated, finally the pixel representation enhanced by context information is obtained; 2) FCN is used as the semantic segmentation head to obtain the multi-class semantic segmentation result of the front and rear period images; 3) the difference between the obtained front and rear period remote sensing image semantic segmentation results is taken and the absolute value is sent into the difference discriminant network composed of FCN to obtain the final change detection result image. 2.The UAV and satellite-based forest change detection method according to claim 1, wherein: Said step one includes: Step 1.1: take a piece of image in the aerial photograph data set with the maximum level, denoted as IMG1; the resolution of its aerial photograph image is K×K, and the latitude and longitude of its upper left corner and lower right corner are set as <x1, y1, x2, y2>; Step 1.2: find all images within the latitude and longitude range of <x1, y1, x2, y2> from the flight log and the corresponding video of the unmanned aerial vehicle, and perform orthographic correction and image stitching on them to obtain a large orthographic geographic information image; Step 1.3: from the orthographic geographic information image generated in step 1.2, a new unmanned aerial vehicle image is cropped according to the GPS position <x1, y1, x2, y2>, and scaled to K×K size; thus, the aerial photograph image is generated from the unmanned aerial vehicle video, denoted as IMG2. 3.The UAV and satellite-based forest change detection method of claim 2, wherein: Said step two includes: Step 2.1: convert IMG1 and IMG2 into images in LAB space to obtain new image data, denoted as IMG1_1 and IMG2_1; Step 2.2: Zero-mean and unit-variance is performed on the regenerated IMG1_1 to get the normalized image IMG1_2; Step 2.3: Then the normalized IMG1_2 is multiplied by the variance of each channel of IMG2_1 image, and then the mean of each channel of IMG2_1 image is added to get the image IMG1_3; Step 2.4: The new generated image IMG1_3 is converted from LAB space back to RGB color space; and is saved as IMG1_4.

Images