Forest land state change rapid detection method
By using a self-developed U-Net network and attention mechanism, combined with GPS and IMU data, high-precision geometric correction and pattern comparison of forest land images are performed, solving the problems of long correction time and difficult matching in existing technologies for forest land change detection, and realizing rapid and accurate detection of forest land status changes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SANYA FORESTRY RES INST
- Filing Date
- 2024-02-22
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies for forest change detection suffer from problems such as long correction time, severe ghosting, and difficulty in achieving accurate matching in complex terrain, especially when there is a lack of control points, making effective image correction impossible.
This method employs a self-developed U-Net network structure combined with an attention mechanism to achieve rapid detection of forest land status changes through high-precision geometric correction and pattern comparison of on-site images. It utilizes GPS and IMU data for image geometric correction, introduces coordinate offsets, and combines Siamese networks and an attention mechanism for feature extraction and fusion. It supports both vector and raster image inputs and can quickly identify forest land changes.
It enables real-time and rapid correction and change detection of forest images, improving detection efficiency. It can accurately identify changes in forest status under complex terrain, supports input of multiple data formats, and meets the needs of real-time monitoring.
Smart Images
Figure CN121962879A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of forest land condition change detection, and specifically to a method for detecting forest land condition changes after rapid correction of forest land images. Background Technology
[0002] The common method for image correction is to first correct the image using control points and POS (Position and Orientation System) data, and then stitch multiple images together using feature matching methods such as SIFT, SURF, and ORB to obtain an orthophoto. This method requires control points, and feature matching methods struggle to achieve both short stitching times and no ghosting to meet the requirements for forest change detection. The collinearity equation can be used to derive coordinate transformation relationships using various parameters, but it cannot be solved without control points. It can only be solved by assuming constant height while ignoring topographic relief and not using DEM data. However, the corrected image obtained in this way cannot correct distortions caused by large topographic reliefs, making it difficult to match with previous images.
[0003] U-Net is a deep convolutional neural network (CNN) architecture used for image segmentation tasks. A powerful image segmentation architecture, U-Net boasts excellent segmentation performance and broad application prospects. Its flexibility and performance make it a crucial tool in deep learning image segmentation. U-Net employs an encoder-decoder structure, where the encoder extracts features from the input image, and the decoder maps these features back to a segmentation result at the same resolution as the input image. The decoder part of U-Net typically includes multiple feature maps at different resolutions, allowing the network to handle objects at different scales and better capture detailed information. Attention mechanisms are techniques that mimic human attention patterns, allowing neural network models to dynamically and selectively focus on or highlight different parts of the input data. This enables the model to assign different weights and importance based on the different content of the input. Incorporating attention mechanisms improves model performance, allows the model to better understand the structure and semantics of the input data, and increases the model's interpretability. Summary of the Invention
[0004] The purpose of this invention is to provide a rapid detection method for forest land status changes, including a high-precision image processing stage and a map comparison stage. This method improves the speed of image correction and provides assistance for field forest land investigation using forest land status change detection methods, which has important scientific and practical significance for forest land monitoring.
[0005] This invention is achieved using the following technical solution: The method described in this invention involves acquiring high-precision images on-site and immediately comparing them with previous base maps to quickly obtain results on forest land change status in real time. This includes a high-precision image processing stage and a map comparison stage, specifically: (1) The high-precision processing stage of the on-site acquired images, that is, after the geometric correction of the change recognition images, the self-developed model is input to enhance the on-site acquired images. The self-developed model supports two input methods for the on-site images: vector images and raster images. If it is a vector image, it is rasterized and then data enhancement is carried out to enhance the boundary features. If it is a raster image, it is directly used as the input image. The process includes the following steps: a. Geometric correction is performed on single images collected using GPS and an inertial measurement unit (IMU). Collinearity equations are established using flight attitude, latitude, longitude, and altitude recorded in the POS system. By transforming the collinearity condition equations, the plane coordinates (x, y) of the image point and the spatial coordinates (x, y) of the corresponding ground point can be obtained. A , Y A Z A The mutual transformation between the image points and their corresponding ground points is obtained by:
[0006] Where X S , Y S Z S Let a be the spatial coordinates of the projection center. 1-3 b 1-3 and c 1-3 It is the cosine of each direction composed of three attitude angles, x0 and y0 are the principal point offset of the camera, f is the camera focal length, and Z is the focal length. A -Z S It is a negative number of the flight altitude H; b. Use the standard transformation relationship obtained in step a to correct and resample the image, calculate the pixel correspondence between the corrected image and the original image, and select the bilinear interpolation method that balances interpolation speed and image quality in the interpolation method selection; C. Add a coordinate header file to the resampled image from step b, assigning coordinate system information. Correcting images from a single image often results in discrepancies between the corrected image and the actual ground features. Therefore, obtain the offset by analyzing the deviations of the previous few images along the same flight path. Subsequent images can then be adjusted using this offset. The offset consists of three parameters: xfix, yfix, and zfix. xfix is the offset of the projection center on the x-axis, yfix is the offset of the projection center on the y-axis, and zfix is the offset of the projection center on the z-axis (also the offset of flight altitude H); that is, the spatial coordinates of the projection center shift from (X...) to (...). S , Y S ZS Adjust to (X) S + xfix, Y S +yfix, Z S +zfix), while Z A -Z S It is a negative number of the flight altitude H; d. Compare the matching degree between the image after being assigned coordinate system information and the previous image base map, and calculate the coordinate offsets xfix, yfix, and zfix for this flight path, where xfix is the offset of the projection center on the x-axis, yfix is the offset of the projection center on the y-axis, and zfix is the offset of the flight altitude of the projection center on the z-axis: The coordinate transformation relationship after introducing the offsets shows the image point's planar coordinates (x, y) and the corresponding ground point's spatial coordinates (x, y). A , Y A Z A The corresponding formula for ) is converted to:
[0007] Where X S , Y S Z S Let a be the spatial coordinates of the projection center. 1-3 b 1-3 and c 1-3 It is a cosine of each direction composed of three attitude angles, x0 and y0 are the principal point offset of the camera, f is the camera focal length, and H is the flight altitude; e. After obtaining the coordinate offsets xfix, yfix, and zfix, the coordinate offsets described in step d are added to all subsequent geometrically corrected images to ensure that the image matches the previous base map; f. Input the matched image and the previous image base map into the self-developed model to enhance the boundary features. The self-developed model supports two input methods: vector map and raster map of the previous image. If the previous image is a vector map, it is rasterized and then data augmented to enhance the boundary features. If the previous image is a raster map, it is directly used as the input map. The enhancement of boundary features involves extracting features from the two input images, fusing the features, and using an attention mechanism to help obtain the forest change result image. Self-developed model foundation: The framework of this self-developed forest condition change detection model uses the U-Net structure. It mainly utilizes an encoder-decoder structure, where the encoder is responsible for extracting features from the input image, and the decoder is responsible for mapping these features back to the segmentation result with the same resolution as the input image.
[0008] Image Input for Self-Developed Model: Typical neural networks only support bitmaps as input. To accommodate the need for vector graphics data in the initial stages, a feature allowing vector graphics as input was added. The self-developed model first identifies the input file. If it's a bitmap, it's directly used as input for feature extraction. If it's a vector graphic, it's rasterized according to the number of channels in the later image processing. To prevent feature loss during subsequent dimensionality enhancement, the rasterized image undergoes data augmentation, increasing the maximum pixel value to 255.
[0009] Self-developed model feature extraction: Since the input consists of two phases of images, a Siamese network is required for feature extraction. A Siamese network comprises two neural network branches with shared weights, meaning that both branches use the same parameters to extract features.
[0010] (2) The pattern comparison stage, that is, the comparison of the changes after the image has gone through the processing stage (1) above. The change results are obtained by multiplying the fused feature map with the spatial attention map. The values of the changed areas are larger, while the values of the unchanged areas are smaller. The values are compressed to between 0 and 1 by the sigmoid activation function. Values below the threshold of 0.5 are determined to be no change and assigned a value of 0, and values above the threshold of 0.5 are determined to be change and assigned a value of 1, thereby forming a change map and obtaining the result of the change of forest status in this area of the image.
[0011] Furthermore, the change recognition image geometric correction only uses a single image for geometric correction, without the need for stitching, thereby achieving a faster response speed.
[0012] Furthermore, the geometric correction addresses the two-phase image matching problem by adding coordinate offsets xfix, yfix, and zfix.
[0013] Furthermore, the method uses a self-developed model forest change detection network improved with the U-Net network, and uses a two-branch VGG-16 model as a replacement for feature extraction.
[0014] Furthermore, the self-developed forest land change detection network uses an attention mechanism module to perform feature fusion during upsampling and then extracts the range of forest land change.
[0015] Furthermore, the self-developed model forest change detection network supports both raster images and vector files as input formats at the input end of the previous images.
[0016] The beneficial effects of this invention are: 1. This method can perform real-time forest land image correction using geometric correction.
[0017] 2. The self-developed model of this method can perform comparative recognition of image and vector data.
[0018] 3. This method enables the full-process detection and extraction of forest land status changes, providing results on-site in real time and with high efficiency. Attached Figure Description
[0019] Figure 1 A diagram illustrating the process of recognition by a convolutional neural network.
[0020] Figure 2 This is a flowchart for detecting changes in forest land status.
[0021] Figure 3 This is the original image of the example.
[0022] Figure 4 The image is corrected as an example. Detailed Implementation
[0023] This invention discloses a rapid detection method for forest land status changes, which generally involves first detecting forest land status changes, then identifying forest land status changes, thereby obtaining the results of the status changes. For forest land status change detection, this method is based on a self-developed detection model. For previous imagery, it supports two input methods: vector maps and raster maps. If it is a vector map, it is rasterized and then data augmentation is performed to enhance boundary features; if it is a raster map, it is directly used as the input image. Then, feature extraction is performed on the two input images, and after feature fusion, an attention mechanism is used to help obtain the forest land status change result image. For forest land status change identification, this method identifies the extracted features based on the forest land status change result image obtained from the self-developed model, and then finally identifies the forest land status change situation, thus obtaining the forest land status change result.
[0024] I. Correction and identification of images refer to Figure 1 and 2 This embodiment uses five convolutional modules (Blocks 1-5) to obtain features in five dimensions. For example... Figure 1 As shown, the first layer is the input layer, where the input image is cropped to 512×512 pixels. Next are five 3×3 convolutional modules (Blocks 1-5), with Blocks 1-2 containing two convolutional layers and Blocks 3-5 containing three convolutional layers; each layer ends with a 2×2 max-pooling layer. The feature size is as follows: Block 1: 512×512×64; Block 2: 256×256×128; Block 3: 128×128×256; Block 4: 64×64×512; and Block 5: 32×32×512. Figure 2The self-developed model for forest change detection first superimposes the highest-dimensional features from two image phases, followed by three convolutions to obtain a fused feature map. Then, a spatial attention map is generated in the spatial attention module. In this map, each pixel is assigned a weight value, which is learnable within the network; pixels that have changed receive a greater weight than those that haven't. By multiplying the fused feature map with the spatial attention map, the changed regions in the fused feature map are extracted, forming a first-level change map CM1 with a size of 32×32. CM1 is then upsampled to the same size as Block 4 using deconvolution, and Block 4 features from both image phases are superimposed. Unlike Block 5, before convolution, a channel attention module assigns weights to each channel. After convolution, spatial attention is used to widen the gap between the changed forest regions and other regions, improving the quality of the change map. In this way, while restoring dimensionality through upsampling, features extracted from the self-developed model's image input are fused, ultimately identifying forest changes.
[0025] First, a forest image and its POS data (Table 1) are used for correction. The image plane coordinates of the four corners are calculated: top left (-w / 2*s, h / 2*s), top right (w / 2*s, h / 2*s), bottom right (w / 2*s, -h / 2*s), and bottom left (-w / 2*s, -h / 2*s). The projected coordinates (X, y, y) of the photocenter S are then calculated using latitude and longitude. S , Y S Then, using the above parameters, we construct a collinear equation between the plane coordinates (x, y) of the image point and the spatial coordinates (X, Y) of the corresponding ground point:
[0026] After introducing the offset, the equation is adjusted to:
[0027] Where a(1-3), b(1-3), and c(1-3) are the direction cosines composed of the three exterior azimuth angles, calculated as follows:
[0028] The projected coordinates of the four corners can be obtained through calculation. Then, the spatial resolution d=s*H / f in the source image can be calculated. Finally, the number of rows and columns in the corrected image can be calculated.
[0029]
[0030] Table 1 shows the various parameters of the image.
[0031] Secondly, resampling: Calculate the row and column number of each pixel in the correction frame corresponding to the original image using the transformation relationship obtained in the previous step, and select bilinear resampling.
[0032] Finally, the result is as follows Figure 3 The original image shown and as Figure 4 The corrected image shown.
[0033] II. Pattern Comparison The first step is to determine the type of the input image. If it is a raster image, proceed directly to the second step of cropping. If it is a vector image, rasterize it and enhance the rasterized pixels.
[0034] The second step is to crop the corrected image to a size of 512*512.
[0035] The third step involves the following neural network structure for forest change detection: The overall structure is a "W" shape formed by combining two "U" shapes. Features are acquired through continuous downsampling on the left and right sides, while the middle layer is upsampled and fused with the features from both sides to identify changes. The first layer on each side is the input layer, followed by five convolutional modules (Blocks 1-5). Blocks 1-2 contain two convolutional layers, and Blocks 3-5 contain three. Each layer ends with a 2×2 max-pooling layer. The feature size is as follows: Block 1: 512×512×64; Block 2: 256×256×128; Block 3: 128×128×256; Block 4: 64×64×512; and Block 5: 32×32×512. Through continuous downsampling on both sides, features of different dimensions from 512×512 to 32×32 are acquired.
[0036] At the lowest level, Block 5, the highest-dimensional features obtained from both sides are superimposed after convolution, and then three convolutions are performed to obtain a fused feature map. This fused feature map is then processed by a spatial attention module to extract the change regions, forming the first-level change map CM1, which is 32×32 in size. Deconvolution is then used to upsample CM1 to the same size as Block 4, and Block 4 features from both images are superimposed. Unlike Block 5, before the convolution operation, a channel attention module is used to assign weights to each channel. After the convolution operation, spatial attention is used to widen the gap between the forest's change regions and other regions, improving the quality of the change map. During the generation of each level of change map, corresponding ground-value images are used for loss calculation to accelerate network convergence.
[0037] The fourth step involves using the cropped 512*512 image as the input layer of the network for prediction, then reassembling the output forest land change results, and finally reassigning the coordinate system information of the original image to complete the forest land status change detection process.
Claims
1. A method for rapid detection of changes in forest land condition, characterized in that, The method involves acquiring high-precision images on-site and immediately comparing them with previous base maps to quickly obtain results on forest land change status in real time. This includes a high-precision image processing stage and a map comparison stage, specifically: (1) The high-precision processing stage of the on-site acquired images, that is, after the geometric correction of the change recognition images, the self-developed model is input to enhance the on-site acquired images. The self-developed model supports two input methods for the on-site images: vector images and raster images. If it is a vector image, it is rasterized and then data enhancement is carried out to enhance the boundary features. If it is a raster image, it is directly used as the input image. The process includes the following steps: a. Using the flight attitude, latitude, longitude, and altitude recorded in the POS system, the collected single images are used to establish collinearity equations, thus obtaining the coordinate transformation relationship between image points and corresponding ground objects: ； Where XS, YS, and ZS are the spatial coordinates of the projection center, a1-3, b1-3, and c1-3 are the cosines of each direction composed of three attitude angles, x0 and y0 are the principal point offset of the camera, f is the camera focal length, and ZA-ZS is the negative number of the flight altitude H. b. Use the standard transformation relationship obtained in step a to correct and resample the image; C. Add a coordinate header file to the image after resampling in step b, and assign coordinate system information; d. Compare the matching degree between the image with the coordinate system information and the previous image base map, and calculate the coordinate offsets xfix, yfix, and zfix of this flight path, where xfix is the offset of the projection center on the x-axis, yfix is the offset of the projection center on the y-axis, and zfix is the offset of the flight altitude of the projection center on the z-axis: ； Where XS, YS, and ZS are the spatial coordinates of the projection center, a1-3, b1-3, and c1-3 are the cosines of each direction composed of three attitude angles, x0 and y0 are the principal point offset of the camera, f is the camera focal length, and H is the flight altitude; e. After obtaining the coordinate offsets xfix, yfix, and zfix, the coordinate offsets described in step d are added to all subsequent geometrically corrected images to ensure that the image matches the previous base map; f. Input the matched image and the previous image base map into the self-developed model to enhance the boundary features. The self-developed model supports two input methods: vector map and raster map of the previous image. If the previous image is a vector map, it is rasterized and then data augmented to enhance the boundary features. If the previous image is a raster map, it is directly used as the input map. The enhancement of boundary features involves extracting features from the two input images, fusing the features, and using an attention mechanism to help obtain the forest change result image. (2) The pattern comparison stage, that is, the comparison of the changes after the image has gone through the processing stage (1) above. The change results are obtained by multiplying the fused feature map with the spatial attention map. The values of the changed areas are larger, while the values of the unchanged areas are smaller. The values are compressed to between 0 and 1 by the sigmoid activation function. Values below the threshold of 0.5 are determined to be no change and assigned a value of 0, and values above the threshold of 0.5 are determined to be change and assigned a value of 1, thereby forming a change map and obtaining the result of the change of forest status in this area of the image.
2. The rapid detection method for forest land status changes as described in claim 1, characterized in that, The change recognition image geometric correction only uses a single image for geometric correction, without the need for stitching, thus achieving a faster response speed.
3. The method for rapid detection of forest land status changes as described in claim 1, characterized in that, The problem of matching two images was solved by adding coordinate offsets xfix, yfix, and zfix.
4. The rapid detection method for forest land status changes as described in claim 1, characterized in that, The method uses a self-developed model forest change detection network improved from the U-Net network, and uses a two-branch VGG-16 model as a replacement for feature extraction.
5. The rapid detection method for forest land status changes as described in claim 4, characterized in that, The self-developed forest change detection network uses an attention mechanism module to perform feature fusion during upsampling and then extracts the range of forest change.
6. The rapid detection method for forest land status changes as described in claim 4, characterized in that, The self-developed forest change detection network supports both raster and vector image input formats at the input end of the previous imagery.