Terrain change monitoring system based on image analysis

Abstract

The invention relates to the technical field of terrain monitoring, and discloses a terrain change monitoring system based on image analysis, and the system is characterized in that an image obtaining module is used for obtaining multi-source image data, carrying out the fusion, and generating a fused image; the preprocessing module is used for preprocessing the fused image to obtain a preprocessed to-be-analyzed image; the analysis module is used for constructing a dual-channel deep convolutional network, analyzing the preprocessed to-be-analyzed image and detecting a topographic change area; the generation module is used for processing the detected terrain change area by adopting an incremental point cloud processing technology, and generating a three-dimensional terrain difference model and a settlement area volume change thermodynamic diagram in real time by combining a Poisson surface reconstruction algorithm; the early warning module is used for analyzing the generated three-dimensional terrain difference model and the volume change thermodynamic diagram, calculating parameters of terrain change and performing early warning processing; according to the method, the topographic change is accurately evaluated, and the accuracy and reliability of early warning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of terrain monitoring, and particularly relates to a terrain change monitoring system based on image analysis. Background Technique

[0002] Terrain change monitoring is of great significance in many fields such as geological disaster warning, urban planning, and ecological environment research. Traditional terrain change monitoring methods mainly include on-site measurement methods based on surveying instruments such as total stations and GPS, and monitoring methods based on remote sensing satellite data. Although the on-site measurement method has high measurement accuracy, it requires a large amount of manpower, material resources and time, and the measurement range is limited, making it difficult to achieve real-time dynamic monitoring of large-area terrain; although the monitoring method based on remote sensing satellite data can achieve large-area monitoring, due to factors such as the resolution and revisit period of satellite images, the monitoring effect of some minor terrain changes and local terrain changes is not ideal, and at the same time, the data processing process is complex and the timeliness is poor. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and design a terrain change monitoring system based on image analysis.

[0004] The present invention provides a terrain change monitoring system based on image analysis, and the system includes: An image acquisition module, configured to acquire multi-source image data of a target monitoring area, fuse the multi-source image data by using a dynamic adaptive registration algorithm, and generate a fused image; A preprocessing module, configured to preprocess the fused image, remove noise in the image, enhance the contrast of the denoised image by using an adaptive histogram equalization algorithm, highlight terrain features, and obtain a preprocessed image to be analyzed; An analysis module, configured to construct a two-channel deep convolutional network, analyze the preprocessed image to be analyzed, and detect terrain change areas; A generation module, configured to process the detected terrain change areas by using incremental point cloud processing technology, and generate a three-dimensional terrain difference model and a heat map of volume change in the settlement area in real time in combination with the Poisson surface reconstruction algorithm; An early warning module, configured to analyze the generated three-dimensional terrain difference model and the heat map of volume change, calculate terrain change parameters, and perform early warning processing.

[0005] Optionally, in the first implementation manner of the present invention, the image acquisition module includes: A feature point extraction sub-module, configured to collect multi-source image data of a target monitoring area and extract feature points by using the SIFT algorithm; A feature point matching sub-module, which is used to perform feature point matching based on the extracted sets of image feature points by using the method of Euclidean distance measurement; A region growing sub-module, which is used to take the obtained matching point pairs as seed points and perform region growing operations in the images, and adjust the images that have been preliminarily aligned by region growing by using the position information and attitude information of the image acquisition device to achieve pixel-level alignment; A fusion sub-module, which is used to fuse the aligned multi-source images by using a weighted average fusion method to generate a fused image.

[0006] Optionally, in the second implementation manner of the present invention, the region growing sub-module includes: A preliminary alignment unit, which is used to take the obtained matching point pairs as seed points, start from each seed point, check its adjacent pixel points, and if the features of the adjacent pixel points meet the region growing criterion, merge the pixel point into the current growing region and continuously expand it to the surrounding until there are no adjacent pixel points that meet the growing criterion, thus completing the preliminary alignment of the image; A establishment unit, which is used to establish the spatial relationship between different images by using the position information and attitude information of the image acquisition device to obtain the translation and rotation parameters between the images; A geometric transformation unit, which is used to perform geometric transformation on the images that have been preliminarily aligned by region growing according to the translation and rotation parameters between the images, and adjust the multi-source images in terms of spatial position and direction to achieve pixel-level alignment.

[0007] Optionally, in the third implementation manner of the present invention, the preprocessing module includes: A first traversal sub-module, which is used to set the noise type and threshold, traverse each pixel point of the fused image and make a judgment, and if the pixel point meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove the noise; A division sub-module, which is used to divide the denoised image into several sub-blocks of equal size and overlapping with each other, calculate the histogram of each sub-block respectively, and calculate the gray-level mapping relationship according to the histogram distribution; A first calculation sub-module, which is used to calculate the average gray mean value and average gray variance of all sub-blocks to obtain the global statistical information of the image. For the pixel points in the overlapping area of the sub-blocks, use a weighted average method to determine the weight according to the distance from the pixel point to the center of each sub-block, and calculate the final gray value of the pixel points in the overlapping area; A merging sub-module, which is used to merge all the sub-blocks that have been processed by adaptive histogram equalization in the original image arrangement order to obtain the preprocessed image and highlight the terrain features.

[0008] Optionally, in the fourth implementation manner of the present invention, the analysis module includes: A marking sub-module, used to construct a dual-channel deep convolutional network architecture, collect multiple groups of image data of the target monitoring area at different times, mark the real change areas, and divide the multiple groups of image data into a training set, a validation set, and a test set; An input sub-module, used to input the image data in the training set into the dual-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, and channel two analyzes the temporal change trend of the image. After passing through the fusion layer and the output layer, a predicted pixel-level change probability map is obtained; A first comparison sub-module, used to compare the predicted pixel-level change probability map with the marked real change areas, and calculate the error between the predicted result and the real result using the cross-entropy loss function; An update sub-module, used to propagate the error of the loss function from the output layer to each layer of the network through the backpropagation algorithm, calculate the gradients of the parameters of each layer according to the error, and update the network parameters using stochastic gradient descent; An evaluation sub-module, used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network; A second traversal sub-module, used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output a pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change area.

[0009] Optionally, in the fifth implementation manner of the present invention, the dual-channel deep convolutional network architecture includes: Channel one architecture: A network structure including multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers use convolutional kernels of different sizes to extract the texture and elevation features of the image. The pooling layers use max pooling to reduce the data dimension to obtain key features. The fully connected layers integrate the extracted features and output feature vectors; Channel two architecture: Composed of a convolutional layer, an LSTM layer, and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal change trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector; Fusion layer: A fusion layer is set after the two channels to splice the feature vectors output by channel one and channel two to form a comprehensive feature vector; Output layer: A fully connected layer using the Softmax activation function is used as the output layer to output a pixel-level change probability map.

[0010] Optionally, in the sixth implementation manner of the present invention, the generation module includes: A determination sub-module, used to obtain initial point cloud data for the detected terrain change area through stereo matching and lidar data analysis and processing, and determine the boundary of the change area; The removal sub-module is used to analyze the distance distribution between each point and its neighboring points in the point cloud data by statistical filtering, remove outliers, and reduce the number of points in the dense point cloud by grid sampling; The transformation sub-module is used to construct an octree data structure based on the point cloud data, calculate the implicit function value corresponding to each point according to the position and normal vector information of the point cloud, sample the implicit function using the marching cubes algorithm, extract the zero isosurface, and transform it into a triangular mesh form to obtain the reconstructed terrain surface model; The second comparison sub-module is used to compare the currently reconstructed terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the magnitude and direction of the displacement vector; The second calculation sub-module is used for the settlement area. According to the displacement vector in the three-dimensional terrain difference model, calculate the volume change of each triangular mesh unit, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change in the settlement area.

[0011] Optionally, in the sixth implementation manner of the present invention, a method for implementing a terrain change monitoring system based on image analysis is provided. The method includes the following steps: Obtain multi-source image data of the target monitoring area, and fuse the multi-source image data using a dynamic adaptive registration algorithm to generate a fused image; Preprocess the fused image to remove noise in the image, and enhance the contrast of the denoised image using an adaptive histogram equalization algorithm to highlight terrain features, obtaining a preprocessed image to be analyzed; Construct a two-channel deep convolutional network, analyze the preprocessed image to be analyzed, and detect terrain change areas; Process the detected terrain change areas using incremental point cloud processing technology, and combine the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change in the settlement area in real time; Analyze the generated three-dimensional terrain difference model and heat map of the volume change, calculate the parameters of the terrain change, and perform early warning processing.

[0012] Optionally, in the sixth implementation manner of the present invention, a method for implementing a terrain change monitoring system based on image analysis is provided. The method includes the following steps: Collect multi-source image data of the target monitoring area, and extract feature points using the SIFT algorithm; Based on the extracted image feature point sets, perform feature point matching using the Euclidean distance metric; Using the obtained matching point pairs as seed points, perform region growing operations in the image, and use the position information and attitude information of the image acquisition device to adjust the images that have been preliminarily aligned by region growing to achieve pixel-level alignment; For the aligned multi-source images, use the weighted average fusion method for fusion to generate the fused image.

[0013] Optionally, in the sixth implementation manner of the present invention, a method for implementing a terrain change monitoring system based on image analysis is provided, and the method includes the following steps: Set the noise type and threshold, traverse each pixel point of the fused image and make a judgment. If the pixel point meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove the noise; Divide the denoised image into several sub-blocks of equal size and overlapping each other, calculate the histogram of each sub-block respectively, and calculate the gray-level mapping relationship according to the histogram distribution; Calculate the average gray mean and average gray variance of all sub-blocks to obtain the global statistical information of the image. For the pixel points in the overlapping area of the sub-blocks, use the weighted average method, determine the weight according to the distance from the pixel point to the center of each sub-block, and calculate the final gray value of the pixel points in the overlapping area; Merge all the sub-blocks after adaptive histogram equalization processing in the arrangement order of the original image to obtain the preprocessed image, highlighting the terrain features.

[0014] In the technical solution provided by the present invention, the terrain change monitoring system based on image analysis includes an image acquisition module, a preprocessing module, an analysis module, a generation module, and a warning module; the present invention greatly improves the accuracy of terrain change monitoring, can timely detect small terrain changes, realizes real-time monitoring of terrain changes, and can output a warning report on the same day, timely master the terrain change situation, effectively eliminates interference such as seasonal illumination differences through a dynamic adaptive registration algorithm, can more comprehensively and accurately evaluate the impacts and risks that terrain changes may bring, improves the accuracy and reliability of warnings, reduces a large amount of human and material resources investment; at the same time, through optimization algorithms such as incremental point cloud processing technology, it improves the data processing efficiency, reduces the calculation cost, has a lower monitoring cost in local areas, and can achieve higher-frequency monitoring, with high cost-effectiveness. Description of the Drawings

[0015] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0016] Figure 1Schematic diagram of the first embodiment of the terrain change monitoring system based on image analysis provided by the embodiments of the present invention; Figure 2 Schematic diagram of the second embodiment of the terrain change monitoring system based on image analysis provided by the embodiments of the present invention; Figure 3 Schematic diagram of the third embodiment of the terrain change monitoring system based on image analysis provided by the embodiments of the present invention. Detailed implementation manners

[0017] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that shown or described herein. In addition, the terms "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or device comprising a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific processes of the embodiments of the present invention are described below. Please refer to Figure 1 Schematic diagram of the first embodiment of the terrain change monitoring system based on image analysis provided by the embodiments of the present invention. The system includes: An image acquisition module for acquiring multi-source image data of a target monitoring area, fusing the multi-source image data using a dynamic adaptive registration algorithm, and generating a fused image; A preprocessing module for preprocessing the fused image, removing noise in the image, enhancing the contrast of the denoised image using an adaptive histogram equalization algorithm to highlight terrain features, and obtaining a preprocessed image to be analyzed; An analysis module for constructing a dual-channel deep convolutional network, analyzing the preprocessed image to be analyzed, and detecting terrain change areas; A generation module for processing the detected terrain change areas using incremental point cloud processing technology, and generating a 3D terrain difference model and a heat map of the volume change of the settlement area in real time in combination with the Poisson surface reconstruction algorithm; An early warning module for analyzing the generated 3D terrain difference model and heat map of volume change, calculating terrain change parameters, and performing early warning processing.

[0019] In this embodiment, the preprocessing module includes: The first traversal sub-module is used to set the noise type and threshold, traverse each pixel point of the fused image and make a judgment. If the pixel point meets the noise type and threshold, median filtering and bilateral filtering are combined to remove the noise; The division sub-module is used to divide the denoised image into several sub-blocks of equal size and overlapping each other, calculate the histogram of each sub-block respectively, and calculate the gray-level mapping relationship according to the histogram distribution; The first calculation sub-module is used to calculate the average gray mean and average gray variance of all sub-blocks to obtain the global statistical information of the image. For the pixel points in the overlapping area of the sub-blocks, a weighted average method is adopted to determine the weight according to the distance from the pixel point to the center of each sub-block, and the final gray value of the pixel points in the overlapping area is calculated; The merging sub-module is used to merge all the sub-blocks after adaptive histogram equalization processing in the original image arrangement order to obtain the preprocessed image and highlight the terrain features.

[0020] In this embodiment, the generation module includes: The determination sub-module is used to obtain the initial point cloud data through stereo matching and lidar data analysis and processing for the detected terrain change area, and determine the boundary of the change area; The removal sub-module is used to analyze the distance distribution between each point and its neighborhood points in the point cloud data by statistical filtering to remove outliers, and use grid sampling to reduce the number of points in the dense point cloud; The conversion sub-module is used to construct an octree data structure based on the point cloud data, calculate the implicit function value corresponding to each point according to the position and normal vector information of the point cloud, sample the implicit function by using the marching cubes algorithm, extract the zero isosurface, and convert it into a triangular mesh form to obtain the reconstructed terrain surface model; The second comparison sub-module is used to compare the currently reconstructed terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the magnitude and direction of the displacement vector; The second calculation sub-module is used to calculate the volume change amount of each triangular mesh unit in the settlement area according to the displacement vector in the three-dimensional terrain difference model for the settlement area, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change in the settlement area.

[0021] In this embodiment, the possible types of noise existing in the image are determined, such as Gaussian noise, salt-and-pepper noise, etc. For different types of noise, corresponding noise recognition thresholds are set respectively. For Gaussian noise, the fluctuation ranges of its mean value and variance are set as the threshold. For salt-and-pepper noise, the proportion threshold of the pixel value deviating from the normal range (such as approaching 0 or 255) is set. Each pixel point of the fused image is traversed and judged according to the noise type and threshold set in the first step. If the pixel point conforms to the noise characteristics, a processing method combining median filtering and bilateral filtering is adopted. Taking the current noisy pixel point as the center, a filtering window of a suitable size is selected, the median of the pixel points in the window is calculated, and the noisy pixel point is replaced with the median. Then, using bilateral filtering, according to the spatial distance and pixel value difference of the pixel points, the pixel points are further smoothed to remove the noise while trying to retain the edge information of the image. The denoised image is divided into several sub-blocks of equal size and overlapping. The size of the sub-blocks is determined according to the image resolution and the complexity of the terrain features. For each sub-block, its histogram is calculated respectively. The frequency of each gray level appearing in the sub-block is counted to obtain the gray histogram distribution of the sub-block. At the same time, the mean value and variance of the gray values in the sub-block are calculated as the basis for adjusting the histogram equalization parameters later. Local contrast enhancement: For each sub-block, according to its gray histogram distribution, the gray level mapping relationship is calculated. When calculating the gray mapping according to the traditional histogram equalization method, the mapped gray value is multiplied by an adjustment factor, so that the sub-block with a larger variance (more noise) has a relatively smaller contrast enhancement amplitude, and the sub-block with a smaller variance (less noise) has a relatively larger contrast enhancement amplitude. The average gray mean value and average gray variance of all sub-blocks are calculated to obtain the global statistical information of the image. For each pixel point in the sub-block, based on the gray value after local contrast enhancement, a secondary adjustment is made according to the difference between the sub-block where the pixel point is located and the global statistical information. If the gray mean value of the sub-block is higher than the global average gray mean value, the gray value of the pixels in the sub-block is appropriately reduced; otherwise, it is appropriately increased. At the same time, according to the difference between the sub-block variance and the global average variance, the adjustment factor is finely tuned to further balance the local and global contrast enhancement effects. For the pixel points in the overlapping area of the sub-blocks, the processing results in multiple sub-blocks are comprehensively considered. By using the weighted average method, the weight is determined according to the distance from the pixel point to the centers of each sub-block, and the closer the distance, the greater the weight. The final gray value of the pixel points in the overlapping area is calculated to ensure the smooth transition of the image at the sub-block splicing. All sub-blocks after the improved adaptive histogram equalization processing are merged in the arrangement order of the original image to obtain the preprocessed image, highlighting the terrain features.

[0022] In this embodiment, for the detected terrain change area, initial point cloud data of this area is obtained from the original image data through methods such as stereo matching and lidar data parsing. The point cloud data contains the three-dimensional coordinate (X, Y, Z) information of each point, and some may also carry attribute information such as color and reflection intensity; According to the results of the previously detected terrain change area, the boundary range of the change area is determined. The contour information obtained through image segmentation can be transformed into a boundary polygon in three-dimensional space to define the area range that needs to be incrementally processed; New data acquisition: When new image data is collected, for the area within the boundary of the change area, new point cloud data is obtained again through methods such as stereo matching or lidar data parsing; Data fusion: The newly obtained point cloud data is fused with the initial point cloud data. For the points in the overlapping area, screening and merging are performed according to the attribute information of the points (such as timestamp, accuracy, etc.), and more accurate point information is retained to complete the incremental update of the point cloud data; Outlier removal: Methods such as statistical filtering or radius filtering are used to analyze the distance distribution of each point in the point cloud data from its neighboring points. A distance threshold is set, and the points whose distances exceed the threshold range are determined as outliers and removed to improve the quality of the point cloud data; Point cloud simplification: Methods such as voxel filtering or grid sampling are used to simplify the dense point cloud while maintaining the shape characteristics of the point cloud, reducing the number of points in the point cloud and reducing the subsequent computational workload; Octree structure construction: Based on the point cloud data, an octree data structure is constructed. The three-dimensional space containing the point cloud is divided into eight subspaces, and continuous recursive division is performed until the number of points in each subspace meets the set threshold or reaches the maximum division level, which is used to accelerate the subsequent surface reconstruction calculation; Implicit function calculation: According to the position and normal vector information of the point cloud, the implicit function value corresponding to each point is calculated. By solving the Poisson equation, a continuous implicit function in the entire three-dimensional space is constructed, and the zero isosurface of this function is the reconstructed surface; Surface mesh extraction: Methods such as the Marching Cubes algorithm are used to sample the implicit function, extract the zero isosurface, and transform it into a triangular mesh form to obtain the reconstructed terrain surface model; The currently reconstructed terrain surface model is compared with the historical terrain surface model. By calculating the three-dimensional coordinate differences between the corresponding points of the two models, the displacement vector of each point is obtained; According to the magnitude and direction of the displacement vector, the difference results are visually processed. A color-coding method can be used, where different colors represent different displacement magnitudes, to generate a three-dimensional terrain difference model to visually display the terrain changes; For the settlement area, according to the displacement vectors in the three-dimensional terrain difference model, the volume change of each small area (such as a triangular mesh unit) is calculated. Through integration or accumulation, the total volume change of the entire settlement area is obtained; The settlement area is divided into multiple sub-areas according to certain rules, and the average volume change rate of each sub-area is calculated.Draw a heat map using different colors and transparencies according to the magnitude of the volume change rate. The darker the color and the lower the transparency, the greater the volume change rate, thereby generating a heat map of the volume change in the settlement area.

[0023] In this embodiment, determine the terrain change parameters to be calculated, including but not limited to the area, volume, slope change amount, elevation change amount, etc. of the terrain change; for each terrain change parameter, set corresponding risk thresholds according to the geological conditions of the monitoring area, historical terrain change data, and actual application requirements. For example, according to the historical settlement data of the mining area, set the volume change amount exceeding 100 cubic meters as the high-risk threshold, and 50 - 100 cubic meters as the medium-risk threshold; the slope change amount exceeding 15° is high risk, and 8° - 15° is medium risk, etc.; extract the three-dimensional coordinate data of the terrain change area from the three-dimensional terrain difference model, including the X, Y, and Z coordinate information of each change point; obtain the volume change rate data and the corresponding area range information of each sub-region from the heat map of the volume change in the settlement area; project the extracted three-dimensional coordinate data of the terrain change area onto a two-dimensional plane (such as the XY plane), and use polygon area calculation methods (such as the vector cross product method, segmentation summation method) to calculate the projected area of the terrain change area on the two-dimensional plane; combine the Z coordinate change amount of the change points in the three-dimensional terrain difference model and the corresponding area data, and calculate the total volume of the terrain change by integration or accumulation. For the sub-regions in the heat map, calculate the volume change amount of each sub-region according to the volume change rate and the initial volume of the sub-region; calculate the slope values of each region before and after the terrain change respectively. The slope value can be obtained by calculating the tangent value of the angle between the normal vector of the terrain surface at a certain point and the vertical direction vector. Compare the slope values before and after the change to obtain the slope change amount of each region; extract the elevation data (Z coordinates) of each point before and after the terrain change in the terrain change area, calculate the elevation difference of each point, and statistically obtain parameters such as the average elevation change amount and the maximum elevation change amount of the entire region; compare the calculated terrain change parameters with the risk thresholds set in the first step; adopt a hierarchical evaluation mechanism. If a certain parameter exceeds the high-risk threshold, it is determined that the region is in a high-risk state; if it is within the medium-risk threshold range, it is determined to be in a medium-risk state; if it is lower than the medium-risk threshold, it is determined to be in a low-risk state. For the case of comprehensive evaluation of multiple parameters, weights can be set according to the importance of each parameter, and the comprehensive risk level can be obtained through weighted calculation; when the evaluation result is in a high-risk or medium-risk state, trigger the early warning mechanism; according to the preset early warning method, send early warning information to relevant personnel (such as geological disaster monitoring departments, regional management personnel, etc.) through channels such as text messages, emails, and audible and visual alarms. The early warning information includes the name of the monitoring area, risk level, main terrain change parameters and change situations, etc., so that relevant personnel can take corresponding measures in a timely manner.

[0024] Please refer to Figure 2, the second embodiment schematic diagram of the terrain change monitoring system based on image analysis provided by the embodiments of the present invention. The image acquisition module includes: A feature point extraction sub-module, configured to collect multi-source image data of a target monitoring area and perform feature point extraction using the SIFT algorithm; A feature point matching sub-module, configured to perform feature point matching based on each set of extracted image feature points by using the Euclidean distance metric; A region growing sub-module, configured to use the obtained matching point pairs as seed points to perform region growing operations in the image, and adjust the images that have been initially aligned by region growing by using the position information and attitude information of the image acquisition device to achieve pixel-level alignment; A fusion sub-module, configured to fuse the aligned multi-source images by using a weighted average fusion method to generate a fused image.

[0025] In this embodiment, the region growing sub-module includes: A preliminary alignment unit, configured to use the obtained matching point pairs as seed points, start from each seed point, check its adjacent pixel points, if the features of the adjacent pixel points meet the region growing criterion, then merge the pixel point into the current growing region, and continuously expand around until there are no adjacent pixel points that meet the growing criterion, thus completing the preliminary alignment of the image; A establishment unit, configured to establish the spatial relationship between different images by using the position information and attitude information of the image acquisition device to obtain the translation and rotation parameters between the images; A geometric transformation unit, configured to perform geometric transformation on the images that have been initially aligned by region growing according to the translation and rotation parameters between the images, and adjust the multi-source images in terms of spatial position and direction to achieve pixel-level alignment.

[0026] In this embodiment, multi-source image data of the target monitoring area is collected, including satellite multispectral images, UAV oblique photography images, ground camera video streams, etc. At the same time, metadata such as the corresponding GPS position information, IMU attitude information (including pitch angle, yaw angle, roll angle), and acquisition time of each image acquisition device during image acquisition is obtained, providing basic reference information for subsequent image registration; for the collected multi-source image data, the SIFT (Scale-Invariant Feature Transform) algorithm is respectively used to extract feature points. Extreme points at different scales are detected in the images, stable key points are found through the Gaussian difference pyramid, and the directions and descriptors of the key points are calculated, thereby obtaining the feature point sets of each image. Each feature point contains information such as position, scale, direction, and descriptor; based on the extracted feature point sets of each image, the Euclidean distance metric is used for feature point matching. The Euclidean distance between the feature point descriptors in different images is calculated, and a distance threshold is set. The feature point pairs with a distance less than the threshold are regarded as matching point pairs. In this way, a preliminary correspondence relationship between multi-source images is established, and the obtained matching point pairs are used as seed points for region growing operations in the images. Starting from each seed point, according to a certain similarity criterion (such as similarity in features such as color, texture, grayscale, etc.), adjacent pixel points are gradually merged into the same region until the stop condition is met (for example, the region growing reaches a certain area, the difference between adjacent pixel points exceeds the set threshold, etc.). Through region growing, the corresponding matching regions in the multi-source images are determined, further refining the correspondence relationship between the images; the obtained GPS position information and IMU attitude information are introduced to precisely adjust the images that have been preliminarily aligned through region growing. According to the positions and attitudes of the image acquisition devices, the spatial transformation relationship between the images (including transformation parameters such as translation, rotation, scaling, etc.) is calculated, and these transformation parameters are used to perform geometric transformation on the images, so that the multi-source images are more precisely aligned in spatial position and direction, eliminating the interference caused by shooting angle, position differences, and seasonal lighting differences, and achieving pixel-level alignment; for the multi-source images that have been precisely aligned, a weighted average fusion method is used for fusion. According to the characteristics and importance of different images, different weights are assigned to each image. For example, high-resolution UAV images have an advantage in detail expression and can be given a higher weight; satellite images have an advantage in large-area coverage and spectral information, and corresponding weights are also assigned according to the actual situation. Then, the pixel values of the corresponding pixel points of each image are weighted and summed to generate a fused image, integrating the advantageous information of the multi-source images.

[0027] In this embodiment, the criteria for determining whether pixels belong to the same region include, but are not limited to, color difference threshold, texture similarity threshold, and grayscale difference threshold. For example, the color difference threshold is set such that in the RGB color space, the difference in each channel does not exceed 20; the texture similarity is calculated by computing the local binary pattern (LBP) features of the pixel neighborhood, and the feature difference is set not to exceed a certain value; the grayscale difference threshold is set to not exceed 15. Using the obtained matching point pairs as seed points, starting from each seed point, its adjacent pixel points are examined. For each adjacent pixel point, its features such as color, texture, and grayscale are compared with the corresponding features of the seed point and the pixels within its growing region. If the features of the adjacent pixel point meet the region growing criteria set in the first step, then the pixel point is merged into the current growing region. This process is repeated, continuously expanding outwards until there are no adjacent pixel points that meet the growing criteria, or the growing region reaches pre-set stopping conditions such as the maximum area or maximum number of growing steps, thus completing the preliminary alignment of the image. Using the position information (GPS coordinates) and attitude information (pitch angle, yaw angle, roll angle) of the image acquisition device, the spatial relationship between different images in the world coordinate system is established. Through coordinate transformation and geometric calculation, the transformation parameters such as the translation amount (displacement in the X, Y, and Z axis directions), rotation angle (rotation around the X, Y, and Z axes), and scaling ratio between the images are determined. For example, the relative position between two devices is calculated based on their GPS coordinates, and the difference in the shooting directions of the devices is calculated in combination with the attitude information, thereby obtaining the translation and rotation parameters between the images. According to the spatial transformation parameters calculated in the third step, geometric transformation is performed on the images that have been preliminarily aligned by region growing. For translation, the image is directly moved by the corresponding displacement amount in the X, Y, and Z axis directions; for rotation, the rotation matrix is used to transform the pixel coordinates of the image; for scaling, the image is enlarged or reduced according to the calculated scaling ratio. Through these geometric transformations, the multi-source images are precisely adjusted in terms of spatial position and orientation to achieve pixel-level alignment.

[0028] Please refer to Figure 3 , the schematic diagram of the third embodiment of the terrain change monitoring system based on image analysis provided by the embodiment of the present invention. The analysis module includes: A marking sub-module, used to construct a dual-channel deep convolutional network architecture, collect multiple groups of image data of the target monitoring area at different times, mark the real change areas, and divide the multiple groups of image data into a training set, a validation set, and a test set; An input sub-module, used to input the image data in the training set into the dual-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, and channel two analyzes the temporal change trend of the image. After passing through the fusion layer and the output layer, a predicted pixel-level change probability map is obtained; The first comparison sub-module is used to compare the predicted pixel-level change probability map with the labeled true change region, and calculates the error between the predicted result and the true result using the cross-entropy loss function; The update sub-module is used to propagate the error of the loss function from the output layer to each layer of the network through the backpropagation algorithm, calculate the gradients of the parameters of each layer according to the error, and update the network parameters using stochastic gradient descent; The evaluation sub-module is used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network; The second traversal sub-module is used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output the pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change region.

[0029] In this embodiment, the architecture of the dual-channel deep convolutional network includes: The architecture of Channel 1: A network structure including multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers use convolutional kernels of different sizes to extract the texture and elevation features of the image. The pooling layers use max pooling to reduce the data dimension to obtain key features. The fully connected layers integrate the extracted features and output feature vectors; The architecture of Channel 2: It consists of a convolutional layer, an LSTM layer, and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal change trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector; The fusion layer: A fusion layer is set after the two channels to splice the feature vectors output by Channel 1 and Channel 2 to form a comprehensive feature vector; The output layer: A fully connected layer using the Softmax activation function is used as the output layer to output the pixel-level change probability map. The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A terrain change monitoring system based on image analysis, characterized in that, The system includes: An image acquisition module, which is used to acquire multi-source image data of the target monitoring area, fuse the multi-source image data by using a dynamic adaptive registration algorithm, and generate a fused image; A preprocessing module, which is used to preprocess the fused image, remove the noise in the image, enhance the contrast of the denoised image by using an adaptive histogram equalization algorithm, highlight the terrain features, and obtain the preprocessed image to be analyzed; An analysis module, which is used to construct a two-channel deep convolutional network, analyze the preprocessed image to be analyzed, and detect the terrain change area; A generation module, which is used to process the detected terrain change area by using incremental point cloud processing technology, and combine the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time; An early warning module, which is used to analyze the generated three-dimensional terrain difference model and the heat map of volume change, calculate the parameters of terrain change, and perform early warning processing.

2. The terrain change monitoring system based on image analysis according to claim 1, wherein The image acquisition module includes: A feature point extraction sub-module, which is used to collect multi-source image data of the target monitoring area and extract feature points by using the SIFT algorithm; A feature point matching sub-module, which is used to perform feature point matching based on the extracted image feature point sets by using the Euclidean distance metric; A region growing sub-module, which is used to perform region growing operations in the image with the obtained matching point pairs as seed points, and adjust the images preliminarily aligned by region growing by using the position information and attitude information of the image acquisition device to achieve pixel-level alignment; A fusion sub-module, which is used to fuse the aligned multi-source images by using a weighted average fusion method to generate a fused image.

3. A terrain change monitoring system based on image analysis according to claim 2, characterized in that, The region growing sub-module includes: A preliminary alignment unit, which is used to use the obtained matching point pairs as seed points, start from each seed point, check its adjacent pixel points, if the features of the adjacent pixel points meet the region growing criterion, merge the pixel point into the current growing region, and continuously expand to the surrounding until there are no adjacent pixel points that meet the growing criterion, and complete the preliminary alignment of the image; A establishing unit, which is used to establish the spatial relationship between different images by using the position information and attitude information of the image acquisition device to obtain the translation and rotation parameters between the images; A geometric transformation unit, which is used to perform geometric transformation on the images preliminarily aligned by region growing according to the translation and rotation parameters between the images, and adjust the multi-source images in terms of spatial position and direction to achieve pixel-level alignment.

4. A terrain change monitoring system based on image analysis according to claim 1, characterized in that, The preprocessing module includes: A first traversal sub-module, which is used to set the noise type and threshold, traverse each pixel point of the fused image and make a judgment, if the pixel point meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove the noise; A dividing sub-module, which is used to divide the denoised image into several sub-blocks of equal size and overlapping with each other, calculate the histogram of each sub-block respectively, and calculate the gray-level mapping relationship according to the histogram distribution; The first calculation sub-module is used to calculate the average gray mean and average gray variance of all sub-blocks to obtain the global statistical information of the image. For the pixel points in the overlapping area of the sub-blocks, a weighted average method is adopted, and the weight is determined according to the distance from the pixel point to the center of each sub-block, and the final gray value of the pixel point in the overlapping area is calculated; The merging sub-module is used to merge all the sub-blocks after the adaptive histogram equalization process in the arrangement order of the original image to obtain the preprocessed image and highlight the terrain features.

5. The terrain change monitoring system based on image analysis according to claim 1, characterized in that, The analysis module includes: The marking sub-module is used to construct a dual-channel deep convolutional network architecture, collect multiple groups of image data in different periods of the target monitoring area, mark the real change areas, and divide the multiple groups of image data into a training set, a validation set and a test set; The input sub-module is used to input the image data in the training set into the dual-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, and channel two analyzes the temporal change trend of the image. After passing through the fusion layer and the output layer, a predicted pixel-level change probability map is obtained; The first comparison sub-module is used to compare the predicted pixel-level change probability map with the marked real change area, and use the cross-entropy loss function to calculate the error between the predicted result and the real result; The update sub-module is used to propagate the error of the loss function from the output layer to each layer of the network through the backpropagation algorithm, calculate the gradient of the parameters of each layer according to the error, and update the network parameters using stochastic gradient descent; The evaluation sub-module is used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network; The second traversal sub-module is used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output a pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change area.

6. The terrain change monitoring system based on image analysis according to claim 5, characterized in that, The dual-channel deep convolutional network architecture includes: Channel one architecture: a network structure containing multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer uses convolutional kernels of different sizes to extract the texture and elevation features of the image. The pooling layer uses max pooling to reduce the data dimension to obtain key features. The fully connected layer integrates the extracted features and outputs a feature vector; Channel two architecture: composed of a convolutional layer, an LSTM layer and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal change trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector; Fusion layer: A fusion layer is set after the two channels to splice the feature vectors output by channel one and channel two to form a comprehensive feature vector; Output layer: A fully connected layer using the Softmax activation function is used as the output layer to output a pixel-level change probability map.

7. The terrain change monitoring system based on image analysis according to claim 1, characterized in that The generation module includes: The determination sub-module is used to obtain the initial point cloud data for the detected terrain change area through stereo matching and lidar data analysis and processing, and determine the boundary of the change area; The removal sub-module is used to analyze the distance distribution between each point and its neighboring points in the point cloud data by statistical filtering, remove outliers, and reduce the number of points in the dense point cloud by grid sampling; The transformation sub-module is used to construct an octree data structure based on the point cloud data, calculate the implicit function value corresponding to each point according to the position and normal vector information of the point cloud, sample the implicit function using the marching cubes algorithm, extract the zero isosurface, and transform it into a triangular mesh form to obtain the reconstructed terrain surface model; The second comparison sub-module is used to compare the currently reconstructed terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the magnitude and direction of the displacement vector; The second calculation sub-module is used for the settlement area. According to the displacement vector in the three-dimensional terrain difference model, calculate the volume change of each triangular mesh unit, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change in the settlement area.

8. A method for implementing a terrain change monitoring system based on image analysis as described in claim 1, characterized in that, The method includes the following steps: Obtain multi-source image data of the target monitoring area, and fuse the multi-source image data using a dynamic adaptive registration algorithm to generate a fused image; Preprocess the fused image to remove noise in the image, and use an adaptive histogram equalization algorithm to enhance the contrast of the denoised image to highlight terrain features, obtaining a preprocessed image to be analyzed; Construct a two-channel deep convolutional network to analyze the preprocessed image to be analyzed and detect terrain change areas; Use an incremental point cloud processing technology to process the detected terrain change areas, and combine with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change in the settlement area in real time; Analyze the generated three-dimensional terrain difference model and heat map of volume change, calculate the parameters of terrain change, and perform early warning processing.

9. A method for implementing a terrain change monitoring system based on image analysis as described in claim 1, characterized in that, The method includes the following steps: Collect multi-source image data of the target monitoring area and use the SIFT algorithm to extract feature points; Based on the extracted image feature point sets, use the Euclidean distance metric for feature point matching; Use the obtained matching point pairs as seed points to perform region growing operations in the image, and use the position information and attitude information of the image acquisition device to adjust the images that have been initially aligned by region growing to achieve pixel-level alignment; For the aligned multi-source images, use a weighted average fusion method for fusion to generate a fused image.

10. A method for implementing a terrain change monitoring system based on image analysis as described in claim 1, characterized in that, The method includes the following steps: Set the noise type and threshold, traverse each pixel point of the fused image and make a judgment. If the pixel point meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove noise; Divide the denoised image into several sub-blocks of equal size and overlapping each other, calculate the histogram of each sub-block respectively, and calculate the gray-level mapping relationship according to the histogram distribution; Calculate the average gray mean and average gray variance of all sub-blocks to obtain the global statistical information of the image. For the pixel points in the overlapping area of the sub-blocks, use a weighted average method to determine the weight according to the distance from the pixel point to the center of each sub-block, and calculate the final gray value of the pixel points in the overlapping area; Merge all the sub-blocks after adaptive histogram equalization processing in the arrangement order of the original image to obtain the pre-processed image, highlighting the terrain features.

Images