A terrain change monitoring system based on image analysis

Abstract

The present invention relates to the technical field of terrain monitoring and discloses a terrain change monitoring system based on image analysis. The system comprises an image acquisition module for acquiring multi-source image data, fusing them, and generating a fused image. The system comprises a preprocessing module for preprocessing the fused image to obtain a preprocessed image to be analyzed. The system comprises an analysis module for constructing a dual-channel deep convolutional network to analyze the preprocessed image to be analyzed and detect terrain change areas. The system comprises a generation module for processing the detected terrain change areas using incremental point cloud processing technology and generating a three-dimensional terrain difference model and a volume change heat map of the subsidence area in real time in combination with a Poisson surface reconstruction algorithm. The system comprises an early warning module for analyzing the generated three-dimensional terrain difference model and volume change heat map, calculating parameters of terrain change, and performing early warning processing. The system accurately assesses terrain changes and improves the accuracy and reliability of early warnings.

Description

Technical Field

[0001] The present invention relates to the technical field of terrain monitoring, and in particular to a terrain change monitoring system based on image analysis. Background Art

[0002] Monitoring terrain change is crucial in numerous fields, including geological disaster early warning, urban planning, and ecological and environmental research. Traditional methods for monitoring terrain change primarily include field measurements using instruments such as total stations and GPS, and monitoring methods based on remote sensing satellite data. While field measurements offer high accuracy, they require significant manpower, material resources, and time, and their limited measurement range makes it difficult to achieve real-time dynamic monitoring of large terrain areas. Remote sensing satellite data-based monitoring methods, while capable of monitoring large areas, are limited by factors such as satellite imagery resolution and revisit cycles. This makes monitoring of minor terrain changes and localized changes less effective. Furthermore, data processing is complex and time-sensitive. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and to design a terrain change monitoring system based on image analysis.

[0004] The present invention provides a terrain change monitoring system based on image analysis, the system comprising:

[0005] The image acquisition module is used to acquire multi-source image data of the target monitoring area, fuse the multi-source image data using a dynamic adaptive registration algorithm, and generate a fused image;

[0006] The preprocessing module is used to preprocess the fused image, remove the noise in the image, and use the adaptive histogram equalization algorithm to enhance the contrast of the denoised image, highlight the terrain features, and obtain the preprocessed image to be analyzed;

[0007] The analysis module is used to build a dual-channel deep convolutional network to analyze the pre-processed image to be analyzed and detect the area of ​​terrain change;

[0008] The generation module is used to process the detected terrain change areas using incremental point cloud processing technology and combine it with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time;

[0009] The early warning module is used to analyze the generated three-dimensional terrain difference model and volume change heat map, calculate the parameters of terrain change, and perform early warning processing.

[0010] Optionally, in a first implementation of the present invention, the image acquisition module includes:

[0011] The feature point extraction submodule is used to collect multi-source image data of the target monitoring area and use the SIFT algorithm to extract feature points;

[0012] The feature point matching submodule is used to perform feature point matching based on the extracted feature point sets of each image using the Euclidean distance metric;

[0013] The region growing submodule is used to perform region growing operations in the image using the obtained matching point pairs as seed points. The image is adjusted after the initial alignment by region growing using the position and posture information of the image acquisition device to achieve pixel-level alignment.

[0014] The fusion submodule is used to fuse the aligned multi-source images using a weighted average fusion method to generate a fused image.

[0015] Optionally, in a second implementation of the present invention, the region growing submodule includes:

[0016] The preliminary alignment unit is used to use the obtained matching point pairs as seed points. Starting from each seed point, it checks its adjacent pixels. If the features of the adjacent pixels meet the region growth criterion, the pixel is merged into the current growth region and continuously expanded to the surrounding area until there are no adjacent pixels that meet the growth criterion, thus completing the preliminary alignment of the image.

[0017] An establishing unit, for establishing a spatial relationship between different images using position information and posture information of an image acquisition device, and obtaining translation and rotation parameters between images;

[0018] The geometric transformation unit is used to perform geometric transformation on the images that have been initially aligned by region growing according to the translation and rotation parameters between the images, adjust the spatial position and direction of the multi-source images, and achieve pixel-level alignment.

[0019] Optionally, in a third implementation of the present invention, the preprocessing module includes:

[0020] The first traversal submodule is used to set the noise type and threshold, traverse each pixel of the fused image and make a judgment. If the pixel meets the noise type and threshold, the noise is removed by combining median filtering and bilateral filtering;

[0021] The division submodule is used to divide the denoised image into several equal-sized and overlapping sub-blocks, calculate the histogram of each sub-block respectively, and calculate the grayscale mapping relationship according to the histogram distribution;

[0022] The first calculation submodule is used to calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image. For the pixels in the overlapping area of ​​the sub-blocks, a weighted average method is used to determine the weight according to the distance between the pixel point and the center of each sub-block to calculate the final grayscale value of the pixel point in the overlapping area;

[0023] The merging submodule is used to merge all sub-blocks processed by adaptive histogram equalization according to the arrangement order of the original image to obtain a pre-processed image and highlight the terrain features.

[0024] Optionally, in a fourth implementation of the present invention, the analysis module includes:

[0025] The labeling submodule is used to build a dual-channel deep convolutional network architecture, collect multiple sets of image data from the target monitoring area at different times, mark the real change areas, and divide the multiple sets of image data into training sets, validation sets, and test sets;

[0026] The input submodule is used to input the image data in the training set into a two-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, while channel two analyzes the temporal change trend of the image. After the fusion layer and the output layer, a predicted pixel-level change probability map is obtained.

[0027] The first comparison submodule is used to compare the predicted pixel-level change probability map with the marked real change area, and calculate the error between the predicted result and the real result using the cross entropy loss function;

[0028] The update submodule is used to propagate the error of the loss function from the output layer to each layer of the network through the back-propagation algorithm, calculate the gradient of the parameters of each layer based on the error, and update the network parameters using stochastic gradient descent;

[0029] The evaluation submodule is used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network;

[0030] The second traversal submodule is used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output a pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change area.

[0031] Optionally, in a fifth implementation of the present invention, the dual-channel deep convolutional network architecture includes:

[0032] Channel 1 architecture: A network structure consisting of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers use convolution kernels of different sizes to extract texture and elevation features of the image. The pooling layers use max pooling to reduce the data dimension to obtain key features. The fully connected layers integrate the extracted features and output feature vectors.

[0033] Channel 2 architecture: It consists of a convolutional layer, an LSTM layer, and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector.

[0034] Fusion layer: A fusion layer is set after the two channels to concatenate the feature vectors output by channel one and channel two to form a comprehensive feature vector;

[0035] Output layer: A fully connected layer with a Softmax activation function is used as the output layer to output a pixel-level change probability map.

[0036] Optionally, in a sixth implementation of the present invention, the generating module includes:

[0037] The determination submodule is used to obtain the initial point cloud data of the detected terrain change area through stereo matching and lidar data analysis and processing, and determine the boundary of the change area;

[0038] The removal submodule is used to analyze the distance distribution between each point in the point cloud data and its neighboring points using statistical filtering, remove outliers, and use grid sampling to reduce the number of points in dense point clouds;

[0039] The conversion submodule is used to construct an octree data structure based on the point cloud data. According to the position and normal vector information of the point cloud, the implicit function value corresponding to each point is calculated. The implicit function is sampled using the marching cube algorithm, the zero isosurface is extracted, and the zero isosurface is converted into a triangular mesh to obtain the reconstructed terrain surface model.

[0040] The second comparison submodule is used to compare the reconstructed current terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the size and direction of the displacement vector;

[0041] The second calculation submodule is used to calculate the volume change of each triangular grid unit in the settlement area according to the displacement vector in the three-dimensional terrain difference model, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change of the settlement area.

[0042] Optionally, in a sixth implementation of the present invention, a method for implementing a terrain change monitoring system based on image analysis is implemented, the method comprising the following steps:

[0043] Acquire multi-source image data of the target monitoring area, fuse the multi-source image data using a dynamic adaptive registration algorithm, and generate a fused image;

[0044] The fused image is preprocessed to remove noise from the image, and an adaptive histogram equalization algorithm is used to enhance the contrast of the denoised image and highlight the terrain features, thus obtaining the preprocessed image to be analyzed.

[0045] Construct a dual-channel deep convolutional network to analyze the pre-processed image to be analyzed and detect areas of terrain change;

[0046] Incremental point cloud processing technology is used to process the detected terrain change areas, and combined with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time;

[0047] The generated three-dimensional terrain difference model and volume change thermal map are analyzed, the parameters of terrain change are calculated, and early warning processing is carried out.

[0048] Optionally, in a sixth implementation of the present invention, a method for implementing a terrain change monitoring system based on image analysis is implemented, the method comprising the following steps:

[0049] Collect multi-source image data of the target monitoring area and use SIFT algorithm to extract feature points;

[0050] Based on the extracted feature point sets of each image, feature point matching is performed using the Euclidean distance metric;

[0051] Using the obtained matching point pairs as seed points, a region growing operation is performed in the image. The image that has been initially aligned by region growing is adjusted using the position and posture information of the image acquisition device to achieve pixel-level alignment.

[0052] The aligned multi-source images are fused using a weighted average fusion method to generate a fused image.

[0053] Optionally, in a sixth implementation of the present invention, a method for implementing a terrain change monitoring system based on image analysis is implemented, the method comprising the following steps:

[0054] Set the noise type and threshold, traverse each pixel of the fused image and make a judgment. If the pixel meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove the noise;

[0055] Divide the denoised image into several equal-sized and overlapping sub-blocks, calculate the histogram of each sub-block separately, and calculate the grayscale mapping relationship based on the histogram distribution;

[0056] Calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image. For the pixels in the overlapping area of ​​the sub-blocks, use weighted averaging to determine the weight according to the distance from the pixel to the center of each sub-block to calculate the final grayscale value of the pixel in the overlapping area.

[0057] All sub-blocks processed by adaptive histogram equalization are merged according to the arrangement order of the original image to obtain a preprocessed image that highlights the terrain features.

[0058] In the technical solution provided by the present invention, the terrain change monitoring system based on image analysis includes an image acquisition module, a preprocessing module, an analysis module, a generation module and an early warning module; the present invention greatly improves the accuracy of terrain change monitoring, can promptly detect minor terrain changes, realize real-time monitoring of terrain changes, and can output early warning reports on the same day to promptly grasp the terrain changes. Through the dynamic adaptive registration algorithm, it effectively eliminates interference such as seasonal lighting differences, can more comprehensively and accurately assess the possible impacts and risks of terrain changes, improve the accuracy and reliability of early warnings, and reduce a large amount of manpower and material resources. At the same time, through optimization algorithms such as incremental point cloud processing technology, data processing efficiency is improved, computing costs are reduced, monitoring costs in local areas are lower, and higher-frequency monitoring can be achieved, with high cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0060] Figure 1 A schematic diagram of a first embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of a second embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of a third embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The terms "first," "second," "third," "fourth," and so forth (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar items and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product, or device.

[0064] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A schematic diagram of a first embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention includes:

[0065] The image acquisition module is used to acquire multi-source image data of the target monitoring area, fuse the multi-source image data using a dynamic adaptive registration algorithm, and generate a fused image;

[0066] The preprocessing module is used to preprocess the fused image, remove the noise in the image, and use the adaptive histogram equalization algorithm to enhance the contrast of the denoised image, highlight the terrain features, and obtain the preprocessed image to be analyzed;

[0067] The analysis module is used to build a dual-channel deep convolutional network to analyze the pre-processed image to be analyzed and detect the area of ​​terrain change;

[0068] The generation module is used to process the detected terrain change areas using incremental point cloud processing technology and combine it with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time;

[0069] The early warning module is used to analyze the generated three-dimensional terrain difference model and volume change heat map, calculate the parameters of terrain change, and perform early warning processing.

[0070] In this embodiment, the preprocessing module includes:

[0071] The first traversal submodule is used to set the noise type and threshold, traverse each pixel of the fused image and make a judgment. If the pixel meets the noise type and threshold, the noise is removed by combining median filtering and bilateral filtering;

[0072] The division submodule is used to divide the denoised image into several equal-sized and overlapping sub-blocks, calculate the histogram of each sub-block respectively, and calculate the grayscale mapping relationship according to the histogram distribution;

[0073] The first calculation submodule is used to calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image. For the pixels in the overlapping area of ​​the sub-blocks, a weighted average method is used to determine the weight according to the distance between the pixel point and the center of each sub-block to calculate the final grayscale value of the pixel point in the overlapping area;

[0074] The merging submodule is used to merge all sub-blocks processed by adaptive histogram equalization according to the arrangement order of the original image to obtain a pre-processed image and highlight the terrain features.

[0075] In this embodiment, the generation module includes:

[0076] The determination submodule is used to obtain the initial point cloud data of the detected terrain change area through stereo matching and lidar data analysis and processing, and determine the boundary of the change area;

[0077] The removal submodule is used to analyze the distance distribution between each point in the point cloud data and its neighboring points using statistical filtering, remove outliers, and use grid sampling to reduce the number of points in dense point clouds;

[0078] The conversion submodule is used to construct an octree data structure based on the point cloud data. According to the position and normal vector information of the point cloud, the implicit function value corresponding to each point is calculated. The implicit function is sampled using the marching cube algorithm, the zero isosurface is extracted, and the zero isosurface is converted into a triangular mesh to obtain the reconstructed terrain surface model.

[0079] The second comparison submodule is used to compare the reconstructed current terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the size and direction of the displacement vector;

[0080] The second calculation submodule is used to calculate the volume change of each triangular grid unit in the settlement area according to the displacement vector in the three-dimensional terrain difference model, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change of the settlement area.

[0081] In this embodiment, the possible noise types in the image are determined, such as Gaussian noise, salt and pepper noise, etc., and corresponding noise recognition thresholds are set for different noise types. For Gaussian noise, the fluctuation range of its mean and variance is set as the threshold; for salt and pepper noise, a threshold is set for the ratio of pixel values ​​deviating from the normal range (such as close to 0 or 255); each pixel point of the fused image is traversed and judged according to the noise type and threshold set in the first step; if the pixel point meets the noise characteristics, it is processed by combining median filtering and bilateral filtering; with the current noise pixel point as the center, a filter window of appropriate size is selected, and the median value of the pixel points in the window is calculated. , replace the noise pixels with the median; then use bilateral filtering to further smooth the pixels according to the spatial distance and pixel value difference of the pixels, remove the noise while retaining the image edge information as much as possible; divide the denoised image into several equal-sized and overlapping sub-blocks, and the sub-block size is determined according to the image resolution and the complexity of the terrain features; for each sub-block, calculate its histogram respectively; count the frequency of each gray level in the sub-block to obtain the grayscale histogram distribution of the sub-block; at the same time, calculate the mean and variance of the grayscale value in the sub-block as the basis for subsequent adjustment of the histogram equalization parameters; local contrast enhancement: for each sub-block, according to its grayscale Histogram distribution, calculate the grayscale mapping relationship, when calculating the grayscale mapping according to the traditional histogram equalization method, the mapped grayscale value is multiplied by the adjustment factor, so that the contrast enhancement amplitude of the sub-block with larger variance (more noise) is relatively small, and the contrast enhancement amplitude of the sub-block with smaller variance (less noise) is relatively large; calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image; for the pixel point in each sub-block, based on the grayscale value after local contrast enhancement, make a secondary adjustment according to the difference between the sub-block where the pixel point is located and the global statistical information; if the sub-block grayscale mean is higher than the global average grayscale mean, appropriately reduce the grayscale mean of the sub-block. Otherwise, it is appropriately increased. At the same time, the adjustment factor is fine-tuned according to the difference between the sub-block variance and the global average variance to further balance the local and global contrast enhancement effects. For the pixels in the overlapping area of ​​sub-blocks, the processing results in multiple sub-blocks are comprehensively considered. The weighted average method is used to determine the weight according to the distance from the pixel to the center of each sub-block. The closer the distance, the greater the weight. The final grayscale value of the pixel in the overlapping area is calculated to ensure a smooth transition of the image at the sub-block splicing point. All sub-blocks processed by the improved adaptive histogram equalization are merged according to the arrangement order of the original image to obtain the pre-processed image, highlighting the terrain features.

[0082] In this embodiment, for detected terrain change areas, initial point cloud data for that area is obtained from the original image data through methods such as stereo matching and LiDAR data analysis. This point cloud data contains the 3D coordinates (X, Y, Z) of each point, and some may also include attribute information such as color and reflection intensity. Based on the previously detected terrain change areas, the boundaries of the changed areas are determined. The contour information obtained through image segmentation can be converted into boundary polygons in 3D space to define the area requiring incremental processing. New data acquisition: When new image data is collected, new point cloud data is again acquired within the boundaries of the changed areas through methods such as stereo matching or LiDAR data analysis. Data fusion: The newly acquired point cloud data is fused with the initial point cloud data. Points in duplicate areas are filtered and merged based on their attribute information (such as timestamp and accuracy), retaining more accurate point information and completing the incremental update of the point cloud data. Outlier removal: Methods such as statistical filtering or radius filtering are used to analyze the distance distribution between each point in the point cloud data and its neighboring points. A distance threshold is set to identify points outside the threshold as outliers and remove them, improving the quality of the point cloud data. Point cloud simplification: Using methods such as voxel filtering or grid sampling, dense point clouds are streamlined while preserving their shape characteristics, reducing the number of points and subsequent computational complexity. Octree data structure is constructed: Based on the point cloud data, an octree data structure is constructed. The 3D space containing the point cloud is divided into eight subspaces, and the subdivision is repeated recursively until the number of points in each subspace meets the set threshold or reaches the maximum number of subdivision levels, accelerating subsequent surface reconstruction. Implicit function calculation: Based on the position and normal information of the point cloud, the implicit function value corresponding to each point is calculated. By solving the Poisson equation, an implicit function that is continuous throughout the entire 3D space is constructed. The zero isosurface of this function is the reconstructed surface. Surface mesh extraction: Using methods such as marching cubes, the implicit function is sampled to extract the zero isosurface, which is converted into a triangular mesh to obtain the reconstructed terrain surface model. The reconstructed current terrain surface model is compared with the historical terrain surface model. By calculating the 3D coordinate difference between corresponding points in the two models, the displacement vector of each point is obtained. The difference results are visualized based on the magnitude and direction of the displacement vectors. A color-coded approach can be used, with different colors representing different displacement magnitudes, to generate a 3D terrain difference model that visually displays terrain changes. For subsidence areas, the volume change of each microregion (such as a triangular mesh element) is calculated based on the displacement vectors in the 3D terrain difference model. The total volume change of the entire subsidence area is obtained through integration or accumulation. The subsidence area is then divided into multiple subregions according to specific rules, and the average volume change rate of each subregion is calculated.According to the magnitude of the volume change rate, a heat map is drawn using different colors and transparencies. The darker the color and the lower the transparency, the greater the volume change rate, thereby generating a heat map of the volume change in the settlement area.

[0083] In this embodiment, the terrain change parameters to be calculated are determined, including but not limited to the area, volume, slope change, elevation change, etc. of the terrain change; for each terrain change parameter, a corresponding risk threshold is set based on the geological conditions of the monitoring area, historical terrain change data and actual application needs. For example, based on the historical settlement data of the mining area, a volume change exceeding 100 cubic meters is set as a high-risk threshold, and 50-100 cubic meters is set as a medium-risk threshold; a slope change exceeding 15° is set as a high risk, and 8°-15° is set as a medium risk, etc.; from the three-dimensional terrain difference model, the three-dimensional coordinate data of the terrain change area is extracted, including the X, Y, and Z coordinate information of each change point; from the volume change heat map of the settlement area, the volume change rate data of each sub-area and the corresponding area range information are obtained; based on the extracted three-dimensional coordinate data of the terrain change area, it is projected onto a two-dimensional plane (such as the XY plane), and the polygon area calculation method (such as the vector cross product method and the partition sum method) is used to calculate the projected area of ​​the terrain change area on the two-dimensional plane; combined with the Z coordinate change of the change point in the three-dimensional terrain difference model and the corresponding area data, the total volume of the terrain change is calculated by integration or accumulation. For the sub-regions in the heat map, the volume change of each sub-region is calculated based on the volume change rate and the initial volume of the sub-region. The slope value of each region before and after the terrain change is calculated. The slope value can be obtained by calculating the tangent of the angle between the normal vector of the terrain surface at a certain point and the vertical direction vector. The slope values ​​before and after the change are compared to obtain the slope change of each region. The elevation data (Z coordinate) of each point before and after the change in the terrain change area is extracted, and the elevation difference of each point is calculated. The average elevation change and maximum elevation change of the entire region are statistically obtained. The calculated terrain change parameters are compared with the risk threshold set in the first step. A hierarchical assessment mechanism is used. If a parameter exceeds the high-risk threshold, the region is judged to be in a high-risk state; if it is within the medium-risk threshold, it is judged to be in a medium-risk state; if it is below the medium-risk threshold, it is judged to be in a low-risk state. For comprehensive assessments of multiple parameters, weights can be assigned based on their importance, and a comprehensive risk level is calculated through weighted calculation. When the assessment results in a high or medium risk state, an early warning mechanism is triggered. Based on pre-set warning methods, early warning information is sent to relevant personnel (such as geological hazard monitoring departments and regional management personnel) via SMS, email, audible and visual alarms, and other channels. This information includes the name of the monitored area, the risk level, and the main terrain change parameters and their status, allowing relevant personnel to take timely response measures.

[0084] See also Figure 2, a schematic diagram of a second embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention, wherein the image acquisition module includes:

[0085] The feature point extraction submodule is used to collect multi-source image data of the target monitoring area and use the SIFT algorithm to extract feature points;

[0086] The feature point matching submodule is used to perform feature point matching based on the extracted feature point sets of each image using the Euclidean distance metric;

[0087] The region growing submodule is used to perform region growing operations in the image using the obtained matching point pairs as seed points. The image is adjusted after the initial alignment by region growing using the position and posture information of the image acquisition device to achieve pixel-level alignment.

[0088] The fusion submodule is used to fuse the aligned multi-source images using a weighted average fusion method to generate a fused image.

[0089] In this embodiment, the region growing submodule includes:

[0090] The preliminary alignment unit is used to use the obtained matching point pairs as seed points. Starting from each seed point, it checks its adjacent pixels. If the features of the adjacent pixels meet the region growth criterion, the pixel is merged into the current growth region and continuously expanded to the surrounding area until there are no adjacent pixels that meet the growth criterion, thus completing the preliminary alignment of the image.

[0091] An establishing unit, for establishing a spatial relationship between different images using position information and posture information of an image acquisition device, and obtaining translation and rotation parameters between images;

[0092] The geometric transformation unit is used to perform geometric transformation on the images that have been initially aligned by region growing according to the translation and rotation parameters between the images, adjust the spatial position and direction of the multi-source images, and achieve pixel-level alignment.

[0093] In this embodiment, multi-source image data from the target monitoring area is collected, including satellite multispectral imagery, drone oblique photography images, and ground camera video streams. Metadata such as the GPS location information, IMU attitude information (including pitch, yaw, and roll angles), and acquisition time corresponding to each image acquisition device at the time of image acquisition are also obtained to provide basic reference information for subsequent image registration. Feature points are extracted using the SIFT (Scale Invariant Feature Transform) algorithm from the collected multi-source image data. Extreme points at different scales are detected in the image. Stable key points are found using a Gaussian Difference Pyramid. The orientation and descriptors of the key points are calculated to obtain a feature point set for each image. Each feature point contains information such as location, scale, orientation, and descriptor. Based on the extracted feature point sets for each image, feature point matching is performed using the Euclidean distance metric. The Euclidean distance between the descriptors of feature points in different images is calculated. A distance threshold is set, and feature point pairs with a distance less than the threshold are considered matched. In this way, a preliminary correspondence between the multi-source images is established. Region growing is performed within the image using the resulting matched point pairs as seed points. Starting from each seed point, adjacent pixels are gradually merged into the same region according to certain similarity criteria (such as similarity in color, texture, and grayscale) until a stopping condition is met (e.g., region growing reaches a certain area, or the difference between adjacent pixels exceeds a set threshold). Through region growing, corresponding matching regions are identified in the multi-source images, further refining the correspondence between the images. GPS location and IMU attitude information is incorporated to precisely adjust the images initially aligned through region growing. Based on the position and attitude of the image acquisition device, spatial transformation relationships (including translation, rotation, and scaling parameters) are calculated between the images. These transformation parameters are used to perform geometric transformations on the images, achieving more precise alignment of the multi-source images in terms of spatial position and orientation, eliminating interference caused by differences in shooting angle, location, and seasonal illumination, and achieving pixel-level alignment. The precisely aligned multi-source images are then fused using a weighted average fusion method. Different weights are assigned to each image based on its characteristics and importance. For example, high-resolution drone imagery, which excels in detail, can be given a higher weight; satellite imagery, which excels in large-area coverage and spectral information, is also given a corresponding weight based on the actual situation. Then, the pixel values ​​of the corresponding pixels in each image are weighted and summed according to the weights to generate a fused image, integrating the advantageous information of multiple source images.

[0094] In this embodiment, the criteria used to determine whether pixels belong to the same region include, but are not limited to, color difference thresholds, texture similarity thresholds, and grayscale difference thresholds. For example, the color difference threshold is set to a value of no more than 20 for each channel in the RGB color space; the texture similarity threshold is set to a value of no more than a certain feature difference by calculating the local binary pattern (LBP) features of the pixel neighborhood; and the grayscale difference threshold is set to no more than 15. Using the resulting matching point pairs as seed points, starting from each seed point, the adjacent pixels are examined. For each adjacent pixel, its color, texture, and grayscale features are compared with the corresponding features of the seed point and the pixels within the grown region. If the features of the adjacent pixel meet the region growing criteria set in the first step, the pixel is merged into the current growing region. This process is repeated, expanding until no adjacent pixels meet the growing criteria, or the growing region reaches a predetermined stop condition, such as a maximum area or maximum number of growing steps, completing the initial image alignment. The spatial relationship between the different images in the world coordinate system is established using the image acquisition device's position information (GPS coordinates) and attitude information (pitch, yaw, and roll angles). Through coordinate transformation and geometric calculations, transformation parameters such as translation (displacement in the X, Y, and Z axes), rotation angle (rotation around the X, Y, and Z axes), and scaling are determined between the images. For example, the relative position of the two devices is calculated based on their GPS coordinates, and the difference in the device shooting orientation is calculated based on posture information, thereby obtaining the translation and rotation parameters between the images. Based on the spatial transformation parameters calculated in the third step, the images initially aligned through region growing are geometrically transformed. For translation, the images are directly shifted in the X, Y, and Z axes by the corresponding displacement; for rotation, the image pixel coordinates are transformed using a rotation matrix; and for scaling, the images are enlarged or reduced according to the calculated scaling ratio. Through these geometric transformations, the spatial position and orientation of the multi-source images are precisely adjusted to achieve pixel-level alignment.

[0095] See also Figure 3 , a schematic diagram of a third embodiment of a terrain change monitoring system based on image analysis provided by an embodiment of the present invention, wherein the analysis module includes:

[0096] The labeling submodule is used to build a dual-channel deep convolutional network architecture, collect multiple sets of image data from the target monitoring area at different times, mark the real change areas, and divide the multiple sets of image data into training sets, validation sets, and test sets;

[0097] The input submodule is used to input the image data in the training set into a two-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, while channel two analyzes the temporal change trend of the image. After the fusion layer and the output layer, a predicted pixel-level change probability map is obtained.

[0098] The first comparison submodule is used to compare the predicted pixel-level change probability map with the marked real change area, and calculate the error between the predicted result and the real result using the cross entropy loss function;

[0099] The update submodule is used to propagate the error of the loss function from the output layer to each layer of the network through the back-propagation algorithm, calculate the gradient of the parameters of each layer based on the error, and update the network parameters using stochastic gradient descent;

[0100] The evaluation submodule is used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network;

[0101] The second traversal submodule is used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output a pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change area.

[0102] In this embodiment, the dual-channel deep convolutional network architecture includes:

[0103] Channel 1 architecture: A network structure consisting of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers use convolution kernels of different sizes to extract texture and elevation features of the image. The pooling layers use max pooling to reduce the data dimension to obtain key features. The fully connected layers integrate the extracted features and output feature vectors.

[0104] Channel 2 architecture: It consists of a convolutional layer, an LSTM layer, and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector.

[0105] Fusion layer: A fusion layer is set after the two channels to concatenate the feature vectors output by channel one and channel two to form a comprehensive feature vector;

[0106] Output layer: A fully connected layer with a Softmax activation function is used as the output layer to output a pixel-level change probability map.

[0107] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A terrain change monitoring system based on image analysis, characterized in that: The system includes: The image acquisition module is used to acquire multi-source image data of the target monitoring area, fuse the multi-source image data using a dynamic adaptive registration algorithm, and generate a fused image; The preprocessing module is used to preprocess the fused image, remove the noise in the image, and use the adaptive histogram equalization algorithm to enhance the contrast of the denoised image, highlight the terrain features, and obtain the preprocessed image to be analyzed; The analysis module is used to build a dual-channel deep convolutional network to analyze the pre-processed image to be analyzed and detect areas of terrain change; The generation module is used to process the detected terrain change areas using incremental point cloud processing technology and combine it with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time; The early warning module is used to analyze the generated three-dimensional terrain difference model and volume change heat map, calculate the parameters of terrain change, and perform early warning processing.

2. The terrain change monitoring system based on image analysis according to claim 1, characterized in that: The image acquisition module includes: The feature point extraction submodule is used to collect multi-source image data of the target monitoring area and use the SIFT algorithm to extract feature points; The feature point matching submodule is used to perform feature point matching based on the extracted feature point sets of each image using the Euclidean distance metric; The region growing submodule is used to perform region growing operations in the image using the obtained matching point pairs as seed points. The image is adjusted after the initial alignment by region growing using the position and posture information of the image acquisition device to achieve pixel-level alignment. The fusion submodule is used to fuse the aligned multi-source images using a weighted average fusion method to generate a fused image.

3. The terrain change monitoring system based on image analysis according to claim 2, characterized in that: The region growing submodule includes: The preliminary alignment unit is used to use the obtained matching point pairs as seed points. Starting from each seed point, it checks its adjacent pixels. If the features of the adjacent pixels meet the region growth criterion, the pixel is merged into the current growth region and continuously expanded to the surrounding area until there are no adjacent pixels that meet the growth criterion, thus completing the preliminary alignment of the image. An establishing unit, for establishing a spatial relationship between different images using position information and posture information of an image acquisition device, and obtaining translation and rotation parameters between images; The geometric transformation unit is used to perform geometric transformation on the images that have been initially aligned by region growing according to the translation and rotation parameters between the images, adjust the spatial position and direction of the multi-source images, and achieve pixel-level alignment.

4. The terrain change monitoring system based on image analysis according to claim 1, characterized in that: The pre-processing module comprises: The first traversal submodule is used to set the noise type and threshold, traverse each pixel of the fused image and make a judgment. If the pixel meets the noise type and threshold, the noise is removed by combining median filtering and bilateral filtering; The division submodule is used to divide the denoised image into several equal-sized and overlapping sub-blocks, calculate the histogram of each sub-block respectively, and calculate the grayscale mapping relationship according to the histogram distribution; The first calculation submodule is used to calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image. For the pixels in the overlapping area of ​​the sub-blocks, a weighted average method is used to determine the weight according to the distance between the pixel point and the center of each sub-block to calculate the final grayscale value of the pixel point in the overlapping area; The merging submodule is used to merge all sub-blocks processed by adaptive histogram equalization according to the arrangement order of the original image to obtain a pre-processed image and highlight the terrain features.

5. The terrain change monitoring system based on image analysis according to claim 1, characterized in that: The analysis module includes: The labeling submodule is used to build a dual-channel deep convolutional network architecture, collect multiple sets of image data from the target monitoring area at different times, mark the real change areas, and divide the multiple sets of image data into training sets, validation sets, and test sets; The input submodule is used to input the image data in the training set into a two-channel deep convolutional network. Channel one extracts the texture and elevation features of the image, while channel two analyzes the temporal change trend of the image. After the fusion layer and the output layer, a predicted pixel-level change probability map is obtained. The first comparison submodule is used to compare the predicted pixel-level change probability map with the marked real change area, and calculate the error between the predicted result and the real result using the cross entropy loss function; The update submodule is used to propagate the error of the loss function from the output layer to each layer of the network through the back-propagation algorithm, calculate the gradient of the parameters of each layer based on the error, and update the network parameters using stochastic gradient descent; The evaluation submodule is used to evaluate and adjust the network using the validation set after each round of training to obtain a trained dual-channel deep convolutional network; The second traversal submodule is used to input the preprocessed image to be analyzed into the trained dual-channel deep convolutional network, output a pixel-level change probability map, and traverse each pixel point in the pixel-level change probability map to determine the terrain change area.

6. The terrain change monitoring system based on image analysis according to claim 5, characterized in that: The dual-channel deep convolutional network architecture includes: Channel 1 architecture: A network structure consisting of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layers use convolution kernels of different sizes to extract texture and elevation features of the image. The pooling layers use max pooling to reduce the data dimension to obtain key features. The fully connected layers integrate the extracted features and output feature vectors. Channel 2 architecture: It consists of a convolutional layer, an LSTM layer, and a fully connected layer. The convolutional layer performs preliminary feature extraction on the image. The LSTM layer receives the feature sequence output by the convolutional layer and analyzes the temporal trend of the image. The fully connected layer processes the output of the LSTM layer to generate a temporal feature vector. Fusion layer: A fusion layer is set after the two channels to concatenate the feature vectors output by channel one and channel two to form a comprehensive feature vector; Output layer: A fully connected layer with a Softmax activation function is used as the output layer to output a pixel-level change probability map.

7. The terrain change monitoring system based on image analysis according to claim 1, characterized in that: The generation module includes: The determination submodule is used to obtain the initial point cloud data of the detected terrain change area through stereo matching and lidar data analysis and processing, and determine the boundary of the change area; The removal submodule is used to analyze the distance distribution between each point in the point cloud data and its neighboring points using statistical filtering, remove outliers, and use grid sampling to reduce the number of points in dense point clouds; The conversion submodule is used to construct an octree data structure based on the point cloud data. According to the position and normal vector information of the point cloud, the implicit function value corresponding to each point is calculated. The implicit function is sampled using the marching cube algorithm, the zero isosurface is extracted, and the zero isosurface is converted into a triangular mesh to obtain the reconstructed terrain surface model. The second comparison submodule is used to compare the reconstructed current terrain surface model with the historical terrain surface model, and generate a three-dimensional terrain difference model according to the size and direction of the displacement vector; The second calculation submodule is used to calculate the volume change of each triangular grid unit in the settlement area according to the displacement vector in the three-dimensional terrain difference model, divide the settlement area into multiple sub-areas, calculate the average volume change rate of each sub-area, and generate a heat map of the volume change of the settlement area.

8. A method for implementing a terrain change monitoring system based on image analysis as claimed in claim 1, characterized in that: The method comprises the following steps: Acquire multi-source image data of the target monitoring area, fuse the multi-source image data using a dynamic adaptive registration algorithm, and generate a fused image; The fused image is preprocessed to remove noise from the image, and an adaptive histogram equalization algorithm is used to enhance the contrast of the denoised image and highlight the terrain features, thus obtaining the preprocessed image to be analyzed. Construct a dual-channel deep convolutional network to analyze the pre-processed image to be analyzed and detect areas of terrain change; Incremental point cloud processing technology is used to process the detected terrain change areas, and combined with the Poisson surface reconstruction algorithm to generate a three-dimensional terrain difference model and a heat map of the volume change of the settlement area in real time; The generated three-dimensional terrain difference model and volume change thermal map are analyzed, the parameters of terrain change are calculated, and early warning processing is carried out.

9. A method for implementing a terrain change monitoring system based on image analysis as claimed in claim 1, characterized in that: The method comprises the following steps: Collect multi-source image data of the target monitoring area and use SIFT algorithm to extract feature points; Based on the extracted feature point sets of each image, feature point matching is performed using the Euclidean distance metric; Using the obtained matching point pairs as seed points, a region growing operation is performed in the image. The image that has been initially aligned by region growing is adjusted using the position and posture information of the image acquisition device to achieve pixel-level alignment. The aligned multi-source images are fused using a weighted average fusion method to generate a fused image.

10. A method for implementing a terrain change monitoring system based on image analysis as claimed in claim 1, characterized in that: The method comprises the following steps: Set the noise type and threshold, traverse each pixel of the fused image and make a judgment. If the pixel meets the noise type and threshold, use a combination of median filtering and bilateral filtering to remove the noise; Divide the denoised image into several equal-sized and overlapping sub-blocks, calculate the histogram of each sub-block separately, and calculate the grayscale mapping relationship based on the histogram distribution; Calculate the average grayscale mean and average grayscale variance of all sub-blocks to obtain the global statistical information of the image. For the pixels in the overlapping area of ​​the sub-blocks, use weighted averaging to determine the weight according to the distance from the pixel to the center of each sub-block to calculate the final grayscale value of the pixel in the overlapping area. All sub-blocks processed by adaptive histogram equalization are merged according to the arrangement order of the original image to obtain a preprocessed image that highlights the terrain features.

Images