Disaster deformation image offset registration method and system

Abstract

The invention discloses an offset registration method and system for disaster deformation images, and relates to the technical field of disaster deformation monitoring, and the method comprises the steps: obtaining to-be-registered images at the same place before and after disaster deformation through a plurality of platforms, and dividing the to-be-registered images into a plurality of blocks; acquiring a reference image with geographic reference information before disaster deformation, extracting feature points on the reference image, determining a search area in the to-be-registered image according to the geographic reference information of the feature points, and detecting homonymy points between the reference image and each to-be-registered image; and carrying out error correction on each image to be registered through the homonymy points, splicing a plurality of corrected images to be registered to obtain a registered image, and carrying out optical offset processing on the registered image to obtain the offset of the registered image. According to the method, the defect of insufficient data or insufficient time resolution of a single platform is overcome, and the time resolution and the space resolution are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of disaster deformation monitoring, and in particular to an offset registration method and system for disaster deformation images. Background Art

[0002] Collapse, landslide, debris flow, earthquake and ice and rock avalanche are the most common natural geological disasters in mountainous areas. Due to their high frequency and great destructive power, there is an urgent need to monitor and investigate the deformation of geological disaster risk areas to provide knowledge for the disaster early warning system. These disaster chains have a profound impact on the safety of life and property in the region and the development of water conservancy projects. Deformation monitoring of natural geological disasters poses a major challenge to traditional methods such as global navigation satellite systems, total stations and levels. These methods are difficult to meet the requirements of large-scale mobile monitoring of glaciers in high mountain valleys. In contrast, the rapidly developing remote sensing technology uses non-contact methods such as satellites and aircraft to provide all-weather and all-weather observations, effectively overcoming the limitations of traditional technologies.

[0003] The image cross-correlation technology of optical images uses a cross-correlation algorithm to estimate the pixel displacement between images, so that the relative movement of ground feature points can be tracked and sub-pixel horizontal deformation information can be obtained. Many researchers use platform observations with sensors to quantify the displacement signals of ground disasters.

[0004] Usually, research on disaster deformation offset monitoring focuses on the result fusion of a single platform, which requires the platform to obtain image data with a longer temporal resolution and a large amount of reference data. However, existing platforms have insufficient temporal resolution of image data and insufficient reference data, which makes it difficult to meet the needs of deformation monitoring within the expected time period when the results are fused, greatly increasing the difficulty of studying and predicting disaster chains, and the accuracy of offset monitoring is not high. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and provide a method and system for offset registration of disaster deformation images, so as to solve the problem that the prior art is difficult to meet the needs of deformation monitoring within the expected time period, which greatly increases the difficulty of studying and predicting disaster chains in remote areas and leads to low accuracy of deformation monitoring.

[0006] The present invention specifically provides the following technical solutions:

[0007] A method for offset registration of disaster deformation images comprises the following steps:

[0008] Acquire the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks;

[0009] Obtain a reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered based on the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered;

[0010] The error of each image to be registered is corrected through the same-name points, and the corrected multiple images to be registered are spliced ​​to obtain the registered image, and the registered image is optically offset processed to obtain the offset of the image after the registration is completed.

[0011] Preferably, the extracting feature points from the reference image comprises the following steps:

[0012] Divide the reference image into n×n non-overlapping grids;

[0013] In each grid, the Harris value of the point feature extraction operator of each pixel is obtained, and k pixels with the largest Harris value are selected as feature points, and n×n×k feature points are extracted in the reference image.

[0014] Preferably, detecting the same-name points between the reference image and each image to be registered includes:

[0015] The CFOG algorithm and frequency domain space accelerated matching algorithm are used to detect the same-name points between the reference image and each image to be registered.

[0016] Preferably, the method of using the CFOG algorithm and the frequency domain space acceleration matching algorithm to detect the same-name points between the reference image and each image to be registered includes:

[0017] Obtaining the gradient features of multiple directions of the reference image and each image to be registered, and arranging the gradient features of each direction in the Z direction to obtain the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered;

[0018] Use an approximately three-dimensional Gaussian convolution kernel to convolve the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered, and generate a CFOG descriptor for the reference image and each image to be registered;

[0019] The square sum of the difference between the reference image and each image to be registered is used for matching, and the square sum of the difference is accelerated using fast Fourier transform to obtain the same-name points between the reference image and each image to be registered.

[0020] Preferably, after detecting the same-name points between the reference image and each image to be registered, the method further includes:

[0021] Use all the same-name points to build a geometric mapping model;

[0022] Use the least squares method to obtain the residuals and root mean square error (RMSE) of the same-name points, and remove the same-name points with the largest residuals;

[0023] The process of removing the homonymous points with the largest residual is repeated until the RMSE is less than the given threshold, and the homonymous points after error elimination are obtained.

[0024] Preferably, the selection of the geometric mapping model depends on the geometric distortion between images.

[0025] The present invention provides an offset registration system for disaster deformation images, comprising:

[0026] An image acquisition module is used to acquire the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks;

[0027] The same-name point detection module is used to obtain the reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered through the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered;

[0028] The offset acquisition module is used to correct the error of each image to be registered through the same-name points, splice the corrected images to obtain the registered image, and perform optical offset processing on the registered image to obtain the offset of the registered image.

[0029] The present invention provides a computer device, comprising a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor executes the steps of the above-mentioned offset registration method of disaster deformation images.

[0030] The present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned offset registration method of disaster deformation images are implemented.

[0031] Compared with the prior art, the present invention has the following significant advantages:

[0032] The present invention adopts the setting of reference images, extracts the features of the reference images, and then determines the search area of ​​the image to be registered, obtains the same-name points between the two, that is, uses multi-platform data for reference to supplement the time resolution of the area, overcomes the defects of insufficient data from a single platform or insufficient time resolution, and can fully explore the value of multi-source data. At the same time, each piece of the image to be registered is corrected and spliced ​​through the same-name points to prevent the overall deviation caused by excessive local errors, thereby obtaining the registered image and offset, and improving the accuracy and quality of deformation monitoring within the expected time period. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of an offset registration method of a disaster deformation image in an embodiment of the present invention;

[0034] Figure 2 It is a structural schematic diagram of an image offset acquisition system based on cross-platform registration in an embodiment of the present invention;

[0035] Figure 3 This is the block division and offset calculation diagram for the 20171107 image and the 20171204 image; Figure 3 (a) is a block diagram of the 20171107 image. Figure 3 (b) is a block diagram of the 20171204 image. Figure 3 (c) Figure 3 (d) and Figure 3 (e) The unregistered image, the overall registered image, and the four-block registered image in the north-south direction, respectively. Figure 3 (f) Figure 3 (g) and Figure 3 (h) The unregistered image, the overall registered image, and the four-block registered image in the east-west direction respectively;

[0036] Figure 4 This is the block division and offset calculation diagram for the 20171107 image - 20171221 image; Figure 4 (a) is the upper and lower block diagram of 20171107. Figure 4 (b) Figure 4 (c) and Figure 4 (d) are the upper and lower blocks of 20171221, the upper four blocks of 20171221, and the lower four blocks of 20171221. Figure 4 (e) Figure 4 (f) Figure 4 (g) and Figure 4 (h) are the unregistered, overall registered, upper and lower block registered, and four-block registered images in the north-south direction. Figure 4 (i) Figure 4 (j) Figure 4 (k) and Figure 4 (l) The unregistered, overall registered, upper and lower block registered, and four-block registered images in the east-west direction respectively;

[0037] Figure 5 middle Figure 5 (a) Schematic diagram of section line ABC and stable region, Figure 5 (b) is a schematic diagram of the profile deformation;

[0038] Figure 6The mean MEV, standard deviation STD and root mean square error RMSE in the statistical stability area are quantitatively improved; among them, Figure 6 (a) is the mean MEV and standard deviation STD of the north-south and east-west excursions in the stable region from 20171107 to 20171204. Figure 6 (b) is the mean MEV and standard deviation STD of the north-south and east-west excursions in the stable region from 20171107 to 20171221. Figure 6 (c) is the percentage of north-south RESE reduction and east-west RESE reduction in the stable region from 20171107 to 20171204. Figure 6 (d) The percentage of north-south RESE reduction and the percentage of east-west RESE reduction in the stable area from 20171107 to 20171221;

[0039] in, Figure 2 101 is an image four-block module; 102 is a feature point detection module; 103 is a feature point matching module; 104 is an error elimination module; 105 is an image correction and stitching module; and 106 is an offset calculation module. DETAILED DESCRIPTION

[0040] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] Part I:

[0042] The first aspect of the present invention provides an offset registration method for disaster deformation images, such as Figure 1 As shown, including:

[0043] In the embodiment of the present invention, the Sedongpu area before and after the Milin earthquake on November 18, 2017 was selected as the research object, and the Gaojing-1 image data on November 7, 2017, the Pleiades image data on December 4, 2017, and the Beijing-2 image data on December 21, 2017 were used to unify the three different source data into one coordinate system. When the method of the present invention is used, the following steps are specifically included:

[0044] Step S1: Acquire the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks.

[0045] When the image to be registered is divided into multiple blocks, an area with a certain length and width ratio and overlapped in four quadrants of the image to be registered is obtained, and the area is divided into multiple blocks. In one embodiment of the present invention, the area is divided into four blocks.

[0046] The Gaojing-1 image data of November 7, 2017 was selected as the main image, thus generating two image pairs. The purpose of the four-block is to prevent the overall deviation caused by excessive local errors, and the registration is changed from one polynomial to four polynomials.

[0047] Step S2: Obtain a reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered through the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered.

[0048] Extract feature points from the reference image, including:

[0049] Divide the reference image into n×n non-overlapping grids.

[0050] In each grid, the Harris value of the point feature extraction operator of each pixel is obtained, and k pixels with the largest Harris value are selected as feature points, and n×n×k feature points are extracted in the reference image.

[0051] Detect the same-name points between the reference image and each image to be registered, including:

[0052] The dense structural feature descriptor CFOG (channel features of orientated gradients) algorithm and the frequency domain space accelerated matching algorithm (fast matching similarity measurement) are used to detect the same-name points between the reference image and each image to be registered. Including:

[0053] CFOG Description:

[0054] Obtain the gradient features of multiple directions (for example, 9) of the reference image and each image to be registered, and arrange the gradient features of each direction in the Z direction to obtain the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered.

[0055] Use an approximately three-dimensional Gaussian convolution kernel to convolve the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered, and generate the CFOG descriptor of the reference image and each image to be registered. The function of the three-dimensional Gaussian convolution kernel is that the convolution in the two-dimensional XY direction can replace the trilinear interpolation to generate the direction histogram, and the convolution in the Z direction is operated in the gradient direction, which can reduce the influence of local geometric distortion on the feature vector.

[0056] Frequency domain space accelerated matching algorithm: Use the sum of squared differences (SSD (Sum of Squared Difference)) of the CFOG descriptors of the reference image and each image to be registered for matching. In order to speed up the operation, the fast Fourier transform FFT is used to accelerate the sum of squared differences SSD (Sum of Squared Differences) to obtain the same-name points between the reference image and each image to be registered. Because the point multiplication operation in the frequency domain is equivalent to the convolution operation in the spatial domain, the use of FFT can increase the operation speed by dozens or even hundreds of times.

[0057] Step S3: performing error correction on each image to be registered through the same-name points, splicing the corrected images to be registered to obtain the registered image, and performing optical offset processing on the registered image to obtain the offset of the registered image.

[0058] In the above image matching, mismatches (i.e., same-name points with large errors) are inevitable due to some occlusions such as clouds and shadows.

[0059] After detecting the same-name points between the reference image and each image to be registered, the following steps are also included:

[0060] Use all points of the same name to model the geometric mapping.

[0061] The least squares method is used to obtain the residual error and root mean square error (RMSE) of the same-name points, and the same-name points with the largest residual error are removed.

[0062] The process of removing the same-name points with the largest residual is repeated until the RMSE is less than a given threshold (e.g., 2 pixels), and the same-name points after error elimination are obtained.

[0063] In the error removal process, the choice of geometric mapping model (univariate, bivariate, or cubic polynomial model) depends on the geometric distortion between images.

[0064] The image to be registered is corrected using the same-name points eliminated by the geometric mapping model, and finally the four corrected images are spliced ​​to obtain the registered image.

[0065] When the registered images are processed for optical offset, cosi-corr software is used to obtain the offset of two image pairs (20171107-20171204, 20171107-20171221). After processing, the obvious errors are removed.

[0066] like Figure 2As shown, based on the above method, the present invention provides an offset registration system for disaster deformation images, including: an image acquisition module (image four-block module 101), a same-name point detection module (feature point detection module 102, feature point matching module 103) and an offset acquisition module (error elimination module 104, image correction and stitching module 105, offset calculation module 106).

[0067] Among them, the image acquisition module is used to obtain the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks; the same-name point detection module is used to obtain the reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered through the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered; the offset acquisition module is used to correct the errors of each image to be registered through the same-name points, splice the corrected multiple images to be registered, obtain the registered image, and perform optical offset processing on the registered image to obtain the offset of the image after the registration is completed.

[0068] The research area selected for this experiment is the Sedongpu gully disaster chain on the left bank of the Yarlung Zangbo River in Linzhi City, Tibet Autonomous Region. The time period is the Sedongpu area before and after the Milin earthquake on November 18, 2017. The data used are the Gaojing-1 image data on November 7, 2017, the Pleiades image data on December 4, 2017, and the Beijing-2 image data on December 21, 2017. The offset calculation of cross-platform data is realized by using the offset method based on cross-platform registration of the present invention, such as Figure 3 and 4 First, the image is divided into blocks, and four blocks are registered. Then, the optical offset is processed using Cosi-corr software to obtain Figure 3 and 4 North-south and east-west deformation. Figure 5 The section line ABC shown in the figure is extracted to obtain the deformation between the two image pairs, that is, the deformation caused by the Milin earthquake on November 18, 2017. Figure 5 The green stable area shown in the figure quantifies the improvement effect by counting the mean (MEV), standard deviation (STD) and root mean square error (RMSE) of the stable area, as shown in the figure. Figure 6 As shown, the improvement effect of the four-block for the north-south and east-west directions is very obvious, and the root mean square error reduction percentage reaches 80%, which shows that this method has a significant improvement in the registration accuracy and the monitoring accuracy.

[0069] The offset method and system based on cross-platform registration of the present invention can use data from different platforms to perform registration and offset calculation, and can monitor large-scale disaster deformation in remote mountainous areas. The registration speed is fast and the accuracy is significantly improved.

[0070] The present invention also provides a computer device, including a memory and a processor. The memory stores a program. When the program is executed by the processor, the processor executes the steps of an offset registration method for disaster deformation images.

[0071] In accordance with the disclosed embodiments, a computing device may communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth communications, etc.), or with any device (e.g., routers, modems, etc.) that enables a computing device to communicate with one or more other computing devices.

[0072] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of an offset registration method for disaster deformation images are implemented.

[0073] According to the disclosed embodiments, the storage medium may be a non-volatile computer-readable storage medium, such as but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0074] The above content is a further detailed description of the present invention in combination with a specific preferred embodiment. For technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for offset registration of disaster deformation images, characterized in that: The steps include: Acquire the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks; Obtain a reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered based on the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered; The error of each image to be registered is corrected through the same-name points, and the corrected multiple images to be registered are spliced ​​to obtain the registered image, and the registered image is optically offset processed to obtain the offset of the image after the registration is completed.

2. The offset registration method of disaster deformation image according to claim 1, characterized in that: The extracting feature points from the reference image comprises: Divide the reference image into n×n non-overlapping grids; In each grid, the Harris value of the point feature extraction operator of each pixel is obtained, and k pixels with the largest Harris value are selected as feature points, and n×n×k feature points are extracted in the reference image.

3. The offset registration method of disaster deformation image according to claim 1, characterized in that: The detecting of the same-name points between the reference image and each image to be registered comprises: The dense structural feature descriptor CFOG algorithm and the frequency domain space accelerated matching algorithm are used to detect the same-name points between the reference image and each image to be registered.

4. The offset registration method of disaster deformation image according to claim 3, characterized in that: The method of using the CFOG algorithm and the frequency domain space acceleration matching algorithm to detect the same-name points between the reference image and each image to be registered includes: Obtaining the gradient features of multiple directions of the reference image and each image to be registered, and arranging the gradient features of each direction in the Z direction to obtain the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered; Use an approximately three-dimensional Gaussian convolution kernel to convolve the three-dimensional multi-directional gradient feature map of the reference image and each image to be registered, and generate a CFOG descriptor for the reference image and each image to be registered; The square sum of the difference between the reference image and each image to be registered is used for matching, and the square sum of the difference is accelerated using fast Fourier transform to obtain the same-name points between the reference image and each image to be registered.

5. The offset registration method of disaster deformation image according to claim 1, characterized in that: After detecting the same-name points between the reference image and each image to be registered, the method further includes: Use all the same-name points to build a geometric mapping model; Use the least squares method to obtain the residuals and root mean square error (RMSE) of the same-name points, and remove the same-name points with the largest residuals; The process of removing the homonymous points with the largest residual is repeated until the RMSE is less than the given threshold, and the homonymous points after error elimination are obtained.

6. The offset registration method of disaster deformation image according to claim 5, characterized in that: The selection of the geometric mapping model depends on the geometric distortion between the images.

7. An offset registration system for disaster deformation images, characterized in that: include: An image acquisition module is used to acquire the images to be registered at the same location before and after the disaster deformation through multiple platforms, and divide the images to be registered into multiple blocks; The same-name point detection module is used to obtain the reference image with geographic reference information before the disaster deformation, extract feature points on the reference image, determine the search area in the image to be registered through the geographic reference information of the feature points, and detect the same-name points between the reference image and each image to be registered; The offset acquisition module is used to correct the error of each image to be registered through the same-name points, splice the corrected images to obtain the registered image, and perform optical offset processing on the registered image to obtain the offset of the registered image.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein a program is stored in the memory, and when the program is executed by the processor, the processor executes the steps of the offset registration method of disaster deformation images as claimed in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the offset registration method of disaster deformation images described in any one of claims 1 to 6 are implemented.

Images