An InSAR deformation monitoring method using the extraction results of bridge targets

Abstract

The present disclosure provides an InSAR deformation monitoring method using the bridge target extraction result, including: acquiring an optical remote sensing image and performing rotated rectangle annotation on its bridge target, and obtaining a bridge multi-target detection model through a target detection algorithm; inputting the remote sensing image to be detected into the bridge multi-target detection model for detection to obtain the final detection result; acquiring the corner coordinates corresponding to the detection result, generating a SHP vector bridge extraction result through geocoding, and obtaining a bridge extraction boundary mask file for the research area; processing the time-series SAR images of the research area by the PS-InSAR method, and taking into account the above bridge boundary mask file when selecting PS points, and finally obtaining the cumulative deformation result of the bridges in the research area. The present disclosure uses a deep learning method to automatically and accurately extract the position, length, width, axial direction and other information of the bridge target in the remote sensing image, connects it with the InSAR data processing technology, increases the PS points of the bridge target in the research area, and effectively improves the processing speed and accuracy of deformation monitoring.

Description

Technical Field

[0001] The present disclosure relates to the field of radar technology, and particularly to an InSAR deformation monitoring method using the extraction result of bridge targets. Background Art

[0002] In recent years, satellite remote sensing technology has developed rapidly, and remote sensing satellites with high resolution and low revisit period have emerged continuously. With the gradual improvement of the space-based observation system, remote sensing technology has now been widely applied to various aspects of our daily life, having a significant impact on civilian and military fields such as traffic highway detection, environmental and agricultural detection, and port environment perception. Remote sensing technology has also received increasing attention from countries around the world and has even been elevated to a strategic height as a strategic goal for the long-term stable development of the country.

[0003] Regarding the research on remote sensing images, different branches have emerged according to the development of different fields. Among them, accurately finding bridge targets from large-scale remote sensing images is a research hotspot in land transportation remote sensing images. In the civilian aspect, by detecting the area near the bridge in the remote sensing image, bridge settlement and collapse accidents can be discovered and warned in a timely manner; in the military field, it can be used for all-day search and positioning of important bridge targets, real-time tracking and perception, etc.

[0004] The commonly used method for detecting bridge targets in remote sensing images is the threshold segmentation detection algorithm, but this method has problems such as high false alarms and poor anti-interference ability in complex scenarios. In response to this problem, researchers introduced a deep learning detection algorithm, taking advantage of the powerful feature extraction ability of convolutional neural networks to enhance the detection performance of the algorithm. The existing deep learning object detection network predicts the position information of bridge targets, with significant errors and the inability to accurately predict the true length, width, and axial information of bridge targets. There are also problems such as adhesion in detection and low recall rate when multiple bridge targets are densely arranged. Summary of the Invention

[0005] To solve the above-mentioned existing defects, the present invention provides an InSAR deformation monitoring method using the extraction result of bridge targets, aiming to automatically and accurately extract information such as the position, length, width, and axis of bridge targets in remote sensing images using deep learning methods, and connect it with InSAR data processing technology, which can increase the PS points of bridge targets in the research area and effectively improve the processing speed and accuracy of deformation monitoring using InSAR technology.

[0006] The present invention provides an InSAR deformation monitoring method using bridge target extraction results, comprising: acquiring optical remote sensing images and performing rotated rectangular annotation on bridge targets in a study area, and obtaining a bridge multi-target detection model through a target detection algorithm; inputting the remote sensing images to be detected into the bridge multi-target detection model for processing to obtain a final detection result; acquiring the corner point coordinates corresponding to the detection result, and generating a SHP vector bridge extraction result by geocoding the corner point coordinates to obtain a bridge extraction boundary mask file in the study area; using the PS-InSAR method to process the time series SAR images of the study area, and taking the bridge extraction boundary mask file into account when selecting PS points, and finally obtaining the cumulative deformation results of the bridges in the study area.

[0007] Furthermore, the remote sensing images annotated with rotated rectangles are divided into a training set and a test set, and a multi-target bridge detection model for detecting bridge targets is obtained after learning and training respectively; the multi-target bridge detection model includes a basic feature extraction network and a classification regression network.

[0008] Furthermore, the rotated rectangle is a minimum area circumscribed rectangle that includes the bridge target.

[0009] Furthermore, after overlapping cropping and segmentation and positioning of the remote sensing image to be detected, the image is input into the bridge multi-target detection model for detection, and a preliminary detection result including a pre-selected box of the bridge target and the corresponding confidence level is obtained;

[0010] Calculate the rotation intersection and union ratio of the pre-selected box with the highest confidence and other pre-selected boxes, delete the pre-selected boxes whose rotation intersection and union ratio is greater than the set threshold, and obtain the final detection result of the overall detection;

[0011] All detection results are integrated and calculated to obtain all bridge recognition results. The corresponding corner point coordinates are obtained by performing minimum outer envelope calculation on them.

[0012] Furthermore, the SHP vector bridge extraction result is the bridge geometric boundary result.

[0013] Furthermore, the bridge extraction boundary mask file is obtained according to the geographic coordinate range of the optical image, and the corresponding digital elevation model file is obtained. The intensity image of the main image in the optical image and the external digital elevation model are used for geocoding to obtain a lookup table of the mapping relationship between the geographic coordinate system and the radar image coordinate system. According to the mapping relationship between different coordinate systems in the lookup table, the bridge geometric boundary result is converted from the geographic coordinate system to the radar coordinate system, and a corresponding mask file is generated.

[0014] Furthermore, geocoding uses the range Doppler model to convert the row and column numbers of the pixel points on the radar influence coordinate system into actual plane geographic coordinates.

[0015] Further, select the main image from the time-series SAR image sequence;

[0016] Select the interferometric pairs in the main image and perform differential interferometry on the interferometric pairs;

[0017] Use the coherence coefficient or amplitude deviation to select the permanent scatterer coherent target points on each interferometric pair, and at the same time add the bridge extraction boundary mask file to retain all bridge observation point targets, and finally obtain the cumulative deformation result of the bridge in the study area.

[0018] Further, construct a Delaunay triangulation for the high-coherence target points among the permanent scatterer coherent target points, and solve the parameters of the elevation residual difference and velocity difference for the arcs between the high-coherence target points to obtain the solution parameters of the arcs.

[0019] Further, select stable permanent scatterer coherent target points in the SAR image as reference points, perform indirect adjustment on the solution parameters of all reference points, and finally obtain the deformation value and deformation rate of each reference point, and finally obtain the cumulative deformation result of the bridge in the study area.

[0020] According to the embodiments of the present disclosure, it can be used for bridge deformation detection by using the bridge target extraction result. The algorithm is robust and reliable, easy to program and implement, and provides a new method for bridge deformation detection, which realizes the following technical effects:

[0021] (1) By redefining the bridge target position, without significantly modifying the existing target detection network structure, the minimum area circumscribed rectangle of all bridge targets in the large-scale remote sensing image can be predicted. Through the overall non-maximum suppression (Non Maximum Suppression, NMS), the redundant prediction rectangle frames can be accurately and quickly removed, further improving the speed and accuracy of bridge prediction;

[0022] (2) Convert the recognition result into a SHP vector result in the geographic coordinate system to generate a thematic map of the bridge recognition result for subsequent data warehousing and processing;

[0023] (3) Geocode the bridge SHP vector extraction result to obtain the bridge range in the radar coordinate system, and can retain the bridge observation targets to the greatest extent during the process of extracting observation targets for subsequent data processing;

[0024] (4) Using the automatically extracted bridge geometric boundary range can not only ensure the effectiveness of the observation area, but also avoid the processing of non-target areas, thereby effectively improving the efficiency of InSAR data processing.

[0025] It should be understood that the content described in the Summary of the Invention section is not intended to limit the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0027] Figure 1 is a schematic flow chart of the InSAR deformation monitoring method of the present disclosure;

[0028] Figure 2 is the overall flow chart of the InSAR deformation monitoring method of the present disclosure;

[0029] Figure 3 is the remote sensing image before bridge identification of the InSAR deformation monitoring method of the present disclosure;

[0030] Figure 4 is the remote sensing image after bridge identification of the InSAR deformation monitoring method of the present disclosure;

[0031] Figure 5 is the coherence before superimposing the bridge identification result in the SAR coordinate system of the InSAR deformation monitoring method of the present disclosure;

[0032] Figure 6 is the coherence after superimposing the bridge identification result in the SAR coordinate system of the InSAR deformation monitoring method of the present disclosure;

[0033] Figure 7 is the bridge deformation detection result map before adding the mask file in the geographic coordinate system of the InSAR deformation monitoring method of the present disclosure;

[0034] Figure 8 is the bridge deformation detection result map after adding the mask file in the geographic coordinate system of the InSAR deformation monitoring method of the present disclosure;

[0035] Figure 9 is the radar imaging geometric principle diagram of the InSAR deformation monitoring method of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0037] In addition, the term "and / or" in this document is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after.

[0038] In the present disclosure, regarding the problem of how to use the bridge target extraction result for InSAR deformation measurement, an InSAR deformation monitoring method using the bridge target extraction result is provided. The purpose is to accurately output information such as the position, length, width, and axial direction of the bridge target, improve the processing speed and accuracy of using InSAR technology to detect the deformation of the bridge, and contribute to the display of the bridge InSAR deformation monitoring results.

[0039] The present disclosure provides an InSAR deformation monitoring method using the bridge target extraction result, including: acquiring an optical remote sensing image and performing rotated rectangle annotation on the bridge target therein, and obtaining a bridge multi-target detection model through a target detection algorithm; inputting the remote sensing image to be detected into the bridge multi-target detection model for detection to obtain the final detection result; acquiring the corner coordinates corresponding to the detection result, generating a SHP vector bridge extraction result through geocoding, and obtaining a bridge extraction boundary mask file for the research area; processing the time-series SAR images of the research area using the PS-InSAR method, and considering the above bridge boundary mask file when selecting PS points, and finally obtaining the cumulative deformation result of the bridge in the research area.

[0040] In the above embodiment, the detection steps of the InSAR deformation monitoring method using the bridge target extraction result can refer to Figure 1 and Figure 2 , specifically including:

[0041] Step 1: Collect remote sensing images containing bridge targets and crop the remote sensing images to a set size;

[0042] Step 2: Perform rotated rectangle annotation on the bridge targets contained in the remote sensing images respectively, and use the remote sensing images annotated with rotated rectangles as the training set and the test set; the rotated rectangle is the minimum area circumscribed rectangle containing the bridge target;

[0043] Step 3: Use the training set and the test set to train a bridge multi-object detection model including a basic feature extraction network and a classification and regression network, and obtain a bridge multi-object detection model for detecting bridge targets;

[0044] Step 4: Overlap and crop the large-scale remote sensing data to be detected and perform segmentation and positioning, then input it into the trained bridge multi-object detection model for detection, and obtain a preliminary detection result including the preselected bounding boxes of bridge targets and the corresponding confidence levels;

[0045] Step 5: Calculate the rotational intersection over union (IoU) between the preselected bounding box with the highest confidence level and other preselected bounding boxes, and delete the preselected bounding boxes with a rotational IoU greater than the set threshold to obtain the overall detection result of the large-scale remote sensing image;

[0046] Step 6: Obtain the corner coordinates corresponding to the detection result, generate a SHP vector bridge extraction result through geocoding, and obtain a bridge extraction boundary mask file for the study area;

[0047] Step 7: Process the time-series SAR images of the study area using the PS-InSAR method, and take into account the above-mentioned bridge boundary mask file when selecting PS points, and finally obtain the cumulative deformation result of the bridges in the study area.

[0048] In the above embodiment, the resolution of the remote sensing image can be selected as 15000*15000, and it is segmented and cropped into 608*608 according to an overlap ratio of 0.3. The bridge targets included in the remote sensing image are respectively marked with rotated rectangles. Taking the upper left corner of the remote sensing image as the origin, the bridge targets with any rotation angle are marked as (x, y, w, h, θ).

[0049] In the above embodiment, perform integrated calculation on all the cropped image results to obtain all the bridge recognition results, and then calculate the minimum bounding box of the bridges to obtain the corresponding corner coordinates (x1, y1, x2, y2, x3, y3, x4, y4).

[0050] In the above embodiment, the steps of finally obtaining the cumulative deformation result of the bridges by combining the PS-InSAR method with the bridge extraction boundary mask file are as follows:

[0051] Select the master image among the time-series SAR sequences;

[0052] Perform differential interferometry processing on the selected interferometric pairs, where

[0053] The phase of the PS (permanent scatterer) points on each interferometric pair can be expressed as:

[0054]

[0055] In the formula, is the phase value of the i-th PS point on the n-th interference pair, represents the integer ambiguity, represents the topographic residual phase, represents the phase influence caused by the atmosphere, represents other observation noises, represents the phase influence caused by deformation;

[0056] Select permanent scatterer coherent target points by using methods such as coherence coefficient or amplitude deviation, and at the same time add a bridge extraction boundary mask file to ensure that the bridges identified in the remote sensing image can be observed by the SAR image at the same time; specifically, obtain the DEM file of the area according to the geographic coordinate range of the optical image; perform geocoding on the intensity image of the master image and the external DEM to obtain the mapping relationship between the radar coordinate system and the geographic coordinate system, that is, the lookup table; the lookup table can be used to convert the SHP vector extraction result from the geographic coordinate system to the radar coordinate system, obtain the bridge identification range in the radar coordinate system, and generate the corresponding bridge extraction boundary mask file; during the process of coherent target recognition, add this mask file to retain the observed point target of the bridge, that is, the coherent target, to the greatest extent.

[0057] Construct a Delaunay triangulation network for high coherence target points, and solve the parameters of the difference in elevation residual and the difference in velocity for the arcs between points; specifically, the phase of the PS points on each interference pair can be expressed as:

[0058]

[0059] where is the elevation residual of each PS point, and v i is the deformation rate of each PS point;

[0060] Take the difference between every two PS points on an interference pair to get:

[0061]

[0062] Using the LAMDBA method to perform time-dimensional phase unwrapping can calculate the difference in elevation residual of the PS arc The difference in deformation rate Δv i,j , continue to perform unwrapping in the spatial dimension, and the deformation value and deformation rate on the PS points can be obtained;

[0063] Select stable permanent scatterer coherent targets in the image as reference points, perform indirect adjustment on the solved parameters on all arcs, and finally obtain the deformation value and deformation rate of each PS point.

[0064] In the above embodiment, the main step in the process of forming the bridge extraction boundary mask file is geocoding, and the main step of geocoding is to convert the row and column numbers of the pixel points on the radar coordinate system to the actual planar geographic coordinates by using the range-Doppler model.

[0065]

[0066] Among them, as Figure 9 shown in the radar imaging geometry of , the vectors are respectively the flight speed of the satellite and the position vector from the earth center to the satellite, F is the frequency, λ is the wavelength, and the Doppler angle Ω can be obtained from Equation (2-1); the can be obtained from the image parameter file, and the slant range unit vector can be obtained by using Equation (2-3); according to (2-4), (2-5) and the given geodetic height h, the slant range l is calculated; by changing the , the formulas (2-3), (2-4) and (2-5) are repeated until the slant range l is consistent with the true distance, and then the iteration is stopped; the Cartesian coordinates (X, Y, Z) of the pixel in the radar coordinate system are obtained according to Equation (2-4).

[0067] Taking the Cartesian coordinate system as an intermediate bridge, a mapping relationship between the SAR image coordinate system and the geographic coordinate system is established. Using this mapping relationship, the geographic coordinates of the bridge range are converted to the radar coordinate system, and a mask file is generated.

[0068] The coherence coefficient in space is expressed as:

[0069]

[0070] In the formula, M(i, j) and S(i, j) represent the complex values at the position (i, j) in the pixel coordinate system of the master and slave images, * is the conjugate complex number, and the value range of γ is [0, 1].

[0071] On the time scale, the amplitude deviation index is expressed as:

[0072]

[0073] In the formula, σ A represents the standard deviation of the image intensity (amplitude) in the time dimension, σ v represents the standard deviation of the image phase in the time dimension, m A represents the mean value of the image intensity (amplitude) in the time dimension. D A is the amplitude deviation index.

[0074] In order to retain as many bridge point targets as possible, within the area corresponding to the bridge range mask file, the standard for reducing its amplitude deviation index is lowered, so that both the bridge observation point targets can be improved and the point targets in the non-observation area can be reduced, thereby improving the calculation efficiency.

[0075] In the above embodiment, as Figure 3 and Figure 4 shown, Figure 3 is the remote sensing image before bridge recognition, Figure 4 is the remote sensing image after bridge recognition through this embodiment.

[0076] In the above embodiment, as Figure 5 and Figure 6 shown, Figure 5 is the coherence before superimposing the bridge recognition result in the SAR coordinate system. The weak point targets with lower coherence on the bridge cannot be observed as shown by the boxes in the figure; Figure 6 is the coherence after superimposing the bridge recognition result in the SAR coordinate system. The weak point targets with lower coherence on the bridge can be clearly observed after being restricted by the recognition range as shown by the boxes in the figure.

[0077] In the above embodiment, as Figure 7 and Figure 8 shown, Figure 7 is the bridge deformation detection result map before adding the mask file in the geographic coordinate system; Figure 8 is the bridge deformation detection result map after adding the mask file in the geographic coordinate system. It can be seen that the front and back images are accurately displayed.

[0078] In the above embodiment, the InSAR deformation monitoring method using the bridge target extraction result accurately predicts the true length, width, and axial information of the bridge target through the center coordinates, length, and width information of the minimum bounding rectangle of the bridge target and the existence of the length and width information of the minimum bounding rectangle of the bridge target and the true length and width of the bridge target; by introducing the rotated region extraction algorithm, the minimum area bounding rectangle of the bridge target is predicted, and while realizing the refined prediction of the position, length, width, and axial information of the bridge target, the problem of mutual "inhibition" of adjacent prediction frames in the ordinary target detection model is effectively avoided.

[0079] According to the embodiments of the present disclosure, the following technical effects are achieved:

[0080] (1) By redefining the position of the bridge target, without significantly modifying the existing target detection network structure, the minimum area bounding rectangle of all bridge targets in the large-scale remote sensing image can be predicted. Through the overall non-maximum suppression (Non Maximum Suppression, NMS), the redundant prediction rectangles can be accurately and quickly removed, further improving the speed and accuracy of bridge prediction;

[0081] (2) Convert the recognition result into an SHP vector result in the geographic coordinate system, and generate a thematic map of the bridge recognition result for subsequent data storage and processing;

[0082] (3) Geocode the extracted SHP vector result of the bridge to obtain the bridge range in the radar coordinate system, and retain the bridge observation target to the greatest extent during the process of extracting the observation target for subsequent data processing;

[0083] (4) Using the automatically extracted geometric boundary range of the bridge can not only ensure the effectiveness of the observation area, but also avoid the processing of non-target areas, thereby effectively improving the efficiency of InSAR data processing.

[0084] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0085] It should be understood that various forms of the processes shown above can be used, reordering, adding or deleting steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. There is no limitation herein.

[0086] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An InSAR deformation monitoring method using the extraction result of bridge targets, characterized in that include: Obtain optical remote sensing images and mark the bridge targets in the study area with rotated rectangles, and obtain a multi-target detection model for bridges through target detection algorithms; Inputting the remote sensing image to be detected into the bridge multi-target detection model for processing to obtain the final detection result; The corner point coordinates corresponding to the detection result are obtained, and the corner point coordinates are used to generate SHP vector bridge extraction results through geocoding to obtain the bridge extraction boundary mask file of the study area, wherein: The SHP vector bridge extraction result is a bridge geometric boundary result; The bridge extraction boundary mask file is obtained according to the geographic coordinate range of the optical image, and the corresponding digital elevation model file is obtained. The intensity image of the main image in the optical image and the external digital elevation model are used to perform the geographic coding, and a lookup table of the mapping relationship between the geographic coordinate system and the radar image coordinate system is obtained. According to the mapping relationship between different coordinate systems in the lookup table, the bridge geometric boundary result is converted from the geographic coordinate system to the radar coordinate system, and a corresponding mask file is generated; The PS-InSAR method is used to process the time-series SAR images of the study area, and the bridge extraction boundary mask file is taken into account when selecting PS points, and finally the cumulative deformation results of the bridges in the study area are obtained.

2. The method according to claim 1, characterized in that The remote sensing image annotated with the rotated rectangle is divided into a training set and a test set, and the multi-target bridge detection model for detecting bridge targets is obtained after learning and training respectively; the multi-target bridge detection model includes a basic feature extraction network and a classification regression network.

3. The method according to claim 1, characterized in that The rotated rectangle is a minimum area circumscribed rectangle that includes the bridge object.

4. The method according to claim 1, characterized in that: After overlapping cropping and segmenting and locating the remote sensing image to be detected, the image is input into the bridge multi-target detection model for detection to obtain a preliminary detection result including a bridge target pre-selection box and a corresponding confidence level; Calculate the rotation intersection-and-union ratio of the pre-selected box with the highest confidence and the other pre-selected boxes, delete the pre-selected boxes whose rotation intersection-and-union ratio is greater than a set threshold, and obtain the final detection result of the overall detection; All the detection results are integrated and calculated to obtain all the bridge recognition results, and the corresponding corner point coordinates are obtained by performing minimum outer envelope calculation on them.

5. The method according to claim 1, characterized in that The geocoding is to convert the row and column numbers of the pixel points on the radar influence coordinate system into the actual plane geographic coordinates using the range Doppler model.

6. The method according to claim 1, characterized in that Selecting a main image from the time-series SAR images; Selecting an interference pair in the main image, and performing differential interference processing on the interference pair; The coherence coefficient or amplitude deviation is used to select the coherent target points of the permanent scatterer on each interference pair, and the bridge extraction boundary mask file is added at the same time to retain all bridge observation point targets, and finally the cumulative deformation results of the bridges in the study area are obtained.

7. The method according to claim 6, characterized in that a Delaunay triangulation network is constructed for the high coherence target points among the permanent scatterer coherent target points, and parameter solutions of the difference between elevation residuals and the difference between velocities are carried out for the arcs between the high coherence target points to obtain the solution parameters of the arcs.

8. The method according to claim 7, characterized in that stable permanent scatterer coherent target points are selected as reference points in the SAR image, indirect adjustment is carried out on the solution parameters of all the reference points, and finally the deformation values and deformation rates of each reference point are obtained, and finally the cumulative deformation result of the bridge in the study area is obtained.

Images