FPGA Registration Method and Structure of 2D InSAR Images

Abstract

A FPGA registration method and registration structure for two-dimensional InSAR images. In the embodiment, the method includes: first, FPGA uses a coarse registration algorithm of the maximum correlation function method to obtain the range and azimuth offsets of the main and auxiliary InSAR images, obtains the addresses of the main and auxiliary images in DDR according to the azimuth offsets, and filters the azimuth data; then, with the help of AXI Interconnect, the filtered main and auxiliary image data are converted from AXI-Master to AXI-Stream data streams and sent to RAM in parallel; finally, the range data of the main and auxiliary images are filtered out in RAM according to the range offsets, and the interference phase and coherence coefficient are calculated according to the filtered azimuth and range data. The FPGA registration method for two-dimensional InSAR images adopted by the present invention simplifies the complex address mapping formula caused by selecting data in both azimuth and range directions, and has advantages in terms of real-time, simplicity, upgradeability and maintainability, ensuring high-speed and high-efficiency InSAR data processing.

Description

Technical Field

[0001] The present invention relates to the field of radar real-time imaging technology, and more particularly to an FPGA registration method and registration structure for two-dimensional interferometric synthetic aperture radar images. Background Art

[0002] The basic principle of InSAR (Interferometric Synthetic Aperture Radar Technology) is that the interferometric synthetic aperture radar generates a SAR image after imaging the same area twice with a small geometric parallax through the same orbit (SAR image is slant range imaging, the direction along the orbit is called azimuth, and the direction along the radar wave emission is range). The distance between the radar and the area is not equal during the two imagings, which causes a phase difference between the two images at the same image points, forming an interference pattern. The phase value in the interference pattern is the phase difference measurement value of the two imagings. According to the geometric relationship between the phase difference of the two imagings and the three-dimensional spatial position of the ground target, the three-dimensional coordinates (azimuth, range, elevation) of the ground target can be calculated using the flight orbit parameters. Due to the deviation of the imaging orbit, viewing angle or time of the two images, there will be certain misalignment and distortion in the range and azimuth directions. Before generating the interference pattern, the two images of the same scene must be accurately matched and superimposed. One of the images is used as the reference image, regarded as the main image, and the other image is used as the image to be registered, regarded as the auxiliary image. The process of registering the main and auxiliary images in azimuth and range is called two-dimensional interferometric synthetic aperture radar image registration. Interferometric synthetic aperture radar image registration is mainly implemented in DSP and GPU. At present, there is no index to the registration process for implementing two-dimensional rough registration on FPGA.

[0003] For example, FPGA+DSP heterogeneity is used to implement SAR / InSAR (synthetic aperture radar / interferometric synthetic aperture radar) imaging, in which FPGA implements SAR imaging processing and control scheduling functions and then three DSPs in the system jointly implement the InSAR process. There are also technical solutions based on FPGA-based data interaction, two-dimensional coarse alignment optimization to one-dimensional alignment, ping-pong to improve data throughput, and DSP-based engineering to implement one-dimensional interferometric synthetic aperture radar image alignment. There are also technical solutions that use CPU+GPU heterogeneity to implement interference coefficient maps and filtering calculation functions, but these solutions all have their own problems.

[0004] For example, in DSP, image registration takes up relatively more time for data scheduling. Although it was optimized in later research, it also resulted in loss of phase accuracy. GPU is not suitable for onboard processing in terms of power consumption when implementing image registration. On the other hand, these technical solutions often use DDR for data selection in both azimuth and distance, resulting in discontinuous addresses and loss of data access efficiency, thus reducing the real-time performance and access efficiency of the system. Summary of the invention

[0005] The present invention discloses at least one embodiment to provide an FPGA registration method for a two-dimensional InSAR image, comprising: FPGA uses a coarse registration algorithm of a maximum correlation function method to obtain the range and azimuth offsets of the InSAR main and auxiliary images; FPGA obtains the addresses of the main and auxiliary images in DDR according to the azimuth offset and filters the azimuth data; FPGA uses AXIInterconnect to convert the filtered main and auxiliary image data from AXI-Master to AXI-Stream data streams and sends them to RAM in parallel; FPGA filters the range data of the main and auxiliary images in RAM according to the range offset; and after filtering the range and azimuth data, FPGA uses a self-compiled IP core to calculate the interference phase and coherence coefficient of the main and auxiliary InSAR images.

[0006] The present invention discloses at least one embodiment, which provides an FPGA registration structure for a two-dimensional InSAR image, including: an offset calculation module, which uses a maximum function correlation method to calculate the range and azimuth offsets of a primary and secondary image; a primary data screening module, which specifies a read data address to screen azimuth data according to the azimuth offset; an AXI-Interconnect conversion module, which converts the screened data from an AXI-Master to an AXI-Stream and sends it to a RAM array; a secondary screening module, which screens out range data for calculation according to the range offset; and a coherence coefficient and interference phase calculation module, which receives the registered data and calculates the coherence coefficient and interference phase.

[0007] The present invention simplifies the complex address mapping formula caused by selecting data in both the azimuth and distance directions, has advantages in terms of real-time performance, simplicity, upgradeability and maintainability, and ensures high speed and high efficiency of InSAR data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the drawings described below are only multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 This is a schematic diagram of the algorithm implementation of the two-dimensional registration of InSAR images;

[0010] Figure 2 It is a schematic diagram of the offset calculation process of the FPGA registration method for two-dimensional InSAR images;

[0011] Figure 3It is a schematic diagram of the FPGA step-by-step screening structure of InSAR image two-dimensional registration in combination with the embodiments of this specification;

[0012] FIG4( a ) is an embodiment of the present specification, illustrating the data flow between the DDR and RAM after the primary screening of the two-dimensional InSAR image when the azimuth offset is positive;

[0013] FIG4( b ) is an embodiment of the present specification, illustrating the data flow between the DDR and RAM after the primary screening of the two-dimensional InSAR image when the azimuth offset is negative;

[0014] Figure 5 Schematic diagram of the process of realizing data alignment in RAM array buffer by azimuth offset in the secondary screening stage;

[0015] Figure 6 It is a pipeline diagram of FPGA alignment structure offset data scheduling in the data screening stage;

[0016] Figure 7 It is an implementation process of an FPGA two-dimensional registration method of InSAR images according to an embodiment of the specification;

[0017] Figure 8 It is a structural diagram of FPGA registration of two-dimensional InSAR images according to this specification. DETAILED DESCRIPTION

[0018] The scheme provided by the embodiments of this specification is described below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the invention. The described embodiments are only part of the embodiments of this specification, rather than all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0019] like Figure 1As shown in the figure, two 512×512 point images are SAR images generated after imaging the same area twice. One of the images is used as the reference image and is represented by A, which is the main image. The other image B is used as the auxiliary image. First, a control point is selected on image A, and then the target interval is set with the point as the center. Then, the search interval is selected at the corresponding position of image B based on the target interval. By finding the offset vectors in the azimuth and range directions in the search interval, and then translating the entire image according to the offset vector, the same pixel point in image A and image B coincides with the corresponding ground sample point. The offset vectors of this position are represented by X_axis (range offset) and Y_axis (azimuth offset), and the positive and negative X_axis and Y_axis are defined according to the different directions of image offset. The offsets in different directions are divided into four cases. If the offsets Y_axis and X_axis are both taken in DDR, the pseudo algorithm is used for further explanation as shown below:

[0020]

[0021] Since the offset coordinates Y_axis and X_axis act in two loops respectively, the complexity of the algorithm is O(n 2 ). More importantly, since the address of the selected data is related to the offset coordinate X_axis, the DDR address space is discontinuous, affecting storage efficiency. When only the offset Y_axis fetches data in DDR, the influence of X_axis is transferred from DDR to the back-end RAM, then the pseudo algorithm is also used to illustrate:

[0022]

[0023] In this way, the complexity of DDR reading data is reduced to O(n). We only need to filter according to Y_axis and send the data to RAM, avoiding efficiency loss caused by discontinuous address space.

[0024] Based on this, the present invention discloses an FPGA registration method for two-dimensional InSAR images. The maximum function correlation method is used to calculate the offset of the main and auxiliary images in the FPGA, and the result is stored in the DDR. The offsets in the azimuth and distance directions are screened step by step, and finally the interference phase and coherence coefficient are generated.

[0025] Specifically, Figure 2 The calculation process of the FPGA registration method of the two-dimensional InSAR image is shown:

[0026] Different from the image processing operations performed by DSP, GPU and CPU, etc. DSP, GPU and CPU process images basically in frames. InSAR image data is stored in memory, and then GPU reads the InSAR image in memory for processing. Taking the frame rate of 30 frames as an example, if DSP or GPU can complete the processing of one frame of image within 1 / 30 second, it can basically be regarded as real-time processing, while FPGA performs real-time pipeline calculation on images in lines.

[0027] First, the InSAR image is converted from a time domain image to a frequency domain image through FFT fast calculation, and the same size of primary and secondary image blocks are intercepted from two 16K×4K InSAR images. In this embodiment, the size of the image block is selected as 512×512 points, and the distance FFT (Fast Fourier Transform) operation is performed point by point in the row unit to obtain the frequency of each pixel point;

[0028] Secondly, FPGA saves the distance of the main and auxiliary images to the FFT calculation results, and the main and auxiliary images are transposed and stored in DDR1 and DDR2 respectively;

[0029] Next, the main and auxiliary images are respectively subjected to FFT operation of azimuth data, and the maximum value is calculated by conjugate multiplication to obtain the spectrum diagram of interference fringes, where the maximum value obtained is the frequency value corresponding to the fringe with the highest gray value in the spectrum;

[0030] Next, FPGA selects the azimuth data for IFFT (inverse Fourier transform), and then saves the calculation results. The main and auxiliary image data are transposed and stored in DDR1 and DDR2 respectively;

[0031] Finally, FPGA takes out the main and auxiliary image range data from DDR1 and DDR2 and performs inverse Fourier transform to complete the range calculation, obtaining the cross-correlation result of the two images. The position of the maximum value can be used to calculate the offset coordinates (X_axis, Y_axis) during registration and restore the InSAR image from the frequency domain to the time domain, thus entering the interference pattern generation process.

[0032] After obtaining the offset coordinates, the two images are aligned according to the two-dimensional offset coordinates, the interference phase is calculated after conjugate multiplication, and the coherence coefficient is obtained based on the interference phase.

[0033] In addition, the data transposition of the two-dimensional matrix operation of the InSAR image on the satellite generates a large amount of process data. The huge amount of data cannot be directly stored in the block memory inside the FPGA, but must be stored in the external DDR. However, the rate of DDR line skipping reading and writing is extremely low, so the present invention adopts a step-by-step screening registration structure such as Figure 3 shown.

[0034] FPGA filters the data in the azimuth and distance directions respectively, avoiding discontinuous DDR access and greatly improving the data processing flow rate.

[0035] 4(a) and (b) are an embodiment of the present invention, respectively illustrating the data processing of InSAR images and the data flow between DDR and RAM when the azimuth offset is positive or negative in the primary screening stage. The InSAR main and auxiliary images are respectively stored in DDR1 and DDR2 of the FPGA plug-in.

[0036] First, FPGA obtains the addresses of the main and auxiliary images in DDR according to the azimuth offset Y_axis. The different signs of Y_axis determine the different screening methods of the main and auxiliary images.

[0037] As shown in FIG4(a), when Y_axis>0 is 1, 2, and 3 respectively, FPGA determines to read out data from the corresponding 2nd, 3rd, and 4th rows in DDR1. In the illustrated embodiment, when Y_axis=1, FPGA skips row 1 in DDR1, and starts to fetch 4K-1 rows of main image data from row 2 to row 4K, and reads 4K rows of auxiliary image data from row 1 in DDR2;

[0038] As shown in Figure 4(b), when Y_axis<0 is -1, -2, and -3 respectively, the FPGA determines to read out data from the corresponding 2nd, 3rd, and 4th rows in DDR2. In the illustrated embodiment, when Y_axis=-2, the FPGA reads 4K rows of main image data starting from row 1 in DDR1, and skips rows 1 and 2 in DDR2, and fetches auxiliary image data from row 3 to row 4K, a total of 4K-2 rows.

[0039] With the help of Xilinx's own IP core AXI Interconnect, FPGA obtains data from DDR1 and DDR2 to complete data transmission with the on-chip RAM. The transmission adopts the AXI4 protocol, and the image data is converted from AXI_Master to AXI_Stream. The IP core adopts the VALID / READY handshake mechanism. When FPGA obtains the initial screening data from DDR, it needs to set the VALID signal high to indicate that the image data is ready and keep it on the message bus. Once the RAM sets the READY signal high, it means that it is ready to receive data, and data reception is started to send the data to the RAM.

[0040] As shown in Figure 4 (a) and (b), the main and auxiliary image data when Y_axis is equal to 1 and -2 respectively are converted into two AXI_Streams through AXIInterconnect and sent to the corresponding RAM arrays S1_PRE and S2_PRE respectively. Every two columns of image data are transmitted once, and each column of data includes 16K points, which fully utilizes the parallel data processing capability of FPGA. It can also be seen from the figure that the order of FPGA's access to DDR1 and DDR2 is from 1 to 4K rows in sequence, and the address is continuous and does not jump.

[0041] The process of secondary data screening is as follows Figure 5 As shown, S1_PRE and S2_PRE are self-compiled IP cores, which receive the distance data of the main and auxiliary images at the same time. As mentioned above, FPGA filters out the distance data of the main and auxiliary images in RAM according to the distance offset X_axis, and there are different filtering methods according to the different X_axis symbols.

[0042] When X_axis>0, data is selected by offsetting on the main image. When X_axis<0, data is selected by offsetting on the auxiliary image. FPGA divides these two sets of data into three parts:

[0043] The data in segment A is the data outside the selected image block and is discarded;

[0044] Segment C is used to calculate the coherence coefficient and phase;

[0045] The B segment data is reset to zero and the data length is supplemented to achieve the unchanged data length in the distance direction, making the system compatible and upgradeable.

[0046] In the FPGA registration structure of the two-dimensional InSAR image disclosed in the present invention, the calculation of the coherence coefficient and the interference phase is completed in the self-compiled IP core Cordic. After obtaining the X_axis and Y_axis offset coordinates, the interference graph process is generated. The specific process is shown in formulas (1), (2), and (3):

[0047]

[0048] Where R(u,v) is the coherence coefficient, M (I,j) 、M (i+u,j+v) They are the pixels of the main and auxiliary images respectively.

[0049]

[0050] S1(m,n) represents the conjugate of S2(m,n).

[0051]

[0052] γ represents the coherence coefficient.

[0053] When two 16384×4096 images are selected as the SAR images generated after imaging the same area twice, the coherence coefficient and interference phase are calculated using the above formula, which takes 164.7ms. Calculate the number of data points of the image:

[0054] B=D / T

[0055] Where D is the number of image data points, T is the time, and B is the image data throughput. The data throughput of this solution is obtained:

[0056] B=407.46×10 6

[0057] Because the FPGA registration structure is used, the offset calculation, data transposition, data screening, and the calculation of the coherence coefficient and the interference phase can all be realized through the pipeline. The following mainly describes the pipeline of the FPGA registration structure in the data screening stage. Figure 6 shown.

[0058] First, the FPGA calculates the offset of the two-dimensional InSAR image;

[0059] Then, according to Y_axis, the addresses of the main and auxiliary images in DDR are obtained and the azimuth data is filtered;

[0060] Secondly, while the data is being sent to the RAM array inside the FPGA, the FPGA then selects the directional data to specify the data to be read from the DDR next time.

[0061] Moreover, the data filtering at this stage only acts on the Y_axis, continuously taking data out of DDR and transferring it to RAM without address jumping.

[0062] As the data flows, the range offset is screened in RAM, and finally the coherence coefficient and interference phase are calculated based on the range and azimuth offsets. There is no need to screen the offset (X_axis, Y_axis), which is then sent to DDR and taken out to calculate the coherence coefficient and interference phase, so as to realize real-time processing of InSAR images.

[0063] like Figure 7 The embodiment of the present invention provides an FPGA registration method for a two-dimensional InSAR image, the method comprising the following steps:

[0064] Step S701: Calculate the two-dimensional offset of the InSAR image in the FPGA. First, perform two-dimensional FFT operations on the main and auxiliary images respectively to obtain the frequency value of each pixel point. According to the obtained frequency values, conjugate multiplication is performed to obtain the spectrum peak. Then, an IFFT operation is performed to obtain the cross-correlation result of the main and auxiliary images and the offset (X_axis, Y_axis) is calculated.

[0065] Step S702: FPGA filters the azimuth data in DDR according to the data address specified by Y_axis. FPGA obtains each image data from DDR and reads it line by line through Y_axis. The address is continuous and uninterrupted.

[0066] Step S703: FPGA uses AXI Interconnect to convert the filtered main and auxiliary image data from AXI-Master into two AXI-Streams respectively and send them to RAM arrays S1_PRE and S2_PRE.

[0067] Step S704: Filter out the distance data for calculation according to X_axis. According to the positive and negative values ​​of X_axis, filter out the distance data by modifying the lengths of S1_PRE and S2_PRE according to the offset, delete and fill the data, ensure that the length of the distance data remains unchanged, and filter out the distance data.

[0068] Step S705: Calculate the interference phase and the coherence coefficient. Calculate the coherence coefficient and the interference phase according to the filtered azimuth and range data.

[0069] like Figure 8 The present invention provides an FPGA registration structure for a two-dimensional InSAR image, wherein the FPGA registration structure adopts an FPGA of Xilinx xc7vx690tffg1761, with a processing clock frequency of 100MHz; a DDR of model MT41K512M16HA-125, with a working clock of 200MHz. The FPGA registration structure includes an offset calculation module, a primary screening module, an AXI-Stream conversion module, a secondary screening module, and a calculation module for a coherence coefficient and an interference phase.

[0070] The offset calculation module 801 mainly uses FPGA to complete the two-dimensional FFT operation of the image, and then obtains the maximum value through conjugate multiplication, and then performs two-dimensional inverse FFT operation to obtain the offset coordinates in the range and azimuth directions, and stores the structure in DDR.

[0071] The primary screening module 802 mainly completes the work of azimuth data screening. The address of the data to be read is stored according to the distance offset, and the azimuth data is first screened. According to the Y_axis, it is determined to start reading from a certain row of the original data storage space, and the screening work of the stage is completed in DDR.

[0072] The AXI-Stream conversion module 803 includes an AXI Interconnect IP core, which mainly completes the work of converting the initially screened offset data stored in DDR from AXI-Master to AXI-Stream.

[0073] The secondary screening module 804 includes a self-written IP core, which mainly completes the screening of distance data. The data is converted into AXI-Stream in the AXI-Stream conversion module and enters the RAM arrays S1_PRE and S2_PRE. S1_PRE and S2_PRE simultaneously receive the X_axis information of the main and auxiliary images to screen the distance data. The screening work of the stage is completed in RAM.

[0074] The coherence coefficient and interference phase calculation module 805 includes an IP core provided by an FPGA, and mainly obtains the coherence coefficient and interference phase of the image through calculation after step-by-step screening.

[0075] The FPGA step-by-step registration structure disclosed in the present invention avoids the step of calculating the distance to the effective point number, reduces the complexity of the calculation, and simplifies the complexity of back-end data storage and reading. The use of the FPGA step-by-step registration structure is also conducive to system upgrade and maintenance.

Claims

1. A FPGA registration method for two-dimensional InSAR images, comprising: FPGA uses the maximum correlation function method to obtain the range and azimuth offsets of the InSAR main and auxiliary images. FPGA obtains the addresses of the main and auxiliary images in DDR according to the azimuth offset and filters the azimuth data; FPGA uses AXI Interconnect to convert the filtered main and auxiliary image data from AXI-Master into AXI-Stream data streams and send them to RAM in parallel; FPGA selects the distance data of the main and auxiliary images in RAM according to the distance offset; After filtering the range and azimuth data, the FPGA uses the self-compiled IP core to calculate the interference phase and coherence coefficient of the main and auxiliary InSAR images.

2. The method according to claim 1, characterized in that: FPGA is used to calculate the main and auxiliary images pixel by pixel in units of lines to finally obtain the main and auxiliary image offsets.

3. The method according to claim 1, characterized in that: FPGA uses a step-by-step screening method in distance and azimuth directions to save the steps of calculating the precise address of each row and reduce the complexity of calculation.

4. The method according to claim 1, characterized in that: The FPGA external DDR adopts a step-by-step screening method to read the main and auxiliary image data from the DDR line by line according to the azimuth offset. The access address is continuous and uninterrupted during the whole line of data reading, which avoids the efficiency loss caused by address jumping and quickly realizes the registration of two-dimensional InSAR images on the satellite.

5. The method according to claim 1, characterized in that: The RAM array in the FPGA contains two self-programmed IP cores that use the characteristics of FPGA parallel data processing to simultaneously filter the main and auxiliary image distance data according to the distance offset. The offset acts on DDR and RAM respectively to achieve data screening at each level.

6. The method according to claim 1, characterized in that: The FPGA pipeline structure is used to realize real-time processing of two-dimensional InSAR images according to five stages: offset calculation, data transposition, data screening, and coherence coefficient and interference phase calculation.

7. An FPGA registration device for a two-dimensional InSAR image, comprising an offset calculation module, a primary screening data module, an AXI-Interconnect conversion module, a secondary screening module and a calculation module for a coherence coefficient and an interference phase, which are connected in sequence; wherein: The offset calculation module uses the maximum correlation function method to obtain the range and azimuth offsets of the InSAR main and auxiliary images. The primary data screening module obtains the addresses of the main and auxiliary images in the DDR according to the azimuth offset and screens the azimuth data; AXI-Interconnect conversion module converts the filtered main and auxiliary image data from AXI-Master into AXI-Stream data streams and sends them to RAM in parallel; A secondary filtering module filters the distance data of the primary and secondary images in RAM according to the distance offset; The calculation module of the coherence coefficient and the interference phase receives the above-screened range and azimuth data and uses the self-compiled IP core to calculate the coherence coefficient and the interference phase of the main and auxiliary InSAR images.

Images