WasSAR image offset estimation method and device for three-dimensional information extraction

Abstract

The application relates to a WasSAR image offset estimation method and device for three-dimensional information extraction. Two sub-aperture images with different observation position angles, namely a main image and a secondary image, are preprocessed, coarse offset position estimation is performed on the preprocessed main image and secondary image based on a block-based enhanced correlation coefficient, coarse offset position estimation of the secondary image relative to the main image is obtained, a secondary image after offset correction is correspondingly obtained, image pyramid processing is performed on the secondary image after offset correction and the preprocessed main image, a main pyramid image and a secondary pyramid image at multiple levels are respectively obtained, flow vector calculation is performed from the lowest resolution layer of the main pyramid image and the secondary pyramid image, the calculation result is transmitted to flow vector calculation of the next layer, and the flow vector obtained at the highest resolution layer is taken as the result of WasSAR image fine estimation. The method can better realize the precision and robustness of SAR image matching.

Description

Technical Field

[0001] This application relates to the field of WasSAR three-dimensional information extraction, and in particular to a WasSAR image offset estimation method and apparatus for three-dimensional information extraction. Background Technology

[0002] Wide-Angle Starring Synthetic Aperture Radar (WasSAR) is an innovative SAR observation technology that overcomes some limitations of traditional straight-line SAR modes, providing a new perspective for acquiring three-dimensional information. WasSAR systems utilize curved or circular motion trajectories on airborne platforms, achieving a focused working mode by continuously pointing a radar beam at the observation area. The advantage of WasSAR technology lies in its ability to collect information from multiple angles, thus increasing the angular dimension of observation. This provides multi-angle observation information to traditional two-dimensional SAR imaging, greatly enriching the connections between data and making it possible to extract three-dimensional information from two-dimensional images.

[0003] Similar to the principle of binocular stereo vision, the WasSAR system introduces angular dimension observation. By utilizing the correspondence between corresponding points in SAR images with similar azimuth angles, the horizontal or vertical offset of these points between images is calculated. Combined with known observation platform motion parameters, this enables accurate 3D reconstruction of the observed scene. This innovative method based on angular dimension observation can effectively improve the observation and analysis capabilities of complex scenes.

[0004] Therefore, estimating the offset of corresponding points in SAR images observed at similar azimuth angles has become crucial for WasSAR 3D information acquisition technology. However, due to the complex characteristics of SAR images, such as radiation attenuation, salt-and-pepper noise, overlay, and shadows, the task of estimating the offset of corresponding points in SAR images at similar azimuth angles becomes challenging. Summary of the Invention

[0005] Therefore, it is necessary to provide a WasSAR image offset estimation method and apparatus for three-dimensional information extraction that can achieve better accuracy and robustness in addressing the above-mentioned technical problems.

[0006] A WasSAR image offset estimation method for three-dimensional information extraction, characterized in that the method includes:

[0007] Two sub-aperture images with different viewing angles at different observation positions are acquired. The two sub-aperture images are obtained by imaging two echo data segments obtained when observing the same scene based on the WasSAR platform.

[0008] The two sub-aperture images are taken as a main image and a sub-image respectively, and the main image and the sub-image are pre-processed to make the brightness distribution of the pre-processed main image and the pre-processed sub-image consistent;

[0009] Based on the block-based enhanced correlation coefficient, coarse estimation of the offset position of the pre-processed main image and the pre-processed sub-image is performed to obtain coarse estimation of the offset position of the sub-image relative to the main image, and a sub-image after offset correction is obtained accordingly;

[0010] The sub-image after offset correction and the pre-processed main image are subjected to image pyramid processing to obtain a main pyramid image and a sub-pyramid image of multiple levels respectively;

[0011] The calculation of the optical flow vector is performed from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and the calculation result is transmitted to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is taken as the result of the fine estimation of the WasSAR image offset.

[0012] In one embodiment, the azimuth angle interval of the main image and the sub-image is greater than 10°.

[0013] In one embodiment, the pre-processing of the main image and the sub-image includes:

[0014] The main image is subjected to brightness equalization processing to obtain a brightness-equalized main image;

[0015] The brightness-equalized main image and the sub-image are subjected to histogram matching to obtain a matched main image and a matched sub-image;

[0016] The matched main image and the matched sub-image are respectively subjected to smoothing processing to obtain the pre-processed main image and the pre-processed sub-image.

[0017] In one embodiment, the histogram matching of the brightness-equalized main image and the sub-image includes:

[0018] The cumulative distribution functions of the brightness-equalized main image and the sub-image are respectively calculated;

[0019] The cumulative distribution function of the brightness-equalized main image is taken as a standard to construct a mapping function for the sub-image, and each intensity value in the sub-image is mapped to a corresponding intensity value in the brightness-equalized main image to complete the histogram matching.

[0020] In one embodiment, the coarse estimation of the offset position of the pre-processed main image and the pre-processed sub-image based on the block-based enhanced correlation coefficient includes:

[0021] The preprocessed main image and the preprocessed auxiliary image are respectively divided into multiple sub-image blocks by using a uniform division strategy, and multiple sub-image blocks corresponding to the main image and the auxiliary image are obtained respectively.

[0022] Each sub-image block in the main image is respectively matched with each sub-image block in the auxiliary image by using an enhanced correlation coefficient, and a coarse estimation result of the offset position is obtained.

[0023] In one embodiment, the uniform division strategy includes:

[0024] The preprocessed main image and the preprocessed auxiliary image are respectively divided into multiple sub-image blocks according to different division numbers, that is, for each division number, the preprocessed main image and the preprocessed auxiliary image will obtain a corresponding subset, and the subset includes a corresponding number of sub-image blocks.

[0025] The correlation between each sub-image block in each subset is calculated, and a correlation measurement matrix is obtained.

[0026] The minimum value is selected from each correlation measurement matrix, and the minimum values of each subset are compared. The division number corresponding to the minimum value is taken as the final image division number.

[0027] The preprocessed main image and the preprocessed auxiliary image are divided by using the final image division data.

[0028] In one embodiment, when the offset-corrected auxiliary image and the preprocessed main image are processed by using the image pyramid, the minimum value of the pyramid layer number is calculated by using the following formula:

[0029]

[0030] In the above formula, represents the point target offset distance, d c represents the imaging network size, δ r represents the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, and the symbol is the minimum integer not less than the value in the symbol.

[0031] The application also provides a WasSAR image offset estimation device for three-dimensional information extraction, and the device includes:

[0032] A to-be-matched image acquisition module is configured to acquire two sub-aperture images with different observation position perspectives, and the two sub-aperture images are obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform.

[0033] a preprocessing module, configured to take the two sub-aperture images as a main image and a sub-image respectively, and perform preprocessing on the main image and the sub-image, so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0034] a coarse estimation module, configured to perform coarse estimation of the offset position of the preprocessed main image and the sub-image based on the block-based enhanced correlation coefficient, to obtain coarse estimation of the offset position of the sub-image relative to the main image, and to obtain a sub-image after offset correction accordingly;

[0035] an image pyramid processing module, configured to perform image pyramid processing on the sub-image after offset correction and the preprocessed main image, to obtain a main pyramid image and a sub-pyramid image at multiple levels respectively;

[0036] a fine estimation module, configured to perform calculation of an optical flow vector from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and to pass the calculation result to the calculation of the optical flow vector of the next layer, and to take the optical flow vector obtained at the highest resolution layer as the result of fine estimation of the WasSAR image offset.

[0037] A computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0038] obtaining two sub-aperture images with different viewing angles of observation positions, the two sub-aperture images being obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform;

[0039] taking the two sub-aperture images as a main image and a sub-image respectively, and performing preprocessing on the main image and the sub-image, so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0040] performing coarse estimation of the offset position of the preprocessed main image and the sub-image based on the block-based enhanced correlation coefficient, to obtain coarse estimation of the offset position of the sub-image relative to the main image, and to obtain a sub-image after offset correction accordingly;

[0041] performing image pyramid processing on the sub-image after offset correction and the preprocessed main image, to obtain a main pyramid image and a sub-pyramid image at multiple levels respectively;

[0042] performing calculation of an optical flow vector from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and to pass the calculation result to the calculation of the optical flow vector of the next layer, and to take the optical flow vector obtained at the highest resolution layer as the result of fine estimation of the WasSAR image offset.

[0043] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the following steps:

[0044] Obtaining two sub-aperture images with different observation position perspectives, the two sub-aperture images being obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform;

[0045] Taking the two sub-aperture images as a main image and a secondary image respectively, pre-processing the main image and the secondary image so that the brightness distributions of the pre-processed main image and the secondary image are consistent;

[0046] Based on a block-based enhanced correlation coefficient, performing coarse estimation of the offset position of the pre-processed main image and the secondary image to obtain coarse estimation of the offset position of the secondary image relative to the main image, and accordingly obtaining an offset-corrected secondary image;

[0047] Performing image pyramid processing on the offset-corrected secondary image and the pre-processed main image to obtain a main pyramid image and a secondary pyramid image at multiple levels respectively;

[0048] Performing calculation of an optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and transferring the calculation result to the calculation of the optical flow vector of the next layer, and taking the optical flow vector obtained at the highest resolution layer as the result of the fine estimation of the WasSAR image offset.

[0049] The WasSAR image offset estimation method and device for three-dimensional information extraction described above, by pre-processing two sub-aperture images with different observation position perspectives, i.e., a main image and a secondary image, performing coarse estimation of the offset position of the pre-processed main image and the secondary image based on a block-based enhanced correlation coefficient to obtain coarse estimation of the offset position of the secondary image relative to the main image, and accordingly obtaining an offset-corrected secondary image, performing image pyramid processing on the offset-corrected secondary image and the pre-processed main image to obtain a main pyramid image and a secondary pyramid image at multiple levels respectively, and performing calculation of an optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and transferring the calculation result to the calculation of the optical flow vector of the next layer, and taking the optical flow vector obtained at the highest resolution layer as the result of the fine estimation of the WasSAR image offset. The method can better realize the precision and robustness of SAR image matching. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A flowchart of the WasSAR image offset estimation method for three-dimensional information extraction in one embodiment;

[0051] Figure 2 A schematic diagram of the geometric structure of the WasSAR imaging system in one embodiment;

[0052] Figure 3 This is a schematic diagram of the image preprocessing process in one embodiment;

[0053] Figure 4 This is a schematic diagram of image segmentation in one embodiment;

[0054] Figure 5 This is a schematic diagram illustrating the implementation process of a WasSAR image offset estimation method for three-dimensional information extraction in one embodiment.

[0055] Figure 6 An optical photograph of a scene observed in an experiment;

[0056] Figure 7 This section presents the image preprocessing and histogram distribution results from an experiment, where... Figure 7 (a) is the original main image. Figure 7 (b) is the histogram of the original main image. Figure 7 (c) is the preprocessed main image. Figure 7 (d) is the histogram of the preprocessed main image. Figure 7 (e) is the preprocessed sub-image. Figure 7 f is the histogram of the preprocessed sub-image;

[0057] Figure 8 This is a schematic diagram of the offset estimation results in an experiment (pseudocolor contrast display), where, Figure 8 (a) The rough estimate result, Figure 8 (b) The precise estimation result;

[0058] Figure 9 This is a schematic diagram illustrating the measurement results of a similarity index in an experiment, where... Figure 9 (a) is the processed CC result. Figure 9 (b) is the CC result before processing. Figure 9 (c) is the processed SSIM result. Figure 9 (d) represents the SSIM result before processing. Figure 9 (e) represents the CC frequency distribution results before and after processing. Figure 9 (f) represents the frequency distribution before and after SSIM processing;

[0059] Figure 10 This is a schematic diagram illustrating the offset estimation results of different methods in an experiment, where... Figure 10 (a) is the initial image. Figure 10 (b) is the MS method. Figure 10 (c) is the MI method. Figure 10 (d) is KAZE. Figure 10(e) is Bspline, Figure 10 (f) is TPS, Figure 10 (g) is original Demons, Figure 10 (h) is the method described herein;

[0060] Figure 11 is a schematic diagram of the positional relationship between the sub-block image and the original image in an experiment;

[0061] Figure 12 is a schematic diagram of the three-dimensional information extraction result in an experiment, Figure 12 (a) is the optical mapping result, Figure 12 (b) is the 5-level pyramid extraction result, Figure 12 (c) is the 1-level pyramid extraction result, Figure 12 (d) is the 10-level pyramid extraction result;

[0062] Figure 13 is a structural block diagram of the WasSAR image offset estimation device for three-dimensional information extraction in an embodiment;

[0063] Figure 14 is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0064] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0065] Currently, the SAR image offset estimation for similar azimuth angles mainly includes two methods: feature-based method and region-based method.

[0066] Specifically, the feature-based method estimates by extracting feature points (such as corner points, spots, etc.) of the image. This method is more reliable when processing SAR images, because the feature points are usually not affected by radiation attenuation and salt and pepper noise. The feature-based method usually includes two steps: feature point extraction and feature point matching. Feature point extraction refers to extracting feature points such as corner points, spots, etc. from the image, and these feature points usually have strong robustness and distinguishability. Feature point matching refers to matching the feature points in two images to find the corresponding relationship between the two images.

[0067] The ratio of coarse error matching on optical images based on the feature matching method of the descriptor is generally more than 50%, while on SAR images it can reach 90% or even higher. However, in mountainous areas and other areas where features are not obvious, the number of feature points in SAR images is small, so the registration method based on feature points is not applicable. Although setting a more stringent threshold for the descriptor can improve the correct matching ratio, it will also result in sacrificing a large number of correct matches, and usually only a small number of registration feature points can be obtained. Therefore, although it has applications in the fields of image fusion and change detection, this method cannot meet the needs of full-scene three-dimensional information acquisition, especially in complex terrain areas such as mountains.

[0068] Specifically, the region-based SAR image registration method evaluates the matching degree between images by calculating the similarity measure between images, thereby achieving image registration. This method can effectively utilize the texture information in the image without the need to extract feature points, thus to some extent overcoming the influence of factors such as salt and pepper noise and radiation attenuation on feature point extraction. Commonly used similarity measure functions include cross-correlation information, mutual information, mean square error, etc. Cross-correlation information is a commonly used similarity measure function, which is a measure of the correlation between two signals. Mutual information is a measure of the difference between the joint probability distribution and the marginal probability distribution of two random variables. The mean square error is the average of the square difference between two signals, which is the most commonly used error measure.

[0069] However, the SAR image registration method based on this type of similarity measure has some defects. First, this method needs to calculate the entire image, which is computationally intensive, time-consuming, and inefficient. Second, due to the complexity of SAR images, the selected similarity function may have multiple local maxima, which will result in inaccurate registration results.

[0070] The deformation of the WasSAR image targeted in this application is essentially a non-rigid deformation due to the differences in observation angles and the characteristics of SAR images, which includes bending, twisting, etc., making the registration process more complex. The registration method relying solely on similarity measure cannot meet the demand for fast and high-precision registration.

[0071] Lucas-Kanade optical flow algorithm is an advanced image registration technique that uses a global coordinate transformation model to handle the non-rigid motion of objects in image sequences. The core of this method lies in capturing the changes of pixels in the time domain and the intrinsic relationship between images, thereby identifying and calculating the motion trajectory of the object in consecutive frames. Unlike traditional rigid registration, Lucas-Kanade algorithm accurately calculates the relative displacement of each pixel between two images by meticulously analyzing the gray value differences and gradient information between the reference image and the floating image. This calculation not only reveals the dynamic changes of the object, but also provides key information for non-rigid registration between images.

[0072] While the optical flow algorithm is a powerful tool in the field of optical image registration, it faces some challenges and limitations when applied to processing SAR images acquired at different azimuth angles. First, unlike optical images, the brightness of SAR images reflects the radar backscatter intensity, which is closely related to the scattering characteristics of the object, as well as factors such as the roughness of the target surface, geometric structure, and terrain. Second, under different observation angles, the scattering center of the image will change, affecting the performance of the optical flow algorithm. In addition, the optical flow algorithm usually assumes that the motion in the image is smooth. When encountering large deformations, such as severe positional shifts or nonlinear deformations, this smoothness assumption may no longer hold, leading to a decline in algorithm performance. In the case of large deformations, the optical flow algorithm is prone to local minima, i.e., the algorithm may find a local optimal solution rather than a global optimal solution, thereby affecting the registration accuracy.

[0073] In this application, as shown in the figure, a WasSAR image offset estimation method for three-dimensional information extraction is provided, which can improve the applicability and effect of the optical flow algorithm in SAR images, specifically including the following steps:

[0074] Step S100, two sub-aperture images with different observation position angles are obtained, which are obtained by imaging two pieces of echo data obtained by observing the same scene based on the WasSAR platform.

[0075] Step S110, the two sub-aperture images are respectively taken as the main image and the auxiliary image, and the main image and the auxiliary image are preprocessed to make the brightness distribution of the preprocessed main image and auxiliary image consistent.

[0076] Step S120, based on the block-based enhanced correlation coefficient, the offset position of the preprocessed main image and auxiliary image is coarsely estimated, the offset position of the auxiliary image relative to the main image is coarsely estimated, and the offset corrected auxiliary image is obtained.

[0077] Step S130: Perform image pyramid processing on the offset-corrected sub-image and the preprocessed main image to obtain multi-level main pyramid images and sub-pyramid images respectively.

[0078] Step S140: Calculate the optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and pass the calculation result to the optical flow vector calculation of the next layer. Use the optical flow vector obtained from the highest resolution layer as the result of the fine estimation of WasSAR image offset.

[0079] In this embodiment, firstly, image preprocessing is performed to reduce image noise, balance the brightness distribution of the primary and secondary images, and ensure their consistency in grayscale distribution, thus laying a good foundation for subsequent registration work. Secondly, a block-based processing method based on enhanced correlation coefficient (ECC) is used for coarse registration. This method estimates and reduces the offset range of large deformation regions by calculating the correlation between image blocks. Through ECC block segmentation, the relative positions between images can be quickly located over a large area, providing an initial estimate for the fine registration stage. Finally, an improved optical flow algorithm is used for fine registration of the images. This improved optical flow algorithm is optimized for the characteristics of SAR images, enabling more accurate capture and estimation of small offsets in the image, achieving high-precision offset estimation.

[0080] This method effectively overcomes the problems that SAR images may encounter in traditional optical flow algorithms, such as large deformation, inconsistent brightness, and differences in scattering characteristics, by integrating image preprocessing, coarse registration, and improved optical flow fine registration. This results in higher accuracy and robustness in SAR image registration tasks.

[0081] In step S100, the two sub-aperture images are obtained from, as follows: Figure 2 The observations were taken of the same scene under the geometry of the WasSAR imaging system shown. First, in Figure 2 In this system, a spatial rectangular coordinate system OXYZ is established, where the OXY plane is defined as the imaging plane, and the OZ axis represents the altitude axis. The airborne radar platform is located on a plane with a height of H above the horizontal plane and moves around the vertical axis OZ in a curved motion with a radius of R, maintaining a tangential velocity of V. During the motion, the radar transmits signals always aligned with the center point O of the imaging area, thus ensuring that a circular imaging area with radius R centered at O ​​is formed after the aircraft completes one revolution. At any given moment, the radar's position can be represented by (Rcosθ, Rsinθ, H) in the spherical coordinate system, where θ is the angle between the radar's current position and the positive X-axis. For a point target P located at the observation position, its position in the coordinate system is (x, y, z). According to geometric relationships, the instantaneous distance of the radar to the point target P can be expressed as formula (1).

[0082]

[0083] Let the imaging plane be the z=0 plane. If a point target P is imaged using backpropagation (BP), then its point spread function on the imaging plane is:

[0084]

[0085] In formula (2), R xy Performing a Fourier transform on the distances from the radar platform to each pixel on the imaging plane yields the following result:

[0086]

[0087] Let Φ = -2KR lp +2KR xy -K x xK y Given y, find the first-order partial derivatives of Φ with respect to x and y respectively. The corresponding instantaneous frequencies are:

[0088]

[0089] If we set the first-order partial derivative to 0, then formula (3) can be expressed as follows:

[0090] G(K x ,K y )=exp{jΦ(x(K x ),y(K y ))} (5)

[0091] Performing an inverse Fourier transform on it yields:

[0092]

[0093] in, make

[0094]

[0095] Find Ψ(K) in formula (7) x ,K y Regarding K x K y The first-order partial derivatives yield:

[0096]

[0097] Setting the first-order partial derivative to 0, we get:

[0098] (xx p ) 2 +(yy p ) 2= (z p tan θ 2 (9)

[0099] As can be seen from equation (9), when there is a height difference between the target height plane and the imaging plane, the target will be defocused when WasSAR focusing imaging is performed, resulting in geometric distortion of the image, and a circle with (x p , y p ) as the center and |z p · tan θ| as the radius (the absolute value is taken because z is negative when the target height is lower than the imaging height) will be formed on the imaging plane.

[0100] WasSAR imaging technology is an imaging method that relies on target height information. In the case where the exact position of the target is unknown, the three-dimensional spatial information of the target can be derived by analyzing the offset information of the target in the WasSAR image sequence. The key to this process is to accurately estimate the offset of the same point in different azimuth WasSAR images.

[0101] In this embodiment, when performing three-dimensional information reconstruction based on view angle offset, it is generally required that the interval of the azimuth angle reach at least 10° in order to ensure the accuracy of the extracted height data. That is, the interval of the azimuth angle of the main image and the secondary image is greater than 10°.

[0102] Considering that SAR images often exhibit obvious differences in brightness and contrast due to their unique imaging mechanism, which are mainly caused by different scattering intensities of the target. Before performing SAR image registration, the main and secondary SAR images need to be preprocessed to make their brightness distributions consistent.

[0103] In step S110, as shown in FIG. 1, the preprocessing of the main image A and the secondary image B includes: performing brightness equalization processing on the main image A to obtain a brightness-equalized main image A, performing histogram matching on the brightness-equalized main image A and the secondary image B to obtain a matched main image A and a matched secondary image B, and performing smoothing processing on the matched main image A and the matched secondary image B respectively to obtain a preprocessed main image A and a preprocessed secondary image B. Figure 3 In one embodiment, the histogram matching on the brightness-equalized main image A and the secondary image B includes: calculating the cumulative distribution functions of the brightness-equalized main image A and the secondary image B respectively, taking the cumulative distribution function of the brightness-equalized main image A as a standard to construct a mapping function for the secondary image B, and mapping each intensity value in the secondary image B to a corresponding intensity value in the brightness-equalized main image A to complete the histogram matching.

[0104]

[0105] ​Specifically, first, the main image A is subjected to brightness equalization processing. In this embodiment, the pixel histogram distribution is used as a basis for histogram equalization processing, and the pixel values are redistributed to make the image brightness distribution more uniform, thereby improving the contrast and visual quality of the image. The core idea of the adaptive histogram equalization technique is to divide the image into multiple small regions and independently perform histogram equalization processing in each small region. The local histogram H(i, region) is shown in equation (10):

[0106]

[0107] In equation (10), δ is the Kronecker function, I(x, y) is the pixel value of the original image at (x, y), and δ is 1 when I(x, y) = i, otherwise 0.

[0108] Based on the local histogram, the cumulative distribution function (CDF) of each small region is calculated to reflect the cumulative probability distribution of the pixel value. The cumulative distribution function C(i, region) is shown in equation (11):

[0109]

[0110] Using the cumulative distribution function, an equalization mapping function T(i, region) is constructed, which maps the original pixel value to a new equalization value, as shown in equation (12). The equalized pixel I'(x, y) is shown in equation (13):

[0111]

[0112] I′ A (x,y)=T(I A (x,y),region) (13)

[0113] The equalization results of all small regions are combined to obtain the final adaptive histogram equalization image.

[0114] Next, for the equalized image A and the to-be-matched image B, their cumulative distribution functions are calculated, as shown in equation (14):

[0115]

[0116] In equation (14), H A (i) and H B (i) are the histogram values of images A and B, respectively, N is the total number of pixels of the image, and k is the pixel intensity value.

[0117] Next, using the CDF of image A as a standard, a mapping function is constructed for image B to map each pixel intensity value in B to the corresponding intensity value in A, as shown in equation (15):

[0118]

[0119] In equation (15), I B is the pixel value of the original image B, I′ B is the mapped pixel value, and M is the mapping function that maps the gray level i in image B to the gray level j in image A that has the closest CDF value. The matched image B (I′ B ) will have a similar brightness distribution and contrast to image A.

[0120] Finally, the histogram is smoothed to reduce the effect of noise. A Gaussian filter is used in this step, as shown in equation (16):

[0121]

[0122] In equation (16), H′ A (i) and H′ B (i) are the smoothed histogram values, and G is the Gaussian kernel function. Through the above image preprocessing, we can effectively adjust the brightness and contrast of two SAR images to achieve consistency in both visual and statistical aspects. This consistency is crucial for subsequent image registration operations and can effectively improve the robustness and accuracy of the registration algorithm.

[0123] Next, in step S120, the pre-processed primary and secondary images are subjected to rigid registration using the Enhanced Correlation Coefficient (ECC). The Enhanced Correlation Coefficient (ECC) is a measurement method used for image registration, which provides a means to evaluate the similarity between two images, especially when performing image fusion or registration tasks. The core of the ECC method lies in quantifying the correlation between two image regions and using this correlation to estimate the best alignment between them. The ECC registration method is mainly used for rigid or affine transformation image registration, including translation, rotation, and scaling. For non-rigid deformation, i.e., those involving local deformation, bending, or other complex geometric transformations, traditional ECC registration may not be sufficient to achieve high-precision registration. This method divides the image into multiple sub-blocks and applies ECC registration to each image sub-block independently to adapt to local changes and achieve coarse registration of the image.

[0124] In the embodiment, the coarse offset estimation of the preprocessed primary image and the preprocessed secondary image based on the block-based enhanced correlation coefficient comprises: performing block processing on the preprocessed primary image and the preprocessed secondary image respectively by using a uniform block strategy, to obtain a plurality of sub-image blocks corresponding to the primary image and the secondary image respectively; and performing enhanced correlation coefficient registration on each sub-image block in the primary image and each sub-image block in the secondary image respectively, to obtain a coarse offset estimation result.

[0125] As shown in the figure, when performing coarse registration, the two images are first subjected to block processing. In the method, a uniform block strategy is used. In order to avoid discontinuity at the boundaries of the blocks, overlapping block processing can be used, that is, each sub-block has a partial area overlapping with other sub-blocks. Figure 4

[0126] Further, the uniform block strategy comprises: dividing the preprocessed primary image and the preprocessed secondary image into a plurality of sub-image blocks according to different division numbers respectively, that is, the preprocessed primary image and the preprocessed secondary image will obtain a corresponding subset for each division number, the subset includes a corresponding number of sub-image blocks, the correlation between each sub-image block in each subset is calculated, and a correlation measurement matrix is obtained, the minimum value is selected from each correlation measurement matrix, the minimum values of each subset are compared, and the division number corresponding to the minimum value is taken as the final image block number, finally, the preprocessed primary image and the preprocessed secondary image are subjected to block processing according to the final image block data.

[0127] Specifically, the strategy of image block processing is as follows:

[0128] (1) Uniform block: the primary image and the secondary image are equally divided into m x m small blocks, each block contains a subset of the image (m = 1, 2, 3, …);

[0129] (2) Correlation calculation: analyze the sub-image blocks, calculate the correlation between each pair of sub-image blocks, and obtain m x m correlation measurements;

[0130] (3) Minimum correlation value determination: identify the minimum value in the obtained m x m correlation measurements, and record it as Corr Min,m ;

[0131] (4) Optimal sub-block size determination: by comparing Corr Min,m corresponding to different m values, select the m that makes Corr Min,m minimum as the final sub-image block division parameter.

[0132] In the embodiment, the minimum correlation value Corr Min,m ​As a criterion, because when the correlation between the sub-image pairs is low, it indicates that the matching degree between them is poor, which usually means that these areas need more accurate registration. Therefore, focus on the sub-image pairs with the lowest correlation to achieve more effective registration.

[0133] It should be noted that selecting a too large m value can lead to a significant increase in the amount of calculation and can increase the risk of sub-image block mismatch, which is not in line with the goal of coarse registration. Therefore, preferably, the value of m will be controlled within 10 to balance the registration effect and calculation efficiency.

[0134] Then, the ECC registration algorithm is used to register each image sub-block, and ECC is usually defined as:

[0135]

[0136] In formula (17), i r and i w are the zero-mean versions of the pixel intensity vectors of the reference image (main image) and the deformed image (secondary image), respectively, and p is the vector of transformation parameters. The core purpose of the registration algorithm is to maximize the enhanced correlation coefficient ρ(p). Wherein, is a constant, indicating the normalized zero-mean reference vector. Gradient descent method is used in this paper to solve the maximum problem of ρ(p).

[0137] In this method, the ECC algorithm is used because it is robust to overall brightness changes of the image, mainly focuses on the geometric changes in the image, and has the ability to correct image deformation. However, when facing images with complex geometric distortion or large size, using a single homography matrix for global solution may not be enough to accurately correct the geometric mismatch between images caused by different degrees of distortion. In this case, the ECC algorithm may have difficulty in efficiently achieving accurate registration of local images. By block processing, the ECC algorithm is independently applied to each image block for registration, and the registered image blocks are finally combined, which can effectively reduce the calculation difficulty brought by large image size or complex geometric distortion, and provide accurate initial estimation for the subsequent fine registration stage.

[0138] Optical flow method image offset estimation is a method of using optical flow technology to determine the relative position change between two images. Applying the optical flow method to WasSAR images needs to follow the following three assumptions:

[0139] Gray constancy: The optical flow algorithm is based on the basic assumption that if the pixel points appear in the overlapping area in the image sequence, their gray values remain unchanged between different images.

[0140] Small motion assumption: When the motion amplitude of the object is small, the displacement of the pixel points between the two images will not be too large.

[0141] Spatial coherence: The projections of adjacent regions on the 2D image should also be close. That is, neighboring pixels have similar motion speed. Spatial coherence allows us to construct a set of linear equations using a set of neighboring pixels, and solve these equations by least squares to obtain the final velocity estimate.

[0142] After the image pre-processing and coarse registration in steps S110 and S120, the offset-corrected sub-image and the pre-processed main image satisfy the three assumptions mentioned above.

[0143] To ensure that the WasSAR image satisfies the basic assumption of the optical flow algorithm. Assume that a point target on the image is located at I(x, y, t) and I(x + dx, y + dy, t + dt) in the adjacent two sub-aperture images, respectively. According to the gray constant assumption, we can get:

[0144] I(x, y, t) = I(x + dx, y + dy, t + dt) (18)

[0145] According to the small motion assumption, the first-order Taylor expansion of the right side of formula (18) is obtained:

[0146]

[0147] According to formula (18) and formula (19), we can get:

[0148]

[0149] When dt is small enough and approaches to 0:

[0150]

[0151] Let u = dx / dt, v = dy / dt, formula (21) can be rewritten as:

[0152] -I t = I x u + I y v (22)

[0153] Formula (22) is the basic optical flow constraint equation.

[0154] When performing three-dimensional information acquisition based on perspective shift, in order to ensure the accuracy of the extracted height data, it is usually required that the azimuth angle interval reaches at least 10°. However, for mountainous scenes with large terrain undulations, due to the large parallax under different perspectives, even if the ECC method is used for preliminary coarse registration processing, there may still be significant positional deviations. These deviations will lead to geometric inconsistency between images, thereby violating the small motion premise assumption of the optical flow algorithm.

[0155] In step S130, the image pyramid technique is introduced to reduce the displacement offset between images. The image pyramid is a multi-scale image representation method, which constructs a pyramid structure by reducing the image resolution step by step, and the image size of each level is halved, thereby reducing the displacement amount between each layer of images. On a lower resolution level, the same physical displacement causes less change at the pixel level, which helps to meet the small motion assumption requirement of the optical flow algorithm.

[0156] For image I, the next layer of the image is constructed by Gaussian filtering and downsampling, and the formula of Gaussian filtering is:

[0157]

[0158] In formula (23), G(x, y) is the Gaussian kernel function, and σ is the standard deviation.

[0159] Downsampling usually uses the nearest even point for downsampling, such as formula:

[0160] I n (x, y) = I n-1 (2x, 2y) (24)

[0161] In formula (24), I n (x, y) is the image of the nth layer of the pyramid after downsampling, I n-1 is the (n-1)th layer image, and I0 is the original image. If the resolution of the original image is M x N, then the resolution of the kth layer is

[0162] When passing the optical flow from one layer of the pyramid to the next layer, the update of the optical flow vector is shown in formula:

[0163]

[0164] In formula (25), v n and u n are the optical flow vectors of the nth layer after downsampling, v n-1 and u n-1 are the optical flow vectors of the (n-1)th layer.

[0165] However, the more pyramid levels are not necessarily better, the more pyramid levels, the more serious the loss of image details, which will also affect the performance of the algorithm, and the number of pyramid levels should be the lowest number that meets the small motion assumption premise of the image. Small motion assumption usually means that the displacement on the image does not exceed 5-10 pixels. But this is only a rough empirical value, and the specific situation needs to be adjusted according to the actual application.

[0166] In the WasSAR three-dimensional information extraction, the relationship between the target offset and the target is shown in the formula:

[0167]

[0168] In the formula (26), X TA and X TB are the horizontal coordinates of the point target T in the sub-aperture images A and B, respectively, y TA and y TB are the corresponding vertical coordinates, h T is the height of the point target T, and are the azimuth angles of the centers of the two sub-aperture observation positions. θ A and θ B are the corresponding incident angles. Then the relationship between the number of pyramid layers and the point target offset is shown in the formula:

[0169]

[0170] In the formula (27), d represents the point target offset distance, d c is the imaging grid size, the number of image pyramid layers is k, and δ r is the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm. Then the minimum value k min of the number of pyramid layers is:

[0171]

[0172] In the formula (28), d represents the point target offset distance, d c represents the imaging network size, δ r represents the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, and the symbol is the minimum integer not less than the value in the symbol.

[0173] In this embodiment, the data processing flow of the WasSAR image offset estimation method for three-dimensional information extraction is shown in Figure 5 .

[0174] In order to comprehensively verify the effectiveness and practicability of the method (the present method) proposed in this paper in actual application, the following part will introduce the specific application of the algorithm in processing the actual measured data of the airborne Ku band synthetic aperture radar (WasSAR). These measured data are collected by the airborne Ku band WasSAR system, ensuring the reliability of the data and the scientificity of the experiment.

[0175] In this experiment, the selected observation scene is a typical mountainous area, which is very suitable for testing the adaptability and accuracy of the algorithm to complex terrain. In order to more intuitively show the observation scene, the optical photo of the scene is provided as shown in Figure 6 .

[0176] First, two SAR images with different azimuth angles are selected, and the azimuth angles of the two SAR images are separated by 15°, and the main image is shown in Figure 7 (a). According to the image preprocessing scheme of the method, the main image after brightness equalization processing is shown in Figure 7 (c). The histogram of the secondary image is registered, and the histogram filtering processing of the main and secondary images is performed, and the preprocessed image is shown in Figure 7 (e).

[0177] Then, the offset estimation operation is performed on the preprocessed image, and the experimental results of the coarse registration are shown in Figure 8 (a). It can be seen that for the area with small elevation fluctuation (farmland, etc.), since the deformation degree of this part is small, a certain degree of correction can be achieved through ECC coarse registration, while for the mountainous area with large elevation fluctuation, the coarse registration can only reduce the deformation degree, and cannot achieve effective correction.

[0178] In the final stage of the method in this paper, the preliminary results of image registration are further refined to accurately estimate the offset of each pixel point, and the number of Gaussian pyramid layers is determined to be 5 according to formula (28). As shown in Figure 9 (b), the optical flow algorithm is used to successfully estimate the motion vector of all pixel points in the whole image. This estimation not only improves the accuracy of registration, but also enhances the adaptability of the algorithm to complex non-rigid transformation.

[0179] Further, the similarity information before and after image processing is carefully compared and analyzed. Specifically, the correlation (CC) and structural similarity (SSIM) of the image before and after processing are evaluated to quantitatively measure the processing effect. As shown in Figure 9 .

[0180] By comparative analysis, it is obvious to observe the significant improvement in correlation and structural similarity after image processing. It should be noted that in the correlation map, there is a region where the correlation is zero at the edge, which indicates that the edge region of the image may have lost information after offset correction. In addition, the frequency distribution curve of the similarity information shows that the frequency distribution of the image after offset correction is more concentrated, and the concentration degree is enhanced. Specifically, the correlation index of the processed image reaches 0.9587, and the structural similarity index (SSIM) is improved to 0.9319. Both values are significantly higher than the corresponding indicators of the original image, further confirming the effectiveness of the processing effect.

[0181] In order to systematically evaluate the effectiveness of existing image offset correction algorithms, a series of comparative experiments were conducted. The algorithms involved in the experiment include: registration strategy based on gray intensity information, which is subdivided into affine transformation, rigid transformation, similarity-based transformation and translation transformation; advanced registration technology using KAZE features for scale-invariant feature transformation; non-rigid registration method using thin plate splines (TPS); elastic registration technology based on B-spline; and widely recognized classic Demons algorithm.

[0182] As shown in Figure 10 , the registration effect of each algorithm is intuitively displayed through comparative experiments.

[0183] In order to comprehensively evaluate the effect of image correction, a multi-scale analysis method is used, not only to comprehensively evaluate the whole image, but also to pay special attention to four areas of significant change in the image. In the selection of evaluation indicators, a series of widely recognized measurement standards including SSD (Sum of Squared Differences), correlation, mutual information, PSNR (Peak Signal-to-Noise Ratio), structural similarity (Structural Similarity Index, SSIM) and RMSE (Root Mean Square Error) are used. In addition, in order to further improve the accuracy of evaluation, advanced image alignment quality evaluation methods are introduced, in order to provide more in-depth and comprehensive evaluation of the image registration results.

[0184] Whole image comparison results: Different correction methods are used for the whole image, and a series of widely recognized metrics are used to evaluate the performance, and the results are summarized in Table 1. After further analyzing the data in the table, it can be clearly observed that the algorithm proposed in this method has superior performance in all performance evaluation indicators. Specifically, the correlation coefficient (CC) achieves a significant improvement of 25-33%, the mutual information (MI) index increases by more than 80%, and the structural similarity index (SSIM) increases by more than 50%. In addition, the peak signal-to-noise ratio (PSNR) also achieves a robust improvement of 13-15%. In terms of error metrics, the root mean square error (RMSE) is reduced by more than 75% of the existing algorithm, and the sum of absolute errors (SSD) is only 50-60% of the existing algorithm. In terms of algorithm efficiency, the calculation time of the algorithm in this paper is significantly shortened, which is generally better than most existing algorithms, and the time consumption is only 20%-50% of the existing algorithm.

[0185] Table 1 Performance evaluation results of different methods for whole image offset estimation

[0186]

[0187] Sub-block image comparison results: The effectiveness of local image correction is further discussed, and four regions with significant offset and rich features are particularly focused on. As shown in Figure 11 , the position relationship between these sub-block images and the original image is clearly shown. In order to improve efficiency, four excellent existing algorithms are selected for performance evaluation, and their correction results are compared with the algorithm proposed in this paper, and the evaluation results are listed in Table 2.

[0188] Table 2 Performance evaluation results of different methods for sub-block image offset estimation

[0189]

[0190] The comparative analysis shows that the proposed method is significantly better than other existing algorithms in each evaluation indicator, which is consistent with the correction results of the whole image. Considering the large degree of offset in the selected sub-block regions, it means that more precise adjustment and more complex calculation are needed in the correction process, which to some extent increases the difficulty of correction. Therefore, the relative deficiency of sub-block image evaluation indicators also reflects the challenge of the correction algorithm in dealing with complex scenes.

[0191] Through the analysis of the experimental results, it is fully proved that the proposed algorithm has significant advantages in the accuracy and reliability of image offset estimation. Whether it is for the correction of the whole image or the local offset significant region, the proposed algorithm has shown excellent performance, which verifies the effectiveness and innovation of the algorithm.

[0192] Further, the focus is on the application field of three-dimensional information extraction. Specifically, the image offset information obtained by the proposed algorithm is used to explore its performance in three-dimensional information extraction and analysis. This step is crucial for verifying the applicability and robustness of the algorithm in handling higher dimensional data. By comprehensively evaluating the offset values of the three-dimensional information extraction results, it is expected to fully understand the potential of the proposed algorithm in practical applications and its impact on three-dimensional modeling and visualization.

[0193] To verify the practicability of the algorithm, it is applied to the task of three-dimensional information extraction, and the three-dimensional information extraction results are shown in Figure 12 . By analyzing the errors of three-dimensional information extraction, it is found that for traditional optical flow algorithms, it is often difficult to achieve accurate offset estimation due to excessive target offset, resulting in large errors in three-dimensional information reconstruction. On the other hand, if the number of pyramid layers is set too high, important detail information in the target scene may be lost, although the performance may only show a slight decline in evaluation indicators, but there may be a large deviation in the accuracy of three-dimensional information extraction. However, through the method in this paper, the number of sub-aperture division layers of the pyramid optical flow algorithm is optimized after image preprocessing and coarse registration, thereby realizing high-precision three-dimensional information extraction of the whole scene.

[0194] Based on the above detailed experimental analysis and targeted application verification, the experiments fully confirm the significant practicality and efficiency of the method in the application of three-dimensional information extraction of the observed scene. The experimental data and application results not only verify the theoretical advantages of the algorithm, but also demonstrate its potential in solving practical problems. In addition, through comparison with existing technologies, the outstanding performance of the proposed algorithm in accuracy, efficiency and stability further highlights its application value in the field of three-dimensional information extraction.

[0195] In the above WasSAR image offset estimation method for three-dimensional information extraction, an innovative multi-view image offset estimation method is proposed to address the key issues of WasSAR imaging technology in three-dimensional information extraction. This method realizes accurate estimation of image offset through brightness equalization preprocessing, ECC block coarse estimation, and improved optical flow algorithm fine estimation, significantly improving the accuracy and reliability of WasSAR image offset information acquisition. Through the use of mountain scene measured data, the method is comprehensively experimentally verified. The experimental results and the current mainstream of a variety of offset estimation methods are compared and analyzed in detail, thus fully demonstrating the significant advantages of the method in multiple key performance indicators. In addition, the performance of the proposed method in three-dimensional information extraction applications is comprehensively analyzed, and the experimental data and analysis results clearly confirm the high efficiency and strong practical value of the method in practical applications

[0196] It should be understood that, althoughFigure 1 The steps in the flowchart are shown in sequence according to the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order limitation on the execution of the steps, and the steps can be executed in other orders. Moreover, Figure 1 At least some of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the sub-steps or stages is not necessarily sequential, but can be alternated or interleaved with at least some of the other steps or sub-steps or stages of other steps.

[0197] In one embodiment, as shown in Figure 13 a WasSAR image offset estimation device for three-dimensional information extraction is provided, comprising: a to-be-matched image acquisition module 200, a preprocessing module 210, a coarse matching module 220, an image pyramid processing module 230, and an offset fine estimation module 240, wherein:

[0198] The to-be-matched image acquisition module 200 is configured to acquire two sub-aperture images with different viewing angles at an observation position, the two sub-aperture images being obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform;

[0199] The preprocessing module 210 is configured to take the two sub-aperture images as a main image and a secondary image respectively, and pre-process the main image and the secondary image to make the brightness distributions of the pre-processed main image and the secondary image consistent;

[0200] The coarse matching module 220 is configured to perform coarse offset estimation on the pre-processed main image and the secondary image based on a block-based enhanced correlation coefficient, to obtain coarse offset estimation of the secondary image relative to the main image, and to obtain a secondary image corrected for offset accordingly;

[0201] The image pyramid processing module 230 is configured to perform image pyramid processing on the secondary image corrected for offset and the pre-processed main image, to obtain a main pyramid image and a secondary pyramid image at multiple levels respectively;

[0202] The offset fine estimation module 240 is configured to perform calculation of an optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and to pass the calculation result to the calculation of the optical flow vector at the next layer, and to take the optical flow vector obtained at the highest resolution layer as a result of WasSAR image offset fine estimation.

[0203] The specific limitations of the WasSAR image offset estimation device for three-dimensional information extraction can refer to the limitations of the WasSAR image offset estimation method for three-dimensional information extraction described above, and will not be repeated here. Each module in the above WasSAR image offset estimation device for three-dimensional information extraction can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0204] In one embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram of the computer device can be as shown in Figure 14 The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with external terminals through network connections. The computer program is executed by the processor to implement a WasSAR image offset estimation method for three-dimensional information extraction. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or can be a key, trackball, or touchpad provided on the shell of the computer device, or can be an external keyboard, touchpad, or mouse, etc.

[0205] Those skilled in the art can understand that Figure 14 the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0206] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0207] Obtaining two sub-aperture images with different viewing angles at the observation position, the two sub-aperture images being obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform;

[0208] two sub-aperture images are taken as a main image and a sub-image respectively, the main image and the sub-image are pre-processed so that the brightness distribution of the pre-processed main image and the sub-image is consistent;

[0209] a coarse estimation of the offset position of the pre-processed main image and the sub-image is performed based on a block-based enhanced correlation coefficient, a coarse estimation of the offset position of the sub-image relative to the main image is obtained, and a sub-image after offset correction is correspondingly obtained;

[0210] the sub-image after offset correction and the pre-processed main image are subjected to image pyramid processing, and a main pyramid image and a sub-pyramid image of multiple levels are respectively obtained;

[0211] the calculation of the optical flow vector is performed from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and the calculation result is transmitted to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is taken as the result of the fine estimation of the WasSAR image offset.

[0212] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the following steps:

[0213] two sub-aperture images with different observation position perspectives are obtained, the two sub-aperture images are obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform;

[0214] two sub-aperture images are taken as a main image and a sub-image respectively, the main image and the sub-image are pre-processed so that the brightness distribution of the pre-processed main image and the sub-image is consistent;

[0215] a coarse estimation of the offset position of the pre-processed main image and the sub-image is performed based on a block-based enhanced correlation coefficient, a coarse estimation of the offset position of the sub-image relative to the main image is obtained, and a sub-image after offset correction is correspondingly obtained;

[0216] the sub-image after offset correction and the pre-processed main image are subjected to image pyramid processing, and a main pyramid image and a sub-pyramid image of multiple levels are respectively obtained;

[0217] the calculation of the optical flow vector is performed from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and the calculation result is transmitted to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is taken as the result of the fine estimation of the WasSAR image offset.

[0218] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0219] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0220] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for WasSAR image offset estimation for three-dimensional information extraction, characterized in that, The method comprises: Obtaining two sub-aperture images with different observation position perspectives, the two sub-aperture images being obtained by imaging two pieces of echo data obtained by observing the same scene based on a WasSAR platform; Respectively taking the two sub-aperture images as a main image and a secondary image, pre-processing the main image and the secondary image to make the brightness distributions of the pre-processed main image and secondary image consistent; Based on a block-based enhanced correlation coefficient, performing coarse estimation of the offset position of the pre-processed main image and secondary image to obtain coarse estimation of the offset position of the secondary image relative to the main image, and accordingly obtaining an offset-corrected secondary image; Performing pyramid processing on the offset-corrected secondary image and the pre-processed main image to obtain a multi-level main pyramid image and a secondary pyramid image; From the lowest resolution layer of the main pyramid image and the secondary pyramid image, calculating an optical flow vector, and passing the calculation result to the optical flow vector calculation of the next layer, and taking the optical flow vector obtained at the highest resolution layer as the result of the fine estimation of the offset of the WasSAR image.

2. The WasSAR image offset estimation method of claim 1, wherein, The azimuth angle interval of the main image and the secondary image is greater than 10°.

3. The WasSAR image offset estimation method of claim 2, wherein, The pre-processing of the main image and the secondary image comprises: Performing brightness equalization processing on the main image to obtain a brightness-equalized main image; Performing histogram matching on the brightness-equalized main image and the secondary image to obtain a matched main image and a matched secondary image; Respectively performing smoothing processing on the matched main image and the matched secondary image to obtain the pre-processed main image and the pre-processed secondary image.

4. The WasSAR image offset estimation method of claim 3, wherein, The histogram matching on the brightness-equalized main image and the secondary image comprises: Respectively calculating the cumulative distribution functions of the brightness-equalized main image and the secondary image; Taking the cumulative distribution function of the brightness-equalized main image as a standard, constructing a mapping function for the secondary image, and mapping each intensity value in the secondary image to a corresponding intensity value in the brightness-equalized main image to complete the histogram matching.

5. The WasSAR image offset estimation method of claim 2, wherein, The coarse estimation of the offset position of the pre-processed main image and secondary image based on the block-based enhanced correlation coefficient comprises: Respectively performing block processing on the pre-processed main image and the secondary image by using a uniform block strategy to obtain a plurality of sub-image blocks corresponding to the main image and the secondary image; Respectively performing enhanced correlation coefficient registration on each sub-image block in the main image and each sub-image block in the secondary image to obtain a coarse estimation result of the offset position.

6. The WasSAR image offset estimation method of claim 5, wherein, The uniform block strategy comprises: Respectively dividing the pre-processed main image and the secondary image into a plurality of sub-image blocks according to different division numbers, i.e., for each division number, the pre-processed main image and the secondary image will obtain a corresponding subset, and the subset includes a corresponding number of sub-image blocks; Calculating the correlation between each sub-image block in each subset to obtain a correlation measurement matrix; Selecting the minimum value from each correlation measurement matrix, comparing the minimum values of each subset, and taking the division number corresponding to the minimum value as the final image block data; According to the final image block data, performing block processing on the pre-processed main image and the secondary image.

7. The WasSAR image offset estimation method of any one of claims 1-6, characterized in that, When the offset-corrected secondary image and the preprocessed primary image are subjected to image pyramid processing, the minimum value of the number of pyramid layers is calculated by using the following formula: In the above equation, denotes the point target offset distance, d c denotes the imaging network size, δ r denotes the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, the symbol is the smallest integer not less than the value within the symbol.

8. A WasSAR image offset estimation device for three-dimensional information extraction, characterized by, The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device comprises: The device

Images