WasSAR image offset estimation method and device for three-dimensional information extraction

Abstract

The invention relates to a WasSAR image offset estimation method and device for three-dimensional information extraction, and the method comprises the steps: carrying out the preprocessing of two sub-aperture images, i.e., a main image and an auxiliary image, at different observation position visual angles, carrying out the offset position rough estimation of the preprocessed main image and auxiliary image based on a segmented enhanced correlation coefficient, and carrying out the offset position rough estimation of the preprocessed auxiliary image; performing image pyramid processing on the auxiliary image after offset correction and the preprocessed main image to respectively obtain a multi-level main pyramid image and an auxiliary pyramid image, and performing image pyramid processing on the multi-level main pyramid image and the multi-level auxiliary pyramid image to obtain a multi-level main pyramid image and a multi-level auxiliary pyramid image; and calculating the optical flow vector from the lowest resolution layer of the main pyramid graph and the secondary pyramid graph, transmitting the calculation result to the optical flow vector calculation of the next layer, and taking the optical flow vector obtained at the highest resolution layer as the result of WasSAR image offset fine estimation. By adopting the method, the precision and robustness of SAR image matching can be better realized.

Description

Technical Field

[0001] The present application relates to the field of WasSAR three-dimensional information extraction, and in particular to a WasSAR image offset estimation method and device for three-dimensional information extraction. Background Art

[0002] Wide Angle Staring Synthetic Aperture Radar (WasSAR) is an innovative SAR observation technology that overcomes some limitations of the traditional linear SAR mode and provides a new perspective for the acquisition of three-dimensional information. The WasSAR system uses a curved or circular motion trajectory on the airborne platform and realizes a beam-forming working mode by continuously pointing the radar beam at the observation area. The advantage of WasSAR technology is that it can collect information from multiple angles, thereby increasing the dimension of observation in the angle dimension, providing multi-angle observation information for traditional two-dimensional SAR imaging, greatly enriching the connection between data, and making it possible to extract three-dimensional information from two-dimensional images.

[0003] Similar to the principle of binocular stereo vision technology, the WasSAR system introduces angle-dimensional observation, and uses the correspondence between the same-name points in SAR images with similar azimuth angles to deduce the horizontal or vertical offset of the same-name points between images. Combined with the known motion parameters of the observation platform, it achieves accurate three-dimensional reconstruction of the observed scene. This innovative method based on angle-dimensional observation can effectively improve the observation and analysis capabilities of complex scenes.

[0004] Therefore, the estimation of the offset of the same-name points in SAR images observed at similar azimuth angles becomes the key to the WasSAR 3D information acquisition technology. However, due to the complex characteristics of SAR images, such as radiation attenuation, salt and pepper noise, overlap and shadow, the task of estimating the offset of the same-name points in SAR images with similar azimuth angles becomes challenging. Summary of the invention

[0005] Based on this, it is necessary to provide a WasSAR image offset estimation method and device for three-dimensional information extraction that can achieve better accuracy and robustness in response to the above technical problems.

[0006] A WasSAR image offset estimation method for three-dimensional information extraction, characterized in that the method comprises:

[0007] Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0008] The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0009] Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction;

[0010] Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0011] The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

[0012] In one embodiment, the azimuth angle interval between the main image and the auxiliary image is greater than 10°.

[0013] In one embodiment, the preprocessing of the main image and the sub-image includes:

[0014] Performing brightness equalization processing on the main image to obtain a main image with balanced brightness;

[0015] Performing histogram matching using the brightness-balanced main image and the sub-image to obtain a matched main image and sub-image;

[0016] The matched main image and sub-image are respectively smoothed to obtain the pre-processed main image and sub-image.

[0017] In one embodiment, performing histogram matching using the brightness-balanced main image and the auxiliary image includes:

[0018] Calculating the cumulative distribution functions of the brightness-balanced main image and the secondary image respectively;

[0019] Taking the cumulative distribution function of the brightness-balanced main image as a standard, a mapping function is constructed for the secondary image, and each intensity value in the secondary image is mapped to a corresponding intensity value in the brightness-balanced main image, thereby completing histogram matching.

[0020] In one embodiment, the roughly estimating the offset position of the pre-processed main image and the auxiliary image based on the block-based enhanced correlation coefficient includes:

[0021] Using a uniform block division strategy to divide the preprocessed main image and the auxiliary image into blocks, respectively, to obtain a plurality of sub-image blocks corresponding to the main image and the auxiliary image;

[0022] Enhanced correlation coefficient registration is performed on each sub-image block in the main image and each sub-image block in the auxiliary image to obtain a rough estimation result of the offset position.

[0023] In one embodiment, the uniform block partitioning strategy includes:

[0024] Respectively dividing the preprocessed main image and auxiliary image into a plurality of sub-image blocks according to different division numbers, that is, the preprocessed main image and auxiliary image will obtain a corresponding subset for each division number, and the subset includes a corresponding number of sub-image blocks;

[0025] Calculating the correlation between each sub-image block in each of the subsets and obtaining a correlation measurement matrix;

[0026] Selecting a minimum value from each of the correlation measurement matrices, comparing the minimum values ​​of each subset, and taking the number of divisions corresponding to the minimum value as the final number of image blocks;

[0027] The pre-processed main image and auxiliary image are subjected to block processing according to the final image block data.

[0028] In one embodiment, when performing image pyramid processing on the offset-corrected secondary image and the pre-processed primary image, the minimum number of pyramid layers is calculated using the following formula:

[0029]

[0030] In the above formula, Indicates the offset distance of the point target, d c represents the size of the imaging network, δ r It represents the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, symbol is the smallest integer not less than the value within the symbol.

[0031] The present application also provides a WasSAR image offset estimation device for three-dimensional information extraction, the device comprising:

[0032] The image acquisition module to be matched is used to acquire two sub-aperture images with different viewing angles at the observation position, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0033] A preprocessing module, used to use the two sub-aperture images as a main image and a sub-image respectively, and preprocess the main image and the sub-image so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0034] A rough estimation module, used for roughly estimating the offset position of the preprocessed main image and the auxiliary image based on the block enhancement correlation coefficient, obtaining a rough estimate of the offset position of the auxiliary image relative to the main image, and correspondingly obtaining an auxiliary image after offset correction;

[0035] An image pyramid processing module is used to perform image pyramid processing on the offset-corrected secondary image and the pre-processed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0036] The offset precision estimation module is used to calculate the optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and pass the calculation result to the optical flow vector calculation of the next layer, and use the optical flow vector obtained at the highest resolution layer as the result of the WasSAR image offset precision estimation.

[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0039] The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0040] Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction;

[0041] Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0042] The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0045] The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0046] Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction;

[0047] Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0048] The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

[0049] The above-mentioned WasSAR image offset estimation method and device for three-dimensional information extraction preprocesses two sub-aperture images with different observation position viewing angles, namely the main image and the sub-image, and roughly estimates the offset position of the preprocessed main image and the sub-image based on the block enhancement correlation coefficient, so as to obtain a rough estimate of the offset position of the sub-image relative to the main image, and correspondingly obtain the offset-corrected sub-image, and perform image pyramid processing on the offset-corrected sub-image and the preprocessed main image to obtain multi-level main pyramid images and sub-pyramid images respectively, calculate the optical flow vector from the lowest resolution layer of the main pyramid image and the sub-pyramid image, and pass the calculation result to the optical flow vector calculation of the next layer, and use the optical flow vector obtained at the highest resolution layer as the result of the WasSAR image offset precise estimation. The use of this method can better achieve the accuracy and robustness of SAR image matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of a WasSAR image offset estimation method for three-dimensional information extraction in one embodiment;

[0051] Figure 2 A schematic diagram of the geometric structure of a WasSAR imaging system in one embodiment;

[0052] Figure 3 A schematic diagram of an image preprocessing process in one embodiment;

[0053] Figure 4 A schematic diagram of image segmentation in one embodiment;

[0054] Figure 5 It is a schematic diagram of an implementation flow of a WasSAR image offset estimation method for three-dimensional information extraction in one embodiment;

[0055] Figure 6 An optical photograph of a scene observed in an experiment;

[0056] Figure 7 is the image preprocessing and histogram distribution results in an experiment, where: Figure 7 (a) is the original main image, Figure 7 (b) is the histogram of the original main image. Figure 7 (c) is the preprocessed main image. Figure 7 (d) is the histogram of the main image after preprocessing. Figure 7 (e) is the preprocessed secondary image. Figure 7 (f) is the histogram of the preprocessed secondary image;

[0057] Figure 8 This is a schematic diagram of the offset estimation results (pseudo-color contrast display) in an experiment, where: Figure 8 (a) Rough estimation results, Figure 8 (b) accurate estimation results;

[0058] Fig. 9 This is a schematic diagram of the similarity index measurement results in an experiment, where: Fig. 9 (a) is the CC result after processing. Fig. 9 (b) is the CC result before processing. Fig. 9 (c) is the SSIM result after processing. Fig. 9 (d) is the SSIM result before processing. Fig. 9 (e) is the CC frequency distribution results before and after processing. Fig. 9 (f) is the frequency value distribution result before and after SSIM processing;

[0059] Fig.10 Schematic diagram of the offset estimation results of different methods in an experiment, where: Fig.10 (a) is the initial image, Fig.10 (b) is the MS method, Fig.10 (c) is the MI method, Fig.10 (d) is KAZE, Fig.10 (e) is Bspline, Fig.10(f) is TPS, Fig.10 (g) are original Demons, Fig.10 (h) is the method proposed in this paper;

[0060] Fig.11 A schematic diagram of the positional relationship between a sub-block image and an original image in an experiment;

[0061] Fig.12 This is a schematic diagram of the results of three-dimensional information extraction in an experiment. Fig.12 (a) is the result of optical mapping. Fig.12 (b) is the 5-level pyramid extraction result. Fig.12 (c) is the first-level pyramid extraction result. Fig.12 (d) is the result of 10-level pyramid extraction;

[0062] Fig.13 is a structural block diagram of a WasSAR image offset estimation device for three-dimensional information extraction in one embodiment;

[0063] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] At present, there are mainly two types of methods for SAR image offset estimation with similar azimuth angles: feature-based methods and region-based methods.

[0066] Specifically, the feature-based method estimates by extracting feature points (such as corner points, spots, etc.) from the image. This method is more reliable when processing SAR images because feature points are usually not affected by radiation attenuation and salt and pepper noise. Feature-based methods usually include two steps: feature point extraction and feature point matching. Feature point extraction refers to extracting feature points from an image, such as corner points, spots, etc. These feature points usually have strong robustness and distinguishability. Feature point matching refers to matching the feature points in two images to find the correspondence between the two images.

[0067] The gross error matching ratio of the descriptor-based feature matching method on optical images is generally above 50%, while it can reach 90% or even higher on SAR images. However, in areas where features are not obvious, such as mountains, the number of feature points in SAR images is small, so the feature point-based registration method is not applicable. Although setting a stricter threshold for the descriptor can increase the correct matching ratio, this will also result in sacrificing a large number of correct matches, and usually only a small number of registered feature points can be obtained. Therefore, although it has applications in fields such as image fusion and change detection, this method cannot meet the needs of obtaining three-dimensional information in the entire scene, especially in areas such as mountains with complex terrain.

[0068] Specifically, the region-based SAR image registration method evaluates the degree of matching between images by calculating the similarity measure between images, thereby achieving image registration. This method can effectively utilize the texture information in the image without extracting feature points, so it can overcome the influence of factors such as salt and pepper noise and radiation attenuation on feature point extraction to a certain extent. Commonly used similarity measurement functions include cross-correlation information, mutual information, mean square error, etc. Cross-correlation information is a commonly used similarity measurement function, which is a measure of the correlation between two signals. Mutual information is a measure of the difference between the joint probability distribution and the marginal probability distribution of two random variables. The mean square error is the average of the sum of the squared differences between two signals and is one of the most commonly used error measures.

[0069] However, the SAR image registration method based on this type of similarity measure has some defects. First, this method needs to calculate the entire image, which is computationally intensive, time-consuming, and inefficient. Second, due to the complexity of SAR images, the selected similarity function may have multiple local maxima, which will lead to inaccurate registration results.

[0070] The deformation of WasSAR images targeted in this application is essentially a non-rigid deformation due to different observation angles and the characteristics of SAR images, which includes bending, twisting, etc., making the registration process more complicated. The registration method that simply relies on similarity measurement cannot meet the requirements of fast and high-precision registration.

[0071] The Lucas-Kanade optical flow algorithm is an advanced image registration technology that uses a global coordinate transformation model to handle the non-rigid motion of objects in an image sequence. The core of this method is to capture the changes in pixels in the time domain and the intrinsic connections between images, thereby identifying and calculating the motion trajectory of objects in consecutive frames. Unlike traditional rigid registration, the Lucas-Kanade algorithm accurately calculates the relative displacement of each pixel between the two images by carefully analyzing the grayscale value difference and gradient information between the reference image and the floating image. This calculation not only reveals the dynamic changes of the object, but also provides key information for non-rigid registration between images.

[0072] Although the optical flow algorithm is a powerful tool in the field of optical image registration, it faces some challenges and limitations when applied to SAR images acquired at different azimuth angles. First, unlike optical images, the brightness of SAR images reflects the intensity of radar backscattering, which is closely related to the scattering characteristics of the object, and is also related to factors such as the roughness, geometric structure and terrain of the target surface. Secondly, the scattering center of the image will also change at different observation angles, affecting the performance of the optical flow algorithm. In addition, the optical flow algorithm usually assumes that the motion in the image is smooth. When encountering large deformations, such as drastic position offsets or nonlinear deformations, this smoothness assumption may no longer hold, resulting in a decrease in algorithm performance. In the case of large deformations, the optical flow algorithm is prone to fall into a local minimum, that is, the algorithm may find a local optimal solution instead of a global optimal solution, thus affecting the registration accuracy.

[0073] In the present application, as shown in the figure, a WasSAR image offset estimation method for three-dimensional information extraction is provided, which can improve the applicability and effect of the optical flow algorithm in SAR images, and specifically includes the following steps:

[0074] Step S100, obtaining two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform.

[0075] Step S110 , taking the two sub-aperture images as the main image and the sub-image respectively, and preprocessing the main image and the sub-image so that the brightness distribution of the preprocessed main image and the sub-image is consistent.

[0076] Step S120 , based on the enhanced correlation coefficients of the blocks, a rough estimate of the offset position of the preprocessed main image and the sub-image is performed to obtain a rough estimate of the offset position of the sub-image relative to the main image, and a sub-image after offset correction is obtained accordingly.

[0077] Step S130 , performing image pyramid processing on the offset-corrected secondary image and the pre-processed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image.

[0078] Step S140, calculating the optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and passing the calculation result to the optical flow vector calculation of the next layer, and taking the optical flow vector obtained at the highest resolution layer as the result of the WasSAR image offset precision estimation.

[0079] In this embodiment, first, image preprocessing is performed to reduce image noise, balance the brightness distribution of the main and auxiliary images, and ensure their consistency in grayscale distribution, thereby laying a good foundation for subsequent registration work. Secondly, a block processing method based on enhanced correlation coefficient (ECC) is used for coarse registration. This method estimates and reduces the offset range of large deformation areas by calculating the correlation between image blocks. Through ECC blocking, the relative position between images can be quickly located within a large range, providing an initial estimate for the fine registration stage. Finally, an improved optical flow algorithm is used to perform fine registration of the images. This improved optical flow algorithm is optimized for the characteristics of SAR images, and can more accurately capture and estimate small offsets in images, achieving high-precision offset estimation.

[0080] This method can effectively overcome the problems that SAR images may encounter in traditional optical flow algorithms, such as large deformation, inconsistent brightness and differences in scattering characteristics, by integrating image preprocessing, coarse registration and improved optical flow fine registration, thereby achieving higher accuracy and robustness in SAR image registration tasks.

[0081] In step S100, two sub-aperture images are obtained as follows: Figure 2 The same scene was observed under the geometric structure of the WasSAR imaging system shown in Figure 1. Figure 2 In the paper, a spatial rectangular coordinate system OXYZ is established, where the OXY plane is defined as the imaging plane and the OZ axis represents the height axis. The airborne radar platform is located on a plane at a height of H from the horizontal plane, and performs a curved motion with a radius of R around the vertical axis OZ, and its tangential velocity is maintained at V. During the motion, the radar transmit signal is always aimed at the center point O of the imaging area, thereby ensuring that a circular imaging area with O as the center and a radius of R can be formed after the aircraft completes a circle of flight. At any time, the position of the radar can be expressed by (Rcosθ, Rsinθ, H) in the spherical coordinate system, where θ is the angle between the current position of the radar and the positive X-axis. For a point target P located at the observation position, its position in the coordinate system is (x, y, z). According to the geometric relationship, the instantaneous distance from the radar to the point target P can be expressed as formula (1).

[0082]

[0083] Assuming the imaging plane is the z=0 plane, and performing BP imaging on the point target P, its point spread function on the imaging plane is:

[0084]

[0085] In formula (2), R xy is the distance from the radar platform to each pixel point on the imaging plane. After Fourier transform, the result is:

[0086]

[0087] Let Φ = -2KR lp +2KR xy -K x xK y y, respectively find the first-order partial derivatives of Φ with respect to x and y, and the corresponding instantaneous frequency is:

[0088]

[0089] Let the first-order partial derivative be 0, then formula (3) can be expressed as follows:

[0090] G(K x ,K y )=exp{jΦ(x(K x ),y(K y ))} (5)

[0091] Performing the inverse Fourier transform, we can get:

[0092]

[0093] in, make

[0094]

[0095] For formula (7), find Ψ(K x ,K y )About K x , K y The first-order partial derivative of , we can get:

[0096]

[0097] Let the first-order partial derivative be 0, we can get:

[0098] (xx p ) 2 +(yy p ) 2=(z p tanθ) 2 (9)

[0099] It can be seen from formula (9) that when there is a height difference between the target height plane and the imaging plane, the target will be defocused during WasSAR focusing imaging, resulting in geometric deformation of the image, and a (x p ,y p ) is the center of the circle. p tanθ| is the radius (the absolute value is taken because z is negative when the target height is lower than the imaging height).

[0100] WasSAR imaging technology is an imaging method that relies on target height information. When the exact position of the target is unknown, the three-dimensional spatial information of the target can be derived by analyzing the offset information of the target in the WasSAR image sequence. The key to this process is to accurately estimate the offset of the same-name points in WasSAR images at different azimuths.

[0101] In this embodiment, when reconstructing three-dimensional information based on perspective shift, in order to ensure the accuracy of the extracted elevation data, it is usually required that the azimuth interval is at least 10°, that is, the azimuth interval between the main image and the auxiliary image is greater than 10°.

[0102] Considering that SAR images often show obvious differences in brightness and contrast due to their unique imaging mechanism, these differences are mainly due to different scattering intensities of the targets. Before SAR image registration, the primary and secondary SAR images need to be preprocessed to make their brightness distribution consistent.

[0103] In step S110, Figure 3 As shown, the preprocessing of the main image A and the sub-image B includes: performing brightness equalization processing on the main image A to obtain the brightness-balanced main image A, performing histogram matching using the brightness-balanced main image A and the sub-image B to obtain the matched main image A and sub-image B, and respectively performing smoothing processing on the matched main image A and sub-image B to obtain the preprocessed main image A and sub-image B.

[0104] In one embodiment, performing histogram matching using a brightness-balanced main image A and a sub-image B includes: respectively calculating the cumulative distribution functions of the brightness-balanced main image A and the sub-image B, using the cumulative distribution function of the brightness-balanced main image A as a standard, constructing a mapping function for the sub-image B, mapping each intensity value in the sub-image B to the corresponding intensity value in the brightness-balanced main image A, and completing the histogram matching.

[0105] Specifically, first, the main image A is subjected to brightness equalization. In this embodiment, the histogram distribution is used as the basis for histogram equalization, and the pixel values ​​are redistributed to make the image brightness distribution more uniform, thereby improving the image contrast and visual quality. The core idea of ​​adaptive histogram equalization technology is to divide the image into multiple small regions and perform histogram equalization independently in each small region. The local histogram H(i, region) is shown in formula (10):

[0106]

[0107] In formula (10), δ is the Kronecker function, I(x, y) is the pixel value of the original image at (x, y), and when I(x, y) = i, δ is 1, otherwise it is 0.

[0108] Based on the local histogram, the cumulative distribution function (CDF) of each small region is calculated to reflect the cumulative probability distribution of pixel values. The cumulative distribution function C(i, region) is shown in formula (11):

[0109]

[0110] Using the cumulative distribution function, the equalization mapping function T(i, region) is constructed, which maps the original pixel value to the new equalization value, as shown in formula (12). The equalized pixel I′(x, y) is shown in formula (13):

[0111]

[0112] I′ A (x,y)=T(I A (x,y),region) (13)

[0113] The equalization results of all small areas are combined to obtain the final adaptive histogram equalization image.

[0114] Next, for the equalized image A and the image to be matched B, their cumulative distribution functions are calculated, as shown in formula (14):

[0115]

[0116] In formula (14), H A (i) and H B (i) are the histogram values ​​of images A and B respectively, N is the total number of pixels in the image, and k is the pixel intensity value.

[0117] Next, using the CDF of image A as a standard, a mapping function is constructed for image B to map each pixel intensity value in B to the corresponding intensity value of A. The mapping function is shown in formula (15):

[0118]

[0119] In formula (15), I B is the pixel value of the original image B, I′ B is the pixel value after mapping, M is the mapping function, which maps the gray level i in image B to the gray level j in image A that makes the CDF value closest. The matched image B(I′ B ) will have similar brightness distribution and contrast as image A.

[0120] Finally, the histogram is smoothed to reduce the influence of noise. A Gaussian filter is used in this step. As shown in formula (16):

[0121]

[0122] In formula (16), H′ A (i) and H′ B (i) is the smoothed histogram value, and G is the Gaussian kernel function. Through the above image preprocessing, we can effectively adjust the brightness and contrast of the two SAR images to make them visually and statistically consistent. This consistency is crucial for subsequent image registration operations and can effectively improve the robustness and accuracy of the registration algorithm.

[0123] Next, in step S120, the preprocessed main image and auxiliary image are rigidly registered using the Enhanced Correlation Coefficient (ECC). The Enhanced Correlation Coefficient (ECC) is a metric used for image registration, which provides a means of evaluating the similarity between two images, especially when performing image fusion or registration tasks. The core of the ECC method is to quantify the correlation between two image regions and use this correlation to estimate the optimal alignment between them. The ECC registration method is mainly used for image registration of rigid or affine transformations, including translation, rotation and scaling. For non-rigid deformations, that is, those involving local deformation, bending or other complex geometric transformations, traditional ECC registration may not be sufficient to achieve high-precision registration. This method achieves coarse image registration by dividing the image into multiple sub-blocks and applying ECC registration independently to each image sub-block to adapt to local changes.

[0124] In this embodiment, the rough estimation of the offset position of the preprocessed main image and the sub-image based on the enhanced correlation coefficient of the block includes: using a uniform blocking strategy to block the preprocessed main image and the sub-image respectively, to obtain a plurality of sub-image blocks corresponding to the main image and the sub-image respectively, and to align each sub-image block in the main image with each sub-image block in the sub-image by enhancing the correlation coefficient to obtain a rough estimation result of the offset position.

[0125] like Figure 4 As shown, when performing rough registration, the two images are first divided into blocks. In this method, a uniform block division method is adopted. In order to avoid discontinuity at the boundary of the block, overlapping block division can be used, that is, each sub-block has a partial area overlapping with other sub-blocks.

[0126] Furthermore, the uniform blocking strategy includes: dividing the preprocessed main image and sub-image into multiple sub-image blocks according to different division numbers, that is, the preprocessed main image and sub-image will obtain corresponding subsets for each division number, and the subset includes a corresponding number of sub-image blocks, calculating the correlation between each sub-image block in each subset, and obtaining a correlation measurement matrix, selecting the minimum value from each correlation measurement matrix, comparing the minimum value of each subset, and taking the division number corresponding to the minimum value as the final image block number, and finally, the preprocessed main image and sub-image are blocked according to the final image blocking data.

[0127] Specifically, the image segmentation strategy is as follows:

[0128] (1) Uniform block division: divide the main image and the auxiliary image into m×m small blocks, each of which contains a subset of the image (m=1, 2, 3, ...);

[0129] (2) Correlation calculation: Analyze these sub-image blocks, calculate the correlation between each pair of sub-image blocks, and obtain m×m correlation metrics;

[0130] (3) Determine the minimum correlation value: Among the m×m correlation measures obtained, identify the minimum value, which is recorded as Corr Min,m ;

[0131] (4) Determine the optimal sub-block size: By comparing the Corr corresponding to different m values Min,m , choose so that Corr Min,m The smallest m is used as the final sub-image block division parameter.

[0132] In this embodiment, the minimum correlation value Corr is selected Min,mAs a criterion, when the correlation between sub-image pairs is low, it indicates that they are poorly matched, which usually means that these areas need more accurate registration. Therefore, we focus on the sub-image pairs with the lowest correlation to achieve more effective registration.

[0133] It should be noted that choosing too large a value of m may lead to a significant increase in the amount of computation and may increase the risk of sub-image block mismatch, which is inconsistent with the goal of coarse registration. Therefore, preferably, the value of m will be controlled within 10 to balance the registration effect and computational efficiency.

[0134] Next, the ECC registration algorithm is used to register each image sub-block. ECC is usually defined as:

[0135]

[0136] In formula (17), i r and i w are the zero-mean versions of the pixel intensity vectors of the reference image (primary image) and the deformed image (secondary image), respectively, and p is the vector of transformation parameters. The core purpose of the registration algorithm is to maximize the enhancement correlation coefficient ρ(p). Among them, is a constant, representing a normalized zero-mean reference vector. In this paper, the gradient descent method is used to solve the ρ(p) maximization problem.

[0137] In this method, the ECC algorithm is used because it is robust to changes in the overall brightness of the image, focuses on geometric changes in the image, and has the ability to correct image deformation. However, when faced with images with complex geometric distortion or large size, using a single homography matrix for global solution may not be sufficient to accurately correct the geometric mismatch between images caused by different degrees of distortion. In this case, the ECC algorithm may be difficult to efficiently achieve accurate registration of local images. Through block processing, the ECC algorithm is applied independently to each image block for registration, and finally the registered image blocks are recombined, which can effectively reduce the computational difficulty caused by the large image size or the complexity of geometric distortion, while providing an accurate initial estimate for the subsequent fine registration stage.

[0138] Optical flow image offset estimation is a method that uses optical flow technology to determine the relative position change between two images. Applying the optical flow method to WasSAR images requires the following three assumptions:

[0139] Grayscale constancy: The optical flow algorithm is based on a basic assumption that in an image sequence, if pixels appear in overlapping areas, their grayscale values ​​remain unchanged between different images.

[0140] Small motion assumption: When the object's motion is small, the displacement of pixels between the two images will not be too large.

[0141] Spatial coherence: The projections of adjacent regions on a two-dimensional image should also be close. That is, adjacent pixels have similar movement speeds. Spatial coherence allows us to use a set of adjacent pixels to construct a set of linear equations and solve these equations through the least squares method to obtain the final velocity estimate.

[0142] After performing image preprocessing and rough registration operations in step S110 and step S120 , the offset-corrected secondary image and the preprocessed primary image already meet the above three assumptions.

[0143] Ensure that the WasSAR image meets the basic assumptions of the optical flow algorithm. Assume that the positions of a point target in two adjacent sub-aperture images are I(x, y, t) and I(x+dx, y+dy, t+dt), respectively. According to the grayscale constancy assumption, we can get:

[0144] I(x,y,t)=I(x+dx,y+dy,t+dt) (18)

[0145] According to the small motion assumption, the first-order Taylor expansion of the right side of formula (18) yields:

[0146]

[0147] According to formula (18) and formula (19), we can get:

[0148]

[0149] When dt is small enough and approaches 0:

[0150]

[0151] Let u = dx / dt, v = dy / dt, formula (21) can be rewritten as:

[0152] -I t =I x u+I y v (22)

[0153] Formula (22) is the basic optical flow constraint equation.

[0154] When acquiring three-dimensional information based on perspective shift, in order to ensure the accuracy of the extracted elevation data, the azimuth interval is usually required to be at least 10°. However, for mountainous scenes with large terrain fluctuations, due to the large parallax under different perspectives, even if the ECC method is used for preliminary coarse registration processing, there may still be significant position deviations. These deviations will lead to geometric inconsistencies between images, which in turn violates the premise of small motion of the optical flow algorithm.

[0155] In step S130, an image pyramid technique is introduced to reduce the displacement offset between images. Image pyramid is a multi-scale image representation method that builds a pyramid structure by gradually reducing the image resolution. The image size of each level is halved, thereby reducing the displacement between each level of images. At a lower resolution level, the same physical displacement causes less change at the pixel level, which helps meet the requirements of the optical flow algorithm for the small motion assumption.

[0156] For image I, the next layer of the image is constructed by Gaussian filtering and downsampling. The formula for Gaussian filtering is:

[0157]

[0158] In formula (23), G(x,y) is the Gaussian kernel function and σ is the standard deviation.

[0159] Downsampling usually uses the closest even-numbered point for downsampling, as shown in the formula:

[0160] I n (x,y)=I n-1 (2x,2y) (24)

[0161] In formula (24), I n (x, y) is the image of the nth pyramid after downsampling, I n-1 is the n-1th layer image, I 0 is the original image. If the resolution of the original image is M×N, then the resolution at layer k is

[0162] When passing optical flow from one layer of the pyramid to the next, the optical flow vector is updated as shown in the formula:

[0163]

[0164] In formula (25), v n and u n is the optical flow vector of the nth layer after downsampling, v n-1 and u n-1 is the optical flow vector of the n-1th layer.

[0165] However, the more pyramid levels there are, the better. The more pyramid levels there are, the more serious the loss of image details will be, which will also affect the performance of the algorithm. The number of pyramid levels should be set to the minimum number of levels that meets the small motion assumption of the image. The small motion assumption usually means that the offset on the image does not exceed 5-10 pixels. But this is just a rough empirical value, and the specific situation needs to be adjusted according to the actual application.

[0166] In WasSAR 3D information extraction, the relationship between target offset and target is as shown in the formula:

[0167]

[0168] In formula (26), X TA and X TB are the horizontal coordinates of the point target T in the sub-aperture images A and B, y TA and TB are the corresponding ordinates, h T is the height of the point target T, and is the azimuth angle of the center of the two sub-aperture observation positions. A and θ B is the corresponding incident angle. Then the relationship between the number of pyramid layers and the point target offset is as shown in the formula:

[0169]

[0170] In formula (27), Indicates the offset distance of the point target, d c is the imaging grid size, the number of layers of the image pyramid is k, δ r To satisfy the maximum pixel offset of the optical flow algorithm's small motion assumption, the minimum number of pyramid layers k is min for:

[0171]

[0172] In formula (8), Indicates the offset distance of the point target, d c represents the size of the imaging network, δ r It represents the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, symbol is the smallest integer not less than the value within the symbol.

[0173] In this embodiment, the data processing flow of the WasSAR image offset estimation method for three-dimensional information extraction is as follows: Figure 5 shown.

[0174] In order to fully verify the effectiveness and practicality of the proposed method (this method) in practical applications, the following section will introduce in detail the specific application of the algorithm in processing airborne Ku-band synthetic aperture radar (WasSAR) measured data. These measured data are collected by the airborne Ku-band WasSAR system, ensuring the reliability of the data and the scientific nature of the experiment.

[0175] In this experiment, the observation scene selected is a typical mountainous area. The complex and changeable terrain in this area is very suitable for testing the adaptability and accuracy of the algorithm to complex terrain. In order to more intuitively display the observation scene, optical photos of the scene are provided, such as Figure 6 shown.

[0176] First, two SAR images with different azimuth angles are selected. The azimuth angle interval between the two SAR images is 15°. The main image is as follows: Figure 7 (a) As shown. According to the image preprocessing scheme of this method, the main image after image brightness equalization processing is as follows Figure 7 (c) As shown. The secondary image is histogram-aligned, and the primary and secondary images are histogram-filtered. The preprocessed image is as follows: Figure 7 (e) as shown.

[0177] Next, the offset estimation operation is performed on the preprocessed image. The experimental results after rough registration are as follows: Figure 8 As shown in (a), it can be seen that for areas with small elevation fluctuations (such as farmland), due to the small degree of deformation in this part, ECC coarse registration can achieve a certain degree of correction. However, for mountainous areas with large elevation fluctuations, coarse registration can only reduce the degree of deformation and cannot achieve effective correction.

[0178] In the final stage of the proposed method, the preliminary results of image registration are further refined to accurately estimate the offset of each pixel. The number of layers of the Gaussian pyramid is determined to be 5 according to formula (28). Fig. 9 As shown in (b), the optical flow algorithm is used to successfully achieve accurate estimation of the motion vectors of all pixels in the entire image. This estimation not only improves the accuracy of registration, but also enhances the algorithm's adaptability to complex non-rigid transformations.

[0179] Furthermore, a detailed comparative analysis of the similarity information of the images before and after processing was conducted. Specifically, the correlation (CC) and structural similarity (SSIM) of the images before and after processing were evaluated to quantitatively measure the processing effect. Fig. 9 shown.

[0180] Through comparative analysis, it is clearly observed that the image processing has significantly improved in terms of correlation and structural similarity. It should be noted that in the correlation graph, there are areas with zero correlation at the edges, which indicates that after the image is offset corrected, information may have been lost in the edge areas. In addition, the frequency distribution curve of similarity information shows that the image after offset correction is more concentrated in frequency distribution, and the concentration is enhanced. Specifically, the correlation index of the processed image reached 0.9587, and the structural similarity index (SSIM) was increased to 0.9319. Both values ​​are significantly higher than the corresponding indicators of the original image, further confirming the effectiveness of the processing effect.

[0181] In order to systematically evaluate the effectiveness of existing image offset correction algorithms, this study conducted a series of comparative experiments. The algorithms involved in the experiment include: grayscale intensity information-based registration strategy, which is subdivided into four modes: affine transformation, rigid transformation, similarity-based transformation, and translation transformation; advanced registration technology using KAZE features for scale-invariant feature transformation; non-rigid registration method using Thin Plate Splines (TPS); elastic registration technology based on B-spline; and the widely recognized classic Demons algorithm.

[0182] like Fig.10 As shown in the figure, the registration effect of each algorithm is intuitively demonstrated through comparative experiments.

[0183] In order to comprehensively evaluate the effect of image correction, a multi-scale analysis method was used. Not only was the entire image comprehensively evaluated, but also four areas of significant changes in the image were paid special attention to. In the selection of evaluation indicators, a series of widely recognized metrics including SSD (Sum of Squared Differences), correlation, mutual information, PSNR (Peak Signal-to-Noise Ratio), Structural Similarity Index (SSIM) and RMSE (Root Mean Square Error) were used. In addition, in order to further improve the accuracy of the evaluation, an advanced image alignment quality assessment method was introduced to provide a more in-depth and comprehensive evaluation of the image registration results.

[0184] Comparison results of the whole image: The performance of the whole image was evaluated using different correction methods and a series of widely recognized metrics, and the results are summarized in Table 1. After further analysis of the data in the table, it can be clearly observed that the algorithm proposed in this method shows excellent performance that surpasses the existing algorithms in all performance evaluation indicators. Specifically, the correlation coefficient (CC) achieved a significant improvement of 25-33%, the mutual information (MI) index increased by more than 80%, and the structural similarity index (SSIM) increased by more than 50%. In addition, the peak signal-to-noise ratio (PSNR) also achieved a robust improvement of 13-15%. In terms of error metrics, the reduction in root mean square error (RMSE) reached less than 75% of the existing algorithms, while the sum absolute error (SSD) was only 50-60% of the existing algorithms. In terms of algorithm efficiency, the computation time of the algorithm in this paper is significantly shortened, generally better than most existing algorithms, and its time consumption is only 20%-50% of the existing algorithms.

[0185] Table 1 Performance evaluation results of different methods for full-image offset estimation

[0186]

[0187] Sub-block image comparison results: The effectiveness of local image correction was discussed in depth, with special attention paid to four areas with significant offsets and rich features. Fig.11 As shown in Figure 2, the positional relationship between these sub-block images and the original image is clearly shown. In order to improve efficiency, four existing algorithms with excellent performance were selected during the performance evaluation and compared with the correction results of the algorithm proposed in this paper. The evaluation results are listed in Table 2.

[0188] Table 2 Performance evaluation results of different sub-block image offset estimation methods

[0189]

[0190] Comparative analysis shows that the proposed method is significantly better than other existing algorithms in all evaluation indicators, and this trend is consistent with the correction results of the entire image. Considering that the offset degree of the selected sub-block area is large, it means that more precise adjustments and more complex calculations are required during the correction process, which increases the difficulty of correction to a certain extent. Therefore, the relative insufficiency of the sub-block image evaluation indicators also reflects the challenge of the correction algorithm in dealing with complex scenes.

[0191] The analysis of the comprehensive experimental results fully proves that the proposed algorithm has significant advantages in the accuracy and reliability of image offset estimation. Whether it is the correction of the entire image or the local significant offset area, the proposed algorithm has shown excellent performance, verifying the effectiveness and innovation of this algorithm.

[0192] Furthermore, the focus will be placed on the application field of 3D information extraction. Specifically, the image offset information obtained by the proposed algorithm will be used to explore its performance in 3D information extraction and analysis. This step is crucial to verify the applicability and robustness of the algorithm when processing higher dimensional data. By comprehensively evaluating the offset values ​​of the 3D information extraction results, it is expected that the potential of the proposed algorithm in practical applications and its impact on 3D modeling and visualization can be fully understood.

[0193] In order to verify the practicality of the algorithm, it is applied to the 3D information extraction task and the 3D information extraction results are presented in Fig.12 In the analysis of the errors in 3D information extraction, it is found that for traditional optical flow algorithms, due to the large target offset, it is often difficult to achieve accurate offset estimation, resulting in large errors in 3D information reconstruction. On the other hand, if the number of pyramid layers is too large, it may cause the loss of important detail information in the target scene. Although it may only show a slight performance degradation in the evaluation indicators, it may cause a large deviation in the accuracy of 3D information extraction. However, through the method in this paper, after image preprocessing and coarse alignment, the number of sub-aperture division layers of the pyramid optical flow algorithm is optimized, thereby achieving high-precision 3D information extraction for the entire scene.

[0194] Based on the above detailed experimental analysis and targeted application verification, the experiment fully confirms that this method has significant practicality and efficiency in the application of three-dimensional information extraction of observation scenes. The experimental data and application results not only verify the theoretical advantages of the algorithm, but also demonstrate its potential in solving practical problems. In addition, by comparing with the existing technology, the excellent performance of the proposed algorithm in terms of accuracy, efficiency and stability further highlights its application value in the field of three-dimensional information extraction.

[0195] In the above-mentioned WasSAR image offset estimation method for three-dimensional information extraction, an innovative multi-view image offset estimation method is proposed to address the key issues of WasSAR imaging technology in three-dimensional information extraction. This method achieves accurate estimation of image offset through brightness equalization preprocessing, ECC block coarse estimation, and improved optical flow algorithm fine estimation, which significantly improves the accuracy and reliability of WasSAR image offset information acquisition. This method was fully experimentally verified using actual measured data of mountain scenes. The experimental results were carefully and deeply compared with various current mainstream offset estimation methods, fully demonstrating the significant advantages of this method in multiple key performance indicators. In addition, a comprehensive analysis was conducted on the performance of the proposed method in three-dimensional information extraction applications, and both the experimental data and analysis results clearly confirmed the high efficiency and strong practical value of this method in practical applications.

[0196] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0197] In one embodiment, Fig.13 As shown, a WasSAR image offset estimation device for three-dimensional information extraction is provided, comprising: an image acquisition module 200 to be matched, a preprocessing module 210, a coarse matching module 220, an image pyramid processing module 230 and an offset fine estimation module 240, wherein:

[0198] The to-be-matched image acquisition module 200 is used to acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0199] A preprocessing module 210 is used to use the two sub-aperture images as a main image and a sub-image respectively, and preprocess the main image and the sub-image so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0200] A coarse matching module 220 is used to perform a coarse estimation of the offset position of the preprocessed main image and the secondary image based on the enhanced correlation coefficient of the blocks, obtain a coarse estimation of the offset position of the secondary image relative to the main image, and obtain a secondary image after offset correction accordingly;

[0201] An image pyramid processing module 230 is used to perform image pyramid processing on the offset-corrected secondary image and the pre-processed primary image to obtain a multi-level primary pyramid map and a secondary pyramid map respectively;

[0202] The offset precision estimation module 240 is used to calculate the optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and pass the calculation result to the optical flow vector calculation of the next layer, and use the optical flow vector obtained at the highest resolution layer as the result of the WasSAR image offset precision estimation.

[0203] For the specific definition of the WasSAR image offset estimation device for three-dimensional information extraction, please refer to the definition of the WasSAR image offset estimation method for three-dimensional information extraction above, which will not be repeated here. Each module in the above-mentioned WasSAR image offset estimation device for three-dimensional information extraction can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0204] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a WasSAR image offset estimation method for three-dimensional information extraction is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0205] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0206] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0207] Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0208] The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0209] Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction;

[0210] Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0211] The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

[0212] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0213] Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform;

[0214] The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent;

[0215] Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction;

[0216] Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively;

[0217] The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

[0218] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0219] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0220] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A WasSAR image offset estimation method for three-dimensional information extraction, characterized in that: The method comprises: Acquire two sub-aperture images with different viewing angles at observation positions, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform; The two sub-aperture images are used as a main image and a sub-image respectively, and the main image and the sub-image are preprocessed so that the brightness distribution of the preprocessed main image and the sub-image is consistent; Based on the enhanced correlation coefficient of the blocks, a rough estimate of the offset position of the preprocessed main image and the auxiliary image is performed to obtain a rough estimate of the offset position of the auxiliary image relative to the main image, and accordingly obtain an auxiliary image after offset correction; Performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively; The optical flow vector is calculated from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and the calculation result is passed to the optical flow vector calculation of the next layer, and the optical flow vector obtained at the highest resolution layer is used as the result of the WasSAR image offset precision estimation.

2. The WasSAR image offset estimation method according to claim 1, characterized in that: The azimuth angle interval between the main image and the auxiliary image is greater than 10°.

3. The WasSAR image offset estimation method according to claim 2, characterized in that: The preprocessing of the main image and the secondary image comprises: Performing brightness equalization processing on the main image to obtain a main image with balanced brightness; Performing histogram matching using the brightness-balanced main image and the sub-image to obtain a matched main image and sub-image; The matched main image and sub-image are respectively smoothed to obtain the pre-processed main image and sub-image.

4. The WasSAR image offset estimation method according to claim 3, characterized in that: The performing histogram matching using the brightness-balanced main image and the auxiliary image comprises: Calculating the cumulative distribution functions of the brightness-balanced main image and the secondary image respectively; Taking the cumulative distribution function of the brightness-balanced main image as a standard, a mapping function is constructed for the secondary image, and each intensity value in the secondary image is mapped to a corresponding intensity value in the brightness-balanced main image, thereby completing histogram matching.

5. The WasSAR image offset estimation method according to claim 2, characterized in that: The method of roughly estimating the offset position of the preprocessed main image and the auxiliary image based on the block-based enhanced correlation coefficient includes: Using a uniform block division strategy to divide the preprocessed main image and the auxiliary image into blocks, respectively, to obtain a plurality of sub-image blocks corresponding to the main image and the auxiliary image; Enhanced correlation coefficient registration is performed on each sub-image block in the main image and each sub-image block in the auxiliary image to obtain a rough estimation result of the offset position.

6. The WasSAR image offset estimation method according to claim 5, characterized in that: The uniform block partitioning strategy includes: Respectively dividing the preprocessed main image and auxiliary image into a plurality of sub-image blocks according to different division numbers, that is, the preprocessed main image and auxiliary image will obtain a corresponding subset for each division number, and the subset includes a corresponding number of sub-image blocks; Calculating the correlation between each sub-image block in each of the subsets and obtaining a correlation measurement matrix; Selecting a minimum value from each of the correlation measurement matrices, comparing the minimum values ​​of each subset, and taking the number of divisions corresponding to the minimum value as the final number of image blocks; The pre-processed main image and auxiliary image are subjected to block processing according to the final image block data.

7. The WasSAR image offset estimation method according to any one of claims 1 to 6, characterized in that: When performing image pyramid processing on the offset-corrected secondary image and the preprocessed primary image, the minimum number of pyramid layers is calculated using the following formula: In the above formula, Indicates the offset distance of the point target, d c represents the size of the imaging network, δ r It represents the maximum pixel offset that satisfies the small motion assumption of the optical flow algorithm, symbol is the smallest integer not less than the value within the symbol.

8. A WasSAR image offset estimation device for three-dimensional information extraction, characterized in that: The device comprises: The image acquisition module to be matched is used to acquire two sub-aperture images with different viewing angles at the observation position, wherein the two sub-aperture images are obtained by imaging two segments of echo data obtained when observing the same scene based on the WasSAR platform; A preprocessing module, used to use the two sub-aperture images as a main image and a sub-image respectively, and preprocess the main image and the sub-image so that the brightness distribution of the preprocessed main image and the sub-image is consistent; A rough estimation module, used for roughly estimating the offset position of the preprocessed main image and the auxiliary image based on the block enhancement correlation coefficient, obtaining a rough estimate of the offset position of the auxiliary image relative to the main image, and correspondingly obtaining an auxiliary image after offset correction; An image pyramid processing module is used to perform image pyramid processing on the offset-corrected secondary image and the pre-processed primary image to obtain a multi-level primary pyramid image and a secondary pyramid image respectively; The offset precision estimation module is used to calculate the optical flow vector from the lowest resolution layer of the main pyramid image and the secondary pyramid image, and pass the calculation result to the optical flow vector calculation of the next layer, and use the optical flow vector obtained at the highest resolution layer as the result of the WasSAR image offset precision estimation.

Images