Target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar image

Abstract

This invention discloses a method for target projection 3D reconstruction based on multi-azimuth synthetic aperture radar (DEM) images, comprising: projecting a set of oblique-plane multi-azimuth SAR images onto a ground plane using a reference DEM; extracting target images from the ground-plane multi-azimuth SAR image set; binarizing the target image set to obtain a multi-azimuth mask image set; calculating the initial center position of the observed target based on the projection relationship obtained from the projection, and constructing an initial 3D structural model centered on this position; verifying the authenticity of the oblique projection point coordinates of the structural units using the multi-azimuth mask image set, and determining the true 3D structural model based on the verification results; and performing 3D reconstruction based on the true 3D structural model. This invention eliminates the need for registration based on feature points and image coherence, avoids dependence on strong scattering points, and improves the success rate of 3D reconstruction when the azimuth span is greater than 90 degrees.

Description

Technical Field

[0001] This invention belongs to the field of synthetic aperture radar (SAR) technology, specifically relating to a method for three-dimensional reconstruction of target projection based on multi-azimuth synthetic aperture radar images. Background Technology

[0002] Synthetic Aperture Radar (SAR) has been widely used in military reconnaissance, national economic construction, and scientific research due to its all-weather, all-day global observation capabilities. To achieve fine-grained resolution, high-precision positioning, and omnidirectional description of observed targets, multi-azimuth SAR systems have been proposed in recent years. These systems utilize multiple flybys or multi-satellite networks to complete multi-azimuth observations of the same scene. With appropriate orbital parameters and by switching between left and right views of the radar beam, up to 360° multi-azimuth observations of the same scene can be achieved. By repeatedly observing the same area from different azimuths, multi-view geometric and scattering information of the terrain features in that area can be obtained, thus compensating for the severe information loss and poor image interpretation of single-view SAR systems. Therefore, SAR systems can not only acquire richer target features within the scene but also possess the potential for 3D reconstruction, greatly improving the detection performance of SAR systems.

[0003] In existing technologies, 3D reconstruction is typically achieved through registration based on SAR image coherence, which requires establishing the correspondence between corresponding points in SAR images at different azimuth angles. However, when the observation azimuth angles differ significantly, the occlusion relationship between the observed target and the scene changes, potentially resulting in the absence of corresponding points between SAR images at different azimuth angles, thus making registration impossible. For most observed targets, their scattering characteristics change drastically with azimuth variation, similarly causing traditional SAR image coherence-based registration methods to fail. Furthermore, existing methods are highly dependent on strong scattering points within the observed target. When strong scattering points are present in the image, the geometric structure of the observed target can be constructed by 3D locating a finite number of strong scattering points; however, when the observed target lacks strong scattering points or its overall scattering characteristics are relatively uniform, it is impossible to extract strong scattering points, thus hindering 3D reconstruction of the target through 3D locating these points. Therefore, existing methods for 3D reconstruction based on strong scattering points are only suitable for targets with stable scattering characteristics that do not change with viewing angle. However, in real-world scenarios, apart from corner reflector calibrators, there are very few targets whose scattering characteristics remain stable over a large range of observation azimuth angles.

[0004] For example, in the paper "A Three-Dimensional Imaging Method for Spaceborne SAR Based on Joint Multi-Azimuth Frequency Estimation" (Zhou Chaowei, Li Zhenfang, Wang Yuekun, Xie Jinwei. A Three-Dimensional Imaging Method for Spaceborne SAR Based on Joint Multi-Azimuth Frequency Estimation [J]. Journal of Radar, 2019, 15(9): 935-939.), the method for achieving three-dimensional imaging includes: dividing the full aperture data into sub-aperture sequences; performing two-dimensional imaging on the sub-aperture sequences; selecting a reference sub-aperture from the sub-aperture images, dividing the reference sub-aperture into blocks and statistically analyzing the pixel amplitude within each block, and extracting strong scattering points; taking several image blocks from each sub-aperture as the center of the strong scattering point, performing sampling registration on each image block, and calculating the offset of the image block containing the same strong scattering point in adjacent sub-apertures; calculating the elevation error of adjacent sub-apertures based on the offset using a preset elevation error estimation function, correcting the elevation value of the strong scattering point based on the elevation error, and determining its three-dimensional coordinates; obtaining the three-dimensional coordinates at different azimuth angles and fusing them to obtain the final three-dimensional image.

[0005] This method offers higher elevation measurement accuracy compared to single-azimuth observation when the azimuth span is small, and it requires lower time and economic costs compared to InSAR (Interferometric SAR) and TomoSAR (Tomographic SAR) techniques. However, this method requires the extraction of strong scattering points, so it becomes unsuitable when the azimuth span exceeds 90 degrees, as the scattering characteristics of the observed target change significantly with the azimuth angle. Summary of the Invention

[0006] To address the aforementioned problems in the prior art, this invention provides a method for three-dimensional reconstruction of target projection based on multi-azimuth synthetic aperture radar imagery.

[0007] The technical problem to be solved by this invention is achieved through the following technical solution:

[0008] A method for three-dimensional reconstruction of target projection based on multi-azimuth synthetic aperture radar imagery includes:

[0009] Referring to the elevation information in the digital elevation model (DEM) of the area where the observation target is located, the oblique plane multi-azimuth SAR image set is projected into a ground plane multi-azimuth SAR image set; the oblique plane multi-azimuth SAR image set contains the observation target.

[0010] Target images are extracted from the aforementioned multi-azimuth SAR image set to obtain the multi-azimuth target image set.

[0011] The ground plane multi-azimuth target image set is binarized to obtain the ground plane multi-azimuth mask image set;

[0012] Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, the initial center position of the observed target is calculated, and an initial three-dimensional structural model of the observed target is constructed with the initial center position as the center.

[0013] Calculate the coordinates of the oblique projection point of each structural unit in the initial three-dimensional structural model on the ground plane; wherein the coordinates of the oblique projection point are the intersection of a circular arc with equal oblique distance and the ground plane, and the circular arc with equal oblique distance is located in the plane defined by the radar, the structural unit, and the vertical projection point of the structural unit on the ground plane.

[0014] The authenticity of the oblique projection point coordinates of each structural unit is verified using the multi-azimuth mask image set, and the true three-dimensional structural model of the observed target is determined based on the verification results.

[0015] Three-dimensional reconstruction is performed based on the real three-dimensional structural model.

[0016] Optionally, the elevation information in the digital elevation model (DEM) of the area where the reference observation target is located is used to project the oblique plane multi-azimuth SAR image set into a ground plane multi-azimuth SAR image set, including:

[0017] Obtain radar observation information corresponding to pixels in oblique plane multi-azimuth SAR images;

[0018] The radar observation information corresponding to the acquired pixels is substituted into a set of equations consisting of the rotating ellipsoid equation, the synthetic aperture radar slant range equation, and the synthetic aperture radar Doppler equation to obtain the position information of the pixel's projection on the ground plane; wherein, the rotating ellipsoid equation uses the elevation information in the digital elevation model (DEM) of the area where the observed target is located.

[0019] Based on the position information of the pixels of the oblique plane multi-azimuth SAR image projected onto the ground plane, the oblique plane multi-azimuth SAR image is projected onto the ground plane to obtain the ground plane multi-azimuth SAR image set.

[0020] Optionally, the multi-azimuth target image set is subjected to binarization processing, including:

[0021] The average pixel value is calculated for several pixels located at the same position in the multi-azimuth target image set, and the average pixel value is used to perform binarization processing on the several pixels.

[0022] Optionally, calculating the initial center position of the observed target based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set includes:

[0023] Based on the pixel correspondence between the ground plane multi-azimuth target image set and the ground plane multi-azimuth SAR image set, the coordinates of the center of the observed target in the ground plane multi-azimuth target image set in the ground plane multi-azimuth SAR image set are determined, and the average of the determined coordinates is taken as the target ground plane coordinates.

[0024] Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, the target oblique plane coordinates corresponding to the target ground plane coordinates are determined as the initial center position of the observed target.

[0025] Optionally, the three-dimensional structure of the initial three-dimensional structural model is a cube or cubic structure.

[0026] Optionally, calculating the coordinates of the oblique projection point of each structural unit in the initial three-dimensional structural model on the ground plane includes:

[0027] The coordinates of the projection point of each structural unit in the initial three-dimensional structural model in the ground plane are calculated using a pre-built projection relationship function;

[0028] The projection relationship function is:

[0029]

[0030] Where α is the plane angle of the radar's slant range vector to the observed target in the three-dimensional coordinate system, β is the pitch angle of the observed target's velocity vector in the three-dimensional coordinate system, and θ is the pitch angle of the radar's slant range vector to the observed target in the three-dimensional coordinate system. The plane angle of the velocity vector of the observed target in the triangular coordinate system; the three-dimensional coordinate system uses the ground plane as the XOY plane; T0 is the coordinate of the vertical projection point of the structural unit on the ground plane; h T T represents the elevation value of the observed target relative to the XOY plane. proj These are the coordinates of the oblique projection points of the structural unit onto the ground plane.

[0031] Optionally, the authenticity of the projection point coordinates of each structural unit is verified using the multi-azimuth mask image set, including:

[0032] The authenticity of the projection point coordinates of each structural unit is evaluated using a preset structural unit authenticity evaluation function; the verification is passed when the evaluation score is not lower than the threshold, otherwise the verification fails.

[0033] The authenticity evaluation function for the structural unit is:

[0034]

[0035] Among them, (x i,y i ,z i F(x) represents the three-dimensional coordinates of the i-th structural unit in the initial three-dimensional structural model; i ,y i ,z i (x', y') represents the evaluation score of the structural unit; (x', y') represents the pixel position of the oblique projection point coordinates of the i-th structural unit in the m-th mask image; f m (x',y') is the pixel value located at (x',y') in the m-th mask image.

[0036] Optionally, the authenticity of the projection point coordinates of each structural unit is verified using the multi-azimuth mask image set, including:

[0037] The authenticity of the projection point coordinates of each structural unit is evaluated using a preset structural unit authenticity evaluation function; the verification is passed when the evaluation score is not lower than the threshold, otherwise the verification fails.

[0038] The authenticity evaluation function for the structural unit is:

[0039]

[0040] Among them, (x i ,y i ,z i F(x) represents the three-dimensional coordinates of the i-th structural unit in the initial three-dimensional structural model; i ,y i ,z i (x', y') represents the evaluation score of the structural unit; (x', y') represents the pixel position of the oblique projection point coordinates of the i-th structural unit in the m-th mask image; f m (x',y') is the pixel value located at (x',y') in the m-th mask image; α m Let be the importance coefficient corresponding to the m-th mask image. The importance coefficient corresponding to the mask image is determined by the image quality of the azimuth target image corresponding to the mask image.

[0041] Optionally, when the image quality of the azimuth target image corresponding to the mask image meets the requirements, the importance coefficient of the mask image is 1; when the image quality of the azimuth target image corresponding to the mask image does not meet the requirements, the importance coefficient of the mask image is 0.

[0042] This invention also provides a method for three-dimensional reconstruction of target projection based on multi-azimuth synthetic aperture radar imagery, comprising:

[0043] Referring to the elevation information in the Digital Elevation Model (DEM) of the area where the observation target is located, the oblique plane multi-azimuth SAR image set is projected into a ground plane multi-azimuth SAR image set; the oblique plane multi-azimuth SAR image set contains the observation target; the observation target includes multiple targets.

[0044] Target images are extracted from the aforementioned multi-azimuth SAR image set to obtain a multi-azimuth target image set for each observed target;

[0045] The ground plane multi-azimuth target image set of each observed target is binarized to obtain the ground plane multi-azimuth mask image set of that observed target;

[0046] Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, the initial center position of each observed target is calculated, and the initial three-dimensional structural model of the observed target is constructed with the initial center position as the center.

[0047] For each observation target, calculate the coordinates of the oblique projection point of each structural unit in its initial three-dimensional structural model on the ground plane; wherein, the coordinates of the oblique projection point are the intersection of a circular arc with equal oblique distance and the ground plane, and the circular arc with equal oblique distance is located in the plane defined by the radar, the structural unit, and the vertical projection point of the structural unit on the ground plane.

[0048] The authenticity of the oblique projection point coordinates of each structural unit in the initial three-dimensional structural model of each observed target is verified by using a multi-azimuth mask image set of each observed target, and the true three-dimensional structural model of the observed target is determined based on the verification results.

[0049] Three-dimensional reconstruction is performed based on the real three-dimensional structural models of the multiple observed targets.

[0050] The target projection 3D reconstruction method based on multi-azimuth synthetic aperture radar imagery provided by this invention improves the success rate of 3D reconstruction when the azimuth span is greater than 90 degrees through various technical means, including:

[0051] (1) By projecting the oblique plane multi-azimuth SAR image set onto the same ground plane, there is no need to register images or feature points based on the coherence between the oblique plane multi-azimuth SAR image sets.

[0052] (2) Extract target images from the multi-azimuth SAR image set of the ground plane and perform binarization processing on the multi-azimuth target image set of the ground plane. This operation is applied to all pixel values ​​in the target image, without relying on certain specific strong scattering points. Therefore, it gets rid of the dependence on strong scattering points, making the present invention applicable to targets with strong scattering points or targets with uniform scattering.

[0053] (3) Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, the initial center position of the observed target was calculated, and the initial three-dimensional structural model of the observed target was constructed with the initial center position as the center. Then, the authenticity of the oblique projection point coordinates of each structural unit in the initial three-dimensional structural model was verified by using the multi-azimuth mask image set obtained by binarization. In this process, the judgment result of whether the oblique projection point coordinates of the structural unit fall within the mask is used. Therefore, the requirement for whether the scattering characteristics of the observed target change with the azimuth angle is much more relaxed.

[0054] In summary, this invention does not require image or feature point registration, does not rely on strong scattering points, and has relaxed requirements on the scattering characteristics of the observed target, thus improving the success rate of three-dimensional reconstruction when the azimuth span angle is greater than 90 degrees.

[0055] Furthermore, the present invention can construct the initial three-dimensional structural model of the observed target as a cube or cubic structure. This means that the present invention does not require prior information such as the size and structure of the observed target, except for the prior information of the elevation information in the digital elevation model (DEM) of the region where the observed target is located. In other words, the dependence of the present invention on prior information is greatly reduced, thereby enabling three-dimensional reconstruction of unknown targets.

[0056] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar imagery provided by an embodiment of the present invention;

[0058] Figure 2 This is a flowchart of a target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar imagery provided by an embodiment of the present invention;

[0059] Figure 3 This is a projection relationship diagram between the observed target and the ground plane in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the pitch angle and plane angle corresponding to the slant range vector in an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the pitch angle and plane angle corresponding to the velocity vector in an embodiment of the present invention. Detailed Implementation

[0062] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0063] To improve the success rate of 3D reconstruction when the azimuth span is greater than 90 degrees, this invention provides a target projection 3D reconstruction method based on multi-azimuth synthetic aperture radar imagery. The implementation principle of this method is as follows: Figure 1 As shown, a cuboid is used to represent the initial three-dimensional structural model of the observed target, and the gray cylinder represents the actual three-dimensional structure of the observed target. Without loss of generality, Figure 1 Only two different azimuth angle observation scenarios are shown—observation azimuth A and observation azimuth B—and the target projections for observation azimuth angles A and B are given. The key to the implementation principle of this invention is determining whether each structural unit in the initial 3D structural model belongs to the observation target. When a structural unit does not belong to the observation target, such as... Figure 1 The light-colored structural units in the image will only have their projections fall within the projections of a portion of the azimuth observation target on the ground plane; when the structural unit belongs to the observation target, such as... Figure 1 The dark structural units in the image will have their projections on the ground plane fall within the projections of all observed azimuth targets. Therefore, this embodiment of the invention achieves three-dimensional reconstruction of the observed target by traversing all structural units.

[0064] The implementation process of the embodiments of the present invention will be described in detail below. Figure 2 As shown, the target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar imagery provided in this embodiment of the invention includes the following steps:

[0065] S10: Referencing the elevation information in the digital elevation model (DEM) of the area where the observed target is located, project the oblique plane multi-azimuth SAR image set into a ground plane multi-azimuth SAR image set; the oblique plane multi-azimuth SAR image set contains the observed target.

[0066] The oblique plane multi-azimuth SAR image set is the SAR image set detected by synthetic aperture radar under multi-azimuth span. Multi-azimuth span can greatly reduce the blind zone problem caused by single-view observation, improve the accuracy of target 3D reconstruction, and enhance the scattering integrity and observation stereoscopicity of the reconstructed target. In addition to generating SAR images, the radar also generates a parameter file to record radar observation information, including radar position, velocity, Doppler center, and slant range.

[0067] Specifically, step S10 includes:

[0068] (1) Obtain radar observation information corresponding to the pixels of the oblique plane multi-azimuth SAR image;

[0069] (2) Substitute the radar observation information corresponding to the obtained pixel into the system of equations consisting of the rotating ellipsoid equation, the synthetic aperture radar slant range equation and the synthetic aperture radar Doppler equation to obtain the position information of the pixel on the ground plane projection; wherein, the elevation information in the digital elevation model (DEM) of the area where the observed target is located is used in the rotating ellipsoid equation.

[0070] (3) Based on the position information of the pixel points of the oblique plane multi-azimuth SAR image projected onto the ground plane, the oblique plane multi-azimuth SAR image is projected onto the ground plane to obtain the ground plane multi-azimuth SAR image set.

[0071] In step (2) above, the equation of the rotating ellipsoid is:

[0072]

[0073] In this equation, R e R is the major radius of the Earth's ellipsoid. p The minor radius of the Earth's ellipsoid is a known quantity; h is the elevation information in the Digital Elevation Model (DEM) of the area where the observation target is located; (X t ,Y t Z t ) represents the pixel coordinates of the SAR image in the ground plane multi-azimuth SAR image set, which is the coordinate of the pixel points in the oblique plane multi-azimuth SAR image set projected onto the ground plane, that is, the position information of the pixel points in the oblique plane multi-azimuth SAR image set projected onto the ground plane.

[0074] The slant range equation for synthetic aperture radar is:

[0075]

[0076] In this equation, The position vector of the synthetic aperture radar in the Earth's fixed coordinate system can be constructed based on the radar position information recorded by the radar. This is the slant distance vector.

[0077] Because the satellite mounted on the radar moves radially relative to the ground objects, the frequency of the electromagnetic waves scattered by the ground objects to the radar will be displaced. The magnitude of the displacement is proportional to the relative velocity between the satellite and the ground objects, expressed as:

[0078]

[0079] Among them, f DC The frequency is the Doppler frequency. λ is the velocity vector of the radar relative to the observed target, which can be constructed based on the velocity information recorded by the radar; λ is the wavelength of the radar.

[0080] Given the known fDC , λ, R e R p By substituting the parameters into the system of equations consisting of the three equations above, the solution can be obtained. This involves projecting pixels from a slant-plane multi-azimuth SAR image onto pixels in a ground-plane multi-azimuth SAR image. Specifically, it involves projecting pixels from the slant-plane SAR image onto pixels in the ground-plane multi-azimuth SAR image. Pixels with time-corresponding relationships are assigned to the median of the ground-level SAR image. The pixels at the location are used to obtain the multi-azimuth SAR image of the ground plane, and then the multi-azimuth SAR image set of the ground plane is obtained.

[0081] In addition, in practical applications, the raw SAR echoes from the radar can be used to directly image the ground plane, thereby obtaining a multi-azimuth SAR image set for the ground plane. Specifically, a ground plane grid for imaging the observation scene is established with the center of the observation scene as the origin, and the ground plane is directly imaged using an imaging algorithm. This imaging method is relatively complex compared to the projection method mentioned in step S10. Furthermore, in this embodiment of the invention, the projection relationship obtained after projecting the oblique plane multi-azimuth SAR image set into a ground plane multi-azimuth SAR image set will be used in subsequent steps. Therefore, considering the difficulty of implementation and execution efficiency, this embodiment of the invention chooses the projection method to obtain the ground plane multi-azimuth SAR image set.

[0082] S20: Extract target images from the multi-azimuth SAR image set on the ground plane to obtain the multi-azimuth target image set on the ground plane.

[0083] Specifically, the target detection method based on visual attention extracts the region where the target is located from each SAR image in the multi-azimuth SAR image set. Using the center of the extracted target as the image center, a target image of the same size is generated. After processing the entire multi-azimuth SAR image set, a distributed multi-azimuth target image set can be obtained.

[0084] S30: Binarize the multi-azimuth target image set on the ground plane to obtain the multi-azimuth mask image set on the ground plane.

[0085] Specifically, the average pixel value is calculated for several pixels located at the same position in the multi-azimuth target image set, and this average pixel value is used to binarize these pixels. For example, after calculating the average pixel value, if a pixel at these positions is less than the average pixel value, then that pixel is assigned a value of 0; otherwise, it is assigned a value of 1.

[0086] S40: Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, calculate the initial center position of the observed target, and construct the initial three-dimensional structural model of the observed target with this initial center position as the center.

[0087] Specifically, step S40 includes:

[0088] (1) Based on the pixel correspondence between the multi-azimuth target image set and the multi-azimuth SAR image set, determine the coordinates of the center of the observed target in the multi-azimuth target image set in the multi-azimuth SAR image set, and average the determined coordinates as the target's ground plane coordinates.

[0089] (2) Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, determine the target oblique plane coordinates corresponding to the target ground plane coordinates, and use them as the initial center position of the observed target.

[0090] It is understandable that by performing the above step S10, the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set has been obtained. Therefore, in this step (2), it is not necessary to use the above equations to solve again. The target oblique plane coordinates corresponding to the target ground plane coordinates can be determined directly based on the projection relationship obtained in step S10, and thus used as the initial center position of the observation target. Then, the initial three-dimensional structural model of the observation target is constructed with this initial center position as the center.

[0091] In one implementation, if the specific type of the observed target is unknown, an initial 3D structural model of a cube or cubic structure can be constructed. In another implementation, if the specific type of the observed target is known, an initial 3D structural model similar in shape to the actual observed target can be constructed. In practical applications, the volume of the constructed initial 3D structural model can be set slightly larger, just enough to encompass the actual observed target.

[0092] S50: Calculate the coordinates of the oblique projection point of each structural unit in the initial three-dimensional structural model on the ground plane; wherein, the coordinates of the oblique projection point are the intersection of a circular arc with equal oblique distance and the ground plane, and the circular arc with equal oblique distance is located in the plane defined by the radar, the structural unit, and the vertical projection point of the structural unit on the ground plane.

[0093] For any observation azimuth, the relationship between the observed target and its projection on the ground plane is as follows: Figure 3 As shown, the X and Y axes constitute the ground plane, the Z axis is the height direction, T is the true position vector of the observed target, and its elevation value relative to the XOY plane is h. TIts vertical projection point on the ground plane is T0. This is the slant range vector from the satellite carried by the radar to the observed target. Through SAR imaging and slant-plane projection onto the ground plane, the observed target is projected onto the ground plane as T. proj At point D, the projection vector of the observed target onto the ground plane is D. proj The vertical and horizontal components of the projection vector are D respectively. proj,v With D proj,h The intersection point of the equidistant circular arc L with the ground plane is T. proj It is located in the radar (mounted on) Figure 3 The plane defined by the satellite in the drawing, the observation target (position vector T), and the vertical projection point T0 of the observation target on the ground plane.

[0094] like Figure 4 The slant range vector is shown. In a three-dimensional coordinate system composed of the XYZ axes, the pitch angle and the plane angle are θ and α, respectively; for example Figure 5 As shown, velocity vector The pitch angle and plane angle in the three-dimensional coordinate system are β and β, respectively.

[0095] Therefore, the slant range vector With velocity vector It can be represented by two sets of pitch angles and plane angles:

[0096]

[0097] in, The magnitude of the slant range vector. This represents the magnitude of the satellite's velocity vector.

[0098] Let u R Slant distance vector The unit vector, u V velocity vector unit vector, u proj For the projection vector D proj The unit vector.

[0099] Based on the projection relationship between the observed target and the ground plane, we can obtain:

[0100]

[0101] Let u proj,v and u proj,h They are D proj,v With D proj,h The unit vector, as shown in the above equation, has the following expressions:

[0102]

[0103]

[0104] According to u proj,v and u proj,h From the expression, we can see that the coordinates of the projection point of the observed target in the XOY plane are:

[0105]

[0106] Therefore, in step S50, the T given here can be used directly. proj The expression is used to calculate the coordinates of the oblique projection point of each structural unit in the initial three-dimensional structural model onto the ground plane. Where α is the plane angle of the radar's slant range vector to the observed target in the three-dimensional coordinate system, β is the pitch angle of the observed target's velocity vector in the three-dimensional coordinate system, and θ is the pitch angle of the radar's slant range vector to the observed target in the three-dimensional coordinate system. The plane angle of the velocity vector of the observed target in the triangular coordinate system; this three-dimensional coordinate system uses the ground plane as the XOY plane; T0 is the coordinate of the vertical projection point of the structural unit on the ground plane, which can be obtained from the coordinates of the structural unit in the initial three-dimensional structural model; h T T represents the elevation of the target relative to the XOY plane. proj These are the coordinates of the oblique projection points of the structural unit onto the ground plane.

[0107] S60: Verify the authenticity of the oblique projection point coordinates of each structural unit using a multi-azimuth mask image set, and determine the true three-dimensional structural model of the observed target based on the verification results.

[0108] The verification of the authenticity of the oblique projection point coordinates of each structural unit is carried out using a multi-azimuth mask image set. This includes: evaluating the authenticity of the projection point coordinates of each structural unit using a preset structural unit authenticity evaluation function; verification is passed when the evaluation score is not lower than the threshold, otherwise verification fails.

[0109] In one implementation, the structural unit authenticity evaluation function can be:

[0110]

[0111] Among them, (x i ,y i ,z i F(x) represents the three-dimensional coordinates of the i-th structural unit in the initial three-dimensional structural model; i ,y i ,z i (x', y') represents the evaluation score for the structural unit; (x', y') represents the pixel position in the m-th mask image where the oblique projection point coordinates of the i-th structural unit fall; fm (x',y') is the pixel value located at (x',y') in the m-th mask image.

[0112] In another implementation, the structural unit authenticity evaluation function can be:

[0113]

[0114] Where, α m is the importance coefficient corresponding to the m-th mask image. The importance coefficient corresponding to the mask image is determined by the image quality of the azimuth target image corresponding to the mask image. For the explanation of the other parameters, please refer to the authenticity evaluation function of the previous structural unit.

[0115] For example, when the image quality of the azimuth target image corresponding to the mask image meets the requirements, the importance coefficient of the mask image is 1; when the image quality of the azimuth target image corresponding to the mask image does not meet the requirements, the importance coefficient of the mask image is 0. The image quality of the azimuth target image corresponding to the mask image can be evaluated using PSNR (Peak-Signal to Noise Ratio) or MSE (Mean Square Error), but is not limited to these methods.

[0116] After obtaining the verification results, if the authenticity verification of a structural unit passes, the structural unit is retained; if the verification fails, the structural unit is discarded. The retained structural units ultimately form the true 3D structural model of the observed target. This process can be understood as sculpting the initial 3D structural model based on the verification results of the structural units, ultimately obtaining the true 3D structural model of the observed target.

[0117] S70: Reconstruct the observed target in three dimensions based on a real three-dimensional structural model.

[0118] Specifically, 3D reconstruction can be achieved by visualizing the real 3D structural model. Alternatively, the real 3D structural model can be visualized together with the observed scene, and color rendering can be applied to the real 3D structural model.

[0119] The target projection 3D reconstruction method based on multi-azimuth synthetic aperture radar imagery provided in this invention improves the success rate of 3D reconstruction when the azimuth span is greater than 90 degrees through various technical means, including:

[0120] (1) By projecting the oblique plane multi-azimuth SAR image set onto the same ground plane, there is no need to register images or feature points based on the coherence between the oblique plane multi-azimuth SAR image sets.

[0121] (2) After being uniformly projected onto the ground plane, the target image is extracted from the multi-azimuth SAR image set of the ground plane, and the multi-azimuth target image set of the ground plane is binarized. This operation deals with all pixel values ​​in the target image, without relying on certain specific strong scattering points. Therefore, it gets rid of the dependence on strong scattering points, which makes the embodiments of the present invention applicable to targets with strong scattering points or targets with uniform scattering.

[0122] (3) Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, the initial center position of the observed target was calculated, and the initial three-dimensional structural model of the observed target was constructed with the initial center position as the center. Then, the authenticity of the oblique projection point coordinates of each structural unit in the initial three-dimensional structural model was verified by using the multi-azimuth mask image set obtained by binarization. In this process, the judgment result of whether the oblique projection point coordinates of the structural unit fall within the mask is used. Therefore, the requirement for whether the scattering characteristics of the observed target change with the azimuth angle is much more relaxed.

[0123] In summary, the embodiments of the present invention do not require image or feature point registration, do not rely on strong scattering points, and have relaxed requirements on the scattering characteristics of the observed target, thus improving the success rate of three-dimensional reconstruction when the azimuth span angle is greater than 90 degrees.

[0124] Furthermore, embodiments of the present invention can construct the initial three-dimensional structural model of the observed target as a cube or cubic structure. This means that, apart from using the prior information of the elevation information in the digital elevation model (DEM) of the region where the observed target is located, embodiments of the present invention do not require prior information such as the size and structure of the observed target. In other words, the dependence of the present invention on prior information is greatly reduced, thereby enabling three-dimensional reconstruction of unknown targets.

[0125] Based on the same inventive concept, embodiments of the present invention also provide another method for target projection three-dimensional reconstruction based on multi-azimuth synthetic aperture radar imagery, including:

[0126] (1) Referencing the elevation information in the digital elevation model (DEM) of the area where the observation target is located, the oblique plane multi-azimuth SAR image set is projected into a ground plane multi-azimuth SAR image set; the oblique plane multi-azimuth SAR image set contains the observation target; the observation target includes multiple targets.

[0127] (2) Extract target images from the multi-azimuth SAR image set on the ground plane to obtain the multi-azimuth target image set on the ground plane for each observed target;

[0128] (3) Binarize the ground plane multi-azimuth target image set of each observed target to obtain the ground plane multi-azimuth mask image set of the observed target;

[0129] (4) Based on the projection relationship between the oblique plane multi-azimuth SAR image set and the ground plane multi-azimuth SAR image set, calculate the initial center position of each observation target, and construct the initial three-dimensional structure model of the observation target with the initial center position as the center.

[0130] (5) For each observation target, calculate the coordinates of the oblique projection point of each structural unit in its initial three-dimensional structural model on the ground plane; wherein, the coordinates of the oblique projection point are the intersection of an equal oblique distance circular arc and the ground plane, and the equal oblique distance circular arc is located in the plane defined by the radar, the structural unit and the vertical projection point of the structural unit on the ground plane.

[0131] (6) Use the multi-azimuth mask image set of each observation target to verify the authenticity of the oblique projection point coordinates of each structural unit in its initial three-dimensional structural model, and determine the real three-dimensional structural model of the observation target based on the verification results.

[0132] (7) Three-dimensional reconstruction is performed based on the real three-dimensional structural model of multiple observation targets.

[0133] It is understood that the difference between this method embodiment and the previous method embodiment is that the observed targets include multiple targets; these targets exist simultaneously in the oblique plane multi-azimuth SAR image set, and therefore also exist simultaneously in the ground plane multi-azimuth target image set. In the execution process of this method embodiment, step (1) is the same as step S10 in the previous method embodiment; the execution flow of steps (2) to (6) for each observed target can be referred to steps S20 to S60 in the previous method embodiment; in step (7), after obtaining the real three-dimensional structure model of all distributed observed targets, the overall three-dimensional scene is synthesized according to the relative positional relationship between each observed target.

[0134] It should be noted that the second method embodiment is basically similar to the first method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the first method embodiment.

[0135] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0136] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0137] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0138] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings and the disclosure, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In this specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. While certain measures are described in different embodiments, this does not mean that these measures cannot be combined to produce good results.

[0139] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar image, characterized in that, The method comprises the following steps: projecting the slant plane multi-aspect angle SAR image set into a ground plane multi-aspect angle SAR image set by referring to the elevation information in a digital elevation model (DEM) of the region where the observation target is located; extracting target images from the ground plane multi-aspect angle SAR image set to obtain a ground plane multi-aspect angle target image set; performing binary processing on the ground plane multi-aspect angle target image set to obtain a ground plane multi-aspect angle mask image set; calculating an initial center position of the observation target according to the projection relationship between the slant plane multi-aspect angle SAR image set and the ground plane multi-aspect angle SAR image set, and constructing an initial three-dimensional structure model of the observation target with the initial center position as the center; calculating the slant projection point coordinates of each structure unit in the initial three-dimensional structure model on the ground plane; wherein the slant projection point coordinates are the intersection points of an equal slant distance circular arc and the ground plane, and the equal slant distance circular arc is located in the plane defined by the radar, the structure unit and the vertical projection point of the structure unit on the ground plane; verifying the authenticity of the slant projection point coordinates of each structure unit by using the multi-aspect angle mask image set, and determining a real three-dimensional structure model of the observation target based on the verification result; performing three-dimensional reconstruction based on the real three-dimensional structure model.

2. The method of claim 1, wherein, The method comprises the following steps: obtaining the radar observation information corresponding to the pixel points of the slant plane multi-aspect angle SAR image; solving the equation set composed of a rotating ellipsoid equation, a synthetic aperture radar slant range equation and a synthetic aperture radar Doppler equation by substituting the obtained radar observation information corresponding to the pixel points into the equation set to obtain the position information of the pixel points on the ground plane; wherein the elevation information in the digital elevation model (DEM) of the region where the observation target is located is used in the rotating ellipsoid equation; projecting the slant plane multi-aspect angle SAR image on the ground plane according to the position information of the pixel points of the slant plane multi-aspect angle SAR image on the ground plane to obtain a ground plane multi-aspect angle SAR image set.

3. The method of claim 1, wherein, The method comprises the following steps: calculating the pixel mean value of a plurality of pixels located at the same position in the multi-aspect angle target image set, and performing binary processing on the plurality of pixels by using the pixel mean value.

4. The method of claim 1, wherein, The method comprises the following steps: determining the center of the observation target in the ground plane multi-aspect angle target image set in the ground plane multi-aspect angle SAR image set according to the pixel correspondence relationship between the ground plane multi-aspect angle target image set and the ground plane multi-aspect angle SAR image set, and averaging a plurality of determined coordinates as the target ground plane coordinates. According to a projection relationship between the slant plane multi-aspect SAR image set and the ground plane multi-aspect SAR image set, a target ground plane coordinate corresponding to the target ground plane coordinate is determined as an initial center position of the observation target.

5. The method of claim 1, wherein, The three-dimensional structure of the initial three-dimensional structure model is a cube structure.

6. The method of claim 1, wherein, The slant projection point coordinates of each structural unit in the initial three-dimensional structure model in the ground plane are calculated, including: The slant projection point coordinates of each structural unit in the initial three-dimensional structure model in the ground plane are calculated by using a pre-constructed projection relationship function; The projection relationship function is: Wherein, a is the plane angle of the slant range vector of the radar to the observation target in the three-dimensional coordinate system, β is the pitch angle of the velocity vector of the observation target in the three-dimensional coordinate system; θ is the pitch angle of the slant range vector of the radar to the observation target in the three-dimensional coordinate system, is the plane angle of the velocity vector of the observation target in the three-dimensional coordinate system; the three-dimensional coordinate system takes the ground plane as the XOY plane; T0 is the vertical projection point coordinate of the structural unit on the ground plane; h T is the elevation value of the observation target relative to the XOY plane, T proj is the oblique projection point coordinate of the structural unit on the ground plane.

7. The method of claim 1, wherein, The authenticity of the projection point coordinates of each structural unit is verified by using the multi-aspect mask image set, including: The authenticity of the projection point coordinates of each structural unit is evaluated by using a preset structural unit authenticity evaluation function; when the evaluation score is not lower than a threshold value, the verification is passed, otherwise the verification is failed; The authenticity of the projection point coordinates of each structural unit is evaluated by using a preset structural unit authenticity evaluation function; when the evaluation score is not lower than a threshold value, the verification is passed, otherwise the verification is failed; Among them, (x i ,y i ,z i F(x) represents the three-dimensional coordinates of the i-th structural unit in the initial three-dimensional structural model; i ,y i ,z i (x', y') represents the evaluation score for the structural unit; (x', y') represents the pixel position in the m-th mask image where the oblique projection point coordinates of the i-th structural unit fall; f m (x',y') is the pixel value located at (x',y') in the m-th mask image.

8. The method of claim 1, wherein, The authenticity of the projection point coordinates of each structural unit is evaluated by using a preset structural unit authenticity evaluation function; when the evaluation score is not lower than a threshold value, the verification is passed, otherwise the verification is failed; When the image quality of the azimuth target image corresponding to the mask image meets the requirements, the importance coefficient of the mask image is 1; when the image quality of the azimuth target image corresponding to the mask image does not meet the requirements, the importance coefficient of the mask image is 0. Including: wherein (x i ,y i ,z i ) is the three-dimensional coordinate of the i-th structural unit in the initial three-dimensional structure model; F(x i ,y i ,z i ) is the evaluation score of the structural unit; (x', y') is the pixel position of the oblique projection point coordinate of the i-th structural unit in the m-th mask image, f m (x', y') is the pixel value of the m-th mask image at (x', y'); and α m is the importance coefficient corresponding to the m-th mask image, which is determined by the image quality of the azimuthal target image corresponding to the mask image.

9. The method of claim 8, wherein, The slant plane multi-aspect SAR image set is projected into a ground plane multi-aspect SAR image set by referring to the elevation information in a digital elevation model (DEM) of a region where the observation target is located; the slant plane multi-aspect SAR image set contains the observation target; the observation target includes a plurality of 10. A target projection three-dimensional reconstruction method based on multi-azimuth synthetic aperture radar images, characterized in that, Target images are extracted from the ground plane multi-aspect SAR image set to obtain a ground plane multi-aspect target image set of each observation target; The ground plane multi-aspect target image set of each observation target is subjected to binary processing to obtain a ground plane multi-aspect mask image set of the observation target; According to a projection relationship between the slant plane multi-aspect SAR image set and the ground plane multi-aspect SAR image set, an initial center position of each observation target is calculated, and an initial three-dimensional structure model of the observation target is constructed with the initial center position as the center; For each observation target, the slant projection point coordinates of each structural unit in the initial three-dimensional structure model in the ground plane are calculated; the slant projection point coordinates are intersection points of an equal slant distance circular arc and the ground plane, and the equal slant distance circular arc is located in a plane defined by a radar, a structural unit, and a vertical projection point of the structural unit in the ground plane; The authenticity of the slant projection point coordinates of each structural unit in the initial three-dimensional structure model of each observation target is verified by using the multi-aspect mask image set of the observation target, and a real three-dimensional structure model of the observation target is determined based on the verification result; ​ ​ performing three-dimensional reconstruction based on the real three-dimensional structure model of the plurality of observed targets.

Images