Non-line-of-sight corner building layout three-dimensional reconstruction method suitable for sparse array radar

Abstract

The invention discloses a non-line-of-sight corner building layout three-dimensional reconstruction method suitable for a sparse array radar, is applied to the technical field of non-line-of-sight imaging, and aims at processing a uniform linear array (ULA) receiving signal based on a traditional matched filter algorithm in the prior art. And estimation of building layout parameters in a non-line-of-sight corner is realized through a post-processing algorithm, and the method is difficult to adapt to a vehicle-mounted radar for configuring a distributed sparse linear array (SLA). According to the method, high-resolution reconstruction of radar sparse array receiving signals with different heights is realized by utilizing a convex optimization algorithm, height information of a target is derived through a chromatography method, and finally, identification and relocation of the non-line-of-sight building layout in the acquired three-dimensional point cloud are realized based on an established non-line-of-sight building layout model.

Description

Technical Field

[0001] This invention belongs to the field of non-line-of-sight imaging technology, and specifically relates to a non-line-of-sight corner building layout three-dimensional reconstruction technology. Background Technology

[0002] Non-line-of-sight imaging has attracted widespread attention in recent years. Because it can use the multipath propagation effect of electromagnetic waves to sense non-cooperative targets such as buildings and pedestrians at non-line of sight, it greatly expands the detection range of traditional radar and has great potential to become a key technology in the fields of next-generation assisted driving and autonomous driving.

[0003] Numerous research institutions both domestically and internationally have conducted extensive studies on the reconstruction of non-line-of-sight (NLOS) corner building layouts. The paper "NLOS Building Layout and Target Estimation in an L-Shaped Corner With Complex Geometries, IEEE Trans. Instrum. Meas., Nov. 2024" utilizes the coupling characteristics between NLOS building layouts and target echoes at corners, and leverages multi-frame target echoes to dynamically estimate NLOS building layout parameters. The paper "A hybrid approach for extending automotive radar operation to NLOSurban scenarios, IEEE Trans. Aerosp. Electron. Syst., Aug. 2025" proposes using deep learning networks to identify line-of-sight walls in a scene and then dividing the scene into line-of-sight and non-line-of-sight regions based on the acquired line-of-sight wall parameters.

[0004] The aforementioned methods all rely on traditional matched filter algorithms to process the received signals from uniform linear arrays (ULA) and use post-processing algorithms to estimate building layout parameters in non-line-of-sight corners. However, these methods are difficult to apply to vehicle-mounted radars configured with distributed sparse linear arrays (SLA). In real-world scenarios, non-line-of-sight echo signals are often extremely weak, making it difficult for traditional methods to extract non-line-of-sight signals while maintaining high resolution. Furthermore, most existing methods can only achieve two-dimensional reconstruction of non-line-of-sight corner scenes, failing to distinguish targets at different elevations, making the identification and estimation of non-line-of-sight layouts even more challenging in complex urban environments. Therefore, researching a three-dimensional reconstruction method for non-line-of-sight corner building layouts suitable for sparse array radars has significant practical implications. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a three-dimensional reconstruction method for non-line-of-sight corner building layouts suitable for sparse array radars. This method utilizes a convex optimization algorithm to achieve high-resolution reconstruction of signals received by sparse array radars at different heights, derives the target's height information through tomography, and finally identifies and relocates the non-line-of-sight building layouts in the acquired three-dimensional point cloud based on the established non-line-of-sight building layout model.

[0006] The technical solution adopted in this invention is: a three-dimensional reconstruction method for non-line-of-sight corner building layouts suitable for sparse array radar, comprising the following steps:

[0007] S1. Construct a non-line-of-sight corner building model, including an L-shaped wall formed by the first wall and the second wall, and a third wall parallel to the second wall. The surfaces of the three walls are all perpendicular to the ground. The radar is set directly in front of the first wall. Then the second wall is a non-line-of-sight wall and the third wall is a line-of-sight wall.

[0008] Let the normal vector of the viewing distance wall and the Cartesian coordinate system be given. The angle between the positive axis and the positive axis is The normal vector of the non-line-of-sight wall and the Cartesian coordinate system The angle between the positive axis and the positive axis is ;

[0009] The scattering point on the wall at the line of sight satisfies: the x-coordinate of the scattering point and The product of the cosine values ​​plus the ordinate of the scattering point and The sum of the products of the sine values ​​is the perpendicular distance between the origin of the coordinate system and the wall at that viewing distance;

[0010] The scattering point on the non-line-of-sight wall surface satisfies: the x-coordinate of the scattering point and The product of the cosine values ​​plus the ordinate of the scattering point and The sum of the products of the sine values ​​is the perpendicular distance between the origin of the coordinate system and the non-visual wall surface;

[0011] S2, based on the non-line-of-sight corner building model, derives a non-line-of-sight signal model suitable for sparse arrays;

[0012] S3. Based on the non-line-of-sight signal model suitable for sparse arrays established in S2, the convex optimization method is used to realize two-dimensional imaging of the scene. Furthermore, the pitch information of the target is derived from the phase difference of the image set in the height dimension, thereby obtaining the point cloud position.

[0013] S4. Based on the non-line-of-sight corner 3D point cloud obtained in S3, combined with the non-line-of-sight corner building model established in S1, the non-line-of-sight building layout is identified and relocated.

[0014] The beneficial effects of this invention are as follows: This invention provides a 3D reconstruction method for non-line-of-sight corner building layouts applicable to sparse array radar. It enables high-resolution 3D tomographic imaging of non-line-of-sight corner scenes and derives the line-of-sight and non-line-of-sight building layout parameters within the corner scene. Specifically, a non-line-of-sight corner building model is first established, parameterizing the geometric structure of typical buildings. Based on this, a non-line-of-sight corner receiving signal model for a sparse multiple-input multiple-output (MIMO) radar is constructed. High-resolution 3D imaging of the non-line-of-sight corner scene is achieved through convex optimization methods. Finally, based on the constructed scene building model, the identification and relocation of the non-line-of-sight building layout are realized. The method of this invention has the following advantages:

[0015] 1. This invention can be used for high-resolution three-dimensional tomography imaging of sparse arrays;

[0016] 2. This invention can realize the three-dimensional reconstruction of the layout of buildings at non-visual corners;

[0017] 3. This invention does not require prior information about the building layout;

[0018] 4. This invention can be applied to fields such as 3D mapping and driver assistance. Attached Figure Description

[0019] Figure 1 This is the processing flow of the method of the present invention.

[0020] Figure 2 This is a schematic diagram of a non-line-of-sight corner scene provided in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of a sparse MIMO radar array antenna provided in an embodiment of the present invention.

[0022] Figure 4 The experimental scenario provided for the embodiments of the present invention.

[0023] in, Figure 4 (a) is a photo of the experimental scene. Figure 4 (b) is a schematic diagram of the experimental scenario.

[0024] Figure 5 The two-dimensional imaging results of a non-line-of-sight corner scene provided in the embodiments of the present invention.

[0025] Figure 6 The execution process of the non-line-of-sight building layout recognition and relocation algorithm provided in the embodiments of the present invention.

[0026] in, Figure 6 (a), (b), and (c) are the results of the first, second, and third iterations of the algorithm, respectively.

[0027] Figure 7The three-dimensional reconstruction result of the non-line-of-sight corner building layout provided in the embodiment of the present invention.

[0028] in, Figure 7 (a) is the first perspective of the reconstruction results. Figure 7 (b) is the second perspective of the reconstruction results. Detailed Implementation

[0029] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0030] like Figure 1 As shown, the present invention provides a method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar, comprising the following steps:

[0031] Step 1: Non-visual-distance corner building model

[0032] Consider as Figure 2 In the scenario shown, each building surface contains multiple effective scattering point targets, where the set of scattering points on the line-of-sight surface is represented as follows: ,in The three-dimensional xyz coordinates of the line-of-sight scattering point. This represents the number of point targets scattered at line-of-sight. Assuming the surface at line-of-sight is perpendicular to the ground, its Householder matrix can be expressed as:

[0033]

[0034] in Represents the identity matrix. For the set of complex numbers, Represents the normal vector of the building surface at the line of sight. For the normal vector and the Cartesian coordinate system The angle along the positive direction of the axis. According to the polar-Cartesian coordinate mapping, for any scattering point on the reflecting surface at the line of sight... Its satisfaction

[0035]

[0036] in This represents the perpendicular distance between the origin of the coordinate system and the wall at view distance.

[0037] Assuming the radar location is The two-way distance between it and each scattering point is given by the following formula.

[0038]

[0039] in This represents the l2 norm.

[0040] The set of scattering points on a non-line-of-sight surface is represented as ,in The three-dimensional coordinates of the non-line-of-sight scattering point. This represents the number of non-line-of-sight scattering point targets. Similarly, we can obtain...

[0041]

[0042] in, , , and Let the Householder matrix, normal vector, and the coordinate system of the normal vector on the non-line-of-sight building surface be represented respectively. The included angle and vertical distance along the positive axis. The two-way distance between the radar and each scattering point is:

[0043]

[0044] Step 2: Non-line-of-sight corner signal model

[0045] Consider a single sparse MIMO radar, such as Figure 3 As shown, the radar transmitting array The array elements are distributed in At different pitch heights, its first The number of launch elements at each elevation altitude is The receiving array is A linear array of elements. The center frequency of the transmitting antenna is... bandwidth is The frequency modulation period is The linear frequency modulated continuous wave (LFMCW) signal is generated by the first... The signals emitted by the high-altitude transmission array are scattered by various targets and then return to the radar to form the received signals. For line-of-sight building surface echo Non-line-of-sight building surface echo Dynamic target interference With Gaussian white noise The superposition of can be represented as

[0046]

[0047] in , The following formula is given.

[0048]

[0049] in , , and These represent the number of transmitted pulses and the number of sampling points, respectively. , and These represent the complex scattering coefficients of line-of-sight targets, the complex scattering coefficients of non-line-of-sight targets, and the reflection attenuation, respectively. , These represent the azimuth and elevation angles of a target at line of sight, respectively. , These represent the azimuth and elevation angles of a non-line-of-sight target, respectively. Indicates the first The elevation baseline length of each transmission array; Indicates the first A height-controlled transmission array guide vector, Indicates the receiving guide vector; The radial velocity of the line-of-sight target relative to the radar. The radial velocity of non-line-of-sight targets relative to the radar is considered to be 0 when the platform is stationary, and the radial velocities of both line-of-sight and non-line-of-sight building surface echoes are considered to be 0. and These represent the sampling interval and the pulse repetition period (PRT), respectively. It represents the Kronecker product.

[0050] To separate echoes from building surfaces from moving target interference and improve the signal-to-noise ratio (SNR), the received signal is first subjected to range-Doppler processing. This is based on the considered beat signal frequency. Doppler frequency , , , and Let these represent the number of beat frequency points and the number of Doppler frequency points, respectively. The corresponding matched filter output can be expressed as:

[0051]

[0052] Where n is the index of slow time sampling, k is the beat frequency index, and z is the Doppler frequency index.

[0053] Since the radial velocity of a static building reflector relative to the radar is zero, subsequent processing only considers echoes from zero Doppler elements. For ease of writing, it will be abbreviated as follows in the following text. Since this invention aims to reconstruct the architectural layout of non-line-of-sight corners, where the echo Doppler is 0, while the Doppler of moving targets is not 0, this invention does not consider dynamic echoes.

[0054] Step 3: High-resolution 3D imaging of non-line-of-sight corner sparse array

[0055] In order to achieve low-complexity, high-resolution imaging of corner scenes using a single sparse array radar, this invention adopts a reconstruction framework of two-dimensional imaging and three-dimensional tomography. That is, firstly, two-dimensional imaging of the scene is achieved using arrays at different heights, and then the elevation information of the target is derived based on the phase difference of the image set in the height dimension.

[0056] Consider the set of angles of interest Two-dimensional imaging is achieved by solving the following set of sparse reconstruction problems.

[0057]

[0058] in, This indicates the number of angles of interest being considered. This represents the spatial spectrum corresponding to the m-th transmission array and the k-th range cell. , , Represents the Khatri-Rao product. Indicate l 2,1 Norm (matrix row norm) and Let the transmit steering matrix and receive steering matrix be represented respectively, and defined as follows:

[0059]

[0060] Solving based on the alternating direction multiplier method framework, by introducing auxiliary variables Rewrite the above problem as

[0061]

[0062] in, As dual variables, For regularization parameters, As a penalty factor, This represents the row norm (l-2, 1 norm) of a matrix. This represents the Frobenius norm.

[0063] Solving for the three variables alternately yields

[0064]

[0065] in , , and They represent , and The OK.

[0066] According to the above iterative solution, the iteration stops when the maximum number of iterations is reached, and the optimized result is derived; in this embodiment, the maximum number of iterations is 50. Based on the optimized result... , The angle corresponding to the non-zero element is the estimated angle. ;distance Derived from the beat frequency, for the first... Each distance unit has a distance value of [number] distance units. , This indicates the speed of electromagnetic wave propagation.

[0067] Assuming optimization results No. If a row element is non-zero, then a target is considered to exist at that angle, and its pitch angle is determined. Derived by the following formula

[0068]

[0069] in Indicates the pitch steering vector. For the first The phase difference of each transmitting antenna array relative to the reference position can be expressed as: , This represents the Hadamard product, and the superscript H indicates the conjugate transpose.

[0070] Based on the estimated azimuth angle Pitch angle and round trip distance Thus, the point cloud locations are obtained:

[0071]

[0072] in The rotation matrix used for coordinate transformation is expressed as:

[0073]

[0074] in This represents the angle between the radar array and the negative x-axis.

[0075] Step 4: Non-line-of-sight building layout identification and relocation

[0076] Since non-line-of-sight buildings are not located in their actual positions in the imaging results, but rather within the shadow area behind the reflecting wall, they still need to be identified and relocated. The resulting 3D point cloud set is denoted as... Projecting all point clouds onto the ground yields a two-dimensional point cloud set. The iterative process for non-line-of-sight building layout identification and relocation is as follows:

[0077] (1) Using the Hough transform to map all point clouds to their polar coordinate space, it can be expressed as:

[0078]

[0079] in Let be the possible polar coordinates corresponding to any wall surface. This represents the perpendicular distance between the line and the origin. This represents the angle between the normal vector of the line and the positive x-axis. For any Traverse the areas of interest If the interval satisfies the formula Then the corresponding Increment the grid point value by 1; set the first grid point value to 1. The cumulative result of the Hough transform in each iteration is denoted as . The corresponding estimated results for the building surface parameters at the line of sight are as follows:

[0080]

[0081] These represent the estimated polar coordinate parameters of the wall at the line-of-sight (LOS) at the t-th iteration.

[0082] (2) The DBSCAN algorithm is used to cluster the point clouds projected onto the polar coordinate grid to distinguish point clouds from different buildings but located on the same straight line. The set of point clouds from the same building and on the same straight line is denoted as . This updates the current set of two-dimensional point clouds in the iteration to... ,in Indicates from set Middle Removal Set The elements contained therein Representing sets The first in The x and y coordinates of each point This indicates the number of points in the set.

[0083] (3) Set The slopes of the lines connecting the two farthest points to the origin are denoted as follows: and This leads to the extraction of non-line-of-sight point clouds located behind the building surface at line-of-sight distance. The Householder transformation is used to map it back to its true location.

[0084]

[0085] in , , Finally, the 2D point cloud set is updated to... , for use in the first The next iteration.

[0086] Through multiple iterations, the final mapping of the building layout within the scene is achieved. That is, the mapping of the building layout within the scene is based on the updated two-dimensional point cloud set when the iteration stops. In this embodiment, the number of iterations is 50.

[0087] The following is a specific implementation of the present invention based on an experimental example.

[0088] Experimental scenarios such as Figure 4 As shown, the sensor used in the experiment was a TI AWR2243 Cascade radar, placed at (0.3 m, 0.5 m). Its transmitting array was a sparse planar array, and its receiving array was a sparse linear array. The array made an angle of 55° with the negative x-axis. The x-coordinates of the line-of-sight wall and the non-line-of-sight wall were 1.2 m and 7.96 m, respectively, with corresponding polar coordinate parameters of... and During the experiment, the radar remained stationary, and the transmitting antenna emitted an LFMCW signal with the following waveform parameters: initial frequency 77 GHz, bandwidth 2.05 GHz, sampling frequency 20 MHz, number of sampling points 512, pulse repetition period 420 μs, and number of pulses 128.

[0089] Figure 5 Two-dimensional high-resolution imaging results of the corner scene are presented, showing the building structure clearly, with the non-line-of-sight wall located behind the line-of-sight wall. Figure 6 The iterative process of the non-line-of-sight building layout recognition and relocation algorithm is presented. Through multiple iterations, this invention can estimate the parameters of buildings, including non-line-of-sight walls, in the entire corner scene. The final estimated parameters are as follows: and After identifying and relocating buildings at both the line-of-sight and non-line-of-sight distances in the corner scene, the 3D reconstruction result of the non-line-of-sight corner is derived, as shown below. Figure 7 As shown, compare it with Figure 4 The optical photographs in (a) show that the present invention can achieve high-resolution three-dimensional reconstruction of the layout of buildings at non-line-of-sight corners.

[0090] Those skilled in the art will recognize that the embodiments described herein are for the purpose of helping to understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar, characterized in that, Includes the following steps: S1. Construct a non-line-of-sight corner building model, including an L-shaped wall formed by the first wall and the second wall, and a third wall parallel to the second wall. The surfaces of the three walls are all perpendicular to the ground. The radar is set directly in front of the first wall. Then the second wall is a non-line-of-sight wall and the third wall is a line-of-sight wall. Let the normal vector of the viewing distance wall and the Cartesian coordinate system be given. The angle between the positive axis and the positive axis is The normal vector of the non-line-of-sight wall and the Cartesian coordinate system The angle between the positive axis and the positive axis is ; The scattering point on the wall at the line of sight satisfies: the x-coordinate of the scattering point and The product of the cosine values ​​plus the ordinate of the scattering point and The sum of the products of the sine values ​​is the perpendicular distance between the origin of the coordinate system and the wall at that viewing distance; The scattering point on the non-line-of-sight wall surface satisfies: the x-coordinate of the scattering point and The product of the cosine values ​​plus the ordinate of the scattering point and The sum of the products of the sine values ​​is the perpendicular distance between the origin of the coordinate system and the non-visual wall surface; S2, based on the non-line-of-sight corner building model, derives a non-line-of-sight signal model suitable for sparse arrays; S3. Based on the non-line-of-sight signal model suitable for sparse arrays established in S2, the convex optimization method is used to realize two-dimensional imaging of the scene. Furthermore, the pitch information of the target is derived from the phase difference of the image set in the height dimension, thereby obtaining the point cloud position. S4. Based on the non-line-of-sight corner 3D point cloud obtained from S3, combined with the non-line-of-sight corner building model established in S1, the non-line-of-sight building layout is identified and relocated.

2. The method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar according to claim 1, characterized in that, The implementation process of step S2 is as follows: The transmitting antenna transmits a linear frequency modulated continuous wave signal, which is generated by the first... The signals emitted by the high-altitude transmission array are scattered by various targets and then return to the radar to form the received signal. For line-of-sight wall echo Non-line-of-sight wall echo Dynamic target interference With Gaussian white noise The superposition; For the received signal Perform distance-Doppler processing; Since the radial velocity of a static building reflector relative to the radar is zero, only echoes from zero Doppler elements are considered. abbreviated as .

3. The method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar according to claim 2, characterized in that, In step S3, two-dimensional imaging is achieved by solving the following set of sparse reconstruction problems: ； in This represents the spatial spectrum corresponding to the m-th transmission array and the k-th range cell. , The total number of transmission arrays, , , Represents the Khatri-Rao product. and These represent the transmit steering matrix and the receive steering matrix, respectively. Indicate l 2,1 Norm; Solving based on the alternating direction multiplier method framework, by introducing auxiliary variables The above sparse reconstruction problem can be rewritten as: ； in As dual variables, For regularization parameters, As a penalty factor; Solving for the three variables alternately yields ； in , , and They represent , and The OK; The solution is obtained through iterative solving. When the number of iterations reaches the set maximum, the iteration stops, and the optimized result is derived. , , .

4. The method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar according to claim 3, characterized in that, The process of obtaining the point cloud position in step S3 is as follows; Based on optimization results The angle corresponding to the non-zero element is the estimated angle. ; Assumption No. If a row element is non-zero, it is assumed that a target exists at that angle, and the estimated pitch angle is calculated. Derived by the following formula ； in Indicates the pitch steering vector. It represents the Hadamardi (or Hadama) stack; The estimated two-way distance is derived from the beat frequency. ; Based on the estimated azimuth angle Pitch angle and round trip distance Thus, the point cloud locations are obtained: ； in This represents the rotation matrix used for coordinate transformation.

5. A method for three-dimensional reconstruction of non-line-of-sight corner building layouts suitable for sparse array radar according to claim 4, characterized in that, The implementation process of step S4 is as follows: Let the three-dimensional point cloud set obtained in step S3 be denoted as... Projecting all point clouds onto the ground yields a two-dimensional point cloud set. ; The number of 3D point clouds, This represents the number of points in a two-dimensional point cloud. Represents the coordinates of a three-dimensional point cloud; The Hough transform is used to map all two-dimensional point clouds to their polar coordinate space; For any two-dimensional point cloud coordinates Traverse the areas of interest Grid interval, if it satisfies the formula Then the corresponding Increment the grid point value by 1; set the first grid point value to 1. The cumulative result of the Hough transform in each iteration is denoted as . ;in, This represents the perpendicular distance between the line and the origin. This represents the angle between the normal vector of the line and the positive x-axis. The DBSCAN algorithm is used to cluster the point clouds projected onto the polar coordinate grid to distinguish point clouds from different buildings but located on the same straight line. The set of point clouds from the same building and on the same straight line is denoted as . This updates the current set of two-dimensional point clouds in the iteration to... ,in Indicates from set Middle Removal Set The elements contained therein Representing sets The first in The x and y coordinates of each point Indicates the number of points in the set; set The slopes of the lines connecting the two farthest points to the origin are denoted as follows: and This leads to the extraction of non-line-of-sight point clouds located behind the building surface at line-of-sight distance. The Householder transformation is used to map it back to its real location; Finally, the 2D point cloud set is updated to... , used for the The next iteration, in which... Indicates from set Middle Removal Set The elements contained therein; Through multiple iterations, the mapping of the building layout within the scene was finally achieved.

Images