Millimeter wave radar direct vision and non-direct vision mixed target positioning method, device and equipment

Abstract

The present application relates to a kind of millimeter wave radar direct vision and non-direct vision mixed target positioning method, device and equipment, belong to millimeter wave radar positioning technical field, wherein, millimeter wave radar direct vision and non-direct vision mixed target positioning method includes based on the received radar echo signal constructs observation vector, and by to search area is discretized to construct dictionary matrix;With the residual error between the sparse vector estimation value of current round and the observation vector and the iteration number as input, generate the regularization parameter of the first area and the second area by the depth learning enhanced iterative optimization algorithm;The sparse vector estimation value is updated based on the regularization parameter, while iteratively executing the depth learning enhanced iterative optimization algorithm until meeting the convergence condition, and output joint estimation result;Based on the joint estimation result, the real spatial position of each target is output to complete joint positioning.Using the present application can improve the joint positioning precision of direct vision and non-direct vision.

Description

Technical Field

[0001] This invention relates to the field of millimeter-wave radar positioning technology, and in particular to a method, apparatus, and equipment for locating targets using a hybrid approach of direct and indirect line-of-sight using millimeter-wave radar. Background Technology

[0002] In urban environments with obstructed visibility, millimeter-wave radar is often used for joint localization of both line-of-sight and non-line-of-sight targets, especially in complex scenarios such as building corners and street canyons. Target signals must propagate indirectly through multipath effects such as wall reflection and diffraction, leading to severe echo energy attenuation and complex path delays. Traditional methods typically use a grid matching model to discretize the detection space, constructing a dictionary matrix containing range, angle, and Doppler dimensions, and then recovering target parameters from radar echoes based on sparse reconstruction algorithms. These algorithms iteratively optimize sparse vectors and utilize norm regularization to enhance the sparsity of the solution, thereby suppressing noise and sidelobe interference and achieving super-resolution localization. To address the significant difference in echo energy between line-of-sight and non-line-of-sight targets, some studies have introduced weighting strategies, applying differentiated regularization weights to different regions in an attempt to balance detection sensitivity.

[0003] However, traditional techniques have significant drawbacks. Regularization parameters are often manually preset or fixed constants, unable to adapt to dynamic changes in the scene. This leads to problems such as over-regulation causing weak signals to be missed, or under-regulation generating false targets, especially in low signal-to-noise ratio or when non-line-of-sight targets dominate. Secondly, traditional iterative algorithms lack the ability to learn multipath channel characteristics. When line-of-sight and non-line-of-sight targets coexist, strong line-of-sight echoes easily mask nearby non-line-of-sight targets, causing boundary shifts, energy distortion, and positioning errors in the reconstruction results. Traditional techniques suffer from the technical problem of non-line-of-sight targets being masked due to significant differences in signal strength and fixed regularization parameters when line-of-sight and non-line-of-sight targets coexist, resulting in decreased positioning accuracy. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, apparatus and equipment for locating mixed direct and non-direct targets using millimeter-wave radar, in order to solve the problem that non-direct targets are masked due to the large difference in signal strength and the fixed regularization parameters when direct and non-direct targets coexist, resulting in a decrease in positioning accuracy.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a millimeter-wave radar method for locating targets using a hybrid line-of-sight and non-line-of-sight approach, comprising: An observation vector is constructed based on the received radar echo signal, and a dictionary matrix is ​​constructed by discretizing the search area. The iterative optimization algorithm enhanced by deep learning generates regularization parameters corresponding to the first region and the second region, respectively, by taking the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input. The first region and the second region are obtained by dividing the search region. The sparse vector estimate is updated based on the regularization parameter, and the deep learning-enhanced iterative optimization algorithm is executed iteratively until the convergence condition is met, and the joint estimation result is output. Based on the joint estimation results, the true spatial location of each target is output to complete the joint localization.

[0006] In one possible implementation, the construction of the observation vector based on the received radar echo signal and the construction of a dictionary matrix by discretizing the search area include: The received radar echo signal is down-converted and matched-filtered to obtain a discrete signal matrix, and the discrete signal matrix is ​​vectorized to obtain the observation vector. The search area is divided into a three-dimensional grid consisting of distance units, angle units, and Doppler units. Each grid point of the three-dimensional grid corresponds to a set of distance parameters, angle parameters, and Doppler parameters. A dictionary matrix is ​​constructed based on the three-dimensional mesh. Each column of the dictionary matrix is ​​obtained by vectorizing the transmit array guide vector and the receive array guide vector of the corresponding mesh point.

[0007] In one possible implementation, the method further includes: Based on the current position of the millimeter-wave radar, determine the first vertical distance between the millimeter-wave radar and the first wall, the second vertical distance between the millimeter-wave radar and the second wall, and the third vertical distance between the second wall and the third wall; Calculate the first angle boundary, the second angle boundary, and the distance threshold based on the first vertical distance, the second vertical distance, and the third vertical distance; The first region is defined as the region in the search area where the angle of arrival of the target to be detected is less than or equal to the first angle boundary, the region between the angle of arrival of the target to be detected being greater than the first angle boundary and less than or equal to the second angle boundary, and the region where the echo distance is less than or equal to the distance threshold. The area between the angle of arrival of the target to be detected that is greater than the first angle boundary and less than or equal to the second angle boundary, and the area where the echo distance is greater than the distance threshold, is divided into the second region.

[0008] In one possible implementation, the deep learning-enhanced iterative optimization algorithm generates regularization parameters corresponding to the first and second regions, respectively, using the residual between the sparse vector estimate of the current round and the observed vector, and the number of iterations as input. The diagonal loading parameters are calculated based on the residual between the sparse vector estimate of the current round and the observed vector using the deep learning-enhanced iterative optimization algorithm. Calculate the regularization parameter based on the diagonal loading parameter and the current iteration number; A weighted matrix in block diagonal form is constructed by setting a first weight value and a second weight value according to the region to which the grid point belongs. The second weight value is greater than the first weight value and is used to compensate for the energy attenuation of the signal in the second region. Regularization parameters corresponding to the first region and the second region are determined based on the regularization parameters, the first weight value, and the second weight value, respectively. The regularization parameters include region-specific regularization parameters and weighting matrices for the first region and the second region.

[0009] In one possible implementation, updating the sparse vector estimate based on the regularization parameter, while iteratively executing the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and outputting the joint estimation result, includes: Construct a diagonal weighted matrix based on the regularization parameters; Update the sparse vector estimate for the current round based on the diagonal weighted matrix, the dictionary matrix, and the observation vector; The updated sparse vector estimate is used as the input for the next iteration. The steps of generating region-specific regularization parameters and weighting matrices and updating sparse vector estimates are repeated. The iteration stops when the iterative change or residual of the sparse vector estimate meets the convergence condition, and the output includes the joint estimation result of the target in terms of distance, angle and Doppler dimension.

[0010] In one possible implementation, outputting the true spatial location of each target based on the joint estimation result includes: Based on the first angle boundary, the second angle boundary, and the distance threshold, each target point in the joint estimation result is determined to be located in a region. The target points identified as the first region are confirmed as line-of-sight target positions, and the target points identified as the second region are mapped to their real coordinates in physical space through a symmetric projection function. The set of line-of-sight target positions and the set of non-line-of-sight target positions are output. The symmetric projection function is constructed based on the geometric relationship of wall reflection. It calculates the real coordinates of the target point in physical space using the echo distance and angle of arrival of the target point as input. Before the area determination, the two-dimensional area of ​​distance and angle is filtered according to the actual scene to remove target points located in the invalid area.

[0011] In one possible implementation, the deep learning-enhanced iterative optimization algorithm is a network-enhanced iterative sparse reconstruction algorithm. This algorithm replaces fixed regularization parameters with learnable network parameters. In each iteration, the error between the current estimated sparse vector and the scattering coefficients of the real scene is calculated using a loss function. The network weights are then updated via backpropagation, allowing the regularization parameters and weighting matrix to adaptively adjust with the iteration process. The network-enhanced iterative sparse reconstruction algorithm includes an input module, an initialization module, a target reconstruction and update module, a parameter training and adjustment module, a parameter adaptive update submodule, and a parameter momentum update submodule. The parameter adaptive update submodule dynamically generates regularization parameters, stability control terms, and periodic momentum coefficients based on the current iteration state and echo residuals. The parameter momentum update submodule uses historical gradient information to smoothly update the sparse vector to suppress iterative oscillations.

[0012] Secondly, the present invention also provides a millimeter-wave radar hybrid target localization device that combines direct-line and non-direct-line views, comprising: The signal processing module is used to construct observation vectors based on the received radar echo signals and to construct a dictionary matrix by discretizing the search area. The parameter generation module is used to generate regularization parameters corresponding to the first region and the second region respectively by using the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as inputs through the deep learning-enhanced iterative optimization algorithm. The first region and the second region are obtained by dividing the search region. The iterative update module is used to update the sparse vector estimate based on the regularization parameter, and simultaneously execute the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and output the joint estimation result. The mapping output module is used to output the true spatial location of each target based on the joint estimation result, so as to complete the joint localization.

[0013] Thirdly, the present invention also provides an electronic device, including a signal acquisition unit, a memory, and a processor, wherein the signal acquisition unit is used to acquire radar echo signals; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the millimeter-wave radar direct-line and non-direct-line hybrid target localization method described in any of the above implementations.

[0014] Fourthly, the present invention also provides a millimeter-wave radar direct-line and non-direct-line hybrid target positioning system, which, as described above, includes an electronic device, a radar, and a result output device. The radar and the result output device are respectively connected to the electronic device. The radar is used to acquire radar echo signals, and the result output device is used to output and display the joint positioning results.

[0015] The beneficial effects of this invention are as follows: The millimeter-wave radar hybrid target localization method for both line-of-sight and non-line-of-sight targets provided by this invention constructs an observation vector based on the received radar echo signal, generates a dictionary matrix by discretizing the search region, and achieves sparse reconstruction by combining a grid matching model to obtain a set of target candidate points. Furthermore, through a deep learning-enhanced iterative optimization algorithm, the residual between the sparse vector estimate and the observation vector in the current round and the number of iterations are used as inputs to dynamically generate adaptive regularization parameters corresponding to the first and second regions, respectively. The first and second regions are divided according to the spatial characteristics of the search region and are used to characterize different environmental sub-regions with significant differences in the attenuation of line-of-sight and non-line-of-sight signals. This algorithm uses a deep learning model to nonlinearly model the evolution trend of the residual and the iterative process, accurately adjusts the sparse constraint strength of each region, effectively preserves the weak signal caused by multipath attenuation of non-line-of-sight targets, and suppresses the overfitting interference caused by the strong signal of line-of-sight targets. During the iteration process, the sparse vector estimate is continuously updated until convergence, and finally the joint estimation result containing all targets is output, and their true spatial positions are inverted and restored. This successfully solved the technical challenge of masking non-direct-view targets and reducing positioning accuracy when direct-view and non-direct-view targets coexist in urban occlusion environments due to significant differences in signal strength and the fixed nature of traditional regularization parameters. It achieved high-precision, adaptive joint positioning of multiple types of targets in complex scenarios, significantly improving the perception robustness and reliability of millimeter-wave radar in real urban environments. Attached Figure Description

[0016] Figure 1 A flowchart illustrating an embodiment of the millimeter-wave radar hybrid target localization method combining direct and indirect line-of-sight methods provided by the present invention; Figure 2 A typical T-shaped scene diagram of an embodiment of the millimeter-wave radar hybrid target localization method combining direct and non-direct line of sight provided by the present invention; Figure 3 A schematic diagram of a grid matching model for a detection scene in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 4 A schematic diagram of the transceiver array and multidimensional parameter estimation model of an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 5 A schematic diagram of the RIRM-Net iterative algorithm of an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 6 This is a LOS / NLOS target mapping diagram of an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention; Figure 7 A schematic diagram of a sector area is provided for an embodiment of the millimeter-wave radar hybrid target localization method combining direct and non-direct line of sight provided by the present invention. Figure 8 A schematic diagram of DAS reconstruction results under a distance-angle two-dimensional cross-section in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 9 A schematic diagram of the IAA reconstruction result under a range-angle two-dimensional cross-section in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 10 A schematic diagram of the RIRM iteration reconstruction result under a distance-angle two-dimensional cross-section in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 11 A schematic diagram of the RIRM iteration reconstruction result after 5 iterations in a range-angle two-dimensional cross-section case of an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 12 A schematic diagram of DAS reconstruction results under a range-Doppler two-dimensional cross-section in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 13 This is a schematic diagram of the IAA reconstruction result under the range-Doppler two-dimensional cross section in an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention. Figure 14 A schematic diagram of the RIRM iteration reconstruction result under a range-Doppler two-dimensional cross-section in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 15 A schematic diagram of the RIRM iteration reconstruction result after 5 iterations in a range-Doppler two-dimensional cross-section case of an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention; Figure 16A schematic diagram of the simulated letter "WIT" from an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention; Figure 17 This is a schematic diagram of RIRM reconstruction results during the performance verification of multi-target localization in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention. Figure 18 A schematic diagram of RIRM-Net reconstruction results during the performance verification of multi-target localization in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention. Figure 19 This is a schematic diagram of the IAA reconstruction results during the performance verification of multi-target localization in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention. Figure 20 A RIRM two-dimensional energy map of an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention; Figure 21 A RIRM-Net two-dimensional energy map of an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention; Figure 22 An IAA two-dimensional energy map of an embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method provided by the present invention; Figure 23 A schematic diagram showing the time comparison of reconstruction results of different algorithms under different signal-to-noise ratios in an embodiment of the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization method provided by the present invention. Figure 24 A functional block diagram of a millimeter-wave radar hybrid target localization device that combines direct and indirect line of sight according to an embodiment of this application; Figure 25 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0019] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Before demonstrating the embodiments, the following terms will be explained.

[0022] LOS: Line of Sight (LOS) refers to the path through which a radar signal can propagate directly between itself and a target when there are no obstructions.

[0023] NLOS: Non-Line of Sight (NLOS) refers to a path where there are obstructions such as buildings between the radar and the target, and the signal must travel through multiple paths such as reflection and diffraction from walls, i.e., not a direct line of sight.

[0024] IAA: Iterative Adaptive Approach (IAA).

[0025] RIRM (Regularized Iterative Reweighted Minimization): A regularized iterative reweighted minimization algorithm.

[0026] Grid matching model: A model that discretizes a continuous space into grid points and determines the target location by comparing the degree of matching between the theoretical multipath characteristics of each grid point and the measured data.

[0027] Sparse reconstruction: the process of recovering a sparse signal from a small amount of observation data, which in this invention is used to recover the position information of a target from an echo signal.

[0028] This invention provides a method, apparatus, and device for locating targets using a hybrid approach of direct and indirect line of sight using millimeter-wave radar, which will be described in detail below.

[0029] Figure 1 This is a schematic flowchart of an embodiment of the millimeter-wave radar hybrid line-of-sight and non-line-of-sight target localization method provided by the present invention, as shown below. Figure 1 As shown, the millimeter-wave radar hybrid target localization method, which combines line-of-sight and non-line-of-sight approaches, includes: S101. Construct an observation vector based on the received radar echo signal, and construct a dictionary matrix by discretizing the search area.

[0030] Before proceeding with the explanation, this embodiment will first describe the application scenarios involved in the millimeter-wave radar hybrid target localization method (combining line-of-sight and non-line-of-sight) provided by the present invention. Specifically, please refer to... Figure 2 As shown, a representative T-shaped building hybrid scenario is constructed by combining typical urban buildings and street layouts to describe the detection environment where LOS and NLOS targets coexist. This T-shaped building hybrid scenario consists of three walls: Wall 1 (Wall-1), Wall 2 (Wall-2), and Wall 3 (Wall-3). The radar is positioned at the origin O of the coordinate system, and the positions and layouts of Wall 1, Wall 2, and Wall 3 are known information. The vertical distances between the radar and Wall 1, Wall 2, and Wall 3 are denoted as [insert values ​​here]. , and The angle between line segment OD and the x-axis is... The extension of OC intersects wall 3 at point E, and the angle between line segment OE and the x-axis is . .

[0031] It should be noted that in this embodiment, the observation vector is constructed based on the received radar echo signal. This process involves vectorizing the discrete signal after down-conversion and matched filtering at the receiving end to form observation data reflecting the characteristics of the target echo. Subsequently, the detection scene is discretized, and the continuous spatial domain is divided into a three-dimensional grid of multiple range units, angle units, and Doppler units. Each grid point corresponds to a possible combination of target parameters. Then, based on the radar system's transmitting and receiving array structure, signal frequency offset, and propagation model, the theoretical steering vector corresponding to each grid point is calculated. The steering vectors of all grid points are arranged in columns to form a dictionary matrix. This matrix represents the linear mapping relationship between the observation vector and the parameters of each grid point.

[0032] In some embodiments, step S101 includes: obtaining a discrete signal matrix by down-converting and matched filtering the received radar echo signal; obtaining an observation vector by vectorizing the discrete signal matrix; dividing the search area into a three-dimensional grid composed of range units, angle units, and Doppler units, wherein each grid point of the three-dimensional grid corresponds to a set of range parameters, angle parameters, and Doppler parameters; and constructing a dictionary matrix based on the three-dimensional grid, wherein each column of the dictionary matrix is ​​obtained by vectorizing the transmit array steering vector and the receive array steering vector of the corresponding grid point.

[0033] It should be understood that in the NLOS region, radar cannot directly detect targets; however, it can indirectly obtain target-related information by utilizing multipath signals generated by wall reflections and diffraction. The grid matching model can transform the NLOS localization problem into an optimal matching problem within a discretized search space. By comparing the degree of matching between the theoretical multipath characteristics of each grid point and the measured data, the target location can be determined.

[0034] Specifically, such as Figure 3 As shown, a grid matching model is used to grid the search area, with each grid point corresponding to a set of three-dimensional parameters, respectively. , and It means that, among them Represents the distance to the target to be detected. Represents the angle of the target to be detected. This represents the Doppler frequency of the target to be detected. The scene of interest is divided into... distance unit and Each angle unit is used, and the Doppler interval of interest is divided into... One Doppler unit.

[0035] Furthermore, such as Figure 4 As shown, the radar system includes Each launch element and There are 1 receiving array element, where T represents the target to be located. The transmission signal of each transmitting element It can be represented as: The radar system includes Each launch element and There are 3 receiving array elements, and the spacing between the transmitting and receiving array elements is denoted as d. t and d r . No. The transmission signal of each transmitting element It can be represented as:

[0036] in, Indicates the transmission of energy. Indicates the baseband signal, frequency increment. Defined as:

[0037] in, express The transmission frequency of the transmitting antenna. Indicates frequency offset. .

[0038] It should be noted that this is considering a single snapshot scenario, and assuming that the detection scenario includes... There are several targets, the first target is located at a distance of... and angle Doppler is ,but The received signal of each receiving array element can be represented as:

[0039] in, and These are the transpose and conjugate transpose operators, respectively. Indicates the first The scattering amplitude of each target, Indicates noise and interference terms. Represents the baseband signal vector. and Let represent the steering vectors of the transmitting array and the receiving array, respectively, and define them as follows:

[0040]

[0041] in, The number of the transmitting array The phase difference between the first unit and the second unit is defined as:

[0042] in, Corresponding to the The center frequency of the Doppler interval.

[0043] Furthermore, based on Figure 3 The mesh matching model for the detection scene in the model divides the target scene of interest into... distance unit and Each angular unit (relative to the array) is used to divide the Doppler interval of interest into... One Doppler unit. Then the received signal... The discrete signal after down-conversion and matched filtering at the receiving end is:

[0044] After vectorization, it can be represented as , for A sparse vector representing the target's range, angle, and Doppler cell information; for The dictionary matrix represents the guiding vector of the array. and They are defined as follows:

[0045]

[0046] in, vec represents the vectorization operator.

[0047] It is worth noting that target localization is achieved by solving for sparse vectors in the model. This allows for the acquisition of detailed information about targets both at and outside the line of sight. To this end, this paper employs an iterative adaptive method and a deep learning enhancement algorithm to process sparse vectors. The system reconstructs the data and, by introducing a region-adaptive weighting strategy and a learnable regularization mechanism, effectively distinguishes and accurately extracts the distance, angle, and Doppler information of LOS and NLOS targets.

[0048] S102. An iterative optimization algorithm enhanced by deep learning is used to generate regularization parameters corresponding to the first region and the second region, respectively, with the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input.

[0049] It should be noted that the first region and the second region in this embodiment are obtained by dividing the search region.

[0050] In some embodiments, the region division process includes determining a first vertical distance between the millimeter-wave radar and a first wall, a second vertical distance between the millimeter-wave radar and a second wall, and a third vertical distance between the second wall and a third wall based on the current position of the millimeter-wave radar; calculating a first angle boundary, a second angle boundary, and a distance threshold based on the first vertical distance, the second vertical distance, and the third vertical distance; dividing the region in the search area where the angle of arrival of the target to be detected is less than or equal to the first angle boundary, and the region between the angle of arrival of the target to be detected being greater than the first angle boundary and less than or equal to the second angle boundary, and the region where the echo distance is less than or equal to the distance threshold, into the first region; dividing the region between the angle of arrival of the target to be detected being greater than the first angle boundary and less than or equal to the second angle boundary, and the region where the echo distance is greater than the distance threshold, into the second region.

[0051] For example, with Figure 2 For example, suppose the angle of arrival of the target to be detected is... ,Will The region is defined as a LOS region, and The region is defined as a mixed LOS region. Given that the echo energy of LOS targets is typically several times that of NLOS targets, within the mixed LOS region, LOS and NLOS targets can be further distinguished based on echo energy intensity. and That is, the first angular boundary and the second angular boundary. Additionally, when... and When different size relationships exist, the non-line-of-sight region will also exhibit different characteristics: The non-visual distance area appears as a right-angled triangle; The non-visual distance area appears as an acute triangle; The non-line-of-sight (LOS) region presents as an obtuse triangle. Line-of-sight and non-line-of-sight targets in the mixed LOS region are classified based on echo energy intensity. , , That is, the first vertical distance, the second vertical distance, and the third vertical distance. and That is, the first angular boundary and the second angular boundary; the first region is also the LOS region, and the second region is also the NLOS region. O represents the radar position, and C is the angle between the vertex line segment OD of the included angle between wall 1 and wall 2 and the x-axis. The extension of OC intersects wall 3 at point E, and the angle between line segment OE and the x-axis is... .

[0052] Furthermore, an iterative optimization algorithm enhanced by deep learning is used as input to generate regularization parameters corresponding to the first and second regions, respectively, by utilizing the residual between the sparse vector estimate and the observed vector in the current round and the number of iterations. This process dynamically derives regularization parameters applicable to the first and second regions based on the estimation error information reflected in the residuals and the optimization progress represented by the number of iterations. This allows the regularization strength to adaptively change with estimation accuracy and iteration stage, thereby achieving differentiated control of sparse constraints in different regions without relying on preset fixed values.

[0053] In some embodiments, step S102 includes using the deep learning-enhanced iterative optimization algorithm to calculate diagonal loading parameters based on the residual between the sparse vector estimate of the current iteration and the observation vector; to calculate regularization parameters based on the diagonal loading parameters and the current iteration number; and to construct a block diagonal weighted matrix by setting a first weight value and a second weight value according to the region to which the grid point belongs.

[0054] It should be noted that the second weight value is greater than the first weight value and is used to compensate for the energy attenuation of the signal in the second region; regularization parameters corresponding to the first region and the second region are determined according to the regularization parameters, the first weight value and the second weight value, respectively. The regularization parameters include region-specific regularization parameters and weighting matrices for the first region and the second region.

[0055] Specifically, assuming These represent the distance, angle, and Doppler cell of the target, respectively. To reduce the complexity of the estimation algorithm, this paper achieves 3D parameter estimation by traversing each 2D plane. Based on the above model, the joint localization problem of LOS and NLOS targets is transformed into finding the target from the observation vectors. Recovery of sparse vectors The optimization problem. Considering that LOS target signals are typically stronger than NLOS target signals. A weighted strategy needs to be designed to balance detection performance, therefore a weighting matrix is ​​introduced:

[0056] Among them, weight Based on the region to which the grid point belongs, set as follows:

[0057] in, This is the NLOS region weighting enhancement factor, used to compensate for NLOS signal attenuation. The first weight value is 1, and the second weight value is... .

[0058] Based on the grid matching model, the weighted sparse reconstruction problem can be formulated as follows:

[0059] Or equivalent regularized form:

[0060] in, This is a regularization parameter that controls the overall sparsity. Indicates an extremely small quantity.

[0061] In weighted sparse reconstruction, the weighting matrix W is typically used to impose differentiated sparsity constraints on the reflection coefficients at different spatial locations. In LOS / NLOS hybrid scenes, to handle the energy differences between the two types of paths more precisely, W can be designed as a block diagonal form, corresponding to the LOS and NLOS regions respectively:

[0062] At this point, weighted The norm can be written as:

[0063] If the overall regularization parameter is... With regional weight , Merging, Then the regularization form can be simplified to a region-adaptive form. The sum of penalty terms. Therefore, the weighting matrix. The effect of this is absorbed into the region-specific regularization parameter, resulting in the following intuitive form:

[0064] in, , These represent the sub-vectors corresponding to the LOS and NLOS regions, respectively. , A region-specific regularization parameter.

[0065] In some embodiments, the deep learning-enhanced iterative optimization algorithm is a network-enhanced iterative sparse reconstruction algorithm. This algorithm replaces fixed regularization parameters with learnable network parameters. In each iteration, the error between the current estimated sparse vector and the scattering coefficients of the real scene is calculated using a loss function. The network weights are then updated via backpropagation, allowing the regularization parameters and weighting matrix to adaptively adjust with the iteration process. The network-enhanced iterative sparse reconstruction algorithm includes an input module, an initialization module, a target reconstruction and update module, a parameter training and adjustment module, a parameter adaptive update submodule, and a parameter momentum update submodule. The parameter adaptive update submodule dynamically generates regularization parameters, stability control terms, and periodic momentum coefficients based on the current iteration state and echo residuals. The parameter momentum update submodule uses historical gradient information to smoothly update the sparse vector to suppress iterative oscillations.

[0066] Specifically, regularization parameters were introduced. and The adaptive update mechanism. Among them, The expression reflects the relationship between the data fitting error and the regularized weighting matrix. The balance, achieved by introducing dynamic adjustment of the high-dimensional echo autocorrelation matrix, can effectively suppress parameter drift caused by noise during the iteration process. Load regular expressions, parameters and It can be represented as:

[0067] in, It represents the regularized weighted matrix of the high-dimensional echo autocorrelation matrix that varies with the target parameter during each iteration. This indicates the number of iterations in the current round. This design allows the regularization strength to gradually increase as the iteration progresses, thus maintaining good parameter update flexibility in the early stages of the algorithm and strengthening constraints in the later stages to avoid overfitting or noise amplification, achieving a smooth transition from coarse estimation to fine estimation.

[0068] Therefore, the regularized weighted iterative RIRM-Net localization algorithm processes the first... The iteration process can be represented as:

[0069] in, Indicates the number of iterations. Indicates the first The regularized weighted matrix in the next iteration It is a dictionary matrix (also known as an observation matrix or a sensing matrix). It is an identity matrix.

[0070] further, exist In the estimation, as a denominator term that decreases with each iteration, it is important to avoid the denominator being too small or zero in the early stages of iteration or under certain circumstances, thereby enhancing numerical stability. exist In the estimation, the adjustment term used for periodic oscillations, parameters Dynamic adjustments can prevent the optimization process from getting stuck in flat regions or local minima. Furthermore, As an adjustment term, periodic perturbations of the learning rate or search direction can enhance the ability to escape local optima.

[0071] The RIRM-Net iterative algorithm can be divided into six modules: input module, initialization module, target reconstruction and update module, parameter training and adjustment module, parameter adaptive update submodule, and parameter momentum update submodule. Input module: Input real beam echo. and the true scattering coefficient of the target scene The initialization module primarily initializes the echo autocorrelation matrix (using DAS estimation in this paper) and sets the regularization parameters. The target reconstruction and update module primarily updates the echo autocorrelation matrix and solves for the target scattering coefficients. The parameter training and adjustment module primarily calculates the loss function and updates the weighted matrix of the regularization parameters. The adaptive parameter update submodule is responsible for dynamically generating regularization parameters based on the current iteration state and the echo residuals. Stability control items and periodic momentum coefficient These parameters are obtained through lightweight networks or learnable mapping functions, and their updates depend on the number of iterations. And the current estimation error. Parameter momentum update submodule: uses historical gradient information to update the target scattering coefficient. By performing smooth updates, iterative oscillations are suppressed and convergence is accelerated. Through the above modular design, RIRM-Net significantly improves its robustness and super-resolution imaging accuracy in complex scenes through an adaptive parameter scheduling mechanism.

[0072] Furthermore, such as Figure 5 As shown, the RIRM-Net network iterative structure introduces a deep learning module into the traditional RIRM algorithm to achieve adaptive parameter learning and iterative path optimization. This structure is designed to receive echo data. and initial target scattering coefficient estimation As input, after multiple iterations, the final high-resolution parameter estimate is output. In each iteration, the network adjusts the loss function. Calculate the current estimate With real-world scenarios The error is used to optimize the adaptive parameters and generate weights in the network through backpropagation. This allows RIRM-Net to learn the optimal regularization strategy and iterative control mechanism for complex scenes during the training phase, and achieve fast and robust super-resolution localization during the testing phase. In this embodiment, the reconstruction loss... The time is used as the termination condition for the iteration.

[0073] S103. Update the sparse vector estimate based on the regularization parameter, and iteratively execute the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and output the joint estimation result.

[0074] It should be noted that updating the sparse vector estimate based on the regularization parameter means adjusting the current estimate of the sparse vector according to the set regularization parameter, so that the estimate is closer to the real target distribution while satisfying the constraints. This update process is achieved by iteratively executing a deep learning-enhanced iterative optimization algorithm. In each iteration, the algorithm dynamically guides the correction direction of the sparse vector according to the parameter adjustment mechanism learned by the deep learning model, rather than relying on fixed rules. The whole process continues until the preset convergence condition is met. At this point, the algorithm stops iterating and outputs the distance, angle and Doppler information jointly estimated after multiple adaptive corrections, ensuring that the final result is determined by the algorithm's internal iteration and regularization mechanism without introducing external assumptions.

[0075] Specifically, adaptive iterative optimization algorithms, such as IAA and RIRM, estimate sparse vectors through multiple iterations. They utilize adaptive weighting or regularization strategies on the observed signals to progressively improve the accuracy of target parameter estimation. However, these methods typically rely on fixed regularization parameters or weights and cannot adaptively adjust to adapt to the differences in signal characteristics across different regions (such as LOS and NLOS). The iterative update rule of the IAA algorithm is:

[0076] in, Indicates the first iteration Represents the data covariance matrix. This represents the diagonal loading regularization parameter.

[0077] RIRM through weighted Norm enhances sparsity, and the iterative update rule is as follows:

[0078] in, Represents a diagonal weighted matrix. This represents the diagonal weighted regularization parameter.

[0079] IAA continuously updates the covariance matrix. It utilizes its inverse matrix to optimize sparse vectors, primarily focusing on adaptive estimation of signal power, but lacks explicit sparsity constraints. RIRM, on the other hand, introduces weighted... The norm, as a regularization term, is adjusted by the weight matrix. To enhance the sparsity of the estimation.

[0080] S104. Output the true spatial location of each target based on the joint estimation results to complete the joint positioning.

[0081] In this embodiment, the true spatial positions of each target are output based on the joint estimation results to complete joint localization. This means that after jointly estimating the target distance, angle, and Doppler parameters, the estimated three-dimensional parameter units are directly converted into coordinate positions in the real world according to preset geometric mapping rules, thereby achieving the final determination of the spatial position of each target. This process does not rely on additional filtering, trajectory association, or physical model correction. It only takes the estimation results as input and maps each estimated point from the parameter space to the real coordinate system through known scene structure parameters (such as wall distance and angle boundaries) and predefined mapping functions. This allows the system to clearly output the specific coordinates of the target in the environment, completing the final closed loop from signal processing to spatial localization.

[0082] In some embodiments, step S103 includes: determining the region of each target point in the joint estimation result based on the first angle boundary, the second angle boundary, and the distance threshold; confirming the target points determined to be in the first region as line-of-sight target positions; mapping the target points determined to be in the second region to their real coordinates in physical space through a symmetric projection function; and outputting a set of line-of-sight target positions and a set of non-line-of-sight target positions.

[0083] It should be noted that the symmetric projection function is constructed based on the geometric relationship of wall reflection. It calculates the real coordinates of the target point in physical space using the echo distance and angle of arrival of the target point as input. Before the area determination, the two-dimensional area of ​​distance and angle is filtered according to the actual scene to remove target points located in the invalid area.

[0084] It should be noted that when the non-line-of-sight (LOS) area is an obtuse angle, a simple energy partitioning is used to distinguish between LOS and NLOS targets. When the non-line-of-sight (LOS) area is a right angle or an acute angle, the LOS and NLOS target mapping method specifically includes: the input is the original radar echo signal yt, the distance d from the radar corner C to Wall-3, the angle boundaries θ1 and θ2, and the output is the set of LOS target positions R and the set of NLOS target positions B. 1: Initialization: R←Ø, B←Ø, M←Preprocessing yt, 2: while M≠Ø do, 3: R, θ←NextCandidateM, 4: if θ≤θ1 or θ2≤θ≤θ1, R≤d then, 5: R←R∪Rcosθ, Rsinθ, 6: else if θ2≤θ≤θ1, R>d, 7: B←B∪2D3 Rcosθ, Rsinθ, 8: end if, 9: end while, 10: return R,B.

[0085] Specifically, the first step is to model the detection scene, measure the distance from corner C to Wall-3, and calculate the angular boundaries. , and acquisition of echo signals Based on this, iterative algorithms or network augmentation algorithms are used to... Preprocessing is performed to obtain the reconstructed 3D parameter estimation model. During this process, the possible locations of the target are initially filtered and identified, and these identified points are assigned values. Next, take them out one by one. For each point in the set, perform LOS and NLOS target mapping judgments on that point until... It is an empty set. The final result is... These are the sets of target points at line-of-sight and non-line-of-sight, respectively. Furthermore, before mapping LOS / NLOS targets, the distance-angle two-dimensional region needs to be filtered according to the actual scene to effectively eliminate false and invalid targets. This can be mapped to a central angle of [missing information - likely a value]. , radius is The radar is defined as a sector-shaped region. Other areas within this sector are considered invalid regions and are filtered out before actual mapping. Targets located within invalid regions, such as those too close to the radar or behind walls, are effectively removed after filtering. Figure 6 The region shown can be mapped to polar coordinates as follows: Figure 7 The central angle shown is , radius is The diagram shows a sector-shaped region. The horizontal axis (Range) represents the range unit, i.e., the echo distance from the target to the radar, corresponding to the range parameter in the mesh matching model. The vertical axis (Angle) represents the angle unit, i.e., the target's angle of arrival, corresponding to the angle parameter in the mesh matching model. Other areas within the sector are considered invalid regions and are filtered out before actual mapping.

[0086] Furthermore, several simulation results are presented to systematically evaluate the performance of the proposed algorithm. For example: To compare the target information estimation of the adaptive iterative algorithm, and to verify the performance of the IAA and RIRM algorithms in multi-target 3D parameter estimation, five targets were set in the simulation scene. Their distance, angle, and Doppler cell information are shown in Table 1 below (simulated target 3D parameters):

[0087] The simulation system uses the designed weighted CAN sequence as the transmitted signal. The simulation parameters are shown in Table 2 (Simulation Parameters of CAN Sequence Simulation System). Wavelength, noise variance To facilitate observation of the simulation results, we selected two reference surfaces with angular elements of -10° and Doppler elements of 0°.

[0088] Table 2:

[0089] Case 1 (range-angle 2D section): Targets 1, 4 and 5 are on a 2D section with Doppler elements of 0. Figure 8-11 The reconstruction results using DAS, IAA, and RIRM algorithms are presented.

[0090] Case 2 (range-Doppler 2D cross section): Targets 1, 2 and 3 are on a 2D cross section with an angle element of -10. Figure 12-15 The reconstruction results using DAS, IAA, and RIRM algorithms are presented.

[0091] Furthermore, the performance of adaptive iteration and network enhancement algorithms in multi-target localization is verified, for example: To verify the multi-target processing performance of the IAA and RIRM algorithms and the processing advantages of the network augmentation algorithm, this section sets up 148 targets in the simulation scenario to simulate the character "WIT". Their distance, angle, and Doppler cell information are as follows: Figure 16 As shown. Additionally, the analog system transmits an IWR6843ISK sawtooth wave signal.

[0092] Figure 17 The results of the RIRM algorithm are presented. The algorithm performs poorly when the target sparsity is low, and the reconstruction results will shift to the boundary, resulting in serious distortion of the final result. Figure 18 The results of the RIRM-Net algorithm are presented. By introducing an adaptive learning mechanism, intelligent regularization parameter adjustment and improved iterative strategy, the algorithm significantly improves the performance in terms of target sparsity variation, boundary localization accuracy and noise suppression, and overcomes the limitations of traditional RIRM algorithms in complex scenarios. Figure 19 The results of the IAA algorithm are presented. Under ideal conditions, the algorithm can provide high-resolution reconstruction, but in real complex environments, it may suffer from estimation bias and false targets due to limitations such as regularization parameter settings.

[0093] To more intuitively demonstrate the multi-objective processing performance of the IAA and RIRM algorithms and the processing advantages of network augmentation algorithms. Figure 20-22 Two-dimensional energy maps are presented for different algorithms when the Doppler element is 0 (stationary surface). Figure 20 The image is a two-dimensional energy map generated by the RIRM algorithm. Due to boundary offset during the estimation process, the stationary target shifts towards the moving surface. Figure 21 This is a two-dimensional energy map of the RIRM-Net algorithm. Compared to the RIRM algorithm, it introduces an adaptive parameter learning mechanism to make each iteration of the algorithm more appropriate and the target estimation more focused. Figure 22 The image is a two-dimensional energy map of the IAA algorithm. Since the estimation effect is good because it is independent of sparsity during the iteration process, a small number of ghost images are generated.

[0094] Mean Squared Error (MSE) is used to evaluate the performance of different reconstruction algorithms. The MSE calculation formula for the reconstruction result versus the original scene can be defined as follows:

[0095] in, This represents the number of effective target elements in the three-dimensional parametric cube. This represents the reconstruction result vectors of different algorithms. This represents the original scene vector. The MSE calculation results for different algorithms are shown in Table 3.

[0096] Table 3:

[0097] Table 3 shows that the traditional RIRM algorithm has a large reconstruction error and limited performance in multi-object sparse scenarios, resulting in a large MSE. Furthermore, the traditional IAA algorithm suffers from a decrease in MSE because its iteration process is independent of sparsity. The network augmentation algorithm, by adaptively adjusting key parameters, eliminates its dependence on fixed parameters and ideal assumptions, achieving a better MSE and validating its processing advantages.

[0098] Furthermore, a comparison of the localization processing time between adaptive iteration and network enhancement algorithms is provided, for example: To compare the computational complexity of different iterations and network augmentation algorithms, this paper studies the execution time of the algorithms under different signal-to-noise ratios. To control for variable factors, the reconstruction loss was selected. The time is used as the termination condition for the iteration.

[0099] like Figure 23 As shown, with As the value increases, the computation time of the IAA algorithm generally shows a decreasing trend. This phenomenon is due to the different... The convergence difficulty varies, and the IAA algorithm takes roughly the same amount of time for each iteration, while at lower convergence difficulties... Under these conditions, due to severely contaminated data, IAA will start iterating from a very poor initial estimate and require a very large number of iterations to reach convergence, thus increasing computation time. Furthermore, with... Despite the changes, the computation time of the other algorithms remained largely consistent without significant fluctuations. The IAA algorithm, however, has a higher computation time due to the need to calculate and update the complete spectral covariance matrix in each iteration, involving computationally intensive operations such as matrix inversion. The RIRM algorithm, by introducing a simplified approximate update method, reduces the computational load in each step, thus resulting in a lower computation time. The RIRM-Net network enhancement algorithm accelerates convergence through adaptive selection of regularization parameters, and due to the independence of inter-beam echoes, it reduces the dimensionality of the echo correlation matrix inversion by cropping the test echoes, thereby lowering the computation time.

[0100] Furthermore, the multi-dimensional parameter model mapping method is validated. For example, to verify the accuracy of the LOS and NLOS target mapping methods, this section sets up four targets in the simulation: two LOS targets and two NLOS targets. The specific parameters of the targets are shown in Table 4. Additionally, scene parameters... , .

[0101] Table 4:

[0102] The calculation formula is as follows:

[0103] In this design, the radar panel's x-axis is at 0°, and each angle element represents 1°. Range element. For a range gate corresponding to the IWR6843ISK radar board, assuming Indicates sampling rate, Indicates frequency modulation slope, This indicates the total number of doors at a distance.

[0104] This embodiment constructs an observation vector based on the received radar echo signal and builds a dictionary matrix by discretizing the search area. A deep learning-enhanced iterative optimization algorithm is used as input to generate regularization parameters corresponding to the first and second regions, respectively, based on the residual between the sparse vector estimate and the observation vector in the current round and the number of iterations. The sparse vector estimate is updated based on the regularization parameters, and the deep learning-enhanced iterative optimization algorithm is iteratively executed until the convergence condition is met, outputting a joint estimation result. The true spatial position of each target is output based on the joint estimation result to complete joint localization, which can improve the joint localization accuracy of both line-of-sight and non-line-of-sight methods.

[0105] Based on the same concept as the millimeter-wave radar direct-line and non-direct-line hybrid target localization method in the above embodiments, this application also provides a millimeter-wave radar direct-line and non-direct-line hybrid target localization device. This device can be used to execute the above-described millimeter-wave radar direct-line and non-direct-line hybrid target localization method. For ease of explanation, the structural schematic diagram of the embodiment of the millimeter-wave radar direct-line and non-direct-line hybrid target localization device only shows the parts related to the embodiments of this application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0106] like Figure 24 As shown, the millimeter-wave radar hybrid target localization device 2400, which combines direct and indirect line-of-sight capabilities, includes: The signal processing module 2401 is used to construct an observation vector based on the received radar echo signal and to construct a dictionary matrix by discretizing the search area. The parameter generation module 2402 is used to generate regularization parameters corresponding to the first region and the second region respectively by using the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input through a deep learning-enhanced iterative optimization algorithm. The first region and the second region are obtained by dividing the search region. The iterative update module 2403 is used to update the sparse vector estimate based on the regularization parameter, and simultaneously iteratively execute the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and output the joint estimation result. The mapping output module 2404 is used to output the true spatial position of each target based on the joint estimation result, so as to complete the joint localization.

[0107] Please refer to Figure 25 , Figure 25 This is a schematic diagram of an embodiment of the electronic device of this application. In this embodiment of the invention, the electronic device 2500 includes a processor 2501, a memory 2502, a display 2503, and a signal acquisition device 2504. Figure 25 Only some components of the electronic device 2500 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0108] In some embodiments, processor 2501 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 2502 or process data, such as the millimeter-wave radar direct-line and non-direct-line hybrid target localization method of the present invention.

[0109] In some embodiments, display 2503 may be an LED display, a liquid crystal display, a touch-screen liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 2503 is used to display information from electronic device 2500 and to display visual user applications. Components 2501-2503 of electronic device 2500 communicate with each other via a system bus.

[0110] In one embodiment, when processor 2501 executes the millimeter-wave radar line-of-sight and non-line-of-sight hybrid target localization program stored in memory 2502, the following steps can be implemented: An observation vector is constructed based on the received radar echo signal, and a dictionary matrix is ​​constructed by discretizing the search area. The iterative optimization algorithm enhanced by deep learning generates regularization parameters corresponding to the first region and the second region, respectively, by taking the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input. The first region and the second region are obtained by dividing the search region. The sparse vector estimate is updated based on the regularization parameter, and the deep learning-enhanced iterative optimization algorithm is executed iteratively until the convergence condition is met, and the joint estimation result is output. Based on the joint estimation results, the true spatial location of each target is output to complete the joint localization.

[0111] It should be understood that when the processor 2501 executes the millimeter-wave radar direct-line and non-direct-line target localization program in the memory 2502, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0112] Furthermore, the embodiments of the present invention do not specifically limit the type of the electronic device 2500 mentioned. The electronic device 2500 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 2500 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0113] Accordingly, this application also provides a millimeter-wave radar direct-line and non-direct-line hybrid target positioning system. The millimeter-wave radar direct-line and non-direct-line hybrid target positioning system includes an electronic device, a radar, and a result output device. The radar and the result output device are respectively connected to the electronic device. The radar is used to acquire radar echo signals, and the result output device is used to output and display the joint positioning results.

[0114] For example, at the corner of a T-shaped corridor in a school building, due to wall obstruction, people on both sides of the corridor cannot see each other, easily leading to collisions. This system is deployed in this scenario. The radar is installed on the ceiling or wall at the corridor corner, and the electronic equipment is deployed in a nearby equipment room or integrated into the radar. The output device is a display screen or audible and visual alarm at the corridor corner. When someone is walking normally within the line-of-sight area of ​​the corridor, and someone is about to pass through the non-line-of-sight area on the other side of the corner, the radar transmits millimeter-wave signals in real time and receives multipath echoes reflected from the wall. The electronic equipment processes the echo signals, and through constructing a grid matching model of the search area and a deep learning-enhanced iterative optimization algorithm, it performs joint localization of the line-of-sight and non-line-of-sight targets. The output device displays the real-time position information of people on both sides and issues a warning when both are about to approach the corner, effectively preventing collisions.

[0115] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable system. The computer-readable system may be a disk, optical disk, read-only memory, or random access memory, etc.

[0116] The above provides a detailed description of the millimeter-wave radar direct-line and non-direct-line hybrid target localization method, apparatus, electronic device, and computer-readable system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for locating targets using a hybrid line-of-sight and non-line-of-sight approach with millimeter-wave radar, characterized in that, include: An observation vector is constructed based on the received radar echo signal, and a dictionary matrix is ​​constructed by discretizing the search area. The iterative optimization algorithm enhanced by deep learning generates regularization parameters corresponding to the first region and the second region, respectively, by taking the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input. The first region and the second region are obtained by dividing the search region. The sparse vector estimate is updated based on the regularization parameter, and the deep learning-enhanced iterative optimization algorithm is executed iteratively until the convergence condition is met, and the joint estimation result is output. Based on the joint estimation results, the true spatial location of each target is output to complete the joint localization.

2. The millimeter-wave radar hybrid target localization method combining direct and non-direct line-of-sight methods according to claim 1, characterized in that, The observation vector is constructed based on the received radar echo signal, and a dictionary matrix is ​​constructed by discretizing the search area, including: The received radar echo signal is down-converted and matched-filtered to obtain a discrete signal matrix, and the discrete signal matrix is ​​vectorized to obtain the observation vector. The search area is divided into a three-dimensional grid consisting of distance units, angle units, and Doppler units. Each grid point of the three-dimensional grid corresponds to a set of distance parameters, angle parameters, and Doppler parameters. A dictionary matrix is ​​constructed based on the three-dimensional mesh. Each column of the dictionary matrix is ​​obtained by vectorizing the transmit array guide vector and the receive array guide vector of the corresponding mesh point.

3. The millimeter-wave radar hybrid target localization method combining direct and non-direct line-of-sight approaches according to claim 1, characterized in that, The method further includes: Based on the current position of the millimeter-wave radar, determine the first vertical distance between the millimeter-wave radar and the first wall, the second vertical distance between the millimeter-wave radar and the second wall, and the third vertical distance between the second wall and the third wall; Calculate the first angle boundary, the second angle boundary, and the distance threshold based on the first vertical distance, the second vertical distance, and the third vertical distance; The first region is defined as the region in the search area where the angle of arrival of the target to be detected is less than or equal to the first angle boundary, the region between the angle of arrival of the target to be detected being greater than the first angle boundary and less than or equal to the second angle boundary, and the region where the echo distance is less than or equal to the distance threshold. The area between the angle of arrival of the target to be detected that is greater than the first angle boundary and less than or equal to the second angle boundary, and the area where the echo distance is greater than the distance threshold, is divided into the second region.

4. The millimeter-wave radar hybrid target localization method combining direct and non-direct line-of-sight approaches according to claim 1, characterized in that, The iterative optimization algorithm enhanced by deep learning generates regularization parameters corresponding to the first and second regions, respectively, using the residual between the sparse vector estimate of the current round and the observed vector, and the number of iterations as input. The diagonal loading parameters are calculated based on the residual between the sparse vector estimate of the current round and the observed vector using the deep learning-enhanced iterative optimization algorithm. Calculate the regularization parameter based on the diagonal loading parameter and the current iteration number; A weighted matrix in block diagonal form is constructed by setting a first weight value and a second weight value according to the region to which the grid point belongs. The second weight value is greater than the first weight value and is used to compensate for the energy attenuation of the signal in the second region. Regularization parameters corresponding to the first region and the second region are determined based on the regularization parameters, the first weight value, and the second weight value, respectively. The regularization parameters include region-specific regularization parameters and weighting matrices for the first region and the second region.

5. The millimeter-wave radar hybrid target localization method combining direct and non-direct line-of-sight approaches according to claim 1, characterized in that, The process of updating the sparse vector estimate based on the regularization parameter, simultaneously iteratively executing the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and outputting the joint estimation result includes: Construct a diagonal weighted matrix based on the regularization parameters; Update the sparse vector estimate for the current round based on the diagonal weighted matrix, the dictionary matrix, and the observation vector; The updated sparse vector estimate is used as the input for the next iteration. The steps of generating region-specific regularization parameters and weighting matrices and updating sparse vector estimates are repeated. The iteration stops when the iterative change or residual of the sparse vector estimate meets the convergence condition, and the output includes the joint estimation result of the target in terms of distance, angle and Doppler dimension.

6. The millimeter-wave radar hybrid target localization method combining direct and non-direct line-of-sight approaches according to claim 3, characterized in that, The step of outputting the true spatial location of each target based on the joint estimation result includes: Based on the first angle boundary, the second angle boundary, and the distance threshold, each target point in the joint estimation result is determined to be located in a region. The target points identified as the first region are confirmed as line-of-sight target positions, and the target points identified as the second region are mapped to their real coordinates in physical space through a symmetric projection function. The set of line-of-sight target positions and the set of non-line-of-sight target positions are output. The symmetric projection function is constructed based on the geometric relationship of wall reflection. It calculates the real coordinates of the target point in physical space using the echo distance and angle of arrival of the target point as input. Before the area determination, the two-dimensional area of ​​distance and angle is filtered according to the actual scene to remove target points located in the invalid area.

7. The millimeter-wave radar hybrid target localization method combining line-of-sight and non-line-of-sight as described in claim 1, characterized in that, The deep learning-enhanced iterative optimization algorithm is a network-enhanced iterative sparse reconstruction algorithm. The network-enhanced iterative sparse reconstruction algorithm replaces the fixed regularization parameters with learnable network parameters. In each iteration, the error between the current sparse vector estimate and the real scene scattering coefficient is calculated through the loss function. The network weights are updated through backpropagation, so that the regularization parameters and weighting matrix are adaptively adjusted with the iteration process. The network-enhanced iterative sparse reconstruction algorithm includes an input module, an initialization module, a target reconstruction and update module, a parameter training and adjustment module, a parameter adaptive update submodule, and a parameter momentum update submodule. The parameter adaptive update submodule dynamically generates regularization parameters, stability control terms, and periodic momentum coefficients based on the current iteration state and echo residuals. The parameter momentum update submodule uses historical gradient information to smoothly update the sparse vector to suppress iterative oscillations.

8. A millimeter-wave radar hybrid target positioning device combining direct-line and non-direct-line-of-sight methods, characterized in that, include: The signal processing module is used to construct observation vectors based on the received radar echo signals and to construct a dictionary matrix by discretizing the search area. The parameter generation module is used to generate regularization parameters corresponding to the first region and the second region respectively by using the iterative optimization algorithm enhanced by deep learning, with the residual between the sparse vector estimate of the current round and the observation vector and the number of iterations as input. The first region and the second region are obtained by dividing the search region. The iterative update module is used to update the sparse vector estimate based on the regularization parameter, and simultaneously execute the deep learning-enhanced iterative optimization algorithm until the convergence condition is met, and output the joint estimation result. The mapping output module is used to output the true spatial location of each target based on the joint estimation result, so as to complete the joint localization.

9. An electronic device, characterized in that, It includes a signal acquisition unit, a memory, and a processor, wherein the signal acquisition unit is used to acquire radar echo signals; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the millimeter-wave radar direct-line and non-direct-line hybrid target localization method according to any one of claims 1 to 7.

10. A millimeter-wave radar hybrid target localization system combining line-of-sight and non-line-of-sight methods, characterized in that, The millimeter-wave radar direct-line and non-direct-line hybrid target positioning system includes the electronic device, radar, and result output device as described in claim 9. The radar and the result output device are respectively connected to the electronic device. The radar is used to acquire radar echo signals, and the result output device is used to output and display the joint positioning results.

Images