Localization methods for locating at least one object using wave-based signals and a localization system

Abstract

The present invention relates to a localization method for locating at least one object using wave-based signals in which a wave field emanates from the object and the wave field emanating from the object is received by one or more receivers, in each receiver at least one measurement signal is generated using one or more receiving antennas of the receiver(s), which depends on the spatial and / or temporal distribution of the wave field and / or whose phase and / or amplitude profile is characteristically influenced by the signal propagation time from the object to the respective receiver, wherein a position of the object is determined using an artificial neural network and the measurement signal.

Description

[0001] The present invention relates to a location method for locating at least one object using wave-based signals, in which a wave field emanates from the object and the wave field emanating from the object is received by one or more receivers, in each receiver using one or more receiving antennas of the receiver or receivers, at least one measurement signal is formed which is dependent on the spatial and / or temporal distribution of the wave field and / or whose phase and / or amplitude curve is characteristically influenced by the signal propagation time from the object to the respective receiver.

[0002] The following state of the art is known: [BSLG22] BRÜCKNER, STEFAN ; SIPPEL, ERIK ; LIPKA, MELANIE ; GEISS, JOHANNA ; VOSSIEK, MARTIN: Phase Difference Based Precise Indoor Tracking of Common Mobile Devices Using an Iterative Holographic Extended Kalman Filter. In: IEEE Open Journal of Vehicular Technology Bd. 3 (2022), S. 55-67. [LiSV00] LIPKA, MELANIE ; SIPPEL, ERIK ; VOSSIEK, MARTIN: Locating method for localizing at least one object using wave-based signals and locating system. [PRDG17] PAVLENKO, T. ; REUSTLE, C. ; DOBREV, Y. ; GOTTINGER, M. ; JASSOUME, L. ; VOSSIEK, M.: Design and Optimization of Sparse Planar Antenna Arrays for Wireless 3-D Local Positioning Systems. In: IEEE Transactions on Antennas and Propagation Bd. 65 (2017), Nr. 12, S. 7288-7297. [ScHV23] SCHUESSLER, CHRISTIAN ; HOFFMANN, MARCEL ; VOSSIEK, MARTIN: Super-Resolution Radar Imaging With Sparse Arrays Using a Deep Neural Network Trained With Enhanced Virtual Data. In: IEEE Journal of Microwaves Bd. 3 (2023), Nr. 3, S. 980-993. [SGBG21] SIPPEL, ERIK ; GEISS, JOHANNA ; BRÜCKNER, STEFAN ; GRÖSCHEL, PATRICK ; HEHN, MARKUS ; VOSSIEK, MARTIN: Exchanging Bandwidth With Aperture Size in Wireless Indoor Localization - Or Why 5G / 6G Systems With Antenna Arrays Can Outperform UWB Solutions. In: IEEE Open Journal of Vehicular Technology Bd. 2 (2021), S. 207-217. [SHBU21] SCHÜßLER, CHRISTIAN ; HOFFMANN, MARCEL ; BRÄUNIG, JOHANNA ; ULLMANN, INGRID ; EBELT, RANDOLF ; VOSSIEK, MARTIN: A Realistic Radar Ray Tracing Simulator for Large MIMO-Arrays in Automotive Environments. In: IEEE Journal of Microwaves Bd. 1 (2021), Nr. 4, S. 962-974. [Sipp22] SIPPEL, ERIK: Holographic 3D Indoor Localization, Friedrich-Alexander University Erlangen-Nuremberg (FAU), 2022. [SLGH20] SIPPEL, ERIK ; LIPKA, MELANIE ; GEIß, JOHANNA ; HEHN, MARKUS ; VOSSIEK, MARTIN: In-Situ Calibration of Antenna Arrays Within Wireless Locating Systems. In: IEEE Transactions on Antennas and Propagation Vol. 68 (2020), No. 4, pp. 2832-2841. From [SGBG21] it is known that the localization accuracy of phase-difference-of-arrival (PDOA) systems depends directly on the aperture size of receiver arrays in relation to the measurement distance.

[0003] Known angle-based PDOA localization approaches do not work for large receiver arrays because a transmitter to be located is located in the near field of a receiver or a receiver array. This contradicts the assumption of plane received waves of angle estimators. Due to the spherical waves emitted by the transmitter, this assumption only applies to small spatial regions, especially in a far field, and thus to small receiver arrays. A near field is often referred to as a Fresnel region, where the Fresnel approximation, but not the Fraunhofer approximation, applies. A far field is often referred to as a Fraunhofer region, where the Fraunhofer approximation applies.

[0004] Furthermore, receiver arrays for angle estimation are typically fully populated with receiving antennas, with the entire area covered by receiving antennas, usually spaced at half the wavelength, which is very labor-intensive and costly for large receiver arrays. Therefore, for positioning with large receiver arrays, receiver arrays sparsely populated with receiving antennas are often used. These are referred to as "sparse arrays," as known from [PRDG17].

[0005] Consequently, precise PDOA-based positioning systems require the evaluation of raw data, for example complex amplitudes of the measurement signal, without angle estimation.

[0006] It is known from [BSLG22], [LiSV00], and [Sipp22] that the position can be efficiently determined by recursively evaluating the phase differences between the spatially distributed receiving antennas of the receiver arrays. This is usually done by applying recursive filters, in particular the extended Kalman filter in the form of the holographic extended Kalman filter (HEKF), as described in [BSLG22].

[0007] The metric is always implicitly "least squares." A position estimate of the transmitter's position thus implicitly assumes normally distributed additive noise, or "Additive White Gaussian Noise" (AWGN). Position estimation is therefore performed by inverting the phase measurement model.

[0008] The biggest challenge in locating a transmitter in a room, besides the implementation of the hardware, is the interference caused by multipath propagation of a wave-based signal via the reflecting environment, which superimposes the signal on the line of sight (LOS), as stated in [Sipp22].

[0009] The performance of direct least-squares localization is degraded because an AWGN assumption is incorrect.

[0010] In cases of strong multipath propagation, most localization approaches fail, especially when localizing in three dimensions.

[0011] Direct extraction of phases from complex-valued measurements for application in HEKF is no longer possible in cases of strong multipath propagation.

[0012] Different approaches already exist to suppress multipath propagation in localization systems, e.g. the use of multiple signal classification (MUSIC), compressed sensing, estimation of signal parameters via rotational invariant techniques, etc.

[0013] These separate the LOS signal and the signal by multipath propagation on the same hypothesis space, i.e. distance or angle, to isolate the LOS and then use the angle and / or distance of the LOS for localization.

[0014] For physically large receivers, which are necessary for precise PDOA positioning, differently shaped wavefronts result for the LOS and multipath propagation, since the far-field approximation no longer applies or is only partially applicable. Separating the LOS signal from the multipath propagation using known methods on the same hypothesis space is not possible.

[0015] It is known that receiver arrays are calibrated for intended use, as described in [SLGH20]. This applies both to receiver-specific parameters such as antenna phases and / or attenuation, antenna coupling, relative antenna positions, etc., as well as the global position of the receivers within the measurement environment.

[0016] It is known from [LiSV00] that at least implicitly an ideally calibrated measurement system is assumed.

[0017] Against this background, the present invention is based on the object of improving an above-mentioned method, in particular with regard to the evaluation of the measurement signal.

[0018] This object is achieved by the method having the features of independent claim 1. Advantageous developments of the invention are the subject of the dependent claims.

[0019] Accordingly, the invention provides that a position of the object is determined using an artificial neural network and the measurement signal.

[0020] Preferably, the measurement signal is used by the artificial neural network to determine and / or determine the position of the object.

[0021] Preferably, PDOA-based localization is performed using an artificial neural network. Preferably, PDOA measurements are performed by multiple receivers at spatially distributed receiving antennas to locate an object or transmitter.

[0022] The wave field preferably comprises or is formed by electromagnetic waves.

[0023] The wave field is preferably designed such that it corresponds to a communication standard, in particular any one, for example the Bluetooth, WLAN, 5G and / or 6G standard.

[0024] The object preferably comprises or forms a transmitter configured to emit the wave field. The transmitter can be a Bluetooth, Wi-Fi, 5G, and / or 6G transmitter.

[0025] The receiver(s) are preferably receiver arrays with multiple receiving antennas.

[0026] Localization can also be understood as locating or determining a position.

[0027] Preferably, the artificial neural network is used to invert a measurement model. Preferably, few to no prior assumptions about the measurement model are required.

[0028] Preferably, an artificial neural network is used for PDOA-based positioning with large receiver arrays.

[0029] The artificial neural network is preferably not used for angle estimation, but rather directly estimates and / or determines the position of the object or transmitter. The measured raw data or measurement signals can be preprocessed for this purpose.

[0030] Preferably, machine learning is used for PDOA-based localization.

[0031] The artificial neural network can take into account effects caused by shadowing of individual receivers or receiver arrays.

[0032] Preferably, it is provided that the position of the object is determined from the phase and / or amplitude curve of the measurement signal or only from the phase curve of the measurement signal.

[0033] Preferably, the position of the object is determined only from measured phase differences of at least two receiving antennas.

[0034] Preferably, a sample covariance matrix is ​​created from the measurement signal and input into the artificial neural network.

[0035] Preferably, only phase measurement values ​​are input into the artificial neural network.

[0036] Preferably, only measured phase differences from at least two receiving antennas are input into the artificial neural network.

[0037] Preferably, phases and / or amplitudes or their equivalent complex numbers are input into the artificial neural network.

[0038] Preferably, sample covariance matrices are input into the artificial neural network.

[0039] In particular, the phase differences between receiving antenna pairs can be determined and / or the phase and / or amplitude of individual channels can be calculated and input to the artificial neural network. Those skilled in the art will preferably be aware that phase and / or amplitude, or their course, can be equivalently described by complex numbers.

[0040] Furthermore, sample covariance matrices can be created from complex-valued received data or measurement signals. The sample covariance matrices can be input to the artificial neural network. The sample covariance matrices can be input to the artificial neural network according to their real and / or imaginary components and / or amplitude and / or phase.

[0041] Preferably, it is provided that the artificial neural network is trained using real and / or simulated measurement data, in particular before using the artificial neural network to determine the position of the object.

[0042] Preferably, the real and / or simulated measurement data include multipath propagation.

[0043] Due to its flexibility, the artificial neural network can take into account the effect of a multipath environment or multipath propagation.

[0044] Multipath propagation is preferably understood to mean that waves of the wave field do not travel and / or are transmitted directly on a straight line of sight or, in the sense of wave optics, a straight line of sight, but indirectly, i.e. not directly, for example via one or more surfaces which reflect the waves, between the object and the respective receiver.

[0045] Preferably, it is provided that, in at least one receiver, one or more parameters of the receiver are not calibrated or are only partially calibrated before the artificial neural network is taught and / or one or more of the parameters are also taught in the artificial neural network.

[0046] Preferably, the position and / or orientation of at least one receiver is not calibrated or is only partially calibrated before the artificial neural network is trained. Calibration of the receiver's global information, i.e., the overall positioning system, is preferably performed.

[0047] The artificial neural network can correct errors in the calibration of a receiver or receiver array or can be used without prior calibration.

[0048] Preferably, no or only erroneous prior information about at least one arrangement, position, orientation, phase offsets, and / or antenna coupling of at least one antenna, in particular a receiving antenna, is available at at least one receiver. Preferably, the local information and / or the receiver-specific parameters of the individual receivers are calibrated.

[0049] Preferably, the effect of multipath propagation is also learned.

[0050] Preferably, at least one receiver is large enough for the object to be in the near field of the receiver.

[0051] Preferably, at least one receiver is sparsely populated with receiving antennas.

[0052] The artificial neural network can efficiently handle sparsely populated receiver arrays or receiver arrays with few spatially distributed receiving antennas.

[0053] Preferably, additional sensor values, in particular an inertial sensor, a Doppler evaluation, a magnetic field-based position determination, an optical sensor and / or an ultrasonic sensor are taken into account for determining the position.

[0054] Preferably, the artificial neural network is or comprises a “recurrent neural network”.

[0055] In principle, all types of artificial neural networks are suitable. The artificial neural network can be or include a feedforward neural network, a convolutional neural network, and / or a recurrent neural network.

[0056] Preferably, the artificial neural network is a transformer network, in particular a recurrent neural network. A recurrent neural network is a particularly promising variant, since a recurrent neural network has a memory in which a system state can be stored according to a HEKF.

[0057] Preferably, the wavelength of a wave of the wave field is smaller than the maximum distance between two points within a space in which the object is to be located.

[0058] Preferably, the maximum distance between two points within a space in which the object is to be located is greater than the largest uniqueness range resulting from the phase differences between the receiving antenna pairs of individual receivers.

[0059] The invention also relates to a positioning system with at least one receiver for determining the position of at least one object and with means for carrying out the method according to the invention.

[0060] The positioning system may be or include a measuring system.

[0061] It should be noted here that the terms "a" and "an" do not necessarily refer to exactly one of the elements, although this represents a possible embodiment, but can also refer to a plurality of the elements. Likewise, the use of the plural also includes the presence of the element in question in the singular, and conversely, the singular also encompasses several of the elements in question. Furthermore, all features of the invention described herein can be combined with one another as desired or claimed in isolation from one another.

[0062] Further advantages, features, and effects of the present invention will become apparent from the following description of preferred embodiments with reference to the figures, in which identical or similar components are designated by the same reference numerals. Herein: Fig. 1: a flowchart of an embodiment of a method according to the invention. Fig. 2: a flow diagram of a further embodiment of a method according to the invention. Fig. 3: a sketch of an embodiment of a locating system according to the invention.

[0063] In Fig. 1, after the measurement signal has been formed from sensor data or from data which are determined by a receiver by means of a wave field emanating from an object in step S1, the measurement signal is preprocessed in step S2, followed by the use of an artificial neural network and the measurement signal in step S3 to determine and / or estimate the position of an object in step S4.

[0064] Fig. 1 thus shows an embodiment in which an artificial neural network is used to localize an object.

[0065] Training the artificial neural network for PDOA-based positioning is particularly challenging because the relative spatial distribution of the receivers and the resulting multipath propagation vary depending on the environment. Therefore, the artificial neural network is preferably retrained for each spatial configuration of the receivers.

[0066] To train the artificial neural network, measurement data generated by a reference measurement are preferably available as “ground truth” data.

[0067] Learning can be performed using real measurement data with prior information. This can be complex, but it can compensate for calibration errors of the receivers or receiver arrays and / or incorrectly assumed arrangements or positions of the receivers, and reduce the influence of multipath propagation.

[0068] The learning process can be performed using real measurement data with limited prior information. Preferably, all or individual receiver-specific parameters, in particular the relative antenna positions and / or the antenna phase offsets and / or the antenna coupling, in particular of the receiving antenna(s), are pre-calibrated, while the global positions and orientations of the receiver(s), in particular of the receiving antenna(s), are unknown. Thus, a partially calibrated measurement system or positioning system is preferably available.

[0069] The training can be performed using real measurement data without prior information. Preferably, the artificial neural network is trained without prior knowledge of the arrangement or position of the receiver(s) and / or the antenna positions, the antenna phase offsets, and / or the antenna coupling, in particular of the receiving antenna(s) of the receiver(s), relative to each other. Therefore, preferably, there is no ideally calibrated measurement system or positioning system.

[0070] Training can be achieved by first calibrating the entire measurement system and then simulating the artificial neural network based on the determined arrangement or positions of the receivers using a simple AWGN simulation model without multipath propagation. This advantageously enables simple training with minimal measurement effort and a very extensive data set, or training or learning data set.

[0071] Training can be achieved by first calibrating the entire measurement system and recreating the actual measurement environment in a simulation environment. The artificial neural network is then trained using the simulation environment. Compared to training using real measurement data, this results in a simple training process with minimal measurement effort and a very large data set. The influence of multipath propagation can be at least partially trained in. This approach has been successful in radar imaging systems. The measurement environment is preferably known and / or is measured at the same time.

[0072] Training can be achieved by first calibrating the measurement system and simulating the artificial neural network under changing multipath conditions. For this purpose, it is advisable to train the artificial neural network using different spatial geometries, e.g., from databases. Compared to training using real measurement data, this approach is advantageously simple and requires little measurement effort. The changing random multipath conditions allow the creation of an artificial neural network that is robust against multipath propagation and particularly adaptable to changing environmental conditions.

[0073] The teaching can be carried out by a combination of the above-mentioned teaching using real measurement data and the above-mentioned teaching using simulations.

[0074] Fig. 2 shows an embodiment of the method according to the invention with a learning and / or training of the artificial neural network.

[0075] In Fig. 2, after the measurement signal has been formed from sensor data or from data which are determined by a receiver by means of a wave field emanating from an object in step S1, the measurement signal is preprocessed in step S2, followed by the use of an artificial neural network and the measurement signal in step S3 to determine and / or estimate the position of an object in step S4.

[0076] Starting from step S1, training with ground truth data takes place in step S15, whereby a loss function is determined in step S5.

[0077] Fig. 3 shows a sketch of a hypothetical measuring environment 100, for example a room with an embodiment of a positioning system according to the invention.

[0078] The positioning system comprises a transmitter 10 and two receivers 40, each with multiple receiving antennas. The receivers are thus receiver arrays.

[0079] The transmitter 10 emits a wave-based signal or a wave field containing waves. The transmitter 10 or the object has a transmitting antenna.

[0080] The signal or waves of the wave field are received on the one hand directly via the direct LOS 20 and on the other hand via multipath propagation 30 by the two receivers 40, each of which has eight receiving antennas. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] BRÜCKNER, STEFAN ; SIPPEL, ERIK ; LIPKA, MELANIE ; GEISS, JOHANNA ; VOSSIEK, MARTIN: Phase Difference Based Precise Indoor Tracking of Common Mobile Devices Using an Iterative Holographic Extended Kalman Filter. In: IEEE Open Journal of Vehicular Technology Bd. 3 (2022), S. 55-67

[0002] LiSV00] LIPKA, MELANIE ; SIPPEL, ERIK ; VOSSIEK, MARTIN: Locating method for localizing at least one object using wave-based signals and locating system

[0002] PAVLENKO, T. ; REUSTLE, C. ; DOBREV, Y. ; GOTTINGER, M. ; JASSOUME, L. ; VOSSIEK, M.: Design and Optimization of Sparse Planar Antenna Arrays for Wireless 3-D Local Positioning Systems. In: IEEE Transactions on Antennas and Propagation Bd. 65 (2017), Nr. 12, S. 7288-7297

[0002] SCHUESSLER, CHRISTIAN ; HOFFMANN, MARCEL ; VOSSIEK, MARTIN: Super-Resolution Radar Imaging With Sparse Arrays Using a Deep Neural Network Trained With Enhanced Virtual Data. In: IEEE Journal of Microwaves Bd. 3 (2023), Nr. 3, S. 980-993

[0002] SIPPEL, ERIK ; GEISS, JOHANNA ; BRÜCKNER, STEFAN ; GRÖSCHEL, PATRICK ; HEHN, MARKUS ; VOSSIEK, MARTIN: Exchanging Bandwidth With Aperture Size in Wireless Indoor Localization - Or Why 5G / 6G Systems With Antenna Arrays Can Outperform UWB Solutions. In: IEEE Open Journal of Vehicular Technology Bd. 2 (2021), S. 207-217

[0002] SCHÜßLER, CHRISTIAN ; HOFFMANN, MARCEL ; BRÄUNIG, JOHANNA ; ULLMANN, INGRID ; EBELT, RANDOLF ; VOSSIEK, MARTIN: A Realistic Radar Ray Tracing Simulator for Large MIMO-Arrays in Automotive Environments. In: IEEE Journal of Microwaves Bd. 1 (2021), Nr. 4, S. 962-974

[0002] SIPPEL, ERIK: Holographic 3D Indoor Localization, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 2022

[0002] SIPPEL, ERIK; LIPKA, MELANIE; GEIß, JOHANNA; HEHN, MARKUS; VOSSIEK, MARTIN: In-Situ Calibration of Antenna Arrays Within Wireless Locating Systems. In: IEEE Transactions on Antennas and Propagation, Vol. 68 (2020), No. 4, pp. 2832–2841

[0002]

Claims

[1] Location method for locating at least one object using wave-based signals in which a wave field emanates from the object and the wave field emanating from the object is received by one or more receivers, in each receiver, using one or more receiving antennas of the receiver(s), at least one measurement signal is formed which depends on the spatial and / or temporal distribution of the wave field and / or whose phase and / or amplitude characteristic is characteristically influenced by the signal propagation time from the object to the respective receiver, characterized by , that a position of the object is determined using an artificial neural network and the measurement signal. [2] Method according to claim 1, wherein the position of the object is determined from the phase and / or amplitude curve of the measurement signal or only from the phase curve of the measurement signal. [3] Method according to claim 1, wherein the position of the object is determined only from measured phase differences of at least two receiving antennas. [4] Method according to one of the preceding claims, wherein a sample covariance matrix is ​​created from the measurement signal and input into the artificial neural network. [5] Method according to one of the preceding claims, wherein the artificial neural network is trained by real and / or simulated measurement data, in particular before the artificial neural network is used to determine the position of the object. [6] The method of claim 5, wherein the real and / or simulated measurement data comprise multipath propagation. [7] Method according to one of claims 5 to 6, wherein in at least one receiver, one or more parameters of the receiver are not calibrated or are only partially calibrated before the artificial neural network is taught and / or one or more of the parameters are also taught in the artificial neural network. [8] Method according to one of claims 5 to 7, wherein in at least one receiver a position and / or an orientation of the receiver is not calibrated or is only partially calibrated before the artificial neural network is taught. [9] Method according to one of claims 5 to 8, wherein no or only erroneous prior information about at least one arrangement, position, orientation, phase offsets and / or antenna coupling of at least one antenna is available at at least one receiver. [10] Method according to one of claims 5 to 9, wherein an effect of multipath propagation is also learned. [11] Method according to one of the preceding claims, wherein at least one receiver is so large that the object is in the near field of the receiver. [12] Method according to one of the preceding claims, wherein at least one receiver is sparsely populated with receiving antennas. [13] Method according to one of the preceding claims, wherein additional sensor values, in particular an inertial sensor, a Doppler evaluation, a magnetic field-based position determination, an optical sensor and / or an ultrasonic sensor are taken into account for determining the position. [14] Method according to one of the preceding claims, wherein the artificial neural network is or comprises a recurrent neural network. [15] Positioning system with at least one receiver for determining the position of at least one object and with means for carrying out the method according to one of the preceding claims.

Images