Three-dimensional tracking of a transmitter within a volume

Abstract

A system (100) that has the following features: a plurality of arrival angle receivers (142) operated such that they measure a plurality of directions between a transmitter (74) and the plurality of arrival angle receivers (142) (294), wherein the transmitter (74) moves within a volume (64); a plurality of arrival time receivers (122) configured to measure a plurality of distances between the transmitter (74) and the plurality of arrival time receivers (122) (294); and a processing circuit (104) coupled to the plurality of arrival angle receivers (142) and the plurality of arrival time receivers (122), wherein the processing circuit (104) is configured: to determine a position (78) of the transmitter (74) in three dimensions within the volume (64) based on the multiple directions and the multiple distances; to report the location (78) of the transmitter (74) to additional circuits (106), and to perform a two-stage localization operation (280) with the majority of directions and the majority of distances, wherein a first stage of the two-stage localization process (280) comprises determining a plurality of pseudo-locations (78) of the transmitter (74) within the volume (64) based on the plurality of distances, comprising a second stage of the two-stage localization process (280): a determination of one of the plurality of pseudo-locations as the location (78) of the transmitter (74) on the basis of the plurality of directions, and a removal of one or more non-visible pseudo-locations from the plurality of pseudo-locations that are hidden from one or more of the plurality of arrival angle receivers (142) prior to the determination of the location (78).

Description

[0001] The present description refers to a system and a method for the three-dimensional tracking of a transmitter within a volume.

[0002] Existing in-vehicle phone tracking systems have accuracy issues in three-dimensional space. Communication latency and sensor dependencies lead to inefficient localization solutions when tracking moving phones. Furthermore, angle-of-arrival techniques struggle when the field of view is limited.

[0003] DE 199 36 846 A1 discloses a method and apparatus for determining the position and speed of a mobile phone using either the time of arrival (TOA) of a signal transmitted by the mobile phone, its reception phase (POA), its reception frequency (FOA), or a combination thereof at multiple antennas located at a number of monitoring stations (MSs). To resolve the coordinates of the mobile phone, either hyperbolic multilateration based on the reception time difference (TDOA), linear multiangulation based on the reception phase difference (PDOA), or both are used. To resolve the speed of the mobile phone, FOA based on the reception frequency difference (FDOA) is used.

[0004] US Patent 2023 / 0007441 A1 discloses a system and a method for acquiring location data for a portable device relative to an object. The system and method may include an object device positioned in a fixed location relative to the object. The object device has an antenna configured to communicate wirelessly with the portable device via UWB over a communication link. The system may include a control system, such as a robot and / or a remote control, configured to acquire one or more samples relating to the communication between the portable device and the object device.

[0005] Accordingly, experts are continuing their research and development efforts in the field of locating a transmitter, e.g. a mobile phone, inside a vehicle.

[0006] A system is provided here. The system comprises multiple angle-of-arrival receivers, multiple time-of-arrival receivers, and a processing circuit. The multiple angle-of-arrival receivers measure multiple directions between a transmitter and the multiple angle-of-arrival receivers. The transmitter moves within a volume. The multiple time-of-arrival receivers measure multiple distances between the transmitter and the multiple time-of-arrival receivers. The processing circuit is connected to the multiple angle-of-arrival receivers and the multiple time-of-arrival receivers. The processing circuit is capable of determining the transmitter's position in three dimensions within the volume based on the multiple directions and multiple distances and of reporting the transmitter's position to additional circuits.

[0007] The processing circuit for determining the transmitter's location also performs a two-stage localization operation for the majority of directions and the majority of distances.

[0008] The first stage of the two-stage localization process involves determining a plurality of pseudo-positions of the transmitter within the volume based on the plurality of distances.

[0009] A second stage of the two-stage localization process involves determining one of the several pseudo-locations as the location of the transmitter based on the plurality of directions.

[0010] The second stage of the two-stage localization process involves removing one or more invisible pseudo-locations from the plurality of pseudo-locations that are hidden from one or more of the plurality of arrival angle receivers prior to determining the location.

[0011] In one or more embodiments of the system, the processing circuit for determining the location of the transmitter performs a time-dependent weighted linear least squares operation with the multiple directions and the multiple distances.

[0012] In one or more embodiments of the system, the time-dependent weighted linear least squares operation comprises a confidence analysis of a plurality of successive measurements of the plurality of directions and the plurality of distances.

[0013] In one or more embodiments of the system, the confidence analysis includes a spatial consistency analysis that determines a plurality of deviations of the plurality of sequential measurements from a global estimate, a temporal consistency analysis that calculates a plurality of standard deviations of the plurality of sequential measurements over time, and determines consistency based on the plurality of deviations and the plurality of standard deviations.

[0014] In one or more embodiments of the system, the time-dependent weighted linear least squares operation includes the calculation of a plurality of weights in response to a plurality of measurement times, a plurality of localization timestamps, and the consistency and construction of a weighted matrix based on the plurality of weights.

[0015] In one or more embodiments of the system, the transmitter is a mobile phone capable of transmitting a signal that can be detected by the majority of reception angle receivers and the majority of reception time receivers.

[0016] A method for the three-dimensional tracking of a transmitter within a volume is presented here. The method involves measuring multiple directions between the transmitter and multiple arrival-angle receivers. The transmitter moves within the volume. The method includes measuring multiple distances between the transmitter and multiple arrival-time receivers, determining a three-dimensional location of the transmitter within the volume using a processing circuit based on the multiple directions and distances, and reporting the transmitter's location from the processing circuit to additional circuits.

[0017] The procedure includes determining the location of the transmitter and performing a two-stage localization process for the majority of directions and the majority of distances.

[0018] The first stage of the two-stage localization process involves determining a plurality of pseudo-positions of the transmitter within the volume based on the plurality of distances.

[0019] A second stage of the two-stage localization process involves determining one of the several pseudo-locations as the location of the transmitter based on the plurality of directions.

[0020] The second stage of the two-stage localization process involves removing one or more invisible pseudo-locations from the plurality of pseudo-locations that are hidden from one or more of the plurality of arrival angle receivers prior to determining the location.

[0021] In one or more embodiments of the method, determining the location of the transmitter involves performing a time-dependent weighted linear least squares operation on the plurality of directions and the plurality of distances.

[0022] In one or more embodiments of the method, the time-dependent weighted linear least squares operation comprises analyzing confidence on a plurality of successive measurements of the plurality of directions and the plurality of distances.

[0023] In one or more embodiments of the method, the confidence analysis includes analyzing spatial consistency, which determines a plurality of deviations of the plurality of successive measurements from a global estimate; analyzing temporal consistency, which calculates a plurality of standard deviations of the plurality of successive measurements over time; and determining consistency based on the plurality of deviations and the plurality of standard deviations.

[0024] In one or more embodiments of the method, the time-dependent weighted linear least squares operation comprises the calculation of a plurality of weights in response to a plurality of measurement times, a plurality of localization timestamps, and the consistency and construction of a weighted matrix based on the plurality of weights.

[0025] A vehicle is provided here. The vehicle comprises a cabin, a plurality of arrival angle receivers, a plurality of arrival time receivers, and a processing circuit. The cabin defines a volume. The volume is sized to accommodate one or more occupants and a transmitter. The occupant(s) are able to move the transmitter within the volume. The plurality of arrival angle receivers are arranged in the cabin and serve to measure a plurality of directions between the transmitter and the plurality of arrival angle receivers. The plurality of arrival time receivers are arranged in the cabin and serve to measure a plurality of distances between the transmitter and the plurality of arrival time receivers. The processing circuit is connected to the plurality of arrival angle receivers and the plurality of arrival time receivers.The processing circuit is able to determine the position of the transmitter in three dimensions within the volume based on multiple directions and distances, and to report the position of the transmitter to additional circuits. Fig. Figure 1 is a schematic representation illustrating the context of a vehicle. Fig. Figure 2 is a schematic perspective diagram of an arrival time measurement in accordance with one or more exemplary embodiments. Fig. Figure 3 is a schematic perspective representation of an angle of incidence measurement according to one or more exemplary embodiments. Fig. Figure 4 is a diagram of the valid ranges of arrival angle receivers in accordance with one or more exemplary embodiments. Fig.Figure 5 is a schematic functional block diagram of a time-delay-dependent weighted sensor fusion technique according to one or more exemplary embodiments. Fig. Figure 6 is a diagram of a time-delay association for moving objects in accordance with one or more exemplary embodiments. Fig. Figure 7 is a schematic perspective representation of a localization according to one or more exemplary embodiments. Fig. Figure 8 is a schematic functional block diagram of a two-stage receive time / receive angle localization technique in accordance with one or more exemplary embodiments. Fig. Figure 9 is a schematic floor plan diagram of the results for a two-state localization method with arrival time and arrival angle in accordance with one or more exemplary embodiments. Fig.Figure 10 is a schematic top view of a non-visible transmitter according to one or more exemplary embodiments. Fig. Figure 11 is a schematic diagram of a neural sensor fusion network according to one or more exemplary embodiments. Fig. Figure 12 is a schematic representation of the architecture of a neural network in accordance with one or more exemplary embodiments.

[0026] Embodiments of the description provide a hybrid three-dimensional localization system with arrival time (TOA) / arrival angle (AOA) for in-vehicle smartphone tracking using ultra-wideband (UWB) sensors. To reduce communication latency and sensor dependency, the localization system incorporates fewer multi-antenna sensors than existing designs. Regarding line-of-sight limitations and accuracy issues, various localization system designs employ at least one of two techniques. A first technique accounts for signal reception delay and improves localization accuracy through a weighted strategy. A second technique focuses on balancing the contributions of arrival time and arrival angle during sensor fusion.

[0027] Fig.Figure 1 shows a schematic diagram illustrating a relationship with a vehicle 60. The vehicle 60 generally includes a cabin 62, which defines an interior volume 64. The cabin 62 is sized to accommodate one or more occupants 70, one or more mobile phones 72 (e.g., smartphones), and at least part of a system 100. The mobile phones 72 (e.g., transmitters 74) can transmit wireless signals 76, which are received by the system 100.

[0028] To clarify: A direction of travel of vehicle 60 from front to back can define a positive X-direction. A lateral direction of vehicle 60 from left to right (viewed from above) can define a positive Y-direction. A direction of vehicle 60 from bottom to top (viewed from one side of vehicle 60) generally defines a positive Z-direction. The X-direction, the Y-direction, and the Z-direction can be orthogonal to each other.

[0029] System 100 implements a localization system. System 100 generally comprises several (e.g., two) sensor mounts 102a-102b, a processing circuit 104, and additional circuits 106. In various embodiments, a front sensor mount 102a is placed within the volume 64 in or near a rearward-facing (e.g., in the positive X direction) front console. The mounting location [x, y, z] and orientation [orientation-azimuth, orientation-elevation] of the sensors in the front sensor mount 102a can be [0, 0, 0] in centimeters and [0, 0] in degrees. A rear sensor mount 102b is placed within the volume 64 near a rear window facing forward (e.g., in the negative X direction). The installation location [x, y, z] and the orientation [alignment-azimuth, alignment-elevation] of the sensors in the rear anchorage 102b can be [150, 0, 0] in centimeters and [180, 0] in degrees.Other placements and / or orientations can be implemented to meet the design criteria of a particular application.

[0030] Vehicle 60 can include, among other things, mobile objects such as a car, a truck, an autonomous vehicle, a gas-powered vehicle, an electric vehicle, a hybrid vehicle, a motorcycle, a boat, an agricultural vehicle, a train, and / or an aircraft. In some embodiments, Vehicle 60 can also include stationary objects such as buildings. Other types of Vehicle 60 can be implemented to meet the design criteria of a specific application.

[0031] The sensor anchors 102a-102b each have several (e.g., three) antennas for TOA and AOA sensors. The sensor anchors 102a-102b serve to determine the location 78 of the mobile phone 72 / transmitter 74. In various embodiments, the TOA sensors and / or the AOA sensors can be configured as ultra-wideband sensors. The sensor data signals 103a-103b can be forwarded to the processing circuit 104.

[0032] The processing circuit 104 contains one or more digital circuits. The processing circuit 104 is capable of transmitting the current location 78 of the transmitter 74 to the additional circuit 106. The processing circuit 104 can receive direction and angle data from the sensor armatures 102a-102b via the sensor data signals 103a-103b. The location data determined by the processing circuit 104 can be transmitted to the additional circuit 106 via a location signal 105.

[0033] In various embodiments, the processing circuit 104 is implemented as at least one microcontroller. The at least one microcontroller can comprise one or more processors, each of which can be implemented as a separate processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a dedicated electronic control unit. The at least one microcontroller can be an electronic processor (implemented in hardware, software running on hardware, or a combination of both). The at least one microcontroller can also include tangible, non-volatile memory (e.g., read-only memory in the form of optical, magnetic, and / or flash memory).For example, the at least one microcontroller may include application-appropriate amounts of random access memory, solid-state memory, flash memory and other types of electrically erasable, programmable solid-state memory, as well as accompanying hardware in the form of a high-speed clock or timer, analog-to-digital and digital-to-analog circuits and input / output circuits and devices, as well as suitable signal conditioning and buffer circuits.

[0034] Computer-readable and executable instructions embodying this method can be recorded (or stored) in memory and executed as described herein. The executable instructions can be a set of instructions used to run applications on the at least one microcontroller (either in the foreground or background). The at least one microcontroller can receive commands and information in the form of one or more input signals from various control units or components and transmit instructions to the other electronic components.

[0035] The additional circuit 106 contains further digital circuits. The additional circuit 106 is capable of performing a variety of operations on the location information received by the processing circuit 104. For example, the additional circuit 106 can determine whether an occupant 70 is in a rear seat of vehicle 60 by determining the location 78 of a mobile phone 72 near the rear seat. Other use cases can be implemented by the additional circuit 106 to meet the design criteria of a particular application.

[0036] In Fig. 2 is referred to Fig.Figure 1 shows a schematic perspective diagram 120 of an exemplary time-of-air measurement according to one or more exemplary embodiments. TOA measurements generally provide accurate distance estimates between the transmitter and the receiver. AOA measurements determine directional information about the signal arriving at the receiver from the transmitter. By using multiple antennas or antenna groups, the receiver can estimate the angle of arrival of the signal. In hybrid TOA / AOA localization, the hybrid tracking system 100, by combining distance and angle measurements, locates and tracks the transmitter 74 in both the distance and angle dimensions, with the final position in three-dimensional space being determined using a few anchor sensors 102a-102b.

[0037] The TOA measurement uses a first arrival time of the signal 76 and the front sensor anchor 102a (e.g., a TOA sensor 122) to calculate a first distance 124 (e.g., a radius of a first sphere 126). A second arrival time of the signal 76 and the rear sensor anchor 102b (e.g., another TOA sensor 122) is used to calculate a second distance 128 (e.g., a radius of a second sphere 130). Knowing a distance 132 between the TOA sensors 122, a transmitting device (e.g., the mobile phone 72) of the signal 76 can be located at several pseudo-locations (or points) along a circle 134 in a plane 136 where the first sphere 126 and the second sphere 130 intersect.

[0038] With reference to Fig. 3 and back to Fig.Figure 2 shows a schematic perspective diagram 140 of an exemplary angle of incidence measurement according to one or more exemplary embodiments. The AOA measurement uses one or more of the sensor anchors 102a-102b (e.g., AOA sensors 142) to measure one or more angles of arrival 144 of the signal 76. Each angle 144 generally defines a direction 146 in the direction of the transmitter 74. In conjunction with the circle 134 from the TOA measurements ( Fig. 2) The location 78 of transmitter 74 can be refined to a specific pseudo-location 148 where direction 146 intersects circle 134.

[0039] In some situations, the ability of the AOA sensor 142 to accurately determine the angles of arrival of the signal in certain spatial regions may be limited by restrictions in the coverage of the angles 144 from which the signals 76 can be accurately measured. Furthermore, measurements of the angle of arrival are generally considered less accurate than measurements of the time of arrival. Additionally, due to communication limitations, the independent sensors 122 and 142 typically do not provide measurements simultaneously. The measurements provided by sensors 122 and 142 may not be synchronized or time-aligned, leading to potential discrepancies or gaps in the data.

[0040] In Fig.Figure 4 is a diagram showing examples of valid ranges for AOA receivers according to one or more exemplary embodiments. Diagram 160 is shown as an example in two dimensions. Diagram 160 generally has an X-axis 162 in degrees and a Y-axis 164 in feet.

[0041] Curve 170 shows an example of the distance to transmitter 74. Curve 172 illustrates an example measurement of the arrival angle as detected by the AOA sensor at the front sensor anchor 102a. The AOA sensor at the front sensor anchor 102a can have valid measurement ranges 174 over several angle bands (or ranges). Curve 182 illustrates an example measurement of the arrival angle as detected by the AOA sensor at the rear sensor anchor 102b. The AOA sensor at the rear sensor anchor 102b can have valid measurement ranges 178 over several angle bands (or ranges). Therefore, the sensor anchors 102a-102b can continuously cover the arrival angles between zero degrees and 70 degrees.

[0042] In Fig.Figure 5 is a schematic functional block diagram of an example of a time-delay-aware weighted sensor fusion technique according to one or more exemplary embodiments. The time-delay-aware weighted sensor fusion technique 200 generally comprises steps (or blocks) 202 to 218, as shown. The inputs for the time-delay-aware weighted sensor fusion technique 200 include sequential TOA / AOA measurements 220a-220c over multiple measurement times 222. Location 78 can be represented as the output of the time-delay-aware weighted sensor fusion technique 200. The sequence of steps is shown as a representative example. Other step sequences can be implemented to meet the criteria of a particular application. The time-delay-aware weighted sensor fusion technique 200 can be implemented by System 100.

[0043] In step 202, the consecutive measurements (220a-220c) undergo data processing. The processed measurements can be fed to steps 204 and 208. In step 204, a sensor orientation mapping is performed. In step 206, the oriented measurement data is used to create an AOA matrix. The AOA matrix is ​​then fed to step 218.

[0044] In step 208, a coordinate transformation is performed on the processed measurements. The transformed measurements are used in step 210 to create a TOA matrix. The TOQ matrix is ​​presented in step 218.

[0045] In step 212, a confidence analysis of the successive measurements 220a-220c is performed. In step 214, a time / delay mapping is performed on the data obtained in step 212. The results of the time / delay mapping are used in step 216 to create a weighting matrix. The weighting matrix is ​​presented in step 218.

[0046] In step 218, a weighted least squares operation is performed based on the TOA matrix from step 210, the AO matrix from step 206, and the weight matrix from step 216. The weighted least squares operation generally yields location 78.

[0047] A confidence analysis of the sensors (e.g., measurement consistency, spatial consistency, temporal consistency) involves several calculations. A chi-square test measures the deviation of local measurements from a global estimate. Lower deviation values ​​indicate greater consistency and thus higher spatial consistency. The deviation values ​​can be determined using equations 1-3 as follows: Devs(j)=1m∑j=1m(Lj−L0)2

[0048] L is involved j the observed local measurement of sensor j and is determined as follows: Lj=[xj,yj,zj]=[xj+cos(θj−αj)(dj),yj+sin(θj−βj)(dj),zj+sin(θ''j−γj)(dj)]

[0049] L o is the global estimate, whereby: Lo=∑j=1m(Lj)m

[0050] Standard deviation: Calculate the standard deviation of the local measurements over time to check whether the signals behave predictably and stably over time. A lower standard deviation indicates greater temporal consistency and thus higher reliability. The standard deviation values ​​can be determined using Equation 4 as follows: Devt(k)=1n∑k=1n(xt−x^)2

[0051] where x t the measurement at time t; x̂ is the mean of the measurements; and n is the total number of measurements.

[0052] Taking into account both spatial and temporal consistency, the consistency is C j , defined as follows: Cj=1Devs(j)∗Devt(j)

[0053] In Fig. 6, which are based on Fig.Reference 1 shows a diagram of an exemplary time delay assignment for moving objects according to one or more exemplary embodiments. Diagram 240 shows timestamps 242, which progress in time (e.g., from left to right), and a position estimate 244.

[0054] To track the moving transmitter 74, closer measurements are usually more accurate due to signal delays (phase shifts).

[0055] The delay between the measurements and the localization timestamps is given as follows: δts = [δt1, δt2, ..., δt n ].

[0056] The weighting, W k , the TOA / AOA of for the k th The measurements received from sensor q are as follows: wk=δtk−max(δts)max(δts)−min(δts)∗Cj

[0057] In Fig.Figure 7 shows a schematic perspective diagram 260 of an example localization according to one or more exemplary embodiments. The diagram 260 illustrates the front sensor armature 102a relative to a transmitter 74 at position 78 in three-dimensional space (e.g., within volume 64).

[0058] Let us consider the technique of weighted linear least squares (LLS matrix). Given the weighting W k From TOA / AOA, the weighting matrix W is constructed with respect to each measurement as follows, based on the k-th received measurement from sensor q according to equation 6: w=(ω10⋯00ω2⋯0⋮⋮⋱⋮00⋯ωn)

[0059] Additionally, the hybrid TOA / AOA matrix can be defined as follows: A=[A1A2…An];B=[B1B2…Bn] Ak=[1,0,00,1,00,0,1];Bk=[xj+cos(θk−αj)(dk)yj+sin(θk−βj)(dk)zj+sin(θ'''k−γj)(dk)]

[0060] The measurement k this from the sensor anchoring j th 102a-102b observed, and: Wk=[δtk−max(δts)max(δts)−min(δts)∗Cj,0.00,δtk−max(δts)max(δts)−min(δts)∗Cj,00,0δtk−max(δts)max(δts)−min(δts)∗Cj]

[0061] Weighted linear least squares can be used for localization.

[0062] In Fig.Figure 8 is a schematic functional block diagram of an exemplary two-stage TOA / AOA localization technique according to one or more exemplary embodiments. The two-stage TOA / AOA localization technique 280 generally comprises steps (or blocks) 282 to 300, as shown. The inputs for the two-stage TOA / AOA localization technique 280 include sequential TOA / AOA measurements. Location 78 can be represented as the output of the two-stage TOA / AOA localization technique 280. The sequence of steps is shown as a representative example. Other step sequences can be implemented to meet the criteria of a particular application. The two-stage TOA / AOA localization technique 280 can be implemented by System 100.

[0063] In step 282, TOA measurements can be taken from sensor anchors 102a-102b. The TOA measurements are then fed to steps 284 and 298. In step 284, a dimensional reduction of the TOA measurements is performed. If no in-vehicle validation is in progress, the two-stage TOA / AOA localization procedure 280 is terminated in step 288. If in-vehicle validation is in progress, the two-stage TOA / AOA localization procedure 280 continues with step 290.

[0064] In step 290, the valid AOA ranges are checked. In step 292, a sensor selection is performed. The selection is passed to the AOA sensor. In step 294, the selected sensors generate the AOA measurements. The AOA measurements are made available in step 296. In step 296, a coordinate transformation is performed.

[0065] In step 298, the TOA measurements and the transformed AOA measurements result in a hybrid TOA / AOA localization using the least squares (LLS) method. The three-dimensional position 78 is output in step 300.

[0066] With reference to Fig. 9 and back to Fig. Figure 1 shows a schematic floor plan diagram 320 of an example result for a two-state TOA / AOA localization technique according to one or more exemplary embodiments. The diagram 320 shows a view measured from one of the sensor anchors 102a-102b along the X-direction. A first stage of the two-stage TOA / AOA localization is generally in Fig.Figure 2 illustrates this. In the first stage, the circle 134 is generally determined within the three-dimensional volume 64. Advantages of two-stage localization over raw three-dimensional space search include a higher priority for accuracy and improved (e.g., simpler) computational complexity.

[0067] The second stage of the two-stage TOA / AOA localization is generally in Fig. Figure 3 shows that a first AOA sensor can determine several pseudo-positions 322a-322c. A second AOA sensor can determine several pseudo-positions 322d-322f. In different embodiments, different techniques can be used to determine the location 78 ( Fig. 1) to be determined under pseudo-positions 322a-322f.

[0068] In various embodiments, a dimensional reduction can be performed for precise single-stage localization. Assuming that the sensor anchors 102a-102b are placed on a line such that y i = y j and z i = z j In the first step, the variable x can be removed using TOA measurements. The original formulas for the TOA measurements are reduced to: (x−xi)2+(y−yi)2+(z−zi)2=(di)2 (x−xj)2+(y−yj)2+(z−zj)2=(dj)2

[0069] The combination of equations 11 and 12 yields the x-coordinate of transmitter 74: x=(di)2−(dj)2−(xi)2+(xj)22(xj−xi)

[0070] In Fig.Figure 10 shows a schematic floor plan diagram 340 of an example of an out-of-sight transmitter in accordance with one or more exemplary embodiments. Valid AOA range verification can be performed to remove certain AOA data to avoid erroneous measurements of the out-of-sight transmitter 74's location 78. Due to the limited angular coverage of certain sensors, out-of-sight AOA data can be filtered out to improve localization accuracy. In the example, the transmitter 74 is located at a position 78 that is visible from the front sensor anchor 102a but out of sight from the rear sensor anchor 102b.

[0071] The sensor can be checked out of sight as follows: Rk=(dk)2−(x−xk)2

[0072] If arcsin (Rθk)>θk, Day θk and θ' k If the measurement is invalid, it will be rejected (mask m). k = 0), otherwise the anchor AOA measurement is accepted (mask m k = 1).

[0073] The computational complexity for the second stage of the hybrid TOA / AOA LLS localization technique can be reduced. After filtering out the unreliable measurements by creating a mask matrix with m k Can the y- and z-coordinates of transmitter 74 (e.g., location L) be estimated in the second stage using AOA from valid measurements? A=[mi,0mj,00,mj0,mj];B=[yi+sin(θi−βi)(di)yj+sin(θj−βj)(dj)zi+sin(θ'''i−γi)(di)zj+sin(θ'''j−γj)(dj)] L=AB−1

[0074] The basic symbols / equations for TOA / AOA localization are as follows. Several parameters are given in Table I for multiple (e.g., four) timestamps. Table I Timestamp anchoring Installation location Measured distance Azimut AOA ElevationAOA 1 Front anchor (x , y 11 , z)1 r1 θ1 θ'1 2 Rear anchorage (X , y 2 2 , z)2 r2 θ2 θ'2 3 Front anchor (x , y 3 3 , z)3 r3 θ3 θ'3 4 Rear anchorage (x, y 4 4 , z)4 r4 θ4 θ'4 TOA=(x−x1)2+(y−y1)2+(z−z1)2=r12 (x−x2)2+(y−y2)2+(z−z2)2=r22 (x−x3)2+(y−y3)2+(z−z3)2=r32 (x−x4)2+2=y−y AOA= (y−y1) / (x−x1)=tan(θ1) (y−y2) / (x−x2)=tan(θ2) (y−y3) / (x−x3)=tan(θ3) (y−y4) / (x−x4)=tan(θ4) (z−z1) / )(tanx−θ1) (z−z2) / (x−x2)=tan(θ'2) (z−z3) / (x−x3)=tan(θ'3) (z−z4) / (x−x4)=tan(θ'4)

[0075] Zusätzliche Symboldefinitionen: x j , y j , z j : Montageort of the j th sensor. α j , β j , y j : Buckling of the j th sensor. d k : die k th OLD-Messung. θ k : die k th Azimuth AOA-Messung. θ' k die k th AOA-Messung in der Höhe. L = [x, y, z]: from the Standard of Heaven. A, B: The matrix of the TOA- and AOA-related parameter values. FLUENT k , B k: the LLS matrix from the k th Measurement. W: the weighted matrix resulting from the trustworthiness analysis in the weighted LLS approach. W k : the weighted matrix from the k th Measurement. Additional symbols in the two-stage LLS. R k : the distance between the center of the circle (from the first stage of localization) and the k th Sensor. m k : the mask parameter for the j th Sensor (m k = 1: valid measurement; m k = 0: invalid measurement).

[0076] In Fig. 11 is referred to Fig. 1 and Fig.Figure 2 shows a schematic diagram of an exemplary neural sensor fusion network according to one or more exemplary embodiments. The neural sensor fusion network 360 generally comprises a local feature block 362, a sensor fusion feature block 364, and a neural network 366. The local feature block 362 includes the TOA sensors 122 and the AOA sensors 142. The sensor fusion feature block 364 includes a signal consistency block 370, a valid AOA range verification block 372, and a communication delay block 374. The neural network 366 comprises an identical subnetwork 38, a differential subnetwork 382, ​​and a multi-attention transformer encoder 384. The multi-attention transformer encoder 384 generates the location 78 in three dimensions.In various embodiments, the block 362 for local features, the sensor fusion block 364 and the neural network 366 can be implemented in the processing circuit 104.

[0077] The neural sensor fusion network 360 is capable of isolating measurement noise caused by dynamic sounds from a subset of sensors 122 and 142 in a wireless sensor network. By employing a Siamese neural network architecture and a multi-attention model, a small number (e.g., two) of the sensor sets 122 and 142 can be implemented while demonstrating adaptability to different noise patterns. The Siamese neural network generally uses the same weights while operating in tandem on two different input vectors to compute comparable output vectors.

[0078] The multi-attention model incorporates data from both block 362 for localization features and block 364 for sensor fusion features. The network processes direct measurements of arrival time and angle using the Siamese neural network, while simultaneously highlighting performance differences between sensors 122 and 142 by integrating additional features. Both the direct and additional features can be systematically processed by the sensor fusion neural network 360 to jointly localize transmitter 74 within three-dimensional volume 64.

[0079] Direct characteristics. The TOA measurements are used to estimate the distance D between the transmitter 74 and the receivers 122. Assuming the speed of light c and the time difference δt between sending and receiving the signal 76 for a round trip, the distance D is obtained from equation 19 as follows: D=(c×δt) / 2

[0080] Furthermore, the antenna arrays in the AOA sensors 142 can be used to measure the arrival angle of the incoming signals 76. By comparing the phase differences of signals 76 received by different elements of the antenna array, both the azimuth and elevation angles of arrival can be estimated. Assuming that d is the distance between the elements of the antenna array and γ is the wavelength of the signal 74, a phase difference ΔΘ between two adjacent antennas can be related to the arrival angle (AoA) according to Equation 20 as follows: AoA=arcsin(ΔΘ×γ) / (2π×D)

[0081] Block 364, with its sensor fusion features, can incorporate additional features, including arrival angle range validation, signal communication delay, and temporal signal consistency. Since the arrival angle is calculated by analyzing the phase differences between signals received by different antennas, such measurements may be considered less accurate than arrival time measurements.

[0082] To detect unreliable arrival angle data based on a pair of arrival time measurements, a verification approach can be implemented. Based on the sphere geometry (as in Fig.(Figure 2) shows the first sphere 126 and the second sphere 130 as centers relative to the mounting locations of the sensor anchors 102a-102b, and the arrival time measurements as radii. The intersection of the two spheres 126 and 130 forms circle 134. The ratio between the radius of circle 134 and the arrival time measurement indicates whether the transmitter 73 is within or outside the valid range.

[0083] Based on the above configuration, a pair of anchors is placed at (x i , y i , z i ) and x j , y j , z j ) assumed, where the exact distance measurements are given as d i and d j denoted as in equations 11, 12, and 13 above. To determine whether the measurement of the arrival angle by sensor j is reliable, a confidence indicator αj can be defined according to equation 21 as follows: αj=arcsin((dj)2−(x−xj)2θj)−θj

[0084] After that, block 372 can have an AoA verification factor as V j determine where V j = 1 only if α j <= 0; otherwise V j = 0. This allows the neural sensor fusion network 360 to validate the arrival angle data and avoid the use of erroneous measurements out of sight, resulting in more accurate hybrid localization by providing additional features as inputs for the neural network 366.

[0085] Block 374 can account for the signal transmission delay and the temporal consistency of the individual signals 74 to provide valuable features. Due to the signal delay when tracking a moving object, measurements that are closer in time tend to be more accurate. If we denote the communication delay of sensor j as δtj, then a normalized delay factor λ can be used. k Sensor j is defined by equation 22 as follows: λk=δtj−max(δt)max(δt)−min(δt)

[0086] Furthermore, define Cj to represent temporal consistency across n consecutive measurements, estimated by a standard deviation, to verify whether signal 76 has behaved predictably and stably over time. The temporal consistency can be determined in Block 370 according to Equation 23 as follows: Cj=11n∑k=1n(xt−x^)2

[0087] Fig.12 shows with reference to Fig. 2 and Fig.Figure 11 shows a schematic diagram of an exemplary architecture of a neural network in accordance with one or more exemplary embodiments. The architecture of the neural network 400 comprises several sets of sensors 122 and 124, several delayed, non-visual standard blocks 404, a layer 406, the multi-attention transform encoder 384, and a feedforward block 422 representing the positions 78 of the transmitters 74. The layer 406 may comprise several pairs of identical subnetworks 380 and differential subnetworks 382. Several input signals 402a-402n may be received by the several pairs of identical subnetworks 380 and differential subnetworks 382. The multi-attention transform encoder 384 generally comprises a feature fusion and embedding block 410, a position coding block 412, and a network 414.The network 414 comprises a multi-head attention block 416, a first addition and normalization block 418, a feed-forward block 420, and a second addition and normalization block 422. In various embodiments, the multi-attention encoder 384, layer 406, and feed-forward block 422 can be implemented in the processing circuit 104.

[0088] Neural network for sensor fusion. The use of multiple sensors is generally advantageous for the localization system, as it increases the range and improves accuracy through redundancy. Formulating the noise model and balancing the contributions of the different sensors can be challenging. Uncertain signal noise can arise from signal blocking, Gaussian noise, and signal reflections. Conventional least-squares triangulation and trigonometric functions cannot accurately capture the relationship between the location of transmitter 74 and the sensor measurements under such conditions. Therefore, a machine learning model is provided that incorporates multiple arrival time data points, multiple arrival angle data points, communication delays (λ), signal temporal consistency (C), and arrival angle validation factors (V) as input features for location estimation.

[0089] The design of the neural network 400 architecture generally comprises three sections: the identical subnetworks 380, the differential subnetwork 382, ​​and the block 364 for sensor fusion features. Based on the neural network 400 architecture, several (e.g., two) types of sensor fusion neural networks can be implemented: a semantic-function neural network (SFNN) and a fuzzy-feature neural network (T-SFFN) by Takagi-Sugeno. In SFNN, the network is trained directly on data collected in real time. In contrast, T-SFNN uses a knowledge transfer mechanism where the identical subnetworks share the same weights, which are transferred from the single-sensor localization network. These weights are then frozen in the second phase of training during sensor fusion.

[0090] Identical Siamese subnetworks. Assuming that each sensor follows the same logic for location determination in a self-coordinating system, identical subnetworks can be used to represent the logic of localizing a single sensor. Based on the Siamese network structure, where the subnetworks have the same weights and architecture, the identical subnetworks are built on each individual sensor, processing all local arrival time and arrival angle features in the same way.

[0091] The single-sensor neural localization network model, consisting of several (e.g., two) layers of interconnected neurons, learns to map input features (e.g., arrival time data and arrival angle) to an estimated position of transmitter 74. The weights from the internal layers are first trained and then transferred to the various sensors in the T-SFNN model.

[0092] Differential subnetwork: This component highlights the performance differences between multiple sensors. Although the data from each sensor are processed by common layers of the Siamese network to extract meaningful features, there is no guarantee that the sensors will have identical performance. The performance differences in sensor fusion can be attributed to the following three factors. Since the target may be located in regions outside the valid arrival angle range for a subset of sensors (as in Fig.(as shown in Figure 10), the reliability (α) varies between sensors when a sensor is intended to cover a large area (e.g., 180 degrees) but reliably obtains accurate measurements within a narrower area (e.g., 120 degrees). In a unicast model with time-weighted feedback (TWR), the communication delay (δt) also plays a role in emphasizing or diminishing the contribution of certain sensors, particularly in localizing a moving object. Furthermore, the temporal signal consistency (C) indicates whether some sensors have more difficulty than others providing accurate measurements in a noisy environment.

[0093] The differential subnetwork 404 can take the aforementioned features as input to generate an additional feature map for each sensor during localization. Although the subnetworks have the same structure, the weighting parameters can be updated in the second training stage.

[0094] The multi-attention encoder 384 is built on several identical and differential networks in layer 406. The multi-attention encoder 384 can be connected to the feed-forward network 422 to provide accurate three-dimensional coordinates during sensor fusion. This subnetwork extracts deep features from both the identical subnetwork 380 and the differential subnetwork 382 of each sensor. The position encoder 412 helps to distinguish features based on their positions / orders relative to the sensors, thereby capturing dependencies in the feature list. (For example, the ToA of sensor #1 may have a stronger dependency on both the ToAs of other sensors and the AoA of the same sensor, but is likely less relevant to the AoA of sensor #2 outside the field of view.)In addition to position encoding, the multiple attention heads 416 operate in parallel, allowing the model to jointly consider information from different representational subspaces at different positions. The feed-forward and normalization layers 418, 420, and 422 are added to further learn the logic of sensor fusion and accelerate convergence.

[0095] By utilizing attention mechanisms, the Encoder 384 can focus on more reliable sensor measurements and reduce the impact of noisy or erroneous data. The Encoder 384 dynamically adjusts the weighting of each sensor's contribution in real time based on data quality and consistency, thereby refining the estimation of location 78 even in complex scenarios where some sensors are outside the valid arrival angle range.

[0096] A hybrid TOA and AOA architecture is provided, minimizing the number of sensors (fewer dependencies and faster localization). By combining TOA and AOA, the system can rely on only one sensor anchor, resulting in much more efficient and faster localization. A time-weighted LLS method was developed to improve the accuracy of localizing the moving transmitter. This method considers both the reliability of the anchor sensors and communication latency when creating the weighted matrix. A two-stage localization technique, balancing the contributions of TOA and AOA, generally improves accuracy. In the first stage, TOA is used to categorize and filter out inaccurate measurements. In the second stage, TOA and AOA are used together to determine the final location.

[0097] Various implementations of the system generally enable a multitude of resident-centric applications (e.g., location-based personalization and automation). The system can be implemented lightweight and efficiently on embedded processors, while the transmitters are localized in three-dimensional space. In various implementations, the number of anchor sensors is reduced. Multiple antennas in modern sensors provide both distance and angle measurements, further reducing the number of anchor sensors required. The reduced number of anchor sensors also decreases hardware and system complexity and improves localization efficiency.

[0098] The system offers a hybrid TOA and AOA approach, minimizing the number of sensors and thus reducing dependencies. A time-weighted LLS method is implemented to improve the accuracy of locating moving transmitters. The two-stage localization technique balances the contributions of TOA and AOA to improve overall accuracy.

[0099] Embodiments of the description generally provide a system comprising multiple angle-of-arrival receivers, multiple time-of-arrival receivers, and a processing circuit. The angle-of-arrival receivers measure multiple directions between a transmitter and the receivers. The transmitter is movable within a volume. The time-of-arrival receivers measure multiple distances between the transmitter and the receivers. The processing circuit is connected to the angle-of-arrival and time-of-arrival receivers. Based on the directions and distances, the processing circuit determines the transmitter's location in three dimensions within the volume and transmits this location to additional circuitry.

Claims

[1] A system (100) that has the following features: a plurality of arrival angle receivers (142) operated such that they measure a plurality of directions between a transmitter (74) and the plurality of arrival angle receivers (142) (294), wherein the transmitter (74) moves within a volume (64); a plurality of arrival time receivers (122) configured to measure a plurality of distances between the transmitter (74) and the plurality of arrival time receivers (122) (294); and a processing circuit (104) coupled to the plurality of arrival angle receivers (142) and the plurality of arrival time receivers (122), wherein the processing circuit (104) is configured: to determine a position (78) of the transmitter (74) in three dimensions within the volume (64) based on the multiple directions and the multiple distances; to report the location (78) of the transmitter (74) to additional circuits (106), and to perform a two-stage localization operation (280) with the majority of directions and the majority of distances, wherein a first stage of the two-stage localization process (280) comprises determining a plurality of pseudo-locations (78) of the transmitter (74) within the volume (64) based on the plurality of distances, comprising a second stage of the two-stage localization process (280): a determination of one of the plurality of pseudo-locations as the location (78) of the transmitter (74) on the basis of the plurality of directions, and a removal of one or more non-visible pseudo-locations from the plurality of pseudo-locations that are hidden from one or more of the plurality of arrival angle receivers (142) prior to the determination of the location (78). [2] System (100) according to claim 1, wherein the processing circuit (104) is further configured to determine the location (78) of the transmitter (74): to perform a time-dependent weighted linear least squares operation on the majority of directions and the majority of distances. [3] System (100) according to claim 2, wherein the time-dependent weighted linear least squares operation comprises: a trustworthiness analysis (212) of a plurality of successive measurements of the plurality of directions and the plurality of distances. [4] System (100) according to claim 3, wherein the trustworthiness analysis (212) comprises: a spatial consistency analysis that determines a plurality of deviations of the majority of successive measurements from a global estimate; a temporal consistency analysis that calculates a plurality of standard deviations of the plurality of successive measurements over time; and Determine (298) consistency based on the plurality of deviations and the plurality of standard deviations. [5] System (100) according to claim 4, wherein the time-dependent weighted linear least squares operation comprises: Calculating a plurality of weights depending on a plurality of measurement times (222), a plurality of localization timestamps and consistency; and Construction of a weighted matrix (216) based on the plurality of weights. [6] A method (280) for three-dimensional tracking of a transmitter (74) within a volume (64): Measuring (294) a plurality of directions between the transmitter (74) and a plurality of arrival angle receivers (142), wherein the transmitter (74) moves within the volume (64); Measuring (282) a plurality of distances between the transmitter (74) and a plurality of arrival time receivers (122); Determining (298) a position (78) of the transmitter (74) in three dimensions within the volume (64) using a processing circuit (104) based on the multiple directions and the multiple distances; Reporting the location (78) of the transmitter (74) from the processing circuit (104) to additional circuits (106); and Performing a two-stage localization operation with the majority of directions and the majority of distances; wherein a first stage of the two-stage localization process (280) comprises determining a plurality of pseudo-locations (78) of the transmitter (74) within the volume (64) based on the plurality of distances, comprising a second stage of the two-stage localization process (280): a determination of one of the plurality of pseudo-locations as the location (78) of the transmitter (74) on the basis of the plurality of directions, and a removal of one or more non-visible pseudo-locations from the plurality of pseudo-locations that are hidden from one or more of the plurality of arrival angle receivers (142) prior to the determination of the location (78).

Images