Personal device sensing based on multipath measurements

Abstract

Certain aspects of the present disclosure provide techniques for training and using a machine learning model to predict a location of a stationary object and a location of a non-stationary object in a spatial environment. One example method generally includes measuring, by a device, a plurality of signals within a spatial environment. Timing information is extracted from the measured plurality of signals. And predicting the position of a stationary reflection point and the position of a non-stationary reflection point in the space environment based on a machine learning model, the measured multiple signals in the space environment and the extracted timing information. One or more actions are taken by the device based on predicting the position of a stationary reflection point and the position of a non-stationary reflection point in the spatial environment.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application serial number 17 / 934,598, filed on September 23, 2022, which is hereby incorporated by reference into this application.

[0003] introduction

[0004] Aspects of the present disclosure relate to using machine learning to detect objects and locations of objects in a spatial environment based on wireless communication data.

[0005] In wireless communication systems, measurements such as channel state information (CSI) measurements, signal strength measurements (e.g., received signal strength indicator (RSSI), reference signal received power (RSRP), etc.), and / or other types of measurements of wireless signals may be used for various purposes, such as locating a device or estimating the position of a device in a spatial environment. In one example, a device may perform a position estimate for itself or for other devices in a spatial environment based on triangulation or trilateration of signaling received from multiple anchors. The position of a device that sends signaling for position estimation may be identified using time difference of arrival (TDoA) and / or time of flight (ToF) information and angle of arrival (AoA) information, and thereby triangulate the position of the device in the spatial environment. In another example, fingerprinting based on data related to position information (e.g., received signal strength indicator (RSSI), channel state information (CSI) measurements, etc.) may be used to predict the positions of various objects in a spatial environment. However, these techniques may impose timing coordination requirements on anchors in the network, may be specific to a given spatial environment, and may rely on signaling between anchors and devices (e.g., between a transmitting device and a receiving device). Thus, these techniques may be applicable to certain circumstances and may involve signaling that may expose sensitive information about users in wireless communication systems.

[0006]

[0006] Therefore, there is a need for improved techniques for passive position estimation of devices in wireless communication systems. Summary of the invention

[0007] Certain embodiments provide a method for predicting stationary objects and non-stationary objects in a spatial environment using a machine learning model. An example method generally includes measuring multiple signals in a spatial environment by a device. Extracting timing information from the measured multiple signals. Determining the position of a stationary reflection point and the position of a non-stationary reflection point in the spatial environment based on the machine learning model, the measured multiple signals in the spatial environment, and the extracted timing information. Taking one or more actions by the device based on determining the position of the stationary reflection point in the spatial environment and the position of the non-stationary reflection point.

[0008] Certain embodiments provide a method for training a machine learning model to predict the position of a stationary object and the position of a non-stationary object in a spatial environment. An example method generally includes receiving a data set including signal measurements. Extracting a data set of timing information from the signal measurements. Training a machine learning model to predict the position of a stationary reflection point in a spatial environment and the position of a non-stationary reflection point in the spatial environment based on the data set of signal measurements and the data set of timing information.

[0009] Other embodiments provide a processing system configured to perform the aforementioned methods and those described herein; a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods and those described herein; a computer program product embodied on a computer-readable storage medium comprising code for performing the aforementioned methods and those further described herein; and a processing system comprising components for performing the aforementioned methods and those further described herein.

[0010] The following description and the associated drawings set forth in detail certain illustrative features of one or more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings depict certain aspects of the one or more embodiments and therefore are not to be considered limiting of the scope of the present disclosure.

[0012] Figure 1 An example of virtual anchor positioning in a spatial environment based on non-line-of-sight signals received at a receiving device is depicted.

[0013] Figure 2 An example environment is depicted in which wireless signals in a spatial environment reflect off fixed surfaces in the spatial environment.

[0014] Figure 3 Depicted are example operations for training a machine learning model to predict locations of stationary objects and locations of non-stationary objects in a spatial environment in accordance with aspects of the present disclosure.

[0015] Figure 4 Depicted are examples of predicting a location of an object in a spatial environment in which a transmitting device and a receiving device are decoupled in accordance with aspects of the present disclosure.

[0016] Figure 5 Depicted are example operations for predicting locations of stationary objects and non-stationary objects in a spatial environment based on a machine learning model and timing information extracted from CSI measurements in accordance with aspects of the present disclosure.

[0017] Figure 6 Depicted are example implementations of a processing system on which a machine learning model is trained to predict the positions of stationary objects and non-stationary objects in a spatial environment in accordance with aspects of the present disclosure.

[0018] Figure 7 Depicted are example implementations of a processing system on which a machine learning model is used to predict locations of stationary objects and locations of non-stationary objects in a device spatial environment in accordance with aspects of the present disclosure.

[0019] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION

[0020] Aspects of the present disclosure provide techniques for detecting objects in a spatial environment based on wireless sensing and machine learning models.

[0021] Position prediction (or estimation) can be a powerful tool to facilitate varied object sensing tasks such as intrusion detection, object counting, activity recognition and tracking, and boundary entry / exit detection. For example, a wireless device may use active positioning to predict the position of the wireless device in a space environment based on signals received from one or more transmitters (e.g., base stations, gNodeBs, wireless (e.g., Wi-Fi) access points, etc.) in the space environment. In another example, position estimation may be used in passive positioning. In passive positioning, a wireless device may use radio frequency measurements to predict the position of other devices in the space environment. Generally speaking, the position of other devices in the space environment may be determined based on disturbances to the wireless signal caused by objects that obstruct a direct line-of-sight path between a receiving device and a transmitting device.

[0022] In order to detect objects in a spatial environment and determine the positions of these objects in the spatial environment (e.g., relative to a reference point), various techniques can be used to define the spatial environment in which object detection and position determination are performed. For example, environmental fingerprinting can generally relate measured signal properties (e.g., CSI measurements, etc.) to a specific location within a specific spatial environment; however, environmental fingerprinting is specific to a given spatial environment and cannot be generalized to other spatial environments. Additionally, because position prediction in a spatial environment can be configured by personnel controlling the spatial environment and can be based on wide area signaling between a receiving device (e.g., a UE) and a transmitting device (e.g., a gNodeB), object detection and position prediction may not be personalized for a specific user and may expose information about the owner of the receiving device.

[0023] Various aspects of the present disclosure provide techniques that allow the use of signal measurements and machine learning models to sense or detect objects in a spatial environment. By sensing or detecting objects in a spatial environment based on signal measurements (such as CSI) associated with reflection points in the spatial environment (which can also be interchangeably referred to as "wave interaction points" and include reflection, refraction, absorption, and scattering characteristics), various aspects of the present disclosure can sense objects within the spatial environment without knowing the layout of the spatial environment. In addition, various aspects of the present disclosure can allow personalized sensing of objects within the spatial environment, for example, by allowing users to customize detection radius, sensitivity, and other parameters that can define what sensed objects are objects of interest in the spatial environment. Still further, because various aspects of the present disclosure can allow detection of resident objects and non-resident objects in a spatial environment without sending signaling from a receiving device to a transmitting device, various aspects of the present disclosure can maintain the privacy of devices used to identify objects within the spatial environment.

[0024] Example Multipath Wireless System

[0025] Figure 1 An example environment 100 is illustrated in which a non-line-of-sight component of a signal sent by a real transmitter to a UE may be considered a line-of-sight component of a signal sent by a virtual transmitter to the UE. As illustrated, the environment 100 includes a UE 102 positioned at p=(x,y) and a real transmitter 104 positioned at p0=(x0,y0). The line-of-sight component of the signal sent by the real transmitter 104 has a flight time τ0 to the UE 102 and an angle of arrival θ0, and the non-line-of-sight component of the signal sent by the real transmitter 104 has a flight time τ1 to the UE 102 and an angle of arrival θ1. The non-line-of-sight component is generated by the signal sent by the real transmitter 104 reflecting from a reflector 108 in a building environment (e.g., a wall or another surface in the environment 100 that may reflect the transmitted signal).

[0026] However, the reflected path of the non-line-of-sight signal may be equivalent to the direct path from the virtual transmitter 106 in the environment 100. For example, the non-line-of-sight may be equivalent to the line-of-sight path from the mirror image of the real transmitter 104, which is mirrored relative to the surface that reflects the non-line-of-sight component. Therefore, in the environment 100, the non-line-of-sight component of the signal transmitted by the real transmitter 104 with the flight time τ1 and the arrival angle θ1 to the UE 102 may be regarded as the line-of-sight component of the signal transmitted by the virtual transmitter 106 located at p1=(x1, y1).

[0027] The UE 102 may generate various measurements based on measurements of received signaling that may be used to identify the locations of the real transmitter 104 and one or more virtual transmitters 106 in the environment 100. For example, the UE may calculate CSI measurements, time of flight measurements, etc. The time of flight measurement ToF may be calculated according to the following formula:

[0028]

[0029] Where n represents the nth transmitter in the environment 100 (eg, the real transmitter 104 or the virtual transmitter 106), p represents the location of the UE 102, and p n represents the location of the nth transmitter, and c represents the speed of light.

[0030] In some aspects, UE 102 (representing a receiver) and real transmitter 104 may be distributed or co-located. Generally speaking, in a distributed system where UE 102 is located at a different location from real transmitter 104 (e.g., a base station, a wireless router, etc.), the UE may passively listen to signaling sent by real transmitter 104 (which, as discussed above, may be viewed as signals received from real transmitter 104 and one or more virtual transmitters 106) without sending any signaling itself. Meanwhile, when UE 102 and transmitter 104 are co-located, the UE sends and receives signaling, as described below with respect to Figure 2 In this case, the received signaling may be considered as signaling received from multiple virtual transmitters, which may allow positioning to be performed independent of other wireless infrastructure (eg, real transmitters 104) with which UE 102 may communicate.

[0031] Figure 2 An example environment 200 is illustrated in which a wireless communication system is deployed and in which wireless signals in the environment 200 are reflected from fixed surfaces in the environment 200. As shown, the environment 200 includes a user equipment (UE) 202 in the environment 200 defined by a fixed boundary 204, which receives signals from multiple virtual transmitters. In this example, signals may be sent by the UE 202, and reflections of these signals from other objects within the fixed boundary 204 or the spatial environment may be received at the UE 202 and treated as signals received from virtual transmitters located outside the fixed boundary 204. In some aspects, although not illustrated, the signals may also or alternatively be sent by one or more transmitters (not illustrated) located within the fixed boundary 204 and received at the UE 202.

[0032] As shown, signals in environment 200 are reflected from objects in environment 200. For example, signals 210, 214, 218, 222, and 226 correspond to different reflection paths of a signal transmitted from UE 202 reflected from fixed boundary 204. Meanwhile, signal 230 corresponds to a reflection of a signal transmitted by UE 202 off object 206, which, as shown, is a fixed object within boundary 204 of environment 200. In general, the distance traveled by a signal may be defined as the distance between UE 202 and a virtual transmitter associated with the signal, regardless of whether the signal is reflected directly from boundary 204 and / or object 206 or reflected from boundary 204 multiple times.

[0033] Thus, as shown, signals 210, 214, 218, 222, and 226 are signals transmitted to and received from virtual transmitter sources 208 located outside of boundary 204. Signal 210 transmitted from UE 202 at a compass bearing of 0° may be considered a signal transmitted to UE 202 from virtual transmitter 208A located at a distance from boundary 204 equal to the distance between UE 202 and boundary 204. Similarly, signal 214 may be considered a signal transmitted from virtual transmitter 208B located outside of boundary 204 at a compass bearing of 30°; signal 218 may be considered a signal transmitted from virtual transmitter 208C located outside of boundary 204 at a compass bearing of 90°; signal 122 may be considered a signal transmitted from virtual transmitter 208D located outside of boundary 204 at a compass bearing of 120°; and signal 226 may be considered a signal transmitted from virtual transmitter 208E located outside of boundary 204 at a compass bearing of 180°.

[0034] Furthermore, as shown, signal 230 is reflected from a stationary object 206 located within boundary 204. Thus, signal 230 may be considered to have been transmitted from a virtual transmitter 208F located at a distance from object 206 equal to the distance between UE 202 and stationary object 206. Thus, in this example, virtual transmitter 208F located outside boundary 204 may be considered to be the transmitter that generated signal 230. However, it should be appreciated that a signal reflected from a stationary object located within boundary 204 may be considered to be a signal transmitted from a virtual transmitter also located within boundary 204 (e.g., when the distance between the stationary object and the UE is less than half the distance between UE 102 and boundary 204 along the same compass bearing).

[0035] In some cases, signaling within environment 200 may include multipath components caused by reflections from fixed surfaces (e.g., boundary 204 and / or object 206). The multipath components of the transmitted signal may include a direct reflection from boundary 204 back to UE 202 and one or more indirect reflections from boundary 103 to the UE (e.g., signal 214 or signal 222 reflected from boundary 204 at a non-perpendicular angle). From the perspective of UE 202, each multipath component may be associated with unique timing information and angle of arrival information. Because each multipath component may be associated with unique timing information and angle of arrival information, each multipath component may effectively serve as a line-of-sight component from a different virtual transmitter in environment 200 (as described above with respect to FIG. 1 ). Figure 1 described).

[0036] Example machine learning model for identifying the localization of stationary objects and the localization of non-stationary objects in a spatial environment type

[0037] In order to use passive positioning techniques to identify stationary objects and non-stationary objects in a spatial environment (these stationary objects and non-stationary objects can be targets based on, for example, different machine-learnable signal propagation / reflection characteristics or signatures of the materials that make up these objects), signal measurements and timing information extracted from these timing measurements are used as inputs into a machine learning model that is trained to perform various object sensing tasks, such as predicting the positions of stationary objects and non-stationary objects in the spatial environment, probabilistic identification of objects entering and leaving a defined space, object counting, etc. The resolution at which these positions can be predicted can vary based on the bandwidth of the signaling used within the spatial environment and the frequency band over which the signaling is sent. For example, in the frequency range 1 (FR1) band used in 5G communications (e.g., using a bandwidth of 100+ MHz at frequencies below 6 GHz), the spatial resolution of the predicted (or determined) position can be approximately 3 meters. In the frequency range 2 (FR2) band used in 5G communications (e.g., a millimeter wave band at frequencies above 24 GHz), and at a bandwidth of 400 MHz, the spatial resolution of the predicted position can be as fine as approximately 0.75 meters (75 centimeters). In another example, for wireless signals in a Wi-Fi network (e.g., an 802.11ac or 802.11ax network) transmitted using a 160 MHz bandwidth in the 2.4 GHz to 5 GHz frequency band, the spatial resolution may be as fine as approximately 1.875 meters (187.5 centimeters).

[0038] Various models can be trained to detect stationary objects and non-stationary objects in a spatial environment based on signal measurements and timing information extracted from these signal measurements. For example, it can be assumed that humans or other moving objects act as physical filters that produce detectable signal reflections, refraction scattering, and penetration patterns. Therefore, machine learning models can be trained to distinguish stationary objects and non-stationary objects (e.g., humans in motion) in a spatial environment and predict the positions of stationary objects and non-stationary objects in a spatial environment relative to the device that performs position prediction on them. In some aspects, these models can also use Doppler shift information relative to the device that detects stationary objects and non-stationary objects in the spatial environment for signal measurements to detect not only the presence (or absence) of people in the spatial environment, but also human activities within the spatial environment. For example, limited or no Doppler shift between different times may indicate that humans are stationary, while other Doppler shift characteristics may indicate different types of activities, such as walking, running, jumping, etc.

[0039] In some aspects, a model that uses signal measurements and timing information extracted from these signal measurements to predict the presence of non-resident objects (e.g., humans in motion) in a spatial environment and the locations of these non-resident objects can be a Gaussian mixture model. In general, a Gaussian mixture model can be a probabilistic model implemented as a convolutional neural network that assumes that data points from the spatial environment are generated from a finite number of Gaussian distributions with unknown parameters. These Gaussian mixture models can be trained, for example, based on maximizing the energy of received signals.

[0040] In some aspects, the Gaussian mixture model can be a Bayesian model, where a probability distribution is used to represent uncertainty in the model. The Bayesian model can be defined according to the following formula:

[0041]

[0042] where p(θ) represents the prior distribution, M represents the number of mixture components (or clusters), N represents the number of observations, and λ i is the mixing weight (or prior probability) of the ith component, μ i represents the Gaussian mean of the i-th component, and σ i represents the Gaussian variance of the i-th component.

[0043] In some aspects, the Gaussian mixture model can be a posterior multivariate Gaussian mixture model, where the prediction of the location in the spatial environment is conditioned based on a set of features x. The set of features x can be, for example, features derived from signal strength and timing information selected from a set of non-human object interference (or other interference associated with objects other than a particular object or object type of interest) and noise measurements. x can be defined according to the following formula:

[0044]

[0045] This formula usually defines the signal model of a wireless multipath channel in the propagation space, where The phasor represented is defined according to the following formula:

[0046] e j =cos(wt)-j×sin(wt)

[0047] where wt is a real number and j 2 = -1. w represents the angular frequency and can be defined according to the following formula:

[0048] w=2×π×k

[0049] Term a i is a non-negative real number used to model the scaling of the i-th component, and the term t represents time. L typically represents the number of nominal multipath components in the propagation channel, and x is the sum of the L multipath signal components at the receiver. In some aspects, the feature set x can be selected to learn the time and space subintervals using various algorithms such as multiple signal classification (MUSIC), principal component analysis (PCA), etc.

[0050] The posterior multivariate Gaussian mixture model can be defined according to the following formula:

[0051]

[0052] in and Denotes a version of λ, μ, and σ generated based on the expectation maximization algorithm. The energy maximization algorithm generates λ, μ, and σ based on an iterative process of finding local maximum estimates of these parameters. In general, the prior distribution and the posterior distribution allow identification of the probability distribution of the CSI sample over the M multipath components identified in the sample.

[0053] In some aspects, a model that uses signal measurements and timing information extracted from these signal measurements to predict the presence of non-resident objects (e.g., humans or other objects in motion) in a spatial environment and the locations of these non-resident objects can be a probabilistic convolutional neural network. The convolutional neural network can be configured to predict the locations of resident objects and non-resident objects in a spatial environment based on the time and space segmentation of the measured signals. The probabilistic model may include one or more kernels having activation parameters associated with the detection of a human entering an area, which may be an a priori defined area or a user-defined area (e.g., as a radius of a device that monitors humans or other non-resident objects entering an area). In some aspects, the predictions made by the probabilistic model can be used to maintain a counter that tracks the number of humans entering an area over time.

[0054] In order to generalize the machine learning models for predicting or otherwise identifying the location of stationary objects and the location of non-stationary objects in a spatial environment to any spatial environment, the dataset of CSI measurements (or other signal measurements) used to train the machine learning models may include CSI measurements from environments different from the environment in which the machine learning models are deployed. For example, the dataset of CSI measurements may include CSI measurements from many different spatial environments with different characteristics (e.g., floor layouts, stationary reflection points, etc.). By training these machine learning models using a dataset of CSI measurements that is not specifically designed for a spatial environment, the machine learning models can be trained once and deployed for use in any spatial environment.

[0055] Figure 3 exemplifies the method that can be performed by a computing system (e.g., Figure 6 The system 600 illustrated in FIG. 1 and described below performs example operations to train a machine learning model to predict the locations of stationary objects and non-stationary objects in a spatial environment. Generally speaking, these objects can be considered reflection points from which wireless signals can be reflected toward the device.

[0056] As shown, operation 300 may begin at block 310 with receiving a data set of signal measurements. The data set of signal measurements may include, for example, a data set of CSI measurements. The measurements included in the data set of signal measurements may be measurements from an environment different from the spatial environment in which the machine learning model is deployed, such that the machine learning model is decoupled from the specific spatial environment.

[0057] At block 320, operation 300 continues with extracting a data set of timing information from the signal measurements. As discussed, to extract timing information from the signal measurements, various techniques may be used to identify a virtual transmitter associated with a location from which the signal was reflected (e.g., reflected to another reflection point, reflected back toward a measurement device, etc.). The timing information for any given signal measurement may be determined based on angular information (e.g., angle of arrival), the location of the device generating the measurement, the location of the transmitting device, etc.

[0058] In general, the location of the transmitting device may include a real transmitter and a virtual transmitter associated with a reflection in the spatial environment. To account for the increased amount of time it takes for the reflected signal to reflect back to the device generating the measurement, the virtual transmitter associated with the reflected signal may be considered as a transmitter located outside of the spatial environment generating the measurement. For example, if the spatial environment is a room in a building, the virtual transmitter may be located in a different room of the building or even a different building.

[0059] At block 330, operation 300 continues with training a machine learning model to predict the locations of stationary reflection points in the spatial environment and the locations of non-stationary reflection points in the spatial environment based on the data set of signal measurements and the data set of timing information. As discussed, the machine learning model may include a Gaussian mixture model, a probabilistic convolutional neural network, or other machine learning models that may be used to predict the locations of reflection points in the spatial environment given the input of signal measurements and timing information derived from the signal measurements.

[0060] In general, the characteristics of signals reflected from stationary objects (e.g., walls, pillars, ceilings, etc. in a spatial environment) differ significantly from the characteristics of signals reflected from non-stationary objects. In some aspects, to exploit these differences, supervised learning techniques can be used to train machine learning models based on labeled signal measurements and timing information indicating whether the signal measurements are associated with stationary objects or non-stationary objects. The resulting model can be a probabilistic model that can generate probabilities indicating the likelihood that any given measurement is associated with a stationary object or a non-stationary object. In some aspects, self-supervised learning or semi-supervised learning techniques can be used to train machine learning models, which can allow the use of unlabeled data or partially annotated ground truth data to train the model.

[0061] In some aspects, when the machine learning model is a Gaussian mixture model, the Gaussian mixture model can be a Bayesian model or a posterior multivariate Gaussian mixture model. The Gaussian mixture model can be trained based on maximizing the energy of the received signal.

[0062] In some aspects, where the machine learning model is a probabilistic convolutional neural network, the machine learning model may be trained to predict the location of stationary reflection points and non-stationary reflection points (corresponding to stationary and non-stationary objects in the spatial environment) relative to the device based on the time and space segmentation of the measured signal. As discussed, the time and space segmentation of the measured signal may result in the use of various time and space sub-interval algorithms (e.g., MUSIC, PCA, etc.) to create a data set, so that the data set includes signal measurement information for stationary objects in the spatial environment and timing information derived from the signal measurement information. The probabilistic convolutional neural network may include one or more convolution kernels having activation parameters associated with detecting a human (or other type of mobile object of interest) entering an area. In some aspects, the probabilistic convolutional neural network may be configured to identify a human (or other mobile object) entering an area (which may be defined based on an a priori fixed radius from the device or a radius defined by a user of the device) and maintain a counter that tracks the number of humans (or other mobile objects) entering the area over time.

[0063] In some aspects, the location of non-stationary reflection points in the spatial environment may include the location of humans in motion in the spatial environment. Whether a human is present in the spatial environment and the location of the human may be determined, for example, based on Doppler shift or other information that can be used to identify the motion of objects in the environment. For example, different Doppler shift characteristics may indicate motion toward the device (e.g., when the timing information derived from the signal measurement decreases between different samples), motion away from the device (e.g., when the timing information derived from the signal measurement increases between different samples), or different types of motion (e.g., walking, running, jumping, etc. based on the magnitude of Doppler shift, where slower motion is associated with lower Doppler shift magnitudes than faster motion).

[0064] In general, a machine learning model can be trained to detect stationary objects and non-stationary objects and predict or otherwise determine the locations of these objects in any spatial environment (subject to resolution limitations associated with the frequency bands and bandwidths over which signaling is sent and received in a wireless communication network implemented in the spatial environment), without regard to the specific layout of the spatial environment in which object detection and location estimation are performed. Because the model can be trained to distinguish between stationary objects and non-stationary objects, the model does not need to be trained using signal strength fingerprints specific to a particular spatial environment (e.g., a particular floor plan of a room or building). Therefore, the model can be portable and can be used across different devices in different spatial environments without being customized for any particular user or any particular spatial environment. In addition, because the machine learning model uses signal reflections from various objects within the spatial environment to detect stationary objects and non-stationary objects and predict / determine the locations of these objects, sensitive or otherwise private information may not be exposed to network operators, unlike models that use transmitter-to-receiver (e.g., base station to user equipment or user equipment to base station) signaling to perform various object detection and location prediction tasks.

[0065] Use machine learning models to predict the location of stationary objects and non-stationary objects in a spatial environment Example

[0066] After training, the above-described machine learning model can be deployed (e.g., deployed to a user equipment (UE) or other terminal device in a wireless communication system) for detecting the presence and location of stationary objects and the presence and location of non-stationary objects in a spatial environment. As discussed, because the machine learning model described herein can be trained using a data set of signal measurements captured from many different spatial environments and timing information derived from these signal measurements, the machine learning model described herein can predict the location and presence of stationary objects and the location and presence of non-stationary objects in any spatial environment, and does not need to be trained to make predictions for a specific spatial environment.

[0067] The above machine learning models can be used for a variety of range-based or location-based tasks. For example, the machine learning model can be used to detect humans by treating humans as physical filters that produce detectable patterns (e.g., detectable patterns of signal measurements and timing information derived from these signal measurements). In addition, the model can be trained to deactivate responses to non-human objects that are typically associated with different signal measurement and timing information patterns from humans, and can be trained to detect human activity using Doppler shift information (as discussed in further detail below).

[0068] In another aspect, machine learning models can be used for time selection and tracking. In general, a region can be defined relative to a transmitter and a receiver (which can be co-located or distributed, as discussed above). Predictions of the presence of non-resident objects and the locations of these non-resident objects (e.g., relative to one or both of the transmitter and / or receiver) can be used to identify the entry of non-resident objects into the region within a given time period and track the movement of the non-resident objects in the region.

[0069] In yet another aspect, the machine learning models described herein can be used for directional selection and tracking. The machine learning model can be trained to predict the direction in which a non-resident object is moving relative to a transmitter and a receiver (again, the transmitter and the receiver can be co-located or distributed). Objects predicted to be approaching the transmitter and / or receiver can be tracked, while objects predicted to be moving away from the transmitter can be (at least temporarily) ignored.

[0070] In some aspects, as discussed in further detail below, the machine learning model can use probabilistic mixture modeling to enable detection of non-resident objects (e.g., humans) within a spatial environment. Based on detecting the presence of non-resident objects, counters can be maintained to track the number of non-resident objects that have entered an area defined relative to a transmitter and / or receiver, the number of non-resident objects that are currently located within an area defined relative to a transmitter and / or receiver, etc.

[0071] The machine learning model can be further personalized for a specific user. For example, a user can define specific measurements (e.g., mass, shape, activity pattern, etc.) that can be ignored by the model. Thus, the model can be retrained (or at least refined) to treat objects that meet these defined measurements as objects to be ignored for tracking purposes. Subsequently, when the machine learning model receives signal strength measurements and timing information associated with an object with these defined measurements, the machine learning model can treat the signal strength measurements and timing information as data belonging to a resident object (or other ignored object) rather than marking the object as a non-resident object of interest (e.g., for location prediction, tracking, counting, etc., as discussed above).

[0072] In one aspect, the transmitter and the receiver may be co-located with each other. For example, the transmitter and the receiver used to generate signaling in the space environment and to generate measurements based on such signaling may be components of the same device (e.g., a transceiver included in a UE). In this case, based on the predicted position of a stationary object and the position of a non-stationary object, the device may eliminate various components within multiple signals measured by the device in the space environment. For example, the device may apply various orthogonal or partition codes (e.g., Walsh codes) to eliminate signals associated with various components, or may use various transmission and reception logic or circuits to eliminate these signals. In one example, in the case where a signal associated with a non-stationary object (or a reflection point) is of interest, the device may eliminate the signal associated with the stationary object to reduce the number of signals to be processed. By doing so, an increase in the number of measured signals may indicate the entry of another non-stationary object into the space environment, while a decrease in the number of measured signals may indicate that a non-stationary object leaves the space environment.

[0073] In some aspects, predictions of the locations of stationary objects and non-stationary objects can be used to refine the machine learning model in accordance with known stationary and non-stationary objects (e.g., from a training data set used to train the machine learning model). For example, the machine learning model can be retrained so that the model ignores some objects as non-stationary objects based on the correlation between the radio measurements associated with the objects and the size and shape information associated with the objects. For example, assume that the machine learning model is used to detect the presence of humans in a space environment. Due to the similarities between humans and other living organisms in reflecting radio signals, the model may mark these other living organisms as humans entering and leaving the space environment. However, the model can be retrained to ignore some signals when detecting the presence or absence of humans in the space environment because humans may have different sizes and shapes than other living organisms that may be present in the space environment (e.g., dogs, cats, etc.).

[0074] In some aspects, in order to identify the entry of an object into a spatial environment (e.g., an area defined based on a radius from the device), the device may monitor the predicted (or estimated or determined) position of a non-stationary reflection point in the spatial environment over a period of time. For example, assume that the spatial environment in which the device operates is larger than the area defined based on the radius from the device. If the device predicts (or estimates or determines) that the non-stationary reflection point is outside the area defined based on the radius from the device at time t-1, and predicts (or estimates or determines) that the non-stationary reflection point is within the area at time t, the device may determine that the object associated with the non-stationary reflection point has entered the area. In response, the device may generate an alarm indicating the entry of the object into the area. The device may also maintain a counter of objects in the area, increment the counter when it is determined that the object has entered the area, and (in some aspects) decrement the counter when it is determined that the object has left the area.

[0075] In some aspects, the device may detect or predict other information about the location and presence of non-resident reflection points in the spatial environment. For example, based on the predicted location of non-resident reflection points in the spatial environment over time, angle information relative to the device may be predicted for objects associated with the non-resident reflection points. Thus, in addition to predicting when a non-resident object (e.g., a human) has entered an area defined based on a radius from the device, the device may also predict the direction in which the non-resident object will approach the device.

[0076] The radius from the device based on its defined area can be associated with the characteristics of the radio technology used to receive signals in the spatial environment. For example, the radius can be based on the frequency band in which multiple signals are received or the bandwidth in which multiple signals are received. In general, the radius can decrease with increasing bandwidth and increasing frequency band, and can increase with decreasing bandwidth and decreasing frequency band to accommodate the spatial resolution achieved by using different bandwidths and frequency bands for sending and receiving signaling in the wireless communication network, as discussed above.

[0077] In some aspects, such as Figure 4As shown, the transmitter and receiver may be distributed in the space environment 400. The transmitter may be located at a first focus 410 in the ellipse 405, and the receiver may be located at a second focus 420 in the ellipse 405 (or vice versa). In some aspects, the transmitter and receiver may be time-synchronized peer devices that coordinate to send and receive signaling in the space environment (e.g., to enable device-specific or manufacturer-specific functionality in the wireless communication system). In some aspects, these time-synchronized peer devices may operate cooperatively with each other using manufacturer-specific signaling to implement manufacturer-specific functionality in the wireless communication system. When coordinating the transmission and reception of signaling, the transmitter and receiver (located at the foci 410 and 420 of the ellipse 405) may coordinate the timing and arrival angle of the signal so that the peer devices can eliminate some components of the received signal based on the predicted positions of stationary objects and the predicted positions of non-stationary objects in the space environment. The prediction of the presence of non-stationary objects and the positions of those non-stationary objects can therefore be performed based on the distance from the transmitter and the distance from the receiver. Furthermore, because position prediction may be performed based on transmitters and receivers distributed within a spatial environment, the resolution at which position prediction may be performed may be increased relative to the resolution at which position prediction is performed when the transmitters and receivers are co-located.

[0078] In some aspects, an object may be detected based on triangulation from a first focus and a second focus (eg, Figure 4 430) into the area defined by the ellipse. In general, the position of the object may be defined by the distance from each focus and the angle from each focus. Because the distance between the first focus and the second focus may be known, the position of the object in the spatial environment may be considered to be the final vertex of the triangle formed by the position of the object, the first focus and the second focus. As with the example of the transmitter and receiver co-location discussed above, if the device predicts that the non-stationary reflection point is outside the area defined by the ellipse at time t-1 and predicts that the non-stationary reflection point is within the area at time t, the device may determine that the object associated with the non-stationary reflection point has entered the area. In response, the device may generate an alarm indicating the entry of the object into the area. The device may also maintain a counter of objects in the area, increment the counter when it is determined that the object has entered the area, and (in some aspects) decrement the counter when it is determined that the object has left the area.

[0079] In some aspects, the device may detect or predict other information about the location and presence of non-resident reflection points in the spatial environment. For example, based on the predicted location of the non-resident reflection points in the spatial environment over time, angle information relative to the device may be predicted for objects associated with the non-resident reflection points. Thus, in addition to predicting when a non-resident object has entered the area defined by the ellipse, the device may also predict the direction in which the non-resident object will approach the device.

[0080] Figure 5 The example may be provided by a computing device (e.g., Figure 7 5. The system 700 illustrated in FIG. 5 is performed to use a machine learning model to predict the locations of stationary reflection points and non-stationary reflection points in a spatial environment (these stationary reflection points and non-stationary reflection points may correspond to stationary objects and non-stationary objects). The device may be, for example, a smart phone, tablet, laptop, wearable device, or other computing device that can receive signaling in a wireless network, measure such signaling, extract timing information from such signaling, and predict the locations of stationary reflection points and non-stationary reflection points based on the measurements and timing information.

[0081] As shown, the operation 500 begins with a block 510 of measuring a plurality of signals within a space environment. These measurements may be, for example, CSI measurements generated based on various signals transmitted by the device and reflected back to the device by stationary and non-stationary objects in the space environment. These signals may include, for example, CSI reference signals used to measure various signal quality metrics in a wireless communication system or other reference signals that may be transmitted by a device and reflected back to the device by reflection points in the space environment.

[0082] At block 520, operations 500 continue with extracting timing information from the measured plurality of signals. As discussed, to extract timing information from the signal measurements, various techniques may be used to identify a virtual transmitter associated with a location from which the signal was reflected (e.g., reflected to another reflection point, reflected back toward a measurement device, etc.). The timing information for any given signal measurement may be determined based on angular information (e.g., angle of arrival), the location of the device generating the measurement, the location of the transmitting device, etc.

[0083] At block 530, operation 500 continues by determining the measured plurality of signals within the spatial environment and timing information extracted from the measured plurality of signals, the location of stationary reflection points in the spatial environment, and the presence of non-stationary reflection points based on the machine learning model. As discussed, the machine learning model may include a Gaussian mixture model, a probabilistic convolutional neural network, or other machine learning models that may be used to predict the location of stationary reflection points and the location of non-stationary reflection points in the spatial environment given the input of signal measurements and timing information derived from the signal measurements.

[0084] In some aspects, when the machine learning model is a Gaussian mixture model, the Gaussian mixture model can be a Bayesian model or a posterior multivariate Gaussian mixture model. The Gaussian mixture model can be trained based on maximizing the energy of the received signal.

[0085] In some aspects, where the machine learning model is a probabilistic convolutional neural network, the machine learning model can be trained to predict the locations of resident reflection points and non-resident reflection points based on temporal and spatial segmentation of the measured signal.

[0086] At block 540, the operations 500 continue with taking one or more actions at the device based on determining the locations of the stationary reflection points and the non-stationary reflection points in the spatial environment.

[0087] In some aspects, the device may include a co-located transmitter and receiver. The one or more actions may include canceling one or more components within the plurality of signals based on the determined locations of stationary reflection points in the spatial environment. By canceling the components based on the determined locations of stationary reflection points in the spatial environment, the device may generate a signal including components primarily associated with non-stationary reflection points in the spatial environment.

[0088] In some aspects, the one or more actions may include detecting entry of a specified object or object type into an area defined by a radius from the device based on the determined location of a non-stationary reflection point in the spatial environment. Based on detecting entry of the specified object or object type into the area, an alarm indicating entry of the specified object or object type into the area may be generated at the device.

[0089] In some aspects, the one or more actions may include detecting entry of a specified object or object type into an area defined by a radius from the device based on the location of a non-stationary reflection point in a determined spatial environment defined based on the radius from the device within a time window. Based on detecting entry of the specified object or object type into the area, an alert indicating entry of the specified object or object type into the area may be generated.

[0090] In some aspects, the one or more actions may include detecting a departure angle or an arrival angle of a specified object or type of object relative to the device based on the determined location of a non-resident reflection point in the spatial environment.

[0091] In some aspects, the one or more actions may include detecting the entry of a specified object or object type into an area defined by a radius from the device based on the location of a non-resident reflection point in the determined spatial environment. A counter of objects within the radius from the device (which counter may be defined on a per-object or per-object type basis) may be updated based on detecting the entry of a specified object or object type into the area. For example, when the entry of a specified object or object type is detected, a counter may be incremented to allow a cumulative count of objects entering the area to be maintained. In some cases, the counter may be configured to track the current number of non-resident objects in the area, such that the counter is decremented when the specified object or object type leaves the area.

[0092] In some aspects, the device may coordinate the sending and receiving of signaling with one or more time synchronized peer devices based on the determined locations of stationary reflection points and non-stationary reflection points in the spatial environment. The synchronized peer devices may be located at one focus of an ellipse, and the device may be located at another focus of the ellipse. Thus, the area defined by the ellipse that monitors the entry and exit of non-stationary objects may be defined by the distance between the foci. In some aspects, coordinating the sending and receiving of signaling may include coordinating the timing and angle of arrival of one or more signals so that one or more synchronized peer devices may eliminate one or more components within a received signal based on the determined locations of stationary reflection points and non-stationary reflection points in the spatial environment.

[0093] In some aspects, the one or more actions may include detecting entry of a specified object or type of object into an area defined by an ellipse based on the location of a non-stationary reflection point in the determined spatial environment and triangulation from a first focus and a second focus. Based on detecting entry of a specified object or type of object into the area, an alarm indicating entry of the specified object or type of object into the area may be generated at the device. For example, in a personal safety application, the specified object may be a human being, so that an alarm is not generated for other objects (such as non-human animals or specific types of animals) entering the area.

[0094] In some aspects, the one or more actions may include detecting entry of a specific object or type of object into an area defined by an ellipse based on a position of a non-stationary reflection point in a spatial environment defined based on a radius from the device determined within a time window and triangulation from a first focus and a second focus. Based on detecting entry of the specific object or type of object into the area, an alert indicating entry of the specific object or type of object into the area may be generated.

[0095] In some aspects, the one or more actions may include detecting a departure angle or an arrival angle of a particular object or type of object relative to the device or one of the one or more synchronized peer devices based on the determined location of a non-resident reflection point in the spatial environment.

[0096] In some aspects, the one or more actions may include detecting entry of a specified object or object type into an area defined by an ellipse based on the location of a non-resident reflection point in the determined spatial environment and triangulation from a first focus and a second focus. A counter of objects within a radius of the device may be updated based on detecting entry of a specific object or object type into the area. For example, when entry of a specific object or object type is detected, a counter may be incremented to allow a cumulative count of objects entering the area to be maintained. In some cases, a counter may be configured to track the current number of non-resident objects in the area, such that the counter is decremented when a specified object or object type leaves the area.

[0097] In some aspects, the machine learning model may be retrained to ignore certain specified objects as non-resident objects. For example, the machine learning model may be retrained to recognize certain objects based on their shape and size, and consider these objects to be resident objects based on a correlation between radio measurements associated with these objects and the shapes and sizes of these objects.

[0098] Example processing system for using machine learning models to predict device and anchor locations in a spatial environment

[0099] Figure 6 Describes a method for training a machine learning model to predict the positions of stationary objects (or reflection points) and non-stationary objects in a spatial environment (such as the one described in this article, for example, regarding Figure 3 An example processing system 600 described).

[0100] Processing system 600 includes a central processing unit (CPU) 602, which in some examples may be a multi-core CPU. Instructions executed at CPU 602 may be loaded, for example, from a program memory associated with CPU 602, or may be loaded from memory 624.

[0101] The processing system 600 also includes additional processing components customized for specific functions, such as a graphics processing unit (GPU) 604 , a digital signal processor (DSP) 606 , a neural processing unit (NPU) 608 , a multimedia processing unit 610 , and a wireless connectivity component 612 .

[0102] An NPU, such as NPU 608, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0103] NPUs, such as NPU 608, are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, multiple NPUs may be part of a dedicated neural network accelerator.

[0104] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can generally still be performed independently.

[0105] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing data set (usually labeled or tagged), iterating on the data set, and then adjusting model parameters (such as weights and biases) to improve model performance. Typically, optimization based on error predictions involves passing back through the layers of the model and determining the gradient to reduce the prediction error.

[0106] NPUs designed to accelerate inference are generally configured to operate on complete models. Thus, such NPUs can be configured to input new data slices and quickly process them through the trained model to generate model outputs (e.g., inferences).

[0107] In one specific implementation, the NPU 608 is part of one or more of the CPU 602 , the GPU 604 , and / or the DSP 606 .

[0108] In some examples, wireless connectivity component 612 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 6G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Wireless connectivity component 612 is further connected to one or more antennas 614.

[0109] In some examples, one or more of the processors of processing system 600 may be based on the ARM or RISC-V instruction set.

[0110] The processing system 600 also includes a memory 624, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 624 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 600.

[0111] Specifically, in this example, the memory 624 includes a data set receiving component 624A, a timing information extraction component 624B, and a machine learning model training component 624C. The depicted components, as well as other components not depicted, may be configured to perform various aspects of the methods described herein.

[0112] Generally speaking, the processing system 600 and / or its components may be configured to perform the methods described herein.

[0113] It is worth noting that in other embodiments, aspects of the processing system 600 may be omitted, such as where the processing system 600 is a server computer, etc. For example, in other embodiments, the multimedia processing unit 610, the wireless connectivity component 612, the sensor 616, the ISP 618, and / or the navigation component 620 may be omitted. Furthermore, aspects of the processing system 600 may be distributed, such as training a model and using the model to generate inferences.

[0114] Figure 7 Describes a method for using a machine learning model to predict the positions of stationary objects (reflection points) and non-stationary objects in a spatial environment (such as the one described herein, for example, with respect to Figure 5 An example processing system 700 described).

[0115] Processing system 700 includes a central processing unit (CPU) 702, which in some examples may be a multi-core CPU. Processing system 700 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 704, a digital signal processor (DSP) 706, and a neural processing unit (NPU) 708. CPU 702, GPU 704, DSP 706, and NPU 708 may be similar to those described above with respect to Figure 6 The CPU 702, GPU 704, DSP 706 and NPU 708.

[0116] In some examples, wireless connectivity component 712 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Wireless connectivity component 712 may be further connected to one or more antennas (not shown).

[0117] In some examples, one or more of the processors of processing system 700 may be based on the ARM or RISC-V instruction set.

[0118] The processing system 700 also includes a memory 724, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 724 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 700.

[0119] Specifically, in this example, the memory 724 includes a signal measurement component 724A, a timing information extraction component 724B, a location determination component 724C, an action taking component 724D, and a machine learning model component 724E (such as, Figure 6 The depicted components, as well as other components not depicted, may be configured to perform various aspects of the methods described herein.

[0120] Generally speaking, the processing system 700 and / or its components may be configured to perform the methods described herein.

[0121] It is noted that in other embodiments, aspects of the processing system 700 may be omitted, such as where the processing system 700 is a server computer, etc. For example, in other embodiments, the multimedia component 710, the wireless connectivity component 712, the sensor 717, the ISP 718, and / or the navigation component 720 may be omitted.

[0122] Sample Clauses

[0123] Specific implementation details for each aspect are described in the following numbered clauses.

[0124] Item 1: A method comprising: measuring, by a device, a plurality of signals within a spatial environment; determining, by the device based on a machine learning model and the plurality of signals measured within the spatial environment, a position of a stationary reflection point and a position of a non-stationary reflection point in the spatial environment; and taking one or more actions at the device based on determining the positions of the stationary reflection points and the positions of the non-stationary reflection points in the spatial environment.

[0125] Clause 2: The method of clause 1, wherein the device comprises a co-located transmitter and receiver.

[0126] Clause 3: The method of clause 2, wherein the taking one or more actions comprises: canceling one or more components within the plurality of signals based on the determined locations of the stationary reflection points in the spatial environment.

[0127] Clause 4: A method according to any one of clauses 2 or 3, wherein taking the one or more actions comprises: detecting entry of an object into an area defined by a radius from the device based on the determined position of the non-stationary reflection point in the spatial environment; and generating an alarm at the device indicating entry of the object into the area based on detecting entry of the object into the area.

[0128] Clause 5: A method according to any one of clauses 2 to 4, wherein taking the one or more actions includes: detecting entry of an object into an area defined by a radius from the device based on the position of the non-stationary reflection point in the determined spatial environment defined by the radius from the device within a time window; and generating an alarm at the device indicating entry of the object into the area based on detecting entry of the object into the area.

[0129] Clause 6: A method according to any one of clauses 2 to 5, wherein the one or more actions include: detecting a departure angle or an arrival angle of an object relative to the device based on the determined position of the non-resident reflection point in the spatial environment.

[0130] Clause 7: A method according to any one of clauses 2 to 6, wherein the one or more actions include: detecting entry of an object into an area defined by a radius from the device based on the determined position of the non-stationary reflection point in the spatial environment; and updating a counter of objects within the radius from the device based on detecting entry of the object into the area.

[0131] Clause 8: The method according to any one of clauses 2 to 7, further comprising: retraining the machine learning model to ignore objects as non-resident objects based on a correlation between radio measurements associated with the objects and size and shape information associated with the objects.

[0132] Clause 9: A method according to any one of clauses 1 to 8, wherein taking one or more actions includes: coordinating the sending and receiving of signaling with one or more synchronized peer devices based on the determined positions of the stationary reflection points and the non-stationary reflection points in the spatial environment.

[0133] Clause 10: A method according to clause 9, wherein the transmission and reception of coordinated signaling includes coordinating the timing and arrival angle of one or more signals so that the one or more synchronized peer devices can eliminate one or more components within the received signal based on the determined positions of the stationary reflection points and the non-stationary reflection points in the spatial environment.

[0134] Clause 11: The method of any of clauses 9 or 10, wherein the device is located at a first focus in an ellipse and the one or more synchronized peer devices are located at a second focus in the ellipse.

[0135] Clause 12: A method according to clause 11, wherein taking the one or more actions includes: detecting entry of an object into an area defined by the ellipse based on the determined position of the non-stationary reflection point in the spatial environment and triangulation from the first focus and the second focus; and based on detecting entry of the object into the area, generating an alarm at the device indicating that the object has entered the area.

[0136] Clause 13: A method according to any one of clauses 11 or 12, wherein taking the one or more actions includes: detecting entry of an object into an area defined by the ellipse based on the position of the non-stationary reflection point in the determined spatial environment defined by a radius from the device within a time window and triangulation from the first focus and the second focus; and based on detecting entry of the object into the area, generating an alarm at the device indicating entry of the object into the area.

[0137] Clause 14: A method according to any one of clauses 11 to 13, wherein the one or more actions include: detecting a departure angle or an arrival angle of an object relative to the device or one of the one or more synchronized peer devices based on the determined position of the non-stationary reflection point in the spatial environment.

[0138] Clause 15: A method according to any one of clauses 11 to 14, wherein the one or more actions include: detecting entry of an object into an area defined by the ellipse based on the determined position of the non-stationary reflection point in the spatial environment and triangulation from the first focus and the second focus; and updating a counter of objects within the ellipse based on detecting entry of the object into the area.

[0139] Clause 16: The method according to any one of clauses 11 to 15, further comprising: retraining the machine learning model to ignore objects as non-resident objects based on a correlation between radio measurements associated with the objects and size and shape information associated with the objects.

[0140] Aspect 17: A method according to any one of Aspects 1 to 16, wherein the machine learning model comprises a Gaussian mixture model.

[0141] Clause 18: A method according to any one of clauses 1 to 17, wherein the machine learning model comprises a probabilistic convolutional neural network configured to predict the position of the stationary reflection point and the position of the non-stationary reflection point based on time and spatial segmentation of the measured signal.

[0142] Clause 19: The method according to any one of clauses 1 to 18 further includes: retraining the machine learning model to ignore one or more specified objects when predicting the position of the stationary reflection point and the position of the non-stationary reflection point in the spatial environment.

[0143] Clause 20: A method according to any one of clauses 1 to 19, wherein the multiple signals include measuring the multiple signals including one or more reference signals; and measuring the multiple signals includes measuring channel state information (CSI) from the one or more reference signals.

[0144] Clause 21: The method of any one of clauses 1 to 20, wherein the device comprises one of a smartphone, a tablet, a laptop, or a wearable device.

[0145] Clause 22: A method comprising: receiving a data set of signal measurements; extracting a data set of timing information from the data set of signal measurements; and training a machine learning model to predict the location of a stationary reflection point in a spatial environment and the location of a non-stationary reflection point in the spatial environment based on the data set of signal measurements and the data set of timing information.

[0146] Clause 23: A method according to clause 22, wherein the machine learning model comprises a Gaussian mixture model.

[0147] Clause 24: The method of clause 23, wherein the Gaussian mixture model comprises one of a Bayesian model trained based on maximizing received signal energy or a posteriori multivariate Gaussian mixture model trained based on maximizing received signal energy.

[0148] Clause 25: A method according to any one of clauses 22 to 24, wherein the machine learning model comprises a probabilistic convolutional neural network configured to predict the position of the stationary reflection point and the position of the non-stationary reflection point based on time and spatial segmentation of the measured signal.

[0149] Clause 26: A method according to clause 25, wherein the probabilistic convolutional neural network includes one of the following: one or more convolution kernels, the one or more convolution kernels having activation parameters associated with detecting a human entering an area, or a probability model, the probability model is configured to identify a human entering an area and maintain a counter that tracks the number of humans entering the area over time.

[0150] Clause 27: The method of any one of clauses 22 to 26, wherein the locations of non-stationary reflection points in the spatial environment include locations of humans in motion in the spatial environment.

[0151] Clause 28: A method according to any one of clauses 22 to 27, wherein the dataset of signal measurements includes a dataset of channel state information (CSI) measurements from an environment different from the spatial environment in which the machine learning model is deployed.

[0152] Clause 29: A processing system comprising: a memory including computer executable instructions; and one or more processors configured to execute the computer executable instructions and cause the processing system to perform the method according to any one of clauses 1 to 28.

[0153] Clause 30: A processing system comprising: means for performing the method according to any one of clauses 1 to 28.

[0154] Clause 31: A non-transitory computer readable medium comprising: computer executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any one of clauses 1 to 28.

[0155] Clause 32: A computer program product embodied on a computer readable storage medium, the computer readable storage medium comprising code for performing the method according to any one of clauses 1 to 28.

[0156] Additional considerations

[0157] The previous description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein are not limitations on the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the universal principles defined herein may be applied to other embodiments. For example, without departing from the scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various examples may omit, replace, or add various processes or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, the features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods practiced using other structures, functionality, or structures and functionality in addition to or different from the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the claims.

[0158] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0159] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items (including single members). As an example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0160] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Furthermore, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Furthermore, "determining" may include resolving, selecting, choosing, establishing, and the like.

[0161] The method disclosed herein includes one or more steps or actions for implementing the method. The steps and / or actions of the method can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims. In addition, the various operations of the above-mentioned method can be performed by any suitable component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. Typically, in the case of operations illustrated in the accompanying drawings, those operations may have corresponding corresponding components with similar numbers plus functional components.

[0162] The following claims are not intended to be limited to the embodiments shown herein, but should be granted the full scope consistent with the language of the claims. Within the claims, unless explicitly stated, reference to an element in the singular form is not intended to mean "one and only one", but "one or more". Unless specifically stated, the term "some" refers to one or more. No claim element should be interpreted according to the provisions of 35 U.S.C. § 112 (f), unless the element is explicitly recorded using the phrase "a component for...", or in the case of a method claim, the element is recorded using the phrase "a step for...". All structural and functional equivalents of the elements of the various aspects described throughout the present disclosure that are known or will be known later to a person of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recorded in the claims.

Claims

1. A processor-implemented method, comprising: A device measures multiple signals in a spatial environment; Extracting, by the device, timing information from the measured plurality of signals within the spatial environment; Determining, by the device, a position of a stationary reflection point and a position of a non-stationary reflection point in the spatial environment based on a machine learning model, a plurality of signals measured in the spatial environment, and the extracted timing information; as well as One or more actions are taken at the device based on determining the locations of stationary reflection points and the locations of non-stationary reflection points in the spatial environment.

2. The method of claim 1, wherein the device comprises a co-located transmitter and receiver.

3. The method of claim 2, wherein taking one or more actions comprises: One or more components within the plurality of signals are cancelled based on the determined locations of the stationary reflection points in the spatial environment.

4. The method of claim 2, wherein taking the one or more actions comprises: detecting entry of an object into an area defined by a radius from the device based on the determined position of the non-stationary reflection point in the spatial environment; as well as Based on detecting entry of the object into the area, an alarm is generated at the device indicating entry of the object into the area.

5. The method of claim 2, wherein taking the one or more actions comprises: detecting entry of an object into an area defined by a radius from the device based on a position of the non-stationary reflection point in the determined spatial environment defined based on the radius from the device within a time window; as well as Based on detecting entry of the object into the area, an alarm is generated at the device indicating entry of the object into the area.

6. The method of claim 2, wherein the one or more actions include: A departure angle or an arrival angle of an object relative to the device is detected based on the determined position of the non-stationary reflection point in the spatial environment.

7. The method of claim 2, wherein the one or more actions include: detecting entry of an object into an area defined by a radius from the device based on the determined position of the non-stationary reflection point in the spatial environment; as well as A counter of objects within the radius from the device is updated based on detecting entry of the object into the area.

8. The method according to claim 2, further comprising: The machine learning model is retrained to ignore objects as non-resident objects based on a correlation between radio measurements associated with the objects and size and shape information associated with the objects.

9. The method of claim 1 , wherein taking one or more actions comprises: The transmission and reception of signaling with one or more synchronized peer devices is coordinated based on the determined locations of the stationary reflection points and the non-stationary reflection points in the spatial environment.

10. The method according to claim 9, wherein the sending and receiving of coordination signaling comprises: Coordinating the timing and angle of arrival of one or more signals enables the one or more synchronized peer devices to cancel one or more components within a received signal based on the determined locations of the stationary reflection points and the non-stationary reflection points in the spatial environment.

11. The method of claim 9, wherein the device is located at a first focus in an ellipse and the one or more synchronized peer devices are located at a second focus in the ellipse.

12. The method of claim 11, wherein taking the one or more actions comprises: detecting entry of an object into an area defined by the ellipse based on the determined position of the non-resident reflection point in the spatial environment and triangulation from the first focus and the second focus; as well as Based on detecting entry of the object into the area, an alarm is generated at the device indicating entry of the object into the area.

13. The method of claim 11, wherein taking the one or more actions comprises: detecting entry of an object into an area defined by the ellipse based on a position of the non-resident reflection point in the determined spatial environment defined based on a radius from the device within a time window and triangulation from the first focus and the second focus; as well as Based on detecting entry of the object into the area, an alarm is generated at the device indicating entry of the object into the area.

14. The method of claim 11, wherein the one or more actions include: A departure angle or an arrival angle of an object relative to the device or one of the one or more synchronized peer devices is detected based on the determined position of the non-resident reflection point in the spatial environment.

15. The method of claim 11, wherein the one or more actions include: detecting entry of an object into an area defined by the ellipse based on the determined position of the non-resident reflection point in the spatial environment and triangulation from the first focus and the second focus; as well as A counter of objects within the ellipse is updated based on detecting entry of the object into the area.

16. The method according to claim 11, further comprising: The machine learning model is retrained to ignore objects as non-resident objects based on a correlation between radio measurements associated with the objects and size and shape information associated with the objects.

17. The method of claim 1, wherein the machine learning model comprises a Gaussian mixture model.

18. The method of claim 1, wherein the machine learning model comprises a probabilistic convolutional neural network configured to predict the location of the stationary reflection point and the location of the non-stationary reflection point based on temporal and spatial segmentation of the measured signal.

19. The method of claim 1, further comprising: The machine learning model is retrained to ignore one or more specified objects when predicting the positions of the stationary reflection points and the positions of the non-stationary reflection points in the spatial environment.

20. The method of claim 1, wherein: The plurality of signals includes measuring the plurality of signals including one or more reference signals; and Measuring the plurality of signals includes measuring channel state information (CSI) from the one or more reference signals.

21. The method of claim 1, wherein the device comprises one of a smartphone, a tablet, a laptop, or a wearable device.

22. A processor-implemented method comprising: a data set of received signal measurements; extracting a data set of timing information from said data set of signal measurements; as well as The machine learning model is trained to predict the location of stationary reflection points in a spatial environment and the location of non-stationary reflection points in the spatial environment based on the data set of signal measurements and the data set of timing information.

23. The method of claim 22, wherein the machine learning model comprises a Gaussian mixture model.

24. The method of claim 23, wherein the Gaussian mixture model comprises one of a Bayesian model trained based on maximizing received signal energy or a posteriori multivariate Gaussian mixture model trained based on maximizing received signal energy.

25. The method of claim 22, wherein the machine learning model comprises a probabilistic convolutional neural network configured to predict the locations of the resident reflection points and the locations of the non-resident reflection points based on temporal and spatial segmentation of the measured signal.

26. The method of claim 25, wherein the probabilistic convolutional neural network comprises one of: one or more convolution kernels having activation parameters associated with detecting a human entering an area, or A probabilistic model configured to identify a human entering an area and to maintain a counter that tracks the number of humans entering the area over time.

27. The method of claim 22, wherein the locations of non-stationary reflection points in the spatial environment include locations of humans in motion in the spatial environment.

28. The method of claim 22, wherein the dataset of signal measurements comprises a dataset of channel state information (CSI) measurements from an environment different from the spatial environment in which the machine learning model is deployed.

29. A system comprising: A memory, wherein executable instructions are stored in the memory; and a processor configured to execute the executable instructions so that the system: Measuring multiple signals within a spatial environment; extracting timing information from the measured plurality of signals within the spatial environment; Determine the position of a stationary reflection point and the position of a non-stationary reflection point in the spatial environment based on a machine learning model, the measured plurality of signals within the spatial environment, and the extracted timing information; and One or more actions are taken based on predicting the positions of stationary reflection points and the positions of non-stationary reflection points in the spatial environment.

30. A system comprising: A memory, wherein executable instructions are stored in the memory; and a processor configured to execute the executable instructions so that the system: a data set of received signal measurements; extracting a data set of timing information from said data set of signal measurements; as well as The machine learning model is trained to predict the location of stationary reflection points in a spatial environment and the location of non-stationary reflection points in the spatial environment based on the data set of signal measurements and the data set of timing information.

Images