Wireless sensor

Abstract

This disclosure provides an approach that detects one or more radio signals within a geographic area. Each of the one or more radio signals includes channel status information (CSI) data. Based on the CSI data, this disclosure generates a channel status information (CSI) representation that displays several channel responses corresponding to the one or more radio signals. This disclosure filters the CSI representation to remove at least one of the channel responses corresponding to a stationary object within the geographic area, in order to produce a filtered CSI representation. Based on the filtered CSI representation, this disclosure predicts the presence of a moving object within the geographic area.

Description

[0001] This patent application claims priority to and the benefit of United States Provisional Patent Application No. 63 / 538,218, filed September 13, 2023, the entire contents of which are incorporated herein by reference. FIELD OF THE INVENTION

[0002] Aspects of the present disclosure relate to wireless sensing and, in particular, to improving activity detection in wireless sensing systems. GENERAL STATE OF THE ART

[0003] Radio sensing is a technology that uses radio signals to detect the presence of moving objects. As a radio signal propagates through an area, its properties are affected by stationary objects (e.g., walls, furniture, etc.) and moving objects (people, pets, robots, curtains, doors, etc.). When the radio signal encounters an object, it can be reflected, refracted, scattered, or absorbed, depending on the object's material properties. Furthermore, moving objects in the area can also create shifts in the radio signal's frequency.

[0004] A wireless sensing receiver, such as a Wireless Fidelity (Wi-Fi™) receiver, identifies changes in the characteristics of the wireless signal and uses the identified changes to generate predictions, such as whether the area includes the presence and movement of an object (e.g., walking, sitting, opening door, etc.). BRIEF DESCRIPTION OF THE CHARACTERS

[0005] The described embodiments and the advantages thereof can best be understood by reference to the following description in conjunction with the accompanying drawings. These drawings in no way limit any changes in form and details that may be made to the described embodiments by one skilled in the art without departing from the spirit and scope of the described embodiments. Fig. 1 is a block diagram illustrating an example of an advanced wireless sensing system according to some embodiments of the present disclosure. Fig. 2 is a block diagram illustrating an example preprocessing pipeline according to some embodiments of the present disclosure. Fig. 3 is a block diagram illustrating an example system for training a Bayesian CNN using contrastive data augmentation and self-supervised learning according to some embodiments of the present disclosure. Fig. 4 is a flowchart of a method for using wireless sensing to accurately predict human activity, according to some embodiments of the present disclosure. Fig. 5 is a block diagram of an example computing device that can perform one or more of the operations described herein, according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0006] As discussed above, radio sensing is a technique that uses radio signals to detect the presence of moving objects. Radio sensing makes use of channel state information (CSI) data in radio signals to perceive and identify changes in a given environment. CSI is a measure of the radio signal's properties that reflect how the signal propagates from a transmitter to a receiver and characterizes a combined effect of scattering, attenuation, and energy decay with distance. In radio sensing, CSI is used to identify variations in the radio signal's attributes as it interacts with moving objects in its propagation path. These interactions cause CSI values ​​to fluctuate, providing a signature for specific events or movements.Channel Frequency Response (CFR) data is included in the CSI data (CFR-CSI data), which provides a frequency domain representation of the radio signal and characterizes the behavior of the radio signal over different frequencies.

[0007] One challenge in wireless sensing is that the complexity and variability of real-world environments can introduce many confounding factors into the analysis of wireless signal changes, such as multipath. Multipath occurs when signals bounce off walls and objects, creating a complex mixture of signals at the receiver, making it challenging to attribute signal changes to specific causes. Another challenge in wireless sensing is that wireless signals can be affected by stationary objects, objects that are typically stationary but that occasionally move, and electromagnetic waves, making accurate detection and interpretation difficult.For example, CSI data variations can occur due to RF interference (automatic gain control (AGC) gain, packet detection delay, residual sampling frequency offset (SFO)), open windows, moving curtains, fans, an air conditioner, powered speakers, furniture rearrangement, etc., as well as artifacts outside the room boundaries.

[0008] Another challenge in wireless sensing is the ability to obtain enough training data to properly train models to detect presence and activity. In an ideal scenario, the training data would cover an exhaustive range of environmental conditions and variations, different types of movement, varying numbers and types of objects, and different spatial arrangements. However, creating such a large and diverse training dataset can be time-consuming, expensive, and sometimes impractical, especially accurately labeling the training data. Furthermore, as discussed above, wireless CSI data is highly sensitive to the environment and changes in physical layout. Even atmospheric conditions can lead to different CSI distributions.

[0009] The present disclosure provides an approach to address the above and other weaknesses by using a preprocessing pipeline and a self-supervised learning framework that trains a Bayesian convolutional neural network (CNN) to enhance radio sensing. The present disclosure provides an approach that detects one or more radio signals comprising channel state information (CSI) data within a geographic area. The approach generates a channel state information (CSI) representation based on the CSI data that indicates a plurality of channel responses corresponding to the radio signals. The approach then filters the CSI representation to remove at least one of the channel responses corresponding to a stationary object within the geographic area to generate a filtered CSI representation.The approach, in turn, predicts the presence of a moving object within the geographical area based on the filtered CSI representation.

[0010] In some embodiments, the CSI representation is filtered using a moving target indicator (MTI) filter. The approach calculates a weighted average of past CSI based on past radio signals that include past CSI data received over time. The approach then subtracts the weighted average of past CSI from the CSI representation to generate the filtered CSI representation.

[0011] In some embodiments, the approach transforms the filtered CSI representation into a Doppler trace image. The approach inputs the Doppler trace image to a Bayesian CNN trained to detect one or more human activities. The approach, in turn, generates a prediction using the Bayesian CNN that identifies at least one of the one or more human activities within the geographic area. For example, the Bayesian CNN may output a probability distribution across possible outcomes (no presence, significant movement (walking or running), minor movement (standing or sitting), jerky movements (falling or physical exercise), etc.). The probability distributions quantify how confident (or uncertain) the Bayesian CNN is in its prediction. Common measures include the variance or standard deviation of the output distribution.

[0012] In some embodiments, the approach trains the Bayesian CNN in a self-supervised training mode. The approach creates an augmented Doppler trace image from the Doppler trace image, which is a deformable transformation of the Doppler trace image related to one or more physical properties of the one or more radio signals. The approach inputs the Doppler trace image and the augmented Doppler trace image to the Bayesian CNN. The Bayesian CNN transforms the Doppler trace image and the augmented Doppler trace image into an original latent space representation and an augmented latent space representation, respectively. The approach then computes a contrastive loss between the original latent space representation and the augmented latent space representation and, based on the contrastive loss, adjusts one or more properties of the Bayesian CNN.

[0013] In some embodiments, the approach creates the augmented Doppler trace image using at least one of random augmentation of magnitude and noise, random augmentation of punctuation and deletions, random augmentation of cyclic shifts, or random augmentation of time dilation and normalization.

[0014] In some embodiments, the approach to train the Bayesian CNN inputs the original representation of the latent space and the augmented representation of the latent space into a probe to generate a probe classification. The probe includes both a focal loss function and an evidence-lower-bound (ELBO) loss function. The approach, in turn, uses the probe classification and the contrastive loss to train the Bayesian CNN. In some embodiments, the probe classification includes activity logits corresponding to at least one of the human activities.

[0015] As discussed herein, the present disclosure provides an approach that improves the operation of a computer system by creating augmented Doppler trace images using contrastive data augmentations regarding physical properties of radio signals to train a Bayesian CNN. Furthermore, the present disclosure provides an improvement in the field of radio sensing by including a moving target indicator (MTI) as a filter in a preprocessing pipeline to improve the accuracy of human activity predictions (e.g., predictions of human activity in a geographically limited space).

[0016] Fig. Figure 1 is a block diagram illustrating an example of an advanced wireless sensing system according to some embodiments of the present disclosure. The wireless sensing system 100 includes a wireless front-end 110, which includes, for example, an RF receiver, packet detection and symbol synchronization logic, demodulation logic, channel estimation logic, equalization logic, decoding logic, and beacon frame filtering logic. The wireless front-end 110 receives wireless signals (e.g., transmitted from an access point, station, etc.) and outputs CFR-CSI data 120 to the pre-processing pipeline 130. As discussed above, the CSI data is a measure of the characteristics of the received wireless signal, and the CFR data provides a frequency-domain representation of the wireless signal and characterizes the behavior of the wireless signal across different frequencies.

[0017] The preprocessing pipeline 130 generates CSI representations from the CFR-CSI data 120, for example, using an inverse fast Fourier transform (IFFT) corresponding to several channel responses of the radio signals. The preprocessing pipeline 130 then uses a moving target indicator (MTI) as a filter to remove some channel responses corresponding to stationary objects from the CSI representation and generate filtered CSI representations. The preprocessing pipeline 130, in turn, generates Doppler trace images 140 based on the CSI representations. Doppler trace images 140 amplify variations in the CFR-CSI data 120 caused by human movements (for further details, see Fig. 2 and the corresponding text).

[0018] The Bayesian convolutional neural network (CNN) 150 is trained to detect the presence of a moving object and, in particular, to detect human activities in a geographically limited space (for further details, see Fig. 3 and the corresponding text). Human activity detection in a geographically confined space refers to the detection of human actions or behaviors within a defined virtual geographical boundary (e.g., a geofence). In some embodiments, the Bayesian CNN 150 is trained using Doppler trace images 140 (or similar Doppler trace images) and augmented Doppler trace images, which are deformable transformations of Doppler trace images 140 relating to physical properties of radio signals (for further details, see Fig. 3 and the corresponding text).

[0019] The Bayesian CNN 150 generates predictions 160 based on Doppler trace images 140. Predictions 160 include probability distributions across possible outcomes (no presence, significant movement (walking or running), slight movement (standing or sitting), jerky movement (falling or exercising), etc.). The activity-perceiving classifier 170, in turn, classifies a human activity based on predictions 160. For example, the predictions 160 may include [.1 no presence, .3 slight movement, .6 significant movement], and the activity-perceiving classifier 170 assigns a classification as significant movement to the corresponding Doppler trace images 140.

[0020] Fig. Figure 2 is a block diagram illustrating an example of a preprocessing pipeline according to some embodiments of the present disclosure. Diagram 200 shows the preprocessing pipeline 130 receiving CFR-CSI data 120 from the radio front-end 110. In some embodiments, the preprocessing pipeline 130 extracts the CFR data from the CFR-CSI data 120. In some embodiments, the CFR data is collected along N = 64 subcarriers over a time T index / packet at an interval of 100 ms (X[1:N,1:T]).

[0021] Data normalization and phase cleaning 210 normalize the CFR data over time T to adjust for changes in the receiver's automatic gain control (AGC), nonlinear variations due to RF subsystems, or a combination thereof. Data normalization and phase cleaning 210 also remove phase drifts due to packet detection delays and sampling offsets. In some embodiments, data normalization and phase cleaning 210 perform polynomial equalization on the phase component of the CFR data as a polynomial function and subtract the polynomial from the raw phase to remove systematic phase error. In some embodiments, the polynomial equalization model is embodied as ϕ(f)=∑i=0laifi where "a" and "f" represent coefficients and the frequency variable of the polynomial model, respectively, and l = 2, and is used to capture the systematic phase error. The phase of the CFR-CSI data 120 is modeled as a polynomial function of frequency, denoted as ϕ(f). This model is used to approximate the systematic errors in the phase data. The terms a i in the polynomial expression represent the coefficients determined during the adjustment process. These coefficients are the factors by which each corresponding power of the frequency variable (f) is scaled and define the shape and slope of the polynomial used to correct the phase errors. Data normalization and phase adjustment 210, in turn, generate normalized and adjusted CFR data 215.

[0022] IFFT (Inverse Fast Fourier Transform) 220 transforms normalized and cleaned CFR data 215 into CSI representations 225 (e.g., X̃[1:Nd, 1:T]) by applying an IFFT, for example, along subcarrier dimensions. CSI representations 225 enhance CSI data's inherent transient properties for advanced Doppler profiling used in subsequent processing steps (discussed below). In some embodiments, CSI representation 225 is channel impulse response (CIR) data.

[0023] Moving target indicator (MTI) filtering 230 removes channel responses corresponding to stationary objects (e.g., walls, furniture, etc.) in a room from the CSI representations 225 to generate filtered CSI representations 235. In some embodiments, to remove the unwanted channel responses, the MTI filtering 230 builds statistics of CSI data over time (stored in the history store 232) and applies weights to the past CSI data to calculate a weighted average of past CSI. Then, the MTI filtering 230 subtracts the weighted average of past CSI from incoming signals to remove channel responses from the stationary objects. In some embodiments, the MTI filtering 230 applies a lower weight to older CSI data by applying an exponential moving average M [1:N,1]with a factor for forgetting (e.g., η = 0.985) along a temporal dimension. The MTI filtering 230 then calculates the filtered CSI representations Z [1:N,t] as Z[1:Nd,t]=X˜[1:Nd,t]−M[1:Nd,t−1] where M [1:Nd,t - 1] is updated recursively as ϑ[1:Nd,t]=hM[1:Nd,t−1]+(1−η)X˜[1:Nd,t] (3).

[0024] In some embodiments, when MTI filtering 230 isolates instantaneous variations in CSI representations 225 and attenuates stationary components, it increases the overall sensitivity of preprocessing pipeline 130 to micro-motions (e.g., human breathing or slight movement).

[0025] FFT transform (fast Fourier transform) 240 generates Doppler time vectors 245 by applying, for example, a one-dimensional (1-D) FFT to filtered CSI representations 235 (Z [1:Nd,1:T]) across packets along a dimension T and averaging over a delay dimension Nd, thereby confining the Doppler variations within a supervised window. FFT transform 240, in turn, generates Doppler time vectors 245 for each time index through a sliding window "w" along time / packets. In some embodiments, MTI filtering 230 is performed after FFT transform 260, in which case MTI filtering 230 performs two-dimensional (2-D) MTI filtering. In some embodiments, Doppler time vectors 245 may be fed into a temporally distributed neural network, such as a long short-term memory (LSTM) network or a temporal convolutional neural network (TCNN).

[0026] Vector stacking 250 stacks the Doppler time vectors 245 (Z̃ [1:Td,w]) column-wise to generate Doppler trace images 140 (e.g., 2-D Doppler trace images). Doppler trace images 140 amplify variations in the CSI data induced by human motion. As discussed herein, Doppler trace images 140 are input to the Bayesian CNN 150 during runtime operations and are also used to generate contrastive data augmentation images to train the Bayesian CNN 150 (for further details, see Fig. 3 and the corresponding text).

[0027] Fig. Figure 3 is a block diagram illustrating an example of a system for training a Bayesian CNN using contrastive data augmentation and self-supervised learning, according to some embodiments of the present disclosure. System 300 includes Bayesian CNN 150, projection head 355, and probe 385. As discussed herein, during training, Bayesian CNN 150 performs feature extraction to generate latent space representations, projection head 355 optimizes the latent space representations for contrastive learning, and probe 385 performs task classification. As discussed below, the system 300 for self-supervised learning of the Bayesian CNN 150 makes use of contrastive loss 380 from the projection head 355 and focal evidence lower bound (F-ELBO) loss in the probe 385.

[0028] System 300 includes contrastive data augmentation 310, which receives Doppler trace images 140 from preprocessing pipeline 130. In some embodiments, Doppler trace images 140 are "training" Doppler trace images that may have been previously created by preprocessing pipeline 130 and stored in a training memory area. Contrastive data augmentation 310 applies contrastive data augmentations to Doppler trace images 140 to generate deformable transformations related to physical properties of radio signals. Contrastive data augmentation 310 may use targeted radio signal type augmentation techniques, such as a) random magnitude and noise augmentation, b) random puncturing and erasure augmentation, c) random cyclic shift augmentation, d) random time dilation and normalization augmentation, or a combination thereof.Random magnitude and noise augmentation applied to Doppler trace images 140 trains the Bayesian CNN 150 to be insensitive to variations in noise in the input data. In some embodiments, the random scaling factor convolves the input trace while a level of noise is randomly applied to Doppler trace images 140. Random puncturing and deletion augmentation involves masking a portion of Doppler trace images 140 with zeros, simulating missing data or occlusions in real-world scenarios, and is a form of dropout applied directly to input data to train the Bayesian CNN 150. Random cyclic shift augmentation involves shifting Doppler trace images 140 along a time axis, accounting for variability in the onset of a Doppler contour to train the Bayesian CNN 150.Random augmentation of time dilation and normalization involves changing the time scale of Doppler trace images 140 to mimic variations in the radio signal from varying motion due to object movement, view angles, or the physical characteristics of a person to train the Bayesian CNN 150.

[0029] Contrastive data augmentation 310 outputs an original Doppler trace image 315 and an augmented Doppler trace image 320 augmented using one or more of the augmentation techniques discussed above. Contrastive data augmentation 310 feeds the original Doppler trace image 315 and the augmented Doppler trace image 320 to encoders 330 and 335 of Bayesian CNN 150. Encoders 330 and 335 generate latent space representations 340 that include an original latent space representation 345 (corresponding to the original Doppler trace image 315) and an augmented latent space representation 340 (corresponding to the augmented Doppler trace image 320).

[0030] The original representation of latent space 345 and the augmented representation of latent space 340 are fed to projection head 355. Projection head 355 includes multilayer perceptrons (MLPs) 360 and 365, which include layers of nodes (or neurons), such as an input layer for receiving the signal, an output layer that generates a decision or prediction about the input, and one or more hidden layers. Projection head 355 generates projection widths 370 and 375 (e.g., shaping the outputs). System 300 then calculates a contrastive loss function (contrastive loss 380) based on the differences and similarities of projection widths 370 and 375.

[0031] In some embodiments, the contrastive loss 380 may be a normalized temperature-controlled cross-entropy (NT-Xent) loss that maximizes the mutual information between positive pairs of 370, 375 and minimizes the mutual information between negative pairs of 370, 375, and uses a temperature parameter τ to control the hardness of the positive-negative sample pairs. For example, the directional NT-Xent loss may be defined as follows: L1→2=−1N∑i=1Nlog(exp(sin(xi,xi+) / r)∑j=1Nexp(sim(xi,xj) / r)) where N represents the number of data points in the batch, x+ represents the image of a positive example xi augmented with contrastive data, sim(x,x') represents the cosine similarity between data pairs, and the temperature-scaled similarities are used as logits in the cross-entropy. Then, the total contrastive loss LC, returned by the Bayesian CNN 150 and the projection head 355, is represented as follows: LC=max(12(L1→2+L2→1)−α,0) where LC represents the contrastive loss used to train the Bayesian CNN 150 and the projection head 355, L 1→2 and L 2→1 denote a directional NT-Xent loss between two data points, and α is a boundary parameter that sets a threshold below which the distances between dissimilar points are not penalized. The directional NT-Xent losses L 1→2 and L 2→1represent the directed (asymmetric) versions of the NT-Xent loss, calculated from two different data points or views. L 1→2 denotes the loss calculated when considering data point 1 as the anchor and point 2 as the positive example, while L 2→1 the loss calculated when considering data point 2 as the anchor and point 1 as the positive example. L 1→2 and L 2→1are calculations of the directional or asymmetric contrastive loss using NT-Xent, which contribute to the overall contrastive loss used in training the Bayesian CNN 150 and the projection head 355 to encourage the Bayesian CNN 150 to learn representations that cluster positive pairs and separate negative pairs, and in turn, learn generalizable and robust features from unlabeled data. As discussed below, the Focal-Evidence-Lower-Bound (F-ELBO) loss further refines the training by incorporating evidence from the data into the learned representations while maintaining regularization.

[0032] Probe 385 includes MLP 390 to generate activity logits 395. Activity logits 395, in some embodiments, are unnormalized output values ​​corresponding to human activity detection. Probe 385 generates activity logits 395 using a combined F-ELBO loss that includes two terms. The first term is a likelihood term represented by a focal cross-entropy loss. The second term is a regularization term that encourages the parameters of Bayesian CNN 150 to remain close to their prior distribution. In some embodiments, the F-ELBO loss is represented as follows: LF-ELBO=Lfocal(pt)+βKL(q(w|D||p(w)|)) where D represents the dataset, w represents the weights of the neural network (Bayesian CNN 150), q(w|D) is the posterior distribution of variation over the weights given the data, p(w) is the prior distribution over the weights, a focal cross-entropy loss is given as follows: Lfocal(pt)=−at(1−pt)Ylog(pt) where p t is the predicted probability of the true class, αt is a weighting factor that assigns varying weights to different classes, beta is a weighting hyperparameter, γ is a focusing parameter that controls the degree of downweighting, and log(p t ) represents the standard cross-entropy loss for a class t.

[0033] In some embodiments, system 300 uses a two-stage training approach. In the first stage, system 300 trains Bayesian CNN 150 using outputs from projection head 355 and probe 385. In some embodiments, an Adaptive Moment Estimation (Adam) optimizer with a learning rate (e.g., 0.001) is used in the first stage of training. In the second stage, system 300 trains Bayesian CNN 150 using outputs from probe 385 to learn maps of activities in a geographically limited space with lower learning rates. In some embodiments, hyperparameter optimization is used for projection widths, batch size, and regularization terms in the F-ELBO loss.

[0034] In some embodiments, the Bayesian CNN 150 learns over time to encode similar representations for augmented views of the same data point and different representations for different data points. Generating two traces (315 and 320) of the same data point allows the Bayesian CNN 150 to understand and learn the invariant features of data points that persist across different augmentations. Thus, the Bayesian CNN 150 learns more robust and generalizable representations by focusing on the underlying structure of the data rather than on idiosyncrasies present in a single view.

[0035] Fig. 4 is a flowchart of a method for using wireless sensing to accurately predict human activity, according to some embodiments of the present disclosure. The method 400 may be performed by processing logic that may include hardware (e.g., circuitry, dedicated logic, programmable logic, a processor, a processing device, a central processing unit (CPU), a system-on-chip (SoC), etc.), software (e.g., instructions running / executing on a processing device), firmware (e.g., microcode), or a combination thereof. In some embodiments, at least a portion of the method 400 may be performed by the preprocessing pipeline 130, the Bayesian CNN 150, the activity-sensing classifier 170, the Fig. 5 or a combination thereof.

[0036] Referring to Fig. 4 illustrates method 400 with example functions used by various embodiments. While specific functional blocks ("blocks") are disclosed in method 400, such blocks are examples. That is, embodiments are well-suited to implementing various other blocks or variations of the blocks recited in method 400. It is recognized that the blocks in method 400 may be performed in a different order than illustrated, and that not all of the blocks in method 400 may be performed.

[0037] Referring to Fig. 4, the method 400 begins at block 405, where processing logic detects one or more radio signals in a geographic area. For example, the processing logic may be associated with a radio sensing system that monitors Wi-Fi signals in an office space. In block 410, the processing logic extracts channel state information (CSI) using channel frequency response (CFR) data from the detected radio signals. In block 415, the processing logic normalizes and cleanses the CFR data to generate normalized and cleansed CFR data. In block 420, the processing logic transforms the normalized and cleansed CFR data into a CSI representation. The CSI representation displays multiple channel responses corresponding to the radio signals.

[0038] In block 425, the processing logic filters the CSI representation to generate a filtered CSI representation. In some embodiments, the processing logic filters the CSI representation using a moving target indicator (MTI) filter. The processing logic calculates a weighted average of past CSI based on past radio signals that include past CSI data received over time. The processing logic then subtracts the weighted average of past CSI from the CSI representation to generate the filtered CSI representation.

[0039] In block 430, the processing logic transforms the filtered CSI representation into Doppler time vectors, such as by applying a 1D FFT to the filtered CSI representations. In block 435, the processing logic stacks the Doppler time vectors column-wise to generate a Doppler trace image. In some embodiments, the Doppler trace image is a two-dimensional Doppler trace image.

[0040] In block 440, the processing logic inputs the Doppler trace image to the Bayesian CNN 150. In block 445, the processing logic determines whether the Bayesian CNN 150 has generated a prediction of human activity. For example, the Bayesian CNN 150 may output a probability distribution across possible outcomes (no presence, significant movement, minor movement, jerky movements, etc.) that quantify how confident (or uncertain) the Bayesian CNN 150 is in its prediction.

[0041] If the prediction of the Bayesian CNN 150 indicated no human activity, the processing logic follows the "No" branch, after which the processing logic continues monitoring the radio signals in block 450. If the prediction of the Bayesian CNN 150 indicates human activity, the processing logic follows the "Yes" branch, after which the processing logic reports the prediction of human activity in block 455.

[0042] Fig. 5 illustrates a diagrammatic representation of a machine in the example form of a computer system 500 having a set of instructions therein for causing the machine to perform any one or more of the wireless sensing activity prediction techniques discussed herein.

[0043] In alternative embodiments, the machine may be connected (e.g., networked) to other machines on a local area network (LAN), an intranet, an extranet, or the Internet. The machine may operate as a server or client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web application, a server, a network router, a switch or bridge, a hub, an access point, a network access controller, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be performed by that machine.Furthermore, while only a single machine is illustrated, the term "machine" should be construed to include any collection of machines that individually or collectively execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein. In some embodiments, computer system 500 may be representative of a server.

[0044] The example computer system 500 includes a processing device (a processor) 502, a main memory 504 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM)), a static memory 506 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device 518 that communicate with each other via a bus 530. Any of the signals provided via various buses described herein may be provided in time-multiplexed fashion with other signals via one or more common buses. Additionally, the interconnection between circuit components or blocks may be shown as buses or as individual signal lines. Each of the buses may alternatively be one or more individual signal lines, and each of the individual signal lines may alternatively be a bus.

[0045] Computing device 500 may further include a network interface device 508 capable of communicating with a network 520. Computer system 500 may also include a video display unit 510 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 512 (e.g., a keyboard), a cursor control device 514 (e.g., a mouse), and a device for generating an audible signal 516 (e.g., a speaker). In some embodiments, video display unit 510, alphanumeric input device 512, and cursor control device 514 may be combined into a single component or device (e.g., an LCD touchscreen).

[0046] Processing device 502 represents one or more general-purpose processing devices, such as a microprocessor, central processor, or the like. In particular, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computer (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 502 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. Processing device 502 is configured to execute API managed service instructions 525 to perform the operations and steps discussed herein.

[0047] The data storage device 518 may include a machine-readable storage medium 528 on which one or more sets of API managed service instructions 525 (e.g., software) are stored, embodying one or more of the methods or functions described herein. The API managed service instructions 525 may also reside, in whole or in part, in the main memory 504 or in the processing device 502 during execution thereof by the computer system 500, with the main memory 504 and the processing device 502 also constituting machine-readable storage media. The API managed service instructions 525 may further be sent or received via the network interface device 508 over a network 520.

[0048] The machine-readable storage medium 528 may also be used to store instructions for performing a method as described herein. While the machine-readable storage medium 528 is shown as a single medium in one embodiment, the term "machine-readable storage medium" is to be understood to include a single medium or multiple media (e.g., a centralized or distributed database or associated caches and servers) that store the one or more sets of instructions. A machine-readable medium includes any mechanism for storing information in a form (e.g., software, processing application) that can be read by a machine (e.g., a computer). The machine-readable medium may be a magnetic storage medium (e.g., floppy disk), an optical storage medium (e.g.,CD-ROM), a magneto-optical storage medium, read-only memory (ROM), random-access memory (RAM), erasable programmable memory (e.g., EPROM and EEPROM), flash memory, or any other type of medium suitable for storing electronic instructions.

[0049] Unless specifically stated otherwise, terms such as "capturing," "generating," "filtering," "predicting," "calculating," "subtracting," "transforming," "inputting," "training," "creating," "adapting," "utilizing," or the like refer to actions and processes performed or implemented by computing devices that manipulate data represented as physical (electronic) quantities within the registers and memories of the computing device and transform them into other data similarly represented as physical quantities in the memories or registers of the computing device or other such information storage, transmission, or display devices. Also, the terms "first," "second," "third," "fourth," etc., as used herein, are intended as identifiers to distinguish between different elements and do not necessarily have an ordering meaning according to their numerical designation.

[0050] Examples described herein also refer to a device for performing the operations described herein. This device may be specifically constructed for the required purposes, or it may include a general-purpose computing device (e.g., programmable logic) selectively programmed by a computer program stored in the computing device. Such a computer program may be stored on a computer-readable, non-transitory storage medium.

[0051] The methods and illustrative examples described herein do not inherently relate to any particular computer or other device. Various general-purpose systems may be used in accordance with the teachings described herein, or it may prove convenient to build a more specialized device to perform the required method steps. The required structure for a variety of these systems will be as set forth in the description above.

[0052] The above description is intended to be illustrative and not restrictive. While the present disclosure has been described with reference to specific illustrative examples, it will be appreciated that the present disclosure is not limited to the described examples. The scope of the disclosure should be determined by reference to the following claims, along with the full scope of equivalents to which the claims are entitled.

[0053] As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is further understood that the terms "includes," "including," "comprises," and / or "comprising," when used herein, specify the presence of stated features, integers, steps, acts, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, acts, elements, components, and / or groups thereof. Therefore, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0054] It should also be noted that in some alternative implementations, the noted functions / acts may occur in a different order than that noted in the figures. For example, two consecutively shown figures may actually be executed essentially simultaneously, or may sometimes be executed in the reverse order, depending on the functionality / acts involved.

[0055] While the method acts have been described in a specific order, it should be understood that other acts may be performed between described acts, described acts may be modified to occur at slightly different times, or the described acts may be distributed in a system that allows the processing acts to occur at different intervals associated with the processing.

[0056] Various units, circuits, or other components may be described or claimed as being "configured" or "configurable" to perform a task or tasks. In such contexts, the phrase "configured" or "configurable" is used to imply structure by indicating that the units / circuits / components include structure (e.g., circuitry) that performs the task or tasks during operation. Thus, the unit / circuit / component can be said to be configured to perform the task or to be configurable to perform the task even if the specified unit / circuit / component is not currently operational (e.g., not powered on).The units / circuits / components used with the phrase "configured to" or "configurable to" include hardware—for example, circuitry, memory that stores program instructions that can be executed to implement the operation, etc. The statement that a unit / circuit / component is "configured" to perform one or more tasks or "configurable" to perform one or more tasks is expressly not intended to be an invocation of 35 USC §112(f) for that unit / circuit / component. Additionally, "configured to" or "configurable to" may include general-purpose structure (e.g., general-purpose circuitry) that is manipulated by software and / or firmware (e.g., an FPGA or a general-purpose processor executing software) to operate in a manner capable of performing the task(s) in question."Configured to do so" may also include adapting a manufacturing process (e.g., a semiconductor fabrication facility) to fabricate devices (e.g., integrated circuits) adapted to implement or perform one or more tasks. "Configurable to do so" is expressly intended not to apply to blank media, an unprogrammed processor or unprogrammed general-purpose computer, or an unprogrammed programmable logic device, programmable gate array, or other unprogrammed device unless accompanied by programmed media that provides the unprogrammed device with the capability of being configured to perform the disclosed function(s).

[0057] The foregoing description has been described with reference to specific embodiments for the purpose of explanation. However, the illustrative discussions above are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the embodiments and their practical applications, thereby enabling others skilled in the art to best utilize the embodiments and various modifications as may be suited to the intended use.Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the present disclosure is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 63 / 538,218

[0001]

Claims

[1] A process comprising: detecting one or more radio signals in a geographic area, each of the one or more radio signals including channel status information (CSI) data; generating a channel status information (CSI) representation based on the CSI data, the CSI representation indicating a plurality of channel responses corresponding to the one or more radio signals; Filtering, by a processing device, the CSI representation to remove at least one of the plurality of channel responses corresponding to a stationary object within the geographic area, wherein the filtering produces a filtered CSI representation; and Predict the presence of a moving object within the geographic area based on the filtered CSI representation. [2] The method of claim 1, wherein the filtering is performed using a motion target indicator (MTI) filter, the method further comprising: Calculating a weighted average of past CSI based on past radio signals that include past CSI data received over time; and Subtract the weighted mean of past CSI from the CSI plot to produce the filtered CSI plot. [3] The method of claim 1, wherein the moving object is a human, the method further comprising: Transforming the filtered CSI image into a Doppler trace image; Inputting the Doppler trace image into a Bayesian convolutional neural network (CNN) trained to detect one or more human activities; and Generating, by the processing device using the Bayesian CNN, a prediction that identifies at least one of the one or more human activities within the geographic area. [4] The method of claim 3, further comprising: Training the Bayesian CNN in a self-supervised training mode, wherein the training further comprises: Creating an augmented Doppler trace image from the Doppler trace image, wherein the augmented Doppler trace image is a deformable transformation of the Doppler trace image relating to one or more physical properties of the one or more radio signals; Inputting the Doppler trace image and the augmented Doppler trace image into the Bayesian CNN, wherein the Bayesian CNN transforms the Doppler trace image and the augmented Doppler trace image into an original representation of the latent space and an augmented representation of the latent space, respectively; Calculating a contrastive loss between the original representation of the latent space and the augmented representation of the latent space; and Adjusting one or more features of the Bayesian CNN based on the contrastive loss. [5] The method of claim 4, wherein the augmented Doppler trace image is created from the Doppler trace image using at least one of random augmentation of magnitude and noise, random augmentation of punctuation and erasures, random augmentation of cyclic shifts, or random augmentation of time dilation and normalization. [6] The method of claim 4, wherein the training further includes: inputting the original representation of the latent space and the augmented representation of the latent space into a probe to generate a probe classification, the probe including both a focus loss function and an evidence lower bound (ELBO) loss function; Using probe classification and contrastive loss to train the Bayesian CNN during a first training stage; and Using probe classification to train the Bayesian CNN during a second training stage. [7] The method of claim 6, wherein the probe classification includes activity logits corresponding to at least one of the one or more human activities. [8] A system including: a processing device; and a memory for storing instructions that, when executed by the processing device, cause the processing device to: detects one or more radio signals in a geographic area, each of the one or more radio signals including channel status information (CSI) data; generates a channel status information (CSI) representation based on the CSI data, the CSI representation indicating a plurality of channel responses corresponding to the one or more radio signals; filters the CSI representation to select at least one of the plurality of channel responses, which correspond to a stationary object within the geographical area, to remove, wherein the filtering produces a filtered CSI representation; and predicts the presence of a moving object within the geographical area based on the filtered CSI representation. [9] The system of claim 8, wherein the filtering of the CSI representation is performed using a motion target indicator (MTI) filter, and wherein the processing device, in response to executing the instructions, further causes the system to: calculates a weighted average of past CSI based on past radio signals that include past CSI data received over time; and the weighted mean of past CSI is subtracted from the CSI representation to produce the filtered CSI representation. [10] The system of claim 8, wherein the moving object is a human, and wherein the processing device, in response to executing the instructions, further causes the system to: transforms the filtered CSI representation into a Doppler trace image; inputs the Doppler trace image into a Bayesian convolutional neural network (CNN) trained to detect one or more human activities; and using Bayesian CNN, generates a prediction that identifies at least one of the one or more human activities within the geographic area. [11] The system of claim 10, wherein the processing device, in response to executing the instructions, further causes the system to: the Bayesian CNN trains the system in a self-supervised training mode: create an augmented Doppler trace image from the Doppler trace image, wherein the augmented Doppler trace image is a deformable transformation of the Doppler trace image relating to one or more physical properties of the one or more radio signals; input the Doppler trace image and the augmented Doppler trace image into the Bayesian CNN, wherein the Bayesian CNN transforms the Doppler trace image and the augmented Doppler trace image into an original representation of the latent space and an augmented representation of the latent space, respectively; to calculate a contrastive loss between the original representation of the latent space and the augmented representation of the latent space; and to adapt one or more properties of the Bayesian CNN based on the contrastive loss. [12] The system of claim 11, wherein the augmented Doppler trace image is created from the Doppler trace image using at least one of random augmentation of magnitude and noise, random augmentation of punctuation and erasures, random augmentation of cyclic shifts, or random augmentation of time dilation and normalization. [13] The system of claim 11, wherein the processing device, in response to executing the instructions, further causes the system to: inputs the original representation of the latent space and the augmented representation of the latent space into a probe to generate a probe classification, wherein the probe includes both a focus loss function and an evidence lower bound (ELBO) loss function; uses probe classification and contrastive loss to train the Bayesian CNN during a first training stage; and uses probe classification to train the Bayesian CNN during a second training stage. [14] The system of claim 13, wherein the probe classification includes activity logits corresponding to at least one of the one or more human activities. [15] A non-transitory computer-readable medium having stored thereon instructions which, when executed by a processing device, cause the processing device to: detects one or more radio signals in a geographic area, each of the one or more radio signals including channel status information (CSI) data; generates a channel status information (CSI) representation based on the CSI data, the CSI representation indicating a plurality of channel responses corresponding to the one or more radio signals; the processing device filters the CSI representation to remove at least one of the plurality of channel responses corresponding to a stationary object within the geographic area, wherein the filtering produces a filtered CSI representation; and predicts the presence of a moving object within the geographical area based on the filtered CSI representation. [16] The non-transitory computer-readable medium of claim 15, wherein the filtering of the CSI representation is performed using a motion target indicator (MTI) filter, and wherein the processing device is to: calculate a weighted average of past CSI based on past radio signals that include past CSI data received over time; and subtract the weighted mean of past CSI from the CSI representation to generate the filtered CSI representation. [17] The non-transitory computer-readable medium of claim 15, wherein the moving object is a human, and wherein the processing device is to: transform the filtered CSI representation into a Doppler trace image; input the Doppler trace image into a Bayesian convolutional neural network (CNN) trained to detect one or more human activities; and using the Bayesian CNN, generate a prediction that identifies at least one of the one or more human activities within the geographic area. [18] The non-transitory computer-readable medium of claim 17, wherein the processing device is to: the Bayesian CNN in a self-supervised training mode trains the processing device to: create an augmented Doppler trace image from the Doppler trace image, wherein the augmented Doppler trace image is a deformable transformation of the Doppler trace image relating to one or more physical properties of the one or more radio signals; input the Doppler trace image and the augmented Doppler trace image into the Bayesian CNN, wherein the Bayesian CNN transforms the Doppler trace image and the augmented Doppler trace image into an original representation of the latent space and an augmented representation of the latent space, respectively; to calculate a contrastive loss between the original representation of the latent space and the augmented representation of the latent space; and to adapt one or more properties of the Bayesian CNN based on the contrastive loss. [19] The non-transitory computer-readable medium of claim 18, wherein the augmented Doppler trace image is created from the Doppler trace image using at least one of random augmentation of magnitude and noise, random augmentation of punctuation and erasures, random augmentation of cyclic shifts, or random augmentation of time dilation and normalization. [20] The non-transitory computer-readable medium of claim 18, wherein the processing device is to: input the original representation of the latent space and the augmented representation of the latent space into a probe to generate a probe classification, wherein the probe includes both a focus loss function and an evidence lower bound (ELBO) loss function; use the probe classification and the contrastive loss to train the Bayesian CNN during a first training stage; and use the probe classification to train the Bayesian CNN during a second training stage.

Images