Methods, apparatus and systems for wireless sensing, monitoring and recognition

Abstract

Methods, apparatus, and systems for wireless material sensing are disclosed. [Solution] In one example, the disclosed system includes a transmitter configured to transmit a first radio signal via a radio multipath channel of a place using a transmitting antenna, a receiver configured to receive a second radio signal via a radio multipath channel using a receiving antenna, and a processor. The second radio signal includes the reflection or refraction of the first radio signal at the surface of a target material of an object at the place. Based on the second radio signal, the processor obtains a plurality of the channel information (CI) of the radio multipath channel, each of which is associated with a respective transmitting antenna and a respective receiving antenna, calculates a material analysis based on the plurality of CIs, and determines the type of the target material of the object based on the material analysis.

Description

[Technical Field]

[0001] This disclosure generally relates to wireless sensing, monitoring, and recognition. More specifically, this disclosure relates to periodic or transient motion detection, e.g., fall event detection, sensing and material detection, wireless monitoring for motion detection and localization, and periodic or transient motion recognition based on wireless signals by wireless channel information (CI) processing, e.g., hand movement / gesture recognition, and wireless tracking of writing on a surface, and improvement of the accuracy and efficiency of wireless monitoring by wireless CI processing. [Background technology]

[0002] As the world ages, society faces an increasingly difficult responsibility to provide healthcare for the elderly. Falls are the most common type of accident among the elderly. In all parts of the world, the mortality rate from falls is the highest among adults aged 60 and over. Furthermore, injuries caused by falls are reflected not only in immediate physical injury, but also in all the subsequent adverse effects caused by a lack of timely assistance, especially for those living alone. Therefore, a real-time indoor fall detection system with timely and automatic alarms is highly desirable and could potentially save lives by promptly requesting external assistance. The great importance of fall detection has driven the development of various systems, which can be broadly divided into two categories: wearable and non-contact systems. Wearable technology requires users to wear specialized devices, including electrocardiogram (ECG) sensors, pressure sensors, accelerometers, gyroscopes, smartwatches, and smartphones, to track the user's body movements. However, in addition to the potential for false alarms in wearable systems, requiring users, especially the elderly, to carry specific sensors is cumbersome and sometimes impractical. This has stimulated the development of non-contact systems. The most common contactless systems are vision-based. Existing contactless systems require the deployment of arrays of cameras, infrared sensors, or depth cameras to monitor areas of interest. Thus, vision-based systems are limited by their visibility requirements and can raise privacy concerns, particularly in certain specific environments such as bathrooms and bedrooms.

[0003] Efforts have been made to enable material identification on ubiquitous computers. In recent years, the proliferation of autonomous devices (e.g., self-driving cars, drones), household robots (e.g., robotic vacuum cleaners, elderly care robots), and other smart devices has further driven this demand for object imaging and material identification information in mobile environments. If enabled on ubiquitous devices, many applications are possible, ranging from security to automotive and robotics. For example, material sensing capabilities would allow users to perform simple in-home detection of suspicious packages containing metal or liquid using their smartphones. By considering the material of the target, for example, by applying different gripping strengths to ceramic or metal targets, robots can operate more effectively. Furthermore, smart pencils and children's educational toys could intelligently react with specific functions depending on what they touch or point to. For all these interesting applications, and much more, to be imagined, accurate, portable, and low-cost ubiquitous object material identification information is needed. However, conventional systems typically rely on specialized hardware such as X-ray and ultrasound diagnostic equipment, and are limited to critical safety and medical use only, which prevents their ubiquitous adoption outside the laboratory. In recent years, radio frequency identification (RFID) tags have been deployed for material identification, while ultra-wideband (UWB) signals have been employed to identify liquids. However, these systems rely on multiple transceivers placed on various sides of the target to measure the signal that penetrates it, limiting their application to fixed, constrained setups and objects through which radio frequency (RF) signals can penetrate. Several other systems require specialized radar or mutually orthogonal antennas.

[0004] In the age of the Internet of Things, where everything is widely connected and sensed, much human-machine interaction has shifted from traditional computer keyboards or mice to hand gestures and writing in the air. While gesture recognition and handwriting recognition have been studied, most existing handwriting tracking systems require cameras, handheld sensors, or dedicated hardware that limits user experience, convenience, and scale of use. A high-precision passive handwriting tracking system is desirable.

[0005] Wireless monitoring using wireless channel information (CI) has attracted considerable attention in the age of the Internet of Things. However, CI, such as channel status information (CSI) or channel frequency information (CFI), can be interrupted by many factors, including hardware defects and thermal noise. Furthermore, without preprocessing, many outliers can exist that significantly affect the performance of wireless monitoring.

[0006] Step counting or gait detection is widely used in indoor localization and positioning systems. It is gaining popularity as a means of measuring and motivating daily exercise. Most existing research is based on the periodicity of walking in normal life to develop autocorrelation-based or zero-cross-check methods. However, existing methods ignore the uncertainty and spurious spikes in acceleration measurements during walking, which can significantly degrade their detection results.

[0007] People are surrounded by numerous smartly connected devices and computers every day. Therefore, human-computer interaction (HCI) forms a significant part of people's daily lives. Innovative solutions and approaches are being developed to simplify HCI. For example, from traditional black / whiteboards to keyboards and computer mice, then touchscreens, from push buttons to touch sensors, fingerprint sensors, motion sensors, and voice control systems. To further simplify HCI, researchers have experimented in various ways to build gesture recognition systems. Visual-based gesture recognition tracks hand (or finger) movements and reconstructs the hand trajectory. Visual-based gestures require ambient light, particularly for line-of-sight (LOS) applications, and can raise privacy concerns. While sensor-based gesture recognition technologies have offered solutions for gesture recognition, passive or device-free approaches are increasingly desirable for better user convenience and a better user experience. [Overview of the project]

[0008] This disclosure generally relates to wireless sensing, monitoring, and recognition. More specifically, this disclosure relates to periodic or transient motion detection, e.g., fall event detection, sensing and material detection, wireless monitoring for motion detection and localization, and periodic or transient motion recognition based on wireless signals by wireless channel information processing (CI), e.g., hand movement / gesture recognition, wireless tracking of writing on a surface, and improvement of the accuracy and efficiency of wireless monitoring by wireless CI processing.

[0009] In one embodiment, a system for detecting the motion of a target is described. This system comprises a transmitter configured to transmit a first radio signal over a radio multipath channel of a location, a receiver configured to receive a second radio signal over the radio multipath channel, and a processor. The second radio signal is different from the first radio signal because the radio multipath channel is affected by the motion of a target object within the location. The processor is configured to acquire time-series channel information (TSCI) of the radio multipath channel based on the second radio signal, calculate time-series spatiotemporal information (STI) of the object based on the TSCI, and detect the motion of the target object based on the time series of STI (TSSTI).

[0010] In another embodiment, a wireless device for a wireless target motion detection system is described. The wireless device includes a processor, memory communicatively coupled to the processor, and a receiver communicatively coupled to the processor. An additional wireless device for the wireless target motion detection system is configured to transmit a first wireless signal to a receiver over a wireless multipath channel of a place. The receiver is configured to receive a second wireless signal over the wireless multipath channel. The second wireless signal is different from the first wireless signal because the wireless multipath channel is affected by the target motion of an object in the place. The processor is configured to obtain time-series channel information (TSCI) of the wireless multipath channel based on the second wireless signal, calculate time-series spatiotemporal information (TSSTI) of an object based on the TSCI, and detect the target motion of an object based on at least one of the TSSTI or TSCI.

[0011] In yet another embodiment, a target motion detection system is described. This method includes: transmitting a first radio signal by a transmitter to a receiver via a radio multipath channel of a place; receiving a second radio signal by the receiver via the radio multipath channel during an online phase of the target motion detection system, wherein the second radio signal is different from the first radio signal due to the radio multipath channel being affected by the target motion of an object in the place; obtaining a time series of channel information (TSCI) of the radio multipath channel based on the second radio signal using a processor, a memory communicably coupled to the processor, and a set of instructions stored in the memory; calculating time series spatiotemporal information (TSSTI) of an object based on the TSCI; and detecting the target motion of an object based on at least one of the TSSTI or TSCI.

[0012] In one embodiment, a system for wireless material sensing is described. The system comprises a transmitter configured to transmit a first wireless signal through a wireless multipath channel of a location using a plurality of transmitting antennas, a receiver configured to receive a second wireless signal through the wireless multipath channel using a plurality of receiving antennas, and a processor. The second wireless signal includes the reflection or refraction of the first wireless signal at a target material surface of an object in the location. The target material surface is the surface of the target material of the object. The processor acquires a plurality of channel information (CIs) of the wireless multipath channel based on the second wireless signal, where each CI is associated with one of the plurality of transmitting antennas of the transmitter and one of the plurality of receiving antennas of the receiver, and each CI comprises at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), and is configured to calculate a material analysis based on the plurality of CIs and determine the type of target material of the object based on the material analysis.

[0013] In another embodiment, a wireless device for a wireless material sensing system is described. The wireless device includes a processor, a memory communicatively coupled to the processor, and a receiver communicatively coupled to the processor. An additional wireless device for the wireless material sensing system is configured to transmit a first wireless signal through a wireless multipath channel of a location using a plurality of transmitting antennas. The receiver is configured to receive a second wireless signal through the wireless multipath channel using a plurality of receiving antennas. The second wireless signal includes the reflection or refraction of the first wireless signal at the material surface of a target object in the location. The material surface of the target object is the surface of the target material of the object. The processor is configured to acquire a plurality of channel information (CIs) of the wireless multipath channel based on the second wireless signal, each CI being associated with one of the plurality of transmitting antennas and one of the plurality of receiving antennas, and to calculate a material analysis based on the plurality of CIs and determine the type of material of the target object based on the material analysis.

[0014] In yet another embodiment, a wireless material sensing system is described. This method comprises: transmitting a first wireless signal over a wireless multipath channel of a location using N1 transmitting antennas of a transmitter; receiving a second wireless signal over the wireless multipath channel using N2 receiving antennas of a receiver, the second wireless signal comprising the reflection or refraction of the first wireless signal on the surface of a target material of an object in the location, where N1 and N2 are positive integers; acquiring a plurality of channel information (CIs) of the wireless multipath channel based on the second wireless signal, each CI associated with one of each of the N1 transmitting antennas and one of each of the N2 receiving antennas, each CI comprising at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal intensity index (RSSI); calculating a material analysis based on the plurality of CIs; and determining the type of target material of an object based on the material analysis.

[0015] In one embodiment, a wireless writing tracking system is described. The system comprises a transmitter configured to transmit a first wireless signal over a wireless multipath channel of a location, a receiver configured to receive a second wireless signal over the wireless multipath channel, and a processor. The second wireless signal includes a reflection of the first wireless signal by the tip of a writing instrument in the location. The processor acquires a time series of channel information (CI) of the wireless multipath channel based on the second wireless signal, each CI including at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), and is configured to track the movement of the tip of a writing instrument based on the time series of CI (TSCI).

[0016] In another embodiment, a wireless device of a wireless writing tracking system is described. The wireless device includes a processor, a memory communicatively coupled to the processor, and a receiver communicatively coupled to the processor. An additional wireless device of the wireless writing tracking system is configured to transmit a first wireless signal over a wireless multipath channel of a location. The receiver is configured to receive a second wireless signal over the wireless multipath channel. The second wireless signal includes a reflection of the first wireless signal by the tip of a writing instrument in the location. The processor is configured to obtain a time series of channel information (CI) of the wireless multipath channel based on the second wireless signal, each CI including at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), and to track the movement of the tip of a writing instrument based on the time series of CI (TSCI).

[0017] In yet another embodiment, a wireless writing tracking system is described. This method includes the steps of transmitting a first radio signal over a wireless multipath channel of a location, and receiving a second radio signal over the wireless multipath channel, the second radio signal including a reflection of the first radio signal by the tip of a writing instrument in the location, and obtaining a time series of channel information (CI) of the wireless multipath channel based on the second radio signal, the acquisition of which each CI includes at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), and tracking writing by the tip of a writing instrument based on the time series of CI (TSCI).

[0018] In one embodiment, a wireless monitoring method is described. The method includes transmitting a first wireless signal from a first wireless device having N1 transmitting antennas over a wireless multipath channel of a place, where N1 is a positive integer greater than 1; receiving a second wireless signal from a second wireless device having N2 receiving antennas over the wireless multipath channel, where the second wireless signal differs from the first wireless signal for wireless multipath channels affected by the movement of objects in the place, where N2 is a positive integer greater than 1; acquiring some time series (TSCI) of channel information of the wireless multipath channel based on the second wireless signal using a processor, memory communicably coupled to the processor, and a set of instructions stored in memory, where each TSCI is associated with one of each of the N1 transmitting antennas and one of each of the N2 receiving antennas; and preprocessing the movement of objects in the wireless signal based on the number of preprocessed TSCIs.

[0019] In another embodiment, a wireless monitoring system is described. This wireless monitoring system comprises: a first wireless device having N1 transmitting antennas and configured to transmit a first wireless signal over a wireless multipath channel of a place, wherein N1 is a positive integer greater than 1; a second wireless device having N2 receiving antennas and configured to receive a second wireless signal over the wireless multipath channel, wherein the second wireless signal differs from the first wireless signal in that the wireless multipath channel is affected by the movement of objects in the place, wherein N2 is a positive integer greater than 1; and a processor configured to acquire some time series (TSCI) of channel information of the wireless multipath channel based on the second wireless signal, wherein each TSCI is associated with one of each of the N1 transmitting antennas and one of each of the N2 receiving antennas; and a processor that preprocesses the number of TSCIs and monitors the movement of objects in the place based on the number of preprocessed TSCIs.

[0020] In yet another embodiment, a smart speaker is described. The smart speaker comprises a receiver and a processor. The receiver has a plurality of receiving antennas configured to receive a first radio signal over a radio multipath channel of a place. The first radio signal is received based on a transmitting radio signal transmitted by a plurality of transmitting antennas of a radio device separate from the smart speaker. The first radio signal is different from the transmitted radio signal due to modulation by the radio multipath channel and objects receiving motion in the place. Based on the first radio signal, the processor obtains several time-series channel information (TSCIs) of the radio multipath channel, each TSCI associated with one of each of the transmitting antennas and one of each of the receiving antennas, and is configured to preprocess some TSCIs by (i) identifying time windows as noisy, anomalous, unconventional, irregular, unreliable, questionable, or inconsistent questionable time windows, and (ii) removing all CIs within the questionable time windows, and based on the number of preprocessed TSCIs, monitors the motion of objects in the place, and based on the monitoring, generates at least one of a response, assistance, or presentation to the object.

[0021] In one embodiment, a system for collaborative wireless motion monitoring is described. The system comprises a set of Type 1 devices, each being a heterogeneous wireless device in a location; a set of Type 2 devices interconnected wirelessly; and a processor. Each Type 2 device is a heterogeneous wireless device in a location. Each Type 2 device is wirelessly interconnected with a corresponding set of at least one other Type 2 device in a location. Each Type 2 device is associated with a corresponding subset of Type 1 devices in a location. Each Type 2 device is configured to asynchronously receive its respective wireless signal from each corresponding subset of Type 1 devices associated with the Type 2 device via each wireless multipath channel affected by the movement of objects in the location, and to asynchronously receive its respective wireless signal from each subset of each set of at least one other Type 2 device wirelessly interconnected with the Type 2 device via each wireless multipath channel affected by the movement of objects in the location, and to obtain a set of asynchronous time series of channel information (CI) associated with the Type 2 device, the asynchronous time series of each CI (ATSCI) being obtained based on each wireless signal associated with each wireless multipath channel and received asynchronously. The processor is configured to monitor the movement of objects within a location based on the set of ATSCIs associated with each type 2 device.

[0022] Another embodiment describes a method for a collaborative wireless motion monitoring system. The method comprises: interconnecting a set of Type 2 devices in a location wirelessly, each Type 2 device being a heterogeneous wireless device of the location, each Type 2 device being wirelessly interconnected with each set of at least one other Type 2 devices of the location, each Type 2 device being associated with each set of Type 1 devices of the location, each Type 1 device being a heterogeneous wireless device of the location; for each Type 2 device, asynchronously receiving each wireless signal from each set of Type 1 devices associated with the Type 2 device via each wireless multipath channel affected by the movement of objects in the location; asynchronously receiving each wireless signal from each subset of each set of at least one other Type 2 devices wirelessly interconnected with the Type 2 device via each wireless multipath channel affected by the movement of objects in that location; obtaining a set of asynchronous time series of channel information (CI) associated with the Type 2 device, each asynchronous time series (ATSCI) of CI being associated with each wireless multipath channel and obtained based on each wireless signal received asynchronously; and monitoring the movement of objects in the location based on the set of ATSCI associated with each Type 2 device.

[0023] In yet another embodiment, a method for a joint wireless motion monitoring system is described. This method involves wirelessly interconnecting a set of Type 2 devices in a location, each Type 2 device being a heterogeneous wireless device in the location, each Type 2 device being wirelessly interconnected with each set of at least one other Type 2 devices in the location, each Type 2 device being associated with each set of Type 1 devices in the location, each Type 1 device being a heterogeneous wireless device in the location. A method comprising interconnecting; asynchronously communicating each radio signal between the Type 2 device and each of the respective sets of Type 1 devices associated with the Type 2 device via each radio multipath channel affected by the movement of objects in the place; asynchronously communicating each radio signal between the Type 2 device and each of the respective sets of at least one other Type 2 devices wirelessly interconnected with the Type 2 device via each radio multipath channel affected by the movement of objects in the place; obtaining a set of asynchronous time series of channel information (CI) associated with the Type 2 device, wherein the asynchronous time series (ATSCI) of each CI is of the respective radio multipath channel and is obtained based on each radio signal received asynchronously; and monitoring the movement of objects in the place based on the set of ATSCI associated with each Type 2 device.

[0024] In one embodiment, a system for wireless motion recognition is described. This system comprises a transmitter configured to transmit a first wireless signal over a wireless multipath channel of a location, a receiver configured to receive a second wireless signal over the wireless multipath channel, and a processor. The second wireless signal is different from the first wireless signal due to the wireless multipath channel being affected by the movement of an object in the location. The processor is configured to acquire time-series channel information (TSCI) of the wireless multipath channel based on the second wireless signal, track the movement of an object based on the TSCI to generate a gesture trajectory of the object, and determine a gesture shape based on the gesture trajectory and a plurality of predetermined gesture shapes.

[0025] In another embodiment, a wireless device of a wireless motion recognition system is described. The wireless device includes a processor, a memory communicatively coupled to the processor, and a receiver communicatively coupled to the processor. An additional wireless device of the wireless motion recognition system is configured to transmit a first wireless signal over a wireless multipath channel of a place. The receiver is configured to receive a second wireless signal over the wireless multipath channel. The second wireless signal is different from the first wireless signal because the wireless multipath channel is affected by the movement of an object in the place. The processor is configured to obtain time-series channel information (TSCI) of the wireless multipath channel based on the second wireless signal, to track the movement of an object based on the TSCI to generate a gesture trajectory of the object, and to determine a gesture shape based on the gesture trajectory and a plurality of predetermined gesture shapes.

[0026] In yet another embodiment, a wireless motion identification system is described. This method includes transmitting a first wireless signal over a wireless multipath channel of a place, receiving a second wireless signal over the wireless multipath channel, the second wireless signal being different from the first wireless signal due to the wireless multipath channel being affected by the movement of an object in the place, obtaining time-series channel information (TSCI) of the wireless multipath channel based on the second wireless signal, tracking the movement of an object based on the TSCI to generate a gesture trajectory of the object, and determining a gesture shape based on the gesture trajectory and a plurality of predetermined gesture shapes.

[0027] Other concepts relate to software for implementing this disclosure with respect to wireless sensing, monitoring, and recognition. Additional novel features are partially described in the following description and may be partially revealed to those skilled in the art by examining the following drawings and accompanying drawings, or may be learned by manufacturing or performing the embodiments. Novel features of this disclosure may be realized and achieved by performing or using various embodiments of the methods, means, and combinations described in the detailed embodiments described below. [Brief explanation of the drawing]

[0028] The methods, systems, and / or devices described herein will be further described in reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These embodiments are non-limiting illustrative embodiments, and similar reference numerals in some of the drawings represent similar structures.

[0029] [Figure 1] This disclosure illustrates exemplary wireless indoor rich-scatter environments according to several embodiments.

[0030] [Figure 2]A flowchart illustrating exemplary methods for wirelessly detecting the target movement of an object, according to several embodiments of this disclosure, is shown.

[0031] [Figure 3A] , [Figure 3B] , [Figure 3C] , [Figure 3D] This figure shows an exemplary spatial autocorrelation function (ACF) and its differential for electromagnetic wave components according to one embodiment of the present disclosure.

[0032] [Figure 4] An illustrative flowchart of a wireless fall detection system according to one embodiment of this disclosure is shown.

[0033] [Figure 5A] , [Figure 5B] This disclosure illustrates an exemplary use of the principle of Segmental locally normalized dynamic time warping (SLN-DTW) in a continuous sanitization, according to one embodiment of this disclosure.

[0034] [Figure 6] An exemplary experimental setup for line-of-sight (LOS) and non-LOS scenarios according to one embodiment of this disclosure is shown.

[0035] [Figure 7A] , [Figure 7B] , [Figure 7C] , [Figure 7D] This figure shows exemplary examples of velocity and acceleration patterns for “walk-and-fall” and “sit-down” scenarios according to one embodiment of the present disclosure.

[0036] [Figure 8A] , [Figure 8B] , [Figure 8C] This shows exemplary templates of velocity and acceleration in one-dimensional and two-dimensional space according to one embodiment of the present disclosure.

[0037] [Figure 9] This figure shows an exemplary performance comparison between a dynamic time stretching (DTW) method and a threshold method according to one embodiment of the present disclosure.

[0038] [Figure 10A] This disclosure describes exemplary environments for wireless material sensing according to several embodiments.

[0039] [Figure 10B] This disclosure presents exemplary scenarios for wireless material sensing according to several embodiments.

[0040] [Figure 11] This disclosure illustrates exemplary workflows for channel impulse response (CIR)-based wireless material sensing according to several embodiments of this disclosure.

[0041] [Figure 12] A flowchart illustrating exemplary methods for wireless material sensing according to several embodiments of this disclosure is shown.

[0042] [Figure 13] Flowcharts illustrating exemplary techniques for CIR interpolation and CIR synchronization according to several embodiments of this disclosure are shown.

[0043] [Figure 14] The flowcharts of exemplary schemes for CIR synchronization according to several embodiments of this disclosure are shown.

[0044] [Figure 15] Flowcharts of other exemplary methods for CIR synchronization according to some embodiments of this disclosure are shown.

[0045] [Figure 16] A flowchart of exemplary techniques for noise reduction and target detection according to some embodiments of the present disclosure is shown.

[0046] [Figure 17] A flowchart of exemplary methods for obtaining background CIR according to several embodiments of this disclosure is shown.

[0047] [Figure 18] A flowchart of another exemplary method for obtaining background CIR according to some embodiments of this disclosure is shown.

[0048] [Figure 19] This flowchart shows an example of a material reflection property (MRF) estimation method according to some embodiments of the present disclosure.

[0049] [Figure 20] A flowchart of another exemplary method for MRF estimation according to some embodiments of this disclosure is shown.

[0050] [Figure 21] A flowchart illustrates an exemplary scheme for multi-frame-based wireless material sensing according to several embodiments of this disclosure.

[0051] [Figure 22] A flowchart illustrating an exemplary method for training a classifier during the training phase of a wireless material sensing system, according to some embodiments of the present disclosure, is shown.

[0052] [Figure 23] The present disclosure provides illustrative settings for material reflection characteristics (MRF) of several common materials according to some embodiments.

[0053] [Figure 24] The present disclosure illustrates exemplary CIR measurements, with or without the presence of a target object in the region of interest, according to several embodiments of this disclosure.

[0054] [Figure 25A] This disclosure shows exemplary CIR interpolation according to several embodiments.

[0055] [Figure 25B] The following are exemplary CIR amplitudes after noise reduction according to some embodiments of the present disclosure.

[0056] [Figure 26A] , [Figure 26B] , [Figure 26C] The following describes the exemplary performance of a wireless material sensing system according to several embodiments of this disclosure.

[0057] [Figure 27A] This disclosure illustrates exemplary wireless environments for wireless handwriting tracking according to several embodiments.

[0058] [Figure 27B] The following are exemplary ground truths of handwritten traces according to several embodiments of the present disclosure.

[0059] [Figure 27C] The present disclosure illustrates an exemplary reconstructed trajectory for handwriting tracking according to several embodiments of this disclosure.

[0060] [Figure 28] A flowchart illustrating exemplary methods of wireless handwriting tracking according to several embodiments of this disclosure is shown.

[0061] [Figure 29A] , [Figure 29B] , [Figure 29C]This disclosure illustrates exemplary radar concepts used in wireless handwriting tracking systems according to several embodiments of this disclosure.

[0062] [Figure 30] The following are exemplary channel impulse responses according to several embodiments of this disclosure.

[0063] [Figure 31A] , [Figure 31B] This disclosure illustrates CA-CFAR (Cell Averaging-Constant False Alarm Rate) technology used for target detection according to several embodiments of this disclosure.

[0064] [Figure 32] The present disclosure provides a workflow for an exemplary wireless handwriting tracking method according to several embodiments of this disclosure.

[0065] [Figure 33] The time-series estimated target position using two highest Doppler power bins according to some embodiments of the present disclosure is illustrated.

[0066] [Figure 34] This disclosure demonstrates a Doppler power shift from a non-zero frequency bin to a zero frequency bin when the target has a low or zero radial velocity.

[0067] [Figure 35A] The Doppler power for different range taps over time for a straight line drawn along a range away from the device is illustrated in some embodiments of this disclosure.

[0068] [Figure 35B] The range estimates before and after subsample peak interpolation (SPI) according to several embodiments of this disclosure are shown.

[0069] [Figure 36A] , [Figure 36B] , [Figure 36C] This disclosure illustrates the target location and trajectory configuration of the word "beam" according to several embodiments of this disclosure.

[0070] [Figure 37] The following are illustrative visual comparisons of ground truth characters and recovered characters according to several embodiments of this disclosure.

[0071] [Figure 38] This disclosure demonstrates the recognition accuracy of exemplary characters at different distances from a device using characters written at different scales, according to several embodiments of this disclosure.

[0072] [Figure 39A] , [Figure 39B] , [Figure 39C] The present disclosure illustrates various shapes of exemplary ground truth and reconstructed trajectories according to several embodiments of this disclosure.

[0073] [Figure 40] The following illustrates exemplary tracking errors at different ranges and azimuth angles from the radar according to several embodiments of this disclosure.

[0074] [Figure 41A] , [Figure 41B] This disclosure provides exemplary demonstrations of handwriting tracking for characters less than 1 cm in size, according to several embodiments of this disclosure.

[0075] [Figure 42] This disclosure presents exemplary scenarios in which object movement is monitored, according to several embodiments of this disclosure.

[0076] [Figure 43A] This disclosure illustrates the exemplary functions of an automated assistant system according to several embodiments.

[0077] [Figure 43B] This disclosure illustrates another exemplary function of an automated assistant system according to several embodiments of this disclosure.

[0078] [Figure 44] A flowchart illustrating an exemplary method of precise wireless monitoring according to several embodiments of this disclosure is shown.

[0079] [Figure 45] The flowcharts below illustrate exemplary methods for preprocessing time-series channel information (TSCI) for accurate wireless monitoring, according to some embodiments of the present disclosure.

[0080] [Figure 46] A flowchart illustrating exemplary techniques for accurate wireless monitoring according to several embodiments of this disclosure is shown.

[0081] [Figure 47] This disclosure provides exemplary floor plans and installations of wireless devices for wireless surveillance with motion detection and localization, according to several embodiments of this disclosure.

[0082] [Figure 48] A flowchart shows exemplary techniques for wireless surveillance involving motion detection and localization, according to several embodiments of the present disclosure.

[0083] [Figure 49] The flowcharts illustrate exemplary methods for managing a collaborative wireless motion monitoring system involving motion detection and localization, according to some embodiments of the present disclosure.

[0084] [Figure 50] This disclosure illustrates exemplary wireless indoor rich-scatter environments for wireless motion detection according to several embodiments of this disclosure.

[0085] [Figure 51]A flowchart of exemplary methods for wireless motion recognition according to several embodiments of this disclosure is shown.

[0086] [Figure 52] The present disclosure provides exemplary examples of multipath length differences when the reflector moves a small distance x, according to several embodiments of this disclosure.

[0087] [Figure 53] The following are exemplary examples of hand movements that rotate at the shoulder, according to several embodiments of this disclosure.

[0088] [Figure 54] The following are exemplary time-reversal resonance intensities during hand movements and trolley movements according to some embodiments of the present disclosure.

[0089] [Figure 55] The following describes the workflow of an exemplary wireless gesture recognition / classification system according to several embodiments of this disclosure.

[0090] [Figure 56] The present disclosure illustrates exemplary gesture trajectories of uppercase English alphabet letters having varying numbers of segments, according to several embodiments of this disclosure.

[0091] [Figure 57A] , [Figure 57B] This disclosure illustrates exemplary gesture splitting in several embodiments.

[0092] [Figure 58A] , [Figure 58B] The following are exemplary TRRS damping related to turning, according to some embodiments of the present disclosure.

[0093] [Figure 59A] , [Figure 59B] , [Figure 59C]The following are exemplary TRRS damping related to a 0-degree turn, according to some embodiments of this disclosure.

[0094] [Figure 60A] , [Figure 60B] , [Figure 60C] The following are exemplary TRRS damping related to a 45-degree turn, as shown by some embodiments of the present disclosure.

[0095] [Figure 61A] , [Figure 61B] , [Figure 61C] The following are exemplary TRRS damping related to a 90-degree turn, as shown in some embodiments of the present disclosure.

[0096] [Figure 62A] , [Figure 62B] This disclosure illustrates exemplary matching / cross-detection and associated TRRs according to several embodiments of this disclosure.

[0097] [Figure 63A] , [Figure 63B] , [Figure 63C] The following are exemplary performance matrices for gesture recognition and classification according to several embodiments of the present disclosure.

[0098] [Figure 64] An exemplary block diagram of a first wireless device of a wireless system according to some embodiments of the present disclosure is shown.

[0099] [Figure 65] An exemplary block diagram of a second wireless device of a wireless system according to some embodiments of the present disclosure is shown. [Modes for carrying out the invention]

[0100] In the following detailed description, numerous specific details are given as examples to provide a complete understanding of the relevant disclosures. However, it should be apparent to those skilled in the art that the disclosures can be implemented without such details. In other examples, well-known methods, procedures, components, and / or circuits are described at a relatively high level without detail, so as not to unnecessarily obscure aspects of the disclosures.

[0101] In one embodiment, this disclosure discloses a method, apparatus, device, system, and / or software (method / apparatus / device / system / software) for a wireless monitoring system. Time-series channel information (CI) of a wireless multipath channel can be obtained (e.g., dynamically) using a processor, memory communicably coupled to the processor, and a set of instructions stored in memory. Time-series CI (TSCI) can be extracted from radio signals (signals) transmitted over the channel between a type 1 heterogeneous radio device (e.g., a radio transmitter, TX) and a type 2 heterogeneous radio device (e.g., a radio receiver, RX) at a venue. The channel may be influenced by representations of objects at the venue (e.g., motion, movement, representation, and / or changes in position / pose / shape / representation). Objects and / or motion characteristics and / or spatial-temporal information (STI, e.g., motion information) of objects may be monitored based on the TSCI. Tasks may be performed based on characteristics and / or STI. Presentations associated with tasks may be generated in a user interface (UI) on the user's device. The TSCI may be a radio signal stream. The TSCI or each CI may be preprocessed. The apparatus may be a station (STA). The symbol "A / B" means "A and / or B" in this disclosure.

[0102] An expression can include arrangement, arrangement of movable parts, place, position, orientation, identifiable place, area, spatial coordinates, presentation, state, representation, static representation, size, length, width, height, angle, scale, shape, curve, surface, area, volume, pose, posture, sign, bodily representation, dynamic representation, dynamic noun, movement, motion sequence, gesture, stretch, contraction, distortion, deformation, bodily representation (e.g., head, face, eyes, mouth, tongue, hair, voice, neck, limbs, arms, hands, legs, feet, muscles, movable parts), surface representation (e.g., shape, texture, material, color, electromagnetic (EM) properties, visual pattern, humidity, reflectivity, translucency, flexibility), material properties (e.g., living tissue, hair, textile, metal, wood, leather, plastic, metal, artificial material, solid, liquid, gas, temperature), movement, activity, behavior, change in expression, and / or any combination thereof.

[0103] Radio signals include transmit / receive signals, EM emissions, RF signals / transmit, licensed / unlicensed / ISM band signals, band-limited signals, baseband signals, radio / mobile / cellular communication signals, radio / mobile / cellular network signals, mesh signals, optical signals / communications, downlink / uplink signals, unicast / multicast / broadcast signals, standard (e.g., WLAN, WWAN, WPAN, WBAN, international, domestic, industry, de facto, IEEE, IEEE 802, 802.11 / 15 / 16, WiFi, 802.11n / ac / ax / be, 3G / 4G / LTE / 5G / 6G / 7G / 8G, 3GPP, Bluetooth, BLE, Zigbee, RFID, UWB, WiMax) compliant signals, protocol signals, standard frames, beacon / pilot / search / query / acknowledgment / handshake / synchronization signals, management / control / data signals, standardized radio / cellular communication protocols, reference signals, source signals, motion probe / detection / It may include sensing signals and / or a set of signals. The radio signals may include line-of-sight (LOS) and / or non-LOS components (or paths / links). Each CI may be extracted / generated / calculated / detected at a layer of a Type 2 device (e.g., the PHY / MAC layer of the OSI model) and obtained by an application (e.g., software, firmware, drivers, apps, radio monitoring software / systems).

[0104] A wireless multipath channel may include: communication channels, analog frequency channels (e.g., analog carrier frequencies around 700 / 800 / 900MHz, 1.8 / 1.8 / 2.4 / 3 / 5 / 6 / 27 / 60GHz), coding channels (e.g., CDMA), and / or wireless network / system channels (e.g., WLAN, WiFi, mesh, LTE, 4G / 5G, Bluetooth, Zigbee, UWB, RFID, microwave). It may include two or more channels. The channels may be contiguous (e.g., adjacent / overlapping bands) or non-contiguous (e.g., non-overlapping WiFi channels, one at 2.4GHz and one at 5GHz).

[0105] TSCI can be extracted from radio signals at the layers of a Type 2 device (e.g., the layers of the OSI reference model: physical layer, data link layer, logical link control layer, media access control (MAC) layer, network layer, transport layer, session layer, presentation layer, application layer, TCP / IP layer, internet layer, link layer). TSCI may also be extracted from derived signals (e.g., baseband signals, motion detection signals, motion sensing signals) derived from radio signals (e.g., RF signals). It may be a (radio) measurement detected by a communication protocol (e.g., a standardized protocol) using existing mechanisms (e.g., radio / cellular communication standards / networks, 3G / LTE / 4G / 5G / 6G / 7G / 8G, WiFi, IEEE 802.11 / 15 / 16). The derived signal may include a packet having at least one of a preamble, header, and payload (e.g., for data / control / management in a radio link / network). TSCI can be extracted from probe signals in the packet (e.g., training sequence, STF, LTF, L-STF, L-LTF, L-SIG, HE-STF, HE-LTF, HE-SIG-A, HE-SIG-B, CEF). Motion detection / sensing signals may be recognized / identified based on the probe signals. Packets may be standards-compliant protocol frames, management frames, control frames, data frames, sounding frames, excitement frames, illumination frames, null data frames, beacon frames, pilot frames, probe frames, request frames, response frames, association frames, reassociation frames, disconnection frames, authentication frames, action frames, report frames, pole frames, announcement frames, extension frames, query frames, acknowledgment frames, RTS frames, CTS frames, QoS frames, CF-Poll frames, CF-Ack frames, block acknowledgment frames, reference frames, training frames, and / or synchronization frames.

[0106] Packets may contain control data and / or motion detection probes. Data (e.g., ID / parameters / characteristics / settings / control signals / commands / instructions / notifications / broadcast-related information for Type 1 devices) can be retrieved from the payload. Radio signals may be transmitted by Type 1 devices and received by Type 2 devices. Databases (e.g., in local servers, hub devices, cloud servers, and storage networks) may be used to store TSCI, characteristics, STI, signatures, patterns, behavior, trends, parameters, analysis, output responses, identification information, user information, device information, channel information, location (e.g., maps, environmental models, networks, proximity devices / networks) information, task information, class / category information, presentation (e.g., UI) information, and / or other information.

[0107] Type 1 / Type 2 devices may include at least one of the following: electronics, circuits, transmitters (TX) / receivers (RX) / transceivers, RF interfaces, "origin satellites" / "tracker bots", unicast / multicast / broadcast devices, wireless power devices, power / destination devices, wireless nodes, hub devices, target devices, motion detection devices, sensor devices, remote / wireless sensor devices, wireless communication devices, wireless-enabled devices, standards-compliant devices, and / or receivers. If multiple examples of Type 1 (or Type 2) devices exist, they may be heterogeneous, having different circuits, enclosures, structures, purposes, auxiliary functions, chips / ICs, processors, memory, software, firmware, network connectivity, antennas, brands, models, appearance, form, shape, color, materials, and / or specifications. Type 1 / Type 2 devices may include access points, routers, mesh routers, Internet of Things (IoT) devices, wireless terminals, one or more wireless / RF subsystems / wireless interfaces (e.g., 2.4GHz wireless, 5GHz wireless, fronthaul wireless, backhaul wireless), modems, RF front ends, RF / wireless chips, or integrated circuits (ICs).

[0108] At least one of the following can be associated with an identification (ID), such as a UUID: a Type 1 device, a Type 2 device, the links between them, objects, characteristics, STI, motion monitoring, and tasks. A Type 1 / Type 2 / another device can acquire / store / retrieve / access / preprocess / condition / process / analyze / monitor / apply TSCI. Type 1 and Type 2 devices can communicate network traffic on other channels (e.g., Ethernet, HDMI®, USB, Bluetooth, BLE, WiFi, LTE, other networks, wireless multipath channels) in parallel with radio signals. A Type 2 device can passively observe / monitor / receive radio signals from a Type 1 device on a wireless multipath channel without establishing a connection (e.g., association / authentication) with a Type 1 device or requesting a service from a Type 1 device.

[0109] A transmitter (i.e., a Type 1 device) can function (perform its role) as a receiver (i.e., a Type 2 device) temporarily, sporadically, continuously, repeatedly, interchangeably, alternately, simultaneously, in parallel, and / or concurrently, and vice versa. A device can function as both a Type 1 device (transmitter) and / or a Type 2 device (receiver) temporarily, sporadically, continuously, repeatedly, simultaneously, in parallel, and / or concurrently. There may be multiple radio nodes, each being a Type 1 (TX) and / or Type 2 (RX) device. TSCI may be acquired for every two nodes when exchanging / communicating radio signals. Object properties and / or STI may be monitored individually based on TSCI or jointly based on two or more (e.g., all) TSCIs.

[0110] The movement of an object can be monitored actively (in that its Type 1 device, Type 2 device, or both, is wearable / associated with the object) and / or passively (in that neither the Type 1 device nor the Type 2 device is wearable / associated with the object). It can be passive because the object may not be associated with a Type 1 device and / or a Type 2 device. The object (e.g., a user, an automated guided vehicle, or an AGV) may not need to carry / install any wearables / fixtures (i.e., Type 1 and Type 2 devices are not wearable / mounted devices that the object needs to carry in order to perform a task). This can be active because the object may be associated with either a Type 1 device or a Type 2 device. The object may carry (or install) wearables / fixtures (e.g., a Type 1 device, a Type 2 device, or a device communicatively coupled to either a Type 1 or Type 2 device).

[0111] Presentations include visuals, audio, images, video, animations, graphic presentations, and text. Task calculations can be performed by a processor (or logic unit) of a Type 1 device, a processor (or logic unit) of a Type 1 device IC, a processor (or logic unit) of a Type 2 device, a processor of a Type 2 device IC, a local server, a cloud server, a data analysis subsystem, a signal analysis subsystem, and / or another processor. This work can be performed with or without radio finger prints or baselines (e.g., collection, processing, transmission, and / or training phases / previous survey / latest survey / initial radio survey, passive markings), training, profiling, trained profile, static profile, static profile, survey, initial radio survey, initial setup, installation, retraining, update, and reset).

[0112] A Type 1 device (TX device) may comprise at least one heterogeneous radio transmitter. A Type 2 device (RX device) may comprise at least one heterogeneous radio receiver. Type 1 and Type 2 devices may be colocations. A Type 1 device and a Type 2 device may be the same device. Any device may have a data processing unit / device, a computing unit / system, a network unit / system, a processor (e.g., a logic unit), memory communicatively coupled to the processor, and a set of instructions stored in the memory to be executed by the processor. Some processors, memory, and sets of instructions may coordinate.

[0113] There may be multiple Type 1 devices interacting with the same Type 2 device (or multiple Type 2 devices) (e.g., communicating, exchanging signals / controls / notifications / other data), and / or multiple Type 2 devices interacting with the same Type 1 device. Multiple Type 1 / Type 2 devices may be synchronous and / or asynchronous, have the same / different window widths / sizes and / or time shifts, have the same / different sync start times and sync end times, etc. Radio signals transmitted by multiple Type 1 devices may be sporadic, transient, continuous, iterative, synchronous, simultaneous, and / or simultaneous. Multiple Type 1 / Type 2 devices may operate independently and / or collaboratively. Type 1 and / or Type 2 devices may have / have heterogeneous hardware circuits (e.g., heterogeneous chips or ICs that can generate / receive radio signals, extract CI from received signals, or make CI available). They may be communicatively coupled to the same or different servers (e.g., cloud servers, edge servers, local servers, hub devices).

[0114] The operation of one device may be based on its operation, state, internal state, storage, processor, memory output, physical location, computing resources, and network of other devices. Different devices may communicate directly and / or via other devices / servers / hub devices / cloud servers. A device may be associated with one or more users and have associated settings. Settings may be selected once, pre-programmed, and / or changed over time (e.g., adjusted, altered, modified). A method may have further steps. The steps and / or additional steps of a method may be performed in the order shown or in a different order. Any step may be performed in parallel, repeatedly, or otherwise iteratively or otherwise. Users may be humans, adults, elderly adults, males, females, infants, children, babies, pets, animals, living organisms, machines, computer modules / software, etc.

[0115] For one or more Type 1 devices interacting with one or more Type 2 devices, any processing (e.g., time domain, frequency domain) may differ for different devices. Processing may be based on location, orientation, direction, role, user-related characteristics, settings, configuration, available resources, available bandwidth, network connectivity, hardware, software, processor, coprocessor, memory, battery life, available power, antenna, antenna type, antenna directivity / unidirectional characteristics, power settings, and / or other parameters / characteristics of the device.

[0116] A wireless receiver (e.g., a Type 2 device) may receive signals and / or other signals from a wireless transmitter (e.g., a Type 1 device). A wireless receiver may receive another signal from another wireless transmitter (e.g., a second Type 1 device). A wireless transmitter may transmit signals and / or other signals to another wireless receiver (e.g., a second Type 2 device). Wireless transmitters, wireless receivers, other wireless receivers and / or other wireless transmitters may move with objects and / or other objects. They may track other objects.

[0117] A Type 1 and / or Type 2 device may be capable of radio coupling with at least two Type 2 and / or Type 1 devices. A Type 1 device may be triggered / controlled to switch / establish a radio coupling (e.g., association, authentication) from a Type 2 device to a second Type 2 device at a different location in the same place. Similarly, a Type 2 device may be triggered / controlled to switch / establish a radio coupling from a Type 1 device to a second Type 1 device at yet another location in the same place. Switching can be controlled by a server (or hub device), a processor, a Type 1 device, a Type 2 device, and / or another device. Different radios may be used before and after switching. A second radio signal (second signal) may be transmitted through a channel between a Type 1 device and a second Type 2 device (or between a Type 2 device and a second Type 1 device). A second TSCI of the channel may be obtained from the second signal. The second signal may be the first signal. Object properties, STI and / or other quantities may be monitored based on the second TSCI. Type 1 and Type 2 devices may be the same. Characteristics, STI, and / or other quantities with different timestamps may form a waveform. The waveform can be displayed in the presentation.

[0118] Radio signals and / or other signals may have data embedded in them. A radio signal may be a series of probe signals (e.g., repeated transmission of probe signals, reuse of one or more probe signals). Probe signals may change / change over time. Probe signals may be standards-compliant signals, protocol signals, standardized radio protocol signals, control signals, data signals, radio communication network signals, cellular network signals, WiFi signals, LTE / 5G / 6G / 7G signals, reference signals, beacon signals, motion detection signals, and / or motion sensing signals. Probe signals may be formatted according to a radio network standard (e.g., WiFi), a cellular network standard (e.g., LTE / 5G / 6G), or another standard. Probe signals may include packets with a header and a payload. Probe signals may have data embedded in them. The payload may include data. Probe signals may be replaced by data signals. Probe signals may be embedded in data signals. A radio receiver, a radio transmitter, another radio receiver and / or another radio transmitter may be associated with at least one processor, memory communicably coupled to each processor, and / or each instruction set stored in memory, which, when executed, causes the processor to perform any and / or all steps necessary to determine the object's STI (e.g., motion information), initial STI, initial time, direction, instantaneous position, instantaneous angle, and / or velocity.

[0119] A processor, memory, and / or instruction set may be associated with a Type 1 device, at least one Type 2 device, an object, a device associated with an object, another device associated with a location, a cloud server, a hub device, and / or another server.

[0120] A Type 1 device can broadcast signals to at least one or more Type 2 devices through a location channel. The signals are transmitted without a Type 1 device establishing a radio connection (e.g., association, authentication) with any Type 2 device, and without a Type 2 device requesting service from a Type 1 device. A Type 1 device can transmit to a specific Media Access Control (MAC) address common to multiple Type 2 devices. Each Type 2 device can adjust its MAC address to a specific MAC address. A specific MAC address can be associated with a location. The association can be recorded in a mapping table of an association server (e.g., a hub device). A location can be identified by a Type 1 device, a Type 2 device, and / or another device based on a specific MAC address, a series of probe signals, and / or at least one TSCI extracted from the probe signals.

[0121] For example, a Type 2 device may be moved to a new location (e.g., from another location). A Type 1 device may be newly configured in a location so that Type 1 and Type 2 devices do not recognize each other. During setup, a Type 1 device may be instructed / guided / triggered / controlled to send a series of probe signals to a specific MAC address (e.g., using a dummy receiver, using hardware pin configuration / connection, using saved configuration, using local configuration, using remote configuration, using downloaded configuration, using a hub device, or using a server). Upon power-up, a Type 2 device may scan for probe signals according to a table of MAC addresses (e.g., stored in a specified source, server, hub device, or cloud server) that can be used to broadcast in different locations (e.g., different MAC addresses used in different locations such as houses, offices, enclosures, floors, multi-story buildings, shops, airports, malls, stadiums, halls, stations, subways, lots, regions, areas, districts, localities, cities, countries, continents, etc.). When a Type 2 device detects a probe signal sent to a specific MAC address, the Type 2 device can use the table to identify the location based on the MAC address.

[0122] The location of a Type 2 device can be calculated based on a specific MAC address, a series of probe signals, and / or at least one TSCI obtained by the Type 2 device from the probe signals. The calculation may be performed by the Type 2 device.

[0123] A specific MAC address may change over time (e.g., adjust, modify, revise). It may change according to a schedule, rules, policies, modes, conditions, circumstances, and / or changes. A specific MAC address may be selected based on MAC address availability, a pre-selected list, collision patterns, traffic patterns, data traffic between Type 1 devices and other devices, available bandwidth, random selection, and / or a MAC address switching plan. A specific MAC address may be the MAC address of a second radio device (e.g., a dummy receiver, or a receiver that functions as a dummy receiver).

[0124] A Type 1 device may transmit a probe signal on a channel selected from a set of channels. At least one CI on the selected channel may be acquired by each Type 2 device from the probe signal transmitted on the selected channel.

[0125] The selected channels may change over time (e.g., adjust, modify, revise). Changes may follow schedules, rules, policies, modes, conditions, circumstances, and / or changes. The selected channels may be chosen based on channel availability, random selection, a pre-selected list, same-channel interference, inter-channel interference, channel traffic patterns, data traffic between Type 1 devices and other devices, effective bandwidth associated with the channel, security criteria, channel switching plans, standards, quality criteria, signal quality conditions, and / or considerations.

[0126] Information about a specific MAC address and / or selected channel can be communicated between a Type 1 device and a server (e.g., a hub device) over a network. Information about a specific MAC address and / or selected channel can also be communicated between a Type 2 device and a server (e.g., a hub device) over another network. A Type 2 device can communicate information about a specific MAC address and / or selected channel to another Type 2 device (e.g., via a mesh network, Bluetooth, WiFi, NFC, ZigBee, etc.). The specific MAC address and / or selected channel may be selected by a server (e.g., a hub device). The specific MAC address and / or selected channel may be signaled within an announcement channel by a Type 1 device, a Type 2 device, and / or a server (e.g., a hub device). Any information may be pre-processed before being communicated.

[0127] A radio connection (e.g., association, authentication) between a Type 1 device and another radio device can be established (e.g., using a signal handshake). The Type 1 device may send a first handshake signal (e.g., a sounding frame, probe signal, request to transmit RTS) to the other device. The other device may respond by sending a second handshake signal (e.g., a command, or transmitable CTS) to the Type 1 device, triggering the Type 1 device to broadcast a signal (e.g., a series of probe signals) to multiple Type 2 devices without establishing a connection with any Type 2 device. The second handshake signal may be a response or acknowledgment (e.g., ACK) to the first handshake signal. The second handshake signal may include data having location and / or information about the Type 1 device. The other device may be a dummy device with the purpose (e.g., primary purpose, secondary purpose) of establishing a radio connection with the Type 1 device, receiving the first signal, and / or sending the second signal. The other device may be physically attached to the Type 1 device.

[0128] In another example, another device may send a third handshake signal to a Type 1 device that triggers the Type 1 device to broadcast a signal (e.g., a series of probe signals) to multiple Type 2 devices without establishing a connection (e.g., association, authentication) with any Type 2 device. The Type 1 device may respond to the third special signal by sending a fourth handshake signal to another device. Another device may be used to trigger multiple Type 1 devices to broadcast. The trigger may be continuous, partially continuous, partially parallel, or fully parallel. Another device may have multiple radio circuits to trigger multiple transmitters in parallel. Parallel triggering may also be achieved by using at least one yet another device to perform a trigger in parallel with another device (similar to what the other device does). After establishing a connection with a Type 1 device, another device may not be able to communicate with (or suspend communication with) the Type 1 device. Suspended communication may be resumed. After establishing a connection with a Type 1 device, another device may enter inactive mode, dormant mode, sleep mode, standby mode, low power mode, OFF mode, and / or power-down mode. Another device may have a specific MAC address so that the Type 1 device transmits signals to a specific MAC address. The Type 1 device and / or the other device may be controlled and / or coordinated by a first processor associated with the Type 1 device, a second processor associated with the other device, a third processor associated with a specified source, and / or a fourth processor associated with the other device. The first and second processors can coordinate with each other.

[0129] A first set of probe signals may be transmitted to at least one first type 2 device by a first antenna of a type 1 device through a first channel at a first location. A second set of probe signals may be transmitted to at least one second type 2 device by a second antenna of a type 1 device through a second channel at a second location. The first set of probe signals and the second set of probe signals may be different or different. The at least one first type 2 device may be different or different from the at least one second type 2 device. The first and / or second sets of probe signals may be broadcast without an established connection (e.g., association, authentication) between the type 1 device and any type 2 device. The first and second antennas may be the same or different.

[0130] The two locations may have different sizes, shapes, and multipath characteristics. The first and second locations may overlap. The immediate vicinity areas of the first and second antennas may overlap. The first and second channels may be the same or different. For example, the first may be WiFi and the second may be LTE. Or, both may be WiFi, but the first may be 2.4GHz WiFi and the second may be 5GHz WiFi. Or, both may be 2.4GHz WiFi, but have different channel numbers, SSID names, and / or WiFi settings.

[0131] Each Type 2 device can acquire at least one TSCI from its respective set of probe signals, where CI is the respective channel between the Type 2 device and the Type 1 device. Some first Type 2 devices (one or more) and some second Type 2 devices (one or more) may be the same. The first and second sets of probe signals may be synchronous / asynchronous. The probe signals may be transmitted with data or replaced with data signals. The first and second antennas may be the same.

[0132] A first series of probe signals may be transmitted at a first speed (e.g., 30 Hz). A second series of probe signals may be transmitted at a second speed (e.g., 200 Hz). The first and second speeds may be the same or different. The first and / or second speeds may be changed (e.g., adjusted, modified) over time. Changes may be in accordance with schedules, rules, policies, modes, conditions, circumstances, and / or changes. Any speed may be changed (e.g., adjusted, modified) over time.

[0133] A first and / or second set of probe signals may be transmitted to the first MAC address and / or the second MAC address, respectively. The two MAC addresses may be the same or different. The first set of probe signals may be transmitted within the first channel. The second set of probe signals may be transmitted on the second channel. The two channels may be the same or different. The first or second MAC address and the first or second channel may change over time. Any changes may be in accordance with schedules, rules, policies, modes, states, circumstances, and / or changes.

[0134] Type 1 devices and other devices may be controlled and / or coordinated, physically mounted to a common device, or be part of / within the common device. They may be controlled by / connected to a common data processor, or connected to a common bus interconnect / network / LAN / Bluetooth network / NFC network / BLE / wired network / wireless network / mesh network / mobile network / cloud. They may share common memory, or be associated with common users, user devices, profiles, accounts, identities (IDs), identifiers, homes, residences, physical addresses, locations, geographic coordinates, IP subnets, SSIDs, home devices, office devices, and / or manufacturing devices.

[0135] Each Type 1 device can be a signal source for its respective set of Type 2 devices (i.e., it sends each signal (e.g., each set of probe signals) to its respective set of Type 2 devices). Each Type 2 device selects a Type 1 device from all Type 1 devices as its signal source. Each Type 2 device can be selected asynchronously. At least one TSCI may be acquired by each Type 2 device from each set of probe signals from a Type 1 device, and the CI is a channel between Type 2 and Type 1 devices.

[0136] Each Type 2 device selects a Type 1 device as its signal source from among all Type 1 devices based on its identity (ID) or Type 1 / Type 2 device identifier, the task to be performed, past signal sources, history (e.g., past signal sources, Type 1 device, another Type 1 device, each Type 2 receiver, and / or another Type 2 receiver), thresholds for switching signal sources, and / or user information, account, access information, parameters, characteristics, and / or signal strength (e.g., associated with Type 1 devices and / or each Type 2 receiver).

[0137] Initially, a Type 1 device can be the signal source for each initial set of Type 2 devices (i.e., each Type 1 device sends its respective signal (a series of probe signals) to each initial set of Type 2 devices). Each initial Type 2 device selects a Type 1 device from among all the Type 1 devices as its signal source.

[0138] A signal source (Type 1 device) of a particular Type 2 device may be modified (e.g., adjusted, altered, or corrected) if: (1) the time interval between two adjacent probe signals received from the current signal source of the Type 2 device (e.g., between the current probe signal and the immediately preceding probe signal, or between the next probe signal and the current probe signal) exceeds a first threshold; (2) the signal intensity associated with the current signal source of the Type 2 device is below a second threshold; (3) the processed signal intensity associated with the current signal source of the Type 2 device is below a third threshold, and the signal intensity is processed with a low-pass filter, band-pass filter, median filter, moving average filter, weighted average filter, linear filter, and / or nonlinear filter; and / or (4) the signal intensity (or processed signal intensity) associated with the current signal source of the Type 2 device is below a fourth threshold for a significant percentage (e.g., 70%, 80%, 90%) of the recent time window, the percentage may exceed a fifth threshold. The first, second, third, fourth, and / or fifth thresholds may change over time.

[0139] Condition (1) can occur when the Type 1 and Type 2 devices gradually move further apart from each other, resulting in some probe signals from the Type 1 device becoming too weak to be received by the Type 2 device. Conditions (2) to (4) can occur when the two devices move so far apart that the signal strength becomes very weak.

[0140] The signal source of a Type 2 device may remain unchanged if other Type 1 devices have a weaker signal strength than the current signal source factor (e.g., 1, 1.1, 1.2, or 1.5).

[0141] If the signal source is changed (adjusted, modified, corrected, etc.), the new signal source may become active in the near future (e.g., the next time each change occurs). The new signal source may be a Type 1 device having the strongest signal strength and / or processed signal strength. The current signal source and the new signal source may be the same or different.

[0142] A list of available Type 1 devices may be initialized and maintained by each Type 2 device. The list may be updated by examining the signal strength and / or processed signal strength associated with each set of Type 1 devices. A Type 2 device may be selected from a first set of probe signals from a first Type 1 device and a second set of probe signals from a second Type 1 device based on the respective probe signal rates, MAC addresses, channels, features / characteristics / states, the tasks to be performed by the Type 2 device, the signal strengths of the first and second sets of signals, and / or other considerations.

[0143] A series of probe signals may be transmitted at a constant rate (e.g., 100 Hz). A series of probe signals may be scheduled at regular intervals (e.g., 0.01 s for 100 Hz), although each probe signal may experience small time perturbations due to timing requirements, timing control, network control, handshake, message passing, collision avoidance, carrier sensing, congestion, resource availability, and / or other considerations.

[0144] The rate may be changed (e.g., adjusted, modified, and corrected). Changes may follow a schedule (e.g., changed hourly), rules, policies, modes, conditions, and / or changes (e.g., changed whenever a certain event occurs). For example, the rate is typically 100Hz but may be changed to 1000Hz in demanding situations and to 1Hz in low-power / standby situations. Probe signals may be transmitted in bursts.

[0145] The probe signal rate may vary based on the task performed by the Type 1 or Type 2 device (e.g., a task may typically require 100Hz, but temporarily require 1000Hz for 20 seconds). In one example, the transmitter (Type 1 device), receiver (Type 2 device), and associated task may be adaptively (and / or dynamically) associated with a class (e.g., low priority, high priority, urgent, critical, normal, privileged, unsubscribed, subscribed, paid, and / or uncharged). The rate (of the transmitter) may be adjusted for several classes (e.g., high priority classes). If the needs of that class change, the rate can be changed (e.g., adjusted, modified, corrected). If the receiver has critically low power, the rate may be reduced to reduce the receiver's power consumption in response to the probe signal. In one example, the probe signal may be used to wirelessly transfer power to the receiver (Type 2 device), and the rate may be adjusted to control the amount of power transferred to the receiver.

[0146] The rate can be changed by (or based on) the following: a server (e.g., a hub device), a Type 1 device, and / or a Type 2 device. Control signals may be communicated between them. The server can monitor, track, predict, and / or forecast the needs of the Type 2 device and / or the tasks performed by the Type 2 device, and control the Type 1 device to change the rate. The server can make scheduled changes to the rate according to a schedule. The server can detect emergencies and change the rate immediately. The server can detect developments and adjust the rate gradually.

[0147] Characteristics and / or STIs (e.g., motion information) may be monitored individually based on TSCIs associated with specific Type 1 devices and specific Type 2 devices, and / or jointly based on any TSCIs associated with specific Type 1 devices and any Type 2 devices, and / or jointly based on any TSCIs associated with specific Type 2 devices and any Type 1 devices, and / or globally based on any TSCIs associated with any Type 1 devices and any Type 2 devices. Any joint monitoring may relate to: users, user accounts, profiles, homes, location maps, location environment models, and / or user history, etc.

[0148] The first channel between a Type 1 device and a Type 2 device may differ from the second channel between another Type 1 device and another Type 2 device. The two channels may relate to different frequency bands, bandwidths, carrier frequencies, modulation, radio standards, coding, encryption, payload characteristics, networks, network IDs, SSIDs, network characteristics, network configurations, and / or network parameters.

[0149] The two channels can relate to two different types of wireless systems (for example, two of the following: WiFi, LTE, LTE-A, LTE-U, 2.5G, 3G, 3.5G, 4G, Beyond 4G, 5G, 6G, 7G, cellular network standards, UMTS, 3GPP, GSM, EDGE, TDMA, FDMA, CDMA, WCDMA®, TD-SCDMA, 802.11 systems, 802.15 systems, 802.16 systems, mesh networks, Zigbee, NFC, WiMAX, Bluetooth, BLE, RFID, UWB, microwave systems, radar-like systems). For example, one could be WiFi and the other LTE.

[0150] The two channels may be associated with similar types of wireless systems, but with different networks. For example, the first channel might be associated with a WiFi network named "Pizza and Pizza" in the 2.4GHz band with a 20MHz bandwidth, while the second channel might be associated with a WiFi network with the SSID "StarBud Hotspot" in the 5GHz band with a 40MHz bandwidth. The two channels may also be different channels within the same network (such as the "StarBud Hotspot" network).

[0151] In one embodiment, the wireless monitoring system may include training a classifier of multiple events at a location based on a training TSCI associated with multiple events. The CI or TSCI associated with an event may consider / include wireless samples / characteristics / fingerprints associated with that event (and / or location, environment, object, object movement, state / emotional state / mental state / state / stage / gesture / gait / action / movement / activity / daily activity / history / object event, etc.).

[0152] For each of several known events occurring at a location during each training time associated with a known event (e.g., survey, wirelee survey, initial wirelee survey), each training radio signal (e.g., each set of training probe signals) may be transmitted to at least one first type 2 heterogeneous radio device through the radio multipath channel of the location during each training time by the antenna of a first type 1 heterogeneous radio device using the processor, memory, and instruction set of a first type 1 device.

[0153] At least one time series of each training CI (training TSCI) can be acquired asynchronously from the training signal by each of at least one first type 2 device. The CI may be a channel between the first type 2 device and the first type 1 device during the training time associated with a known event. At least one training TSCI may be preprocessed. Training may be a radio survey (e.g., between the installation of the type 1 device and / or type 2 device).

[0154] For current events occurring within a location during the current period, current radio signals (e.g., a series of current probe signals) may be transmitted by the antenna of a second type 1 heterogeneous radio device using a processor, memory, and instruction set of a second type 1 device to at least one second type 2 heterogeneous radio device through a channel of the location during the current period related to the current event.

[0155] At least one time series of the current CI (current TSCI) can be acquired asynchronously from the current signal (e.g., a series of current probe signals) by each of at least one second type 2 device. The CI may be the CI of the channel between the second type 2 device and the second type 1 device during the current period related to the current event. At least one current TSCI can be preprocessed.

[0156] A classifier may be applied by at least one second type 2 device to classify at least one current TSCI obtained from a series of current probe signals, to classify at least one portion of a particular current TSCI, and / or to classify a combination of at least one portion of a particular current TSCI and another portion of another TSCI. The classifier can divide the TSCI (or features / STI or other analysis or output response) into clusters and associate the clusters with specific events / objects / subjects / locations / movements / activities. Labels / tags may be generated for the clusters. The clusters can be stored and retrieved. The classifier may be applied to relate the current TSCI (or possibly a characteristic / STI or other analysis / output response related to the current event) to: clusters, known / specific events, classes / categories / groups / groupings / lists / clusters, sets / targets / locations / movements / activity of known events, unknown events, classes / categories / groups / groupings / lists / clusters / sets / targets / locations / movements / activity of unknown events, and / or other events / targets / locations / movements / activity / classes / categories / groups / groupings / lists / clusters. Each TSCI may contain at least one CI, each associated with its respective timestamp. Two TSCIs related to two type 2 devices will differ by different start times, durations, stop times, amounts of CIs, sampling frequencies, and sampling periods. Their CIs may have different characteristics. The first and second type 1 devices may be at the same location in a place. They may be the same device. At least one second type 2 device (or their location) may replace at least one first type 2 device (or their location). A specific second type 2 device and a specific first type 2 device may be the same device.

[0157] A subset of the first type 2 device and a subset of the second type 2 device may be the same. At least one second type 2 device and / or at least one subset of the second type 2 device may be a subset of at least one first type 2 device. At least one first type 2 device and / or at least one subset of the first type 2 device may be a replacement for at least one subset of the second type 2 device. At least one second type 2 device and / or at least one subset of the second type 2 device may be a replacement for at least one subset of the first type 2 device. At least one second type 2 device and / or at least one subset of the second type 2 device may be in the same respective positions as at least one subset of the first type 2 device. At least one first type 2 device and / or at least one subset of the first type 2 device may be in the same respective positions as at least one subset of the second type 2 device.

[0158] The antennas of a Type 1 device and a second Type 1 device may be in the same location. The antennas of at least one second Type 2 device and / or at least one subset of second Type 2 devices may be in the same respective locations as the respective antennas of at least one subset of first Type 2 devices. The antennas of at least one first Type 2 device and / or at least one subset of first Type 2 devices may be in the same respective locations as the respective antennas of at least one subset of second Type 2 devices.

[0159] The first section of the first duration of the first TSCI and the second section of the second duration of the second section of the second TSCI may be aligned. A map between the items in the first section and the items in the second section can be calculated. The first section may include a first segment (e.g., a subset) of the first TSCI having a first start / end time, and / or another segment (e.g., a subset) of the processed first TSCI. The processed first TSCI may be the first TSCI processed by the first operation. The second section may include a second segment (e.g., a subset) of the second TSCI having a second start time and a second end time, and another segment (e.g., a subset) of the processed second TSCI. The processed second TSCI may be the second TSCI processed by the second operation. The first and / or second operations may include subsampling, resampling, interpolation, filtering, transformation, feature extraction, preprocessing, and / or other operations.

[0160] The first item in the first section may be mapped to the second item in the second section. The first item in the first section may also be mapped to another item in the second section. Another item in the first section may also be mapped to the second item in the second section. Mappings can be one-to-one, one-to-many, many-to-one, or many-to-many. At least one function of at least one of the following can satisfy at least one constraint: the first item in the first section of the first TSCI, another item in the first TSCI, the timestamp of the first item, the time difference of the first item, the time difference of the first item, the adjacent timestamp of the first item, the adjacent timestamp of the first item, another timestamp related to the first item, the second item in the second section of the second TSCI, another item in the second TSCI, the timestamp of the second item, the time difference of the second item, the time difference of the second item, the adjacent timestamp of the second item, and another timestamp related to the second item.

[0161] One constraint may be that the difference between the timestamp of the first item and the timestamp of the second item may be limited by an adaptive (and / or dynamically adjusted) upper threshold and limited by an adaptive lower threshold.

[0162] The first section may be the entire first TSCI. The second section may be the entire second TSCI. The first time duration may be equal to the second time duration. The sections of the time duration of a TSCI may be determined adaptively (and / or dynamically). A provisional section of a TSCI can be calculated. The start and end times of a section (e.g., provisional section, section) can be determined. A section can be determined by removing the beginning and end parts of a provisional section. The beginning part of a provisional section can be determined as follows. Iteratively, the items in a provisional section with increasing timestamps can be considered as the current item, which is one item at a time.

[0163] In each iteration, at least one activity metric / indicator may be calculated and / or considered. At least one activity metric may be associated with at least one of the following: the current item associated with the current timestamp, past items in the provisional section with a timestamp not greater than the current timestamp, and / or future items in the provisional section with a timestamp not less than the current timestamp. If at least one criterion (e.g., quality criterion, signal quality condition) associated with at least one activity metric is met, the current item may be added to the beginning of the provisional section.

[0164] At least one criterion related to the activity scale may include at least one of the following: (a) the activity scale is less than an adaptive (dynamically adjusted) upper threshold; (b) the activity scale is greater than an adaptive lower threshold; (c) the activity scale is consecutively less than an adaptive upper threshold for at least a given amount of consecutive timestamps; (d) the activity scale is consecutively greater than an adaptive lower threshold for at least another given amount of consecutive timestamps; (e) the activity scale is consecutively less than an adaptive upper threshold for at least a given percentage of a given amount of consecutive timestamps; (f) the activity scale is consecutively greater than an adaptive lower threshold for at least another given percentage of another given amount of consecutive timestamps; (g) another activity scale related to another timestamp related to the current timestamp is less than another adaptive upper threshold and greater than another adaptive lower threshold; (h) at least one activity scale related to each of the at least one timestamp related to the current timestamp is less than its respective upper threshold and greater than its respective lower threshold. (i) The percentage of timestamps associated with an activity metric that is less than each upper threshold and greater than each lower threshold in a set of timestamps associated with the current timestamp exceeds the threshold, and (j) another criterion (e.g., quality criterion, signal quality condition).

[0165] An activity measure / index associated with an item at time T1 may include at least one of the following: (1) a first function of the item at time T1 and the item at time T1-D1, where D1 is a predetermined positive quantity (e.g., a constant time offset); (2) a second function of the item at time T1 and the item at time T1+D1; (3) a third function of the item at time T1 and the item at time T2, where T2 is a predetermined quantity (e.g., a fixed initial reference time; T2 may change over time (e.g., adjust, change, modify); T2 may be periodically updated; T2 may be the start of a period and T1 may be a sliding time in the period); and (4) a fourth function of the item at time T1 and other items.

[0166] At least one of the first, second, third, and / or fourth functions may be a function (e.g., F(X, Y, ...)) with at least two arguments X and Y, the two arguments may be scalars. The function (e.g., F) may be at least one of the following: X, Y, (XY), (YX), abs(XY), X^a, Y^b, abs(X^aY^b), (XY)^a, (X / Y), (X+a) / (Y+b), (X^a / Y^b), and ((X / Y)^ab), where a and b may be some predetermined values. For example, this function may simply be abs(XY), or (XY)^2, (XY)^4. This function may be a robust function. For example, the function is (XY)^2 when abs(XY) is less than the threshold T, and (XY)+a when abs(XY) is greater than T. Alternatively, when abs(XY) is greater than T, the function may be a constant. Also, when abs(XY) is greater than T, the function may be constrained by a slowly increasing function, and as a result, outliers cannot have a significant impact on the result. Another example of this function may be (abs(X / Y)-a) (where a=1). In this way, when X=Y (i.e., no change or activity), the function gives a value of 0. When X is greater than Y, (X / Y) is greater than 1 (assuming X and Y are positive), and the function is positive. When X is less than Y, (X / Y) is less than 1, and the function is negative. In another example, both arguments X and Y may be n-tuples such that X=(x_1, x_2, ..., x_n) and Y=(y_1, y_2, ..., y_n). This function may be at least one of the following functions: x_i, y_i, (x_i-y_i), (y_ix_i), abs(x_i-y_i), x_i^a, y_i^b, abs(x_i^a-y_i^b), (x_i-y_i)^a, (x_i / y_i), (x_i+a) / (y_i +b), (x_i^a / y_i^b), and ((x_i / y_i)^ab), where i is the component exponent of the n-tuples X and Y, and 1<=i<=n. For example, the component index of x_1 is i=1, and the component index of x_2 is i=2.The function can contain the sum of another function for at least one component from among x_i, y_i, (x_i-y_i), (y_ix_i), abs(x_i-y_i), x_i^a, y_i^b, abs(x_i^a-y_i^b), (x_i-y_i)^a, (x_i / y_i), (x_i+a) / (y_i +b), (x_i ^a / y_i^b), and ((x_i / y_i)^ab), where i is the component index of the n tuple X and Y. For example, this function could be of the form sum_{i=1}^n(abs(x_i / y_i)-1) / n, or sum_{i=1}^nw_i*(abs(x_i / y_i)-1), where w_i is the weight of component i.

[0167] The map can be computed using dynamic time warping (DTW). The DTW may include constraints on the map, items of the first TSCI, items of the second TSCI, the first duration, the second duration, the first section, and / or the second section. Suppose in the map that i^{th} domain items are mapped to j^{th} range items. The constraints may be on the acceptable combinations of i and j (constraints on the relationship between i and j). The mismatch cost between the first section of the first duration of the first TSCI and the second section of the second duration of the second TSCI can be computed.

[0168] The first and second sections can be aligned such that a map containing multiple links can be established between a first item in the first TSCI and a second item in the second TSCI. Each link can associate one of the first items with a first timestamp with one of the second items with a second timestamp. A mismatch cost can be calculated between the aligned first section and the aligned second section. The mismatch cost can be a function of the cost of the items between the first and second items associated by a particular link in the map and the (link-wise) cost of the links associated with a particular link in the map.

[0169] The aligned first section and the aligned second section may be represented as the first and second vectors, respectively, having the same vector length. The mismatch cost may include at least one of the following between the first and second vectors: the inner product, inner product quantifier, correlation-based quantity, correlation indicator, covariance-based quantity, discrimination score, distance, Euclidean distance, absolute distance, Lk distance (e.g., L1, L2, ...), weighted distance, distance quantifier, and / or other similarity values. The mismatch cost may be normalized by the respective vector lengths.

[0170] The parameters derived from the mismatch cost between the first section of the first duration of the first TSCI and the second section of the second duration of the second TSCI can be modeled by a statistical distribution. At least one of the scale parameter, location parameter, and / or other parameters of the statistical distribution can be estimated.

[0171] The first section of the first duration of the first TSCI may be a sliding section of the first TSCI. The second section of the second duration of the second TSCI may be a sliding section of the second TSCI.

[0172] A first sliding window may be applied to a first TSCI, and a corresponding second sliding window may be applied to a second TSCI. The first sliding window of the first TSCI and the corresponding second sliding window of the second TSCI may be aligned.

[0173] The mismatch cost can be calculated between the aligned first sliding window of the first TSCI and the corresponding aligned second sliding window of the second TSCI. Based on the mismatch cost, the current event may be associated with at least one of the known event, the unknown event, and / or another event.

[0174] The classifier may be applied to at least one of the first sections of the first duration of the first TSCI and / or the second sections of the second duration of the second TSCI in order to obtain at least one provisional classification result. Each provisional classification result may be associated with its respective first section and its respective second section.

[0175] The current event can be associated with at least one of the following based on mismatch cost: a known event, an unknown event, a set of classes / categories / groups / groupings / lists / unknown events, and / or another event. The current event can be associated with at least one of the known events, unknown events, and / or another event based on the most numerous provisional classification results in multiple sections of the first TSCI and the corresponding multiple sections of the second TSCI: For example, if the mismatch cost points to a particular known event in N consecutive (e.g., N=10) sequences, the current event can be associated with that particular known event. In another example, the current event can be associated with a particular known event if the percentage of mismatch costs in the N preceding consecutive sequences pointing to that particular known event exceeds a given threshold (e.g., >80%).

[0176] In another example, the current event may be associated with a known event that achieves the minimum mismatch cost for the most times in time. The current event may be associated with a known event that achieves the minimum overall mismatch cost, which is a weighted average of at least one mismatch cost associated with at least one first section. The current event may be associated with a specific known event that achieves a minimum of another overall cost. The current event may be associated with an “unknown event” if none of the known events achieve a mismatch cost lower than a first threshold T1 for a sufficient percentage of at least one first section. The current event may also be associated with an “unknown event” if none of the events achieve an overall mismatch cost lower than a second threshold T2. The current event may be associated with at least one of the known events, unknown events, and / or other events based on the mismatch costs and additional mismatch costs associated with at least one additional section of the first TSCI and at least one additional section of the second TSCI. Known events may include at least one of the following: door closing events, door opening events, window closing events, window opening events, multi-state events, on-state events, off-state events, intermediate-state events, continuous-state events, discrete-state events, human presence events, human absence events, life presence events, and / or life non-presence events.

[0177] The projection for each CI can be trained using a dimensionality reduction method based on the trained TSCI. The dimensionality reduction method may include at least one of the following: principal component analysis (PCA), PCA with different kernels, independent component analysis (ICA), Fisher linear decision formula, vector quantization, supervised learning, unsupervised learning, self-organizing maps, autoencoders, neural networks, deep neural networks, and / or other methods. The projection can be applied to at least one of the trained TSCIs and / or the current TSCI associated with at least one event for the classifier.

[0178] A classifier for at least one event may be trained based on a projection associated with at least one event and its associated training TSCI. At least one current TSCI may be classified / categorized based on the projection and the current TSCI. The projection may be retrained using a dimensionality reduction method and at least one of other dimensionality reduction methods, based on at least one of the training TSCI, at least one current TSCI before retraining the projection, and / or additional training TSCIs. Other dimensionality reduction methods may include at least one of principal component analysis (PCA), PCA with different kernels, independent component analysis (ICA), Fisher linear decision, vector quantization, supervised learning, unsupervised learning, self-organizing maps, autoencoders, neural networks, deep neural networks, and / or yet another method. A classifier for at least one event may be retrained based on the retrained projection, at least one training TSCI associated with at least one event, and / or at least one current TSCI. At least one current TSCI can be classified based on a retrained projection, a retrained classifier, and / or the current TSCI.

[0179] Each CI may contain a vector of complex numbers. Each complex number may be preprocessed to give a magnitude for the complex number. Each CI may be preprocessed to give a vector of non-negative real numbers containing the magnitudes for the corresponding complex numbers. Each training TSCI may be weighted in the training of the projection. The projection may contain multiple projection components. The projection may contain at least one most significant projection component. The projection may contain at least one projected component that may be useful to the classifier.

[0180] Channel / Channel information / Location (venue) / Spatial information (spatia-temporal) / Motion / Object

[0181] Channel information (CI) includes signal strength, signal amplitude, signal phase, spectral power measurement, modem parameters (e.g., used in relation to modulation / demodulation in digital communication systems such as WiFi and 4G / LTE), dynamic beamforming information, transfer function components, radio state (e.g., used in digital communication systems to decode digital data, baseband processing state, RF processing state, etc.), measurable variables, sensing data, coarse / fine-grained layer information (e.g., physical layer, data link layer, MAC layer, etc.), digital settings, gain settings, RF filter settings, RF front-end switch settings, DC offset settings, DC correction settings, IQ correction settings, influence of the environment during propagation (e.g., location) on the radio signal, and the output signal (Type 2) of the input signal (radio signal transmitted by a Type 1 device). Each CI may be associated with / include: conversion to a radio signal received by a device, stable behavior of the environment, state profile, radio channel measurement, received signal strength indicator (RSSI), channel state information (CSI), channel impulse response (CFR), channel frequency response (CFR), characteristics of frequency components (e.g., subcarriers) in the bandwidth, channel characteristics, channel response, timestamp, auxiliary information, data, metadata, user data, account data, access data, security data, session data, status data, supervisory data, home data, identification (ID), identifier, device data, network data, neighborhood data, environmental data, real-time data, sensor data, stored data, encrypted data, compressed data, protected data, and / or other channel information. Each CI may be associated with a timestamp and / or arrival time. The CSI can equalize / undo / minimize / reduce multipath channel effects (transmitting channels) and demodulate a signal similar to that transmitted by the transmitter through the multipath channel. CI can be associated with information related to the frequency bandwidth, frequency signature, frequency phase, frequency amplitude, frequency trend, frequency characteristics, frequency-like characteristics, time-domain elements, frequency-domain elements, time-frequency-domain elements, orthogonal-resolved characteristics, and / or non-orthogonal-resolved characteristics of the signal passing through the channel. TSCI can be a stream of radio signals (e.g., CI).

[0182] CI can be pre-processed, processed, post-processed, stored (e.g., in local memory, portable / mobile memory, removable memory, storage network, cloud memory, in a volatile or non-volatile manner), retrieved, transmitted, and / or received. One or more modem parameters and / or radio state parameters can be kept constant. Modem parameters can be applied to the radio subsystem. Modem parameters can represent radio states. Motion detection signals (e.g., baseband signals, and / or packets decoded / demodulated from baseband signals) can be obtained by processing (e.g., down-converting) a first radio signal (e.g., RF / WiFi / LTE / 5G signal) by the radio subsystem using the radio state represented by the stored modem parameters. Modem parameters / radio states can be updated (e.g., using previous modem parameters or previous radio states). Both previous and updated modem parameters / radio states can be applied to the radio subsystem of a digital communication system. Both previous and updated modem parameters / radio states can be compared / analyzed / processed / monitored in a task.

[0183] Channel information may also be modem parameters (e.g., stored or newly calculated) used to process the radio signal. The radio signal may include multiple probe signals. The same modem parameters can be used to process multiple probe signals. It is also possible to process multiple radio signals using the same modem parameters. Modem parameters may include parameters that indicate the settings or overall configuration for the operation of the radio subsystem or baseband subsystem (or both) of the radio sensor device. Modem parameters may include one or more of the following: gain settings, RF filter settings, RF front-end switch settings, DC offset settings, or IQ compensation settings for the radio subsystem, or digital DC correction settings, digital gain settings, and / or digital filtering settings (e.g., for the baseband subsystem). CI may also relate to information relating to the signal's time, time signature, timestamp, time amplitude, time phase, time trend, and / or time characteristics. CI may relate to information relating to the signal's time-frequency division, signature, amplitude, phase, trend, and / or characteristics. CI may relate to the decomposition of the signal. A CI may relate to information relating to direction, angle of arrival (AoA), angle of the directional antenna, and / or the phase of the signal passing through the channel. A CI may relate to the attenuation pattern of the signal passing through the channel. Each CI may be associated with a Type 1 device and a Type 2 device. Each CI may be associated with the antenna of a Type 1 device and the antenna of a Type 2 device.

[0184] CI can be obtained from communication hardware (e.g., a Type 2 device or a Type 1 device) that can provide CI. Communication hardware may include WiFi-enabled chips / ICs (integrated circuits), chips compliant with 802.11 or 802.16 or other radio / wireless standards, next-generation WiFi-enabled chips, LTE-enabled chips, 5G-enabled chips, 6G / 7G / 8G-enabled chips, Bluetooth-enabled chips, NFC-enabled chips, BLE-enabled chips, UWB chips, and other communication chips (e.g., Zigbee, WiMAX, mesh networks). The communication hardware calculates the CI and stores it in buffer memory so that it can be used for extraction. CI may include data and / or at least one matrix related to channel state information (CSI). At least one matrix may be used for channel equalization and / or beamforming, etc. Channels may be associated with locations. Attenuation can be due to signal propagation in place, signal propagation / reflection / refraction / diffraction through / in / near air (e.g., the air of the place), refracting media / reflective surfaces such as walls, doors, furniture, obstacles and / or barriers. Attenuation can also be due to reflections on surfaces and obstacles (e.g., reflective surfaces, obstacles) such as floors, ceilings, furniture, fixtures, objects, people, pets, etc. Each CI may be associated with a timestamp. Each CI may contain N1 components (e.g., N1 frequency domain components in a CFR, N1 time domain components in a CIR, or N1 decomposed components). Each component may be associated with a component index. Each component may be a real, imaginary, or complex quantity, magnitude, phase, flag, and / or set. Each CI may contain a vector or matrix of complex numbers, a set of mixed quantities, and / or a multidimensional set of at least one complex number.

[0185] The components of a TSCI associated with a specific component index can form their own component time series, each associated with its respective index. A TSCI can be divided into N1 component time series. Each individual component time series is associated with its respective component index. The motion characteristics / STI of an object can be monitored based on the component time series. In one example, one or more ranges of CI components (e.g., one range from component 11 to component 23, a second range from component 44 to component 50, and a third range with only one component) can be selected based on several criteria / cost functions / signal quality metrics (e.g., based on signal-to-noise ratio and / or interference level) for further processing.

[0186] The component-wise properties of the component-feature time series of the TSCI may be calculated. The component-wise properties may be scalars (e.g., energy) or functions with domains and ranges (e.g., autocorrelation function, transformation, inverse transformation). The motion properties / STI of an object may be monitored based on the properties of its components. The total properties of the TSCI (e.g., aggregated properties) may be calculated based on the properties of each component time series of the TSCI. The total properties may be a weighted average of the properties of the components. The motion properties / STI of an object may be monitored based on the total properties. The total may be a weighted average of the individual quantities.

[0187] Type 1 and Type 2 devices may support WiFi, WiMAX, 3G / 3G Beyond, 4G / 4G Beyond, LTE, LTE-A, 5G, 6G, 7G, Bluetooth, NFC, BLE, Zigbee, UWB, UMTS, 3GPP, GSM, EDGE, TDMA, FDMA, CDMA, WCDMA, TD-SCDMA, mesh networks, proprietary wireless systems, IEEE 802.11, 802.15, 802.16 standards, 3GPP standards, and / or other wireless systems.

[0188] A common radio system and / or common radio channel may be shared by a Type 1 transceiver and / or at least one Type 2 transceiver. At least one Type 2 transceiver may transmit its respective signals simultaneously (or: asynchronously, synchronously, sporadically, continuously, repeatedly, in parallel, simultaneously, and / or at one time) using the common radio system and / or common radio channel. A Type 1 transceiver may transmit its signal to at least one Type 2 transceiver using the common radio system and / or common radio channel.

[0189] Each Type 1 and Type 2 device may have at least one transmit / receive antenna. Each CI may be associated with one of the transmit antennas of the Type 1 device and one of the receive antennas of the Type 2 device. Each pair of transmit and receive antennas may be associated with a link, path, communication path, signal hardware path, etc. For example, if a Type 1 device has M (e.g., 3) transmit antennas and a Type 2 device has N (e.g., 2) receive antennas, there may be M x N (e.g., 3 x 2 = 6) links or paths. Each link or path may be associated with a TSCI.

[0190] At least one TSCI may correspond to various pairs of antennas between a Type 1 device and a Type 2 device. A Type 1 device may have at least one antenna. A Type 2 device may also have at least one antenna. Each TSCI may be associated with an antenna of a Type 1 device and an antenna of a Type 2 device. Averaging or weighted averaging may be performed across the antenna links. Averaging or weighted averaging may be performed across at least one TSCI. Averaging may optionally be performed on a subset of at least one TSCI corresponding to a subset of antenna pairs.

[0191] The timestamps of some CIs in a TSCI may be irregular and can be corrected so that the corrected timestamps of time-corrected CIs are spaced evenly in time. For multiple Type 1 devices and / or multiple Type 2 devices, the corrected timestamps may relate to the same or different clocks. An original timestamp associated with each CI can be determined. The original timestamps may not be spaced evenly in time. The original timestamps of all CIs in a particular part of a particular TSCI in the current sliding time window can be corrected so that the corrected timestamps of time-corrected CIs are spaced evenly in time.

[0192] Characteristics and / or STI (e.g., motion information) include position, position coordinates, change in position, position (e.g., initial position, new position), position on map, height, horizontal position, vertical position, distance, displacement, velocity, acceleration, rotational velocity, rotational acceleration, direction, angle of motion, orientation, direction of motion, rotation, path, deformation, transformation, contraction, extension, gait, gait cycle, head movement, repetitive motion, periodic motion, pseudo-periodic motion, impulse motion, sudden movement, fall motion, transient motion, behavior, transient behavior, motion cycle, frequency of motion, temporal trend, temporal profile, temporal characteristics, occurrence, change, temporal change, change in CI, change in frequency, change in timing, change in gait cycle, timing, start time, beginning time, end time, duration, motion history, type of motion, classification of motion, frequency, frequency spectrum, frequency characteristics, presence, absence, proximity, being close , backward movement, object identification / identifier, object components, head movement velocity, head movement direction, mouth-related rate, eye-related rate, respiratory velocity, heart rate, tidal volume, breathing depth, inhalation time, exhalation time, inhalation-to-exhalation time ratio, airflow rate, heart rate interval, heart rate variability, hand movement rate, hand movement direction, leg movement, body movement, walking speed, hand movement velocity, positional characteristics, object movement-related characteristics (e.g., change of position / location), tool movement, machine movement, compound movement, and / or combination of multiple movements, events, signal statistics, signal dynamics, anomalies, motion statistics, motion parameters, motion detection display, motion magnitude, motion phase, similarity score, distance score, Euclidean distance, weighted distance, L_1 norm, L_2 norm, L_k norm for k>2, statistical distance, correlation, correlation indicator,Autocorrelation, covariance, autocovariance, cross-covariance, inner product, Cartesian product, motion signal transformation, motion features, presence of motion, absence of motion, motion localization, motion identification, motion recognition, presence of object, absence of object, object entry, object exit, object change, motion cycle, number of motions, gait cycle, motion rhythm, motion deformation, gesture, handwriting, head movement, mouth movement, cardiac movement, visceral movement, motion trend, size, length, area, volume, capacity, shape, form, tag, start / start position, end position, start / start amount, end amount, event, fall event, se Security events, accident events, home events, office events, factory events, warehouse events, manufacturing events, assembly line events, maintenance events, car-related events, navigation events, tracking events, door events, door open events, door close events, window events, window open events, window close events, repeatable events, one-time events, consumption, unconsumption, state, physical state, health state, comfort state, emotional state, mental state, other events, analysis, output response, and / or other information may be included. Characteristics and / or STI may be calculated / monitored based on features calculated from CI or TSCI (e.g., feature calculation / extraction). Static segments or profiles (and / or dynamic segments / profiles) may be identified / calculated / analyzed / monitored / extracted / acquired / marked / presented / indicated / remembered / communicated based on feature analysis. Analysis may include motion detection / motion evaluation / presence detection. Computational workloads can be shared across Type 1 devices, Type 2 devices, and other processors.

[0193] The Type 1 device and / or the Type 2 device can be a local device. A local device can be a smartphone, smart device, TV, soundbar, set-top box, access point, router, repeater, wireless signal repeater / extender, remote control, speaker, fan, refrigerator, microwave oven, coffee machine, hot water pot, appliance, table, chair, light, lamp, door lock, camera, microphone, motion sensor, security device, fire hydrant, garage door switch, power adapter, computer, dongle, computer peripheral, electronic pad, sofa, tile, accessory, home device, vehicle device, office device, building equipment, manufacturing equipment, wristwatch, glass, clock, TV, oven, air conditioner, accessory, utility, electrical appliance, smart machine, smart vehicle, Internet of Things (IoT), smart home, smart office, smart building, smart parking lot, smart system, and other devices.

[0194] Each Type 1 device may be associated with its own identifier (e.g., ID). Each Type 2 device may also be associated with its own identification (ID). An ID can include numbers, text-and-number combinations, names, passwords, accounts, account IDs, web links, web addresses, indexes to any information, and / or another ID. IDs can be assigned. IDs can be assigned by hardware (e.g., hardwired, via dongles and / or other hardware), software and / or firmware. IDs can be stored (e.g., in a database, in memory, in a server (e.g., a hub device), in the cloud, stored locally, stored remotely, stored permanently, stored temporarily) and retrieved. An ID can be associated with at least one record, account, user, household, address, telephone number, social security number, customer number, another ID, another identifier, timestamp, and / or data collection. The IDs and / or parts of IDs of a Type 1 device may be made available to a Type 2 device. IDs may be used by Type 1 devices and / or Type 2 devices for registration, initialization, communication, identification, verification, detection, recognition, authentication, access control, cloud access, networking, social networking, logging, recording, cataloging, classification, tagging, association, pairing, transactions, electronic transactions, and / or intellectual property control.

[0195] Objects include people, users, subjects, passengers, children, the elderly, infants, sleeping infants, infants in vehicles, patients, workers, high-value workers, professionals, specialists, waiters, customers in malls, travelers at airports / train stations / bus terminals / shipping terminals, staff / workers / customer service personnel in factories / malls / supermarkets / offices / workplaces, service personnel in sewage / air ventilation systems / liftwells, lifts in liftwells, elevators, inmates, people being tracked / monitored, animals, plants, living things, pets, dogs, cats, smartphones, telephone accessories, computers, tablets, mobile computers, dongles, computer accessories, network equipment, WiFi devices, IoT devices, smartwatches, smart glasses, smart devices, speakers, keys, smart keys, wallets, purses, handbags, backpacks, goods, cargo, luggage, equipment, motors, machinery, air conditioners, fans, air conditioning equipment, lighting fixtures, movable lights, televisions, cameras, audio and / or video equipment. This may include stationery, surveillance equipment, parts, signs, tools, carts, tickets, parking tickets, toll tickets, airplane tickets, credit cards, plastic cards, access cards, food packaging, utensils, tables, chairs, cleaning equipment / tools, vehicles, automobiles, vehicles in parking facilities, goods in warehouses / stores / supermarkets / distribution centers, boats, bicycles, airplanes, drones, remote-controlled cars / airplanes / boats, robots, manufacturing equipment, assembly lines, materials / unfinished parts / robots / carts / transportation means on factory floors, trackable objects in airports / shopping marts / supermarkets, non-trackable objects, non-existence of objects, presence of objects, objects with shape, objects with altered shape, objects without shape, mass of fluids, mass of liquids, mass of gases / smoke, fire, flames, electromagnetic (EM) sources, EM media, and / or other objects.

[0196] The object itself can be communicatively coupled to several networks such as WiFi, MiFi, 3G / 4G / LTE / 5G / 6G / 7G, Bluetooth, NFC, BLE, WiMax, Zigbee, UMTS, 3GPP, GSM, EDGE, TDMA, FDMA, CDMA, WCDMA, TD-SCDMA, mesh network, ad hoc network, and / or other networks. The object itself may be bulky with an AC power supply but moves during installation, cleaning, maintenance, renovation, etc. The object can also be installed on a movable platform such as a lift, pad, movable, platform, elevator, conveyor belt, robot, drone, forklift, automobile, boat, vehicle, etc. The object can have multiple parts, and each part can have different movements (e.g., changes in location / position). For example, the object may be a person walking forward. During walking, his left and right hands can move in different directions with different instantaneous speeds, accelerations, and movements.

[0197] A wireless transmitter (e.g., a type 1 device), a wireless receiver (e.g., a type 2 device), another wireless transmitter and / or another wireless receiver can move with the object and / or another object (e.g., in previous movement, current movement, and / or future movement). They can be communicatively coupled to one or more nearby devices. They can transmit TSCI and / or information related to TSCI to nearby devices and / or to each other. They may be together with nearby devices. The wireless transmitter and / or the wireless receiver may be part of a small (e.g., coin-sized, tobacco-box-sized, or even smaller) lightweight portable device. The portable device may be wirelessly coupled to nearby devices.

[0198] A nearby device may be a smartphone, iPhone®, Android phone, smart device, smart appliance, smart vehicle, smart gadget, smart TV, smart refrigerator, smart speaker, smartwatch, smart glasses, smart pad, iPad®, computer, wearable computer, notebook computer, or gateway. A nearby device can connect to a cloud server, a local server (e.g., a hub device), and / or other servers via the Internet, a wired internet connection, and / or a wireless internet connection. A nearby device may be portable. Portable devices, nearby devices, local servers (e.g., hub devices), and / or cloud servers can share calculations and / or storage for tasks (e.g., TSCI, determining object properties / STI related to object movement (e.g., position / change of position), calculating time series of power (e.g., signal strength) information, determining / calculating specific functions, searching for local extrema, classification, identifying specific values ​​of offset time, denoising, processing, simplification, cleaning, wireless smart sensing tasks, extracting CI from signals, switching, segmentation, estimating trajectories / paths / tracks, processing maps, processing trajectories / paths / tracks based on environmental models / constraints / limits, correction, correction adjustment, adjustment, map-based (or model-based) correction, error detection, checking for boundary hits, thresholding) and information (e.g., TSCI). Nearby devices may not move with the object. Nearby devices may be portable / non-portable / mobile / non-mobile. Nearby devices may use battery power, solar power, AC power and / or other power sources. Nearby devices may have replaceable / non-replaceable batteries and / or rechargeable / non-rechargeable batteries. Nearby devices may be similar to the object. Nearby devices may have the same (and / or similar) hardware and / or software as the object.Nearby devices may include smart devices, network-enabled devices, devices with WiFi / 3G / 4G / 5G / 6G / Zigbee / Bluetooth / NFC / UMTS / 3GPP / GSM / EDGE / TDMA / FDMA / CDMA / WCDMA / TD-SCDMA / ad-hoc networks / other network connectivity, smart speakers, smartwatches, smart clocks, smart appliances, smart machines, smart devices, smart tools, smart vehicles, Internet of Things (IoT) devices, Internet-enabled devices, computers, portable computers, tablets, and other devices. Nearby devices and / or at least one processor associated with a radio receiver, radio transmitter, another radio receiver, another radio transmitter and / or a cloud server (in the cloud) may determine the initial STI of an object. Two or more of them may jointly determine the initial spatial-temporal information. Two or more of them may share intermediate information in determining the initial STI (e.g., initial position).

[0199] In one example, a wireless transmitter (e.g., a Type 1 device, or tracker bot) moves with the object. The wireless transmitter can send a signal to a wireless receiver (e.g., a Type 2 device, or an Origin Register) or determine the object's initial STI (e.g., initial position). The wireless transmitter can also send signals and / or other signals to another wireless receiver (e.g., another Type 2 device, or another Origin Register) to monitor the object's movement (spatial-temporal information). The wireless receiver may also receive signals and / or other signals from the wireless transmitter and / or another wireless transmitter to monitor the object's movement. The positions of the wireless receiver and / or another wireless receiver are known. In another example, a wireless receiver (e.g., a Type 2 device, or tracker bot) may move with the object. The wireless receiver may receive signals sent from the wireless transmitter (e.g., a Type 1 device, or an Origin Register) to determine the object's initial spatial-temporal information (e.g., initial position). The wireless receiver may also receive signals and / or other signals from another wireless transmitter (e.g., another Type 1 device, or another origin register) for monitoring the current movement of an object (e.g., spatial-temporal information). The wireless transmitter may also transmit signals and / or other signals to the wireless receiver and / or another wireless receiver (e.g., another Type 2 device, or another tracker bot) for monitoring the movement of an object. The locations of the wireless transmitter and / or other wireless transmitters may be known.

[0200] Locations include sensing areas, rooms, houses, offices, property, workspaces, corridors, lifts, liftwells, escalators, elevators, sewers, ventilation systems, stairs, collective areas, ducts, air ducts, pipes, enclosed spaces, enclosed structures, semi-enclosed structures, enclosed areas with at least one wall, plants, machinery, engines, structures, structures with wood, structures with glass, structures with metal, structures with walls, structures with doors, structures with gaps, structures with reflective surfaces, structures with liquids, buildings, rooftops, shops, factories, assembly lines, etc. Tel rooms, museums, classrooms, schools, universities, government buildings, warehouses, garages, malls, airports, train stations, bus terminals, hubs, transportation hubs, cargo terminals, government buildings, public facilities, entertainment venues, recreation venues, hospitals, pediatric / neonatal wards, nursing homes, elderly care facilities, senior living facilities, community centers, stadiums, playgrounds, fields, basketball courts, tennis courts, soccer stadiums, baseball fields, gymnasiums, garages, shopping malls, knolls, supermarkets, manufacturing facilities, parking facilities Construction sites, mining facilities, transportation facilities, main roads, roads, valleys, forests, trees, topography, landscapes, caves, patios, land, roads, amusement parks, urban areas, rural areas, suburban areas, metropolitan areas, gardens, squares, plazas, music halls, downtown facilities, open facilities, semi-open facilities, enclosed areas, train platforms, train stations, distribution centers, warehouses, shops, storage facilities, underground spaces, space (e.g., above ground, space) facilities, floating facilities, caves, tunnel facilities, indoor facilities, outdoor facilities, outdoor spaces with some walls / doors / reflective barriers These areas include facilities, open facilities, semi-open facilities, automobiles, trucks, buses, vans, containers, ships / boats, submarines, trains, electric trains, airplanes, vehicles, mobile platforms, caverns, tunnels, pipes, channels, metropolitan areas, commercial areas with relatively tall buildings, valleys, wells, ducts, routes, gas lines, oil pipes, water pipes, interconnecting routes / arrays / roads / tubes / cavities / caves / pipe-like structures / spaces / fluid spaces, the human body, animal bodies, body cavities, organs, bones, teeth, soft tissues, hard tissues, rigid tissues, non-hard tissues, blood / fluid channels, air channels, air ducts, burrows, etc. A location may be an indoor space or an outdoor space, and a location may include both the inside and outside of a space.For example, a location can include both the interior and exterior of a building. For example, a location could be a building with one or more floors, and part of the building could be underground. The shape of the building could be, for example, round, square, rectangular, triangular, or irregular. These are merely examples. This disclosure can be used to detect events in other types of locations or spaces.

[0201] A wireless transmitter (e.g., a Type 1 device) and / or a wireless receiver (e.g., a Type 2 device) may be embedded in a portable device (e.g., a module, or a device having a module) that can move with the object (e.g., in previous and / or current movements). The portable device may be coupled to the object in a communicative manner using wired connections (e.g., via USB, micro USB, FireWire, HDMI, serial port, parallel port, and other connectors) and / or wireless connections (e.g., Bluetooth, Bluetooth Low Energy (BLE), WiFi, LTE, NFC, ZigBee). The portable device may be a lightweight device. The portable may be powered by batteries, rechargeable batteries, and / or AC power. The portable device may be very small (e.g., on a sub-millimeter and / or sub-centimeter scale) and / or small (e.g., coin-sized, card-sized, pocket-sized, or larger). The portable device may be large, bulky, and / or cumbersome (e.g., installed heavy machinery). Portable devices include WiFi hotspots, access points, mobile WiFi (MiFi), dongles with USB / micro USB / FireWire / other connectors, smartphones, portable computers, computers, tablets, smart devices, Internet of Things (IoT) devices, WiFi-enabled devices, LTE-enabled devices, smartwatches, smart glass, smart mirrors, smart antennas, smart batteries, smart lights, smart pens, smart rings, smart doors, smart windows, smart clocks, smart wallets, smart belts, smart handbags, smart cloths / garments, smart ornaments, smart packaging, smart paper / books / magazines / posters / printed materials / signage / displays / illuminated systems / lighting systems, smart keys / tools, smart bracelets / chains / necklaces / clothing / accessories, smart pads / cushions, smart tiles / blocks / bricks / building materials / other materials.Smart trash cans / waste containers, smart food carriages / storage, smart balls / rackets, smart chairs / sofas / beds, smart shoes / footwear / carpets / mats / shoe racks, smart gloves / handwear / rings / handwear, smart hats / caps / cosmetics / stickers / tattoos, smart mirrors, smart toys, smart pills, smart cookware, smart bottles / food containers, smart tools, smart devices, IoT devices, WiFi-enabled devices, network-enabled devices, 3G / 4G / 5G / 6G-enabled devices, UMTS devices, 3GPP devices, GSM devices, EDGE devices, TDMA devices, FDMA devices, CDMA devices, WCDMA devices, TD-SCDMA devices, embedded devices, embeddable devices, air conditioners, refrigerators, heaters, furnaces, furniture, ovens, cooking devices This could include televisions / set-top boxes (STBs) / DVD players / audio players / video players / remote controls, hi-fi, audio devices, speakers, lamps / lights, walls, doors, windows, roofs, tiles / roofing boards / structures / attic structures / devices / features / installations / fixtures, lawnmowers / gardening tools / tools / machine tools / garage equipment, garbage cans / containers, 20-foot / 40-foot containers, storage containers, factory / production / manufacturing equipment, repair tools, fluid containers, machinery, installed machinery, vehicles, carts, wagons, warehouse vehicles, automobiles, bicycles, motorcycles, boats, ships, airplanes, baskets / boxes / bags / buckets / containers, smart plates / cups / bowls / pots / mats / appliances / kitchen utensils / kitchenware / kitchen accessories / cabinets / tables / chairs / tiles / lights / water pipes / faucets / gas ranges / ovens / dishwashers / , etc. Portable devices may have batteries that are replaceable, non-replaceable, rechargeable, and / or non-rechargeable. Portable devices may be charged wirelessly. The portable device may be a smart payment card. The portable device may be a payment card used in parking lots, highways, entertainment parks, or other places / facilities where payment is required. As described above, the portable device may have an identity (ID) / identifier.

[0202] Events can be monitored based on TSCI. Events can be object-related events such as objects (e.g., people and / or sick people) falling, spinning, hesitating, resting, impact (e.g., a person hitting a punching bag, door, window, bed, chair, table, desk, cabinet, box, another person, animal, bird, flight, table, chair, ball, bowling ball, tennis ball, football, soccer ball, baseball, basketball, volleyball), two-body actions (e.g., leaving a balloon, catching a fish, shaping clay, writing a paper, a person typing on a computer), moving a car in a garage, a person carrying a smartphone, a person walking around an airport / mall / government building / office / etc., and autonomously moving objects / machines moving around (e.g., vacuum cleaners, utility vehicles, cars, drones, self-driving cars).

[0203] Tasks or wireless smart sensing tasks include object detection, presence detection, proximity detection, object recognition, activity recognition, object verification, object counting, daily activity monitoring, health monitoring, vital signs monitoring, health status monitoring, baby monitoring, elderly monitoring, sleep monitoring, sleep stage monitoring, gait monitoring, motion monitoring, tool detection, tool recognition, tool verification, patient detection, patient monitoring, patient verification, machine detection, machine verification, human detection, human recognition, human verification, baby detection, baby recognition, baby verification, human respiration detection, human respiration recognition, human respiration estimation, human respiration verification, human heart rate detection, human heart rate recognition, human heart rate estimation, human heart rate verification, fall detection, fall recognition, fall estimation, fall verification, emotion detection, emotion recognition, emotion estimation, emotion verification, motion detection, motion degree estimation, Motion recognition, motion estimation, motion verification, periodic motion detection, periodic motion estimation, periodic motion verification, repetitive motion detection, periodic motion recognition, repetitive motion estimation, repetitive motion verification, static motion recognition, static motion detection, static motion estimation, static motion verification, cyclic stationary motion detection, cyclic stationary motion recognition, cyclic stationary motion estimation, cyclic stationary motion verification, transient motion detection, transient motion recognition, transient motion estimation, transient motion verification, trend detection, trend recognition, trend estimation, trend verification, respiration detection, respiration recognition, respiration estimation, human biometric detection, human biometric recognition, human biometric estimation, human biometric verification, environmental informatics detection, environmental informatics recognition, environmental informatics estimation, environmental informatics verification, gait detection, gait recognition, gait estimation, gait verification, gesture detection, gesture recognition, gesture estimation, gesture verification, machine learning, supervised learning, unsupervised learning, semi-supervised learning, clustering, feature extraction, feature training, principal component analysis, eigenvalue decomposition, frequency decomposition, time decomposition,Time-frequency decomposition, function decomposition, other decompositions, training, discrimination training, supervised training, unsupervised training, semi-supervised training, neural networks, sudden motion detection, fall detection, hazard detection, life-threat detection, regular motion detection, static motion detection, periodic steady motion detection, intrusion detection, suspicious movement detection, security, safety monitoring, navigation, guidance, map-based processing, map-based correction, model-based processing / correction, irregularity detection, localization, room sensing, tracking, multiple object tracking, indoor tracking, indoor positioning, indoor navigation, energy management, power transmission, wireless power transmission, object counting, car tracking in parking garages, device / system activation (e.g., security systems, access systems, alarms, sirens, speakers, televisions, entertainment systems, cameras, heater / air conditioning (HVAC) systems, ventilation systems, lighting systems, game systems, coffee machines, cooking appliances, cleaning equipment, housekeeping equipment), geometric estimation This may include, but may also include, augmented reality, wireless communication, data communication, signal broadcasting, networking, coordination, management, encryption, protection, cloud computing, other processing and / or other tasks. Tasks can be performed by Type 1 devices, Type 2 devices, other Type 1 devices, other Type 2 devices, nearby devices, local servers (such as hub devices), edge servers, cloud servers and / or other devices.This task can be based on TSCI between any pair of Type 1 and Type 2 devices. The Type 2 device may be a Type 1 device, and vice versa. The Type 2 device may perform the role (e.g., functionality) of the Type 1 device temporarily, continuously, sporadically, simultaneously, and / or simultaneously and / or vice versa. The first part of the task may include at least one of the following: preprocessing, processing, signal conditioning, signal processing, postprocessing, sporadic / continuous / simultaneous / concurrent / dynamic / adaptive / on-demand / as needed processing, calibration, denoising, feature extraction, coding, encryption, transformation, mapping, motion detection, motion estimation, motion change detection, motion pattern detection, motion pattern estimation, motion pattern recognition, vital sign detection, vital sign estimation, vital sign recognition, periodic motion detection, periodic motion estimation, repetitive motion detection / estimation, respiratory rate detection, respiratory rate estimation, respiratory pattern detection, respiratory pattern estimation, respiratory pattern recognition, heart rate detection, heart rate estimation, cardiac pattern detection, cardiac pattern estimation, cardiac pattern recognition, gesture detection, gesture estimation, gesture recognition, velocity detection, velocity estimation, object positioning, object tracking, navigation, acceleration estimation, acceleration detection, fall detection, change detection, intruder (and / or tort) detection, baby detection, baby monitoring, patient monitoring, object recognition, wireless power transmission, and / or wireless charging.

[0204] The second part of the task is to play at least one of the following sound clips: smart home tasks, smart office tasks, smart building tasks, smart factory tasks (e.g., manufacturing using machines or assembly lines), smart Internet of Things (IoT) tasks, smart system tasks, smart home operations, smart office operations, smart building operations, smart manufacturing operations (e.g., moving supplies / parts / raw materials to machines / assembly lines), IoT operations, smart system operations, turning lights on, turning lights off, controlling lighting in at least one of rooms, areas and / or places, playing sound clips, playing sound clips in at least one of rooms, areas and / or places, playing at least one sound clip of welcome, greeting, farewell, first message and / or second message related to the first part of the task, turning appliances on, turning appliances off, This may include at least one of the following: controlling electrical appliances in at least one of rooms, areas, and / or locations; turning on or off electrical systems; controlling electrical systems in at least one of rooms, areas, and / or locations; turning on or off security systems; controlling security systems in at least one of rooms, areas, and / or locations; turning on or off mechanical systems; controlling mechanical systems in at least one of rooms, areas, and / or locations; and / or controlling at least one of air conditioning systems, heating systems, ventilation systems, lighting systems, heating devices, stoves, entertainment systems, doors, fences, windows, garages, computer systems, networked devices, networked systems, home appliances, office equipment, lighting devices, robots (e.g., robotic arms), smart vehicles, smart machines, assembly lines, smart devices, Internet of Things (IoT) devices, smart home devices, and / or smart office devices.

[0205] This task involves detecting when a user returns home, when a user leaves, when a user moves from one room to another, when windows / doors / garage doors / blinds / curtains / panels / solar panels / sunshades are controlled / locked / unlocked / opened / closed / partially opened, when pets are detected, when a user does something (e.g., sleeping on the sofa, sleeping in the bedroom, running on the treadmill, cooking, sitting on the sofa, watching TV, eating in the kitchen, eating in the dining room, going up and down stairs, going out / returning, being in the bathroom), when user / pet location is monitored / detected, when detected an action is taken automatically (e.g., sending a message, notifying / reporting to someone), when a user is detected an action is taken towards the user, when lights are turned on / off / dimened, when music / radio / home entertainment systems are turned on / off, when TV / This may include turning on / off / adjusting / controlling hi-fi / set-top boxes (STBs) / home entertainment systems / smart speakers / smart devices, turning on / off / adjusting air conditioning systems, turning on / off / adjusting ventilation systems, turning on / off / adjusting heating systems, adjusting / controlling curtains / light shades, turning on / off / starting computers, turning on / off / preheating / controlling coffee machines / hot water kettles, turning on / off / preheating / controlling cookers / ovens / microwaves / other cooking appliances, checking / adjusting temperatures, checking weather forecasts, checking phone message boxes, checking emails, checking systems, controlling / adjusting systems, checking / controlling / preparing / disabling security systems / baby monitors, checking / controlling refrigerators, and reporting (e.g., via speakers such as Google Home and Amazon Echo, via web pages / email / messaging systems / notification systems).

[0206] For example, when a user arrives at home in their car, the task may automatically detect the user or vehicle approaching, open the garage door upon detection, turn on the driveway / garage lights as the user approaches the garage, and turn on the air conditioning / heater / fan. As the user enters the house, the task may automatically turn on the entrance lights, turn off the driveway / garage lights, emit a greeting message to wake the user, turn on music, turn on the radio and adjust it to the user's favorite radio news channel, open the curtains / blinds, monitor the user's mood, adjust the lighting and sound environment according to the user's mood or current / imminent events on the user's daily calendar (for example, if the user has dinner with their girlfriend in an hour, activate romantic lighting and music), warm up food the user prepared in the morning in the microwave, perform diagnostic checks on all systems in the house, check the weather forecast for tomorrow's work, check news of the user's interest, and view the user's calendar and to-do list. Check reminders, check phone answering systems, messaging systems, and emails, deliver verbal reports using dialogue systems / speech synthesis, and remind the user of their mother's birthday (e.g., using auditory tools such as speakers, hi-fi, speech synthesis, sound, voice, music, song, sound field, background sound field, and dialogue systems; using visual tools such as TVs / entertainment systems / computers / notebooks / smartpads / displays / light / color / brightness / patterns; using haptic tools / virtual reality tools / gestures / tools; using smart devices / appliances / materials / furniture / fixtures; using web tools / servers / hub devices / cloud servers / fog servers / edge servers / home networks / mesh networks; using messaging tools / notification tools / communication tools / scheduling tools / email; using user interfaces / GUIs; using scents / odors / fragrances / tastes; using nervous tools / nerve system tools; and using combinations of these); create reports and deliver reports (e.g., using reminder tools as described above).Tasks may include pre-activating the air conditioning / heater / ventilation system or pre-adjusting the temperature setting of a smart thermostat. When a user moves from the entrance to the living room, the task may include turning on the living room lights, opening the living room curtains, opening the windows, turning off the entrance lights behind the user, turning on the TV and set-top box, turning on the set-top box, setting the TV to the user's preferred channel, and adjusting fixtures according to the user's preferences and conditions / state (e.g., adjusting the lighting, selecting / playing music to create a romantic atmosphere).

[0207] Another example could be: When the user wakes up in the morning, the task could be to detect the user moving around the bedroom, open the blinds / curtains, open the windows, turn off the alarm clock, adjust the indoor temperature profile from the night temperature profile to the day temperature profile, turn on the bedroom light, turn on the bathroom light as the user approaches the bathroom, check the radio or streaming channels, play the morning news, turn on the coffee machine, preheat the water, turn off the security system, etc. When the user walks from the bedroom to the kitchen, the task could be to turn on the kitchen and hallway lights, turn off the bedroom and bathroom lights, move music / messages / reminders from the bedroom to the kitchen, turn on the kitchen TV, change the TV to the morning news channel, lower the kitchen blinds, open the kitchen window to let in fresh air, unlock the back door for the user to check the backyard, adjust the kitchen temperature setting, etc. Another example could be: When a user leaves home for work, the task could include detecting the user's departure, sending a farewell and / or a message wishing them a good day, opening and closing the garage door, turning garage and driveway lights on / off, turning them off / dimening them to save energy (only if the user fails to do so), closing / locking all windows / doors (only if the user fails to do so), turning off appliances (especially stoves, ovens, and microwaves), turning on / activating the home security system for protection against intruders, adjusting the HVAC / heating / ventilation system to an "away from home" profile to save energy, and sending alerts / reports / updates to the user's smartphone.

[0208] Movement includes: no movement, resting movement, motionless action, movement, change of location / position, deterministic movement, transient movement, falling movement, repetitive movement, periodic movement, pseudo-periodic movement, periodic / repetitive movement related to respiration, periodic / repetitive movement related to heartbeat, periodic / repetitive movement related to living organisms, periodic / repetitive movement related to machines, periodic / repetitive movement related to artificial objects, periodic / repetitive movement related to nature, complex movement related to transient and periodic elements, repetitive movement, non-deterministic movement, probabilistic movement, chaotic movement, random movement, complex movement with non-deterministic and deterministic elements, stationary random movement, pseudo-stationary random movement, periodic stationary random movement, non-stationary random movement, non-stationary random movement with periodic ACF over time. Random motion, pseudo-steady random motion over a period, random motion where instantaneous ACF has pseudo-periodic / repetitive elements over a period, mechanical motion, vehicle motion, drone motion, air-related motion, wind-related motion, weather-related motion, water-related motion, fluid-related motion, ground-related motion, changes in magnetic properties, subsurface motion, seismic motion, plant motion, animal motion, animal movement, human body motion, normal motion, abnormal motion, dangerous motion, warning motion, suspicious motion, rain, fire, flood, tsunami, explosion, collision, imminent collision, human body motion, head motion, facial motion, eye motion, oral motion, tongue motion, neck motion, finger motion, hand motion, arm motion, shoulder motion, body motion, chest motion, abdominal motion, waist motion, leg motion, foot motion, body joint motion, knee motion, elbow motion, upper body motion, lower body motion, skin motion, subcutaneous motion, subcutaneous tissue motion, This may include at least one of the following: vascular movement, venous movement, organ movement, heart movement, lung movement, stomach movement, intestinal movement, bowel movement, feeding movement, respiratory movement, facial expression, eye expression, mouth expression, voice movement, singing movement, feeding movement, gestures, hand gestures, arm gestures, keystrokes, typing strokes, user interface gestures, man-machine interaction, gait, dance movements, coordinated movements, and / or coordinated bodily movements.

[0209] Heterogeneous ICs of a Type 1 device and / or any Type 2 receiver may include low-noise amplifiers (LNAs), power amplifiers, transmit-receive switches, media access controllers, baseband radios, 2.4 GHz radios, 3.65 GHz radios, 4.9 GHz radios, 5 GHz radios, 5.9 GHz radios, sub-6 GHz radios, 60 GHz radios, sub-60 GHz radios, and / or other radios. Heterogeneous ICs may include a processor, memory communicatively coupled to the processor, and a set of instructions stored in the memory that are executed by the processor. ICs and / or any processors may include at least one of the following: general-purpose processors, special-purpose processors, microprocessors, multiprocessors, multicore processors, parallel processors, CISC processors, RISC processors, microcontrollers, central processing units (CPUs), graphical processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), embedded processors (e.g., ARM), logic circuits, other programmable logic devices, discrete logic, and / or combinations thereof. Heterogeneous ICs include broadband networks, wireless networks, mobile networks, mesh networks, cellular networks, wireless local area networks (WLAN), wide area networks (WAN), metropolitan networks (MAN), WLAN standards, WiFi, LTE, LTE-A, LTE-U, 802.11 standards, 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.11af, 802.11ah, 802.11ax, 802.11ay, mesh network standard 802.16, 3G, 3.5G, 4G, Beyond 4G, 4.5G, 6G, 7G, 8G, 9G, UMTS, 3GPP, GSM, EDGE, TDMA, FDMA, CDMA, WCDMA, TD-SCDMA, Bluetooth, and Bluetooth Low-Energy. It may support (BLE), NFC, Zigbee, WiMAX, and other wireless network protocols.

[0210] The processor may include general-purpose processors, special-purpose processors, microprocessors, microcontrollers, embedded processors, digital signal processors, central processing units, graphical processing units (GPUs), multiprocessors, multicore processors, and / or processors with graphics capabilities, and / or combinations thereof. Memory may be volatile, non-volatile, random-access memory (RAM), read-only memory (ROM), programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), hard disks, flash memory, CD-ROMs, DVD-ROMs, magnetic storage devices, optical storage devices, organic storage devices, storage systems, storage networks, network storage devices, cloud storage devices, edge storage devices, local storage devices, external storage devices, internal storage devices, or other forms of non-temporary storage media known in technology. The set of instructions corresponding to the steps of the method (machine-executable code) can be implemented directly in hardware, software, firmware, or a combination thereof. The set of instructions can be embedded, pre-loaded, loaded at boot time, loaded on the fly, loaded on demand, pre-installed, installed, and / or downloaded.

[0211] Presentations may be visual (e.g., using a combination of visuals, graphics, text, symbols, colors, shades, video, animation, sound, voice, acoustics, etc.), graphical (e.g., using a GUI, animation, video), textual (e.g., web pages with text, messages, animated text), symbolic (e.g., emojis, signs, hand gestures), or mechanical (e.g., vibration, actuator movement, tactile feedback, etc.).

[0212] Basic calculation

[0213] The computational workload associated with this method is shared among processors, Type 1 heterogeneous radio devices, Type 2 heterogeneous radio devices, local servers (e.g., hub devices), cloud servers, and other processors.

[0214] Operations, preprocessing, processing, and / or postprocessing may be applied to data (e.g., TSCI, autocorrelation, TSCI features). Operations may include preprocessing, processing, and / or postprocessing. Preprocessing, processing, and / or postprocessing may also be operations. Operations include preprocessing, processing, postprocessing, scaling, calculation of confidence coefficients, calculation of line-of-sight (LOS) quantities, and non-LOS Calculation of (NLOS) quantities, calculation of quantities including LOS and NLOS, calculation of single-link quantities (e.g., path, communication path, link between transmitting and receiving antennas), calculation of quantities including multiple links, calculation of operand functions, filtering, linear filtering, nonlinear filtering, folding, grouping, energy calculation, low-pass filtering, band-pass filtering, high-pass filtering, median filtering, rank filtering, quartile filtering, percentile filtering, finite impulse response (FIR) filtering, infinite impulse response (IIR) filtering, moving average (MA) filtering, autoregressive (AR) filtering, autoregressive moving average (ARMA) filtering, selection filtering, adaptive filtering, interpolation, decimation, subsampling, upsampling, resampling, time correction, time-based correction, phase correction, magnitude correction, phase washing, amplitude washing, matched filtering, enhancement, restoration, noise reduction, smoothing, signal tuning, enhancement, restoration, linear transformation, nonlinear transformation, inverse transformation, frequency transformation, inverse frequency transformation, Fourier transform (FT), discrete-time FT (DTFT), Discrete FT (DFT), Fast FT (FFT), Wavelet Transform, Laplace Transform, Hilbert Transform, Hadamar Transform, Trigonometric Transform, Syn Transform, Cosine Transform, DCT, Power of 2 Transform, Sparse Transform, Graph-Based Transform, Graph Signal Processing, Fast Transform, Transform combined with zero padding, Cyclic Padding, Padding, Zero Padding, Feature Extraction, Decomposition, Projection, Orthogonal Projection, Non-Orthogonal Projection, Over-projection (oecomleeObjection, eigendecomposition, singular value decomposition (SVD), principle component analysis (ICA), independent component analysis (ICA), grouping, sorting, thresholding, soft thresholding, hard thresholding, clipping, soft clipping, first derivative, second derivative, higher derivative, convolution, multiplication, partitioning, addition, subtraction, integration, maximization, minimization, least squares error, recursive least squares, constrained least squares, batch least squares, least absolute deviation, least mean squares deviation, least absolute deviation, local maximization, local minimization, cost function optimization, neural network, recognition, labeling, training, class Tagging, machine learning, supervised learning, unsupervised learning, semi-supervised learning, comparison with other TSCIs, similarity score calculation, quantization, vector quantization, matching tracking, compression, encryption, coding, storage, transmission, normalization, time normalization, frequency domain normalization, classification, clustering, labeling, tagging, learning, detection, estimation, learning network, mapping, remapping, expansion, storage, retrieval, transmission, reception, representation, joining, merging, splitting, tracking, monitoring, matched filtering, Kalman filtering, particle filtering, interpolation, extrapolation, histogram estimation This may include importance sampling, Monte Carlo sampling, compressed sensing, representation, merge, join, split, scrambling, error protection, forward error correction, do nothing, time variation processing, adjusted averaging, weighted averaging, arithmetic mean, geometric mean, harmonic mean, averaging over selected frequencies, averaging over antenna links, logical operations, substitution, combination, sorting, AND, OR, XOR, join, cross, vector addition, vector subtraction, vector multiplication, vector division, inverse, norm, distance, and / or other operations. Operations may be pre-processing, processing, and / or post-processing. Operations may be applied jointly on multiple time series or functions.

[0215] Functions (e.g., operand functions) include scalar functions, vector functions, discrete functions, continuous functions, polynomial functions, characteristics, features, magnitude, phase, exponential functions, logarithmic functions, trigonometric functions, transcendental functions, logical functions, linear functions, algebraic functions, nonlinear functions, piecewise functions, real functions, complex functions, vector-valued functions, inverse functions, derivatives, integral functions, circular functions, functions of other functions, one-to-one functions, one-to-many functions, many-to-one functions, many-to-many functions, zero crossings, absolute functions, index functions, mean, mode, median, range, statistics, histogram, variance, standard deviation, measurement of change, expansion, variance, deviation, divergence, range, interquartile range, total deviation, absolute deviation, arithmetic mean, geometric mean, harmonic mean, trimmed mean, percentile, square, cube, square root, power, sine, cosine, tangent, cotangent Elliptic functions, parabolic functions, hyperbolic functions, game functions, zeta functions, absolute value, threshold, limit function, floor function, rounding function, sign function, quantization, piecewise constant functions, composite functions, functions of functions, time functions processed by operations (e.g., filtering), probabilistic functions, inferential functions, deterministic functions, periodic functions, iterative functions, transformations, frequency transformations, inverse frequency transformations, discrete-time transformations, Laplace transforms, Hilbert transforms, sine transforms, cosine transforms, trigonometric transforms, wavelet transforms, integer transformations, powers of two transformations, sparse transformations, projections, decomposition, principal component analysis (PCA), neural networks, feature extraction, moving functions, moving window functions for adjacent items in time series, filtering functions, convolutions, mean functions, histograms, variance / standard deviation functions, statistical functions, short-time transformations, discrete transformationsDiscrete Fourier transform, Discrete cosine transform, Discrete sine transform, Hadamard transform, Eigenvalue decomposition, Eigenvalue, Singular value decomposition (SVD), Singular value, Orthogonal decomposition, Matching pursuit, Sparse transform, Arbitrary decomposition, Graph-based processing, Graph-based transform, Graph signal processing, Classification, Class / group / category identification, Labeling, Learning, Machine learning, Detection, Estimation, Feature extraction, Learning network, Feature extraction, Denoising, Signal enhancement, Encoding, Encryption, Mapping, Remapping, Vector quantization, Low-pass filtering, High-pass filtering, Band-pass filtering, Matched filtering, Kalman filtering, Preprocessing, Postprocessing, Particle filtering, FIR filtering, IIR filtering, Autoregressive (AR) filtering, Adaptive filtering, First derivative, Higher derivative, Integration, Zero crossing, Smoothing, Median filtering, Mode filtering, Sampling, Random sampling, Resampling function, Downsampling, Downconverting, Upsampling, Upconverting, Interpolation, Extrapolation, Importance sampling, Monte Carlo sampling, Compressed sensing, Statistics, This may include short-term statistics, long-term statistics, autocorrelation functions, cross-correlation, moment generating functions, time averages, weighted averages, special functions, Bessel functions, error functions, complementary error functions, beta functions, gamma functions, integral functions, Gaussian functions, Poisson functions, etc.

[0216] The steps (or each step) of this disclosure may include machine learning, training, discriminative training, deep learning, neural networks, continuous-time processing, distributed computing, distributed storage, and acceleration using GPUs / DSPs / coprocessors / multicores / multiprocessing.

[0217] Frequency conversion may include Fourier transform, Laplace transform, Hadamard transform, Hilbert transform, sine transform, cosine transform, triangular transform, wavelet transform, integer transform, power-of-two transform, zero padding and combinations of transforms, power Fourier transform with zero padding, and / or another transform. A fast version and / or an approximate version of the transform may be executed. The transform can be performed using floating point and / or fixed point arithmetic.

[0218] Inverse frequency conversion can include inverse Fourier transform, inverse Laplace transform, inverse Hadamard transform, inverse Hilbert transform, inverse sine transform, inverse cosine transform, inverse triangular transform, inverse wavelet transform, inverse integer transform, inverse power-of-two transform, zero padding and combinations of transforms, inverse Fourier transform with zero padding, and / or another transform. A fast version and / or an approximate version of the transform may be executed. The transform can be performed using floating point and / or fixed point arithmetic.

[0219] Quantities / features can be calculated from TSCI. Quantities include motion, position, map coordinates, height, velocity, acceleration, movement angle, rotation, dimensions, volume, time trend, one-time pattern, repetitive pattern, developmental pattern, time pattern, mutually exclusive patterns, association / correlation pattern, cause-and-effect, short-term / long-term correlation, trend, slope, preference, statistics, typical behavior, atypical behavior, time trend, time profile, periodic motion, repetitive motion, repetition, trend, change, rapid change, gradual change, frequency, transient, respiration, gait, behavior, event, suspicious event, dangerous event, warning event, warning, belief, proximity, collision, power, signal, signal power, signal strength, signal quantity, received signal strength index (RSSI), signal amplitude, signal phase, signal frequency components, signal frequency band components, channel state information (CSI), map, time, frequency, time-period This may include at least one statistic from among wavenumber, resolution, orthogonal resolution, non-orthogonal resolution, tracking, respiration, palpitations, statistical parameters, cardiopulmonary statistics / analysis (e.g., output response), daily activity statistics / analysis, chronic disease statistics / analysis, medical statistics / analysis, early (or instantaneous or simultaneous or delayed) indication / suggestion / sign / sign / verifier / detection / symptom / condition / state, biometrics, baby, patient, machine, device, temperature, vehicle, parking lot, place, lift, elevator, space, road, fluid flow, home, room, office, house, building, warehouse, storage, system, ventilation, fan, pipe, duct, people, human, car, boat, truck, airplane, drone, downtown, crowd, impulsive event, periodic stationary, environment, vibration, material, surface, three-dimensional, two-dimensional, local, whole, presence, and / or other measurable quantities / variables.

[0220] Sliding window / algorithm

[0221] A sliding time window may have a time-varying window width. To enable rapid acquisition, it may start smaller and increase over time to a steady-state size. The steady-state size may be related to the monitored frequency, repetitive motion, transient motion, and / or STI. Even in the steady state, the window size may be adaptively (and / or dynamically) changed (e.g., adjusted, modified, corrected) based on battery life, power consumption, available computing power, changes in the quantity being monitored, the nature of the motion being monitored, etc.

[0222] A time shift between two sliding time windows in adjacent time instances can be constant / variable / locally adaptive / dynamically adjusted over time (over time). When shorter time shifts are used, any monitoring updates may be more frequent, which can be used for rapidly changing situations, object movements, and / or objects. Longer time shifts can be used for slower situations, object movements, and / or objects.

[0223] Window width / size and / or time shift may be changed (e.g., adjusted, modified, corrected) according to user requests / selections. Time shift may be changed automatically (e.g., controlled by the processor / computer / server / hub device / cloud server) and / or adaptively (and / or dynamically).

[0224] At least one characteristic (e.g., characteristic value, or characteristic point) of a function (e.g., autocorrelation function, autocovariance function, cross-correlation function, cross-covariance function, power spectral density, time function, frequency domain function, frequency transform) can be determined (e.g., by an object tracking server, a processor, a type 1 heterogeneous device, a type 2 heterogeneous device, and / or another device). At least one property of this function may include: maximum, minimum, extremum, local maximum, local minimum, local extremum, local extremum with a positive time offset, first local extremum with a positive time offset, nth extremum with a positive time offset, first local extremum with a negative time offset, restricted maximum, restricted minimum, restricted extremum, significant maximum, significant minimum, significant extremum, gradient, derivative, higher-order derivative, maximum gradient, minimum gradient, local maximum gradient, local maximum gradient with a positive time offset, local minimum gradient, restricted maximum gradient, restricted minimum gradient, maximum higher-order derivative, minimum higher-order derivative, restricted higher-order derivative, zero crossing, zero crossing with a positive time offset, nth zero crossing with a positive time offset, zero crossing with a negative time offset, nth zero crossing with a negative time offset, restricted zero crossing, zero crossing of the gradient, zero crossing of the gradient of the higher-order derivative, and / or other properties. At least one argument of the function related to at least one property of the function may be identified. A certain quantity (for example, the spatial-temporal information of an object) can be determined based on at least one argument of a function.

[0225] Characteristics (e.g., motion characteristics of an object in a location) include instantaneous characteristics, short-term characteristics, repeating characteristics, repetitive characteristics, history, incremental characteristics, change characteristics, deviation characteristics, phase, amplitude, degree, time characteristics, frequency characteristics, time-frequency characteristics, decomposition characteristics, orthogonal decomposition characteristics, non-orthogonal decomposition characteristics, deterministic characteristics, stochastic characteristics, inferential characteristics, autocorrelation function (ACF), mean, variance, standard deviation, measurement of change, spread, variability, deviation, divergence, range, quartile range, total variation, absolute deviation, total deviation, statistics, duration, timing, trend, periodic characteristics, repeating characteristics, long-term characteristics, history characteristics, mean characteristics, latest characteristics, past characteristics, future characteristics, predictive characteristics, position, distance, height, speed, direction, velocity, acceleration, change in acceleration, angle, angular speed, angular velocity, change in angular velocity, object This includes at least one of the following: angular acceleration, change in angular acceleration, object orientation, angle of rotation, deformation of an object, shape of an object, change in shape of an object, change in size of an object, change in structure of an object, and / or change in properties of an object.

[0226] At least one local maximum and at least one local minimum of the function can be identified. At least one local signal-to-noise ratio analogue (SNR analogue) parameter may be calculated for each pair of adjacent local maximums and local minimums. The SNR analogue parameter may be a function (e.g., linear, logarithmic, exponential, monotonic) of the amount of the local maximum (e.g., power, magnitude) over the same amount of the local minimum. It may also be a function of the difference between the amount of the local maximum and the same amount of the local minimum. Significant local peaks can be identified or selected. Each significant local peak may be a local maximum with an SNR analogue parameter greater than threshold T1, and / or a local maximum with an amplitude greater than threshold T2. At least one local minimum and at least one local minimum in the frequency domain can be identified / calculated using a persistence-based approach.

[0227] A set of selected significant local peaks may be chosen from a set of identified significant local peaks based on selection criteria (e.g., quality criteria, signal quality status). The object's properties / STI may be calculated based on the selected set of significant local peaks and the frequency values ​​associated with that set. In one example, the selection criteria may always correspond to selecting the strongest peak within a range. The strongest peak may be selected, but the unselected peaks may still be significant (or even strong).

[0228] Significant unselected peaks may be saved and / or monitored as “reserved” peaks for use in future selections within future sliding time windows. For example, there may be a particular peak (at a specific frequency) that appears consistently over time. Initially, it may not be selected even if significant (because other peaks may become stronger). However, later in time, the peak may become stronger, more dominant, and thus selected. If it becomes “selected,” it can be traced back in time and considered “selected” at an earlier point when it was significant but not selected. In such cases, the traced peak may replace the previously selected peak at an earlier point. The replaced peak may be a relatively weak peak or a peak that appears temporally isolated (i.e., only appears for a short time).

[0229] In other examples, the selection criteria may not correspond to selecting the strongest peak within the range. Instead, in addition to the "strength" of the peak, the "trace" of peaks that may have occurred in the past, especially those that have been identified for a long time, may also be considered.

[0230] For example, if a finite state machine (FSM) is used, it can select peaks based on the state of the FSM. The decision threshold can be calculated adaptively (and / or dynamically) based on the state of the FSM.

[0231] Similarity scores and / or component similarity scores may be calculated based on a pair of temporally adjacent CIs of a TSCI (e.g., by a server (e.g., a hub device), a processor, a Type 1 device, a Type 2 device, a local server, a cloud server, and / or another device). The pair may be obtained from the same sliding window or from two different sliding windows. Similarity scores may also be based on a pair of temporally adjacent or not-so-adjacent CIs from two different TSCIs. The similarity score and / or component similarity score may be / include the following: time reversal resonating strength (TRRS), correlation, cross-correlation, autocorrelation, correlation indicator, covariance, cross-covariance, autocovariance, dot product of two vectors, distance score, norm, metric, quality metric, signal quality condition, statistical properties, discrimination score, neural network, deep learning network, machine learning, training, discrimination, weighted average, preprocessing, denoising, signal conditioning, filtering, time correction, time adjustment, phase offset compensation, transformation, component-by-component operations, feature extraction, finite state machine, and / or other scores. Properties and / or STI may be determined / calculated based on the similarity score.

[0232] Any threshold can be predetermined, adaptively (and / or dynamically), and / or determined by a finite state machine. Adaptive determination may be based on time, space, location, antenna, path, link, state, battery life, battery level, available power, available computing resources, available network bandwidth, etc.

[0233] Thresholds applied to test statistics to distinguish between two events (or two conditions, or two situations, or two states), A and B, can be determined. Data (e.g., CI, channel status information (CSI), power parameters) can be collected under A and / or B in a training scenario. Test statistics can be calculated based on the data. The distribution of test statistics under A can be compared to the distribution of test statistics under B (reference distribution), and thresholds can be selected according to several criteria. Decision criteria may include maximum likelihood estimation (ML), maximum posterior probability (MAP), discrimination training, minimum type 1 error for a given type 2 error, minimum type 2 error for a given type 1 error, and / or other decision criteria (e.g., quality criteria, signal quality conditions). Thresholds can be adjusted to achieve different sensitivities to A, B, and / or other events / conditions / scenarios / states. Threshold adjustment may be automatic, semi-automatic, and / or manual. Threshold adjustments can be applied once, sometimes, often, periodically, repeatedly, occasionally, sporadically, and / or on demand. Threshold adjustments can be adaptive (and / or dynamically adjusted). Threshold adjustments may depend on objects, object movement / position / orientation / action, object properties / STI / size / characteristics / traits / habits / behavior, location, in / at / of / in / location, features / fixtures / furniture / barriers / materials / machines / living things / objects / boundaries / surfaces / mediums, maps, map constraints (or environment models), events / states / scenes / conditions, time, timing, duration, current state, past history, users, and / or personal preferences, etc.

[0234] The stopping criterion (or criterion for skipping, bypassing, blocking, pausing, passing, or rejecting) of an iterative algorithm may be that the change in the current parameter (e.g., offset value) in the update during an iteration is less than a threshold. The threshold may be 0.5, 1, 1.5, 2, or another number. The threshold may be adaptive (and / or dynamically adjusted). It may change as the iteration progresses. With respect to the offset value, the adaptive threshold may be determined based on the task, a specific initial value, the current time offset value, the regression window, the regression analysis, the regression function, the regression error, the convexity of the regression function, and / or the number of iterations.

[0235] Local extrema can be determined as the corresponding extrema of the regression function within the regression window. Local extrema can be determined based on a set of time offset values ​​and an associated set of regression function values ​​within the regression window. Each of the associated set of regression function values ​​associated with a set of time offset values ​​may be within the range of the corresponding extrema of the regression function within the regression window.

[0236] Searching for local extrema involves robust search, minimization, optimization, statistical optimization, dual optimization, constraint optimization, convex optimization, global optimization, local optimization, energy minimization, linear regression, quadratic regression, higher-order regression, linear programming, nonlinear programming, stochastic programming, combinatorial optimization, constraint programming, constraint satisfaction, calculation of variations, optimal control, dynamic programming, mathematical programming, multi-objective optimization, multimodal optimization, disjunctive programming, spatial mapping, infinite-dimensional optimization, heuristics, metaheuristics, convex programming, semidefinite programming problems, cone programming, quadratic cone programming problems, integer programming, quadratic programming, fractional programming, numerical analysis, simplex algorithms, iterative methods, gradient descent, subgradient methods, coordinate gradient methods, conjugate gradient methods, Newton's algorithm, successive quadratic programming, interior point methods, elliptic methods, shrinking gradient methods, quasi-Newton's method, simultaneous perturbation stochastic approximation, interpolation methods, pattern search methods, line search, non-differential optimization, genetic algorithms, evolutionary algorithms, and dynamic relaxation. This may include hill climbing, particle swarm optimization, gravity search algorithms, simulated annealing, memetic algorithms, differential evolution, dynamic relaxation, stochastic tunneling, tab search, reaction search optimization, curve fitting, least squares, simulation-based optimization, variational calculation, and / or deformation. The search for local extrema may be related to the objective function, loss function, cost function, utility function, fitness function, energy function, and / or energy function.

[0237] Regression can be performed using a regression function to fit sampled data (e.g., CI, CI features, CI components) or another function (e.g., an autocorrelation function) within a regression window. The length and / or position of the regression window may change in at least one iteration. The regression function can be a linear function, a quadratic function, a cubic function, a polynomial function, and / or another function.

[0238] Regression analysis can minimize at least one of the following: error, aggregate error, component error, error in the projection region, error on a selected axis, error on a selected orthogonal axis, absolute error, squared error, absolute deviation, squared deviation, higher-order errors (e.g., cubic, quaternary), robust errors (e.g., squared error for absolute error for smaller errors and for larger errors, or first type of error for smaller errors and second type of error for larger errors), another error, a weighted sum (or weighted average) of absolute / squared errors (e.g., in the case of a radio transmitter with multiple antennas and a radio receiver with multiple antennas, each pair of transmitter and receiver antennas forms a link), mean absolute error, mean squared error, mean absolute deviation, and / or mean squared deviation. Errors associated with different links may have different weights. One possibility is that some links and / or some components with higher noise or lower signal quality metrics may have smaller or larger weights (weighted sum of squared errors, weighted sum of higher-order errors, weighted sum of robust errors, weighted sum of other errors, absolute cost, squared cost, higher-order cost, robust cost, other cost, weighted sum of absolute cost, weighted sum of squared cost, weighted sum of higher-order cost, weighted sum of robust cost, and / or weighted sum of other costs).

[0239] The regression error determined may be an absolute error, a squared error, a higher-order error, a robust error, yet another error, a weighted sum of absolute errors, a weighted sum of squared errors, a weighted sum of higher-order errors, a weighted sum of robust errors, and / or a weighted sum of yet another error.

[0240] The time offset associated with the maximum (or minimum) regression error of a regression function for a particular function within a regression window can be the updated current time offset in an iteration.

[0241] Local extrema can be found based on a quantity that includes the difference between two different errors (e.g., the difference between the absolute error and the squared error). Each of the two different errors may include the absolute error, the squared error, a higher-order error, a robust error, another error, a weighted sum of absolute errors, a weighted sum of squared errors, a weighted sum of higher-order errors, a weighted sum of robust errors, and / or a weighted sum of another error.

[0242] The quantity may be compared to reference data or reference distributions such as the F-distribution, the central F-distribution, another statistical distribution, thresholds, thresholds related to probability / histograms, thresholds related to probability / histograms for finding false peaks, thresholds related to the F-distribution, thresholds related to the central F-distribution, and / or thresholds related to another statistical distribution.

[0243] The regression window may be determined based on at least one of the following: the movement of an object (e.g., change of location / position), a quantity associated with the object, at least one characteristic of the object and / or STI related to the movement of the object, the estimated location of local extrema, noise characteristics, estimated noise characteristics, signal quality metric, F-distribution, central F-distribution, another statistical distribution, threshold, preset threshold, threshold related to probability / histogram, threshold related to desired probability, threshold related to the probability of finding a false peak, threshold related to F-distribution, threshold related to central F-distribution, threshold related to another statistical distribution, the condition that the quantity at the center of the window is maximum within the regression window, the condition that for the first time within the regression window there is only one of the local extrema of a particular function for a particular value, another regression window, and / or other conditions.

[0244] The width of the regression window may be determined based on the specific local extrema being searched for. Local extrema include the first local maximum, second local maximum, higher-order local maximum, first local maximum with a positive time offset, second local maximum with a positive time offset value, higher-order local maximum with a positive time offset value, first local maximum with a negative time offset value, second local maximum with a negative time offset value, second local maximum with a negative time offset value, higher-order local maximum with a negative time offset value, first local minimum, second local minimum, higher-order local minimum, first local minimum with a positive time offset value, This may include a second local minimum with a positive time offset value, a higher-order local minimum with a positive time offset value, a first local minimum with a negative time offset value, a second local minimum with a negative time offset value, a higher-order local minimum with a negative time offset value, a first local extremum, a second local extremum, a higher-order local extremum, a first local extremum with a positive time offset value, a second local extremum with a positive time offset value, a first local extremum with a negative time offset value, a second local extremum with a negative time offset value, and a higher-order extremum with a negative and / or negative time offset value.

[0245] The current parameter (e.g., time offset value) may be initialized based on the target value, target profile, trend, historical trend, current trend, target velocity, velocity profile, target velocity profile, historical velocity trend, motion or movement of the object (e.g., change of location / position), at least one characteristic and / or STI of the object related to the object's motion, position quantity of the object, initial velocity of the object related to the object's motion, a predefined value, initial width of the regression window, duration, value based on the signal's carrier frequency, value based on the signal's subcarrier frequency, signal bandwidth, total antenna value related to the channel, noise characteristics, signal h metric, and / or adaptive (and / or dynamically adjusted) value. The current time offset may be at the center, left, right, and / or another fixed relative position of the regression window.

[0246] In the presentation, information may be displayed along with a map of the location (or environmental model). Information may include location, zone, region, area, coverage region, corrected location, approximate location, location on the map of the location (WRT), location on segmented locations, direction, path, path on map and / or segmentation, trace (e.g., location within a time window such as the last 5 seconds or the last 10 seconds, the time window duration can be adjusted adaptively (and / or dynamically), and the time window duration can be adjusted adaptively (and / or dynamically) with respect to velocity and acceleration), path history, approximate region / zone along the path, history / summary of past locations, history of past locations of interest, frequently visited regions, customer traffic, flock distribution, flock behavior, flock control information, velocity, acceleration, motion statistics, respiratory rate, heart rate, presence / absence of motion, Presence or absence of people, pets or objects, presence or absence of vital signs, gesture control (control of devices using gestures), location-based gesture control, location-based operation information, identity (ID) or identifier of object of interest (e.g., pet, person, self-guided machine / device, vehicle, drone, car, boat, bicycle, unmanned vehicle, machine with fan, air conditioner, TV, machine with moving parts), user identification (e.g., person), user information, location / velocity / acceleration / direction / movement / gesture / gesture control / motion tracing, user ID or identifier, user activity, user state, user sleep / rest characteristics, user emotional state, user vital signs, location This may include environmental information, location weather information, earthquakes, explosions, storms, rain, fires, temperature, collisions, impacts, vibrations, events, door opening events, door closing events, window opening events, window closing events, fall events, combustion events, freezing events, water-related events, wind-related events, air movement events, accident events, pseudo-periodic events (e.g., running on a treadmill, jumping, jump rope, somersaults, etc.), recurring events, crowding events, vehicle events, and user gestures (e.g., hand gestures, arm gestures, foot gestures, body gestures, head gestures, face gestures, mouth gestures, eye gestures, etc.).

[0247] Location can be two-dimensional (e.g., using two-dimensional coordinates) or three-dimensional (e.g., using three-dimensional coordinates). Location can be relative (e.g., midway between point A and point B, around a corner, on an upper floor, on a table, on the ceiling, on the floor, on a sofa, close to point A, at a distance R from point A, within a radius of R from point A, etc.). Location can be expressed in Cartesian coordinates, polar coordinates, and / or other representations.

[0248] Information (e.g., location) may be marked with at least one symbol. The symbol may change over time. The symbol may blink and / or pulsate, or change color / intensity, or not. Its size may change over time. The orientation of the symbol may change over time. The symbol may be a number that reflects instantaneous quantities (e.g., user's vital signs / respiratory rate / heart rate / gestures / situation / state / action / movement, temperature, network traffic, network connectivity, device / machine status, remaining power of device, device status, etc.). The rate of change, size, orientation, color, intensity and / or symbol may reflect their respective movements. The information may be presented visually and / or described orally (e.g., using pre-recorded speech or speech synthesis). The information may be written in text. The information may also be presented in a mechanical way (e.g., animated gadgets, movement of moving parts).

[0249] User interface (UI) devices may include smartphones (e.g., iPhones, Android phones), tablets (e.g., iPads), laptops (e.g., notebook computers), personal computers (PCs), devices with a graphical user interface (GUI), smart speakers, devices with voice / sound / speaker capabilities, virtual reality (VR) devices, augmented reality (AR) devices, smart cars, in-car displays, voice assistants, in-car voice assistants, and the like.

[0250] Maps (or environmental models) may be two-dimensional, three-dimensional, and / or higher-dimensional (e.g., time-varying two-dimensional / three-dimensional maps / environmental models). Walls, windows, doors, entrances, exits, and restricted areas may be marked on the map or model. Maps may include floor plans of facilities. Maps or models may have one or more layers (overlays). Maps / models may be maintenance maps / models including water pipes, gas pipes, wiring, cable wiring, air ducts, crawl spaces, ceiling layouts, and / or basement layouts. Locations can be divided into multiple zones / areas / geographic areas / sectors / sections / territories / districts / administrative areas / sites / neighborhoods / areas / stretches / spacious areas, such as bedrooms, living rooms, storage rooms, walkways, kitchens, dining rooms, foyers, garages, ground floors, second floors, restrooms, offices, conference rooms, reception areas, various office areas, various warehouse areas, and various facility areas. Segments / areas / areas can be presented in the map / model. Different areas may be color-coded. Different areas may be presented with characteristics (e.g., color, brightness, color intensity, texture, animation, blinking, blinking speed, etc.). Logical segmentation of locations can be performed using at least one heterogeneous type 2 device or server (e.g., a hub device) or a cloud server.

[0251] Herein is an example of the disclosed system, apparatus, and method. Stephen and his family want to install the disclosed wireless motion detection system to detect movement within their 2,000-square-foot, two-story townhouse in Seattle, Washington. As his house is two stories tall, Stephen decided to use one Type 2 device (named A) and two Type 1 devices (named B and C) on the first floor. The first floor is centered around three rooms: the kitchen, dining room, and living room, arranged in a straight line with the dining room in the middle. The kitchen and living room are on opposite sides of the house. He placed one Type 2 device (A) in the dining room, one Type 1 device (B) in the kitchen, and another Type 1 device (C) in the living room. With the installation of this apparatus, he specifically partitions the first floor into three zones (dining room, living room, and kitchen) using the motion detection system. When movement is detected by the A / B pair and the A / C pair, the system analyzes the motion information and associates the movement with one of the three zones.

[0252] When Stefan and his family go out on weekends (for example, when they go camping on a long weekend), Stefan turns on a motion detection system using a mobile phone app (e.g., an Android phone app or an iPhone app). When the system detects motion, an alert signal is sent to Stefan (e.g., SMS text message, email, push message to the mobile phone app, etc.). If Stefan pays a monthly fee (e.g., $10 / month), the service company (e.g., a security company) receives the alert signal via a wired network (e.g., broadband) or wireless network (e.g., home WiFi, LTE, 3G, 2.5G, etc.) and takes security actions for Stefan (e.g., calling him to confirm the problem, sending someone to check the house, contacting the police on Stefan's behalf, etc.). Stefan loves his elderly mother very much and cares about her well-being when she is home alone. When his mother is home alone while the rest of the family is out (for example, going to work, shopping, or going on vacation), Stefan uses his mobile app to turn on a motion detection system to ensure she is okay. He then uses the mobile app to monitor his mother's movements around the house. When Stefan uses the mobile app to see his mother moving around the house between three areas, he knows she is okay, according to her daily routine. Stefan appreciates that the motion detection system can help him monitor his mother's well-being while he is away from home.

[0253] On a typical day, his mother would wake up around 7 a.m. She would spend about 20 minutes in the kitchen preparing breakfast. Then she would eat breakfast in the dining room for about 30 minutes. After that, she would sit on the sofa in the living room and do her daily exercises before watching her favorite TV shows. The motion detection system allows Stefan to see the timing of movements in each of the three areas of the house. When the movements match the daily routine, Stefan has a rough idea that his mother is doing well. However, if the movement pattern looks unusual (for example, no exercise until 10 a.m., staying in the kitchen for too long, or remaining motionless for too long), Stefan suspects something is wrong and calls his mother to check on her. Stefan may even ask someone else (for example, a family member, neighbor, paid staff member, friend, social worker, or service provider) to check on his mother.

[0254] Sometimes, Stefan feels like he's just relocating a Type 2 device. He simply unplugs the device from its original AC power outlet and plugs it into a different one. He's pleased that the wireless motion detection system is plug-and-play and that relocation doesn't affect the system's operation. Once powered on, it works immediately.

[0255] On another occasion, Stefan was convinced that our wireless motion detection system could actually detect movement with very high accuracy and very low alarms, and that he could actually monitor movement on the first floor using the mobile app. He decided to install a similar configuration on the second floor (i.e., one Type 2 device and two Type 1 devices) to monitor the bedrooms on the second floor. Again, he found the system setup to be extremely easy, as it only required plugging the Type 2 and Type 1 devices into the AC power outlets on the second floor. No special installation was needed. And he could monitor movement on both the first and second floors using the same mobile app. Each Type 2 device on the first / second floor could interact with all the Type 1 devices on both the first and second floors. Stefan was happy to see that as he doubled his investment in Type 1 and Type 2 devices, the combined system had more than twice the capacity.

[0256] According to various embodiments, each CI (CI) may include at least one of the following: channel state information (CSI), frequency domain CSI, frequency representation of CSI, frequency domain CSI associated with at least one subband, time domain CSI, intradomain CSI, channel response, channel response estimate, channel impulse response (CIR), channel frequency response (CFR), channel characteristics, channel filter response, CSI of a radio multipath channel, radio multipath channel information, timestamp, auxiliary information, data, metadata, user data, account data, access data, security data, session data, status data, management data, family data, identity (ID), identifier, device data, network data, neighbor data, environmental data, real-time data, sensor data, stored data, encrypted data, compressed data, protected data, and / or another CI. In one embodiment, the disclosed system has hardware components (e.g., a radio transmitter / receiver with an antenna, analog circuits, a power supply, a processor, memory) and corresponding software components. According to various embodiments of this disclosure, the disclosed system includes a Bot (referred to as a Type 1 device) and an Origin (referred to as a Type 2 device) for vital sign detection and monitoring. Each device comprises a transceiver, a processor, and memory.

[0257] The disclosed system can be applied in many cases. For example, a Type 1 device (transmitter) may be a small WiFi-enabled device placed on a table. It may also be a WiFi-enabled television (TV), set-top box (STB), smart speaker (e.g., Amazon Echo), smart refrigerator, smart microwave, mesh network router, mesh network satellite, smartphone, computer, tablet, smart plug, etc. For example, a Type 2 (receiver) may be a WiFi-enabled device placed on a table. It may also be a WiFi-enabled television (TV), set-top box (STB), smart speaker (e.g., Amazon Echo), smart refrigerator, smart microwave, mesh network router, mesh network satellite, smartphone, computer, tablet, smart plug, etc. Type 1 and Type 2 devices may be placed in / near a conference room to count people. Type 1 and Type 2 devices may be used in a health monitoring system for the elderly to monitor daily activities and any signs of symptoms (e.g., dementia, Alzheimer's disease). Type 1 and Type 2 devices may be used in an infant monitor to monitor the vital signs (respiration) of a living infant. Type 1 and Type 2 devices may be placed in bedrooms to monitor sleep quality and any sleep apnea. Type 1 and Type 2 devices may be placed in vehicles to monitor the health of passengers and drivers, detect driver sleep, and detect any babies left inside the vehicle. Type 1 and Type 2 devices may be used in logistics to prevent human trafficking by monitoring people hidden in trucks and containers. Type 1 and Type 2 devices may be deployed by emergency services in disaster areas to search for trapped victims in rubble. Type 1 and Type 2 devices may be placed in an area to detect the breathing of any intruders. There are numerous applications for wireless respiratory monitoring that is not wearable.

[0258] The hardware module may be configured to include a Type 1 transceiver and / or a Type 2 transceiver. The hardware module may be sold / used under a variable brand for designing, building, and selling the final commercial product. Products using the disclosed systems and / or methods may include home / office security products, sleep monitoring products, WiFi products, mesh products, TVs, STBs, entertainment systems, HiFi, speakers, home appliances, lamps, stoves, ovens, microwave ovens, tables, chairs, beds, shelves, tools, appliances, torches, vacuum cleaners, smoke detectors, sofas, pianos, fans, doors, windows, door / window handles, locks, smoke detection devices, car accessories, computing devices, office supplies, air conditioners, heaters, pipes, connectors, surveillance cameras, access points, computer equipment, mobile devices, LTE devices, 3G / 4G / 5G / 6G devices, UMTS devices, 3GPP devices, GSM devices, EDGE devices, TDMA devices, FDMA devices, CDMA devices, WCDMA devices, TD-SCDMA devices, game devices, glasses, glass panels, VR goggles, necklaces, watches, waistbands, belts, wallets, pens, hats, clothing, implantable devices, tags, parking tickets, smartphones, etc.

[0259] The summary may include: analysis, output response, selected time window, subsampling, transformation, and / or projection. The presentation may include presenting at least one of the following: monthly / weekly / daily views, simplified / detailed views, cross-sectional views, small / large form factor views, color-coded views, comparison views, summary views, videos, web views, audio announcements, and other presentations relating to the periodic / repetitive characteristics of repeating motion.

[0260] Type 1 / Type 2 devices include antennas, devices with antennas, devices with housings (e.g., radios, antennas, data / signal processing units, wireless ICs, circuits), devices interfaced / attached / connected / linked to another device / system / computer / telephone / network / data aggregator, devices with user interfaces (UI) / graphical UI / displays, devices with wireless transceivers, devices with wireless transmitters, devices with wireless receivers, Internet of Things (IoT) devices, devices with wireless networks, devices with both wired and wireless network functions, devices with wireless integrated circuits (ICs), Wi-Fi devices, devices with Wi-Fi chips (e.g., compliant with 802.11a / b / g / n / ac / ax standards), Wi-Fi access points (APs), Wi-Fi clients, Wi-Fi routers, Wi-Fi repeaters, Wi-Fi hubs, Wi-Fi mesh network routers / hubs / APs, wireless mesh network routers, ad-hoc network devices, wireless mesh network devices, and mobile devices (e.g., 2G / 2.5G / 3G / 3G).5G / 4G / LTE / 5G / 6G / 7G, UMTS, 3GPP, GSM, EDGE, TDMA, FDMA, CDMA, WCDMA, TD-SCDMA), cellular devices, base stations, mobile network base stations, mobile network hubs, mobile network compatible devices, LTE devices, devices with LTE modules, mobile modules (e.g., circuit boards with mobile enable chips (ICs) such as Wi-Fi chips, LTE chips, BLE chips), devices with mobile modules, smartphones, companion devices for smartphones (e.g., dongles, attachments, plug-ins), dedicated devices, plug-in devices, AC-powered devices, battery-powered devices, devices with processor / memory / instruction sets, smart devices / gadgets / Items: may be clocks, stationery, pens, user interfaces, paper, mats, cameras, televisions (TVs), set-top boxes, microphones, speakers, refrigerators, ovens, machines, telephones, wallets, furniture, doors, windows, ceilings, floors, walls, tables, chairs, beds, nightstands, air conditioners, heaters, pipes, ducts, cables, carpets, decorations, gadgets, USB devices, plugs, dongles, lamps / lights, tiles, ornaments, bottles, vehicles, automobiles, AGVs, drones, robots, laptops, tablets, computers, hard drives, network cards, equipment, rackets, balls, shoes, wearable devices, clothing, eyeglasses, hats, necklaces, food, pills, small devices that move within the body of a living organism (e.g., blood vessels, lymphatic fluid, digestive system), and / or other devices. Type 1 devices and / or Type 2 devices may be communicatively coupled to the Internet, another device that accesses the Internet (e.g., a smartphone), a cloud server (e.g., a hub device), an edge server, a local server, and / or storage. Type 1 and / or Type 2 devices may operate under local control, be controlled by another device via a wired / wireless connection, operate automatically, or be controlled by a central system located remotely (e.g., away from home).

[0261] In one embodiment, a Type B device may be a transceiver that may function as both an origin (Type 2 device, Rx device) and a bot (Type 1 device, Tx device), i.e., a Type B device may be both a Type 1 (Tx) device and a Type 2 (Rx) device (e.g., simultaneously or alternately), and may be, for example, a mesh device, a mesh router, etc. In one embodiment, a Type A device may be a transceiver that may function only as a bot (Tx device), i.e., it may be only a Type 1 device or only a Tx, and may be, for example, a simple IoT device. This may have the functionality of an origin (Type 2 device, Rx device), but in embodiments it functions only as a bot in some way. All Type A and Type B devices form a tree structure. The root may be a Type B device that can access a network (e.g., the Internet). For example, this may be connected to a broadband service via a wired connection (e.g., Ethernet, cable modem, ADSL / HDSL modem) or a wireless connection (e.g., LTE, 3G / 4G / 5G, WiFi, Bluetooth, microwave link, satellite link, etc.). In one embodiment, all Type A devices are leaf nodes. Each Type B device may be a root node, a non-leaf node, or a leaf node.

[0262] Type 1 devices (transmitters, or Tx) and Type 2 devices (receivers, or Rx) may reside on the same device (e.g., an RF chip / IC), or simply on the same device. The devices can operate in high-frequency bands such as 28 GHz, 60 GHz, and 77 GHz. An RF chip may have dedicated Tx antennas (e.g., 32 antennas) and dedicated Rx antennas (e.g., another 32 antennas).

[0263] A single transmitting antenna can transmit a radio signal (e.g., a series of probe signals, perhaps at 100 Hz). Alternatively, all Tx antennas can be used to transmit a radio signal with beamforming (in Tx), resulting in the radio signal being focused in a specific direction (e.g., for energy efficiency, to boost the signal-to-noise ratio in that direction, or for low-power operation when "scanning" in that direction, or when an object is known to be in that direction).

[0264] The radio signal encounters objects within a location (e.g., a room) (e.g., a living person lying on a bed 4 feet away from the Tx / Rx antenna, breathing, and heartbeat). Object movement (e.g., pulmonary movement according to respiratory rate, or vascular movement according to heartbeat) can affect / modulate the radio signal. All Rx antennas can be used to receive the radio signal.

[0265] Beamforming (in Rx and / or Tx) may be applied (digitally) to "scan" different directions. Many directions may be scanned or monitored simultaneously. Along with beamforming, "sectors" (e.g., direction, orientation, bearing, zone, region, segment) may be defined in relation to a Type 2 device (e.g., with respect to the center position of the antenna array). For each probe signal (e.g., pulse, ACK, control packet, etc.), channel information or CI (e.g., channel impulse response / CIR, CSI, CFR) is acquired / calculated for each sector (e.g., from an RF chip). In breath detection, CIR can be collected in a sliding window (e.g., 30 seconds, with a 100 Hz pulsating / probing speed, there may be 3000 CIRs over 30 seconds).

[0266] A CIR can have many taps (e.g., N1 components / tap). Each tap may be associated with a time lag or time-of-fright (e.g., the time it takes to hit a person 4 feet away and back). When breathing in a certain direction at a certain distance (e.g., 4 feet), one can look up the CIR in "a certain direction" and then look up the tap corresponding to "a certain distance". Then, from that tap in the CIR, the breathing rate and heart rate can be calculated.

[0267] Each tap within the sliding window (for example, the 30-second window in "Component Time Series") can be considered a time function (for example, "Tap Function," "Component Time Series"). When searching for strong periodic behavior (for example, perhaps corresponding to respiration in the range of 10 bpm to 40 bpm), each tap function can be examined.

[0268] Type 1 devices and / or Type 2 devices may have external connections / links and / or internal connections / links. External connections (e.g., connection 1110) may be associated with 2G / 2.5G / 3G / 3.5G / 4G / LTE / 5G / 6G / 7G / NBIoT, UWB, WiMAX, Zigbee, 802.16, etc. Internal connectivity (e.g., 1114A and 1114B, 1116, 1118, 1120) can be associated with WiFi, IEEE 802.11 standards, 802.11a / b / g / n / ac / ad / af / ag / ah / ai / aj / aq / ax / ay, Bluetooth 1.0 / 1.1 / 1.2 / 2.0 / 2.1 / 3.0 / 4.0 / 4.1 / 4.2 / 5, BLE, mesh networks, and IEEE 802.16 / 1 / 1a / 1b / 2 / 2a / a / b / c / d / e / f / g / h / i / j / k / l / m / n / o / p / standards.

[0269] Type 1 devices and / or Type 2 devices are powered by batteries (e.g., AA batteries, AAA batteries, coin cell batteries, button cell batteries, miniature batteries, battery banks, power banks, car batteries, hybrid batteries, vehicle batteries, container batteries, non-rechargeable batteries, rechargeable batteries, NiCd batteries, NiMH batteries, lithium-ion batteries, zinc-carbon batteries, zinc chloride batteries, lead-acid batteries, alkaline batteries, batteries with wireless chargers, smart batteries, solar batteries, boat batteries, plain batteries, other batteries, temporary energy storage devices, capacitors, flywheels).

[0270] Any device may be powered by DC or direct current (for example, from batteries, generators, power converters, solar panels, rectifiers, DC-DC converters as described above, with various voltages such as 1.2V, 1.5V, 3V, 5V, 6V, 9V, 12V, 24V, 40V, 42V, 48V, 110V, 220V, 380V, etc.), and therefore may have a DC connector or connector having at least one pin for DC power.

[0271] Any device may be powered by AC or alternating current (e.g., household wall outlets, transformers, inverters, Shore Power, with various voltages such as 100V, 110V, 120V, 100-127V, 200V, 220V, 230V, 240V, 220-240V, 100-240V, 250V, 380V, 50Hz, 60Hz, etc.) and therefore may have an AC connector or connector having at least one pin for AC power. Type 1 devices and / or Type 2 devices may be located in or out of place (e.g., installed, positioned, moved).

[0272] For example, in a vehicle (e.g., automobile, truck, lorry, bus, special vehicle, tractor, excavator, teleporter, bulldozer, crane, forklift, electric vehicle, AGV, emergency vehicle, cargo, freight car, trailer, container, boat, ferry, ship, submarine, aircraft, airship, lift, monorail, train, electric train, railway vehicle, track vehicle, etc.), Type 1 devices and / or Type 2 devices may be embedded devices embedded in the vehicle, or add-on devices (e.g., aftermarket devices) plugged into ports inside the vehicle (e.g., OBD port / socket, USB port / socket, accessory port / socket, 12V auxiliary power outlet, and / or 12V cigarette lighter port / socket).

[0273] For example, one device (e.g., a Type 2 device) may be plugged into a 12V cigarette lighter / accessory port or an OBD port or USB port (e.g., of a car / truck / vehicle), and the other device (e.g., a Type 1 device) may be plugged into a 12V cigarette lighter / accessory port or an OBD port or USB port. The OBD port and / or USB port can provide power, signaling, and / or networking (of the car / truck / vehicle). The two devices can jointly monitor passengers, including children / babies, in the vehicle. They may be used to count passengers, recognize the driver, and detect the presence of passengers in specific seats / positions within the vehicle.

[0274] In another embodiment, one device may be plugged into a 12V cigarette lighter / accessory port or OBD port or a USB port in a car / truck / vehicle, while the other device may be plugged into a 12V cigarette lighter / accessory port or OBD port or another USB port in a car / truck / vehicle.

[0275] In another example, many devices of the same Type A (e.g., Type 1 or Type 2) may exist in many different types of vehicles / portable devices / smart gadgets (e.g., automated guided vehicles / AGVs, shopping / luggage / mobile carts, parking tickets, golf carts, bicycles, smartphones, tablets, cameras, recording devices, smartwatches, roller skates, shoes, jackets, goggles, hats, eyewear, wearables, Segways, scooters, luggage tags, cleaning machines, vacuum cleaners, pet tags / colors / wearables / implants), each device may be plugged into a vehicle's 12V accessory port / OBD port / USB port or embedded in the vehicle. There may also be one or more other Type B devices (e.g., if A is Type 2, then B is Type 1, and if A is Type 1, then B is Type 2) installed in locations such as gas stations, streetlights, street corners, tunnels, multi-story parking lots, and scatter locations covering large areas such as factories / stadiums / stations / shopping malls / construction sites. Type A devices can be positioned, tracked, or monitored based on TSCI.

[0276] The area / location does not need to have local connectivity such as broadband service or Wi-Fi. Type 1 and / or Type 2 devices may be portable. Type 1 and / or Type 2 devices may support plug-and-play.

[0277] A pairwise radio link can be established between many pairs of devices, forming a tree structure. In each pair (and associated link), the device (second device) may be a non-leaf (Type B) device. The other device (first device) may be a leaf (Type A or Type B) or a non-leaf (Type B) device. In the link, the first device acts as a bot (Type 1 device or transmitting device) for transmitting radio signals (e.g., probe signals) to the second device via a radio multipath channel. The second device may act as an origin (Type 2 device or Rx device) for receiving radio signals, obtaining TSCI, and calculating "analysis about the link" based on the TSCI.

[0278] In some embodiments, this disclosure discloses a WiFi-based passive fall detection system (hereinafter referred to as "DeFall," or "DeFall") that can operate independently of its environment and without prior training in a new environment. Unlike previous studies, the disclosed DeFall system can examine physiological features inherently related to human falls, namely characteristic patterns of speed and acceleration during a fall. In some embodiments, the DeFall system comprises an offline template generation stage and an online decision stage, both of which take speed estimates as input. In the offline stage, an augmented dynamic time stretching (DTW) algorithm is executed to generate a typical template of speed and acceleration patterns for a typical human fall. In the online stage, the system can compare the real-time speed / acceleration estimate pattern to the template in order to detect a fall. To evaluate the performance of DeFall, prototypes may be built using commercially available WiFi devices and experiments may be conducted in various settings. In some embodiments, evaluation results demonstrated that DeFall achieves a detection rate of 96% with a false alarm rate of less than 1.50% under both line-of-sight (LOS) and non-LOS (NLOS) scenarios with a single link measurement.

[0279] Figure 1 shows an exemplary wireless indoor rich scatter environment 100 according to several embodiments of the present disclosure. As shown in Figure 1, in the wireless indoor rich scatter environment 100, a transmitter 110 (which may be a bot) transmits a wireless signal to a receiver 120 (which may be an origin) via a multipath channel affected by various scatters, including static scatters 131, 132, 133 and / or dynamic scatters 140. In some embodiments, the velocity of motion for which CSI size information is available may be estimated based on a statistical model of EM wave theory assuming the practical rich scatter environment 100 of Figure 1.

[0280] Figure 2 shows a flowchart of an exemplary method 200 for wirelessly detecting the target motion of an object, according to several embodiments of the present disclosure. In operation 202, a first radio signal is transmitted from a first radio device, e.g., a transmitter, to a second radio device, e.g., a receiver, via a radio multipath channel of a location. In operation 204, a second radio signal is received by the second radio device via the radio multipath channel. The second radio signal is different from the first radio signal due to the radio multipath channel being affected by the target motion of an object in the location. In operation 206, a time series of channel information (TSCI) of the radio multipath channel is obtained based on the second radio signal, e.g., using a processor, memory communicably coupled to the processor, and a set of instructions stored in memory. In operation 208, a time series of spatiotemporal information (TSSTI) of an object is calculated based on the TSCI. In operation 210, the target motion of an object is detected based on at least one of the TSSTI or TSCI. The order of operations in Figure 2 can be modified according to various embodiments of the present disclosure.

[0281] The target motion detected by the bot and origin may be transient or periodic motion. In some embodiments, the bot and origin are wireless devices in a WiFi-based, robust, environment-independent fall detection system, DeFall, to detect fall events. The DeFall system utilizes physiological patterns of body velocity and acceleration during fall events, rather than the unexplained data-driven functions previously used. Human fall events involve different velocities and / or accelerations than other everyday activities, so the unique patterns of velocity and acceleration can be used to recognize fall events. Even in NLOS environments, it is possible to passively extract velocity information from WiFi signals, so the DeFall system can perform WiFi-based velocity estimation. Since falls involve unique patterns of velocity transitions and persist for a certain period of time, DeFall can use a continuously captured time series of velocity / acceleration instead of instantaneous values ​​to identify fall events. This can significantly reduce unwanted false alarms in real-world environments. Transient fluctuations in the time series present a challenge. To adapt to nonlinear compression or expansion over time, extended dynamic time stretching (DTW)-based algorithms can be applied for time series processing.

[0282] In some embodiments, the DeFall system includes two key components: an offline template generation phase and an online decision-making phase. In the offline phase, representative templates for velocity and acceleration sequences are generated. The similarity between the real-time velocity and acceleration sequences and the templates is then evaluated in the online phase to detect falls. Because velocity and acceleration are inherent characteristics of human movement that are independent of a static background environment, the DeFall system requires only one light training session and is robust to different environments in an unsupervised manner. Furthermore, due to the rich scatter model used in speed estimation, the system can perform very well in both line-of-sight (LOS) and non-line-of-sight (NLOS) scenarios.

[0283] To evaluate DeFall's performance, extensive experiments were conducted in typical indoor environments under various settings. To calculate the detection rate (DR), over 800 fall experiments were first performed under LOS and NLOS conditions using human-like models with the same size and weight as real humans. The false alarm rate (FAR) could also be tested while real humans performed routine indoor activities, including walking and sitting. Furthermore, samples of real human falls were used to verify the feasibility of the disclosed system. The experimental results show that DeFall can achieve a DR of 96% with an FAR of 1.47%, outperforming existing solutions in terms of both accuracy and coverage.

[0284] The DeFall system is the first to leverage time-series speed and acceleration data to detect falls based on WiFi devices, yet it works well in both LOS and NLOS scenarios, removing coverage limitations while protecting privacy. Long-term testing has shown that the system can function independently in changing environments without the need for retraining.

[0285] Velocity and acceleration are two properties typically used to describe motion. Intuitively, a fall can be seen as a type of unusual indoor event involving abnormal speed and acceleration. Therefore, they are both considered unique properties that help distinguish falls from other everyday activities. The uniqueness lies not only in the absolute values ​​of velocity and acceleration at the time of a fall, but also in how they change over time. More specifically, when a person falls to the ground, the body is initially accelerated rapidly. Once the body hits the floor, the body's velocity drops sharply to near zero. In fact, the majority of unexpected falls exhibit similar patterns, which suggests the feasibility of developing an environment-independent system by monitoring velocity and acceleration fluctuations, as utilized by the DeFall system.

[0286] Speed ​​Estimation from CSI: In wireless communication, channel status information (CSI), or alternatively channel frequency response (CFR), describes the propagation of a signal from the transmitter (Tx) to the receiver (Rx). The estimated value of the CSI across subcarriers at frequency f and time t can be expressed by the following equation: (Formula 1) TIFF2026102532000001.tif1956 Here, X(t,f) and Y(t,f) are the transmitted and received signals. Since the transmitted WiFi signal is subjected to multiple reflections during propagation in an indoor environment, the CSI contains a lot of useful information about the channel state, which means that changes in the surrounding environment can be captured via the CSI.

[0287] Because a unique pattern of velocity and acceleration is utilized, it is crucial to have an accurate and reliable estimate of velocity based on WiFi CSI. This is no ordinary matter due to the multipath effects of indoor propagation. Some device-free CSI-based velocity estimators utilize Doppler frequency shift (DFS) to calculate the velocity of a moving body under LOS coverage constraints, since the moving body should be "visible" by both Tx and Rx. DFS-based methods are based on the assumption of a limited number of propagation paths, which typically does not hold in practical indoor environments with rich multipath propagation. In addition, DFS-based velocity estimators take CSI phase into account. However, due to phase synchronization errors between WiFi Tx and Rx, it is not possible to accurately measure the phase of CSI on commercial WiFi devices.

[0288] The disclosed system can estimate velocity based on a statistical model of EM wave theory assuming a practical, rich scatter environment, as shown in Figure 1, which utilizes only CSI size information. Specifically, the CSI size can be measured through the CSI power response, which is defined as follows: (Formula 2) TIFF2026102532000002.tif11112 Here, The filename is TIFF2026102532000003.tif1268, TIFF2026102532000004.tif1232 represents the propagating signal. We focus on additive noise for ε(t,f), and assume that ξ(t,f) and ε(t,f) are independent of each other. It has been shown that the velocity of a moving object can be reliably estimated by evaluating the autocorrelation function (ACF) of G(t,f). The theoretical ACF of G(t,f), ρ G (τ,f) can be derived as follows: (Formula 3) TIFF2026102532000005.tif23166 Here, τ is the time lag of the ACF, TIFF2026102532000006.tif1224 and TIFF2026102532000007.tif1123 represents the variances of ξ(t,f) and ε(t,f), respectively. ξ (τ,f) and the Dirac delta function δ(·) are the ACF of ξ(t,f) and ε(t,f). When τ ≠ 0, δ(τ) can be made to = 0, and ρ G (τ,f) can be further derived based on the statistical theory of EM waves as follows. (Formula 4) TIFF2026102532000008.tif23157 Here, C1(f) and C2(f) are scaling factors determined by the power reflected by all scatter, ρ Eu (τ,f) is in the u-axis direction This is the ACF of TIFF2026102532000009.tif1231, where u∈{x,y,z}.

[0289] Velocity v along the z-axis i For the i-th dynamic scatter moving in this direction, the scatter signal is It is represented as TIFF2026102532000010.tif1230. And its ACFρ along the {x,y,z} axes. Eiu The components of (τ,f) can be expressed by the following closed-form equations: (Formula 5) TIFF2026102532000011.tif41163 (Formula 6) TIFF2026102532000012.tif23149 Here, k represents the wave number. Intuitively, the above equation establishes a relationship between the ACF ρG(τ, f) and the presence of motion and the moving speed.

[0290] Here, ρ G The relationship between (τ, f) and the presence of motion will be described. From equation (3), when there is motion in the propagation environment of the WiFi signal, due to the characteristics of white noise and the continuity of motion, as τ → 0, δ(τ) = 0 and ρξ(τ, f) → 1 can be achieved. As a result, when τ → 0 TIFF2026102532000013.tif1975 When there is no motion, the surroundings are stationary and the variance is TIFF2026102532000014.tif1241 and thus, as τ → 0, ρ G (τ, f) = 0. Therefore, lim τ→0 ρ G (τ, f) can indicate the presence of motion in the surrounding environment.

[0291] Here, ρ G The relationship between (τ, f) and the moving speed will be described. Without loss of generality, for the simple case of all dynamic scatterers moving at the same speed and direction, assume that the moving direction is along the z-axis, and ρ G (τ, f) can be obtained as in equation (4) with its components represented by equations (5) and (6). Each component and its derivative can be visualized in FIGS. 3A and 3B, respectively.

[0292] FIGS. 3A to 3D are diagrams showing exemplary spatial autocorrelation functions (ACFs) and their derivatives for electromagnetic (EM) wave components according to an embodiment of the present disclosure. FIG. 3A shows the theoretical spatial ACF, FIG. 3B shows the theoretical differential spatial ACF, FIG. 3C shows an instance of the differential spatial ACF of the CSI power, and FIG. 3D shows the differential ACF of the CSI power and the inter-valley positions.

[0293] TIFF2026102532000015.tif1229 The first local valley ofG Observing that the first local peak of (τ) is also ∀u∈{x, y}, we see Δρ G By finding the first local valley of (τ,f), speed information of the moving scatter can be extracted. When a single subject, such as a human, moves within the coverage of the Rx and Tx pair, the dynamic signal is dominated by the portion reflected by the human body. Thus, in this case, it is reasonable to assume that all dynamic scatter are moving at the same speed and in the same direction, and that the speed of the human in fall can be estimated using the disclosed method, and the fall can be further detected.

[0294] The DeFall system primarily consists of two stages, as shown in Figure 4: an offline template generation stage 410 and an online decision-making stage 420. In the offline stage 410, the speed of the fall is estimated from the WiFi CSI in operation 411 by applying a statistical model for radio wave propagation in a rich indoor scatter environment. Next, a dynamic time stretching (DTW) based algorithm can be executed to generate a representative template for a typical human fall. The representative template may be a two-dimensional (2D) template 416 that includes both velocity and acceleration patterns 415.

[0295] For example, the Segment Local Normalized Dynamic Time Warping (SLN-DTW) algorithm and the DTW Centricular Averaging (DBA) algorithm may be performed in operations 412 and 413, respectively, to generate a velocity template. Based on the velocity template generated in operation 413, an acceleration template can be generated in operation 414. The 2D template 416 contains information from both the velocity template and the acceleration template.

[0296] Next, a fall event can be detected in the online phase 420 by evaluating the similarity between the patterns of real-time velocity / acceleration estimates 423 and 424 and the representative template 416. Furthermore, prior to fall detection, an online motion detection module 422 is added as a pre-determination procedure. In this embodiment, fall detection is performed only after the presence of motion has been detected by module 422.

[0297] In some embodiments, during the offline template generation step 410, a CSI sequence of toppling events is randomly selected and based on the corresponding estimated velocity sequence in the "template database". TIFF2026102532000016.tif1159 is constructed. To construct a single representative template, an "average" operation can be performed on the database. Since the collected data is all time sequences, the results of direct point-to-point matching and averaging are easily affected by sequence shifts and misalignments. Therefore, distance measurement operations, as with series alignment, will be performed in DTW space.

[0298] Before and after a fall event, there are redundant velocity segments of other movements, and typical DTW algorithms are sensitive to the sequence endpoint. Therefore, the sequence endpoint should be carefully defined, and template database cleaning may be performed.

[0299] Template database cleaning: To remove redundancy while adapting to possible variability in event instances, we can rely on band relaxation segment local normalization dynamic time stretching (SLN-DTW). Figures 5A and 5B show examples of sanitized velocity sequences obtained by applying SLN-DTW.

[0300] Averaging in the DTW measurement space: then a refined database In TIFF2026102532000017.tif1111, the clean velocity series M is scaled to the same length and averaged in the DTW measurement space to construct a single representative profile. The problem of finding the optimal mean is a set of template time series. TIFF2026102532000018.tif1159 can be formulated as an optimization problem, and the averaged sequence TIFF2026102532000019.tif1111 is as follows: TIFF2026102532000020.tif1111 and This is the sequence that minimizes the sum of the squared DTW distances between all sequences in TIFF2026102532000021.tif1111. (Formula 7) TIFF2026102532000022.tif2590 The distance between two sequences DTW(A,B) is defined as the Euclidean distance between series B and series A along the optimal warping path, as follows: (Formula 8) TIFF2026102532000023.tif40127 Here, P * This is the optimal warping path that minimizes the normalized distance: (Formula 9) TIFF2026102532000024.tif26113 Here, a p and b p This is the index associated with the p-th point on path P.

[0301] To solve the minimization problem of Equation 7 and obtain the optimal mean sequence, the DTW centroid mean-averaging (DBA) algorithm can be implemented. DBA follows a convergence-proven expectation maximization scheme, and the mean sequence in each iteration is reduced. This is an iterative algorithm for refining TIFF2026102532000025.tif1111. Next, the optimal speed time series generated by the DBA is... TIFF2026102532000026.tif1111 will be considered as a speed template.

[0302] In addition to speed, acceleration describes the movement during a fall from a different perspective. To obtain a more comprehensive description of the fall event, use a speed template. Accelerated series from TIFF2026102532000027.tif1111 TIFF2026102532000028.tif1113 is derived, and a 2D template is created. They can be combined by point-to-point stitching to generate TIFF2026102532000029.tif1117.

[0303] Decision-making phase: A fall event experiences a distinct velocity and acceleration pattern that can be used to distinguish the fall from other everyday indoor activities. Since velocity estimation requires a high sampling rate, the decision-making phase includes a low-rate motion detection (MD) module in addition to the fall detection (FD) module to conserve energy and computation.

[0304] Motion detection module: lim as the standard for MD τ→0 ρ G Since (τ,f) is available, we can approximate τ → 0 with ρ G Only (τ=1 / Fs,f) can be used. For efficient energy saving, the MD module is added before the FD module as a pre-detection of human movement, and the FD module is triggered only when movement is present.

[0305] Rollover detection module: In the FD module, a sliding window W can be applied to the incoming CSI stream. The test speed sequence T is estimated from the CSI sequence in window W. The acceleration sequence T' is further derived from T, and the 2-D pattern T 2D Combination operations to form the resulting structure follow.

[0306] Next, the test time series T 2D to template By comparing with TIFF2026102532000030.tif1117, the overturning event can be detected. The corresponding similarities of the two sequences are evaluated in DTW space to accommodate misalignment of the two sequences in the time domain.

[0307] Because fall events involving different people may experience different durations, sequences segmented by a fixed-length sliding window may include other activities before and after the target event, which cannot be handled by conventional DTWs. Therefore, to localize the start and end instances of an event, segment-localized DTW (SLN-DTW) can be re-adopted. Template as target sequence TIFF2026102532000031.tif1117 and T as a test series 2D Regarding templates Since TIFF2026102532000032.tif1117 has already been sanitized, S 2D The length of the start and end bands can be set to 1.

[0308] By conducting SLN-DTW, the test stream T 2D and Similarity with TIFF2026102532000033.tif1117 is evaluated. If the DTW distance between the test sequence and the reference template is less than a predetermined empirical threshold γ, the test sequence T 2D teeth See TIFF2026102532000034.tif1117. It has a similar pattern to the tipping template, and the detector warns when tipping occurs, where γ is determined by the experiment and the need for FAR and DR.

[0309] According to some embodiments, in real-time monitoring, the MD module continues to operate at a lower sampling rate. As long as motion is detected, the FD module begins to operate at a higher sampling rate to detect topple events. When the estimated rate remains below a certain threshold for a sufficiently long period, it can switch back to the MD module to save power consumption and computational costs.

[0310] According to several embodiments, one prototype of DeFall was implemented based on a commodity off-the-shelf (COTS) WiFi device with a carrier frequency of 5.808 GHz and a bandwidth of 40 MHz. The detailed setup for this experiment is shown in Figure 6 with the Tx and Rx positions marked, where the experiment can be conducted under LOS and NLOS scenarios, respectively.

[0311] In a Line of Sight (LOS) scenario where both Tx and Rx can "see" the object, Tx is positioned at position Tx1. The dummy inversion is performed near the direct links of Tx and Rx.

[0312] In the case of NLOS, there is no direct link between the subject and one or more devices, which is very common indoors. Tx is placed on position Tx2, while the position of Rx is kept the same as the position below in the LOS scenario. Tin foil was placed on the wall between Tx2 and Rx to ensure blocking of the direct path. Data collection will be carried out on various days over more than three months, while the surrounding propagation environment is constantly changing, such as changes in furniture placement, opening and closing of doors and windows. To verify the feasibility of DeFall, both template data and test data can be collected first using human-like dummies. Then, samples from actual human falls will be evaluated to further verify the effectiveness of the system.

[0313] The experiment examines two types of fall events: "falling after standing" and "falling after walking." "Falling after standing" is achieved by first having a dummy stand upright and then allowing it to fall freely. On the other hand, "falling after walking" requires the experimenter to walk around a standing dummy at a normal speed before knocking it over. Examples of speed and acceleration patterns for "falling after walking" are shown in Figures 7A and 7B, respectively. For "not falling," high-speed daily activities including walking and sitting are considered. Figures 7C and 7D show examples of speed and acceleration patterns for "sitting." After long-term data collection, there are a total of 846 fall samples and 814 non-fall samples.

[0314] The templates generated after refinement and averaging are shown in Figures 8A–8C, showing that velocity initially rises to a peak and then falls, and acceleration is initially positive and then negative. Figure 8A shows the template velocity series, Figure 8B shows the template acceleration series, and Figure 8C shows the template in two-dimensional space.

[0315] The performance metrics for the system are the detection rate and the false alarm rate. The detection rate, shortened as DR, is defined as the percentage of falls correctly detected out of all falls, and the false alarm rate, simplified as FAR, is the percentage of non-falls mistaken for falls out of all non-falls. A threshold-based method can be applied to detect falls, and two features are disclosed: (i) maximum speed and (ii) maximum change in acceleration within 0.5 seconds. To demonstrate the efficiency of the DeFall system, its receiver operating characteristic (ROC) curve can be compared to that of the threshold-based method in Figure 9. Figure 9 shows the ROC curves for the dynamic time stretching (DTW) method of this disclosure versus the threshold method according to one embodiment of this disclosure. As shown in Figure 9, at the same level of FAR, the DR of DeFall is higher than that of the threshold-based method. The area under the curve (AUC) of the ROC curve for DeFall is similarly large, demonstrating better performance. In particular, as shown in the enlarged portion of Figure 9, when FAR is less than 1.5%, DeFall can still achieve a high DR of over 95%, while the corresponding threshold-based method's DR drops to less than 75%.

[0316] Table 1 summarizes the DR and FAR results for all types of events. According to the results listed in Table 1, DeFall successfully achieves high DR and low FAR under both LOS and NLOS scenarios. Comparing the results for different fall events, the "standing-to-fall" event has a higher DR than the "walk-to-fall" event because "walk-to-fall" can interfere with the estimation of the velocity at the start of the fall. Also, among non-fall events, the FAR for "sitting" is slightly higher than that for "walking" because "sitting" involves acceleration followed by deceleration, making it more similar to a fall pattern than "walking." Table 1: Experimental results regarding FAR and DR JPEG2026102532000035.jpg64154

[0317] In some embodiments, to demonstrate the practical applicability of our system, we ask two volunteers to perform real falls under the protection of a thin, firm medium buffer on the ground. We have a total of 100 real fall samples taken from one male and one female. Using templates generated from dummy falls and taking thresholds selected based on the above results, we achieve a DR of 96% for real falls, which further demonstrates the independence of DeFall from the environment and subjects and presents its great potential for real-world deployment.

[0318] Figure 10A shows an exemplary environment for wireless material sensing according to some embodiments of the present disclosure. For example, in a two-bedroom apartment 1000, as shown in Figure 10A, the origin 1032 may be placed in the living room area 1002 and the bot 1031 may be placed in the dining room area 1006. In this example, when the origin 1032 and bot 1031 work together to perform wireless material sensing as a bot-origin pair, the origin 1032 and bot 1031 are in two separate devices, while there is a line of sight (LOS) between the origin 1032 and bot 1031.

[0319] In another embodiment, as shown in Figure 10A, a device 1020 containing both bot 1021 and origin 1022 is located in bedroom 1 of area 1004. Here, origin 1022 and bot 1021 are located adjacent to each other on device 1020 when working together as a bot-origin pair to perform wireless material sensing.

[0320] In yet another embodiment, as shown in Figure 10A, device 1010, which includes both bot 1011 and origin 1012, is located in bedroom 2 of area 1008. In this example, origin 1012 and bot 1011 are located far apart from each other on device 1010, with a line of sight (LOS) between them, when origin 1012 and bot 1011 work together to perform radio material sensing as a bot-origin pair.

[0321] Each bot in a bot-origin pair can transmit a radio signal via a radio multipath channel, while the corresponding origin in the pair can obtain channel information of the radio multipath channel based on the radio signal. Since the radio signal may experience reflection by the surface of the object's material after being transmitted by the bot but before being received by the origin, the channel information obtained by the origin will contain some information related to the type of material. For example, the origin can calculate material feature information based on the channel information, either by itself or via a third device such as a material detector, and then detect the type of material based on the material feature information.

[0322] Figure 10B shows exemplary scenarios for wireless material sensing according to some embodiments of the present disclosure. The wireless material sensing shown in Figure 10B is performed by a device 1010 having both a bot 1011 and an origin 1012. As shown in Figure 10B, the bot 1011 and the origin 1012 are within each other's line of sight (LOS), with an LOS distance D1 between them. The bot 1011 and the origin 1012 may have both a transmitter and a receiver, but in this embodiment the bot 1011 functions as the transmitter (including one or more transmitting antennas) and the origin 1012 functions as the receiver (including one or more receiving antennas). The bot 1011 can transmit a wireless signal 1014 that is reflected by the surface of an object 1015. The object 1015 may be a wall, part of furniture, a device, a person, an animal, etc. The surface material of the object 1015 may affect the wireless signal 1014 so that the reflected wireless signal 1016 after reflection from the surface can contain information related to the type of surface material. Thus, after origin 1012 receives the reflected radio signal 1016, origin 1012 can calculate or detect the material type of object 1015 based on the reflected radio signal 1016 and / or other contextual information.

[0323] In some embodiments, contextual information may include the distance D1 between bot 1011 and origin 1012, and / or the distance D2 between object 1015 and device 1010, as shown in Figure 10B. In some embodiments of this disclosure, a moving material sensing system (hereinafter referred to as "mSense") using a single millimeter-wave (mmWave) radio is disclosed. Unlike most existing systems that rely on penetrating signals and / or multiple transceivers, mSense utilizes signals reflected from a target and uses a single general-purpose mmWave network radio. Different objects reflect incident electromagnetic waves to different degrees depending on their inherent material. For example, metal typically reflects much more energy than wood. The mSense system avoids the need to place two or more radios on either side of the target, or the need to measure it with any device, and thus enables everyday use in a ubiquitous environment. To identify a target material, the user can simply point the radio at the target and either keep it stationary or move it a short distance. Next, the mSense system measures the channel impulse response (CIR) of the reflected signal and, without involving any unexplained functions or machine learning, can calculate new parameters from the extracted CIR to determine the material type.

[0324] In some embodiments, the mSense system models signal propagation using the reflectivity, an intrinsic property of the material. The mSense system then derives a novel Material Reflectivity Feature (MRF) that quantitatively characterizes the reflectivity of the material and, accordingly, relates it to the target material type. Independent of the environment and propagation distance, MRF enables flexible material identification in mobile contexts. Accurate and reliable estimation of MRF presents several challenges. In particular, CIR measured using the mmWave platform offers limited distance resolution (4.26 cm given a 3.52 GHz bandwidth), includes synchronization drift, and suffers from significant measurement noise, all of which lead to errors in estimating propagation distance and signal amplitude. To overcome these challenges, the mSense system first upsamples the measured CIR to decompose the range estimation accuracy to the sub-centimeter level. Next, a novel synchronization method is presented that utilizes a direct path, i.e., direct leakage between a transmitter (Tx) and receiver (Rx) located in the same location, to synchronize all CIRs. To counteract measurement noise, noise reduction may be utilized to eliminate hardware distortion and measurement noise, obtaining a component related only to the target reflection. Spatial diversity resulting from large antenna arrays to enable robustness was further studied. Next, target detection on CIR was proposed to estimate accurate propagation distance and corresponding amplitude response, and to calculate the MRF accordingly. Finally, the mSense system can directly determine the material type by using the estimated MRF value to search for the best-matching record in a prior database.

[0325] A prototype of the mSense system can be implemented on the mmWave testbed, which enables radar-like operation on a general-purpose high-frequency 802.11ad / ay chipset by attaching one additional antenna array, each containing multiple (e.g., 32) antennas. Using this testbed, experiments can be conducted to verify performance on several (e.g., five) common types of materials: metal (aluminum), plastic, ceramic, water, and wood. The results show that the mSense system achieves an average identification accuracy of 92.87% regardless of various target sizes, thicknesses, and distances to the device. More importantly, the accuracy remains at 89.36% in mobile scenarios where the user holds and moves the device for sensing. The disclosed system sheds light on ubiquitous material identification for everyday use, relying solely on reflected signals measured by a mobile radio. Implementation on a radar platform reusing network devices also takes a step towards collaborative radar communication systems, which could promise a future paradigm for wireless communication and sensing.

[0326] We can aim for mobile applications in a ubiquitous context that require recognition of the material type of a target. For example, one might be interested in knowing the material of everyday objects, such as whether a whiteboard is made of metal or plastic, and a robot (e.g., a robotic vacuum cleaner) can adapt its movements accordingly according to the material type of the surrounding objects. Next-generation educational toys and devices can teach children to perceive the physical world in an interactive way. Novel applications such as interactive input devices that change their functions based on the material they touch will also become available. Taking it a step further, if mmWave wireless becomes available in smartphones, the mSense system will instantly provide smartphones with the ability to perform everyday material sensing. The key to realizing these applications is ubiquitous material sensing in a mobile environment that does not require dedicated hardware or cumbersome setup.

[0327] Specifically, the following usage scenario can be envisioned: the user simply holds a portable radio, points the antenna array towards the target for a short period, and the system automatically recognizes the material of the target. The user can hold the device stationary or, if preferred, move it towards the target. If the user chooses to move the radio, the user can generally move it freely, but only needs to keep the antenna as perpendicular as possible to the target. The radar maintains transmitted and received reflected signals throughout the sensing period. From the received set of signals, the mSense system automatically determines the material type, independent of the environment, target size, thickness, etc. While specialized devices for material recognition have been explored, this disclosure aims to leverage general-purpose radio devices for a low-cost, portable solution. More specifically, the ultimate goal is to enable target material sensing anywhere, enabling such capabilities when a smartphone or other smart device is equipped with a mmWave radio in the future.

[0328] In recent years, wireless sensing and tracking have surged. Another trend is the synergistic design of communication and radar systems that utilize the same hardware and spectral resources for dual functionality of both wireless communication and radar sensing. Currently, millimeter-wave wireless is gaining popularity as the next-generation WiFi technology standardized as 802.11ad / ay, with 60GHz WiFi already available in general-purpose routers and integrated into smartphones and automotive applications. The same wireless technology, with higher bandwidth and phased antenna arrays, also shows promise for short-range radar applications. In industry, in addition to academic research, there is active exploration of reusing 60GHz wireless to enable radar-like capabilities such as gesture recognition and room mapping. This device can transmit and receive on a single 60GHz chip and thus support radar-like features. Such dual networking and radar devices have clear advantages over existing solutions, as they reuse networking devices and will become ubiquitous radar on mobile devices as soon as 60GHz WiFi becomes widely deployed. In fact, it is not surprising that radar capabilities will be integrated into smartphones in the near future. Operating 60GHz radios in radar mode is likely to provide higher quality for material identification and other sensing applications compared to 2.4GHz / 5GHz WiFi. A second motivation for using 60GHz radios is the choice of a design that uses only reflected signals. A bottleneck that prevents existing solutions from being used in ubiquitous applications is that they require a special setup with two or more transceivers on both sides of the target or a special setup where the target is tagged. These devices are limited to use in fixed locations and are not yet ready to enable truly ubiquitous material sensing anywhere. In addition to inconvenience, the use of penetrating signals imposes two fundamental problems. First, EM waves propagating through different media (e.g., lossy media vs. lossless media) attenuate significantly differently. For example, in free space and nearly in the air, the amplitude decreases linearly with respect to where the propagation distance is. In contrast, in lossy media, the amplitude experiences an exponential decrease.As a result, conventional studies either apply general models to different media present in the propagation path or assume specific categories of materials (especially different liquids). A second challenge is that for conductive materials (e.g., metals, graphite), RF signals may not be able to penetrate the target. Radio signals, particularly those in the mmWave frequency band, cannot penetrate very thick objects. In this disclosure, to overcome limitations and improve usability, only reflected signals can be used for analysis, thus avoiding special bifacial setups and bypassing the problem of model versatility. A 60 GHz signal with a wide bandwidth and many antennas provides better resolution when resolving reflected signals.

[0329] Overall, the mSense system aims to achieve target material sensing from reflected signals by using a single 60GHz device, which would promise ubiquitous use in mobile environments. Using reflected signals for material detection is more difficult and, to some extent, indirect, since the received signal does not pass through the material at all. The theoretical basis is that different materials reflect incoming signals to distinctly different degrees, thereby "encoding" material type information in the reflected signal.

[0330] Figure 11 shows an exemplary workflow for wireless material sensing based on channel impulse response (CIR) according to several embodiments of the present disclosure. Specifically, Figure 11 shows a method 1100 of an mSense system for determining material type based on channel impulse response (CIR) obtained from reflected wireless signals. As shown in Figure 11, the operation of the mSense system includes two stages: a training stage 1110 for building a database 1107 of different materials, and a testing stage 1120 for testing a target by matching it against the database 1107.

[0331] In operation 1108 of test phase 1120, multiple operations can be performed to process the raw CIR in order to determine or identify the material type. The mSense system can collect the CIR in operation 1101. For example, the CIR may be collected when the user points the radio at the target object (e.g., nearly perpendicularly). Then, in operation 1102, CIR interpolation is performed. For example, the raw CIR is upsampled to improve distance accuracy. Then, in operation 1103, the time series of CIR measurements is synchronized.

[0332] Next, in operation 1104, the system may use noise reduction to remove hardware strain and measurement noise. Then, in operation 1105, the system may perform target detection to obtain amplitude and distance from all available antenna pairs, from which the system may estimate material reflection features (MRF) in operation 1106. In some embodiments, the MRF can characterize the reflectivity of a material and can be easily calculated from the reflected signals alone. Finally, in operation 1108, the system can determine the material type by finding the best match against a pre-trained database 1107 that stores MRFs for various materials.

[0333] In some embodiments, the training and testing phases share the same processing procedures for MRF extraction and estimation, i.e., operations 1101-1106 as shown in Figure 11. Training needs to be performed only once for each type of material and can be done in a separate setting from the testing phase.

[0334] Figure 12 shows a flowchart of an exemplary method 1200 for wireless material sensing according to several embodiments of the present disclosure. In operation 1202, a first wireless signal is transmitted to a receiver via a wireless multipath channel of a location by a plurality of transmitting antennas of a transmitter. In operation 1204, a second wireless signal is received via the wireless multipath channel by a plurality of receiving antennas of a receiver. The second wireless signal includes the reflection or refraction of the first wireless signal on the surface of a target material of an object in the location. In operation 1206, a plurality of channel information (CIs) of the wireless multipath channel are obtained based on the second wireless signal using, for example, a processor, a memory communicably coupled to the processor, and a set of instructions stored in the memory. Each CI is associated with one of each of the plurality of transmitting antennas and one of each of the plurality of receiving antennas. In operation 1208, a material analysis is calculated based on the plurality of CIs. In operation 1210, the type of target material of an object is determined based on the material analysis. In some embodiments, the first wireless signal is a wireless probe signal. In some embodiments, each CI includes at least one of the following: channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI). The order of operations in Figure 12 can be modified according to various embodiments of this disclosure.

[0335] Figure 13 shows a flowchart of exemplary method 1300 for CIR interpolation and CIR synchronization according to several embodiments of the present disclosure. In operation 1302, all CIRs are interpolated by interpolation coefficients based on at least one of linear interpolation, polynomial interpolation, or spline interpolation. In operation 1304, all interpolated CIRs are synchronized by aligning the CIRs based on LOS tabs, each of which is a CIR tab having a magnitude corresponding to the LOS component of the radio probe signal. In operation 1306, material analysis is calculated based on the synchronized CIRs. The sequence of operations in Figure 13 can be modified according to various embodiments of the present disclosure.

[0336] Figure 14 shows a flowchart of an exemplary method 1400 for CIR synchronization according to several embodiments of the present disclosure. In operation 1402, one of the CIRs is determined as the reference CIR. In operation 1404, the reference window of the tab of the reference CIR, which contains at least one LOS tab, is determined. In operation 1406, a matching score is calculated between the reference window of the reference CIR and the sliding window of the additional CIR, and the best-match sliding window with the highest matching score is determined. In operation 1408, the additional CIR is aligned with the reference CIR based on the reference window and the best-match sliding window. In operation 1410, the additional CIR is rotated so that the best-match sliding window is shifted to the same tab position as the reference window. The order of operations in Figure 14 can be modified according to various embodiments of the present disclosure.

[0337] Figure 15 shows a flowchart of another exemplary method 1500 for CIR synchronization according to several embodiments of the present disclosure. In operation 1502, one of the CIRs is determined to be the reference CIR. In operation 1504, the representative tab position of the reference CIR is calculated. In operation 1506, the representative tab position of an additional CIR is calculated. In operation 1508, the additional CIR is aligned with the reference CIR based on the representative tab positions of the reference CIR and the additional CIR. In operation 1510, the additional CIR is rotated so that its representative tab position is closer to the representative tab position of the reference CIR. The order of operations in Figure 15 can be modified according to various embodiments of the present disclosure.

[0338] Figure 16 shows a flowchart of an exemplary method 1600 for noise reduction and target detection according to several embodiments of the present disclosure. In operation 1602, each background CIR is synchronized with CIR (raw). In operation 1604, the synchronized background CIR is scaled. In operation 1606, the CIR with each background subtracted is calculated by subtracting the scaled and synchronized background CIR from the CIR. In operation 1608, spatiotemporal quantities are calculated based on the CIR with each background subtracted. In operation 1609, it is determined whether any raw CIR remains. If so, the method returns to operation 1602 to perform noise reduction on the raw CIR. If none remains, the method proceeds to operation 1610 to calculate material analysis based on multiple spatiotemporal quantities calculated based on all CIRs. The order of operations in Figure 16 can be modified according to various embodiments of the present disclosure.

[0339] Figure 17 shows a flowchart of an exemplary method 1700 for acquiring background CIR according to several embodiments of the present disclosure. In operation 1702, an additional radio probe signal is transmitted by multiple transmitting antennas of the transmitter through the radio multipath channel of the location, removing an object from the region of interest. In operation 1704, the additional radio probe signal is received by multiple receiving antennas of the receiver through the radio multipath channel. In operation 1706, multiple background CIRs of the radio multipath channel are acquired based on the additional radio probe signal. Both each CIR and each background CIR subtracted from each CIR are associated with the same transmitting antenna and the same receiving antenna. The order of operations in Figure 17 can be modified according to various embodiments of the present disclosure.

[0340] Figure 18 shows a flowchart of another exemplary method 1800 for acquiring background CIR according to several embodiments of the present disclosure. In operation 1802, an additional radio probe signal is transmitted by multiple transmitting antennas of the transmitter through the radio multipath channel of the location, removing an object from the area of ​​interest. In operation 1804, the additional radio probe signal is received by multiple receiving antennas of the receiver through the radio multipath channel. In operation 1806, multiple test CIRs of the radio multipath channel are acquired based on the additional radio probe signal. In operation 1808, it is determined whether there is an object in the area of ​​interest based on the multiple test CIRs, and a determination result is generated. If it is determined in operation 1809 that there is an object in the area of ​​interest, the method returns to operation 1802 and transmits another radio probe signal to acquire background CIR. If it is determined in operation 1809 that there is no object in the area of ​​interest, the method proceeds to operation 1810 and automatically acquires multiple background CIRs based on the multiple test CIRs. The order of operations in Figure 18 can be modified according to various embodiments of the present disclosure.

[0341] Figure 19 shows a flowchart of an exemplary method 1900 for material reflection feature (MRF) estimation according to some embodiments of the present disclosure. In operation 1902, a transmit and receive antenna pair is identified. In operation 1904, it is determined that the background-subtracted CIR is associated with the transmit and receive antennas using a first CIR tab associated with the LOS component of the radio probe signal. In operation 1906, the maximum magnitude of the background-subtracted CIR is identified after the first CIR tab, and the maximum magnitude is associated with a second CIR tab of the background-subtracted CIR. In operation 1908, a candidate material analysis is calculated based on the second CIR tab and the maximum magnitude. In operation 1909, it is determined whether there are any further antenna pairs. If so, the method returns to operation 1902 to calculate a candidate material analysis based on the next antenna pair. Otherwise, the method proceeds to operation 1910 to calculate a material analysis based on an aggregate analysis of the multiple candidate material analyses calculated based on all antenna pairs. The sequence of operations in Figure 19 can be modified according to various embodiments of this disclosure.

[0342] Figure 20 shows a flowchart of another exemplary method 2000 for MRF estimation according to some embodiments of the present disclosure. In operation 2002, a transmit-receive antenna pair is identified. In operation 2004, a first distance between the transmit and receive antennas is determined. In operation 2006, the time difference between the first and second CIR tabs is calculated based on the sampling frequency of the CIR with background subtracted. In operation 2008, a second distance is calculated based on the product of the time difference and the wave propagation speed of the radio probe signal in the location. In operation 2010, a third distance is calculated based on the weighted sum of the first and second distances. In operation 2012, a candidate material analysis is calculated based on the largest size and the third distance. In operation 2013, it is determined whether there are any further antenna pairs. If so, the method returns to operation 2002 and calculates the candidate material analysis based on the next antenna pair. If none exists, the method proceeds to operation 2014, which calculates the material analysis based on an aggregate analysis of multiple candidate material analyses calculated based on all antenna pairs. The sequence of operations in Figure 20 can be modified according to various embodiments of this disclosure.

[0343] Figure 21 shows a flowchart of an exemplary method 2100 for multi-frame wireless material sensing according to several embodiments of the present disclosure. In operation 2102, a third wireless signal is transmitted to a receiver via a wireless multipath channel of a location by multiple transmitting antennas of a transmitter. In operation 2104, a fourth wireless signal is received via the wireless multipath channel by multiple receiving antennas of a receiver. The fourth wireless signal includes the reflection or refraction of the third wireless signal on the surface of the target material of an object in the location. In operation 2106, multiple additional channel information (CIs) of the wireless multipath channel are acquired based on the fourth wireless signal. Each additional CI is associated with one of each of the multiple transmitting antennas and one of each of the multiple receiving antennas. In operation 2108, an additional material analysis is calculated based on the multiple additional CIs. In operation 2110, an aggregate material analysis is calculated based on the additional material analysis and the material analysis calculated based on method 1200 of Figure 12. In operation 2112, the type of target material of the object is determined based on the aggregate material analysis. The sequence of operations in Figure 21 can be modified according to various embodiments of this disclosure.

[0344] Figure 22 illustrates a flowchart of an exemplary method 2200 for training a classifier during the training phase of a wireless material sensing system, according to several embodiments of the present disclosure. In operation 2202, a first wireless training signal is transmitted by a plurality of transmitting antennas of a training transmitter to a training receiver via a training wireless multipath channel of a training location containing the training target material. In operation 2204, a second wireless training signal is received by a plurality of receiving antennas of a training receiver via the wireless multipath channel. The second wireless training signal includes the reflection or refraction of the first wireless training signal at the surface of the training target material. In operation 2206, a plurality of training channel information (CIs) of the training wireless multipath channel are acquired based on the second wireless training signal. Each training CI is associated with one of the plurality of transmitting antennas and one of the plurality of receiving antennas. In operation 2208, a training material analysis is calculated based on the plurality of training CIs. In operation 2209, it is determined whether there is any further training material. If available, the method returns to operation 2202 to calculate the next training material analysis based on the next training material. Otherwise, the method proceeds to operation 2210 to train the classifier based on machine learning and all training material analysis. The order of operations in Figure 22 can be modified according to various embodiments of this disclosure.

[0345] For electromagnetic waves propagating in free space, the signal amplitude decreases as the reciprocal of the propagation distance, which can be well modeled by the Friis transmission formula as follows: (Formula A1) TIFF2026102532000036.tif1857 Here, A0 indicates the amplitude of the transmitted signal, A d represents the received amplitude at propagation distance d, and g t and g rλ is the transmit and receive antenna gains, respectively, and λ is the wavelength. The above model generally applies to atmospheric propagation, which approximates free-space propagation. In reality, due to the effects of multipath, an additional exponential component may be applied to d. This is not true for 60 GHz signals where multipath is minimal.

[0346] When an electromagnetic wave (EM wave) encounters a material boundary and creates an impedance discontinuity in the propagation medium, the incident energy will be partially reflected and partially transmitted to the new material. The ratio of the amplitude of the reflected wave to the amplitude of the incident wave is expressed by the reflection coefficient r. (Formula A2) TIFF2026102532000037.tif1145 Here, A in and A out These represent the amplitudes of the incident and reflected signals, respectively. In normal incidence, the reflection coefficient r is According to the Fresnel equation, represented as TIFF2026102532000038.tif1436, n1 and n2 are expressed as the refractive indices of the incident and transmitted materials, respectively. The refractive index n of a material is defined as n = c / v, where c is the speed of light in a vacuum and v is the speed of electromagnetic waves in the propagation medium. The refractive index n is given by the complex permittivity of the material εr = ε' r +ε'' r These are the inherent properties of materials related to this topic. (Formula A3) TIFF2026102532000039.tif1994 Typically, the incident material is air, and therefore n air = 1. Therefore, r = (n-1) / (n+1), where n represents the refractive index of the target material.

[0347] Assuming a uniform loss medium, the amplitude of an EM wave moving continuously inside the target decreases exponentially over the propagation distance d. (Formula A4) TIFF2026102532000040.tif1257 Here, A din The distance d is to the target surface. in The amplitude is shown at Ain This shows the amplitude at the incident surface. α is a damping constant that depends on the inherent material properties. From equation A4, α is often defined as the reciprocal of the distance δ, called the skin depth (or penetration depth), during which the intensity of the electromagnetic field is attenuated to 1 / e = 0.368 of its original value.

[0348] Similar to the refractive index, the damping constant α is also related to the material's inherent relative permittivity. (Formula A5) TIFF2026102532000041.tif26119

[0349] Therefore, if n and α are available, we can solve the two equations above together to find the two unknowns, namely ε'. r and ε'' r This can be derived.

[0350] An effective metric related to the reflective behavior of a material, called material reflection characteristics (MRF), can be extracted. Please recall equations A1 and A2. The received amplitude at Rx, which is at the same position as Tx, can be expressed as follows: (Formula A6) TIFF2026102532000042.tif1984

[0351] The above equation explains the round-trip propagation distance 2d and the target's reflection coefficient r. It is important to note here that the target distance d is the distance from the target surface to the radio. If the filename is TIFF2026102532000043.tif1431, then the following can be done: (Formula A7) TIFF2026102532000044.tif1483

[0352] The definition of γ has the following properties: Firstly, γ is a linear function of the target reflection coefficient r, which is an intrinsic property of the target material, and is independent of the propagation distance. As shown in the intermediate term, the antenna gain g is a constant of three constants, namely the transmission amplitude A0, Tx, and Rx. t and gr Except for the above, γ is related only to the reflection coefficient r and is therefore specific to a particular material. Secondly, as implied by the above term, the distance d to the target and the corresponding signal amplitude A d As long as we can obtain [the necessary parameters], γ can be easily estimated.

[0353] As a result, γ can be referred to as MRF, which can be used for material identification, successfully avoiding the need to know the exact A0 to derive r. In practice, Only the estimated version of γ, expressed as TIFF2026102532000045.tif1130, is obtained here. TIFF2026102532000046.tif1111 and TIFF2026102532000047.tif1115 contains the estimated target distance and the corresponding measured amplitude.

[0354] Figure 23 shows a set of exemplary material reflection features (MRFs) for several common materials according to some embodiments of the present disclosure. As shown in Figure 23, preliminary measurements show that the MRFγ is identifiable, reliable, and accurate in identifying different material types. The stronger a target reflects the incident signal, the larger its MRF value. Despite the presence of small dispersion, the measured MRFs for the exemplary materials are placed apart as shown in Figure 23.

[0355] The channel impulse response (CIR) profiles the propagation delay and channel response of various signal paths between Tx and Rx, and is shown as follows: (Formula A8) TIFF2026102532000048.tif2582 Here a l , θ l , and τ lThe amplitude, phase, and time delay of the first path are shown, respectively, L is the total number of paths, and δ(τ) is the Dirac delta function. Figure 24 shows an example of CIR measured by a mmWave device. Each impulse tap of CIR1 is h(τ) l ) represents the delayed multipath component. This device uses a bandwidth of 3.52 GHz centered in the 60 GHz band, resulting in a time-of-flight (ToF) resolution of 0.28 nanoseconds, which is equivalent to the speed of light c = 3 × 10⁻¹⁶ 8 Given m / s, this corresponds to a range resolution of 4.26 centimeters (for reflection paths).

[0356] Compared to 2.4GHz / 5GHz WiFi, the CIR of 60GHz mmWave signals has clear advantages due to the numerous antennas, large bandwidth, and high carrier frequency. Applying the reported CIR for material sensing presents practical challenges. While the high range resolution is impressive for room localization, it is unsuitable for material sensing. Furthermore, as demonstrated by experimental measurements, the device contains considerable noise and exhibits unreliable amplitude, making material sensing even more difficult. Specifically, Figure 24 shows the CIR measurements with and without a target in the region of interest. There are several major challenges in estimating the CIR by this embodiment of a radar-like radio. Firstly, the physical range resolution of 4.26cm is insufficient to accurately estimate the MRF. Secondly, the CIR measurements reported over time may be out of sync. Thirdly, there may be measurement noise. Ideally, the CIR should all be close to zero when there are no reflective objects. As shown in Figure 24, CIR measurements involve strong direct leakage from Tx to Rx at the same location, as well as significant internal reflections, which significantly interfere with measurements in the presence of a target.

[0357] To overcome the aforementioned challenges, namely CIR interpolation, CIR synchronization, and noise reduction, the disclosed system performs several operations.

[0358] CIR Interpolation: To obtain finer detail, raw CIRs may be upsampled via interpolation. Since the disclosed mSense system primarily utilizes the amplitude of the CIR for material identification information, spline interpolation is performed on the amplitude, which effectively subsamples the amplitude. The interpolated CIR has denser sample points, and the peak taps of a particular path tend to be closer to the actual maximum tap value. Therefore, interpolation provides finer granularity in delay (and thus range) and amplitude estimation, both of which are important for material identification. The mSense system can perform 8x interpolation to improve ToF accuracy to 35.2 picoseconds and distance accuracy to 0.52 cm⁻¹. Figure 25A shows examples of 2x and 16x interpolation of the original CIR. Figure 25B shows the CIR amplitude after noise reduction, where the maximum CIR amplitude is after the first tab in the second tab.

[0359] CIR Synchronization: The default synchronization of 60GHz radios is not sufficiently accurate due to resolution limitations. For example, a misalignment of one tap would lead to a 4.26 cm deviation in distance estimation. Neither is reliable synchronization. That is, leaky taps change over time (especially with temperature changes), and even change with the antenna pair. To synchronize all CIRs more accurately and reliably, direct leakage from Tx to Rx can be utilized. Direct leakage between Tx and Rx at the same location generates a fixed time of flight (ToF) of approximately 0.13 nanoseconds on a platform with no multipath and a separation of approximately 4 cm between Tx and Rx. Thus, the direct leakage tap can act as a reference tap, and the system can align this leakage tap of all CIR measurements, synchronize them, and derive the relative ToF of each subsequent tap. The system can then obtain the absolute ToF for distance estimation by compensating for the ToF of the direct path from a particular Tx-Rx link over the relative ToF.

[0360] Regarding synchronization, the system can first select a reference CIR to be assigned to a specific tap, where the direct leakage tap is indicated as the l0th tap. Then, for each CIR, the system can correlate it with the reference CIR. The correlation includes only the first l0 taps covering the direct leakage, excluding other taps potentially related to target or background reflections. Since direct leakage remains relatively stable with similar patterns for various measurements, the tap that yields the greatest correlation is declared as the direct leakage tap. Thus, the CIR is synchronized by shifting the direct leakage tap to the l0th tap if the direct leakage tap and the l0th tap are different. To maintain the same length of the CIR, the system can perform a circular movement for alignment, which can alternatively be achieved by padding the first (or last) tap with zeros. Interpolated CIRs may be used for finer delays. The synchronization technique is also applicable to the common case of 60 GHz networking with separate asynchronous Tx and Rx devices. As long as the LOS distance between Tx and Rx is known, all measurements can be synchronized by referencing the LOS delay, provided that the energy of the LOS path dominates the other paths.

[0361] Noise reduction: CIR is the sum of h components reflected from the target. t , components provided by direct leakage, internal reflection, measurement noise, and h n It can be modeled as background reflection (if any), shown as h from the measured value h t To obtain component h n A reference CIR measured without a target in the region of interest can be used to characterize the region. Such a reference CIR can be easily obtained by collecting measurements when the device is pointed into the air without a target. If there are Q such measurements, h n This is estimated as the mean sample. That is, TIFF2026102532000049.tif2076

[0362] Next, h from the new h measured toward the target at time t n Subtract h t This can be obtained. Since the measured CIR amplitude changes slightly over time due to automatic gain control (AGC), a scaling factor β is applied for complete cancellation. (Formula A9) TIFF2026102532000050.tif1179

[0363] Assuming that direct leakage is constant, the scaling factor was calculated based on the first tap L0, which considers only direct leakage and noise, by minimizing the mean squared error (MMSE). (Formula A10) TIFF2026102532000051.tif25108

[0364] In some embodiments, cancellation is primarily used to remove hardware distortion and measurement noise rather than interference tailored to the background. Therefore, the reference CIR can be a one-time calibration, eliminating the need to measure it for every data capture session, which greatly enhances its practical use in moving environments. In practice, the system can automatically collect a reference CIR since it can detect whether a target is within the region of interest (e.g., within 1 meter). If there is no target in the region of interest, a reference CIR can be generated using a corresponding CIR sample. Distant objects in the background (e.g., walls, furniture) may also reflect signals that should be included in the reference CIR, but they are typically outside the range of interest and therefore do not affect the mSense system's wireless material sensing.

[0365] Material Reflectance Feature Estimation: Given that direct leakage and internal reflection are excluded, a CIR tap that holds the maximum amplitude as presented by the target can be easily located, and the relative ToF Δt corresponding to that tap can be derived accordingly. The relevant amplitude is considered to be the amplitude of the signal reflected by the target.

[0366] The ToF Δt derived from synchronized CIR is a relative value with respect to the direct leakage tap. The direct leakage delay can be compensated for to obtain the absolute ToF. Given the geometric shapes of the Tx and Rx antenna arrays, the separation distance between each pair of Tx and Rx antennas can be determined in advance, thereby allowing the direct propagation delay to be derived at the top level. For M Tx antennas and N Rx antennas, an M×N matrix S can be defined as representing the Tx-Rx separation distance, where S m,n represents the entry at the m-th Tx element and the n-th Rx element, where m=1,···,M and n=1,···,N. Then, given a measurement between the m-th Tx antenna and the n-th Rx antenna, the distance to the target can be calculated by the following equation: (Formula A11) TIFF2026102532000052.tif1767 Here, half of the propagation distance is considered to be the distance assuming normal incidence, i.e., assuming the device is pointing at the target at approximately 0 degrees.

[0367] The disclosed system performs the above target detection on the CIR measured by all pairs of each Tx-Rx antenna, and can obtain an M×N estimate of γ overall, for each of the m-th Tx element and n-th Rx element. It is shown as TIFF2026102532000053.tif1650, where A m,n is d m,n Amplitude estimation at g t ,g ris the AGC gain reported by the device. The system can achieve a more robust estimate of γ by leveraging antenna diversity to remove potential outliers in target detection. Specifically, the system excludes tuples that estimate distance dm,n that deviate from the majority of all M×N estimates. (Formula A12) TIFF2026102532000054.tif2361 Here, S is Defined as a set of tuples (m,n) satisfying TIFF2026102532000055.tif14124, where δ is a threshold set to the default distance resolution, i.e., δ = 4.26 cm. TIFF2026102532000056.tif1111 is all d m,n This is the median.

[0368] The disclosed m-sense system simply requires a single frame to generate an estimate of MRFγ. If multiple frames exist over time, the system further improves the estimate by taking their average. (Formula A13) TIFF2026102532000057.tif2548 Here, TIFF2026102532000058.tif1119 is an estimate from the k-th CIR frame, where K is the total number of available frames. The device may be moving or stationary during the measurement of consecutive frames.

[0369] The process involves two steps for identifying materials: an offline step to build an MRF database for different materials, and an online step to search for the best match for a given target.

[0370] Given a set of materials of interest, the system can perform a "scan" on each of them and store the estimated variance γ. To reduce the amount of data required for training, for each type T material, instead of fitting to a specific variance, A histogram can be constructed, shown as TIFF2026102532000059.tif1193, where γ i (T) is the bin value, p i (T) is the corresponding probability (normalized observed saturation), and P is the total number of bins. In some embodiments, a median filter is applied to remove outlier estimates from the training sample before constructing the histogram. Let T be the set of materials under consideration. Then, in the online operating phase, for each material with γ to test, the best match can be found as follows: (Formula A14) TIFF2026102532000060.tif25107

[0371] Furthermore, additional rules can be applied to check for the "null" class, that is, the "null class" of material that is not visible in the training database. Specifically, for a given material, if its measured value γ does not match any of γi(T) for any i=1,...,P and any T∈T, the system treats it as an "unknown" type.

[0372] To demonstrate the system's overall cognitive performance, two frames per sample are used for training, and five frames are averaged as test samples. The effect of the number of frames per sample is then evaluated. Results from both moving and stationary cases are fused. As shown in Figure 26A, the overall recognition accuracy is 92.87%, comparable to existing work that requires penetrating targets in a bilateral device setting and cannot be used for moving material sensing. In some embodiments, the mSense system achieves remarkable recognition accuracy of over 90% for all test materials except ceramics, with estimated MRFs for ceramics showing large variability and overlapping with both plastics and moisture. In some embodiments, metals (e.g., aluminum) are accurately and precisely identified by the disclosed system, even in the mobile case, which is particularly useful for detecting suspicious targets for safety. This accuracy, thanks to the simple setup of the mSense system using a single radio and reflected signals, promises valuable applications for ubiquitous contexts.

[0373] It is interesting to investigate the performance of the mSense system in a moving environment. This demonstrates the accuracy of the system's wireless sensing in moving and stationary scenarios, respectively, as shown in Figures 29B and 26C. As shown in Figures 26B and 26C, the overall performance in the static scenario is 93.66% vs. 89.36%, as expected, being slightly better than the performance in the moving scenario. The retained performance in a moving environment, with an accuracy degradation of approximately 4%, is attractive and, given that mmWave signals are sensitive to small position / direction due to their short wavelength, is accurate enough for many everyday applications.

[0374] According to various embodiments of this disclosure, millimeter-wave (mmWave, e.g., 28 GHz or 60 GHz) signals are used or reused between an mmWave transmitter and an mmWave receiver to detect / recognize / distinguish materials on a target surface. The system utilizes the reflective properties (e.g., reflectivity) of the mmWave signal on the material surface, which is different from the refractive properties of the mmWave signal passing through the material. Thus, the transmitter and receiver can be located on the same side of the material, rather than on two different sides. The transmitter (the transmitting device of the system having a processor / memory / software) and the receiver (the receiving device of the system) may or may not be on the same device. The transmitter and / or receiver may each have an antenna array including a dispersed antenna. There may be multiple receivers, each receiving a mmWave signal from a transmitter. There may be multiple transmitters, each transmitting a mmWave signal to a receiver. There may be multiple transmitters and multiple receivers, each transmitting its respective mmWave signal to one or more receivers.

[0375] The device may have a general-purpose mmWave networking or communications chip / chipset that operates in radar mode by attaching an extra antenna array to the chipset. The chip / chipset may be used to transmit mmWave signals using radio transmission and receive reflected mmWave signals using radio reception. The chip can transmit / receive precisely simultaneously or concurrently. The chip can switch rapidly between transmit and receive to simulate or mimic "concurrent" transmit / receive.

[0376] The user carries a portable radio transmitter and / or receiver, and can briefly (for a short time) orient the antenna array towards the target surface to capture the mmWave signal / CI required for material sensing.

[0377] In some embodiments, a transmitter sends a mmWave signal onto the surface of an unknown material. A receiver (e.g., on the same device as the transmitter) receives the mmWave, extracts channel information (CI such as signal strength / RSSI, channel state information / CSI, channel impulse response / CIR, and / or channel frequency response / CFR), and analyzes the channel information to determine the material. A material reflectivity feature (MRF) is calculated based on the CI which quantitatively characterizes the reflectivity of the material, and the material is detected or recognized based on the MRF.

[0378] In the learning phase (or training phase), the system can capture training CIs by illuminating a number of known material surfaces with mmWave signals, extract the training CIs, calculate the training MRFs, apply machine learning to classify / identify the training MRFs, and calculate reference MRF classes associated with the known materials. A database may be established based on the MRF classes and / or associated classifiers. In the learning phase, a histogram of MRFs can be obtained for each known (training) material. The histograms may be cleaned and normalized (e.g., using a median filter or low-pass filter). An additional class of MRFs called "nulls" can be added to account for unknown materials that are different from the known materials.

[0379] During the operational (test) phase, the system can capture the test CI by irradiating the surface of the test material with a mmWave signal, extract the test CI, calculate the test MRF, and analyze the test MRF. The target material may be calculated or determined as a known material associated with the minimum conditional strain (e.g., absolute strain, squared strain, or monotonically non-increasing feature of absolute strain).

[0380] CI (or CSI or CIR or CFR) or MRF may be pre-processed / processed / post-processed by (a) CI interpolation, (b) CI synchronization, (c) background / noise removal, (d) target detection, (e) MRF estimation, and / or (f) material identification. CI (or CSI or CIR or CFR) or MRF may be pre-processed / post-processed by (1) interpolation to increase the temporal resolution of the CI (with finer temporal detail), (2) synchronization (e.g., direct path-based), (3) background and / or noise removal to eliminate hardware distortion and measurement noise, (4) spatial diversity due to a large antenna array to facilitate robustness, (5) estimation of propagation distance relative to the target plane, (6) estimation of the corresponding amplitude response, (7) calculation of the MRF, (8) associating the MRF with a specific reference MRF class, and (9) determining the test material as a material associated with a specific reference MRF class. One or more of the CI pre-processing / processing may be applied in the learning phase and / or operation phase.

[0381] CI can be interpolated using linear interpolation, piecewise constant interpolation, polynomial interpolation, spline interpolation, fractional interpolation, extrapolation, and / or other interpolation methods. Interpolation can be performed in the complex domain, real domain, imaginary domain, size domain, and / or topological domain.

[0382] The transmitter and receiver may be on the same device such that there is direct leakage (line of sight / LOS transmission) between the transmitter and receiver, leading to a very strong leakage pulse (observable in, for example, CI, or CIR, or CFR) with a very strong maximum value of the magnitude of CI (or the square of the magnitude or any monotonically increasing function of the magnitude).

[0383] In some embodiments, each pair of transmitting and receiving antennas generates a time series of CI (TSCI). Different TSCIs may not be synchronized and may exhibit different patterns due to different multipath propagation. TSCIs can be synchronized by (i) searching for the local / global maxima of each TSCI corresponding to a strong direct leakage pulse (or a time series of features of each CI, where features are magnitude, magnitude squared, time average, moving average, moving median, weighted average, autoregressive moving average (ARMA), correlation, one or more previous functions, and / or other features), and (ii) aligning the TSCIs by aligning the relevant local / global maxima.

[0384] Alternatively, one specific TSCI can be selected as a reference TSCI, and the maximum value of the reference TSCI (or associated feature) corresponding to direct leakage can be identified. As a reference window, a time window (i.e., a section of the reference TSCI) of CIs that encompass the high-energy direct leakage pulse waveform (perhaps including previous CIs but not future CIs, to avoid including the effects of multipath from the target plane) can be selected. The length of the time window may be related to the pulse width of the leakage pulse. Any other TSCIs can be synchronized with the reference TSCI by aligning the reference window with the corresponding time window of the other TSCI that gives the greatest cross-correlation or cross-covariance.

[0385] By performing CI interpolation before CI synchronization, synchronization can be performed at a higher sampling rate for the interpolated CIs, and therefore with higher accuracy.

[0386] To perform background / noise (including direct leakage, internal reflection, measurement noise, and background reflection) rejection, the user can orient the antenna array into a target-free region of interest (e.g., by removing the target and orienting the antenna array into the air) and capture a CI (which is effectively a "background CI" unaffected by the target) for each transmit and receive antenna pair associated with each TSCI. Multiple measurements can be taken, and characteristic values ​​(e.g., simple average, weighted average, trimmed average, median) can be used as robust estimates of the background CI. This process of acquiring the background CI is called "calibration" and can be performed once. Calibration may be self-calibration, as the system can detect whether there is a target within the region of interest. Remote objects are typically outside the region of interest and may not affect material sensing.

[0387] The background CI may be scaled and then subtracted from its respective TSCI. The scaling factor can be obtained by a best fit (e.g., mean squared error, mean absolute error, etc.) within a window that directly encompasses the leakage profile (which may include previous CIs but not future CIs to avoid including the effects of multipath from the target plane). Alternatively, the background CI (for each link or pair of transmit and receive antennas) may be replaced by the obtained / calculated common background CI for all pairs of transmit and receive antennas.

[0388] In some embodiments, the distance from the antenna array to the target surface ("target distance") is required to calculate the MRF. To find the distance, a search is performed for each transmit / receive antenna pair to find a tab with the maximum CI amplitude in the TSCI after subtracting the background (called the "target reflection amplitude," which is the amplitude of the reflection at the target material surface), and the time of flight (ToF) can be obtained from the tab. The target distance can be calculated based on the distance between the transmit (Tx) antenna and the receive (Rx) antenna (or the time of flight in the LOS path between the Tx and Rx antennas, i.e., the time after the millimeter-wave signal is transmitted and before it is received), the ToF, and the correction factor.

[0389] For each Tx / Rx antenna pair, the MRF (Multi-Range Frequency) is calculated based on the amplitude of the target reflection, the target distance, the Tx antenna gain (from the Tx chip, due to the Tx radio's automatic gain control / AGC), and the Rx antenna gain (from the Rx chip, due to the Rx radio's AGC). The MRF may also be calculated as the product of the target reflection amplitude and the target distance, divided by the Tx and Rx antenna gains.

[0390] The combined MRF may be calculated as a representative value of the MRF for each pair. The representative value may be the mean, arithmetic mean, geometric mean, harmonic mean, weighted mean, trimmed mean, median, maximum likelihood (ML) value, maximum posterior probability (MAP) value, expected value / statistical mean, conditional mean, or another representative value.

[0391] The mmWave probe signal may be transmitted multiple times (each time referred to as a "frame"). Each frame may correspond to a different target distance, as the user may move the antenna array and / or the target material surface may move or vibrate. In some embodiments, the total MRF may be calculated as representative of the combined MRF.

[0392] The rapid increase in automation has spurred the development of more efficient and convenient approaches for human-computer interaction (HCI). Touchscreens and smart surfaces (e.g., electronic whiteboards) have emerged as more user-friendly alternatives to traditional input devices such as keyboards and computer mice. However, smart surfaces are typically small shape factors that limit space for HCI and ubiquitous smart environments. Handwriting is a human-friendly form of interaction and can be considered a more common form of gesture, making it a promising technique for HCI. Advanced handwriting recognition systems that enable robust and accurate handwriting tracking can realize countless applications in the HCI field. Handwriting tracking involves tracking and reconstructing the trajectory traced by a writing instrument or writing target (e.g., finger, stylus pen, or marker) and can be achieved actively or passively. Active systems require measuring the moving target using sensors such as smartphone accelerometers, RFID tags, or other radios. In contrast, passive tracking does not involve any electronic devices attached to the target. Camera-based approaches dominate passive tracking systems, but they impose limitations on the availability of ambient light and raise privacy concerns. Other modalities, such as acoustic signals, have been used to distinguish hand movement patterns. However, their performance degrades significantly with distance, and they typically require retraining for each new alphabet and surface.

[0393] Wireless signals, par...

Claims

1. A system for wireless material sensing, A transmitter configured to transmit a first radio signal over a location's radio multipath channel using multiple transmitting antennas, A receiver configured to receive a second radio signal via the radio multipath channel using multiple receiving antennas, wherein the second radio signal includes the reflection or refraction of the first radio signal at the material surface of a target object at the location, and the material surface of the target object is the surface of the target material of the object, A processor that acquires a plurality of channel information (CIs) of the wireless multipath channel based on the second radio signal, wherein each CI is associated with one of the plurality of transmitting antennas of the transmitter and one of the plurality of receiving antennas of the receiver, and each CI includes at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), Calculating material analysis based on the aforementioned multiple CIs, A processor configured to determine the type of material of the target of the object based on the material analysis, A system that includes this.

2. The system according to claim 1, The first wireless signal is a wireless probe signal, The material analysis is a system associated with the reflection of the wireless probe signal on the material surface of the target.

3. The system according to claim 2, The CI associated with the transmitting and receiving antennas includes a CIR containing several tabs. The aforementioned material analysis is, The amplitude of the CIR tab associated with the reflection of the wireless probe signal on the material surface of the target, The distance between the transmitting antenna and the material surface of the target, The distance between the receiving antenna and the material surface of the target, The transmitting antenna gain associated with transmitting the first radio signal with the transmitter, or A system calculated based on at least one of the receiving antenna gains associated with receiving the second radio signal in the receiver.

4. The system according to claim 2, Each CI contains a CIR with several tabs, A system in which the plurality of transmitting antennas of the transmitter and the plurality of receiving antennas of the receiver are located in line-of-sight (LOS) such that the second radio signal includes a line-of-sight (LOS) component of the radio probe signal, and each CIR includes a tab having an amplitude corresponding to the LOS component of the radio probe signal.

5. The system according to claim 4, The transmitter and the receiver are physically connected to each other. The plurality of transmitting antennas and the plurality of receiving antennas are adjacent to each other. The material analysis is proportional to the product of (a) the amplitude of the CIR tab associated with the reflection and (b) the distance between the material surface of the target and the transmitting and receiving antennas. The material analysis is inversely proportional to the product of (c) the transmitting antenna gain associated with transmitting the first radio signal in the transmitter and (d) the receiving antenna gain associated with receiving the second radio signal in the receiver.

6. The system according to claim 5, wherein the processor is All CIRs are interpolated using interpolation coefficients. A system further configured to calculate the material analysis based on the interpolated CIR.

7. The system according to claim 6, wherein the processor is Synchronize all interpolated CIRs, A system further configured to calculate the material analysis based on the synchronized CIR.

8. The system according to claim 7, wherein the processor is For each CIR, Each background CIR is synchronized with the aforementioned CIR, The synchronized background CIR is scaled, The CIR with each background subtracted is calculated by subtracting the scaled and synchronized background CIR from the CIR. A system further configured to calculate the material analysis based on the CIR after subtracting the aforementioned multiple backgrounds.

9. The system according to claim 6, wherein the processor is A system further configured to align the CIRs based on LOS tabs, where each LOS tab is a CIR tab having a size corresponding to the LOS component of the radio probe signal.

10. The system according to claim 9, wherein the processor is One of the aforementioned CIRs is determined to be a reference CIR, Determine the reference window of the tab in the reference CIR that contains at least one LOS tab, It is further configured to calculate the matching score between the reference window of the reference CIR and the sliding window of the additional CIR in order to determine the best-match sliding window with the highest matching score. The tab position of the sliding window is within the search range with respect to the tab position of the reference window. The matching score is calculated based on at least one of the following: cross-correlation, cross-covariance, L1 distance, absolute difference, L2 distance, Euclidean distance, or Lk distance. Based on the reference window and the best-match sliding window, the additional CIR is aligned with the reference CIR. A system that rotates the additional CIR so that the best-matching sliding window is moved to the same tab position as the reference window.

11. The system according to claim 9, wherein the processor is One of the aforementioned CIRs is determined to be the reference CIR, Calculate the typical tab position of the aforementioned reference CIR, It is configured to calculate the representative tab position of an additional CIR, and the representative tab position of each corresponding CIR is: The corresponding CIR is normalized, Based on the tab index of the normalized CIR and the tab size of the normalized CIR, a probability density function is generated. Based on the representative tab positions of the reference CIR and the additional CIR, the additional CIR is aligned with the reference CIR. A system calculated by rotating the additional CIR such that the representative tab position of the additional CIR is close to the representative tab position of the reference CIR.

12. The system according to claim 8, The transmitter is further configured to use the plurality of transmitting antennas to remove the object from the area of ​​interest and transmit additional radio probe signals over the radio multipath channel of the location. The receiver is further configured to receive the additional probe signals via the wireless multipath channel using the plurality of receiving antennas, The processor is further configured to acquire the plurality of background CIRs of the radio multipath channel based on the additional radio probe signals, and both each CIR and each background CIR subtracted from each CIR are associated with the same transmitting antenna and the same receiving antenna, in a system.

13. The system according to claim 12, wherein the processor is Based on the additional wireless probe signals, multiple test CIRs of the wireless multipath channel are acquired. Based on the multiple test CIRs for generating the judgment result, it is determined whether there is an object within the scope of interest, If the determination result indicates that there are no objects within the scope of interest, the multiple background CIRs are automatically acquired based on the multiple test CIRs. The system is further configured to notify the transmitter to transmit another radio probe signal to acquire another set of test CIRs when the determination result indicates that an object is within the range of interest.

14. The system according to claim 8, wherein the processor is It is further configured to scale all tabs of the synchronized background CIR by a scaling factor, The scaling coefficient is selected to maximize the matching score between the CIR and the scaled and synchronized background CIR. The matching score is calculated based on at least one of the following: cross-correlation, cross-covariance, L1 distance, absolute difference, L2 distance, Euclidean distance, or Lk distance.

15. The system according to claim 8, wherein the processor is The system is further configured to calculate each of several candidate material analyses based on the CIR after subtracting the respective backgrounds, and the material analysis is an aggregate analysis of the several candidate material analyses.

16. The system according to claim 15, wherein the calculation of the analysis of each candidate material is For each transmitting antenna and each receiving antenna, The CIR, with the background subtracted, is used to determine whether it is associated with the transmitting antenna and the receiving antenna using the first CIR tab associated with the LOS component of the wireless probe signal. Identify the maximum size of the CIR after subtracting the background following the first CIR tab, and the maximum size is associated with the second CIR tab of the CIR after subtracting the background. A system for calculating the candidate material analysis based on the second CIR tab and the maximum size.

17. The system according to claim 16, wherein the calculation of the analysis of each candidate material is Regarding the aforementioned transmitting antenna and receiving antenna, Determine the first distance between the transmitting antenna and the receiving antenna. Based on the sampling frequency of the CIR after subtracting the background, the time difference between the first CIR tab and the second CIR tab is calculated. A second distance is calculated based on the product of the time difference and the propagation speed of the wireless probe signal at the location. A third distance is calculated based on the weighted sum of the first distance and the second distance. A system further comprising calculating the candidate material analysis based on the maximum size and the third distance.

18. The system according to claim 1, The transmitter is further configured to transmit a third radio signal via the wireless multipath channel of the location using the plurality of transmitting antennas. The receiver is further configured to receive a fourth radio signal via the radio multipath channel using the plurality of receiving antennas, the fourth radio signal including the reflection or refraction of the third radio signal at the material surface of the target of the object at the location. The aforementioned processor, Based on the fourth radio signal, a plurality of additional CIs of the radio multipath channel are obtained, each additional CI being associated with one of the plurality of transmitting antennas and one of the plurality of receiving antennas, each additional CI including at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI), Based on the aforementioned multiple additional CIs, an additional material analysis is calculated, Based on the aforementioned material analysis and the aforementioned additional material analysis, the aggregate material analysis is calculated. A system further configured to determine the type of material of the target object based on the aggregate material analysis.

19. The system according to claim 1, During the training phase of the aforementioned system, for each of the multiple training materials, The transmitter is configured to transmit a first wireless training signal via the training wireless multipath channel of the training location using the plurality of transmitting antennas. The receiver is configured to receive a second wireless training signal via the training wireless multipath channel using the plurality of receiving antennas, the second wireless training signal includes the reflection or refraction of the first wireless training signal at the surface of the training material at the training location. The aforementioned processor, Based on the second wireless training signal, a plurality of training CIs of the training wireless multipath channel are acquired, each training CI being associated with one of the plurality of transmitting antennas and one of the plurality of receiving antennas, and each training CI includes at least one of channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI). Based on the aforementioned multiple training CIs, the training material analysis is calculated, During the training phase of the aforementioned system, a classifier is trained based on machine learning and the analysis of the training materials for each training material. The system is configured, in the operating phase, to determine the type of the target material of the object based on the classifier.

20. A method for a wireless material sensing system, Using N1 transmitting antennas of the transmitter, a first radio signal is transmitted via the location's radio multipath channel, Receiving a second radio signal via the radio multipath channel using N2 receiving antennas of the receiver, wherein the second radio signal includes the reflection or refraction of the first radio signal at the surface of the target material of the object at the location, and N1 and N2 are positive integers. Acquiring multiple channel information (CI) of the radio multipath channel based on the second radio signal, wherein each CI is associated with one of each of the N1 transmitting antennas and one of each of the N2 receiving antennas, and each CI includes at least one of the following: channel state information (CSI), channel impulse response (CIR), channel frequency response (CFR), or received signal strength index (RSSI). Calculating material analysis based on the aforementioned multiple CIs, A method comprising determining the type of material of the target of the object based on the material analysis.

Images