Radio frequency sensing in a television environment

Abstract

Techniques are provided for performing radio frequency (RF) sensing to determine a viewing status of a television user. This can be used to determine user behavior during playback of content (e.g., whether the user is looking at the content), which can be used as a data point in determining a level of interest of the user in the content. Using the status of the television user, embodiments can provide additional or alternative functionality, such as turning the television off and / or on. Furthermore, the RF sensing can be performed by existing television hardware (such as a Wi-Fi transceiver), so the RF sensing functionality can be provided to the television with little or no added cost.

Description

Technical Field

[0001] This invention generally relates to object or motion detection, and more specifically, to the use of radio frequency (RF) sensing of objects or motion in a television environment. Background Technology

[0002] As televisions become increasingly complex, consumers are able to stream content from a multitude of content providers. These providers can further customize content based on user profile information and behavior. While modern televisions are highly sophisticated, the collection of user behavior data is typically limited to viewing history and input during playback (pause, rewind, fast forward, etc.). These coarse mechanisms are universal to all viewers and provide little information to content providers. While televisions could integrate cameras to view user behavior during playback, such integration could increase the cost of these televisions and raise privacy concerns among consumers. Summary of the Invention

[0003] The embodiments described herein address these and other problems by providing RF sensing to determine the status of the television user. This can be used to determine user behavior during content playback (e.g., whether the user is watching the content), which can serve as data points to determine the user's level of interest in the content. Using the television user's status, the embodiments can provide additional or alternative functionalities, such as powering down and / or powering up the television. Furthermore, RF sensing can be performed by existing television hardware (such as Wi-Fi transceivers), thus providing RF sensing functionality to the television with little or no additional cost.

[0004] According to this disclosure, an example method for radio frequency (RF) sensing of a television user includes transmitting a first RF signal using one or more wireless transceivers. The method also includes receiving a first reflected RF signal using the one or more wireless transceivers, the first reflected RF signal including reflections of the first RF signal from one or more objects. The method further includes determining first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal. The method also includes determining condition information based on the first CSI, wherein the condition information may include information about the viewing status of the television user. The method further includes performing an action on the television based on the condition information.

[0005] According to this disclosure, an example device for radio frequency (RF) sensing for a television user includes one or more wireless transceivers, a memory, and one or more processing units communicatively coupled to the one or more wireless transceivers and the memory. The one or more processing units are configured to transmit a first RF signal using the one or more wireless transceivers. The one or more processing units are also configured to receive a first reflected RF signal using the one or more wireless transceivers, the first reflected RF signal including reflections of the first RF signal from one or more objects. The one or more processing units are further configured to determine first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal. The one or more processing units are further configured to determine status information based on the first CSI, wherein the status information may include information related to the viewing status of the television user; and to perform actions on the television based on the status information.

[0006] According to this disclosure, another example device for radio frequency (RF) sensing for a television user includes components for transmitting a first RF signal. The device also includes components for receiving a first reflected RF signal, which may include reflections of the first RF signal from one or more objects. The device further includes components for determining first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal. The device also includes components for determining status information based on the first CSI, wherein the status information may include information about the viewing status of the television user. The device further includes components for performing actions on the television based on the status information.

[0007] According to this disclosure, an example of a non-transitory computer-readable medium stores instructions for radio frequency (RF) sensing for a television user. The instructions include code for transmitting a first RF signal using one or more wireless transceivers. The instructions also include code for receiving a first reflected RF signal using one or more wireless transceivers, the first reflected RF signal potentially including reflections of the first RF signal from one or more objects. The instructions further include code for determining first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal. The instructions also include code for determining status information based on the first CSI, wherein the status information may include information about the viewing status of the television user. The instructions further include code for performing actions on the television based on the status information. Attached Figure Description

[0008] Figure 1 This is a block diagram of an example radio frequency (RF) sensing system capable of performing RF sensing in a television environment.

[0009] Figure 2 and Figure 3This is a perspective view of an example television environment.

[0010] Figures 4A-4C This is an illustration of a television user in a television environment from the perspective of a television, according to an embodiment.

[0011] Figure 5 This is a flowchart illustrating the process of determining a television user's interest in the content being broadcast on a television, according to an embodiment.

[0012] Figures 6A-6C This diagram illustrates different ways user interests can be recorded according to different embodiments.

[0013] Figure 7 This is a flowchart illustrating a process for turning off a television based on information obtained from RF sensing, according to an embodiment.

[0014] Figure 8 This is a flowchart illustrating the process of turning on a television based on information obtained from RF sensing, according to an embodiment.

[0015] Figure 9 This is a flowchart illustrating a method 900 for RF sensing of a television user according to an embodiment.

[0016] Figure 10 This is a block diagram of an embodiment of a computer system that can be used in the embodiments described herein.

[0017] According to certain exemplary embodiments, the same reference numerals in various figures indicate the same elements. Furthermore, multiple instances of an element can be indicated by a first digit followed by a letter or hyphen and a second digit. For example, multiple instances of element 110 can be represented as 110-1, 110-2, 110-3, etc., or 110a, 110b, 110c, etc. When such an element is referred to using only the first digit, any instance of that element will be understood (e.g., element 110 in the preceding examples would refer to elements 110-1, 110-2, and 110-3, or elements 110a, 110b, and 110c). Detailed Implementation

[0018] For the purpose of describing the innovative aspects of this disclosure, the following description is directed to certain embodiments. However, those skilled in the art will readily recognize that the teachings herein can be applied in a variety of different ways. Some examples in this disclosure may be based on wireless local area network (WLAN) communication according to the Institute of Electrical and Electronics Engineers (IEEE) 802.11 wireless standard, including those identified as Wi-Fi technology. However, the described embodiments can be implemented in any device, system, or network capable of transmitting and receiving radio frequency (RF) signals according to any communication standard such as any IEEE 802.11 standard, Standards, Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), GSM / General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunking Radio (TETRA), Wideband-CDMA (W-CDMA), Evolved Data Optimized (EV-DO), 1xEV-DO, EV-DO Rev A, EV-DO Rev B, High-Speed ​​Packet Access (HSPA), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolved High-Speed ​​Packet Access (HSPA+), Long Term Evolution (LTE), AMPS, or other known signals used for communication within wireless, cellular, or Internet of Things (IoT) networks, such as systems using 3G, 4G, 5G, 6G, or further implementations thereof.

[0019] As used herein, an "RF signal" includes electromagnetic waves that transmit information through space between a transmitter (or transmitting device) and a receiver (or receiving device). As used herein, a transmitter may send a single "RF signal" or multiple "RF signals" to a receiver. However, due to the propagation characteristics of RF signals through multipath channels, a receiver may receive multiple "RF signals" corresponding to each transmitted RF signal. The same RF signal transmitted on different paths between the transmitter and receiver can be referred to as a "multipath" RF signal.

[0020] As described above, RF signals can be used for RF sensing. RF signals with relatively high frequencies, such as 2.4 GHz, 5 GHz, or 6 GHz (typically used in WLAN implementations), have sufficiently small wavelengths to provide resolution capable of detecting the presence of an object (e.g., based on the volume occupied by the object and / or the movement made by the object) and identifying the object. Furthermore, such RF sensing can be implemented using existing Wi-Fi / IEEE 802.11 / WLAN transceivers used for communication. Therefore, RF sensing can be implemented with minimal or no increase in television cost using these types of existing transceivers. Even more significantly, RF sensing is already implemented in the television in the field via firmware updates. That is, RF sensing can be implemented using additional or alternative transceivers. For example, according to some embodiments, the transceiver can be located in a separate device (referred to herein as a “connected device”) that is communicatively coupled to the television, such as a set-top box, a television streaming device (e.g., Google Chromecast TM Devices, Amazon Fire (etc.), video game systems, etc.

[0021] Figure 1 This is a block diagram of an example RF sensing system 105 capable of performing RF sensing in a television environment, as described herein. In short, the RF sensing system 105 uses one or more RF signals, including one or more waveforms, sequences, or packets, to determine the presence and / or movement of an object. This can be accomplished by using RF signals for channel acquisition, performing channel estimation to obtain a channel impulse response (CIR), channel frequency response (CFR), and / or other forms of channel state information (CSI) indicating the presence and / or movement of an object. CSI indicates various aspects of the RF signal, such as multipath, reflections along each path, and signal strength. More broadly, CSI can represent, for example, a combination of effects such as scattering, decay, and power attenuation with distance. According to some embodiments, CSI can be obtained using channel estimation used in the form of wireless communication systems (in wireless communication, CSI is used for proper demodulation and decoding of received packets). Therefore, embodiments can utilize existing channel estimation techniques to obtain CSI for RF sensing purposes.

[0022] More specifically, the RF sensing system 105 can obtain a CSI associated with the RF signal 112, which is reflected from the object 110. According to some embodiments, the RF sensing system 105 can use this CSI to calculate the distance and angle of arrival corresponding to the reflected signal received by the Rx antenna 120. For example, the RF sensing system 105 can determine the distance by calculating the time of flight of the reflected signal based on the difference between the leaked signal (not shown) and the reflected signal. In a further example, the RF sensing system 105 can determine the angle of arrival by utilizing an antenna array (e.g., the RX antenna 120) to receive the reflected signal and measuring the received phase difference at each element of the antenna array.

[0023] As shown in more detail below, the RF sensing system 105 can use the distance and / or angle of arrival corresponding to the reflected signal to detect the presence or movement of an object (e.g., a television user) at a location and / or the user's head orientation, eye orientation, body position, etc.

[0024] In some embodiments, the RF sensing system 105 may utilize artificial intelligence or machine learning algorithms to perform motion detection, object classification, head / eye orientation detection, and / or body position determination. In some examples, the machine learning technique may include supervised machine learning techniques, such as those utilizing neural networks, linear and logistic regression, classification trees, support vector machines, any other suitable supervised machine learning techniques, or any combination thereof. For example, a dataset of sample RF sensing data may be selected to train the machine learning algorithm or artificial intelligence.

[0025] The RF sensing techniques described herein can be performed regardless of their association with a Wi-Fi network. For example, RF sensing system 105 can perform RF sensing as discussed herein using its Wi-Fi transmitter and Wi-Fi receiver without being associated with any access point or Wi-Fi network.

[0026] The RF sensing system 105 may include a standalone device or may be integrated into a television or connected device. For example, the RF sensing system 105 may be integrated into a WLAN radio of a television or connected device. Example components of an electronic device including an RF sensing system are described below. Figure 10 The following is illustrated and discussed in detail. As indicated in more detail below, certain embodiments may be implemented such that RF signals are transmitted by one device and received by another device.

[0027] Generally speaking, regarding Figure 1 The RF sensing system 105 described herein functions by detecting an object 110 by generating an RF signal (e.g., including one or more pulses) transmitted by one or more Tx antennas 115, which is reflected from the object 110 and received by one or more Rx antennas 120. The RF sensing system 105 then processes the received signal using digital signal processing (DSP) techniques (including leakage cancellation) to determine the object's presence and / or range. The process of transmitting, receiving, and processing signals is generally referred to herein as an RF sensing "scan." As discussed in more detail herein, the frequency or period of a scan can vary depending on the type of transmission mode (e.g., low-resolution or high-resolution scan), but is typically a few times per second.

[0028] In some embodiments, the RF sensing system 105 may have multiple Rx antennas 120. For example, a WLAN radio typically has two to four antennas. In this embodiment, CSI received at different Rx antennas 120 can be used to determine angular information (e.g., by using Rx beamforming, determining the phase difference of angular information, etc.). In some implementations, for example, embodiments with two antennas have achieved an angular granularity of 10° to 15°, and embodiments with four antennas have achieved a granularity of 2° to 3°. Furthermore, in some other embodiments, the RF sensing system 105 may have multiple Tx antennas 115. For example, a WLAN radio typically has two to four antennas. In this embodiment, the phase of the Tx antennas can be configured to transmit RF signals in a beam pointing in a certain direction. In some implementations, for example, embodiments with two Tx antennas have achieved an angular granularity of 10° to 15°, and embodiments with four Tx antennas have achieved a granularity of 2° to 3°. The change of CSI over time (e.g., from one scan to the next) indicates the movement of object 110. The changes here may include variations in the amplitude or phase of the CSI. Additionally or alternatively, the changes may include variations in metrics extracted and / or estimated from the CSI, such as the time of flight and angle of each reflection path. Therefore, the RF signal can be used to determine the location, volume, and movement of an object.

[0029] This functionality of the RF sensing system 105 is achieved using a processing unit 125, a memory 130, a multiplexer (mux) 135, a Tx processing circuit 140, and an Rx processing circuit 145. The RF sensing system 105 may include additional components not shown, such as a power supply, user interface, or electronic interface. However, it can be noted that in alternative embodiments, these components of the RF sensing system 105 may be rearranged or otherwise modified, depending on the desired functionality. Furthermore, as used herein, the terms "transmitting circuit," "Tx circuit," or "Tx processing circuit" refer to any circuitry used to generate and / or transmit RF signals. Similarly, the terms "receiving circuit," "Rx circuit," or "Rx processing circuit" refer to any circuitry used to detect and / or process the RF signal. Thus, the "transmitting circuit" and "receiving circuit" may include not only the Tx processing circuit 140 and the Rx processing circuit 145, respectively, but also the mux 135 and the processing unit 125. In some embodiments, the processing unit 125 may constitute a modem and / or wireless communication interface (e.g., described below). Figure 10At least a portion of the wireless communication interface 1033 shown herein. In some embodiments, more than one processing unit may be used to perform the functions of the processing unit 125 described herein. Additionally, although the Tx antenna 115 and Rx antenna 120 are shown as separate antennas, some embodiments may use the same one or more antennas for both transmission and reception.

[0030] Tx processing circuit 140 and Rx processing circuit 145 may include sub-components for generating and detecting RF signals, respectively. As those skilled in the art will understand, Tx processing circuit 140 may therefore include a pulse generator, a digital-to-analog converter (DAC), a mixer (for upmixing the signal to the transmission frequency), one or more amplifiers (for powering the transmission via Tx antenna 115), etc. Rx processing circuit 145 may have similar hardware for processing the detected RF signal. Specifically, Rx processing circuit 145 may include an amplifier (for amplifying the signal received via Rx antenna 120), a mixer for downconverting the received signal from the transmission frequency, a digital-to-analog converter (ADC) for digitizing the received signal, and a pulse correlator for providing a matched filter for the pulse generated by Tx processing circuit 140. Rx processing circuit 145 can therefore use the correlator output as a CIR, which can be processed by processing unit 125 (or other circuitry) for, for example, leakage elimination. Other processing of the CSI obtained from the RF signal, such as object detection, range, motion, direction of departure (DoD), or direction of arrival (DoA) estimation, may also be performed.

[0031] It can be noted that the characteristics of the transmitted RF signal 112 can vary depending on the technology used. As previously stated, the technology provided herein can be applied to WLAN technologies that typically operate at 2.4, 5, and 6 GHz, but may include frequencies ranging from 900 MHz to 60 GHz. (That is, some embodiments may use RF frequencies outside this range.) For example, this includes the frequencies used by the 802.11ad Wi-Fi standard (operating at 60 GHz). Because RF sensing can be performed in the same frequency band as communication, the hardware can be used for both communication and RF sensing. For example, Figure 1One or more components of the RF sensing system 105 shown may be included in a television’s wireless modem (e.g., a Wi-Fi or 5G modem). That is, embodiments may utilize the RF sensing system 105 independently of any such communication component. As described above, for example, some embodiments may utilize an ultra-wideband (UWB) transceiver. The described techniques for RF sensing can utilize various types of RF signals, such as Zadoff sequences, orthogonal frequency division multiplexing (OFDM) long training field (LTF) class symbols, for channel acquisition to determine the presence and / or movement of the object 110. Because the RF sensing system may be able to transmit RF signals for communication (e.g., using 802.11 communication technology), embodiments may utilize channel estimation used in communication to obtain CSI for performing RF sensing as provided herein. In Wi-Fi, channel estimation can be accomplished using conventional long training field (L-LTF) and high throughput (HT) / very high throughput (VHT) / efficient long training field (HE-LTF) in the preamble of communication packets. Embodiments may use similar methods, for example, to perform channel estimation using known transmitted signals. Therefore, RF signals may include the same wireless pulses and / or packets used for channel estimation in communications.

[0032] As described above, the embodiments described herein address RF sensing in a television environment, enabling the television (and / or connected devices) to provide additional functionality, such as determining a user's interest in television content, automatically turning the television on and / or off. This avoids the privacy issues, hardware costs, and power consumption associated with using a camera.

[0033] Figure 2 This is a perspective view of an example television environment 200 illustrating, according to one embodiment, how RF sensing can be used to detect one or more television users 210-1, 210-2 (collectively referred to herein as television user 210). Here, a first transceiver 220 (which includes an RF sensing system 105 incorporated into the circuitry of the television 230) can transmit RF signals that are reflected from objects in the television environment 200 and received by a second transceiver 240 (illustrated as a media streaming device inserted into the television 230). The location of the first transceiver 210 is shown in dashed lines, illustrating an example location behind the display of the television, where the first transceiver 210 can be housed within the television. A processor or computer, such as a processor within the television 230, communicatively coupled to the first transceiver 210 and the second transceiver 240, can coordinate the timing of the transmission and reception of the RF signals. The first transceiver 210 and the second transceiver 240 can be communicatively linked to and / or incorporated into the electronic hardware of the television 230. As mentioned earlier, the example electronic hardware is in Figure 10 It is shown in the figure and described in more detail below.

[0034] As a specific example of how the first television user 210-1 is detected, a portion of the RF signal traveling along the RF signal path 250 is reflected from the first television user 210-1. These reflections of the RF signal are received by the second transceiver 240. As previously described, these reflections can be identified in captured RF sensing data (such as CSI) and used to determine the presence of the television user 210-1 by comparing the RF sensing data (including the captured CSI and / or information derived therefrom) with previously acquired RF sensing data in which the television user 210-1 is not present (e.g., CSI captured during the calibration process). The presence of a human user can be determined, for example, based on CSI-derived information indicating the detection of a human-sized object (based on one or more dimensions of the object), which can be verified as a human user during calibration. Additionally or alternatively, movement of the first television user 210-1 can be detected by determining changes in continuously captured RF sensing data (e.g., from continuous scans). Such variations can include, for example, variations in CSI amplitude, CSI phase, angle extracted from CSI, time of flight extracted from CSI, Doppler amplitude extracted from CSI, or any combination thereof. Furthermore, RF sensing data with multiple spatial flows and / or relatively high bandwidth can be used to determine objects and / or smaller motions and / or finer details than RF sensing data with fewer spatial flows and / or relatively low bandwidth.

[0035] It can be noted that, Figure 2 The configuration shown is provided as a non-limiting example. Additionally or alternatively, a second transceiver 240 external to the television set may transmit RF signals, and the first transceiver 210 may receive reflected RF signals. Alternative configurations may have multiple transceivers external to the television set 230, multiple transceivers internal to the television set 230, or any combination of internal / external transceivers. Furthermore, although generally described herein as "transceiver," alternative embodiments may use transmitters and / or receivers in the manner described herein.

[0036] Figure 3 Another embodiment illustrates how RF sensing can be performed, similar to... Figure 2 A perspective view of an example television environment 300. Here, the television 230 does not have a separate transceiver, but rather a single transceiver 310. In this embodiment, the transceiver 310 may include an RF sensing system 105 and may perform... Figure 2The first transceiver 210 and the second transceiver 220 both function to transmit and receive RF signals and process RF signals reflected from television user 210-1 along the RF signal path 330 (as well as processing reflected signals from other objects). Those skilled in the art will understand that because transceiver 310 can perform both transmitting and receiving functions simultaneously, it can implement leakage mitigation and / or similar algorithms to help minimize interference between the transmitting and receiving functions. Therefore, in Figure 2 In the television environment 200, (between the first transceiver 210 and the second transceiver 240) channel capture can be used to capture CSI and perform RF sensing, while Figure 3 In the television environment 300, a single transceiver 310 can capture CSI and perform RF sensing.

[0037] Whether in a single transceiver environment (e.g., Figure 3 Is it a television environment (300) or a multi-transceiver environment (e.g., Figure 2 In a television environment 200, the determination of the presence of a first television user 210-1 and the distinction between the first television user 210-1 and a second television user 210-2 and other objects can be achieved in part through calibration and filtering. For example, television 230 can instruct an authorized user (e.g., a user authorized to change television settings and / or otherwise manage the television) to perform transceiver calibration to identify RF signal reflections from television user 210 (e.g., at the location where they are watching television 230) and ignore RF signal reflections from other objects (pets, tables, chairs, etc.). After calibration, RF sensing can be performed to identify the presence of television user 210 by comparing RF sensing data (e.g., CSI) from the reflections of RF signals with the RF sensing data obtained during calibration. Reference Figures 4A-4C Additional details regarding calibration are provided.

[0038] Figures 4A-4C It is from the angle of TV 230 (not shown). Figure 2 A diagram of television user 210 in a television environment 200. As mentioned earlier, television 230 may not be equipped with a camera, therefore television may not... Figures 4A-4C The television user 210 is "seen" in the manner shown. However, as previously stated, RF sensing in the manner described herein can determine movement in the television environment, the presence of one or more television users 210 and other objects, and the condition or state of one or more television users 210. This can be accomplished by utilizing information obtained during the calibration process.

[0039] During calibration, the television can guide television user 210 and / or other authorized users through a process (e.g., using an on-screen menu), during which RF sensing data (such as CSI or CIR) is acquired as television user 210 is at the viewing location and viewing position. For example, Figure 4A It shows the relationship with Figure 2 In a similar scenario, two users, 210-1 and 210-2, are sitting on a sofa with their heads facing the television. During calibration (which can be performed for one user at a time), television user 210 may sit in a similar position near these locations. RF sensing data, including CSI or information derived from it (e.g., volumetric information of television user 210), can then be compared with corresponding RF sensing data from CSI obtained at a later time to determine whether the user is in the desired location and / or desired position at a later time.

[0040] Calibration may require capturing CSI from users in various locations and at various positions. For example, a single TV user 210 may be asked to sit in a different location where they typically watch TV. This allows the TV (or other device performing RF sensing) to identify instances where one or more TV users 210 are watching TV during subsequent operations (e.g., as shown in the image). Figure 4A (in China), and in places such as Figure 4B The scenario shown ignores television users, where television users 210-3 are in different locations.

[0041] Implementation examples can further identify situations, such as Figure 4C The scenario illustrated involves a television user 210-4 in a location where they can watch television, but not in a position indicating they are watching television. Here, for example, the television user's head is turned to one side, so their attention is likely not on the content being broadcast. According to some embodiments, this can be achieved by simply detecting the RF sensing data when the television user 210-4 is in that position and comparing it with the corresponding RF sensing data from the CSI obtained during calibration (when the television user 210-4 is in a "focused position" of watching television) (similar to...). Figure 4AThese situations are detected by the difference between the position of the television user 210 and the time of the event. As mentioned above, such data may include, for example, measurements of multipath, reflections of each path, and signal strength. If the variance of one or more of these measurements compared to the corresponding measurements obtained during calibration when the television user 210-4 is in an "attentional position" exceeds a threshold amount (e.g., a certain percentage), a determination can be made that the user is not in an attentional position. Additionally or alternatively, according to some embodiments, calibration may include obtaining RF sensing data from the CSI of the television user 210-4 in various "inattentive" positions (e.g., head turned to one side or downward), indicating that the television user 210-4 is not watching television, and such... Figure 4C The situation shown can be identified by comparing the RF sensing data (currently) obtained from CSI with the RF sensing data from CSI obtained when the TV user 210-4 was not in a position of focused attention.

[0042] However, it can be noted that while some embodiments may have a calibration process in which CSIs are obtained when the TV user 210 is in different locations and positions (e.g., attentive and inattentive positions), other embodiments may utilize crowdsourcing and / or machine learning to capture a minimal amount of CSIs. That is, a service provider (e.g., a TV manufacturer) can obtain CSI information from thousands or even millions of TV users 210. Using this information, the service provider can develop processing algorithms capable of “inferring” information from a minimal set of CSIs. For example, using CSIs from thousands of TV users 210 in “attentive” and “inattentive” positions, the service provider can train a machine learning algorithm to identify a TV user’s “inattentive” position based on the basic “attentive” position obtained by that TV user during calibration. The machine learning algorithm can then be used in subsequently manufactured TVs (or other devices) and / or distributed to TVs (or other devices) in the field via firmware updates over the Internet.

[0043] As mentioned above Figure 1As shown, RF sensing is based on the reflection of RF signals from the surface of an object. Based on the reflections at different angles (azimuth and elevation) and ranges, the location and volume of the object can be determined. As mentioned above, this can be used not only to determine whether television user 210 is in a specific location, but also to determine whether the television user is in a position indicating that he or she is watching television. Higher resolution implementations can obtain higher resolution RF scans capable of acquiring additional information. For example, implementations operating at 5 GHz or 6 GHz may be able to determine the dimensions of the television user's head, torso, and limbs based on the captured CSI. For example, an implementation using 60 GHz may be able to determine the position of the television user's eyeballs. Thus, calibration may require additional steps. For example, in an embodiment using 60 GHz, calibration may require capturing the CSI when the television user 210 is looking directly at the television, and optionally, capturing the CSI when the television user 210's gaze moves away from the television.

[0044] RF sensing data, including CSI and / or information extracted from captured CSI (e.g., volumetric information, eye-tracking information, etc.), can be stored in user profiles (e.g., locally stored on the TV and / or stored in the cloud by a service provider). Because the RF sensing data of different TV users 210 can be significantly different, it can be used to identify TV users 210 and distinguish them from other objects (such as pets). Calibration for different users and the setting of different user profiles can be used to allow the TV or connected devices to use RF sensing to identify TV users 210. The creation of new user profiles can be initiated by an authorized user and / or prompted by the TV (e.g., when a new, unidentified TV user is detected at the TV viewing location via RF sensing).

[0045] Each user profile can include information that enables the television (or other RF sensing device) to recognize the user. Therefore, this can include RF sensing data, including captured CSI (e.g., from calibrated RF sensing performed when the user profile was initially created) or information derived therefrom, such as volume information. This volume information can include overall volume information (e.g., height, width, etc.) and / or the volume of different parts of the user (e.g., head, torso, arms, legs), and the location of these volumes. That is, because RF sensing can sense more than just volume to identify a television user, other types of information for detecting and / or identifying the television user can be stored. For example, RF sensing can sense the breathing rate of different users, which can be used, for example, as additional data points to identify the television user. Furthermore, for high-frequency embodiments (e.g., 60 GHz or higher), specific facial features (eyes, nose, mouth, cheeks, etc.) can be identified, and recognition algorithms can be used to identify the face of a television user and distinguish it from the faces of other television users. The user profile can also include information about the television environment, such as the user's preferred seating location relative to the television environment. User profiles can include a user's preferred or usual posture while watching TV, such as sitting, lying down, etc. As mentioned above, when subsequent RF sensing is performed (e.g., using...), Figure 5 and Figures 7-9 The method discussed in the illustrated embodiments allows for the comparison of RF sensing data obtained during RF sensing with corresponding RF sensing data from a user profile to identify television users.

[0046] Additionally, user profiles may include information provided by the user and / or information about the user collected by RF sensing devices and / or service providers. For example, when creating a user profile for a new user, the television can guide the new user through an on-screen menu that allows the user to input different preferences. According to some embodiments, the on-screen menu may include, for example, a series of questions whose answers can indicate user preferences. Furthermore, these preferences can influence not only the main television content (movies, TV shows, etc.) but also the advertising content.

[0047] In some embodiments, certain user accounts may include child accounts. For example, by initiating a user profile creation process using a password, an authorized (adult) user can create a user profile for a child, providing similar information as when creating a user profile for an adult. However, the authorized user can additionally indicate (e.g., using non-screen menu items such as checkboxes, radio buttons, etc.) that the new user is a child. Therefore, the television can enable the authorized user to set content filtering, screen time limits, etc., on the child's user account. In subsequent use of the television, if the television detects the child using RF sensing, it can implement content filtering, restricting the content available to the child to view unless an adult is present and / or provides authorization (e.g., enters a password).

[0048] Once user profiles are created, TV functionality can be customized to suit the preferences of one or more users using the TV. For example, using RF sensing, one or more users can be identified, and on-screen menus can be customized to provide content based on user preferences. If more than one user is identified, content can be based on the preferences of multiple users (e.g., providing content and / or other menu options that can satisfy all user preferences). Therefore, menus and content can be customized for each user and / or user group.

[0049] As mentioned earlier, service providers can gather information from numerous television stations (thousands, millions, etc.) to perform crowdsourcing based on user-provided information. (For example, such crowdsourcing can be performed with the individual user's permission and in compliance with applicable laws regarding the protection of consumer information, etc.) Using user preferences and demographic information (gender, age, city / state / country of residence, etc.) obtained from users, service providers can be able to determine trends in different types of content across different demographic groups. These trends can then be used to recommend new content to television viewers within those demographic groups.

[0050] As described in the various embodiments below, a television (or a connected RF sensing device) may perform RF sensing scans at different times to detect the presence of a television user and, optionally, gauge the television user's interest in the content being broadcast on the television. User profiles may be stored in RF sensing data acquired during a setup process that calibrates RF sensing for a particular television user. User detection can be performed when new RF sensing data acquired during subsequent use of the television matches the stored RF sensing data in the user profile (e.g., within a threshold similarity, such as based on a similarity metric like Sum of Absolute Differences (SAD)). If the new RF sensing data is within a threshold similarity to stored RF sensing data from multiple user profiles, then the user with the stored RF sensing data that most closely matches the new RF sensing data is likely the user to be detected.

[0051] Depending on the required functionality, new RF sensing data can be compared and matched with stored RF sensing data in any of a variety of ways. In some embodiments, for example, one or more “features” can be extracted from the obtained CSI, such as an obtained 2D CSI map, which may include the aforementioned information (e.g., volume, respiratory rate, facial features, etc.). These features can be obtained from CSI, such as 2D angle of arrival (AoA) and time-of-flight (ToF) data or similar data. The distance between the measured features of the new RF sensing data and the corresponding features from the stored RF sensing data can be determined, and a similarity score can be generated using a similarity learning algorithm. The determination of whether a match exists can be based on whether the similarity score exceeds a threshold.

[0052] A threshold for the similarity score can be set to minimize the false positive rate while maintaining a high successful detection rate. According to some embodiments, multiple thresholds may exist to determine whether the similarity score indicates a match (e.g., exceeding a higher similarity value threshold), a mismatch (e.g., below a lower similarity value threshold), or that more testing is needed for confirmation (e.g., the similarity score is between a higher and a lower similarity value threshold). Based on some preliminary results, embodiments have been found to achieve a false detection rate of less than 1 in 100,000 over more than 99% of the time in face recognition.

[0053] Other embodiments may use other techniques to match new RF sensing data with stored RF sensing data. In some embodiments, for example, where processing power allows, machine learning can be used to determine if a match exists by feeding the increment between the CSI of the new RF sensing data and the CSI of the stored RF sensing data into a machine learning algorithm. Additionally or alternatively, filtering may be used to filter out uninteresting reflections and focus on the path reflected by the television user. Interpolation can then be used to recover higher resolution reflections, etc.

[0054] Depending on the required functionality, the stored RF sensing data may be refined over time. Subsequent calibrations may be obtained periodically and / or based on trigger events. In some embodiments, once a user is detected by matching new RF sensing data with stored RF sensing data from a user profile, values ​​in the stored RF sensing data can be updated based on the values ​​from the new RF sensing data (e.g., averaged over time). In this way, the RF sensing data can be updated to help ensure accurate data about the stored user and to accommodate changes that may occur in the user, such as a child growing over time. In some embodiments, before updating the RF sensing data stored in the user profile, the TV or connected RF sensing device may prompt the user to confirm (e.g., via an interactive TV menu) the user's identity (e.g., "Alice has been detected, please confirm."). In some embodiments, this additional confirmation and calibration may be performed after the initial calibration against the user profile until a threshold amount of time (e.g., several weeks) has elapsed to help ensure the accuracy of the RF sensing data stored in a user profile.

[0055] As previously Figures 1-4C The ability to perform RF sensing in the manner described and illustrated here enables televisions to provide functions related to object detection and / or motion detection that would otherwise require cameras, which are not only more expensive but may also compromise the privacy of television users.

[0056] This feature is TV user interest detection. Content has a higher value when it arouses a user's interest. Therefore, playing content that users are interested in not only provides more value to consumers but also to content providers, including advertisers. With this in mind, RF sensing can be performed during the playback of specific content on the TV to determine the level of interest of one or more users in the content being played. This information can be used locally by the TV and / or remotely by the content provider as feedback, enabling the TV / content provider to offer TV users content that is of interest.

[0057] Figure 5 This is a flowchart illustrating, according to an embodiment, the process of determining a television user's interest in the content being broadcast on a television. (Compared to other methods provided herein...) Figure 1 Sample, Figure 5 Provided as a non-limiting example. Alternative embodiments may be added, omitted, rearranged, and / or otherwise modified. Figure 5 The operation shown. In Figure 5 The RF sensing provided in the process shown can be provided by an RF sensing system, such as... Figure 1 The system shown can be used in, for example Figure 2 and Figure 3 In one or more transceivers shown. Figure 5 The implementation of the process shown can be executed by one or more processing units of a television or connected device. The processor (e.g., processing unit) and other example components of the television / connected device... Figure 10 It is shown in the figure and described below.

[0058] The process can begin at box 510, where it is determined whether certain content is being played back by the television. Here, "certain content" can include any content that helps determine the user's interests. Advertisements and other content (such as technical reports) can include content to help determine the user's interests. The type of content can be user-selectable (e.g., in the user menu) and can vary based on desired functionality, user preferences, etc.

[0059] The type of playback detected can be determined based on metadata about the media being played. For example, videos for video-on-demand (VOD) may include metadata containing a title, genre, etc. Additionally or alternatively, embodiments may include explicit data fields through which such videos can be tagged to determine user interests. For advertisements in a live stream, a digital "beep" in the stream may indicate the time period during which an advertisement can be played in the stream. In response to the beep, the television may play advertisements from an ad buffer and / or retrieve advertisements from an ad server in real time. If the advertisement is to identify a type of content that the user is interested in, some embodiments may include (as part of the process of playing an advertisement) implementation via the functionality proceeding to box 520. Figure 5 The functions within.

[0060] At box 520, the functionality includes performing high-resolution (“high-resolution”) RF sensing. In the current implementation, low-resolution (“low-resolution”) RF sensing is an RF sensing mode that captures CSI at a relatively low frequency (e.g., a period of 100 ms or more, alternatively 500 ms or more, alternatively 1 s or more, etc.), a relatively low bandwidth (e.g., 20 MHz or less, or 40 MHz or less), and / or a relatively few spatial streams (e.g., a single spatial stream). Depending on the frequency / bandwidth / number of spatial streams used, low-resolution RF sensing may only be able to detect motion, although some embodiments may be able to detect television users and / or other objects. Conversely, high-resolution RF sensing is an RF sensing mode in which CSI can be captured at a relatively high frequency (e.g., a period of 1 ms or less, alternatively 2 ms or less, alternatively 50 ms or less, etc.), a relatively high bandwidth (e.g., 80 MHz or more, or 160 MHz or more), and / or an increased number of spatial streams (e.g., two or more) relative to the number used in low-resolution detection. High-resolution RF sensing (e.g., to a period of 1 ms or less, alternatively 2 ms or less, alternatively 50 ms or less, etc.), a relatively high bandwidth (e.g., 80 MHz or more, or 160 MHz or more), and / or an increased number of spatial streams (e.g., two or more) relative to the number used in low-resolution detection. Figure 5The method performed at frame 520 can therefore enable the detection of movement and / or objects with higher accuracy than low-resolution detection. For example, high-resolution detection can determine the user's position, including sitting position / pose, head orientation, etc. Furthermore, in embodiments using sufficiently high frequencies, high-resolution detection can determine the eye position of one or more television users.

[0061] At box 530, this function includes determining the interest of one or more television users in a content segment based on high-resolution RF sensing. As described above, high-resolution RF scanning can be used to determine the location and position of one or more television users. By comparing RF sensing data (CSI or information extracted from it) with corresponding data for the user in “inattentive” and “attentive” positions, the television or other electronic device performing the RF sensing can determine the level of interest. For example, multiple scans (e.g., several times per second) can be performed on a given content segment. If the RF sensing data from most scans during that segment matches (and / or does not match) the RF sensing data obtained during calibration in which the television user is in an “inattentive” position, it can be determined that the user is not interested in that content segment. Alternatively, if the television user is determined to be “attentive” during X% of the scanning of the content segment, it can be determined that the television user shows X% interest in that content segment. Additional methods for determining and recording user interest are provided below.

[0062] At box 540, the television or other electronic device performing RF sensing determines whether the specific content identified at box 510 is still playing. If so, the process at boxes 520 and 530 can be repeated for each content segment until the content has finished playing. Once the content has finished playing, the result can be recorded, as shown in box 550.

[0063] The methods used to record RF sensing results for determining television user interests may vary depending on the specific functionality required. Figures 6A-6C This diagram illustrates different ways user interests can be recorded according to different embodiments.

[0064] Figure 6A A first type of viewership log 600-A according to an embodiment is shown. In this embodiment, entries summarizing viewership interest across all content segments are created. Figure 6AIn the specific example shown, the content may include a 30-second advertisement, divided into 30 one-second segments. The ratings log 600-A provides a cumulative summary, where the number of segments (seconds) where the TV user's "attention" is determined is summed, and this sum is included in that TV user's entry. This process is repeated for each detected TV user. Depending on the required functionality, a new ratings log 600 may be created for each content item, and / or a composite ratings log may include entries for multiple content items.

[0065] Figure 6B The second ratings log 600-B is shown, similar to Figure 6A The ratings log 600-6A is shown in the figure. However, the ratings log 600-B includes a "maximum interest segment" field (identified by the start and end times of the content) that identifies the longest continuous content segment in which the corresponding TV user is determined to be in a "focused" position, instead of including a "duration of interest" summary of content segments (as included in ratings log 600-A). This can be particularly useful in determining which part of the content a TV user is most interested in for the longest period of time. (For example, an alternative embodiment could indicate a "maximum disinterest segment," which indicates the longest continuous segment of content in which the corresponding TV user is not in a "focused" position.)

[0066] Figure 6C The third ratings log 600-C is shown to be a more comprehensive log than ratings log 600-A or ratings log 600-B. Here, ratings log 600-C includes, for each television user, an indication of whether the corresponding television user was identified as "attentive" to each content segment. Specifically, for each segment, attention is indicated using binary indicators (e.g., "inattentive" could be indicated as "0," and "attentive" could be indicated as "1"). Thus, this form of user interest recording provides a broad record of when a user showed interest or disinterest in the content being broadcast on television. In an alternative embodiment, the log could provide even finer-grained detail by the percentage or number of scans indicating that a television user was identified as "attentive" for each segment.

[0067] It can be noted that, Figures 6A-6C The viewership log 600 shown is provided as a non-limiting example. Alternative embodiments may record television viewers’ interests in any of a variety of alternative ways, depending on the desired functionality.

[0068] return Figure 5In the diagram, the dashed boxes indicate optional functions that can be included in the process. For example, at box 560, this function includes post-content input requested from one or more television users watching a replay of content on the television. This function could be included, for example, if (multiple) television users are determined to be at least at a threshold level interested in the content. Input requested from the user can indicate the user's interest in specific content and / or content type.

[0069] For example, if a television user is determined to be in an "attentive" position for at least a threshold percentage (e.g., 95%) of the duration of a vehicle advertisement, the television can provide that user with one or more prompts (e.g., via on-screen menus, audio / voice prompts, etc.) to determine how to receive the vehicle advertisement. Example prompts might include asking the television user to indicate whether the vehicle advertisement is applicable to them, whether the television user is interested in receiving more advertising about a specific vehicle (or a certain type of vehicle, or vehicles in general, etc.) featured in the vehicle advertisement, and / or whether the television user is interested in receiving more information about the specific vehicle featured in the vehicle advertisement. In the latter case, if the vehicle user indicates they want more information, the television can play additional advertisements, provide a website and / or phone number for the vehicle on the screen, and / or open a web browser to access a website including sales locations, promotions, and / or other detailed information about the vehicle.

[0070] At box 570, the process may optionally include transmitting the result to a remote device. That is, according to some embodiments, the television or connected RF sensing device may be connected to the Internet or other data communication networks, thereby enabling the transmission of information about television user interests (such as television manufacturers, content providers, advertising providers, etc.) to a remote service provider. This information may include, for example, any recorded information about user interests (e.g., such as...). Figures 6A-6C The viewership log (as shown in 600) and the corresponding content item's identifier. Depending on the required functionality, this information can be transmitted after the content has finished playing back (or even during playback, as described below), or in batches (e.g., transmitting accumulated information about the content watched hourly, daily, weekly, etc.). As a result, customized content can be provided to the television. For example, in the case of playing advertisements, the ad server can send more customized advertisements to the television, which can store them in a buffer and play the advertisement during a time slot reserved for other content (e.g., ad-supported content) during playback. In this way, the buffer can be refreshed with advertisements that more closely match the preferences of (multiple) television viewers.

[0071] It can be noted that, in an alternative embodiment, the functions of blocks 550-570 can be performed when RF sensing occurs. That is, recording, interest gathering, and transmission results can occur in real time as the content is played, as a supplement to or alternative to performing these functions after playback. In this embodiment, the television and / or remote services can be adjusted in real time according to user interests, and the content can be changed accordingly if necessary.

[0072] According to some embodiments, RF sensing can be used to provide additional or alternative functionality. Examples of two such functions are... Figure 7 and Figure 8 As shown in the image.

[0073] Figure 7 This is a flowchart illustrating a process for turning off a television based on information obtained from RF sensing, according to an embodiment. (Compared to other methods provided herein...) Figure 1 Sample, Figure 7 Provided as a non-limiting example. Alternative embodiments may be added, omitted, rearranged, and / or otherwise modified. Figure 7 The operation shown. In Figure 7 The RF sensing provided in the process shown can be provided by an RF sensing system, such as... Figure 1 The system shown can be incorporated into, for example, Figure 2 and Figure 3 In one or more transceivers shown. Figure 7 The implementation of the process shown can be performed by a television or other electronic device (such as...) Figure 10 Executed by one or more processing units of the electronic device shown and described below.

[0074] It can be noted that, Figure 7 The process shown can begin during television playback and can be combined with... Figure 5 The process shown (and described below) Figure 8 The process shown is compatible. For example, Figure 5 The process can be performed during advertising or other content that content providers may want to capture user interest. However, when not performed... Figure 5 During the process, Figure 7 The process can be the default execution process. Furthermore, televisions or other electronic devices performing these processes can perform optimizations, for example, by using RF sensing data obtained in one process from another. As a specific example, RF sensing data obtained from high-resolution RF sensing at box 520 can eliminate the need for low-resolution RF sensing at box 710. Other such optimizations can be performed when these (and possibly other) RF sensing processes are executed in parallel.

[0075] The functionality at box 700 includes performing low-resolution RF sensing. As described above, low-resolution RF sensing can include a form of RF sensing that captures CSI at a relatively low frequency (e.g., 100 ms or more, alternatively 500 ms or more, alternatively 1 s or more, etc.), a relatively low bandwidth (e.g., 20 MHz or less, or 40 MHz or less), and / or a relatively small spatial flow (e.g., a single spatial flow). Due to its relatively low resolution, in some embodiments, RF sensing may not be able to identify a specific television user, but RF sensing may still be able to detect motion and / or the presence of a television user at a certain location, which may be sufficient for implementation. Figure 7 The function of the shutdown process. Furthermore, since low-resolution RF sensing consumes less power than high-resolution RF sensing, it is likely preferred in most cases. That is, alternative embodiments may perform high-resolution RF sensing. As previously mentioned, this could be the case where: [The following is a description of the process:] Performing... Figure 7 The television or another RF sensing device is performing RF sensing in parallel for other functions.

[0076] At box 720, RF sensing data is analyzed to determine whether any user or motion is detected. As previously mentioned, RF sensing data may include CSI and / or data derived therefrom. Changes in RF sensing data between consecutive scans can indicate motion, and low-resolution RF sensing data can enable the detection of human-sized objects at locations where television viewing is taking place (this can be predetermined based on one or more user profiles, specifically based on RF sensing data included in those user profiles). Thus, it may not be necessary to perform user / motion detection by comparing the RF sensing data with corresponding RF sensing data stored in the user profile.

[0077] If a user or motion is detected, the process can continue with low-resolution RF sensing (at box 710). Otherwise, the process can optionally perform the function at box 730 by performing high-resolution RF sensing. This high-resolution RF sensing can be performed to confirm the presence or absence of the television user. For example, in embodiments where low-resolution sensing can only provide motion detection (without detecting the presence of a stationary user), additional confirmation can be performed to determine the user's presence. This can help prevent situations where a user is stationary but still watching television.

[0078] If the function at box 730 is executed, the process can further execute the function at box 740, where it is determined whether an attentive user is detected based on the RF sensing data obtained at box 730. If an attentive user is detected, the process can return to the function at box 710, which involves low-resolution RF sensing. Otherwise, if no attentive user is detected (e.g., the user is not attentive, no user is present, etc.), the process can proceed to box 750.

[0079] The function at box 750 includes determining whether a threshold time has elapsed. If not, the process returns to low-resolution RF sensing at box 710. According to some embodiments, the user-configurable delay period may elapse before the repeated low-resolution RF sensing. This function may help ensure the television is not turned off prematurely. According to some embodiments, the time threshold may be set long enough for the television user to temporarily leave the room (e.g., to get a snack, go to the bathroom, etc.) or recover from a state of inattention (e.g., being asleep), but short enough to allow the television to be turned off once it is understood that the user will not continue watching without leaving it on for an excessively long period. According to some embodiments, the threshold may be configured by the television user (e.g., via a screen user menu).

[0080] If a threshold amount of time has elapsed, the process may optionally proceed to the functionality at box 760, which saves the playback position. This allows television users to subsequently resume playback from the position where playback of a television program, movie, etc., was stopped before the television was turned off. In some embodiments, the playback position may be saved in a profile of one or more television users recently identified using RF sensing. This allows one or more users to subsequently resume playback, while other television users (who may be assumed not to be interested in resuming playback at the saved position) may not be given this option.

[0081] Then, the process can continue to execute the function at frame 770 by turning off the TV. Here, "turning off" the TV can include turning off the TV power, or simply turning the TV off to a low-power state (in which the TV may be able to subsequently... as described below) Figure 8 (Open in the manner described). If Figure 7 The process of turning off the TV is performed by the TV's transceiver or connected devices (e.g., set-top boxes or streaming devices). Turning off the TV may include sending a signal to the TV's application processor or central processing unit to turn it off. For example, connected devices that can provide video output via a High Definition Multimedia Interface (HDMI) or Universal Serial Bus (USB) may additionally or alternatively stop providing video output. This could be an additional or alternative indicator for turning off the TV.

[0082] Figure 8 This is a flowchart illustrating the process of turning on a television based on information obtained from RF sensing, according to an embodiment. As described above, this function can be combined with... Figure 5 and Figure 7 One or two functions are used to execute the process shown. This is in contrast to other functions provided in this article. Figure 1 Sample, Figure 8 Provided as a non-limiting example, alternative embodiments may be added to, omitted, rearranged, and / or otherwise modified. Figure 8 The operation shown. (And) Figure 5 and Figure 7 The process shown is similar, in Figure 8 The RF sensing provided in the process shown can be provided by an RF sensing system, such as... Figure 1 The system shown can be incorporated into, for example, Figure 2 and Figure 3 In one or more transceivers shown. Figure 8 The implementation of the process shown can be executed by one or more processing units of an electronic device, such as... Figure 10 The electronic device shown and described below.

[0083] Here, the process can begin with the functionality of box 810, where low-resolution RF sensing is performed. Similar to... Figure 7 The low-resolution RF sensor at frame 710 can be used to detect the motion and / or presence of a television user. Figure 7 The RF sensing at frame 710 differs from that at frame 810; however, the low-resolution RF sensing is performed when the TV is off (e.g., in a low-power state relative to the active state of the TV playing content). For many embodiments, this may mean that the power budget for performing the low-resolution RF sensing at frame 810 (whether performed by the TV's internal circuitry or by a connected device that draws power from the TV or may itself have a tight power budget) may be particularly low. Therefore, the low-resolution RF sensing performed at frame 810 can be tailored to meet such budget power requirements. This may mean, for example, performing RF sensing at a sufficiently low resolution to meet the budget, but performing scanning with a scan cycle, bandwidth, and / or stream sufficient to detect the movement and / or presence of a TV user.

[0084] At box 820, based on the RF sensing data obtained from the RF sensing scan performed at box 810, it is determined whether any motion and / or presence of a television user has been detected. If not, the process can return to the function at box 810 to continue performing low-resolution RF sensing. A user-configurable delay period may pass before repeating the low-resolution RF sensing. Otherwise, if motion and / or the presence of a television user is detected, the process can continue to the function at box 830.

[0085] At box 830, this function includes high-resolution RF sensing. Similar to box 730 and... Figure 7 The function, in Figure 8 The high-resolution scan performed at frame 830 can be used to confirm whether a television user has been detected, or even whether a specific television user has been detected. According to some embodiments, this functionality can help reduce the likelihood of false positives (false positives) detected by low-resolution sensing of a user / motion, which might indicate that the television has been turned on. Such examples could include, for example, the movement of a pet or cleaning robot, or the detection of a television user in a location not intended for watching television (e.g., as...). Figure 4B As shown in the image), the system detects that the TV user is in a position of inattention (e.g., as shown in the image). Figure 4C (as shown), and / or another person is detected (without a corresponding user profile).

[0086] Furthermore, according to certain embodiments, Figure 8 The automatic opening function can be a user-configurable setting, which can be activated on a per-user basis. Furthermore, in addition to being in a focused position (e.g., as...), Figure 4A In addition to turning on the TV as shown in the diagram, or alternatively, a TV user can configure a unique "on" posture to turn on the TV. This can be done, for example, by allowing the user to select a posture (e.g., which may involve a selected position and / or orientation of the user's head, torso, arms, legs, etc.) and calibrating the RF sensors to detect the posture by capturing RF sensing data when the user is in that posture, during the configuration of a user profile specifically for automatically turning on the TV. Furthermore, depending on the desired functionality, the user can also select a specific location where the "on" posture can be performed (e.g., a seating position in the room where the TV is located), further reducing the chance of the TV turning on unintentionally. Thus, the high-resolution RF sensing performed at box 830 can be used to identify a specific TV user and determine, based on the user profile saved for the TV user, whether the user has selected the automatic turn-on function and / or whether the user is in a pre-configured "on" posture.

[0087] At box 840, the function includes determining whether an attentive television user is detected. As described above, this may include determining whether the television user has selected the automatic turn-on function and / or is in a personalized / customized "on" posture. If not, the process may return to the function at box 810 and continue with low-resolution RF sensing. Otherwise, the process may proceed to the function at box 850 and turn on the television. Here, "turning on" includes changing the television from a power-off or low-power state (where the display, circuitry, and / or other television components may be deactivated or in standby mode) to an active state where content can be played.

[0088] Once the television is turned on, the process may include performing the functions shown at boxes 860 and 870 or 880. The function at box 860 includes determining whether a playback position for previously viewed content has been saved. As previously mentioned, depending on the required functionality, a playback position may be saved for a specific user or a group of users. For such an embodiment, box 860 may include determining whether a playback position has been saved for a specific user or an identified group of users, which can be identified at box 840. As described above, playback positions can be saved automatically, such as in… Figure 7 Box 760 in the process shown. By implementing in Figure 7 and Figure 8 These functions in the process shown, for example, allow the television to adjust according to the user's sleep (and therefore not move and / or be out of focus). Figure 7 The process saves the playback position and closes the window, then reopens and resumes playback at the saved position once the TV user's attention is detected to have returned.

[0089] Additionally or alternatively, the TV may automatically save the playback location when a user or user group stops playback and turns off the TV (e.g., this could be a function that the user can select in the TV settings), or when the user / user group manually selects a location to save the playback (e.g., by interacting with an on-screen menu). If the playback location is saved, the TV may resume playback, as shown in box 870. Otherwise, a user menu may be provided to the user, as shown in box 880.

[0090] As described above, because RF sensing can identify users or user groups watching television, content and / or menus can be customized based on the user preferences of the identified users(s). Since one or more users can be identified at box 840, the menu provided at box 880 can be customized in this way. For example, if only children are identified, a child-friendly menu can be provided. According to some embodiments, if an adult television user is identified via RF sensing and / or an authorization password is entered, the menu can revert from a child-friendly menu to a normal menu.

[0091] Figure 9 This is a flowchart illustrating a method 900 for RF sensing of a television user according to an embodiment. Figure 9 The operations shown in the box can be performed by a television or connected device using one or more transceivers in the manner described above for an RF sensing system. Figure 9 Example components of operating televisions or connected devices in Figure 10 The following is illustrated and described in more detail. Alternative embodiments may differ from method 900 by adding, omitting, combining, and / or rearranging the shown operations, and / or by performing the operations simultaneously. Method 900 describes a general process of RF sensing for a television user and covers many of the embodiments previously described. Similarly, method 900 can be considered as described above. Figure 5 , Figure 7 and Figure 8 The process shown can be implemented in a way that at least some aspects are shown.

[0092] At block 910, the method includes transmitting a first RF signal using one or more wireless transceivers. As described in the embodiments above, the television environment may have one or more transceivers, each of which may include an RF sensing system 105 (or at least a portion thereof). The one or more transceivers may include one or more wireless radios capable of transmitting and receiving RF signals using WLAN standards (e.g., IEEE 902.11 / Wi-Fi) and may be used by the television and / or connected devices for WLAN communication in addition to RF sensing. Thus, the first RF signal may include communication packets used by WLAN standards (e.g., IEEE 802.11). As previously described, embodiments herein may utilize existing techniques for channel estimation to obtain CSI for RF sensing. As described in the IEEE 802.11 standard, for example, two repetitions of a long training sequence may be used for channel estimation. Additionally or alternatively, the one or more transceivers may include UWB transceivers.

[0093] Components for performing the functions of block 910 may include (multiple) processing units 1010, bus 1005, working memory 1035, communication subsystem 1030, wireless communication interface 1033, RF sensing system 105, and / or such as Figure 10 Other components shown and described below. Additional components may include (multiple) Tx antennas 115, Tx processing circuitry 140, multiplexer 135, processing unit 125, memory 130, and / or as... Figure 1 Other components of the RF sensing system 105 shown and described above.

[0094] At box 920, the function includes receiving a first reflected RF signal using one or more wireless transceivers, the first reflected RF signal comprising a reflection of a first RF signal from one or more objects. For example, in the presence of multiple television users, the one or more objects may include multiple television users (among other things). Other objects may include pets, furniture, etc., which the television or connected device may ignore. As described in the above embodiments, the transceiver receiving the first reflected RF signal may be the same transceiver that transmits the RF signal (e.g., such as...). Figure 3 (as shown), or it can be a different transceiver (e.g., such as...). Figure 2 (As shown). Thus, according to certain embodiments of method 900, one or more wireless transceivers may include a single wireless transceiver located at a single location. Alternatively, a first wireless transceiver of the one or more wireless transceivers transmits a first RF signal, and a second wireless transceiver of the one or more wireless transceivers receives a first reflected RF signal, and the first wireless transceiver is located at a different location than the second wireless transceiver. In embodiments using more than one transceiver, one or more processors inside and / or outside the television may coordinate the transmission and reception of RF signals. Additionally or alternatively, the transceivers may communicate with each other (e.g., according to a prescribed wireless standard) to coordinate the transmission and reception of RF signals.

[0095] Components for performing the functions of block 920 may include (multiple) processing units 1010, bus 1005, working memory 1035, communication subsystem 1030, wireless communication interface 1033, RF sensing system 105, and / or such as Figure 10 Other components shown and described below. Additional components may include (multiple) Rx antennas 120, Rx processing circuitry 145, multiplexer 135, processing unit 125, memory 130, and / or as... Figure 1 Other components of the RF sensing system 105 shown and described above.

[0096] The function at block 930 includes determining a first CSI of one or more wireless channels based on the received first reflected RF signal. As described above, this can be determined using channel estimation techniques of a prescribed wireless standard for receiving one or more wireless transceivers of the reflected RF signal. Furthermore, as further noted, the reflected RF signal may be received by multiple antennas and / or received multiple times. Therefore, in some embodiments, this can allow not only the determination of the presence of motion or an object, but also its direction. This can depend on how the RF signal is transmitted and received (e.g., using low-resolution or high-resolution detection).

[0097] Components for performing the functions of block 930 may include (multiple) processing units 1010, bus 1005, working memory 1035, communication subsystem 1030, wireless communication interface 1033, RF sensing system 105, and / or such as Figure 10 Other components shown and described below. Additional components may include processing unit 125, memory 130, and / or such as Figure 1 Other components of the RF sensing system 105 shown and described above.

[0098] At box 940, the function includes determining status information based on a first CSI, wherein the status information includes information about the viewing status of a television user. The viewing status of a television user may include the presence or absence of any television user or a specific television user (e.g., in a specific area of ​​the television environment). The viewing status of a television user may include the identity of the television user present in the television environment, for example, in relation to a specific user profile. In some aspects, the viewing status of a television user may include motion state, head orientation, eye orientation, sitting posture, or gesture, or any combination thereof. Furthermore, at box 950, the function includes performing an action on the television based on the status information. The type of status information obtained at boxes 940 and 950 and the action performed may vary depending on the type of function to be implemented. As noted in the previously described embodiments, this may include user interest determination, automatic television shutdown, and / or automatic television startup. As mentioned above, the status information about the television user may be based on RF sensing data, which may include CSI and / or information derived from CSI (detected movement, volume, eye position, etc.).

[0099] In some embodiments, such as Figure 5 In the illustrated embodiment, the status information may include the television user's attention level. This attention level may be as follows: Figures 6A-6CQuantified as described. That is, according to an alternative embodiment, method 900 may include determining that the television is playing content of a predetermined type, wherein sending a first RF signal is in response to determining that the television is playing content of a predetermined type, and determining status information includes determining the attention level of the television user watching the content based on a first CSI. According to some embodiments, determining the attention level of the television user watching the content includes: determining one or more attributes of the television user while the television is playing content based on the first CSI. These attributes may include head orientation, eye orientation, sitting posture, or gesture or any combination thereof. According to some embodiments, determining one or more attributes of the television user includes comparing information obtained based on the first CSI with stored profile information about the television user. According to some embodiments, performing an action on the television includes transmitting an indication of the attention level from the television to a server of a service provider, a server of a content provider, or both. As previously mentioned, such attention level may be indicated in ratings logs, such as in Figures 6A-6C The log shown in the image.

[0100] Additional or alternative land, such as Figure 7 As shown in the embodiments illustrated, the status information can be used to implement an automatic shutdown function. Therefore, some embodiments of method 900 may further include: determining a lack of detected movement based on a first CSI; transmitting a second RF signal using one or more wireless transceivers in response to determining the lack of movement; and receiving a second reflected RF signal using one or more wireless transceivers, the second reflected RF signal including reflections of the second RF signal from one or more objects. These embodiments may further include determining a second CSI based on the received second reflected RF signal. In this embodiment, determining the status information may also be based on the second CSI, wherein the status information includes an indication that a television user is not watching television, and performing an action on the television includes turning off the television. According to some embodiments, the status information may include information indicating that a television user is no longer detected or that the television user is no longer awake. As described in the embodiments above, different types of RF sensing modes (e.g., high-resolution RF sensing and low-resolution RF sensing) can be used to obtain different levels of RF sensing data. Therefore, according to some embodiments, a first RF signal may be transmitted according to a first transmission mode, and a second RF signal may be transmitted according to a second transmission mode. In these embodiments, the second transmission mode may have a shorter transmission period, a larger transmission bandwidth, or a larger number of spatial streams, or any combination thereof, than the first transmission mode. Finally, embodiments of method 900 may further include: saving the position of the content being played on the television before turning off the television.

[0101] Additional or alternative land, such as Figure 8As shown in the embodiments, the status information can be used to implement an automatic turn-on function. For example, according to some embodiments, method 900 may further include: determining detected movement based on a first CSI; in response to determining the movement, transmitting a second RF signal using one or more wireless transceivers; and receiving a second reflected RF signal using one or more wireless transceivers, the second reflected RF signal including reflections of the second RF signal from one or more objects. This embodiment may also include determining a second CSI from the received second reflected RF signal, and determining the status information may also be based on the second CSI, wherein the status information includes an indication that the television user intends to watch television. Moreover, performing an action on the television may include turning on the television. As described in the above embodiments, the user may adopt an "on" posture or a similar position showing attention to the television to cause the television to turn on automatically in this manner, and these positions may be stored in RF sensing data stored in a user profile. Thus, according to some embodiments, determining that the television user intends to watch television may be based at least in part on comparing information obtained from the second CSI with stored profile information about the television user. Furthermore, the function may include low-resolution and high-resolution RF scanning modes. Thus, according to some embodiments, a first RF signal is transmitted according to a first transmission mode, and a second RF signal is transmitted according to a second transmission mode. In such instances, the second transmission mode may have a shorter transmission cycle, a larger transmission bandwidth, or a larger number of spatial streams, or any combination thereof, than the first transmission mode. Other embodiments enabling the on-screen function may include functions such as obtaining a saved location in the content being played on the television and resuming playback of the content from the saved location. Additionally or alternatively, embodiments may include identifying the television user at least in part based on a second CSI, determining the corresponding stored user profile for the television user based on the television user's identity, and providing a user menu on the television, wherein the content within the user menu is at least in part based on the corresponding stored user profile. According to some embodiments, identifying the television user may include determining that the television user is a child, and providing the user menu may include providing a menu for children.

[0102] Finally, embodiments may additionally or alternatively include one of the following features. According to some embodiments, the status information may include an indication of whether a television user is detected, the identity of the television user, an indication of whether the television user is watching content being broadcast on television, or any combination thereof. According to some embodiments, method 900 further includes determining the identity of the television user by comparing information obtained based on the first CSI with stored profile information about the television user. These and other features may be facilitated by initial calibration. Therefore, according to some embodiments, method 900 may further include performing calibration for the television user before transmitting the first RF signal, in which a second RF signal is transmitted by one or more wireless transceivers while the television user is in the location watching television, and a second reflected RF signal including the second RF signal reflected from the television user is received by one or more wireless transceivers. In this embodiment, a second CSI may be determined based on the received second reflected RF signal, one or more user attributes of the television user may be determined at least in part based on the second CSI, and one or more user attributes may be stored in a user profile.

[0103] Components for performing the functions of blocks 940 and 950 may include (multiple) processing units 1010, bus 1005, working memory 1035, communication subsystem 1030, wireless communication interface 1033, RF sensing system 105, and / or such as Figure 10 Other components shown and described below. Additional components may include processing unit 125, memory 130, and / or such as Figure 1 Other components of the RF sensing system 105 shown and described above.

[0104] Figure 10 This is a block diagram of an embodiment of electronic device 1000, which may incorporate an RF sensing system 105 that can operate in the manner discussed in the foregoing embodiments. As described above, the RF sensing system 105 may be included in each of one or more transceivers, which may be incorporated into one or more subsystems of the electronic device, such as wireless communication interface 1033. The electronic device itself may include a television, set-top box, streaming device, or other device capable of performing RF sensing as described herein. It should be noted that... Figure 10 This is intended only to provide a general overview of the various components; any or all of these components may be used as appropriate. Therefore, Figure 10 It extensively demonstrates how individual system components can be implemented in a relatively discrete or relatively more integrated manner. Furthermore, it can be noted that... Figure 10The components shown can be located in a single device and / or distributed among various networked devices that may be located in different physical locations. For example, some components may be inside the TV, while others may be outside the TV (while being communicatively coupled to the internal components via, for example, an HDMI or USB port).

[0105] Electronic device 1000 is shown to include hardware elements that can be electrically coupled (or otherwise communicated) via bus 1005. The hardware elements may include (but are not limited to) processing units 1010, which may include, but are not limited to, one or more general-purpose processors, one or more special-purpose processors (such as DSPs, graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), and / or other processing architectures, which may be configured to perform one or more methods described herein, including those relating to… Figure 9 The methods and / or Figure 5 , Figure 7 and Figure 8 The process described herein. Electronic device 1000 may also include one or more input devices 1015, which may include, but are not limited to, remote controls, touch interfaces, microphones, buttons, switches, dial pads, etc. One or more output devices 1020 may include, but are not limited to, displays, HDMI outputs and / or other media interfaces, speakers, etc.

[0106] The electronic device 1000 may also include (and / or communicate with) one or more non-transitory storage devices 1025, which may include, but are not limited to, locally and / or network-accessible storage devices, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices (such as random access memory (RAM) and / or read-only memory (ROM)), which may be programmable, flash-updatable, etc. Such storage devices can be configured to implement any suitable data storage, including but not limited to various file systems, database structures, etc.

[0107] Electronic device 1000 may also include a communication subsystem 1030, which may include support for wired communication technologies and / or wireless communication technologies (in some embodiments) managed and controlled by wireless communication interface 1033. Communication subsystem 1030 may include a modem, network interface card (wireless or wired), infrared communication device, wireless communication device, and / or chipset, etc. Communication subsystem 1030 may include one or more input and / or output communication interfaces, such as wireless communication interface 1033 or wired communication interface, to allow the exchange of data and signaling with networks, mobile devices (e.g., via mobile phone applications), other computer systems, and / or any other electronic devices described herein. As previously mentioned, RF sensing system 105 (such as...) Figure 1The wireless communication interface 1033 (shown) can be incorporated into the wireless communication interface 1033, such that the (multiple) Tx antennas 115 and (multiple) Rx antennas 120, as well as the circuitry connected to the antenna elements (e.g., other components of the RF sensing system 105), can be used for both RF sensing and data communication. For example, in some embodiments, the wireless communication interface 1033 may include an 802.11ad-compatible and / or 802.11ay-compatible modem capable of both RF sensing and data communication. Aspects of the wireless communication interface 1033 having the RF sensing system 105 may correspond to... Figure 2 and Figure 3 The transceiver shown and described elsewhere in this document.

[0108] As described above, some embodiments may have an RF sensing system 105 that is not used for wireless communication and is therefore a dedicated system for RF sensing. In such an example, the RF sensing system 105 may be incorporated into other locations within the electronic device 1000. In some embodiments, for example, the RF sensing system 105 may be incorporated into the electronic device 1000 as an input device 1015. Other sensors may also be included as input devices 1015.

[0109] In many embodiments, the electronic device 1000 also includes working memory 1035, which may include RAM and / or ROM devices. Software elements shown as residing within working memory 1035 may include operating system 1040, device drivers, executable libraries, and / or other code, such as applications(s)1045, which may include computer programs provided by various embodiments and / or may be designed to implement methods and / or configure systems provided by other embodiments, as described herein. By way of example only, regarding one or more programs described with respect to the above methods(s), such as those concerning... Figure 9 The described method can be implemented as code and / or instructions stored (e.g., temporarily) in working memory 1035 and executable by a computer (and / or a processing unit within the computer, such as processing unit(s) 1010); then, on the one hand, such code and / or instructions can be used to configure and / or adjust a general-purpose computer (or other device) to perform one or more operations according to the described method.

[0110] These instruction sets and / or code may be stored on non-transitory computer-readable (or machine-readable) storage media, such as (multiple) of the aforementioned storage devices 1025. In some cases, the storage media may be incorporated into a computer system, such as electronic device 1000. In other embodiments, the storage media may be separate from electronic device 1000 (e.g., a removable medium such as an optical disc) and / or provided in an installation package, such that the storage media can be used to program, configure, and / or adapt to a general-purpose computer or electronic device having instructions / code stored thereon. These instructions may be in the form of executable code executable by electronic device 1000 and / or may be in the form of source code and / or installable code, which, when compiled and / or installed on electronic device 1000 (e.g., using various generally available compilers, installers, compression / decompression utilities, etc.), then take the form of executable code.

[0111] It will be apparent to those skilled in the art that substantial changes can be made to suit specific requirements. For example, custom hardware may be used, and / or specific elements may be implemented in hardware, software (including portable software such as applets), or both. Furthermore, connections to other computing devices, such as network input / output devices, may be employed.

[0112] Referring to the accompanying drawings, components that may include memory may include non-transitory machine-readable media. As used herein, the terms "machine-readable media" and "computer-readable media" refer to any medium that participates in providing data that enables a machine to operate in a particular manner. In the embodiments provided above, various machine-readable media may involve providing instructions / code to a processing unit and / or(a plurality of) other devices for execution. Additionally or alternatively, machine-readable media may be used to store and / or carry such instructions / code. In many embodiments, computer-readable media are physical and / or tangible storage media. Such media can take many forms, including but not limited to non-volatile and volatile media. Common forms of computer-readable media include, for example, magnetic and / or optical media, any other physical media having a hole pattern, RAM, programmable ROM (PROM), erasable PROM (EPROM), FLASH-EPROM, any other memory chip or cartridge, the carrier wave described below, or any other medium from which a computer can read instructions and / or code.

[0113] The methods, systems, and devices described herein are examples. Various procedures or components may be omitted, substituted, or added as appropriate in various embodiments. For example, features described with respect to certain embodiments may be combined in various other embodiments. Different aspects and elements of embodiments may be combined in a similar manner. Various components of the accompanying drawings provided herein may be embodied in hardware and / or software. Furthermore, technology is evolving, and therefore many of the elements are examples that do not limit the scope of this disclosure to those particular examples.

[0114] Sometimes, primarily for general reasons, referring to signals as bits, information, values, elements, symbols, characters, variables, terms, numbers, numerical symbols, etc., has proven convenient. However, it should be understood that all such terms, or similar terms, will be associated with the appropriate physical quantity and are merely convenient labels. Unless otherwise explicitly stated, as is apparent from the above discussion, it should be understood that the use of terms such as “processing,” “calculating,” “operating,” “determining,” “ascertaining,” “identifying,” “associating,” “measuring,” “executing,” etc., throughout this specification refers to the action or process of a specific device, such as a dedicated computer or similar dedicated electronic computing device. Therefore, in the context of this specification, a dedicated computer or similar dedicated electronic computing device capable of manipulating or converting signals (generally represented as physical electronic, electrical, or magnetic quantities in memory, registers, or other information storage devices), the transmitting device of the dedicated computer or similar dedicated electronic computing device, or the display device of the dedicated computer or similar dedicated electronic computing device.

[0115] As used herein, the terms “and” and “or” can have a variety of meanings, which are also expected to depend at least in part on the context in which the terms are used. Generally, “or” when used in a list of associations, such as A, B, or C, is intended to mean A, B, and C in an inclusive sense, and A, B, or C in an exclusive sense. Furthermore, the term “one or more” as used herein can be used to describe any feature, structure, or property in the singular, or can be used to describe some combination of features, structures, or properties. However, it should be noted that this is merely an illustrative example and the claimed subject matter is not limited to this example. Additionally, when used in a list of associations, such as A, B, or C, the term “at least one” can be interpreted as meaning any combination of A, B, and / or C, such as A, AB, AA, AAB, AABBCCC, etc.

[0116] Several embodiments have been described, and various modifications, alternative constructions, and equivalents may be used without departing from the scope of this disclosure. For example, the foregoing elements may simply be components of a larger system, in which other rules may be preferred or the application of various embodiments may be modified in other ways. Furthermore, numerous steps may be performed before, during, or after considering the foregoing elements. Therefore, the above description does not limit the scope of this disclosure.

[0117] In view of this description, embodiments may include different combinations of features. Examples of implementations are described in the following numbered clauses:

[0118] Clause 1. A method for radio frequency (RF) sensing of a television user, the method comprising: transmitting a first RF signal using one or more wireless transceivers; receiving a first reflected RF signal using one or more wireless transceivers, the first reflected RF signal including reflections of the first RF signal from one or more objects; determining first channel state information (CSI) of one or more wireless channels based on the received first reflected RF signal; determining status information based on the first CSI, wherein the status information includes information about the viewing status of the television user; and performing an action on the television based on the status information.

[0119] Clause 2. The method according to Clause 1 further includes determining that the television is playing content of a predetermined type, wherein: sending a first RF signal is in response to determining that the television is playing content of a predetermined type, and determining status information includes determining the attention level of the television user watching the content based on a first CSI.

[0120] Clause 3. The method described in Clause 2, wherein determining the attention level of a television user watching content comprises: determining one or more attributes of the television user while the television is playing, based on a first CSI, the one or more attributes including: head orientation, eye orientation, sitting posture, or gesture or any combination thereof.

[0121] Clause 4. The method according to Clause 3, wherein determining one or more attributes of a television user includes comparing information obtained based on a first CSI with stored profile information about the television user.

[0122] Clause 5. The method according to any one of Clauses 1-4, wherein performing an action on the television includes transmitting an indication of attention level from the television to: a server of a service provider, a server of a content provider, or both.

[0123] Clause 6. The method according to any one of Clauses 1-5 further includes determining the identity of the television user by comparing information obtained from the first CSI with stored profile information about the television user.

[0124] Clause 7. The method according to any one of Clauses 1, 5, or 6 further comprises: determining, based on a first CSI, the absence of detected movement; in response to determining the absence of movement, transmitting a second RF signal using one or more wireless transceivers; receiving, using one or more wireless transceivers, a second reflected RF signal including reflections of the second RF signal from one or more objects; determining a second CSI based on the received second reflected RF signal; wherein: the determination of status information is further based on the second CSI, wherein the status information includes an indication that a television user is not watching television; and performing an action on the television includes turning off the television.

[0125] Clause 8. The method described in Clause 7, wherein the status information includes information indicating that a television user is no longer detected or that the television user is no longer awake.

[0126] Clause 9. The method according to any one of Clauses 7 or 8, wherein the first RF signal is transmitted according to a first transmission mode; and the second RF signal is transmitted according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams or any combination thereof.

[0127] Clause 10. The method according to any one of Clauses 7-9 further includes: saving the position of the content being played on the television before turning off the television.

[0128] Clause 11. The method according to any one of Clauses 7-10 further comprises: determining detected movement based on a first CSI; transmitting a second RF signal using one or more wireless transceivers in response to determining movement; receiving a second reflected RF signal using one or more wireless transceivers, the second reflected RF signal including reflections of the second RF signal from one or more objects; determining a second CSI based on the received second reflected RF signal; wherein: the determination of status information is further based on the second CSI, wherein the status information includes an indication that a television user intends to watch television; and performing an action on the television including turning on the television.

[0129] Clause 12. The method according to any one of Clauses 1, 5, 6 or 11 further includes determining, at least in part, based on comparing information obtained from the second CSI with stored profile information about the television user to determine that the television user intends to watch television.

[0130] Clause 13. The method according to Clause 11 or 12, wherein the first RF signal is transmitted according to a first transmission mode; and the second RF signal is transmitted according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams or any combination thereof.

[0131] Clause 14. The method according to any one of Clauses 11-13 further includes: obtaining a saved location in the content being played on the television; and restoring the function of the television to play back the content from the saved location.

[0132] Clause 15. The method according to any one of Clauses 11-14 further includes: identifying the television user at least in part based on the second CSI; determining the corresponding stored user profile of the television user based on the identity of the television user; and providing a user menu on the television, wherein the content of the user menu is at least in part based on the corresponding stored user profile.

[0133] Clause 16. The method according to any one of Clauses 11-15, wherein: identifying the television user includes determining that the television user is a child, and providing the user menu includes providing a menu for children.

[0134] Clause 17. The method according to any one of Clauses 1-16, wherein the status information includes: whether an instruction from a television user is detected, the identity of the television user, an instruction from the television user as to whether the television user is watching content being broadcast on television, or any combination thereof.

[0135] Clause 18. The method according to any one of Clauses 1-17 further comprises, before transmitting the first RF signal: performing calibration for the television user, in which, while the television user is at the location where the television is watching television: the second RF signal is transmitted via one or more wireless transceivers, and a second reflected RF signal, including a reflection of the second RF signal from the television user, is received by the one or more wireless transceivers; and wherein: the second CSI is determined based on the received second reflected RF signal; one or more user attributes of the television user are determined at least in part based on the second CSI; and one or more user attributes are stored in a user profile.

[0136] Clause 19. The method according to any one of Clauses 1-18, wherein a first wireless transceiver of one or more wireless transceivers transmits a first RF signal, and a second wireless transceiver of one or more wireless transceivers receives a first reflected RF signal, wherein the first wireless transceiver is located at a different location from the second wireless transceiver.

[0137] Clause 20. The method according to any one of Clauses 1-19, wherein one or more wireless transceivers include one or more wireless local area network (WLAN) or Wi-Fi transceivers.

[0138] Clause 21. A device for sensing the radio frequency (RF) of a television user, the device comprising: one or more wireless transceivers; a memory; and one or more processing units communicatively coupled to the one or more wireless transceivers and the memory, the one or more processing units being configured to: transmit a first RF signal using the one or more wireless transceivers; receive a first reflected RF signal using the one or more wireless transceivers, the first reflected RF signal including reflections of the first RF signal from one or more objects; determine first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal; determine status information based on the first CSI, wherein the status information includes information about the viewing status of the television user; and perform an action on the television based on the status information.

[0139] Clause 22. The device according to Clause 21, wherein the one or more processing units are further configured to determine that the television is playing content of a predetermined type, wherein: the one or more processing units are configured to send a first RF signal in response to determining that the television is playing content of a predetermined type; and in order to determine status information, the one or more processing units are configured to determine the attention level of the television user watching the content based on a first CSI.

[0140] Clause 23. The device according to Clause 22, wherein, in order to determine the attention level of a television user watching the content, the one or more processing units are configured to determine one or more attributes of the television user while the television is playing, based on a first CSI, the one or more attributes including: head orientation, eye orientation, sitting posture, or gesture or any combination thereof.

[0141] Clause 24. The device according to Clause 23, wherein, in order to determine one or more attributes of a television user, the one or more processing units are configured to compare information obtained based on a first CSI with stored profile information about the television user.

[0142] Clause 25. The device according to any one of Clauses 21-24, wherein, in order to perform an action on the television, the one or more processing units are configured to transmit an indication of attention level from the television to: a server of a service provider, a server of a content provider, or both.

[0143] Clause 26. The device according to any one of Clauses 21-25, wherein the one or more processing units are further configured to determine the identity of a television user by comparing information obtained from the first CSI with stored profile information about the television user.

[0144] Clause 27. The device according to any one of Clauses 21, 25, or 26, wherein the one or more processing units are further configured to: determine, based on a first CSI, that there is a lack of detected movement; in response to determining the lack of movement, transmit a second RF signal using one or more wireless transceivers; receive, using one or more wireless transceivers, a second reflected RF signal comprising reflections of the second RF signal from one or more objects; determine a second CSI based on the received second reflected RF signal; wherein: the one or more processing units are configured to determine status information based on the second CSI, wherein the status information includes an indication that a television user is not watching television; and in order to perform an action on the television, the one or more processing units are configured to turn off the television.

[0145] Clause 28. The device as described in Clause 27, wherein the status information includes information indicating that a television user is no longer detected or that the television user is no longer awake.

[0146] Clause 29. The apparatus according to Clause 27 or 28, wherein the one or more processing units are configured to transmit a first RF signal according to a first transmission mode; and the one or more processing units are configured to transmit a second RF signal according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams or any combination thereof.

[0147] Clause 30. The device according to any one of Clauses 27-29, wherein the one or more processing units are further configured to save the position of the content being played on the television in the memory before the television is turned off.

[0148] Clause 31. The device according to any one of Clauses 27-30, wherein the one or more processing units are further configured to: determine detected movement based on a first CSI; transmit a second RF signal using one or more wireless transceivers in response to determining the movement; receive a second reflected RF signal using one or more wireless transceivers, the second reflected RF signal including reflections of the second RF signal from one or more objects; and determine a second CSI based on the received second reflected RF signal; wherein: the one or more processing units are configured to determine status information based on the second CSI, wherein the status information includes an indication that a television user intends to watch television; and in order to perform an action on the television, the one or more processing units are configured to turn on the television.

[0149] Clause 32. The device according to any one of Clauses 21, 25, 26 or 31, wherein the one or more processing units are further configured to determine, at least in part, that a television user intends to watch television based on a comparison of information obtained from the second CSI with stored profile information about the television user.

[0150] Clause 33. The apparatus according to Clause 31 or 32, wherein the one or more processing units are configured to transmit a first RF signal according to a first transmission mode; and the one or more processing units are configured to transmit a second RF signal according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams or any combination thereof.

[0151] Clause 34. The device according to any one of Clauses 31-33, wherein the one or more processing units are further configured to: obtain a saved location in the content being played by the television; and to restore the playback of the content by the television from the saved location.

[0152] Clause 35. The device according to any one of Clauses 31-34, wherein the one or more processing units are further configured to: identify a television user at least in part based on a second CSI; determine a corresponding stored user profile of the television user based on the identity of the television user; and provide a user menu on the television, wherein the content of the user menu is at least in part based on the corresponding stored user profile.

[0153] Clause 36. The device according to any one of Clauses 31-35, wherein: in order to identify a television user, the one or more processing units are configured to determine that the television user is a child; and in order to provide a user menu, the one or more processing units are configured to provide a menu for children.

[0154] Clause 37. The device according to any one of Clauses 31-36, wherein the status information includes: whether an instruction from a television user is detected, the identity of the television user, an instruction from the television user as to whether the television user is watching content being broadcast on television, or any combination thereof.

[0155] Clause 38. The device according to any one of Clauses 21-37, wherein the one or more processing units are further configured to: perform calibration for a television user before transmitting the first RF signal, wherein, during calibration, when the television user is at a location where the television is being watched, the one or more processing units are configured to: transmit a second RF signal via one or more wireless transceivers, and receive a second reflected RF signal by one or more wireless transceivers, the second reflected RF signal including reflections of the second RF signal from the television user; and wherein the one or more processing units are configured to: determine a second CSI based on the received second reflected RF signal; determine one or more user attributes of the television user based at least in part on the second CSI; and store the one or more user attributes in a user profile in memory.

[0156] Clause 39. The device according to any one of Clauses 21-38, wherein the one or more processing units are further configured to: transmit a first RF signal using a first wireless transceiver among one or more wireless transceivers, and receive a first reflected RF signal using a second wireless transceiver among one or more wireless transceivers, wherein the first wireless transceiver is located at a different location from the second wireless transceiver.

[0157] Clause 40. The device according to any one of Clauses 21-39, wherein the one or more wireless transceivers include one or more wireless local area network (WLAN) or Wi-Fi transceivers.

[0158] Clause 41. A device for radio frequency (RF) sensing of a television user, the device comprising: means for transmitting a first RF signal; means for receiving a first reflected RF signal, the first reflected RF signal including reflections of the first RF signal from one or more objects; means for determining first channel state information (CSI) of one or more wireless channels based on the received first reflected RF signal; means for determining status information based on the first CSI, wherein the status information includes information about the viewing status of the television user; and means for performing an action on the television based on the status information.

[0159] Clause 42. The device according to Clause 41 further includes components for determining that the television is playing content of a predetermined type, wherein: sending a first RF signal is in response to determining that the television is playing content of a predetermined type; and determining status information includes determining the attention level of the television user watching the content based on a first CSI.

[0160] Clause 43. The device according to Clause 41 or 42 further includes a component for determining the identity of a television user by comparing information obtained from the first CSI with stored profile information about the television user.

[0161] Clause 44. The device according to any one of Clauses 41-43 further includes: means for determining, based on a first CSI, the absence of detected movement; means for transmitting a second RF signal in response to determining the absence of movement; means for receiving a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from one or more objects; and means for determining a second CSI based on the received second reflected RF signal; wherein: determining the status information is also based on the second CSI, wherein the status information includes an indication that a television user is not watching television; and performing an action on the television includes turning off the television.

[0162] Clause 45. The device according to any one of Clauses 41-44 further includes: means for determining detected movement based on a first CSI; means for transmitting a second RF signal in response to determining the movement; means for receiving a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from one or more objects; and means for determining a second CSI based on the received second reflected RF signal; wherein: the determination of status information is further based on the second CSI, wherein the status information includes an indication that a television user intends to watch television; and performing an action on the television includes turning on the television.

[0163] Clause 46. The device according to any one of Clauses 41-45 further includes: a component for performing calibration for a television user, wherein, during calibration, when the television user is at a location where the television is being watched: a second RF signal is transmitted by the device, and a second reflected RF signal is received by the device, the second reflected RF signal including reflections of the second RF signal from the television user; and wherein: a second CSI is determined based on the received second reflected RF signal; one or more user attributes of the television user are determined at least in part based on the second CSI; and the one or more user attributes are stored in a user profile.

[0164] Clause 47. The device according to any one of Clauses 41-46, wherein the component for transmitting the first RF signal includes a first wireless transceiver, and the component for receiving the first reflected RF signal includes a second wireless transceiver.

[0165] Clause 48. The device according to any one of Clauses 41-47, wherein the first wireless transceiver is located at a different location from the second wireless transceiver.

[0166] Clause 49. The device according to any one of Clauses 41-48, wherein the first wireless transceiver, the second wireless transceiver, or both include a wireless local area network (WLAN) or Wi-Fi transceiver.

[0167] Clause 50. A non-transitory computer-readable medium storing instructions for radio frequency (RF) sensing of a television user, the instructions including code for performing the following operations: transmitting a first RF signal using one or more wireless transceivers; receiving a first reflected RF signal using one or more wireless transceivers, the first reflected RF signal including reflections of the first RF signal from one or more objects; determining first channel state information (CSI) for one or more wireless channels based on the received first reflected RF signal; determining status information based on the first CSI, wherein the status information includes information about the viewing status of the television user; and performing an action on the television based on the status information.

Claims

1. A method for sensing the radio frequency (RF) of a television user, the method comprising: A first RF signal is transmitted according to a first transmission mode using one or more wireless transceivers; The one or more wireless transceivers are used to receive a first reflected RF signal, the first reflected RF signal including a reflection of the first RF signal from one or more objects; Based on the received first reflected RF signal, determine the first channel state information (CSI) of one or more wireless channels; Based on the first CSI, it is determined that there is a lack of detected movement; In response to determining a lack of movement, the one or more wireless transceivers transmit a second RF signal according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams than the first transmission mode, or any combination thereof; The one or more wireless transceivers are used to receive a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from the one or more objects; The second CSI is determined based on the received second reflected RF signal; Based on the first CSI and the second CSI, status information is determined, wherein the status information includes information about the viewing status of the television user; as well as The television is activated based on the aforementioned status information.

2. The method of claim 1, further comprising: It is determined that the television is playing content of a predetermined type, wherein: Sending the first RF signal is in response to determining that the television is playing content of the predetermined type; and Determining the status information includes determining the attention level of the television user watching the content based on the first CSI.

3. The method of claim 2, wherein, Determining the attention level of the television user watching the content includes: based on the first CSI, determining one or more attributes of the television user while the television is playing, the one or more attributes including: Head facing, Eye direction Posture, or Any combination thereof.

4. The method of claim 3, wherein, Determining one or more attributes of the television user includes comparing information obtained based on the first CSI with stored profile information about the television user.

5. The method according to claim 2, wherein, Performing an action on the television includes transmitting an indication of the attention level from the television to: The service provider's server The content provider's server Or both.

6. The method according to claim 1, further comprising: The identity of the television user is determined by comparing the information obtained from the first CSI with the stored profile information about the television user.

7. The method of claim 1, wherein the status information includes an indication that the television user is not watching television; and Performing an action on the television includes turning it off.

8. The method according to claim 7, wherein, The status information includes information indicating that the television user is no longer detected or that the television user is no longer conscious.

9. The method according to claim 7, further comprising: Before turning off the TV, save the position of the content being played on the TV.

10. The method according to claim 1, wherein, The status information includes: Has the instruction from the TV user been detected? The identity of the television user An indication of whether the television user is watching the content being broadcast on the television, or Any combination thereof.

11. The method according to claim 1, further comprising: Before sending the first RF signal: Calibration is performed for the television user, during which the television user is in the location where they are watching television: The second RF signal is transmitted by the one or more wireless transceivers, and The second reflected RF signal is received by the one or more wireless transceivers; And among them: One or more user attributes of the television user are determined at least in part based on the second CSI; and One or more user attributes are stored in the user profile.

12. The method of claim 1, wherein a first wireless transceiver of the one or more wireless transceivers transmits the first RF signal, and a second wireless transceiver of the one or more wireless transceivers receives the first reflected RF signal, and wherein, The first wireless transceiver is located at a different location than the second wireless transceiver.

13. The method according to claim 1, wherein, The one or more wireless transceivers include one or more wireless local area network (WLAN) or Wi-Fi transceivers.

14. An apparatus for radio frequency (RF) sensing for television users, the apparatus comprising: One or more wireless transceivers; Memory; as well as One or more processing units communicatively coupled to the one or more wireless transceivers and the memory, the one or more processing units being configured to: A first RF signal is transmitted according to a first transmission mode using one or more wireless transceivers; The one or more wireless transceivers are used to receive a first reflected RF signal, the first reflected RF signal including a reflection of the first RF signal from one or more objects; Based on the received first reflected RF signal, determine the first channel state information (CSI) of one or more wireless channels; Based on the first CSI, it is determined that there is a lack of detected movement; In response to determining a lack of movement, the one or more wireless transceivers transmit a second RF signal according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams than the first transmission mode, or any combination thereof; The one or more wireless transceivers are used to receive a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from the one or more objects; The second CSI is determined based on the received second reflected RF signal; Based on the first CSI and the second CSI, situation information is determined, wherein the situation information includes information about the viewing status of the television user; as well as The television is activated based on the aforementioned status information.

15. The device according to claim 14, wherein, The one or more processing units are further configured to determine that the television is playing content of a predetermined type, wherein: The one or more processing units are configured to send the first RF signal in response to determining that the television is playing content of the predetermined type; and In order to determine the status information, the one or more processing units are configured to determine the attention level of the television user watching the content based on the first CSI.

16. The device according to claim 15, wherein, To determine the attention level of the television user while watching the content, the one or more processing units are configured to determine one or more attributes of the television user while the television is playing, based on the first CSI, the one or more attributes including: Head facing, Eye direction Posture, or Any combination thereof.

17. The device according to claim 16, wherein, In order to determine one or more attributes of the television user, the one or more processing units are configured to compare information obtained based on the first CSI with stored profile information about the television user.

18. The device according to claim 15, wherein, In order to perform the action on the television, the one or more processing units are configured to transmit an indication of the attention level from the television to: The service provider's server The content provider's server Or both.

19. The device according to claim 14, wherein, The one or more processing units are further configured to determine the identity of the television user by comparing information obtained from the first CSI with stored profile information about the television user.

20. The device of claim 14, wherein the status information includes an indication that the television user is not watching television; and In order to perform an action on the television, the one or more processing units are configured to turn off the television.

21. The device according to claim 20, wherein, The status information includes information indicating that the television user is no longer detected or that the television user is no longer conscious.

22. The device according to claim 20, wherein, The one or more processing units are also configured to save the position of the content being played on the television in the memory before turning off the television.

23. The device according to claim 14, wherein, The status information includes: Has the instruction from the TV user been detected? The identity of the television user An indication of whether the television user is watching the content being broadcast on the television, or Any combination thereof.

24. The device according to claim 14, wherein, The one or more processing units are also configured to: before transmitting the first RF signal: Calibration is performed for the television user, and during calibration, when the television user is in the location where they are watching television, the one or more processing units are configured to: The second RF signal is transmitted by the one or more wireless transceivers, and The second reflected RF signal is received by the one or more wireless transceivers; Furthermore, the one or more processing units are configured as follows: The television user's one or more user attributes are determined at least in part based on the second CSI; and The one or more user attributes are stored in the user profile in the memory.

25. The device according to claim 14, wherein, The one or more processing units are further configured to: The first RF signal is transmitted using the first wireless transceiver of the one or more wireless transceivers; and The first reflected RF signal is received using the second wireless transceiver among the one or more wireless transceivers; and The first wireless transceiver is located at a different location than the second wireless transceiver.

26. The device according to claim 14, wherein, The one or more wireless transceivers include one or more wireless local area network (WLAN) or Wi-Fi transceivers.

27. An apparatus for radio frequency (RF) sensing for television users, the apparatus comprising: A component for transmitting a first RF signal according to a first transmission mode; A component for receiving a first reflected RF signal, the first reflected RF signal including reflections of the first RF signal from one or more objects; A component used to determine first channel state information (CSI) of one or more wireless channels based on a received first reflected RF signal; Used to determine, based on the first CSI, the missing detected moving component; Components for transmitting a second RF signal according to a second transmission mode in response to determining a lack of movement, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams than the first transmission mode, or any combination thereof; A component for receiving a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from the one or more objects; Components used to determine the second CSI based on the received second reflected RF signal; A component for determining status information based on the first CSI and the second CSI, wherein the status information includes information about the viewing status of the television user; as well as A component for performing actions on the television based on the aforementioned status information.

28. The apparatus of claim 27, further comprising: A component for determining that the television is playing content of a predetermined type, wherein: Sending the first RF signal is in response to determining that the television is playing content of the predetermined type; and Determining the status information includes determining the attention level of the television user watching the content based on the first CSI.

29. The device of claim 27, further comprising a component for determining the identity of the television user by comparing information obtained from the first CSI with stored profile information about the television user.

30. The device of claim 27, wherein the status information includes an indication that the television user is not watching television; and Performing an action on the television includes turning it off.

31. The device according to claim 27, further comprising: A component for performing calibration for the television user, during which the television user is in the location where they are watching television: The second RF signal is transmitted by the device, and The second reflected RF signal is received by the device; And among them: One or more user attributes of the television user are determined at least in part based on the second CSI; and One or more user attributes are stored in the user profile.

32. The device according to claim 27, wherein: The component used to transmit the first RF signal includes a first wireless transceiver, and The component used to receive the first reflected RF signal includes a second wireless transceiver.

33. The device according to claim 32, wherein, The first wireless transceiver is located at a different location than the second wireless transceiver.

34. The device according to claim 32, wherein, The first wireless transceiver, the second wireless transceiver, or both include a wireless local area network (WLAN) or Wi-Fi transceiver.

35. A non-transitory computer-readable medium storing instructions for radio frequency (RF) sensing for a television user, the instructions comprising code for the following operations: A first RF signal is transmitted according to a first transmission mode using one or more wireless transceivers; The one or more wireless transceivers are used to receive a first reflected RF signal, the first reflected RF signal including a reflection of the first RF signal from one or more objects; Based on the received first reflected RF signal, determine the first channel state information (CSI) of one or more wireless channels; Based on the first CSI, it is determined that there is a lack of detected movement; In response to determining a lack of movement, the one or more wireless transceivers transmit a second RF signal according to a second transmission mode, wherein the second transmission mode has: a shorter transmission period than the first transmission mode, a larger transmission bandwidth than the first transmission mode, or a larger number of spatial streams than the first transmission mode, or any combination thereof; The one or more wireless transceivers are used to receive a second reflected RF signal, the second reflected RF signal including a reflection of the second RF signal from the one or more objects; The second CSI is determined based on the received second reflected RF signal; Based on the first CSI and the second CSI, situation information is determined, wherein the situation information includes information about the viewing status of the television user; as well as The television is activated based on the aforementioned status information.

Images