Object embedding for decoding radio frequency signal reflections as contextual triggers

Abstract

This document describes systems and techniques aimed at decoding radio frequency signal reflections as object embedding of contextual triggers. In aspects, a computing device having a radar system and a radar manager is configured to transmit a transmit waveform signal and receive a reflected waveform signal that includes a version of the transmit waveform signal reflected by an object. Based on the reflected waveform signal, the radar manager generates an object embedding associated with the object and compares the object embedding to a previous object embedding to provide a comparison result. The radar manager determines that the object embedding and the previous object embedding are associated with the same object based on a result of the comparison. The radar manager communicates the determination to a computing device, which triggers a contextual event based on the determination.

Description

Background Art

[0001] Computing devices have become ubiquitous sensors in the world around us. Those computing devices that include cameras can visually identify the faces of individuals, the species of animals, and even the names of many plant types. Those computing devices that include microphones can audibly identify the voices of individuals, songs, and even translate languages in real time. Those computing devices that include Global Positioning System (GPS) radios can provide real-time location tracking and direction planning.

[0002] However, some of these capabilities may not be achievable without user input, can raise privacy concerns, and may not be able to identify specific objects. For example, the cameras of computing devices generally cannot operate without user input. Further, due to privacy concerns, cameras cannot operate continuously and can cause battery drain in mobile computing devices. Still further, cameras may be sufficient to detect or identify object categories (e.g., dogs, cats), but they may not be accurate enough to identify specific objects within an object category (e.g., the user's dog, the user's cat). Summary of the Invention

[0003] This document describes systems and techniques for object embedding that aim to reflectively decode radio frequency (RF) signals into context triggers. In various aspects, a computing device (e.g., a smart phone, a tablet computer) having a radar system and a radar manager is configured to transmit a transmit waveform signal and receive a reflected waveform signal, the reflected waveform signal including a reflected version of the transmit waveform signal by an object. The radar system can include an antenna array and a transceiver, the transceiver including at least one transmit channel and at least one receive channel, each channel being coupled to an antenna element of the antenna array respectively. The transmit channel and the corresponding antenna element can be used to transmit the transmit waveform signal, which can be a coded waveform (e.g., fixed, known). Similarly, the receive channel and the corresponding antenna element can be used to receive the reflected waveform signal.

[0004] Based on the reflected waveform signal, the radar manager can generate an object embedding associated with an object. The reflected waveform signal can be sampled and correlated with the stored code of the transmitted waveform signal to generate a channel impulse response (CIR). The CIR, the reflected waveform signal, or a combination of both can be provided as an input (e.g., as a result of an offline-trained neural network) to an embedding model stored in the memory of a computing device to generate an object embedding. When generating the object embedding, any one of a variety of loss functions (e.g., triplet loss) can be used. The radar manager can compare the object embedding with a previous object embedding to provide a comparison result. The previous object embedding can be one that has been generated by the radar manager during an out-of-the-box initialization process, where the user can generate various object embeddings for various objects including personal objects in a camera-like manner (e.g., point-and-shoot). The comparison result can be a boolean result (e.g., indicating a match or a mismatch), a ranked result (e.g., how likely it is to match), etc.

[0005] The radar manager can determine that the object embedding and the previous object embedding are associated with the same object based on the comparison result. For example, the user can calibrate the radar manager by scanning his front door during the initialization process. Later, the user may leave his home, close the front door, and scan the front door to indicate to the radar manager that he is leaving home. The radar manager can communicate this determination to the computing device, which can trigger a context event based on this determination. Continuing with this example, the user can set a smart home application with an automatic lock and a security system to be armed when he leaves his home. Instead of accessing the application and manually arming it, the user can set the context event to be the locking of his automatic lock and the arming of his security system. In this way, the user can simply scan his front door when he leaves his home to lock the locks and arm the security system (e.g., trigger a context event).

[0006] Details of one or more implementations are set forth in the accompanying drawings and the following detailed description. Other features and advantages will be apparent from the detailed description, the drawings, and the claims. This summary is provided to introduce the subject matter further described in the detailed description. Accordingly, the reader should not regard the summary as defining essential features or as limiting the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] This document describes details of one or more aspects of decoding a radio frequency (RF) signal reflection into an object embedding for a context trigger with reference to the following drawings. Throughout the drawings, the use of the same numbers in different instances may indicate similar features or components. Figure 1An example environment of a computing device having a radar system and a radar manager is shown, the radar manager being configured to decode reflected RF signals into object embeddings of context triggers; Figure 2 Shows Figure 1 an example implementation of the computing device in [ ], the computing device being configured to provide a radar manager; Figure 3A A plan view of an example implementation of a computing device having a radar system is shown; Figure 3B Shows in more detail Figure 3A a partial view of an example implementation of the radar system from [ ]; Figure 4 Shows various examples of relative power spectral density and radio frequency that the radar manager can utilize; Figure 5A An example implementation of a computing device transmitting a transmit waveform signal is shown; Figure 5B Shows Figure 5A an example implementation of a computing device receiving a reflected waveform signal from [ ]; Figure 6 An example method of the radar manager generating an object embedding based on a reflected RF signal is shown; Figure 7 An example method of the radar manager triggering a context event based on a reflected RF signal is shown; and Figure 8 An example method for decoding a reflected RF signal into an object embedding of a context trigger is shown. Detailed Description Overview

[0008] A computing device (e.g., a smart phone) typically includes multiple sensors to facilitate various functions and improve the user experience. An image sensor (e.g., a camera) enables the computing device to capture photos and videos. A microphone enables the computing device to capture audio and enables the user to call friends and family. The computing device can further utilize the camera to visually identify objects, including various animals and individual faces. The computing device can further utilize the microphone to identify songs and receive voice commands.

[0009] However, cameras and microphones can have certain limitations. For example, continuously recording video or audio can result in significant battery consumption of the mobile computing device. Further, especially in protected spaces such as restrooms or dressing rooms, continuous video or audio recording can raise privacy concerns. As another example, although cameras can enable a computing device to visually identify objects, they typically identify them as groups (e.g., cats, dogs) rather than specific objects within the group (e.g., the user's cat, the user's dog). Further, many of these functions are active and require direct attention or input from the user, making them not optimal for passive operations such as passive object recognition.

[0010] This document describes systems and techniques for object embedding that aim to reflectively decode RF signals into context triggers. The disclosed systems and techniques can address the drawbacks of sensors that require direct attention or input from the user, the drawbacks of sensors that can result in significant battery consumption in a mobile computing device, and the drawbacks of sensors that can cause privacy concerns in protected spaces. The conflicts among these drawbacks can be resolved by the disclosed systems and techniques, which can provide passive object recognition for triggering context events, thereby improving the user experience.

[0011] The following discussion describes an operating environment, techniques that can be employed in the operating environment, example devices, and example methods. Although systems and techniques for object embedding that reflectively decode RF signals into context triggers are described, it should be understood that the subject matter of the appended claims need not be limited to the specific features or methods described. Rather, the specific features and methods are disclosed as example implementations and are referenced only by way of example. Example Operating Environment

[0012] Figure 1An example environment 100 of a computing device 102 having a radar system 104 and a radar manager 114 is shown. The radar manager is configured to decode RF signal reflections into object embeddings of context triggers. As shown, the computing device 102 further includes a processor 110 and a computer-readable medium 112 (CRM 112). The radar system 104 may include an antenna array 106 and a transceiver 108. The antenna array 106 may include one or more antenna elements. The transceiver 108 may include one or more transmit channels and one or more receive channels respectively coupled to elements of the antenna array 106. The transceiver 108 may be configured to use any of a variety of frequencies or power spectral densities in the RF portion of the electromagnetic spectrum. For example, the transceiver 108 may be configured to use ultra-wideband (UWB) signals that include a bandwidth of approximately 500 megahertz (MHz) centered at a center frequency. The center frequency may include any frequency from approximately 3.1 gigahertz (GHz) to 10.5 GHz. Further, the transceiver 108 may be configured to transmit such signals at a lower power spectral density, providing additional benefits to the radar system 104. The additional benefits may include: operating the radar system 104 in continuous mode without draining a battery (not shown), experiencing reduced in-band interference from other narrowband RF signals, and being safer than alternative power spectral densities because the low power spectral density makes the signal difficult to detect.

[0013] The processor 110 may be any suitable single-core or multi-core processor, including a central processing unit (CPU), a graphics processing unit (GPU), an advanced RISC machine (ARM), etc. The CRM 112 may include a memory medium (e.g., dynamic random access memory (DRAM)) and a storage medium (e.g., solid-state drive (SSD)). The CRM 112 may include various computer-readable instructions executable by the processor 110 to provide at least some of the functions described herein. The computer-readable instructions may include various applications, an operating system (OS), and so on. Figure 1 Shown are computer-readable instructions of the CRM 112 that include a radar manager 114 configured to decode RF signal reflections into object embeddings of context triggers.

[0014] Figure 1 Further shown is a user 116 of the computing device 102 who wishes to use the radar manager 114 to scan a vehicle key 118. To scan the vehicle key 118, the user 116 (not shown) orients the computing device 102 such that the radar system 104 is aligned with the vehicle key 118. During initialization or out-of-the-box experience of the computing device 102 or its radar manager 114, the user 116 may provide user input (e.g., touch input) to scan the vehicle key 118 for the first time.

[0015] In response to a user input, the radar manager 114 (e.g., using the radar system 104) transmits a transmitted waveform signal 120 that points towards the vehicle key 118 along the negative Z-axis, as shown. As described above, the transmitted waveform signal 120 can be a UWB signal or another suitable RF signal. The transmitted waveform signal 120 can be temporally constrained (e.g., limited to less than two nanoseconds (ns)) and includes various RF pulses of various frequencies included in the UWB signal spectrum. The transmitted waveform signal 120 can be optimized such that it includes a known code (e.g., a sequence of pulses with known frequencies).

[0016] The transmitted waveform signal 120 can be reflected by the vehicle key 118 into a reflected waveform signal that includes a version of the transmitted waveform signal 120. This means that, for example, the amplitude, phase, and / or frequency of the reflected waveform signal can be different from the transmitted waveform signal. In this example environment 100, the reflected waveform signal includes a first reflected waveform signal 122a and a second reflected waveform signal 122b. The first reflected waveform signal 122a and the second reflected waveform signal 122b can represent the surface and subsurface reflections of the transmitted waveform signal 120 from the surface of the vehicle key 118. Additionally or alternatively, the first reflected waveform signal 122a and the second reflected waveform signal 122b can represent the surface reflections of the transmitted waveform signal 120 from different and corresponding positions of the vehicle key 118.

[0017] The radar manager 114 receives the first reflected waveform signal 122a and the second reflected waveform signal 122b along the positive Z-axis, as shown. The radar manager 114 can generate an object embedding associated with the vehicle key 118 based on the received signals. The object embedding can be a rotation-invariant representation of the numerical values of the received signals and can be generated by an embedding model. For example, the embedding model can be stored as computer-readable instructions on the CRM 112 as part of the radar manager 114. The embedding model can be the result of an embedding network or other suitable neural network that is trained offline on various reflected waveform signals associated with respective objects as input data. The embedding network can use any of various loss functions (e.g., triplet loss) in order to generate an optimal embedding model as output data.

[0018] When generating an object embedding, the radar manager 114 may preprocess the reflected waveform signal before generating the final object embedding. For example, the radar manager 114 may sample portions of the first reflected waveform signal 122a and / or the second reflected waveform signal 122b to correlate with the transmitted waveform signal 120. The radar manager 114 may generate a channel impulse response (CIR) based on this correlation, which may be stored as an object embedding (e.g., on the CRM 112). The CIR may be a mathematical representation of the reflected waveform signal generated by the reflection of the transmitted waveform signal (e.g., a pulse) by the vehicle key 118. The radar manager 114 may further preprocess the reflected waveform signal by passing the CIR through a logarithmic scaling function, which may improve the dynamic range of the CIR to make it more suitable for upstream neural network processing. The logarithmic scaling function may transform the mathematical representation of the reflected waveform included in the CIR from the linear domain to the logarithmic domain. The radar manager 114 does this because, for example, many data sets (e.g., semiconductor degradation, planetary orbital period or radius) are better analyzed in the logarithmic domain. As another example, some data distributions may be severely skewed towards the high or low end in the linear domain (e.g., compared to the mean or median), but are normally distributed in the logarithmic domain. Once completed, the object embedding associated with the vehicle key 118 may be saved in a look-up table (LUT), e.g., stored as computer-readable data on the CRM 112.

[0019] Although Figure 1 not shown in the figure, the computing device 102 may include a vehicle application associated with the vehicle of the user 116. The user 116 may instruct the radar manager 114 that subsequent scans of the vehicle key 118 should trigger a context event. For example, the user 116 may program the vehicle application to start the vehicle upon subsequent scanning of the vehicle key 118. As another example, the computing device 102 may be set to operate in a vehicle-friendly operation mode due to a subsequent scan of the vehicle key 118. Further, because the radar system 104 may be configured to utilize UWB signals with low power spectral density, the radar manager 114 may operate in a continuous mode without explicit input from the user 116. Even further, because UWB and other suitable RF signals can penetrate materials, the vehicle key 118 does not necessarily need to be in direct line of sight with the computing device 102. For example, the user 116 may passively scan the vehicle key 118 by positioning the computing device 102 such that the radar system 104 is oriented towards the vehicle key 118 within the user 116's pants pocket.

[0020] In this manner, the radar manager 114 decodes the reflection of the RF signal into an object embedding of the context trigger. By doing so, the radar manager 114 enables the user 116 to passively trigger context events (e.g., start a vehicle, enter a vehicle-friendly operation mode of the computing device 102) using the radar system 104, the antenna array 106, and the transceiver 108. Further, the radar manager 114 enables the user 116 to passively benefit from this functionality without privacy concerns, while resulting in reduced battery consumption. Example device

[0021] Figure 2 More particularly, an example implementation 200 of the computing device 102 from Figure 1 is shown, which is configured to provide the radar manager 114. The computing device 102 is shown as various example devices. As a non-limiting example, the computing device 102 can be a smart phone 202a, a tablet computer 202b, a laptop computer 202c, a desktop computer 202d, a smart watch 202e, a pair of smart glasses 202f, a game controller 202g, a smart home speaker 202h, or a vehicle 202i. Although not shown, the computing device 102 can also be implemented as a health monitoring device, a personal medical device, a drone, a household appliance, a security system or its devices, a digital photo frame, and so on. The computing device 102 can be wearable, non-wearable but mobile, or relatively stationary. Further, the computing device 102 can be used with or embedded in many computing devices or peripheral devices (e.g., vehicles, personal computers). The computing device 102 can also include additional interfaces or components omitted from Figure 2 omitted.

[0022] Figure 2 The computing device 102 is shown to include various components described with reference to Figure 1 including a radar system 104, an antenna array 106, a transceiver 108, a processor 110, a CRM 112, and a radar manager 114. Figure 2 It is further shown that the CRM 112 can include a memory medium 202 and a storage medium 204. The memory medium 202 can include one or more non-transitory storage devices, including random access memory (RAM) or DRAM. The storage medium 204 can include one or more transitory storage devices, including SSD or magnetic rotating hard disk drive (HDD). The CRM 112 can further include an operating system 206 (OS 206) and an application 208, which can be stored on the CRM 112 as computer-readable instructions. The processor 110 can execute the computer-readable instructions on the CRM 112 to provide some or all of the functions described herein.

[0023] Figure 2 It is also shown that the computing device 102 includes one or more sensors 210 and a display 212. The sensors 210 may include image sensors, microphones, accelerometers, barometers, ambient light sensors, thermometers, and so on. The display 212 may be implemented as any of a variety of display technologies. Some display technologies include liquid crystal displays (LCDs), light emitting diode (LED) displays, organic LED (OLED) displays, twisted nematic displays, in-plane switching displays, etc. Although not shown, the display 212 may be paired with a touch screen or another suitable touch input device such that a user (e.g., Figure 1 user 116) of) may provide touch input (e.g., during the initialization process of the radar manager 114 described in reference Figure 1 ).

[0024] In an implementation, the radar manager 114 may include one or more integrated circuits (ICs), system on a chip (SOC), secure key storage, hardware embedded with firmware, a printed circuit board (PCB) having various hardware components, or any combination thereof. As described herein, the radar manager 114 may include one or more components of the computing device 102, such as Figure 1 and Figure 2 shown, the one or more components are configured to decode the reflection of RF signals into object embeddings of scene triggers. In other implementations, the radar manager 114 may be implemented as the computing device 102.

[0025] Although not shown, the computing device 102 may also include input / output (I / O) ports, a system bus, interconnects, or another data transfer system coupled to various components of or within the computing device 102. As an example, the I / O ports may enable the computing device 102 to interact with other devices or users via peripheral devices, thereby sending any combination of digital signals and / or analog signals in a wired manner (e.g., Ethernet) or a wireless manner (e.g., radio). The I / O ports may include any combination of internal or external ports, including universal serial bus ports, audio ports, video ports, and so on. Various peripheral devices (e.g., manual input devices, external CRMs, speakers, and displays) may be coupled to the I / O ports.

[0026] Figure 3A A plan view of an example implementation 300 of a computing device 302 having a radar system 304 is shown. In addition to what is detailed below, the computing device 302 and the radar system 304 are the same as Figure 1 and Figure 2The computing device 102 and the radar system 104 shown in and described above are similar. Thus, although not shown, the computing device 102 may include one or more processors, a CRM that stores computer-readable instructions (e.g., Figure 1 or Figure 2 the radar manager 114), an OS (e.g., Figure 2 the OS 206), and an application (e.g., Figure 2 the application 208). The radar system 304 similarly includes an antenna array and a transceiver shown in Figure 3B and described below.

[0027] As Figure 3A shown, an example implementation 300 of the computing device 302 includes a radar system 304 located in a top center position within the housing (not shown) of the computing device 302. Although the top center position is shown, the radar system 304 may be located anywhere on or in the computing device 302. Figure 3A The computing device 302 is further shown to include a camera module 306. The camera module 306 may include a first camera 308, a second camera 310, a microphone 312, and an illuminator 314 (e.g., a "flash"). The first camera 308 may be a wide-angle camera, and the second camera 310 may be a high-power zoom camera. The microphone 312 may be any suitable microphone, including a dynamic microphone, a condenser microphone, a ribbon microphone, etc. The first camera 308, the second camera 310, and the microphone 312 are examples of additional sensors that the computing device 302 may include. The illuminator 314 may be any suitable flash, including an LED flash.

[0028] Figure 3B A partial view of the radar system 304 in Figure 3A is shown in more detail. The radar system 304 may be implemented on a main logic board 316 (MLB 316) or another suitable PCB (including a main board and / or a daughter board). The radar system 304 may further include an antenna array that includes a first antenna 318 and a second antenna 320. The radar system 304 may also include a transceiver 322. The first antenna 318, the second antenna 320, and the transceiver 322 may be coupled (e.g., by soldering, electrically, physically coupled) to the MLB 316.

[0029] Figure 3BAlso shown are a first antenna 318 and a second antenna 320 coupled to a transceiver 322 via a first channel 324 and a second channel 326, respectively. The first channel 324 and the second channel 326 may be implemented as one or more suitable single-bit or multi-bit buses. Further, the first channel 324 and the second channel 326 may be configured as a transmit channel, a receive channel, or both. As an example, the first antenna 318 and the first channel 324 may be a transmit antenna and a transmit channel. Thus, the second antenna 320 and the second channel 326 may be a receive antenna and a receive channel. Alternatively, the configuration of the antennas and channels may be reversed. Additionally or alternatively, both the first antenna 318 and the second antenna 320 and the corresponding first channel 324 and second channel 326 may be transceiver antennas and channels. The radar system 304 and its components may be configured to transmit and receive RF signals of a particular power spectral density and frequency.

[0030] Figure 4 An example graph 400 of relative power spectral density versus frequency that may be utilized by a radar system is shown. In addition to the details described below, the radar system is similar to the radar system 104 shown in Figure 1 and Figure 2 and the radar system 304 shown in Figure 3A and Figure 3B Thus, the radar system may include an antenna array, a transceiver, and may be configured to transmit and receive RF signals of one or more frequencies and power spectral densities.

[0031] The example graph 400 includes power spectral density, typically measured in decibels per milliwatt per megahertz (dBm / MHz), on the Y-axis 402 versus frequency, typically measured in gigahertz (GHz), on the X-axis 404. For simplicity, specific power spectral density values are omitted from Figure 4 The example frequencies are marked along the frequency X-axis 404. Further, the relative shapes and dimensions shown are for illustrative purposes only and should not be construed as quantitative values.

[0032] The example graph 400 includes a noise floor 406 that various RF signals must exceed in terms of relative power spectral density in order to be properly received and interpreted by a computing device. Figure 4 Also shown are respective RF frequency bands having a power spectral density above the noise floor 406. The respective RF frequency bands include a first band 408, a second band 410, a third band 412, and a fourth band 414.

[0033] As an example, the first frequency band 408 can be a sub-1 GHz band, such as an Industrial, Scientific, and Medical (ISM) band, which is an unlicensed band for industrial, scientific, and medical use. The sub-1 GHz band can be used for short-range transmissions by various consumer electronic products including garage door openers, televisions, radio-controlled (RC) cars, etc. The second frequency band 410 can be a Global Positioning System (GPS) band, which can be used for long-range communication between GPS satellites and GPS-enabled devices (e.g., Figure 1 the computing device 102). The third frequency band 412 can be a 2.4 GHz Wireless Local Area Network (WLAN) band utilized in a user's home for WLAN functionality and various applications that may require longer range but slower speed wireless communication. The fourth frequency band 414 can be a 5 GHz WLAN band utilized similarly to the second frequency band 412, but for home WLAN applications that may require higher speed but shorter range wireless communication.

[0034] Finally, Figure 4 a fifth frequency band 416 is shown, which is significantly wider than the first through fourth frequency bands 408 - 414. Additionally, compared to the second through fourth frequency bands 410 - 414, the fifth frequency band 416 utilizes a significantly lower power spectral density. The fifth frequency band can be an Ultra-Wideband (UWB) band configured for short-range, low-power wireless communication. The UWB band can enable multiple RF channels within the UWB band that can have a large bandwidth, such as 500 MHz centered around a center frequency. The center frequency can include any frequency from approximately 3.1 GHz to 10.6 GHz. The large bandwidth channels and wide range of center frequencies enable devices (e.g., Figure 1 and Figure 2 the computing device 102, the computing device 302 of FIG. 3) to perform accurate real-time movement tracking, Line-of-Sight (LoS) calculations, and precise positioning in non-LoS scenarios. Further, the low power spectral density enables UWB devices to operate in a continuous mode without concern for battery depletion or interference with other wireless bands (e.g., the first frequency band 408, the third frequency band 412). From a security perspective, the lower power spectral density is also beneficial for UWB devices because the low power makes it difficult to detect and / or intercept UWB communication. Thus, the fifth frequency band 416 can be an optimal choice for providing a radar manager (e.g., radar manager 114) that utilizes a radar system (e.g., radar system 104) configured to decode RF signals reflected by objects embedded as scene triggers.

[0035] Figure 5A An example implementation 500 of a computing device 502 that transmits a transmit waveform signal 506 is shown. In addition to what is detailed below, the computing device 502 is the same as that in Figure 1 , Figure 2is similar to the computing devices 102 and 302 shown and described above. Thus, the computing device 502 has a radar system 504 and a radar manager (not shown), the radar system including an antenna array and a transceiver (not shown), and the radar manager being configured to transmit a transmit waveform signal 506.

[0036] As shown, the radar manager uses the radar system 504 to transmit the transmit waveform signal 506 along the negative Z-axis towards the object 508. The transmit waveform signal 506 may have, for example, a center frequency of 7 GHz and a bandwidth of 500 MHz. Thus, the transmit waveform signal 506 may utilize the UWB (e.g., Figure 4 the fifth frequency band 416). Further, the transmit waveform signal 506 may be a coded signal such that it includes a predetermined number of pulses having a specific frequency and duration. Continuing with this example, the predetermined number (e.g., five, six, 10 or more) of time-constrained pulses (e.g., one-tenth of a nanosecond (ns), one ns, three ns) have a frequency close to the center frequency of 7 GHz and are within the 500 MHz bandwidth. That is, the frequencies may include 6.7 GHz, 7 GHz, 7.2 GHz, etc. The object 508 may be any object, including a user's personal objects (e.g., car key 118, front door of a home, keyboard, watch, pet) and commercial objects (e.g., cash register, item scanner).

[0037] Figure 5B shows an example implementation 500 of the computing device 502 for the received reflected waveform signal 510 from Figure 5A . The reflected waveform signal 510 includes Figure 5A a version of the transmit waveform signal 506 reflected by the object 508. Since RF signals can pass through some materials, the reflected waveform signal 510 may include one or more surface reflections and / or one or more subsurface reflections. As shown, the reflected waveform signal includes a first reflection 510a and a second reflection 510b. As an example, the object 508 may be an apple, and thus the first reflection 510a may be a surface reflection off the apple skin, and the second reflection 510b may be a subsurface reflection off the apple seed. As another example, both the first reflection 510a and the second reflection 510b may be surface reflections off the apple skin but from different locations (e.g., locations closer to the computing device 502).

[0038] The radar manager of computing device 502 may receive a reflected waveform signal 510 using radar system 504 and its components. The radar manager may generate an object embedding associated with object 508 based on the reflected waveform signal 510. The radar manager may further compare the object embedding with a previous object embedding to determine a comparison result and trigger a context event. The comparison may include comparing the object embedding with a previous object embedding stored in a look-up table (LUT), which may be populated by a user (e.g., user 116) during an initialization or out-of-the-box experience associated with the radar manager. Example method

[0039] Figure 6 An example method 600 is shown in which a radar manager generates object embeddings associated with various objects to populate LUT 612. The radar manager is similar to the previously shown and described radar manager, except as detailed below. Thus, the radar manager may be stored as computer-readable instructions on a CRM (e.g., Figure 1 and Figure 2 the CRM 112 of computing device 102 of FIG. 1, computing device 502 of FIG. 5) of a computing device (e.g., Figure 1 and Figure 2 the CRM 112 of FIG. 1). Further, the radar manager may utilize a radar system (e.g., radar system 504 of FIG. 5) and may be configured to decode RF signals into object embeddings for context triggers.

[0040] As shown, Figure 6 includes a first object 602a, a second object 602b, and a third object 602c. Object 602 may be any of a variety of objects that a user may scan using the radar manager during an initialization process of the radar manager. The term "scan" may refer to transmitting a transmit waveform signal and receiving a reflected waveform signal that includes a version of the transmit waveform signal reflected from an object. In other words, the term "scan" may be considered taking a photograph of an object in the RF domain of the electromagnetic spectrum rather than the visible domain. As an example, object 602 may be the front or back door of a user's home, a garage door, a cat, a dog, a water bottle, fitness equipment, clothing or shoes, a hand, a foot, etc.

[0041] Once the user scans the first object 602a, the second object 602b, and the third object 602b, the radar manager can receive the corresponding reflected waveform signals as data 604 associated with the objects 602. The data 604 can include any suitable data (e.g., numerical values) associated with the reflected waveform signals of the various objects 602. The data 604 can be numerical values representing the original frequency, amplitude, and / or phase of the reflected waveform signals. Alternatively, the data 604 can be numerical values representing the frequency, amplitude, and / or phase difference of the reflected waveform signals compared to the transmitted waveform signals. In this example, the data 604 includes first data 604a associated with the first object 602a, second data 604b associated with the second object 602b, and third data 604c associated with the third object 602c. The data 604 can be the channel impulse response (CIR) of the corresponding reflected waveform signals of the objects 602. The CIR can include samples of the corresponding reflected waveform signals related to the transmitted waveform signal, which is known and fixed (e.g., coded, predetermined). This means that regardless of the object reflecting the transmitted waveform signal, the transmitted waveform signal can include a specific frequency, phase, amplitude, and duration.

[0042] Figure 6 Further shown is a preprocessor 606, which can include a first preprocessor 606a, a second preprocessor 606b, and a third preprocessor 606c. The preprocessors 606a to 606c can be the only cores within the preprocessor 606, the only threads on which the preprocessor 606 operates, or completely independent preprocessors. The preprocessor 606 and its cores can be implemented as processors (e.g., Figure 1 processor 110 of Figure 1 computing device 102) of a computing device (e.g.,

[0043] Figure 6Also shown is an embedder 608, which may include a first embedder 608a, a second embedder 608b, and a third embedder 608c. Similar to preprocessors 606a to 606c, embedders 608a to 608c may be implemented as instances of embedder 608, completely independent embedders, multi-core embedders, and so on. Embedder 608 may utilize an embedding model generated by an embedding network or other suitable neural network trained offline. The offline training of the embedding network may include providing various reflected waveform signals associated with various objects as input data to the embedding network. The embedding network may utilize any of various loss functions (e.g., triplet loss, Euclidean distance, L2 squared distance) to generate an embedding model as output data. Alternatively or additionally, the embedding model may be based on physical transformations (e.g., rotation, translation, scaling). Then, the embedding model may be stored on the CRM of a computing device as, for example, computer-readable instructions for use by a radar manager to decode the RF signal reflection into an object embedding of a scenario trigger.

[0044] Embedders 608a to 608c may utilize the embedding model to generate object embeddings for objects 602a to 602c respectively based on the corresponding preprocessed reflected waveform signals. The object embeddings may be provided to a processor 610, which may be any suitable processor, including a single-core or multi-core CPU or GPU. Processor 610 may minimize translations or other physical transformations for the object embeddings. Processor 610 may additionally or alternatively post-process these object embeddings such that they are rotation-invariant numerical representations. Processor 610 may further use a loss function (e.g., Euclidean loss, triplet loss) to do this.

[0045] Further, processor 610 may organize the object embeddings associated with objects 602 into a LUT 612. LUT 612 may be stored on the CRM (e.g., Figure 1 and Figure 2 the CRM 112 of computing device 102 of FIG. 1, computing device 502 of FIG. 5) of a computing device as a computer-readable medium. The radar manager may refer to LUT 612 to compare real-time object embeddings with previously stored object embeddings in LUT 612. An example of real-time object embedding comparison is described in detail below with respect to Figure 1 and Figure 2 FIG. Figure 7 An example of real-time object embedding comparison is described in detail below.

[0046] Figure 7 Shown is an example method 700 in which a computing device with a radar manager triggers a predetermined action 710 (e.g., running a script, opening an application, switching settings) based on a reflected RF signal. In addition to the details described below, the computing device and the radar manager are related to Figure 1Similar to the computing devices and radar managers shown in FIGS. 3 and 5 and described above. Thus, the computing device may include one or more processors, a CRM, one or more sensors, an OS, a radar system, and so on. The radar manager may be stored as computer-readable instructions on the CRM of the computing device and may utilize the radar system to transmit RF signals and receive RF signal reflections.

[0047] As shown, in addition to the details described below, Figure 7 Method 700 is similar to Figure 6 Method 600. Method 600 details the initialization process that a user may experience for causing the radar manager of a computing device to generate a LUT of previous object embeddings, while method 700 details real-time object embedding comparison. Method 700 may be performed passively by the radar manager, the computing device, and its components. That is, the radar manager may passively and continuously transmit transmit waveform signals without user input. The radar manager may do so by using a low-power UWB band of the RF spectrum (e.g., Figure 4 the fifth band 416 in

[0048] By passively transmitting transmit waveform signals, the radar manager may passively receive reflected waveform signals that include a version of the transmit waveform signal reflected by object 702. As an example, object 702 may be the front door of the user's home. Further, the user may have a smart home application that facilitates communication between the computing device and smart home devices (e.g., a security system, an automatic dead bolt lock). The user may have programmed the smart home application to arm the security system and lock the automatic dead bolt lock based on object embedding matches determined by the radar manager during the radar manager initialization process described with respect to Figure 6

[0049] Figure 7 Data 704 is further shown, which may include the reflected waveform signal reflected from object 702. The radar manager may sample a portion of data 704 to correlate it with a known and encoded transmit waveform signal (e.g., including known pulses with known frequencies and amplitudes). This correlation may generate a CIR for object 702 based on data 704. As shown, the radar manager may deliver data 704 to a preprocessor 606, which may improve the dynamic range of the CIR using logarithmic scaling. The preprocessor 606 may also apply noise reduction or signal amplification techniques to data 704.

[0050] ​The radar manager utilizes the embedder 608 to generate an object embedding associated with the object 702 based on the preprocessed data 704. The embedder 608 may utilize an embedding model generated offline by an embedding network or other suitable neural network (e.g., an L2 neural network). The embedding network may be trained on various object embeddings or CIRs as input data and generate the embedding model as output data based on a cost function. Further, the embedder 608 may generate the object embedding as an embedding vector, which is a rotation-invariant numerical representation of the object embedding or the associated CIR. The embedder 608 may generate the object embedding associated with the object based on the cost function of the embedding model such that the object embedding is grouped together with similar object embeddings and placed far from dissimilar object embeddings in the embedding space.

[0051] As Figure 7 shown, the radar manager provides the object embedding from the embedder 608 to the processor 610. The processor 610 may further post-process these object embeddings to minimize the translation of the object embeddings or make the object embeddings rotation-invariant. The radar manager may test the match of the object embedding associated with the object 702 with the object embeddings included in the LUT 612 at 708. The LUT 612 may include a library of previous object embeddings or other suitable list associated with previous objects that the user may have scanned during the initialization process. If the radar manager determines that the object embedding matches one of the previous object embeddings included in the LUT 612, the radar manager may communicate the determination (e.g., communicate to the computing device) to trigger a context event (e.g., by the computing device).

[0052] As an example, the user may scan a pair of running shoes, either standalone or worn on the user's feet, to generate an object embedding associated with the running shoes. The radar manager may determine that the object embedding matches the previous object embedding associated with the pair of running shoes based on the comparison with the LUT 612. Additionally, the user may have scanned the pair of shoes during the initialization process, as described with respect to Figure 6 above. The radar manager may communicate the determination to the computing device, which may then trigger a fitness application to open or start tracking an exercise. In this example, the determination is a match between the object embedding of the running shoes and the previous object embedding of the running shoes, and the context event includes opening the fitness application.

[0053] As another example, a user can scan his hand— which may include a watch or wedding ring—to trigger a change in authorization (e.g., authentication, unlocking) of high-rights resources (e.g., financial accounts, password managers) associated with a computing device. The reflected waveform signal as part of the scan can include a portion of the transmitted waveform signal reflected from the hand, the watch or ring, or both. A financial account or other high-rights resource may require two-factor authentication (2FA). The method 700 of scanning the hand can be used as one of the two factors of 2FA. The other factor of 2FA can be simultaneous biometric authentication (e.g., fingerprint authentication, face recognition) performed by the computing device. Additionally or alternatively, the radar manager can execute method 700 in response to the computing device being in an unlocked state but a financial account or other high-rights resource being in a locked state.

[0054] As yet another additional example, a user can scan a smartwatch to trigger a context event that includes delivering the contents of the smartwatch to the display of a computing device. The user can scan any of a variety of skeuomorphic printouts (e.g., a toy that looks like a cloud) to open a weather application on the computing device. Another printout can include a toy that looks like a letter to open an email application on the computing device.

[0055] Alternatively, the radar manager can determine that an object embedding associated with object 702 does not match a previous object embedding included in LUT 612. The mismatch can include a tolerance (e.g., percentage) based on the Euclidean or L2 distance between the object embedding and the previous object embedding. The radar manager can communicate the determination to the computing device, which can provide an error message 712 to the user of the computing device.

[0056] Figure 8 An example method 800 for decoding a reflected RF signal into an object embedding for a context trigger is shown. In aspects, a computing device includes a radar manager and a radar system. The radar system can include one or more antenna elements respectively coupled to a transceiver including one or more transmit or receive channels. The radar manager can execute method 800 by leveraging the radar system and other unspecified components of the computing device.

[0057] At 802, the radar manager transmits a transmitted waveform signal. The radar manager may utilize a transmit antenna among one or more antenna elements and a corresponding transmit channel of the transceiver. The transmitted waveform signal may be a time-constrained UWB RF signal such that transmitting the transmitted waveform signal does not require a long duration (e.g., 1 second (s), 2 s, 5 s). Further, the transmitted waveform signal may be encoded to include various pulses having specific frequencies or multiple frequencies centered at a center frequency. The transmitted waveform signal may be a short-range low-power signal such that battery consumption can be minimized for the computing device. Additionally, the low-power signal may be difficult for other devices or users to detect, making method 800 secure.

[0058] At 804, the radar manager receives a reflected waveform signal that includes a version of the transmitted waveform signal reflected by an object. The object may include a surface and a subsurface. The reflected waveform signal may include a version of the transmitted waveform signal reflected from the surface, subsurface, or both of the object. The object may be any of various personal objects (e.g., a pet, a toy, a possession), commercial objects (e.g., a cash register, a fuel pump), body parts (e.g., a hand, a foot), or other objects. Further, the object may include a first object and a second object (e.g., a hand and a wedding ring, a hand and a smartwatch), and the reflected waveform signal may include a version of the transmitted waveform signal reflected by the first object, the second object, or both.

[0059] At 806, the radar manager generates an object embedding associated with the object. As described above, the object embedding may be based on an embedding model generated by offline training of an embedding network. The object embedding may be a rotation-invariant numerical representation of the reflected waveform signal. Additionally or alternatively, the object embedding may be a physical transformation (e.g., translation, rotation) of data associated with the reflected waveform signal.

[0060] At 808, the radar manager determines that the object embedding and a previous object embedding are associated with the same object. For example, the radar manager may make this determination based on a tolerance (e.g., a percentage, statistical significance) between the object embedding and the previous object embedding. The radar manager may make a comparison between the object embedding and the previous object embedding by comparing the rotation-invariant numerical representations of the object embeddings with each other.

[0061] Optionally, at 810, the radar manager communicates the determination. The radar manager may communicate the determination to the computing device via an application programming interface (API) or by being integrated into the OS of the computing device. The computing device may trigger a context event based on receiving the communication of the determination. Various examples of context events include opening a fitness app after scanning a pair of running shoes, starting a car after scanning a car key, locking a smart lock after scanning a door, opening a clock app after scanning a watch, and so on. By performingFigure 8 Method 800 shown, the radar manager effectively decodes the reflection of the RF signal into an object embedding of the scene trigger. Additional Examples

[0062] In the following sections, additional examples are provided.

[0063] Example 1: A method includes: transmitting a transmitted waveform signal; receiving a reflected waveform signal, the received reflected waveform signal including a version of the transmitted waveform signal reflected by an object; generating an object embedding associated with the object based on the reflected waveform signal; comparing the object embedding with a previous object embedding to provide a comparison result; and determining based on the comparison result that the object embedding and the previous object embedding are associated with the same object.

[0064] Example 2: The method according to Example 1, wherein at least one of the following is satisfied: the transmitted waveform signal is fixed; the transmitted waveform signal is a high-bandwidth signal up to 500 megahertz (MHz); the transmitted waveform signal is time-constrained; or the transmitted waveform signal includes radio waves having a frequency from 3.1 gigahertz (GHz) to 10.5 GHz.

[0065] Example 3: The method according to Example 1 or 2, wherein the object is at least one of the following: a personal object; a commercial object; or a body part.

[0066] Example 4: The method according to any of the preceding examples, further comprising: communicating the determination that the object embedding and the previous object embedding are associated with the same object; receiving the determination by a computing device; and triggering a predetermined action by the computing device based on the determination.

[0067] Example 5: The method according to Example 4, wherein: the scene event includes changing the authorization for a high-privilege resource associated with the computing device; the high-privilege resource requires two-factor authentication; the method provides one of the two factors of the two-factor authentication; and the method is executed in response to the computing device being in an unlocked state but the high-privilege resource being in a locked state.

[0068] Example 6: The method according to Example 5, wherein the other of the two factors of the two-factor authentication is simultaneous biometric authentication performed by the computing device.

[0069] Example 7: The method according to any of the preceding examples, wherein: the object embedding is based on an embedding model generated by offline training of an embedding neural network; and at least one of the following is satisfied: the object embedding is a rotation-invariant numerical representation of the reflected waveform signal; or the object embedding is a physical transformation of the data associated with the reflected waveform signal.

[0070] Example 8: The method according to any one of the preceding examples, wherein: the object embedding is a numerical representation of the features of the object; the previous object embedding is a previous numerical representation of the features of the previous object; and comparing the object embedding with the previous object embedding compares the numerical representation with the previous numerical representation.

[0071] Example 9: The method according to any one of the preceding examples, wherein: the object has a surface and a subsurface; and at least one of the following is satisfied: the reflected waveform signal includes a version of the transmitted waveform signal reflected by the surface of the object; or the reflected waveform signal includes a version of the transmitted waveform signal reflected by the subsurface of the object.

[0072] Example 10: The method according to any one of the preceding examples, wherein generating the object embedding further comprises: sampling a portion of the reflected waveform signal; correlating the portion of the reflected waveform signal with the transmitted waveform signal; generating a channel impulse response associated with the portion of the reflected waveform signal based on the correlation; and storing the channel impulse response as the object embedding.

[0073] Example 11: The method according to Example 10, wherein generating the object embedding further comprises: converting the channel impulse response into an embedding vector using an embedding network; and storing the embedding vector as the object embedding.

[0074] Example 12: The method according to Example 11, further comprising: providing the reflected waveform signals of various objects as input data to the embedding network; determining various object embeddings in the embedding space as output data by the embedding network; determining a cost function associated with the various object embeddings by the embedding network based on the various object embeddings; and grouping similar objects in the embedding space by the embedding network based on the cost function.

[0075] Example 13: The method according to any one of the preceding examples, wherein: the object includes a first object and a second object; and at least one of the following is satisfied: the reflected waveform signal includes a version of the transmitted waveform signal reflected by the first object; or the reflected waveform signal includes a version of the transmitted waveform signal reflected by the second object.

[0076] Example 14: The method according to any one of the preceding examples, wherein: the object is at least partially occluded by another object; the transmitted waveform signal penetrates the other object; and the reflected waveform signal penetrates the other object.

[0077] Example 15: A computing device includes: a radar system, the radar system including: an antenna array; and a transceiver, the transceiver including: at least one transmit channel respectively coupled to antenna elements of the antenna array; and at least one receive channel respectively coupled to antenna elements of the antenna array; at least one processor; and a computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to implement a radar manager using the antenna array and the transceiver by performing the method according to any one of Examples 1 to 14.

[0078] Example 16: A computer-readable medium includes instructions that, when executed by at least one processor, cause the at least one processor to perform the method according to any one of Examples 1 to 14. Conclusion

[0079] Unless the context otherwise requires, the use of the word "or" herein may be regarded as an "inclusive or" or the use of a term that permits the inclusion or application of one or more of the items joined by the word "or" (e.g., the phrase "A or B" may be interpreted as permitting only "A", only "B", or both "A" and "B"). Further, as used herein, a phrase referring to "at least one" in a list of items refers to any combination of those items, including a single member. For example, "at least one of a, b, or c" can cover a, b, c, a - b, a - c, b - c, and a - b - c, as well as any combination with multiples of the same element (e.g., a - a, a - a - a, a - a - b, a - a - c, a - b - b, a - c - c, b - b, b - b - b, b - b - c, c - c, and c - c - c, or any other ordering of a, b, and c). Additionally, the items represented in the figures and the terms discussed herein may indicate one or more items or terms, and thus the items and terms in the written description may be referred to interchangeably in their singular or plural forms.

[0080] Although implementations of systems, techniques, and devices for embedding an object that enables the reflection decoding of RF signals into a context trigger have been described in language specific to certain features and / or methods, the subject matter of the appended claims need not be limited to the specific features or methods described. Instead, the specific features and methods are disclosed as example implementations of embedding an object that enables the reflection decoding of RF signals into a context trigger.

Claims

1. A method, comprising: Transmit a transmitted waveform signal; Receive a reflected waveform signal, the received reflected waveform signal including a version of the transmitted waveform signal reflected by an object; Generate an object embedding associated with the object based on the reflected waveform signal; Compare the object embedding with a previous object embedding to provide a comparison result; And Determine that the object embedding and the previous object embedding are associated with the same object based on the comparison result.

2. The system according to claim 1, wherein at least one of the following is satisfied: The transmitted waveform signal is fixed; The transmitted waveform signal is a high - bandwidth signal up to 500 megahertz (MHz); The transmitted waveform signal is time - constrained; or The transmitted waveform signal includes radio waves with frequencies ranging from 3.1 gigahertz (GHz) to 10.5 GHz.

3. The method according to claim 1 or 2, further comprising: Identify the object based on the comparison result; and Wherein the object is at least one of the following: Personal object; Commercial object; Or Body part.

4. The method according to any one of the preceding claims, further comprising: Convey the determination that the object embedding and the previous object embedding are associated with the same object; Receive the determination by a computing device; And Trigger a predetermined action by the computing device based on the determination.

5. The method according to claim 4, wherein: The scenario event includes changing the authorization of a high-privilege resource associated with the computing device; The financial account or other high-privilege resource requires two-factor authentication; The method provides one of the two factors of the two-factor authentication; and The method is executed in response to the computing device being in an unlocked state but the high-privilege resource being in a locked state.

6. The method according to claim 5, wherein the other of the two factors of the two - factor authentication is simultaneous biometric authentication performed by the computing device.

7. The method according to any one of the preceding claims, wherein: The object embedding is based on an embedding model generated by offline training of an embedding neural network; and satisfies at least one of the following: The object embedding is a rotation-invariant numerical representation of the reflected waveform signal; or The object embedding is a physical transformation of data associated with the reflected waveform signal.

8. The method according to any one of the preceding claims, wherein: The object embedding is a numerical representation of the features of the object; The previous object embedding is a previous numerical representation of the features of a previous object; and Comparing the object embedding with the previous object embedding compares the numerical representation with the previous numerical representation.

9. The method according to any one of the preceding claims, wherein: The object has a surface and a subsurface; and satisfies at least one of the following: The reflected waveform signal includes a version of the transmitted waveform signal reflected by the surface of the object; or The reflected waveform signal includes a version of the transmitted waveform signal reflected by the subsurface of the object.

10. The method according to any one of the preceding claims, wherein generating the object embedding further comprises: Sample a portion of the reflected waveform signal; Correlate the portion of the reflected waveform signal with the transmitted waveform signal; Generate a channel impulse response associated with the portion of the reflected waveform signal based on the correlation; And Store the channel impulse response as the object embedding.

11. The method according to claim 10, wherein generating the object embedding further comprises: Convert the channel impulse response into an embedding vector using an embedding network; And Store the embedding vector as the object embedding.

12. The method according to claim 11, further comprising: Provide the reflected waveform signals of various objects as input data to the embedding network; Determine various object embeddings in an embedding space as output data by the embedding network; Determine a cost function associated with the various object embeddings by the embedding network based on the various object embeddings; And Group similar objects in the embedding space by the embedding network based on the cost function.

13. The method according to any one of the preceding claims, wherein: The object includes a first object and a second object; and satisfies at least one of the following: The reflected waveform signal includes a version of the transmitted waveform signal reflected by the first object; or The reflected waveform signal includes a version of the transmitted waveform signal reflected by the second object.

14. The method according to any one of the preceding claims, wherein: The object is at least partially occluded by another object; The transmitted waveform signal penetrates the other object; and The reflected waveform signal penetrates the other object.

15. A computing device, comprising: A radar system, the radar system comprising: An antenna array; and A transceiver, the transceiver comprising: At least one transmit channel, the at least one transmit channel being respectively coupled to antenna elements of the antenna array; and At least one receive channel, the at least one receive channel being respectively coupled to antenna elements of the antenna array; At least one processor; and A computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to implement a radar manager utilizing the antenna array and the transceiver by performing the method according to any one of claims 1 to 14.

16. A computer-readable medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 14.

Images