Radar Application Programming Interface

Abstract

A technology and device for implementing a radar application programming interface (210) are described. The radar application programming interface (210) provides a standardized language for external entities, such as applications (206) of a smart device (104), to communicate with the radar system (102) without needing to understand the design and operational complexity used by the radar system (102). Through the radar application programming interface (210), third parties can easily interact with the radar system (102) and customize the use of the radar system (102) for various applications (206). In this way, the radar application programming interface (210) can provide additional features that enhance the user experience by enabling other entities to extend the utilization of the radar system (102).

Description

Background Technology

[0001] Radar is a useful device capable of detecting objects. Compared to other types of sensors, such as cameras, radar can provide enhanced performance in various environmental conditions, such as low light, fog, or when objects are moving or overlapping. Radar can also detect objects through one or more occlusions, such as wallets or pockets.

[0002] Radar can be a complex technology that considers specific trade-offs to achieve target performance. Examples of trade-off considerations include detection range; range, Doppler, and angular resolution; range and Doppler ambiguity; photosensitivity; false alarm rate; response time; size; power consumption; and cost. Due to complex design and operational considerations, radar is often customized for specific applications. For example, some radars provide navigation assistance, others can map the environment, and still others can assist users in interacting with electronic devices.

[0003] While radar offers many advantages, it may be viewed as a black box by external organizations that may not recognize the complexity required to achieve its target performance. Consequently, it can be difficult for electronic applications to leverage the capabilities provided by radar. As a result, radar may be limited to supporting a few specific applications.

[0004] Technologies and devices for implementing a radar application programming interface (API) are described. The radar application programming interface provides a standardized language to external entities, such as applications on smart devices, enabling them to communicate with the radar system without understanding the design and operational complexities involved. Through the radar application programming interface, third parties can easily interact with the radar system and customize its usage for various applications. In this way, the radar application programming interface can provide additional features that enhance the user experience by allowing other entities to extend the utilization of the radar system.

[0005] The embodiment described below includes a method performed by a radar system of a smart device. The method includes the step of accepting a first request from an application of the smart device through a first layer of a multilayer of the radar application programming interface of the smart device. The first request includes a pattern recognition sequence specifying a sequence of states associated with a gesture. The multilayer of the radar application programming interface is associated with different operational levels of the radar system. The method also includes the step of accepting a second request from an application of the smart device through a second layer of a multilayer of the radar application programming interface. The second request includes a detection range of the radar system for detecting the gesture. The method further includes the step of modifying the hardware configuration of the radar system based on the detection range. The method further includes the step of transmitting and receiving a radar signal using the hardware configuration. At least a portion of the radar signal is reflected by an object performing the gesture. The method also includes the step of recognizing the gesture based on the pattern recognition sequence and the received radar signal. In response to the recognition, the method includes the step of transmitting a response indicating the occurrence of the gesture to the application through the radar application programming interface.

[0006] The embodiment described below also includes a device comprising a radar system configured to perform any of the described methods.

[0007] The embodiment described below includes a computer-readable storage medium comprising computer-executable instructions that cause a radar system to perform any one of the described methods in response to execution by a processor.

[0008] The embodiments described below also include systems equipped with means for providing a radar application programming interface. Brief explanation of the drawing

[0009] Devices and technologies implementing a radar application programming interface are described with reference to the following drawings. The same numbers are used throughout the drawings to refer to similar functions and components. Figure 1 illustrates an exemplary environment in which a smart device-based radar system can be implemented. FIG. 2a illustrates an exemplary implementation of a radar system as part of a smart device. FIG. 2b illustrates an exemplary system medium of a radar system. FIG. 3a illustrates an exemplary radar application programming interface. FIG. 3b illustrates an exemplary relationship between the multiple layers of a radar application programming interface and the operational levels of a radar system. FIG. 4 illustrates an exemplary request provided by an application to a radar application programming interface for gesture recognition. FIG. 5a illustrates an exemplary pattern recognition sequence associated with a pump gesture. FIG. 5b illustrates exemplary object behavior related to pump operation. FIG. 6a illustrates an exemplary pattern recognition sequence associated with a swipe gesture. FIG. 6b illustrates exemplary object behavior related to a swipe gesture. FIG. 7a illustrates an exemplary pattern recognition sequence related to a rich gesture. FIG. 7b illustrates exemplary object behavior related to a rich gesture. FIG. 8 illustrates an exemplary antenna array and an exemplary transceiver of a radar system. FIG. 9 illustrates exemplary operation of a hardware abstraction module, an object tracker, a detection module, a radar application programming engine, and a radar application programming interface. FIG. 10 illustrates an exemplary method implemented by a detection module. FIG. 11 illustrates an exemplary method performed by a radar system using a radar application programming interface. FIG. 12 illustrates an exemplary computing system in which a technology capable of implementing or using a radar application programming interface can be implemented. Specific details for implementing the invention

[0010] Radar is a useful device capable of detecting objects. Compared to other types of sensors, such as cameras, radar can provide enhanced performance in various environmental conditions, such as low light, fog, or when objects are moving or overlapping. Radar can also detect objects through one or more occlusions, such as wallets or pockets.

[0011] Radar can be a complex technology that considers specific trade-offs to achieve target performance. Examples of trade-off considerations include detection range; range, Doppler, and angular resolution; range and Doppler ambiguity; photosensitivity; false alarm rate; response time; size; power consumption; and cost. Due to complex design and operational considerations, radar is often customized for specific applications. For example, some radars provide navigation assistance, others can map the environment, and still others can assist users in interacting with electronic devices.

[0012] While radar offers many advantages, it may be viewed as a black box by external organizations that may not recognize the complexity required to achieve its target performance. Consequently, it can be difficult for electronic applications to leverage the capabilities provided by radar. As a result, radar may be limited to supporting a few specific applications.

[0013] To address these challenges, a technology for implementing a radar application programming interface (API) is described. The radar application programming interface provides a standardized language to external entities, such as applications on smart devices, enabling them to communicate with the radar system without understanding the design and operational complexities involved. Through the radar application programming interface, third parties can easily interact with the radar system and customize its usage for various applications. In this way, the radar application programming interface can provide additional features that enhance the user experience by allowing other entities to extend the utilization of the radar system.

[0014] Operating environment

[0015] FIG. 1 illustrates exemplary environments (100-1 to 100-6) in which a technology using a radar system and a device including the same may be implemented. In the illustrated environments (100-1 to 100-6), a smart device (104) includes a radar system (102) capable of interfacing with various different applications of the smart device (104) through a radar application programming interface (Fig. 2a). The smart device (104) is illustrated as a smartphone in environments 100-1 to 100-5 and a smart vehicle in environment 100-6.

[0016] In environments 100-1 through 100-4, the user performs various types of gestures detected by the radar system (102). In some cases, the user performs the action using an appendage or a body part. Alternatively, the user may perform the gesture using a stylus, an object held in the hand, a ring, or any type of material capable of reflecting radar signals. Through the use of a radar application programming interface, the operation of the radar system (102) can be dynamically customized based on the type of gesture requested for the radar system (102) to recognize.

[0017] In Environment 100-1, the user performs a scrolling gesture by moving their hand over the smart device (104) along a horizontal direction (e.g., from the left side of the smart device (104) to the right side of the smart device (104). In Environment 100-2, the user performs a reaching gesture, which reduces the distance between the smart device (104) and the user's hand. Users in Environment 100-3 make hand gestures to play a game on the smart device (104). In one example, the user performs a pushing gesture by moving their hand over the smart device (104) along a vertical dimension (e.g., from the bottom side of the smart device (104) to the top side of the smart device (104). Using information transmitted by the radar application programming interface, the radar system (102) can recognize the gestures performed by the user. In environment (100-4), the smart device (104) is stored in a wallet, and the radar system (102) provides obscured gesture recognition by detecting gestures that are obscured by the wallet.

[0018] The radar system (102) may also recognize other types of gestures or motions not illustrated in FIG. 1. Exemplary types of gestures include a handle-turning gesture, in which a user bends their fingers to grasp a virtual door handle and rotates their fingers and hand clockwise or counterclockwise to mimic the action of turning the virtual door handle. Another exemplary type of gesture includes a spindle-twisting gesture, performed by a user rubbing their thumb together with at least one other finger. Gestures may be two-dimensional, such as gestures used in touch-detecting displays (e.g., two fingers together, two fingers apart, or tap). Gestures may also be three-dimensional, such as many sign language gestures, such as those of American Sign Language (ASL) and other sign languages ​​around the world. Upon detecting each of these gestures, the smart device (104) may perform actions such as displaying new content, moving the cursor, activating one or more sensors, opening applications, etc. In this way, the radar system (102) provides touchless control of the smart device (104).

[0019] In environment 100-5, the radar system (102) generates a three-dimensional map of the surrounding environment for context awareness. The radar system (102) also detects and tracks multiple users, allowing two users to interact with the smart device (104). The radar system (102) can also perform vital-sign detection. In environment 100-6, the radar system (102) monitors the vital signs of the user driving the vehicle. Examples of vital-signs include heart rate and respiratory rate. For example, if the radar system (102) determines that the driver is asleep, the radar system (102) can cause the smart device (104) to warn the user. Alternatively, if the radar system (102) detects a life-threatening emergency such as a heart attack, the radar system (102) can cause the smart device (104) to warn a medical professional or emergency services. In some implementations, the radar system (102) of the environment (100-6) may support collision avoidance for autonomous driving and / or navigation support. Generally, the radar system may support various applications including touchless gesture control, health monitoring (e.g., sleep tracking or biosignal monitoring), fitness tracking, proximity detection, spatial mapping, and human activity recognition (e.g., fall detection, attention, sitting or standing).

[0020] Some implementations of the radar system (102) are particularly advantageous when applied in the context of a smart device (104) where the problem converges. This may include limitations on the spacing and layout of the radar system (102) and the need for low power. Exemplary overall lateral dimensions of the smart device (104) may be, for example, approximately 8 cm x approximately 15 cm. An exemplary footprint of the radar system (102) may be much more limited, such as approximately 4 mm x 6 mm including the antenna. Exemplary power consumption of the radar system (102) may be approximately several milliwatts to tens of milliwatts (e.g., between approximately 2 milliwatts and 20 milliwatts). These requirements for limited installation space and power consumption for the radar system (102) allow the smart device (104) to include other desirable functions in a space-constrained package (e.g., camera sensor, fingerprint sensor, display, etc.). The smart device (104) and the radar system (102) are further described in relation to FIG. 2a.

[0021] FIG. 2a illustrates a radar system (102) as part of a smart device (104). The smart device (104) is illustrated with various non-limiting exemplary devices including a desktop computer (104-1), a tablet (104-2), a laptop (1043), a television (104-4), a computing watch (104-5), computing glasses (104-6), a gaming system (104-7), a microwave oven (104-8), and a vehicle (104-9). Other devices such as home service devices, smart speakers, smart thermostats, security cameras, baby monitors, Wi-Fi™ routers, drones, trackpads, drawing pads, netbooks, e-readers, home automation control systems, wall displays, and other home appliances may be used. Note that the smart device (104) may be wearable, may be non-wearable but mobile, or may be relatively immobile (e.g., desktops and home appliances). The radar system (102) can be used as a standalone radar system, or can be used with or embedded in various smart devices (104) or peripherals, such as a control panel for controlling home appliances and systems, a car for controlling internal functions (e.g., volume, cruise control, even driving of a vehicle), or an attachment to a laptop computer for controlling computing applications on a laptop.

[0022] A smart device (104) comprises one or more computer processors (202) and at least one computer-readable medium (204) comprising a memory medium and a storage medium. An application and / or operating system (not shown) implemented as computer-readable instructions on the computer-readable medium (204) may be executed by the computer processor (202) to provide some of the functions described herein. The computer-readable medium (204) also includes an application (206) that uses radar data generated by the radar system (102) to perform functions such as presence detection, gesture-based touch-free control, collision avoidance for autonomous driving, health monitoring, fitness tracking, spatial mapping, human activity recognition, etc.

[0023] The smart device (104) may also include a network interface (208) for communicating data through a wired, wireless, or optical network. For example, the network interface (208) may communicate data through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), a Personal Area Network (PAN), a Wired Area Network (WAN), an intranet, the Internet, a peer-to-peer network, a point-to-point network (P2P), a mesh network, etc. The smart device (104) may also include a display (not shown).

[0024] The smart device (104) includes a radar application programming interface (210) (e.g., a radar API (210)). The radar application programming interface (210) provides an interface between the radar system (102) and an external entity, such as an application (206) or another radar system. Generally, the radar application programming interface (210) provides a standardized language that the external entity (e.g., the application (206)) and the radar system (102) can use to communicate with each other. In some embodiments, the radar application programming interface (210) converts various requests provided by one or more external entities (e.g., one or more applications (206)) into an operational configuration that the radar system (102) can execute.

[0025] Through the radar application programming interface (210), the application (206) can control the operation of the radar system (102) and / or receive information from the radar system (102) without needing to understand how the radar system (102) operates. The radar application programming interface (210) defines the interaction between the application (206) and the radar system (102), including the types of available requests and responses, the method of sending requests or responses, data formats, and other rules. Additionally, the radar application programming interface (210) can ensure compliance with other regulations and guidelines, including those provided by agencies such as the Federal Communications Commission (FCC). Sometimes this information is captured in standard documents to allow a third party to program the application (206) accordingly. For example, the radar application programming interface (210) can define a set of common functional blocks across various different types of radar systems (102) and allow non-radar engineers (e.g., developers, hobbyists, researchers) to apply these functional blocks in unique ways to achieve different goals. The radar application programming interface (210) can also enable various applications or external entities to communicate with various types of radar systems (102) (e.g., radar systems with various hardware designs, operational configurations, and / or performance capabilities).

[0026] The radar application programming interface (210) may be implemented in software, programmable hardware, or some combination thereof. The radar application programming interface (210) may be inside or outside the radar system (102). In some implementations, the radar application programming interface (210) includes instructions that are stored within the CRM (204) and executable by the computer processor (202). In other implementations, instructions for implementing the radar application programming interface (210) are stored within the CRM of the radar system (102) (e.g., system medium (220)) and executable by the processor (e.g., system processor (218)).

[0027] The radar system (102) may implement one or more different types of radar, such as frequency modulated continuous wave (FMCW) radar, pulse Doppler radar, continuous wave (CW) radar, phase modulated spread spectrum radar, impulse radar, radar using a "Zadoff-Chu" sequence or a constant-amplitude zero-autocorrelation (CASC) sequence, or multiple input multiple output (MIMO) radar. The radar system (102) includes a communication interface (212) for transmitting radar data to a remote device, but this does not need to be used when the radar system (102) is integrated within a smart device (104). Generally, the radar data provided by the communication interface (212) is in a format available to the radar application programming interface (210) and the application (206).

[0028] The radar system (102) also includes at least one antenna array (214) and at least one transceiver (216) for transmitting and receiving radar signals. The antenna array (214) includes at least one transmitting antenna element and at least one receiving antenna element. In some situations, the antenna array (214) includes multiple transmitting antenna elements and / or multiple receiving antenna elements. By using multiple transmitting antenna elements and multiple receiving antenna elements, the radar system (102) can implement a multiple input multiple output radar capable of transmitting multiple individual waveforms (e.g., different waveforms per transmitting antenna element) at a given time. The antenna elements may be circularly polarized, horizontally polarized, vertically polarized, or a combination thereof.

[0029] Multiple receiving antenna elements of the antenna array (214) may be arranged in a one-dimensional shape (e.g., a line) or a two-dimensional shape (e.g., a rectangular array, a triangular array, or an "L" shaped array) for an implementation including three or more receiving antenna elements. A one-dimensional shape allows the radar system (102) to measure one angular dimension (e.g., an azimuth or elevation), whereas a two-dimensional shape allows the radar system (102) to measure two angular dimensions (e.g., to determine both the azimuth and elevation angles of an object). The element spacing associated with the receiving antenna elements may be less than, greater than, or equal to half the center wavelength of the radar signal.

[0030] The transceiver (216) includes circuitry and logic for transmitting and receiving radar signals through the antenna array (214). Components of the transceiver (216) may include amplifiers, phase shifters, mixers, switches, analog-to-digital converters, or filters for modulating radar signals. The transceiver (216) also includes logic for performing phase / quadruple (I / Q) operations, such as modulation or demodulation. Various modulations may be used, including linear frequency modulation, triangular frequency modulation, step frequency modulation, or phase modulation. Alternatively, the transceiver (216) may generate a radar signal having a relatively constant frequency or a single tone. The transceiver (216) may be configured to support continuous wave or pulse radar operation.

[0031] The frequency spectrum (e.g., frequency range) used by the transceiver (216) to generate a radar signal may cover frequencies of 1–400 GHz, 4–100 GHz, 1–24 GHz, 2–4 GHz, 30–80 GHz, 57–64 GHz, or about 2.4 GHz. In some cases, the frequency spectrum may be divided into several sub-spectrases with similar or different bandwidths. The bandwidth may be 500 MHz, 1 GHz, 2 GHz, etc. In some cases, to implement a UWB (ultra-wideband) radar, the bandwidth may be more than about 20% of the center frequency.

[0032] Other frequency subspectra may include frequencies between approximately 57 and 59 GHz, 59 and 61 GHz, or 61 and 63 GHz, for example. The exemplary frequency subspectra described above is continuous, but other frequency subspectra may not be continuous. To achieve consistency, multiple frequency subspectra (whether continuous or not) having the same bandwidth may be used by the transceiver (216) to generate multiple radar signals transmitted simultaneously or temporally separated. In some situations, multiple continuous frequency subspectra may be used to transmit a single radar signal so that the radar signal has a wide bandwidth.

[0033] The radar system (102) also includes one or more system processors (218) and at least one system medium (220) (e.g., one or more computer-readable storage media). The system medium (220) is further described in relation to FIG. 2b.

[0034] FIG. 2b illustrates an exemplary system medium (220) of a radar system (102). The system medium (220) includes at least one digital signal processing (DSP) module (222), at least one radar-data-processing module (224), and at least one application-programming-interface (API) engine (226). The digital-signal-processing module (222), the radar-data-processing module (224), and / or the application-programming-interface engine (226) may be implemented using hardware, software, firmware, or a combination thereof. In this example, a system processor (218) implements the digital-signal-processing module (222), the data-processing module (224), and the application-programming-interface engine (226). In an alternative implementation (not shown), at least one of the digital-signal-processing module (222), the radar-data-processing module (224), or the application-programming-interface engine (226) is implemented by a computer-readable medium (204) and a computer processor (202). For example, the digital signal processing module (222) may be contained within the system medium (220) and executed by the system processor (218), whereas the radar data processing module (224) and the application programming interface engine (226) may be contained within the computer-readable medium (204) and executed by the computer processor (202). In this case, the radar system (102) may provide radar data from the smart device (104) through the communication interface (212) so that the computer processor (202) can process the radar data.

[0035] The digital-signal-processing module (222) provides real-time signal processing by performing low-level processing on analog and / or digital samples of the received radar signal. Exemplary types of signal processing may include non-coherent integration, clutter removal, detection thresholding, noise removal, Doppler filtering, interferometry, and / or digital beamforming. Some functions of the digital-signal-processing module (222) may include repetitive mathematical operations such as addition, subtraction, and multiplication. These repetitive mathematical operations may be performed to implement Fourier transforms (e.g., Fast Fourier transform). In an exemplary implementation, the digital-signal-processing module (222) includes a hardware-abstraction module (228). Generally, the digital-signal-processing module (222) processes raw data provided by the transceiver (216) and generates data preprocessed into a format usable by the radar-data-processing module (224).

[0036] The radar-data-processing (RDP) module (224) provides real-time data processing by performing intermediate-level and / or upper-level processing on preprocessed data received from the digital-signal-processing module (222). The radar-data-processing module (224) can perform functions such as object tracking, clutter tracking, gesture recognition, presence detection, biometric recognition, navigation assistance, and / or health monitoring. These functions can be implemented using empirical algorithms and / or machine learning algorithms. In an exemplary implementation, the radar-data-processing module (224) includes an object tracker (230) and a detection module (232).

[0037] The digital signal processing module (222) and the radar data processing module (224) may be designed for compatibility across various types of radar systems (102), including radar systems (102) having various types of hardware, various hardware configurations and / or various operational configurations (e.g., FMCW radar, pulse Doppler radar, or impulse radar). That is, the digital signal processing module (222) and / or the radar data processing module (224) may be at least somewhat agnostic with respect to the hardware implementation and operation of the radar system (102).

[0038] Data generated by the digital-signal-processing module (222) and / or the radar-data-processing module (224) may be self-describing. In some cases, the structure of the data itself describes the format and meaning of the data. In other cases, the digital-signal-processing module (222) and / or the radar-data-processing module (224) may provide information regarding the format and meaning of the data through a query via the radar application programming interface (210). Exemplary data formats include packaged or compressed data formats (e.g., 13-bit integers of a specific quantity), unpacked data formats (e.g., 32-bit floating-point arrays), or dimensions of multidimensional data that are row-interleaved or row-by-row.

[0039] Each module (or function) implemented within the digital signal processing module (222) and / or radar data processing module (224) may have defined inputs and / or outputs. Some of these inputs and / or outputs may be standardized using the radar application programming interface (210) so that an external entity (e.g., application 206) can easily customize the behavior of the digital signal processing module (222) and / or radar data processing module (224) for various use cases and various radar types. In particular, the external entity may utilize the radar application programming interface (210) to specify an implementation chain (or implementation sequence) that processes samples of received radar signals and outputs radar data for the application (206). An implementation chain may represent a set of digital signal processing modules (222) and / or radar data processing modules (224) that are executed in a specific order. By specifying an implementation chain, the external entity can customize which modules are enabled or disabled and the order in which these modules operate.

[0040] Inputs and / or outputs may be associated with one or more standard rendering planes. In some cases, a module's input and output may refer to the same standard rendering plane. In this case, the module operates on the input data without changing its format. Therefore, the output data is in the same format as the input data (or is associated with the same standard rendering plane). In other cases, a module's input and output may refer to different standard rendering planes. Modules associated with a specific type may support specific inputs and outputs. In this case, the module operates on the input data and generates output data by converting the format of the input data. Therefore, the output data is in a different format than the input data (or is associated with a different standard rendering plane).

[0041] Based on knowledge of the standard rendering planes associated with the module, the engineer can easily replace or alter other modules to operate the radar system (102) in a manner that achieves the desired computational cost, memory cost, configurability, robustness, or performance. Examples of standard rendering planes may include raw digital samples of radar signals received for one or more channels, information representing a range-Doppler map, information representing a range-Doppler-azimuth-elevation map, information representing an interference pattern, object data (e.g., location information, motion information, and / or physical information about one or more objects), gesture data, biometric data, health data, collision avoidance data, etc. Some rendering planes may include information representing the uncertainty of the module. This uncertainty information may be used to recognize errors or poor detection conditions in other modules.

[0042] Some modules may support multiple standard rendering planes (e.g., data support with alternative representations). For example, the object tracker (230) may output a grid representation that can be considered a "dense" representation. In the grid representation, each grid element (or cell) identifies whether an object is present at that location. Thus, the grid representation may contain additional information about locations where no object is present. Alternatively, the object tracker (230) may output a list of objects that can be considered a "sparse" representation. Each object listed in the list of objects may contain information about its grid location. Since the list of objects may not contain information about grid elements that do not contain objects, the list of objects may have a smaller memory size than the grid representation. In this case, an external entity may utilize the radar application programming interface (210) to specify an appropriate standard rendering plane depending on whether the downstream module can utilize the additional information provided by the grid representation.

[0043] The application programming interface engine (226) may serve as an interface between the radar application programming interface (210) and components of the radar system (102) (e.g., antenna array (214), transceiver (216) and / or system processor (218)). In some cases, the application programming interface engine (226) modifies the operation of the radar system (102) in response to a request received by the radar application programming interface (210). For example, the application programming interface engine (226) may activate a specific number of antenna elements within the antenna array (214) or cause the transceiver (216) to generate a specific type of radar signal specified by the radar application programming interface (210). As another example, the application programming interface engine (226) may cause the system processor (218) to execute an implementation chain specified by the radar application programming interface (210) and / or cause a module within the implementation chain to operate with one or more parameters provided by the radar application programming interface (210).

[0044] The application programming interface engine (226) may also provide information to the radar application programming interface (210), and the radar application programming interface (210) may transmit this to the application (206). Optionally, the application programming interface engine (226) may reformat the data received or provided from the radar application programming interface (210). The operation of the hardware abstraction module (228), object tracker (230), and detection module (232) is further described below.

[0045] The hardware abstraction module (228) converts raw data provided by the transceiver (216) into hardware-independent data. The hardware-independent data can be processed by the object tracker (230) and / or the detection module (232). In particular, the hardware abstraction module (228) adapts complex data of various types of radar signals to the expected input. This enables the object tracker (230) or the detection module (232) to process various types of radar signals received by the radar system (102), including signals utilizing different modulation schemes for frequency-modulated continuous wave radar, phase-modulated spread spectrum radar, or impulse radar. The hardware abstraction module (228) can also normalize complex data of radar signals having different center frequencies, bandwidths, transmission power levels, or pulse widths.

[0046] Additionally, the hardware abstraction module (228) accommodates complex data generated using various hardware architectures. Different hardware architectures may include different antenna arrays (214) located on different surfaces of the smart device (104) or different sets of antenna elements within the antenna array (214). By using the hardware abstraction module (228), the downstream module can process complex data generated by different sets of antenna elements with different gains, different sets of antenna elements with varying quantities, or different sets of antenna elements with different antenna element spacings. Furthermore, the hardware abstraction module (228) enables these downstream modules to operate in the radar system (102) using various restrictions that affect available radar modulation schemes, transmission parameters, or types of hardware architectures.

[0047] The object tracker (230) identifies objects within the external environment and aggregates the history of each object's behavior. In particular, the object tracker (230) can collect information about the object's location (e.g., distance and / or angle), motion (e.g., range rate and / or velocity), physical characteristics (e.g., size, radar cross-section and / or material composition), or some combination thereof. The object tracker (230) can be implemented using various different tracking algorithms, including algorithms related to alpha-beta trackers, Kalman filters, or multiple hypothesis trackers (MHT).

[0048] The detection module (232) can analyze data provided by the object tracker (230) for one or more use cases, such as gesture recognition, presence detection, collision avoidance, and state monitoring. For gesture recognition, the detection module (232) can recognize a gesture and distinguish between motion and non-gesture associated with the gesture. The detection module (232) is further described in relation to FIG. 10. In some implementations, the detection module (232) is implemented as a state machine.

[0049] Although not explicitly illustrated, the system medium (220) or computer-readable medium (204) may also include a sensor fusion module, a radar communication module, a data collection module, a data visualization tool, and / or a radar simulator. The sensor fusion module may combine data from the radar system (102) with data provided by an external sensor, such as another radar system or another type of sensor (e.g., a camera).

[0050] A radar communication module enables the radar system (102) to communicate with other entities outside the smart device (104), such as the radar system of another smart device (104). Through direct communication, these radar systems can be configured to support bistatic radar sensing, utilize techniques such as triangulation, acquire objects of interest more quickly, and / or maintain tracking of objects of interest that are temporarily obscured but detected by other radar systems. Through the radar communication module, the radar systems can share information and improve performance in various environments and situations.

[0051] The data collection module can record data available to the system processor (218) for system development and integration. Exemplary data may include data provided by the transceiver (216) or data generated by the digital-signal-processing module (222), the radar-data-processing module (224), and / or the application programming interface engine (226). In some cases, the data collection module captures raw data provided by the transceiver (216) and does not perform additional functions to reduce computational costs.

[0052] A data visualization tool can provide a visual description of the data collected using a data collection module. In this way, the data visualization tool can make it easier for the developer of the application (206) to view and understand the data generated by the radar system (102). In an example implementation, the data visualization tool tags the data with labels describing the environment or activity contained in the data. In another exemplary implementation, the data visualization tool graphically depicts some of the data generated by the radar system (102), such as information representing a range-Doppler map or information representing the motion of an object. Other data visualization tools can depict operational aspects of the radar system (102), such as the field of view (FOV) used by the radar system (102), radar transmission signals generated by the radar system (102), or the antenna pattern of the radar system (102).

[0053] The radar simulator can inject virtual data into the digital-signal-processing module (222), the radar-data-processing module (224), and / or the application programming interface engine (226). The virtual data may include data provided from an external source or simulation data generated by the radar simulator. In some implementations, the radar simulator can generate simulated data by rendering a virtual environment with objects and / or clutter and analyzing the interaction of virtual radar waves within the virtual environment. The radar simulator can also inject noise to help algorithm developers and integration test engineers understand the impact that a noise (noise) source may have on the performance of the radar system (102). Using the radar simulator, developers of the application (206) can determine the potential for utilizing the radar system (102) for specific use cases. The radar application programming interface (210) is further described in relation to FIG. 3a.

[0054] FIG. 3a illustrates an exemplary radar application programming interface (210). Sometimes, the radar application programming interface (210) may include multiple application programming interfaces that enable an application (206) to communicate with various operational levels of the radar system (102). The multiple application programming interfaces may represent multiple layers or levels of the radar application programming interface (210). For example, the radar application programming interface (210) may include a use case application programming interface (302) (use case API (302)), a machine learning (ML) and algorithm application programming interface (304) (ML and algorithm API (application programming interface) 304), a digital signal processing (DSP) application programming interface (306) (DSP API 306), a hardware abstraction application programming interface (308) (hardware-abstraction API (308)), or a combination thereof.

[0055] The radar application programming interface (210) can be described in terms of multiple layers (310), each layer being associated with a specific operational level of the radar system (102), as further described in relation to FIG. 3b. Generally, the upper layers of the radar application programming interface (210) are designed so that engineers with less radar experience and knowledge can easily interact with higher operational levels of the radar system (102). The lower layers of the radar application programming interface (210) are designed so that engineers with more radar experience and knowledge can control lower operational levels of the radar system (102).

[0056] The use case application programming interface (302) represents the first layer (310-1) (or upper layer) of the radar application programming interface (210). The use case application programming interface (302) can enable a non-radar engineer to interface with the radar system (102) without knowing low-level details regarding the operation of the radar system (102) and / or the performance of the radar system (102) from a hardware or software perspective. The use case application programming interface (320) can standardize the interaction between the application (206) and the radar system (102) for various use cases, including gesture recognition, presence detection, biometric recognition, navigation assistance, and / or health monitoring.

[0057] The machine learning and algorithm application programming interface (304) represents the second layer (310-2) (or upper intermediate layer) of the radar application programming interface (210). Using the machine learning and algorithm application programming interface (304), an engineer with some radar knowledge can build and train machine learning models within the radar system (102). Additionally, the machine learning and algorithm application programming interface (304) allows these engineers to customize core algorithms that enable the radar system (102) to analyze upper-level radar data (e.g., preprocessed radar data).

[0058] The digital signal processing application programming interface (306) represents the third layer (310-3) (e.g., lower-middle layer) of the radar application programming interface (210). The digital signal processing application programming interface (306) can define a library of functions that the radar system (102) can execute to process raw data. These functions can perform spectrogram generation, Doppler calculations, digital beamforming, etc. Using the digital signal processing application programming interface (306), an engineer with some knowledge of radar can customize the digital signal processing performed by the radar system (102).

[0059] The hardware abstraction application programming interface (308) represents the fourth layer (310-4) (e.g., lower layer) of the radar application programming interface (308). The hardware abstraction application programming interface (308) can abstract the details of the hardware implementation of the radar system (102) and enable standardized output independent of the hardware of the radar system (102). Using the hardware abstraction application programming interface (308), an engineer with radar knowledge can customize the operation and configuration of the hardware within the radar system (102), including the antenna array (214) and / or transceiver (216).

[0060] Sometimes, the radar system (102) may operate according to one or more available configurations (312). In some cases, these configurations (312) are derived from the limitations of the radar system (102) and / or the limitations of the smart device (104). Some limitations are considered fixed limitations and are not likely to change over time. For example, fixed limitations (limitations) may be based on hardware limitations associated with the radar system (102) (e.g., associated with the antenna array (214), transceiver (216), and / or system processor (218). Exemplary fixed limitations include the available frequency band, bandwidth, transmission power level, antenna configuration, and / or duplex configurations of the radar system (102). Other limitations are considered dynamic limitations that may change over time. Exemplary dynamic limitations may include the amount of available power (e.g., the battery level of the smart device (104)), the amount of available memory (e.g., the size of the system medium (220) or computer-readable medium (204)) and / or the amount of processing capacity (e.g., the processing capacity of the system processor (218) or computer processor (202)).

[0061] Available configurations (312) represent operational configurations of a radar system (102) for generating a radar signal, transmitting a radar signal, receiving a reflected radar signal, and / or processing a reflected radar signal. In some cases, available configurations (312) specify adjustable characteristics of the radar signal, such as carrier frequency, bandwidth, radar waveform (e.g., modulation type), and / or transmission power level. Available configurations (312) may also include hardware configurations of the radar system (102), software configurations of the radar system (102), radar sensing performance metrics of the radar system (102), available resources of the radar system (102) and / or smart device (104), or some combination thereof. Available configurations (312) may include at least one basic configuration.

[0062] Exemplary hardware configurations include antenna configurations such as a single antenna for transmission and reception, a phase array for transmission and / or reception, or MIMO operation. Other exemplary hardware configurations include duplex configurations such as a half-duplex configuration for implementing pulse Doppler radar or a full-duplex configuration for implementing frequency-modulated continuous wave radar.

[0063] An exemplary software configuration may include a radar signal processing configuration. The radar signal processing configuration specifies signal processing techniques that the radar system (102) can use to determine explicit information about an object. Some radar signal processing configurations may be customized to be less complex and use less memory. For example, a first radar signal processing configuration performs a Fourier transform (e.g., Fast Fourier Transform (FFT)) and uses a detection thresholding algorithm. Using these techniques, the first radar signal processing configuration can detect an object and measure the distance to the object. Other radar signal processing configurations may be more complex and use more memory to reduce false alarms and increase accuracy. For example, a second radar signal processing configuration may include a clutter tracker for monitoring clutter, a tracking algorithm to increase the likelihood of object detection and improve measurement accuracy, and / or a digital beamformer for measuring one or more angles to an object.

[0064] In one embodiment, the radar application programming interface (210) enables an engineer to configure the radar system (102) according to one of the available configurations (312). In another embodiment, the radar application programming interface (210) enables an engineer to reference the available configurations (312) and create a new custom configuration based on the available configurations (312).

[0065] During operation, the application (206) sends a request (314) to the radar application programming interface (210). The request (314) may specify a specific operational configuration of the radar system (102) or request specific information from the radar system (102). Exemplary types of information may include a list of available configurations (312), performance capabilities of the radar system (102), specific types of data collected by the radar system (102), or error reports. The request (314) may be applied to a use case application programming interface (API) (302), a machine learning and algorithm application programming interface (304), a digital signal processing application programming interface (306), and / or a hardware abstraction application programming interface (308). Some requests (314) may reset or restart the radar system (102).

[0066] The radar application programming interface (210) can map a request (314) to a configuration (316) of the radar system (102) (e.g., one of the available configurations (312)). Sometimes, the radar application programming interface (210) may receive multiple requests (314) from the application (206) or various other applications. The radar system (102) can process these requests (314) together and determine an appropriate configuration (316) of the radar system (102) that satisfies the multiple requests (314). In this way, the radar application programming interface (210) enables the radar system (102) to efficiently provide radar sensing for various different applications (206) or use cases. The radar system (102) receives this configuration (316) and operates accordingly.

[0067] The radar system (102) transmits radar data (318) to the radar application programming interface (210). The radar data (318) may include data indicating whether a specific gesture has been detected or data collected by the radar system (102) (e.g., raw digital samples of the received signal or data representing a range Doppler map). In response to the reception of the radar data (318), the radar application programming interface (210) may format the radar data (318) and provide a response (320) to the application (206). In some cases, the response (320) may include the radar data (318). The response (320) may also include error reports related to false-positives (FP). Using this information, the application (206) may adjust the request (314) appropriately to reduce the FP. Various application programming interfaces may be associated with various operating levels of the radar system (102), as further described in relation to FIG. 3b.

[0068] FIG. 3b illustrates an exemplary relationship between multiple layers (310-1 to 310-4) of a radar application programming interface (210) and operation levels (322-1 to 322-4) of a radar system (102). In the illustrated configuration, the radar system (102) is associated with multiple levels (322) (or operation levels). Generally, the upper level (322) includes a module that operates on upper-level radar data (e.g., preprocessed radar data). During reception, this module may be executed towards the end of the reception process to generate radar data (318) requested by the application (206). The lower level (322) may include a module that operates on lower-level radar data (e.g., raw radar data). During reception, these modules may be executed at the beginning of the reception process (or at least prior to the module associated with the upper level (322)). The lower level (322) may also include the hardware configuration of the radar system (102).

[0069] As illustrated in FIG. 3b, the radar-data-processing module (224) is associated with the first level (322-1) (e.g., first operational level) and the second level (322-2) (e.g., second operational level) of the radar system (102). In particular, the detection module (232) is associated with the first level (322-1), and the object tracker (230) is associated with the second level (322-2). The digital-signal-processing module (222) is associated with the third level (322-3) (e.g., third operational level) of the radar system (102). The hardware (324) of the radar system (102), including the antenna array (214) and the transceiver (216), is associated with the fourth level (322-4).

[0070] Each layer (310) of the radar application programming interface (210) may be associated with one or more levels (322) of the radar system (102). In particular, each layer (310) of the radar application programming interface (210) may communicate directly or indirectly with the corresponding level (322) of the radar system (102) through the application programming interface engine (226). Through this communication, each layer (310) of the radar application programming interface (210) may manage operation parameters and / or modules associated with the corresponding level (322) of the radar system (102). The radar application programming interface (210) may also extract information from the corresponding level (322) of the radar system (102) and / or customize the inputs and / or outputs of the modules associated with the corresponding level (322).

[0071] In this example, the use case application programming interface (302) is associated with the detection module (232). Through this relationship, the use case application programming interface (302) can customize the operation of the radar system (102) for gesture recognition, presence detection, biometric recognition, collision avoidance, health monitoring, etc. In particular, the use case application programming interface (302) can specify various states of the detection module (232), as further described in relation to FIG. 10.

[0072] The machine learning and algorithm application programming interface (304) is associated with the object tracker (230). Through this relationship, the machine learning and algorithm application programming interface (304) can customize the behavior of the object tracker (230) and specify the output format of the object tracker (230).

[0073] The digital signal processing application programming interface (306) is associated with the digital signal processing module (222) (or hardware abstraction module (228)). Through this relationship, the digital signal processing application programming interface (306) can customize the digital signal processing operations of the radar system (102), which may include clutter removal, noise removal, detection threshold setting, Doppler filtering, interferometry, and / or digital beamforming. The digital signal processing application programming interface (306) can also specify functions such as integration time and update rate.

[0074] The hardware abstraction application programming interface (308) is associated with the hardware (324). Through this relationship, the hardware abstraction application programming interface (308) can customize the operation of the antenna array (214) and / or the transceiver (216). For example, the hardware abstraction application programming interface (308) can specify waveform parameters of the transmitted radar signal, such as transmission power, frequency, bandwidth, and modulation type (e.g., frequency or phase modulation). The hardware abstraction application programming interface (308) can also specify a radiation pattern (e.g., direction of the main lobe and / or beam width), pulse width, pulse repetition frequency (PRF) (e.g., inter-pulse period (IPP)), pulse quantity (amount of pulses), duty cycle, or polarization (e.g., horizontal polarization, vertical polarization, and / or circular polarization).

[0075] In some cases, the hardware abstraction application programming interface (308) may allow an external party to specify higher-level operational aspects of the radar system (102), such as detection range, and map them to one or more operational parameters, such as transmission power. As another example, the hardware abstraction application programming interface (308) may allow a user to specify a resolution (e.g., range, Doppler, and / or angle) and map it to one or more operational parameters (e.g., bandwidth, integration time, or quantity of active antenna elements). The operation of the radar application programming interface (120) and the radar system (102) to support gesture recognition is further described with reference to FIGS. 4 through 10.

[0076] FIG. 4 illustrates an exemplary request (314) generated by an application (206) and provided to a radar application programming interface (210) for gesture recognition. In this example, the radar application programming interface (210) may represent a use case application programming interface (302). The radar application programming interface (210) enables the application (206) to customize the behavior of the radar system (102) for gesture recognition. In this way, the application (206) can control not only the types of gestures that the radar system (102) can detect, but also the false-positive rate (FPR) of the radar system (102) (e.g., the rate at which the radar system (102) incorrectly recognizes other behaviors as gestures).

[0077] For gesture recognition, the application (206) transmits a request (314) to the radar application programming interface (210). The request (314) includes a pattern recognition sequence (402) that specifies a sequence of behavioral states of an object when performing a gesture. The gesture proceeds through at least two states. Each state may describe one or more attributes (or features) of the object performing the gesture that can be measured or determined using radar sensing. For example, each state may describe the relative or absolute position of the object performing the gesture, motion associated with the object performing the gesture (e.g., range rate, velocity, acceleration, or derivatives thereof), the orientation or radar cross section of the object performing the gesture, the configuration of the object performing the gesture, and / or some combination thereof.

[0078] Each pattern recognition sequence (402) includes at least a starting state describing the starting behavior or starting feature of the gesture and an end state describing the ending behavior or ending feature of the gesture. Depending on the complexity of the gesture, some pattern recognition sequences (402) may also include one or more intermediate states describing one or more behaviors or features of the gesture that occur between the starting state and the end state. Using the pattern recognition sequences (402), the application (206) can define the gesture in a manner distinct from other types of motion or gestures. In this way, the application (206) can control the sensitivity of the radar system (102) for detecting a specific gesture and the ability (function) of the radar system (102) to ignore other motions not associated with the gesture.

[0079] Increasing sensitivity allows the radar system (102) to detect gestures performed by various users through various variations. For example, some users may perform gestures using varying speeds or varying degrees of subtlety or exaggeration. Increasing sensitivity can also improve accessibility for people with disabilities to perform gestures. However, increasing sensitivity may cause the radar system (102) to generate false positives (FP). For example, the radar system (102) may accidentally detect a gesture based on a user performing other motions unrelated to the gesture, such as vacuuming, walking while holding the smart device (104), walking a dog next to the smart device (104), folding clothes next to the smart device (104), making the bed next to the smart device (104), washing dishes next to the smart device (104), or changing the position of an object near the smart device (104). By defining a pattern recognition sequence (402), the application (206) can control the sensitivity level of the radar system (102) and manage the false-positive rate (FPR) of the radar system (102).

[0080] The pattern recognition sequence (402) includes states (404-1, 404-2...404-P), where P represents a positive integer. Each state (404-1 to 404-P) defines the action of an object during a portion of a gesture. Individual states (404-1 to 404-P) may include at least one criterion (406) and optionally transmit one or more outputs (408). The criterion (406) may include one or more entrance criteria, one or more exit criteria, or a combination thereof. The entrance criteria represent criteria for entering states 404-1 to 404-P, and the exit criteria represent criteria for exiting states 404-1 to 404-P. Each criterion (406) can specify a threshold, a conditional comparison (e.g., greater than, less than, equal to), a logical operation (e.g., AND, OR, or NOT), a range of valid values, or a delta change (e.g., displacement).

[0081] Exemplary criteria (406) include a location criterion (410), a motion criterion (412), a physical characteristic criterion (414), and a duration (period) criterion (416). The location criterion (410) includes criteria related to the location of the object performing the gesture. For example, the location criterion (410) may include a range criterion (418) (e.g., a distance criterion or an inclination range criterion) or an angle criterion (420) (e.g., an azimuth criterion and / or an elevation criterion).

[0082] The motion criterion (412) includes criteria related to the motion of the object performing the gesture. For example, the motion criterion (412) may include a range rate criterion (422) or a velocity criterion (424). As another example, the motion criterion (412) may include an acceleration or jerk criterion (not shown). The range rate criterion (422) may be based on a Doppler determined range rate or a range-determined range rate.

[0083] The physical characteristic criteria (414) include criteria related to the physical characteristics of the object performing the gesture. For example, the physical characteristic criteria (414) may include Radar Cross Section (RCS) criteria 426 (RCS criteria 426) describing the behavior of the object's radar cross section. Additionally or alternatively, the physical characteristic criteria (414) may include material composition criteria (428) describing the composition of the object (e.g., human tissue, metal, wood, or cloth). Although not illustrated, the physical characteristic criteria (414) may also include criteria related to the size or dimensions of the object.

[0084] The period criterion (416) specifies a period in which at least one other criterion is satisfied. This period may be specified as a time interval in which an object is detected by the radar system (102) or as a quantity of radar frames (e.g., instances).

[0085] The output (408) enables the current state (404) to transfer information to the next state (404) within the pattern recognition sequence (402). An exemplary output (408) may include location information (e.g., range or angle), motion information (e.g., Doppler frequency, range rate or speed), or physical information (e.g., size, radar cross-section, material composition). By receiving the output (408), the next state (404) can evaluate whether the object's behavior remains relatively similar to the previous state or changes over the gesture.

[0086] Generally, a pattern recognition sequence (402) may include any number of states (404-1 to 404-P) that allow the pattern recognition sequence (402) to define simple or complex gestures. Simple gestures, such as a hand reaching gesture, may have fewer states (404) compared to more complex gestures, such as a pump gesture or sign language. The criteria (406) for each state (4041 to 404-P) may allow for strict or loose tolerances that affect the sensitivity and FPR of the radar system (102) for detecting the gesture. An exemplary pattern recognition sequence (402) is further described in relation to the various gestures of FIGS. 5a through 7b.

[0087] FIG. 5a illustrates an exemplary pattern recognition sequence (402-1) associated with an object (500) performing a pump gesture (502). To perform the pump gesture (502), the object (500) moves toward the smart device (104) and moves away from the smart device (104) at an angle of approximately constant with respect to the radar system (102). In FIG. 5a, the object (500) is depicted as a human hand. However, other types of objects, including inanimate objects, may be used as alternatives.

[0088] In this example, the pattern recognition sequence (402-1) includes states (404-1, 4042, and 404-3). State (404-1) includes an angle criterion (420), a range rate criterion (422-1), and a duration criterion (416-1), which characterizes the behavior of the object (500) when approaching the smart device (104). State (404-2) includes an angle criterion (420) and a range rate criterion (422-2), which characterizes the behavior of the object (500) slowing down at a short distance from the smart device (104). State (404-3) includes an angle criterion (420), a range rate criterion (422-3), and a duration criterion (416-2), which characterizes the behavior of the object (500) when moving away from the smart device (104).

[0089] The criteria across states (404-1 through 404-3) may be similar, as in the case of the angle criterion (420), or different, as in the case of the range rate criterion (422-1, 422-2, 422-3). To reduce false-positives (FP), states 404-1 through 404-3 may include other criteria. For example, states 404-1 through 404-3 may include a radar cross-section criterion (426) to distinguish between a pump gesture (502) made of a finger pointing in a direction perpendicular to the hand motion and another pump gesture (502) made of a finger bent into a fist. As another example, states (404-1 through 404-3) may include a material composition criterion (428) to distinguish between a pump gesture (502) made with the hand and a pump gesture (502) made with a stylus. The criteria (406) associated with the states (404-1 to 404-3) are further explained with reference to FIG. 5b.

[0090] FIG. 5b illustrates an exemplary operation of an object (500) performing a pump gesture (502). Graph (504) illustrates the range rate of the object (500) over the duration of the pump gesture (502). The range rate may be determined based on a change in the measured Doppler frequency or the measured range (e.g., distance) of the object (500). In this case, the range rate changes from a negative value to a positive value. Graph (506) depicts the angle of the object (500) over the duration of the pump gesture (502). In this case, the angle remains relatively constant, which is a characteristic of the pump gesture (502). In this example, consider that the criteria (406) of states 404-1 through 404-3 represent the termination criteria.

[0091] At time T0, the object (500) has a range rate smaller than the range rate criterion (422-1) of the state (404-1). The object (500) also has an angle that is maintained within the margin specified by the angle criterion (420) of the state (404-1). In this case, the object (500) may be at any angle as long as the angle does not change significantly (e.g., as long as it does not change beyond the specified margin). The object (500) satisfies the range rate criterion (422-1) and the angle criterion (420) for a period specified by the duration criterion (416-1). Thus, the first part of the pump gesture (502) is characterized by the state (404-1).

[0092] Between times T1 and T2, the range rate of the object (500) is between the thresholds specified by the range rate reference (422-2) of the state (404-2). These thresholds allow the range rate reference (422-2) to capture the sign of the range rate change (e.g., moving from negative to positive). This indicates that the object (500) changes direction from moving toward the smart device (104) to moving away from the smart device (104). Additionally, the angle of the object (500) is maintained within the angle reference (420). In this way, the second part of the pump gesture (502) is characterized by the state (404-2).

[0093] At time T3, the range rate of the object (500) is greater than the range rate criterion (422-3) of the state (404-3). Additionally, the angle of the object (500) is maintained within the angle criterion (420). The object satisfies the range rate criterion (422-3) and the angle criterion (420) during the period specified by the period criterion (416-2). In this way, the third part of the pump gesture (502) is characterized by the state (404-3).

[0094] By sequentially recognizing objects satisfying the criteria of states 404-1, 404-2, and 404-3, the radar system (102) can recognize a pump gesture (502), and the radar application programming interface (210) can send a response (320) indicating that the pump gesture (502) has occurred to the application (206). The application (206) can perform an action in response to receiving the response (320). For example, the application (206) can stop a timer, click a button, activate a selected option, or pause the media.

[0095] Some applications (206) may reduce false-positives (FP) by adding additional criteria (406) within one or more of the states (404-1 to 404-3) or by tightening the tolerance of the criteria (406). For example, the application (206) may add a range criterion (418) to the state (404-2) to require that the object (500) be within a minimum distance from the smart device (104). As another example, the application (206) adds a radar cross-section criterion (426) or a material composition criterion (428) to the states (404-1 to 404-3) to distinguish between a pump gesture (502) performed by the user and the user flipping a bed sheet next to the smart device (104). As another example, the range rate criteria (422-1 and 422-3) can be appropriately set to enable rejection detections and pump gestures (502) associated with vacuum cleaning that may have a slower range rate that does not satisfy the range rate criteria (422-1 or 422-3).

[0096] FIG. 6a illustrates an exemplary pattern recognition sequence (402-2) associated with an object (500) performing a swipe gesture (602). To perform the swipe gesture (602), the object (500) moves across the smart device (104) along a relatively straight path. The straight path can be oriented along any direction for an omni-swipe gesture. Alternatively, the straight path can be oriented along a specific direction associated with a directional swipe gesture (e.g., from top to bottom, from bottom to top, from left to right, or from right to left). In FIG. 6a, the object (500) is depicted as a human hand. However, other types of objects, including inanimate objects, may also be used as alternatives.

[0097] In this example, the pattern recognition sequence (402-2) includes states (404-1 and 404-2). State (404-1) includes an angle criterion (420-1), a speed criterion (424), and a duration criterion (4161), which characterizes the behavior of the object (500) when it starts swiping across the smart device (104). State (404-2) includes an angle criterion (420-2), a speed criterion (424), and a duration criterion (4162), which characterizes the behavior of the object (500) when it continues swiping across the smart device (104).

[0098] The criteria (406) across the states (404-1, 404-2) may be similar as in the case of the speed criteria (424), or different as in the case of the angle criteria (420-1, 420-2). The criteria (406) associated with the states (404-1 and 404-2) are further explained with reference to FIG. 6b.

[0099] FIG. 6b illustrates an exemplary operation of an object (500) performing a swipe gesture (602). Graph (604) depicts the speed of the object (500) during the duration of the swipe gesture (602). The speed can be determined based on the measured change in position of the object (500). In this case, the speed remains relatively constant. Graph (606) depicts the angle of the object (500) over the duration of the swipe gesture (602). In this case, the angle changes over time, which is a characteristic of the swipe gesture (602). In this example, consider that the criteria (406) of states 404-1 and 404-2 represent the end criteria.

[0100] Between times T0 and T1, the speed of the object (500) is higher than the threshold specified by the speed criterion (424) of the state (404-1). Additionally, the angle of the object (500) changes by an amount specified by the angle criterion (420-1). The object (500) satisfies the speed criterion (424) and the angle criterion (420-1) for a duration specified by the duration criterion (416-1). Thus, the first part of the swipe gesture (602) is characterized by the state (404-1).

[0101] Between times T1 and T2, the speed of the object (500) is maintained above the threshold specified by the speed criterion (424). Additionally, the angle of the object (500) changes by an amount specified by the angle criterion (420-2). The object (500) satisfies the speed criterion (424) and the angle criterion (420-1) for a duration specified by the duration criterion (416-2). Thus, the second part of the swipe gesture (602) is characterized by the state (404-2).

[0102] By sequentially recognizing objects satisfying the criteria of states 404-1 and 404-2, the radar system (102) can recognize a swipe gesture (602), and the radar application programming interface (210) can send a response (320) indicating that a swipe gesture (602) has occurred to the application (206). The application (206) can perform an action in response to receiving the response (320). For example, the application (206) can scroll through content or play the next song in the playlist.

[0103] Another implementation of the pattern recognition sequence (402-2) may include an angle reference (420) that supports a swipe gesture (602) along a specific direction. To recognize a swipe gesture (602) from left to right, the angle reference (420-1) of the state (404-1) can identify whether the object (500) appears on the left side of the smart device (104). Additionally, the angle reference (420-2) of the state (404-2) can identify whether the object (500) moves to the right side of the smart device (104).

[0104] Some applications (206) may reduce false-positives (FP) by adding additional criteria (406) within one or more of the states (404-1 and 404-2) or by tightening the tolerance of the criteria (406). For example, the application (206) may add a material composition criterion (428) to the states (404-1 and 404-2) to distinguish between a user performing a swipe gesture (602) and a user moving an inanimate object (e.g., a cup or paper) across the smart device (104).

[0105] FIG. 7a illustrates an exemplary pattern recognition sequence (402-3) associated with an object (500) performing a reach gesture (702). To perform the reach gesture (702), the object (500) moves toward the smart device (104) along a relatively straight path. In this example, the pattern recognition sequence (402-3) includes states (404-1 and 404-2). State (404-1) includes a range criterion (418-1), a range rate criterion (422), and a duration criterion (416-1), which characterizes the behavior of the object (500) when it begins to reach toward the smart device (104). State (404-2) includes a range criterion (418-2), a range rate criterion (422), and a duration criterion (416-2), which characterizes the behavior of the object (500) as it continues to reach toward the smart device (104).

[0106] The criteria (406) across the states (404-1 and 404-2) may be similar as in the case of the range rate criteria (422), or different as in the case of the range criteria (418-1 and 418-2). The criteria (406) associated with the states (404-1 and 404-2) are further explained with reference to FIG. 7b.

[0107] FIG. 7b illustrates an exemplary action of an object (500) performing a reach gesture (702). Graph (704) depicts the absolute value of the range rate of the object (500) during the duration of the reach gesture (702). The range rate may be determined based on the measured Doppler frequency or the change in the measured range (e.g., distance) of the object (500). In this case, the range rate remains relatively constant. Graph (706) depicts the range (e.g., distance) of the object (500) over the duration of the reach gesture (702). In this case, the range decreases over time, which is a characteristic of the reach gesture (702). In this example, consider that the criteria of the states (404-1 and 404-2) represent the termination criteria.

[0108] Between times T0 and T1, the absolute value of the range rate of the object (500) is higher than the threshold specified by the range rate criterion (422) of the state (404-1). Additionally, the range of the object (500) is smaller than the range criterion (418-1). The object (500) satisfies the range rate criterion (422) and the range criterion (418-1) during the period specified by the period criterion (4161). Thus, the first part of the rich gesture (702) is characterized by the state (404-1).

[0109] Between times T1 and T2, the absolute value of the range rate of the object (500) is higher than the threshold specified by the range rate criterion (422). Additionally, the range of the object (500) is smaller than the range criterion (418-2). The object (500) satisfies the range rate criterion (422) and the range criterion (418-2) during the period specified by the period criterion (416-2). Thus, the second part of the rich gesture (702) is characterized by the state (404-2).

[0110] By sequentially recognizing objects that satisfy the criteria (406) of the states (404-1 and 404-2), the radar system (102) can recognize a rich gesture (702), and the radar application programming interface (210) can transmit a response (320) indicating that the rich gesture (702) has occurred to the application (206). The application (206) can perform an action in response to receiving the response (320). For example, the application (206) can activate a sensor, start face authentication, or turn on a display. The operation of the radar system (102) is further described with reference to FIGS. 8 through 10.

[0111] FIG. 8 illustrates an exemplary antenna array (214) and an exemplary transceiver (216) of a radar system (102). In the illustrated configuration, the transceiver (216) includes a transmitter (802) and a receiver (804). The transmitter (802) includes at least one voltage-controlled oscillator (806) and at least one power amplifier (808). The receiver (804) includes at least two receiving channels (810-1 to 810-M), where M is a positive integer greater than 1. Each receiving channel (810-1 to 810-M) includes at least one low-noise amplifier (LNA) (812), at least one mixer (814), at least one filter (816), and at least one analog-to-digital converter (ADC) (818). The antenna array (214) includes at least one transmitting antenna element (820) and at least two receiving antenna elements (822-1 to 822-M). The transmitting antenna element (820) is connected to a transmitter (802). The receiving antenna elements (822-1 to 822-M) are each connected to receiving channels (810-1 to 810-M). Other implementations of the radar system (102) may include multiple transmitting antenna elements (820) and / or a single receiving antenna element (822).

[0112] During transmission, a voltage-controlled oscillator (806) generates a frequency-modulated radar signal (824) at a radio frequency. A power amplifier (808) amplifies the frequency-modulated radar signal (824) for transmission through a transmitting antenna element (820). The transmitted frequency-modulated radar signal (824) is represented as a radar transmission signal (826) that may contain multiple chirps. For example, the radar transmission signal (826) may contain 16 chirps, which may be transmitted as a continuous burst or as pulses separated by time. For example, the duration of each chirp may be approximately tens or thousands of microseconds (e.g., between approximately 30 microseconds (μs) and 5 milliseconds (ms)).

[0113] Over time, the individual frequencies of the chirp may increase or decrease. The radar system (102) may use a two-slope cycle (e.g., triangular frequency modulation) to linearly increase and linearly decrease the frequencies of the chirp over time. Through the two-slope cycle, the radar system (102) can measure the Doppler frequency shift caused by the motion of the object (500). Generally, the transmission characteristics of the chirp (e.g., bandwidth, center frequency, duration, and transmission power) may be customized to achieve a specific detection range, range resolution, or Doppler sensitivity to detect one or more characteristics of the object (500). In some cases, the characteristics of the chirp are customized by the application (206) using the radar application programming interface (210). For example, the application programming interface engine (226) may selectively configure the transmitter (802) to generate a specific type of radar signal according to the radar application programming interface (210).

[0114] During reception, each receiving antenna element (822-1 to 822-M) receives a radar receiving signal (828-1 to 828-M) representing a delayed version of the radar transmitting signal (826). The amount of delay is proportional to the distance between the antenna array (214) of the radar system (102) and the object (500). In particular, this delay represents the sum of the time it takes for the radar transmitting signal (826) to propagate from the radar system (102) to the object (500) and the time it takes for the radar receiving signal (828-1 to 828-M) to propagate from the object (500) to the radar system (102). Generally, the relative phase difference between the radar receiving signals (828-1 to 828M) is due to the positional difference of the receiving antenna elements (822-1 to 822-M). When the object (500) is moving, the radar received signals (828-1 to 828-M) are shifted in frequency compared to the radar transmitted signals (826) due to the Doppler effect. Similar to the radar transmitted signals (826), the radar received signals (828- to 828-M) consist of one or more chirps.

[0115] In each receiving channel (810-1 to 810-M), a low-noise amplifier (812) amplifies the radar receiving signal (828), and a mixer (814) mixes the amplified radar receiving signal (828) with a frequency-modulated radar signal (824). In particular, the mixer performs a beating operation to down-convert and demodulate the radar receiving signal (828) to generate a beat signal (830).

[0116] The frequency of the beat signal (830) represents the frequency difference between the frequency-modulated radar signal (824) and the radar received signal (828), which is proportional to the slope (gradient) range. Although not illustrated, the beat signal (830) may include multiple frequencies representing reflections from other objects or parts of objects within the external environment. In some cases, these different objects move at different speeds, move in different directions, or are located at different slope ranges relative to the radar system (102).

[0117] A filter (816) filters the bit signal (830), and an analog-to-digital converter (818) digitizes the filtered bit signal (830). Receiving channels (810-1 to 810-M) each generate digital bit signals (832-1 to 832-M) provided to a system processor (218) for processing. The receiving channels (810-1 to 810-M) of the transceiver (216) are connected to the system processor (218) as shown in FIG. 9.

[0118] FIG. 9 illustrates exemplary operation of a hardware abstraction module (228), an object tracker (230), a detection module (232), an application programming interface engine (226), and a radar application programming interface (210). A system processor (218) is connected to receiving channels (810-1 to 810-M) and can also communicate with the radar application programming interface (210). In the illustrated configuration, the system processor (218) implements the hardware abstraction module (228), the object tracker (230), the detection module (232), and the application programming interface engine (226). Although not illustrated, the hardware abstraction module (228), the object tracker (230), the detection module (232), and / or the application programming interface engine (226) may alternatively be implemented by a computer processor (202).

[0119] In some implementations, the application programming interface engine (226) can handle communication between the system processor (218) and the radar application programming interface (210). For example, the application programming interface engine (226) can receive a pattern recognition sequence (402) from the radar application programming interface (210) and transmit the pattern recognition sequence (402) to the detection module (232) or configure the detection module (232) according to the pattern recognition sequence (402). The application programming interface engine (226) can also transmit radar data (318) from the detection module (232) to the radar application programming interface (210). In some cases, the application programming interface engine (226) formats the radar data (318) in a manner specified by the radar application programming interface (210). In other implementations, the radar application programming interface (210) and the detection module (232) can communicate without interfacing through the application programming interface engine (226). In this case, the radar application programming interface (210) transmits a pattern recognition sequence (402) to the detection module (232), and the detection module (232) transmits radar data (318) to the radar application programming interface (210).

[0120] In this example, the hardware abstraction module (228) receives digital bit signals (8321 to 832-M) from receiving channels (810-1 to 810-M). The digital bit signals (832-1 to 832-M) represent raw or unprocessed composite data. The hardware abstraction module (228) performs one or more operations to generate composite radar data (900) based on the digital bit signals (832-1 to 832-M). The composite radar data (900) includes both magnitude and phase information (e.g., in-phase and orthogonal components). In some implementations, the composite radar data (900) includes information representing a range-Doppler map for each receiving channel (810-1 to 810M). In other implementations, the composite radar data (900) includes angle information. The angle information may be implicit within the composite radar data (900), as in a multi-range-Doppler map. Alternatively, the system processor (218) or object tracker (230) performs digital beamforming to explicitly provide angular information, such as in the form of information representing a four-dimensional range-Doppler-azimuth-elevation map. Other forms of complex radar data (900) are also possible. For example, the complex radar data (900) may include complex interferometry data for each receiving channel (810-1 to 810-M). The complex interferometry data is a rectangular representation of the range-Doppler map. In another example, the complex radar data (900) includes a frequency domain representation of a digital bit signal (832-1 to 832-M) for an active radar frame. Sometimes, the complex radar data (900) may include any combination of the above examples. For example, the complex radar data (900) may include magnitude information associated with the range-Doppler map and the complex interferometry data.Although not illustrated, other implementations of the radar system (102) may provide digital bit signals (832-1 to 832M) directly to the application programming interface engine (226).

[0121] The object tracker (230) analyzes the composite radar data (900) and identifies individual objects within the composite radar data (900). Over time, the object tracker (230) collects information about these objects, including location information, motion information, physical characteristic information, or a combination thereof. In this example, the object tracker (230) generates radar timelines (902-1 to 902-O) for objects (500-1 to 500-O), where O represents a positive integer. In some cases, the object tracker (230) generates a single radar timeline (902) associated with the nearest moving object (500). This can increase the execution speed of the application programming interface engine (226).

[0122] The detection module (232) receives radar timelines (902-1 to 902-0) from the object tracker (230) and receives a pattern recognition sequence (402) from the radar application programming interface (210). The detection module (232) recognizes the pattern recognition sequence (402) within at least one of the radar timelines (902-1 to 902-0) and transmits radar data (318) to the radar application programming interface (210). Based on the radar data (318), the radar application programming interface (210) generates a response (320) provided to the application (206). The operation of the detection module (232) is further described with reference to FIG. 10.

[0123] FIG. 10 illustrates an exemplary method implemented by a detection module (232). For gesture recognition, the detection module (232) implements a state machine based on states (4041 to 404-P) specified by a pattern recognition sequence (402). The detection module (232) receives a radar timeline (902) associated with an object (500). The radar timeline (902) includes a list of events (1000-1 to 1000-A), where A represents a positive integer. Each event (1000) may include a timestamp (1002), location information (1004), motion information (1006), physical feature information (1008), or a combination of some of these. The location information (1004), motion information (1006), and physical feature information (1008) may be based on measurement data, smoothed data, and / or prediction data.

[0124] The detection module (232) transitions through states (404-1 to 404-P) as the criteria (406) for each state are satisfied (satisfied) by the radar timeline (902). In response to the completion state (404-P), the detection module (232) transmits radar data (318) to the radar application programming interface (210) so that the radar application programming interface (210) transmits a response (320) to the application (206). This process can be performed for multiple radar timelines from 902-1 to 902-0. In some cases, this operation can be performed in parallel as the object tracker (230) compiles the radar timelines (902-1 to 902-0).

[0125] In some situations, other motions unrelated to the gesture may be observed by the radar system (102). These other motions may include vacuuming, walking, walking a dog, folding clothes, making the bed and / or rearranging objects. When these other motions are observed by the radar system (102), the detection module (232) may determine that these other motions are not associated with the gesture based on the pattern recognition sequence (402). Consider that states (404-1 to 404-P) are defined in a manner that distinguishes the gesture from these other motions. In particular, states (404-1 to 404-P) describe one or more features of the other motions and the other gestures. As the other motions occur, the detection module (232) determines that the criterion (406) for at least one of the states (404-1 to 404-P) is not satisfied. In this way, the radar system (102) determines that the other motions are not associated with the gesture. The radar system (102) is not required to distinguish or identify each of these different motions. Optionally, some radar systems (102) may utilize additional pattern recognition sequences (404) that describe the characteristics of these different motions. In this way, the radar system (102) may explicitly recognize one or more of these different motions.

[0126] Sometimes, the radar system (102) may operate according to multiple modes or support multiple applications (206) during the same time interval. In this case, the system processor (218) may execute multiple instances of detection modules (232) that can be executed in parallel. In this case, each instance of the detection module (232) may be configured by the corresponding application (206) using the radar application programming interface (210). For example, one instance of the detection module (232) may be configured for gesture recognition by the first application (206), and another instance of the detection module (232) may be configured for health monitoring by the second application (206). Additionally, a third instance of the detection module (232) may be configured differently for gesture recognition by the third application (206).

[0127] Example method

[0128] FIG. 11 illustrates an exemplary method (1100) performed by a radar system (102) using a radar application programming interface (210). The method (1100) is illustrated as a set of operations (or actions) performed, but the operations are not necessarily limited to the order or combinations shown herein. Additionally, any of one or more operations may be repeated, combined, reconfigured, or linked to provide a wide range of additional and / or alternative methods. In parts of the following description, the environment of FIG. 1 (100-1 through 100-6) and the entities described in detail in FIG. 2a, 2-2, 3-1, or 4 are referred to merely as examples. The technology is not limited to the performance of a single entity or multiple entities operating on a single device.

[0129] In 1102, a first request from an application of a smart device is accepted through the first layer of the multilayer of the smart device's radar application programming interface. The first request consists of a pattern recognition sequence that specifies a sequence of states associated with a gesture. The multilayer of the radar application programming interface is associated with different operational levels of the radar system. For example, the radar system (102) accepts a first request (314) from an application (206) of the smart device (104) through the first layer (310-1) of the radar application programming interface (210) of the smart device (104), as illustrated in FIG. 3a. The first request (314) includes a pattern recognition sequence (402) that specifies a sequence of states (404-1 to 404P) associated with a gesture, as illustrated in FIG. 4. An exemplary gesture may include a pump gesture (502), a swipe gesture (602), a reach gesture (702), or any of the gestures described above in relation to FIG. 1. The multiple layers (310) of the radar application programming interface (210) are associated with different operating levels (322) of the radar system (102) as illustrated in FIG. 3b. The first layer (310-1) of the radar application programming interface (210) can correspond to a use case application programming interface (302).

[0130] In 1104, a second request from an application of a smart device is accepted through the second layer of the multilayer of the radar application programming interface. The second request includes the detection range of the radar system for detecting gestures. For example, the radar system (102) accepts (accepts) a second request (314) from the application (206) through the fourth layer (310-4) of the radar application programming interface (210). The second request (314) includes the detection range of the radar system (102) for detecting gestures. The fourth layer (310-4) of the radar application programming interface (210) may correspond to a hardware abstraction application programming interface (308).

[0131] In 1106, the hardware configuration of the radar system is modified based on the detection range. For example, the application programming interface engine (226) modifies the hardware configuration (e.g., hardware (324)) of the radar system (102) to achieve a specified detection range. In particular, the application programming interface engine (226) achieves the specified detection range by adjusting the transmit power, the amount of pulses, the amount of active antenna elements in the antenna array (214), and / or the beamforming pattern of the radar system (102). The hardware configuration may include one of the available configurations (312).

[0132] In 1108, radar signals are transmitted and received using a hardware configuration. At least a portion of the radar signal is reflected by an object performing a gesture. For example, the radar system (102) transmits a radar transmission signal (826) and receives radar reception signals (828-1 to 828-M) as shown in FIG. 8. At least a portion of the radar transmission signal (826) is reflected by an object (500) performing a gesture. The object (500) may include an inanimate object such as a user's accessory or a stylus.

[0133] In 1110, a gesture is recognized based on a pattern recognition sequence and a received radar signal. For example, a radar system (102) recognizes a gesture based on a pattern recognition sequence (402) and a received radar signal (828-1 to 828-M). In particular, an object tracker (230) compiles a radar timeline (902) of an object (500). The radar timeline (902) describes the behavior of the object (500) over time. This behavior may include location information (1004), motion information (1006), and / or physical characteristic information (1008). A detection module (232) recognizes the pattern recognition sequence (402) within the radar timeline (902) of the object (500). In particular, the detection module (232) implements a state machine and recognizes a pattern recognition sequence (402) in response to a radar timeline (902) that sequentially satisfies criteria (406) associated with states (404-1 to 404-P).

[0134] In 1112, a response is transmitted to an application through a radar application programming interface. The response indicates the occurrence of a gesture. For example, the radar application programming interface (210) transmits a response (320) indicating the occurrence of a gesture to the application (206).

[0135] Exemplary computing system

[0136] FIG. 12 illustrates various components of an exemplary computing system (1200) that can be implemented as any type of client, server, and / or computing device as described with reference to FIG. 2a and 2-2 to implement a radar application programming interface (210).

[0137] The computing system (1200) includes a communication device (1202) that enables wired and / or wireless communication of device data (1204) (e.g., received data, data in transit, data reserved for broadcasting, or data packets of data). The communication device (1202) or the computing system (1200) may include one or more radar systems (102). The device data (1204) or other device content may include configuration settings of the device, media content stored in the device, and / or information related to the user of the device. Media content stored in the computing system (1200) may include any type of audio, video, and / or image data. The computing system (1200) includes one or more data inputs (1206) to which any type of data, media content, and / or input may be received, such as human speech, inputs selectable by the user (explicit or implicit), messages, music, TV media content, recorded video content, other types of audio, video, and / or image data received from content and / or data sources, etc.

[0138] The computing system (1200) also includes a communication interface (1208) which may be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any type of network interface, a modem, and any other type of communication interface. The communication interface (1208) provides a connection and / or communication link between the computing system (1200) and a communication network through which other electronic, computing, and communication devices communicate data with the computing system (1200).

[0139] A computing system (1200) includes one or more processors (1210) (e.g., any microprocessor, controller, etc.) that process various computer-executable instructions to control the operation of the computing system (1200). Alternatively or additionally, the computing system (1200) may be implemented in any one or a combination of hardware, firmware, or fixed logic circuits implemented in relation to processing and control circuits generally identified in 1212. Although not illustrated, the computing system (1200) may include a system bus or data transfer system that combines various components within the device. The system bus may include one or a combination of various bus structures, such as a memory bus or memory controller (controller), a peripheral device bus, a general-purpose serial bus, and / or a processor or local bus utilizing one of various bus architectures.

[0140] The computing system (1200) includes one or more memory devices (i.e., in contrast to simple signal transmission) that enable continuous and / or non-transient data storage, Random Access Memory (RAM), non-volatile memory (e.g., one or more of Read-Only Memory (ROM), Flash Memory, EPROM, EEPROM, etc.), and computer-readable media (1214) such as disk storage devices. The disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a writable and / or rewritable compact disc (CD), or any type of digital multi-purpose disc (DVD). The computing system (1200) may also include a mass storage media device (storage media) (1216).

[0141] A computer-readable medium (1214) provides a data storage mechanism that stores device data (1204) as well as any other type of information and / or data related to various device applications (1218) and operational aspects of a computing system (1200). For example, an operating system (1220) may be maintained as a computer application via the computer-readable medium (1214) and executed on a processor (1210). Device applications (1218) may include any type of control application, software application, signal processing and control module, code unique to a specific device, a hardware abstraction layer for a specific device, and a device manager.

[0142] The device application (1218) also includes any system component, engine, or manager for implementing the radar application programming interface (210). In this example, the device application (1218) includes the application (206) of FIG. 2b, the radar application programming interface (210), and the application programming interface engine (226).

[0143] conclusion

[0144] Although the technology using a radar application programming interface and the apparatus including the same have been described in language specific to features and / or methods, it should be understood that the essence of the appended claims is not necessarily limited to the specific features or methods described. Rather, specific features and methods are disclosed as exemplary implementations of the radar application programming interface.

[0145] Several examples are provided below.

[0146] Example 1: A method performed by a radar system of a smart device is,

[0147] A step of accepting a first request from an application of the smart device through a first layer of a multilayer radar application programming interface of the smart device—the first request includes a pattern recognition sequence specifying a sequence of states associated with a gesture, and the multilayer of the radar application programming interface is associated with different operation levels of the radar system-;

[0148] A step of accepting a second request from the application of the smart device through the second layer of the multilayers of the radar application programming interface, wherein the second request includes the detection range of the radar system for detecting the gesture;

[0149] A step of modifying the hardware configuration of the radar system based on the above detection range;

[0150] A step of transmitting and receiving a radar signal using the above hardware configuration - at least a portion of the radar signal is reflected by an object performing the gesture -;

[0151] A step of recognizing the gesture based on the pattern recognition sequence and the received radar signal; and

[0152] In response to the above recognition, the method includes the step of sending a response indicating the occurrence of the gesture to the application through the radar application programming interface.

[0153] Example 2: In the method of Example 1, the different operating levels of the radar system are,

[0154] A first operation level associated with a radar-data-processing module of the above radar system; and

[0155] It includes a second operation level associated with the transceiver of the radar system;

[0156] Among the multiple layers of the radar application programming interface, the first layer is associated with the first operation level; and

[0157] Among the multiple layers of the radar application programming interface, the second layer is associated with the second operation level.

[0158] Example 3: As a method of Example 1, the above method is,

[0159] A step of accepting a third request from the application of the smart device through the third layer of the multilayers of the radar application programming interface above—the third request includes an implementation chain specifying a sequence of modules associated with digital signal processing or data processing—; and

[0160] It further includes the step of modifying the software configuration of the radar system based on the above implementation chain.

[0161] Example 4: As a method of the previous example, the sequence of states includes at least two states, and the at least two states are,

[0162] A starting state describing at least one starting feature of the above gesture; and

[0163] It includes a termination state describing at least one termination feature of the above gesture; and

[0164] The above at least one start feature and the above at least one end feature each describe the characteristics of the object performing the gesture, which can be determined using the radar sensing.

[0165] Example 5: In the method of the previous example, each state within the sequence of states includes at least one criterion associated with the gesture;

[0166] The sequence of the above states includes a first state;

[0167] The above first state includes a first standard; and

[0168] The above first standard is,

[0169] Entry criteria for entering the above first state; or

[0170] It includes at least one of the termination criteria for terminating the first state.

[0171] Example 6: In the method of Example 5, the above entry criterion or the above exit criterion is,

[0172] Based on location;

[0173] Motion standard;

[0174] Signal characteristic criteria; or

[0175] It includes at least one of the period criteria.

[0176] Example 7: In the method of any previous example, the step of recognizing the gesture is,

[0177] A step of compiling a radar timeline of the object based on the received radar signal—the radar timeline describes the behavior of the object over time—; and

[0178] It includes the step of detecting the pattern recognition sequence within the radar timeline.

[0179] Example 8: In the method of Example 7, the radar timeline includes a plurality of events, and each event is,

[0180] Timestamp;

[0181] Location information for the above object;

[0182] Motion information for the above object; or

[0183] It includes at least one of the physical characteristic information for the above object.

[0184] Example 9: In the method of Example 7 or 8, the gesture includes a pump gesture; and

[0185] The step of detecting the above pattern recognition sequence is,

[0186] A step of detecting that, according to a first state among the sequence of the above states, the object moves toward the radar system along an angle having an absolute value of a range rate higher than a first threshold value during a first period;

[0187] A step of detecting that, according to a second state among the sequence of the above states, the object changes direction from a direction moving toward the radar system to a direction moving away from the radar system; and

[0188] According to a third state among the sequence of the above states, the method includes the step of detecting that the object moves away from the radar system along an angle having an absolute value of a range rate higher than a second threshold during a second period.

[0189] Example 10: In the method of Example 7 or 8, the gesture includes a swipe gesture; and

[0190] The step of detecting the above pattern recognition sequence is,

[0191] A step of detecting that the object changes its angle at a rate higher than a threshold during a first period according to a first state among the sequence of the above states; and

[0192] According to a second state among the sequence of the above states, the method includes a step of detecting that the object changes its angle along the same direction as the first state at a speed higher than a threshold during the second period.

[0193] Example 11: In the method of Example 10, the swipe gesture includes a directional swipe gesture; and

[0194] The step of detecting that the object changes its angle further includes the step of detecting an object that changes its angle along the direction associated with the directional swipe gesture.

[0195] Example 12: In the method of Example 7 or 8, the gesture includes a reach gesture; and

[0196] The step of detecting the above pattern recognition sequence is,

[0197] A step of detecting that, according to a first state among the sequence of the above states, the object moves toward the radar system and that the object has a range smaller than a first threshold value during a first period; and

[0198] According to a second state among the sequence of the above states, the object moves toward the radar system and detects that the object has a range smaller than a second threshold during a second period.

[0199] Example 13: In the method of any previous example, the object performs a motion other than the gesture; and

[0200] The above method is,

[0201] It includes a step of determining that the other motion is not associated with the gesture based on the pattern recognition sequence above.

[0202] Example 14: In the method of Example 13, the other motion is,

[0203] A person who uses a vacuum cleaner;

[0204] A person walking while holding the above smart device;

[0205] A person walking a dog next to the smart device mentioned above;

[0206] A person folding clothes next to the smart device mentioned above;

[0207] A person making the bed next to the above smart device; or

[0208] It is associated with at least one of the people who change the location of an object near the smart device.

[0209] Example 15: In any previous example method, modifying the hardware configuration is,

[0210] It includes adjusting transmission power, adjusting the quantity of pulses, adjusting the amount of active antenna elements within the antenna array, and / or adjusting the beamforming pattern of the radar system to achieve a specified detection range.

[0211] Example 16: The device includes a radar system configured to perform any one of the methods of Examples 1 to 15.

[0212] Example 17: A computer-readable storage medium includes an instruction that causes a radar system to perform the method of any one of Examples 1 to 15 in response to execution by a processor.

Claims

Claim 1 A method performed by a radar system of a smart device, comprising: accepting a first request from an application of the smart device through a first layer of a multilayer of a radar application programming interface of the smart device, wherein the first request includes a pattern recognition sequence specifying a sequence of states associated with a gesture, and the multilayer of the radar application programming interface is associated with different operational levels of the radar system; accepting a second request from the application of the smart device through a second layer of a multilayer of the radar application programming interface, wherein the second request includes a detection range of the radar system for detecting the gesture; modifying the hardware configuration of the radar system based on the detection range; transmitting and receiving a radar signal using the modified hardware configuration, wherein at least a portion of the radar signal is reflected by an object performing the gesture; recognizing a gesture specified by the application using the pattern recognition sequence included in the first request based on the pattern recognition sequence and the received radar signal; and, in response to the recognition, sending a response indicating the occurrence of the gesture to the application through the radar application programming interface. Claim 2 A method performed by a radar system of a smart device, wherein the different operating levels of the radar system include a first operating level associated with a radar-data-processing module of the radar system; and a second operating level associated with a transceiver of the radar system; wherein the first layer of the multiple layers of the radar application programming interface is associated with the first operating level; and the second layer of the multiple layers of the radar application programming interface is associated with the second operating level. Claim 3 A method performed by a radar system of a smart device according to claim 1, further comprising the step of accepting a third request from the application of the smart device through a third layer of the multiple layers of the radar application programming interface—the third request includes an implementation chain specifying a sequence of modules associated with digital signal processing or data processing—and modifying the software configuration of the radar system based on the implementation chain. Claim 4 A method performed by a radar system of a smart device, wherein, in claim 1, the sequence of states comprises at least two states, and the at least two states comprise a start state describing at least one start feature of the gesture; and a end state describing at least one end feature of the gesture; and the at least one start feature and the at least one end feature each describe a characteristic of the object performing the gesture that can be determined using the radar sensing. Claim 5 A method performed by a radar system of a smart device, wherein, in claim 1, each state within the sequence of states comprises at least one criterion associated with the gesture; the sequence of states comprises a first state; the first state comprises a first criterion; and the first criterion comprises at least one of an entry criterion for entering the first state; or an exit criterion for exiting the first state. Claim 6 A method performed by a radar system of a smart device, wherein the entry criterion or the termination criterion comprises at least one of a position criterion; a motion criterion; a signal characteristic criterion; or a period criterion, in accordance with claim 5. Claim 7 A method performed by a radar system of a smart device, wherein the step of recognizing the gesture comprises: compiling a radar timeline of the object based on the received radar signal—the radar timeline describes the behavior of the object over time—and detecting the pattern recognition sequence within the radar timeline. Claim 8 A method performed by a radar system of a smart device, wherein, in claim 7, the radar timeline comprises a plurality of events, and each event comprises at least one of a time stamp; location information for the object; motion information for the object; or physical characteristic information for the object. Claim 9 A method performed by a radar system of a smart device, wherein, in claim 7, the gesture includes a pump gesture; and the step of detecting the pattern recognition sequence comprises: detecting, according to a first state among the sequence of states, that the object moves toward the radar system along an angle having an absolute value of a range rate higher than a first threshold during a first period; detecting, according to a second state among the sequence of states, that the object changes direction from moving toward the radar system to moving away from the radar system; and detecting, according to a third state among the sequence of states, that the object moves away from the radar system along an angle having an absolute value of a range rate higher than a second threshold during a second period. Claim 10 A method performed by a radar system of a smart device, wherein, in claim 7, the gesture includes a swipe gesture; and the step of detecting the pattern recognition sequence comprises: a step of detecting that, according to a first state among the sequence of states, the object changes its angle at a speed higher than a threshold during a first period; and a step of detecting that, according to a second state among the sequence of states, the object changes its angle along the same direction as the first state at a speed higher than a threshold during a second period. Claim 11 A method performed by a radar system of a smart device, wherein, in claim 10, the swipe gesture includes a directional swipe gesture; and the step of detecting that the object changes its angle further includes the step of detecting an object that changes its angle along a direction associated with the directional swipe gesture. Claim 12 A method performed by a radar system of a smart device, wherein, in claim 7, the gesture includes a reach gesture; and the step of detecting the pattern recognition sequence comprises: detecting, according to a first state among the sequence of states, that the object moves toward the radar system and that the object has a range smaller than a first threshold during a first period; and detecting, according to a second state among the sequence of states, that the object moves toward the radar system and that the object has a range smaller than a second threshold during a second period. Claim 13 In claim 1, the object performs a motion other than the gesture; and the method is performed by a radar system of a smart device, comprising the step of determining that the other motion is not associated with the gesture based on the pattern recognition sequence. Claim 14 In paragraph 13, the other motion is performed by a radar system of a smart device, which is associated with at least one of a person using a vacuum cleaner; a person walking while holding the smart device; a person walking a dog next to the smart device; a person folding clothes next to the smart device; a person making a bed next to the smart device; or a person changing the position of an object near the smart device. Claim 15 A method performed by a radar system of a smart device, wherein, in claim 1, modifying the hardware configuration comprises adjusting the transmission power, adjusting the quantity of pulses, adjusting the amount of active antenna elements in the antenna array, and / or adjusting the beamforming pattern of the radar system to achieve a specified detection range. Claim 16 A device comprising a radar system configured to perform the method of any one of claims 1 to 15. Claim 17 A computer-readable storage medium comprising instructions that cause a radar system to perform the method of any one of claims 1 through 15 in response to execution by a processor.

Images