Radar Application Programming Interface

Abstract

Techniques and apparatus are described for implementing a radar application programming interface (210). The radar application programming interface (210) provides a standardized language for external entities, such as applications (206) on smart devices (104), to communicate with the radar system (102) without having to understand the design and operational complexities used by the radar system (102). The radar application programming interface (210) allows third parties to easily interact with the radar system (102) and customize its use for a wide variety of different applications (206). In this manner, the radar application programming interface (210) can enable other entities to extend the use of the radar system (102), thereby providing additional features that enhance the user experience.

Description

Background Art

[0001] Background A radar is a useful device capable of detecting objects. Compared with other types of sensors such as cameras, radar can provide performance improvements in many different environmental conditions, such as low lighting and fog, or moving or overlapping objects. Radar can also detect objects through one or more shielded states, such as a handbag or pocket.

[0002] Radar can be a complex technology with specific trade-off considerations to achieve target performance. Exemplary trade-off considerations include detection range, range, Doppler, and angular resolution, range and Doppler frequency ambiguity, sensitivity, false detection rate, response time, size, power consumption, and cost. Due to its complex design and operational considerations, radar is often customized for specific applications. For example, some radars can provide navigation assistance, other radars can create environmental maps, and still other radars can assist users interacting with electronic devices.

[0003] Although radar has many advantages, it may be regarded as a black box by other external entities that may not be aware of the complexity that enables radar to achieve target performance. As a result, it can be difficult for applications within an electronic device to utilize the features provided by the radar. Consequently, radar may be limited to supporting some specific applications.

Summary of the Invention

[0004] Summary Techniques and apparatuses for implementing a radar application programming interface (API) are described. The radar application programming interface provides a standardized language for external entities, such as applications on smart devices, to communicate with a radar system without the need to understand the design and operational complexities used by the radar system. Using this radar application programming interface, third parties can easily interact with the radar system and customize the use of the radar system for a variety of different applications. Thus, this radar application programming interface can enable other entities to extend the utilization of the radar system, thereby providing additional functionality to improve the user experience.

[0005] The aspects described below include a method executed by a radar system of a smart device. This method includes receiving a first request from an application of the smart device via a first layer of a plurality of layers of a radar application programming interface of the smart device. The first request includes a pattern recognition sequence that specifies a sequence of states associated with a gesture. The plurality of layers of the radar application programming interface are associated with different operating levels of the radar system. This method also includes receiving a second request from an application of the smart device via a second layer of the plurality of layers of the radar application programming interface. The second request includes a detection range of the radar system for detecting a gesture. The method additionally includes modifying a hardware configuration of the radar system based on the detection range. The method further includes transmitting and receiving radar signals using the hardware configuration. At least a portion of the radar signals is reflected by an object performing a gesture. The method also includes recognizing a gesture based on the pattern recognition sequence and the received radar signals. In response to the recognition, the method includes transmitting a response indicating the occurrence of the gesture to the application via the radar application programming interface.

[0006] The aspects described below also include an apparatus comprising a radar system configured to execute any of the methods described.

[0007] The aspects described below include a computer-readable storage medium comprising computer-executable instructions that, in response to execution by a processor, cause a radar system to execute any one of the methods described.

[0008] The aspects described below also include a system having means for providing a radar application programming interface.

[0009] An apparatus for implementing a radar application programming interface and the technology for implementing it will be described with reference to the following drawings. The same numbers are used throughout the drawings to refer to similar features and components.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2-1

Figure 2-2

Figure 3-1

Figure 3-2

Figure 4

Figure 5-1

Figure 5-2

Figure 6-1

Figure 6-2

Figure 7-1

Figure 7-2

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0011] Detailed Description Radar is a useful device that can detect objects. Compared to other types of sensors such as cameras, radar can provide performance improvements in many different environmental conditions such as low lighting and fog, or moving or overlapping objects. Radar can also detect objects through one or more shielded states, such as a handbag or pocket.

[0012] Radar can be a complex technology with specific trade - off considerations to achieve target performance. Exemplary trade - off considerations include detection range, range, Doppler and angular resolution, range and Doppler frequency ambiguity, sensitivity, false detection rate, reaction time, size, power consumption, and cost. Due to its complex design and operational considerations, radar is often customized for specific applications. For example, some radars provide navigation assistance, others can create environmental maps, and still others can assist users interacting with electronic devices.

[0013] Radar has many advantages, but it may be seen as a black box by other external entities that may not be aware of the complexity that enables radar to achieve target performance. As a result, it can be difficult for applications within an electronic device to utilize the features provided by radar. Consequently, radar may be limited to supporting some specific applications.

[0014] To address this problem, a technique for implementing a radar application programming interface (API) is described. The radar application programming interface provides a standardized language for external entities, such as applications on a smart device, to communicate with a radar system without the need to understand the design and operational complexity used by the radar system. Using this radar application programming interface, a third - party can easily interact with the radar system and customize the use of the radar system for a wide variety of different applications. Thus, this radar application programming interface can enable other entities to extend the utilization of the radar system, thereby providing additional functionality to improve the user experience.

[0015] Operating Environment FIG. 1 is a diagram of exemplary environments 100-1 to 100-6 in which techniques using a radar system and an apparatus including the radar system may be implemented. In the illustrated environments 100-1 to 100-6, a smart device 104 includes a radar system 102 that can interface with a variety of different applications of the smart device 104 via a (FIG. 2-1) radar application programming interface. The smart device 104 is shown as a smartphone in environments 100-1 to 100-5 and a smart vehicle in environment 100-6.

[0016] In environments 100-1 to 100-4, a user performs different types of gestures, which are detected by the radar system 102. In some cases, the user uses an accessory or body part to perform the gesture. Alternatively, the user can also perform the gesture using a stylus, a handheld object, a ring, or any type of material that reflects radar signals. By using the radar application programming interface, the operation of the radar system 102 can be dynamically adjusted based on the type of gesture that the radar system 102 is required to recognize.

[0017] In environment 100-1, the user performs a scroll gesture by moving their hand above the smart device 104 along the horizontal dimension (e.g., from the left side of the smart device 104 to the right side of the smart device 104). In environment 100-2, the user performs a gesture of extending their hand to decrease the distance between the smart device 104 and their hand. The user in environment 100-3 performs a hand gesture for playing a game on the smart device 104. In one example, the user performs a pressing gesture by moving their hand above the smart device 104 along the vertical dimension (e.g., from the lower side of the smart device 104 to the upper side of the smart device 104). Using the information passed by the radar application programming interface, the radar system 102 can recognize the gesture being performed by the user. In environment 100-4, the smart device 104 is housed in a handbag, and the radar system 102 realizes occlusion gesture recognition by detecting the gesture occluded by the handbag.

[0018] The radar system 102 can also recognize other types of gestures or movements not illustrated in FIG. 1. Exemplary types of gestures include a knob rotation gesture in which the user curls his or her fingers to grasp an imaginary doorknob and rotates the fingers and hand clockwise or counterclockwise to mimic the action of turning an imaginary doorknob. Another exemplary type of gesture includes a gesture of twisting an axis of rotation performed by the user rubbing the thumb and at least one other finger together. These gestures can be two-dimensional, such as gestures used with a touch-sensitive display (e.g., pinching with two fingers, spreading with two fingers, or tapping). These gestures can also be three-dimensional, such as many sign language gestures, e.g., American Sign Language (ASL) and sign language gestures from other parts of the world. Upon detecting each of those gestures, the smart device 104 can perform actions such as displaying new content, moving a cursor, activating one or more sensors, opening an application, etc. In this way, the radar system 102 provides contactless control of the smart device 104.

[0019] In environment 100-5, the radar system 102 generates a 3D map of the surrounding environment for situation awareness. The radar system 102 also detects and tracks multiple users to enable both users to interact with the smart device 104. The radar system 102 can also perform biometric signal detection. In environment 100-6, the radar system 102 monitors the biometric signals of the user driving the vehicle. Examples of biometric signals include heart rate and respiratory rate. For example, if the radar system 102 determines that the driver has fallen asleep, the radar system 102 may cause the smart device 104 to generate an alert for the user. Alternatively, if the radar system 102 detects a life-threatening emergency such as a heart attack, the radar system 102 may cause the smart device 104 to generate an alert to a medical professional or emergency service. In some embodiments, the radar system 102 in environment 100-6 can support collision avoidance for autonomous driving and / or navigation assistance. Generally, the radar system can support a wide variety of different applications including non-contact gesture control, health management (e.g., sleep tracking or biometric signal monitoring), fitness tracking, proximity detection, spatial mapping, and human activity recognition (e.g., fall detection, attention, sitting, or standing).

[0020] Some embodiments of the radar system 102 are particularly beneficial when applied in the context of a smart device 104 where there are convergence issues. This can include restrictions in the spacing and layout of the radar system 102, as well as the need for low power. An exemplary overall horizontal dimension of the smart device 104 can be, for example, approximately 8 centimeters by approximately 15 centimeters. An exemplary footprint of the radar system 102, including the antenna, can be further restricted to, for example, approximately 4 millimeters by 6 millimeters. An exemplary power consumption of the radar system 102 can be on the order of a few milliwatts to tens of milliwatts (e.g., approximately 2 milliwatts to 20 milliwatts). Due to such restricted footprint and power consumption requirements of the radar system 102, the smart device 104 can include other desirable features in a spatially restricted package (e.g., camera sensor, fingerprint sensor, display, etc.). The smart device 104 and the radar system 102 are further described with respect to FIG. 2-1.

[0021] FIG. 2-1 shows a radar system 102 as part of a smart device 104. The smart device 104 is shown with various non-limiting exemplary devices including a desktop computer 104-1, a tablet 104-2, a laptop 104-3, a television 104-4, a computing watch 104-5, computing glasses 104-6, a gaming system 104-7, a microwave oven 104-8, and a vehicle 104-9. Other devices may also be used, such as home service devices, smart speakers, smart thermostats, security cameras, baby monitors, Wi-Fi (registered trademark) routers, drones, track pads, drawing pads, netbooks, e-book readers, home automation and control systems, wall displays, and other household appliances. The smart device 104 can be wearable, portable but not wearable, or relatively fixed (e.g., desktop and appliances). The radar system 102 can be used as a stand-alone radar system, or in conjunction with or built into many different smart devices 104 or peripherals, such as a control panel for controlling home appliances and systems, for controlling internal functions (e.g., volume, cruise control, and even driving of a vehicle) within an automobile, or for controlling computing applications on a laptop.

[0022] The smart device 104 includes one or more computer processors 202 and at least one computer-readable medium 204 including a memory medium and a storage medium. An application and / or an operating system (not shown) embodied as computer-readable instructions on the computer-readable medium 204 can be executed by the computer processor 202 to provide some of the functions described herein. The computer-readable medium 204 also includes an application 206, and the application 206 uses radar data generated by the radar system 102 to perform functions such as presence detection, gesture-based non-contact control, collision avoidance for autonomous driving, health management, fitness tracking, space mapping, and human activity recognition.

[0023] The smart device 104 may also include a network interface 208 for communicating data via a wired, wireless, or optical network. For example, the network interface 208 can communicate data via a local area network (LAN), a wireless local area network (WLAN), a personal area network (PAN), a wire area network (WAN), an intranet, the Internet, a peer-to-peer network, a two-point network, a mesh network, etc. The smart device 104 may also include a display (not shown).

[0024] The smart device 104 includes a radar application programming interface 210 (e.g., radar API 210). The radar application programming interface 210 provides an interface between the radar system 102 and an external entity such as the application 206 or another radar system. Generally, the radar application programming interface 210 provides a standardized language that can be used by the external entity (e.g., the application 206) and the radar system 102 for mutual communication. In some aspects, the radar application programming interface 210 converts various requests provided by one or more external entities (e.g., one or more applications 206) into an executable operation configuration for the radar system 102.

[0025] Through the radar application programming interface 210, the application 206 can control the operation of the radar system 102 and / or receive information from the radar system 102 without understanding how the radar system 102 functions. The radar application programming interface 210 defines the interaction between the application 206 and the radar system 102, including the types of requests and available responses, the method of transmitting requests or responses, data formats, and other notation methods. Also, the radar application programming interface 210 can ensure compliance with other rules and guidelines, including those provided by agencies such as the Federal Communications Commission (FCC) of the United States. In some cases, this information is obtained in a standard document that enables a third party to program the application 206 accordingly. For example, the radar application programming interface 210 can define a set of functional blocks that are common across a wide variety of different types of radar systems 102, making it possible for non-radar technicians (e.g., developers, hobbyists, and researchers) to apply those functional blocks in their own ways to achieve different goals. The radar application programming interface 210 can also enable a wide variety of different applications or external entities to communicate with a wide variety of different types of radar systems 102 (e.g., radar systems having different hardware designs, operating configurations, and / or performance capabilities).

[0026] The radar application programming interface 210 can be implemented in software, programmable hardware, or some combination thereof. The radar application programming interface 210 can exist inside or outside the radar system 102. In some embodiments, the radar application programming interface 210 includes instructions stored in the CRM 204 and executable by the computer processor 202. In other embodiments, the instructions for implementing the radar application programming interface 210 are stored in the CRM (e.g., the system medium 220) of the radar system 102 and are executable by a processor (e.g., the system processor 218).

[0027] The radar system 102 can implement one or more different types of radars, such as a frequency modulated continuous wave (FMCW) radar, a pulsed Doppler radar, a continuous wave (CW) radar, a phase modulated spectrum spreading radar, an impulse radar, a radar using a Zadoff-Chu sequence or a constant amplitude zero autocorrelation (CASAC) sequence, or a multiple input multiple output (MIMO) radar. The radar system 102 includes a communication interface 212 for transmitting radar data to a remote device, but this interface need not be used if the radar system 102 is integrated within the smart device 104. Generally, the radar data provided by the communication interface 212 is in a form usable by the radar application programming interface 210 and the application 206.

[0028] The radar system 102 also includes at least one antenna array 214 and at least one transceiver 216 for transmitting and receiving radar signals. The antenna array 214 includes at least one transmit antenna element and at least one receive antenna element. Depending on the situation, the antenna array 214 includes a plurality of transmit antenna elements and / or a plurality of receive antenna elements. Using the plurality of transmit antenna elements and the plurality of receive antenna elements, the radar system 102 can implement a multiple-input multiple-output radar that can transmit a plurality of distinct waveforms (e.g., different waveforms for each transmit antenna element) at a given time. The antenna elements can be circularly polarized, horizontally polarized, vertically polarized, or a combination thereof.

[0029] The plurality of receive antenna elements of the antenna array 214 can be positioned in a one-dimensional shape (e.g., a line) or a two-dimensional shape (e.g., a rectangular arrangement, a triangular arrangement, or an "L" shape arrangement) for embodiments that include three or more receive antenna elements. With a one-dimensional shape, the radar system 102 can measure one angular dimension (e.g., azimuth or elevation angle), while with a two-dimensional shape, the radar system 102 can measure two angular dimensions (e.g., to determine both the azimuth angle and elevation angle of an object). The element spacing associated with the receive antenna elements can be smaller than, larger than, or equal to half of the center wavelength of the radar signal.

[0030] Transceiver 216 includes circuitry and logic for transmitting and receiving radar signals via antenna array 214. The components of transceiver 216 can include amplifiers, phase shifters, mixers, switches, analog-to-digital converters, or filters for conditioning the radar signals. Transceiver 216 also includes logic for performing operations in in-phase / quadrature phase (I / Q) operation, such as modulation or demodulation. A wide variety of modulations can be used, including linear frequency modulation, triangular frequency modulation, stepped frequency modulation, or phase modulation. Alternatively, transceiver 216 can generate radar signals having a relatively constant frequency or tone. Transceiver 216 can be configured to support continuous wave or pulsed radar operation.

[0031] The frequency spectrum (e.g., range of frequencies) used by transceiver 216 to generate radar signals can encompass frequencies between 1 and 400 gigahertz (GHz), between 4 and 100 GHz, between 1 and 24 GHz, between 2 and 4 GHz, between 30 and 80 GHz, between 57 and 64 GHz, or approximately 2.4 GHz. In some cases, the frequency spectrum can be divided into multiple sub-spectra having similar or different bandwidths. The bandwidth can be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, etc. In some cases, the bandwidth is at least approximately 20% of the center frequency to implement ultra-wideband (UWB) radar.

[0032] Different frequency sub - spectra can include frequencies, for example, between approximately 57 - 59 GHz, 59 - 61 GHz, or 61 - 63 GHz. The exemplary frequency sub - spectra described above are continuous, but other frequency sub - spectra may not be continuous. To achieve coherence, multiple frequency sub - spectra (whether continuous or discontinuous) having the same bandwidth may be used by transceiver 216 to generate multiple radar signals, and the generated radar signals are transmitted either simultaneously or separately in time. Depending on the situation, multiple continuous frequency sub - spectra may be used to transmit a single radar signal, whereby the radar signal can have a wide bandwidth.

[0033] Radar system 102 also includes one or more system processors 218 and at least one system medium 220 (e.g., one or more computer - readable storage media). System medium 220 is further described with respect to FIG. 2 - 2.

[0034] FIG. 2-2 is a diagram showing an exemplary system medium 220 of the radar system 102. The system medium 220 includes at least one digital signal processing (DSP) module 222, at least one radar data processing module 224, and at least one application programming interface (API) engine 226. The digital signal processing module 222, the radar data processing module 224, and / or the application programming interface engine 226 can be implemented using hardware, software, firmware, or a combination thereof. In this example, the system processor 218 implements the digital signal processing module 222, the data processing module 224, and the application programming interface engine 226. In an alternative embodiment (not shown), at least one of the digital signal processing module 222, the radar data processing module 224, or the application programming interface engine 226 is included within the computer-readable medium 204 and is implemented by the computer processor 202. For example, the digital signal processing module 222 can be included within the system medium 220 and executed by the system processor 218, while the radar data processing module 224 and the application programming interface engine 226 can be included within the computer-readable medium 204 and executed by the computer processor 202. In this case, the radar system 102 can provide the radar data of the smart device 104 via the communication interface 212 so that the computer processor 202 can process the radar data.

[0035] The digital signal processing module 222 performs low-level processing on the received analog and / or digital samples of the radar signal to achieve real-time signal processing. Exemplary types of signal processing may include non-coherent integration, clutter removal, detection threshold processing, noise cancellation, Doppler filtering, interferometry, and / or digital beamforming. Some functions of the digital signal processing module 222 may involve iterative mathematical operations such as addition, subtraction, and multiplication. These iterative mathematical operations may be performed to implement a Fourier transform (e.g., fast Fourier transform). In one embodiment, the digital signal processing module 222 includes a hardware abstraction module 228. Generally, the digital signal processing module 222 processes the raw data provided by the transceiver 216 and generates preprocessed data in a form usable by the radar data processing module 224.

[0036] The radar data processing (RDP) module 224 performs medium-level and / or high-level processing on the preprocessed data received from the digital signal processing module 222 to achieve real-time data processing. The radar data processing module 224 may perform functions such as object tracking, clutter tracking, gesture recognition, presence detection, biometric recognition, navigation assistance, and / or health management. These functions can be implemented using discovery algorithms and / or machine learning algorithms. In an exemplary embodiment, the radar data processing module 224 includes an object tracker 230 and a detection module 232.

[0037] The digital signal processing module 222 and the radar data processing module 224 can be designed for compatibility across a wide variety of different types of radar systems 102, including radar systems 102 having different types of hardware, different hardware configurations, and / or different operating configurations (e.g., FMCW radar, pulsed Doppler radar, or impulse radar). In other words, the digital signal processing module 222 and / or the radar data processing module 224 can be at least somewhat independent of the hardware implementation and operation of the radar system 102.

[0038] The data generated by the digital signal processing module 222 and / or the radar data processing module 224 can be self - descriptive. In some cases, the structure of the data itself describes the format and meaning of that data. In other cases, the digital signal processing module 222 and / or the radar data processing module 224 can provide information regarding the format and meaning of the data by query via the radar application programming interface 210. Exemplary data formats include packed or compressed data formats (e.g., a specific quantity of 13 - bit integers), unpacked data formats (e.g., an array of 32 - bit floats), or dimensions of multi - dimensional data with row interleaving or row - wise units.

[0039] Each module (or function) implemented within digital signal processing module 222 and / or radar data processing module 224 may have defined inputs and / or outputs. Some of these inputs and / or outputs may be standardized such that external entities (e.g., application 206) using radar application programming interface 210 can easily customize the operation of digital signal processing module 222 and / or radar data processing module 224 for different use cases and various radar types. In particular, an external entity can utilize radar application programming interface 210 to specify an implementation chain (or implementation sequence) for processing samples of received radar signals and outputting radar data for application 206. This implementation chain may represent a set of digital signal processing module 222 and / or radar data processing module 224 that are executed in a specific order. By specifying this implementation chain, the external entity can customize which modules to enable or disable and in what order those modules operate.

[0040] Inputs and / or outputs may be associated with one or more standard rendering planes. In some cases, the inputs and outputs of a module can refer to the same standard rendering plane. In this case, the module operates on the input data without changing the format of the input data. Thus, the output data has the same format as (or is associated with the same standard rendering plane as) the input data. In other cases, the inputs and outputs of a module can refer to different standard rendering planes. A module associated with a particular type can support particular inputs and outputs. In this case, the module operates on its input data and the format of the input data is converted to generate the output data. Thus, the output data has a different format from (or is associated with a different standard rendering plane from) the input data.

[0041] By knowing the standard rendering planes associated with the modules, an engineer can easily swap in or swap out different modules to operate the radar system 102 to achieve the desired computational cost, memory cost, configurability, robustness, or performance. Exemplary standard rendering planes can include raw digital samples of received radar signals for one or more channels, information representing a range-Doppler map, information representing a range-Doppler-azimuth-elevation map, information representing an interferogram, object data (e.g., position information, motion information, and / or physical information regarding one or more objects), gesture data, biometric data, health data, collision avoidance data, etc. Some rendering planes may include information representing the uncertainty of the modules. This uncertainty information can be used by other modules to recognize an error or bad sensing state.

[0042] Some modules may support multiple standard rendering planes (e.g., support data with alternative representations). As an example, the object tracker 230 may output a grid representation that can be considered a "dense" representation. In that grid representation, each grid element (or cell) identifies whether an object is present at that location. Thus, the grid representation may include additional information regarding locations that do not contain objects. Alternatively, the object tracker 230 may output a list of objects that can be considered a "sparse" representation. In the list of objects, each enumerated object may include information regarding its grid position. Since this list of objects may not include information regarding grid elements that do not contain objects, the list of objects may have a smaller memory size than the grid representation. In this case, an external entity can use the radar application programming interface 210 to specify the appropriate standard rendering plane depending on whether downstream modules can utilize the additional information provided by the grid representation.

[0043] The Application Programming Interface Engine 226 can function as an interface between the Radar Application Programming Interface 210 and components of the radar system 102 (e.g., the antenna array 214, the transceiver 216, and / or the system processor 218). In some cases, the Application Programming Interface Engine 226 modifies the operation of the radar system 102 according to requests received by the Radar Application Programming Interface 210. For example, the Application Programming Interface Engine 226 can activate a specific number of antenna elements within the antenna array 214, or cause the transceiver 216 to generate a specific type of radar signal specified by the Radar Application Programming Interface 210. As another example, the Application Programming Interface Engine 226 can cause the system processor 218 to execute an implementation chain specified by the Radar Application Programming Interface 210, and / or cause modules within the implementation chain to operate with one or more parameters provided by the Radar Application Programming Interface 210.

[0044] The Application Programming Interface Engine 226 can also provide information to the Radar Application Programming Interface 210, and the Radar Application Programming Interface 210 can pass that information to the application 206. Optionally, the Application Programming Interface Engine 226 can reformat data received from the Radar Application Programming Interface 210, or data provided to the Radar Application Programming Interface 210. The operation of the Hardware Abstraction Module 228, the Object Tracker 230, and the Detection Module 232 is further described below.

[0045] The hardware abstraction module 228 converts the raw data provided by the transceiver 216 into hardware-independent data. This hardware-independent data can be processed by the object tracker 230 and / or the detection module 232. In particular, the hardware abstraction module 228 matches the complex data from a wide variety of different types of radar signals to the expected input. This enables the object tracker 230 or the detection module 232 to process different types of radar signals received by the radar system 102, including those that utilize different modulation schemes for frequency-modulated continuous-wave radar, phase-modulated spectrum-spread radar, or impulse radar. The hardware abstraction module 228 can also normalize the complex data from radar signals having different center frequencies, bandwidths, transmission power levels, or pulse widths.

[0046] In addition, the hardware abstraction module 228 matches the complex data generated using different hardware architectures. Different hardware architectures can include different antenna arrays 214 positioned on different surfaces of the smart device 104, or different sets of antenna elements within the antenna array 214. By using the hardware abstraction module 228, downstream modules can process complex data generated by different sets of antenna elements having different gains, different sets of various numbers of antenna elements, or different sets of antenna elements having different antenna element spacings. Further, the hardware abstraction module 228 enables those downstream modules to operate in the radar system 102 with different limitations that affect the available radar modulation schemes, transmission parameters, or types of hardware architectures.

[0047] The object tracker 230 identifies objects within the external environment and compiles a history of the behavior of each object. In particular, the object tracker 230 can compile information regarding the position of the object (e.g., distance and / or angle), movement (e.g., range rate and / or velocity), physical characteristics (e.g., size, radar cross section, and / or material composition), or combinations thereof. The object tracker 230 can be implemented using a variety of different tracking algorithms, including those associated with alpha-beta trackers, Kalman filters, or multiple hypothesis trackers (MHT).

[0048] The detection module 232 can analyze the data provided by the object tracker 230 for one or more use cases such as gesture recognition, presence detection, collision avoidance, health management, etc. In the case of gesture recognition, the detection module 232 can recognize gestures and distinguish the movements associated with gestures and non-gestures. The detection module 232 is further described with respect to FIG. 10. In some embodiments, the detection module 232 is implemented as a state machine.

[0049] Although not explicitly stated, the system medium 220 or the computer-readable medium 204 can also include a sensor fusion module, a radar communication module, a data collection module, a data visualization tool, and / or a radar simulator. The sensor fusion module can combine data from the radar system 102 with data provided by an external sensor such as another radar system or another type of sensor (e.g., a camera).

[0050] With the radar communication module, the radar system 102 can communicate with another entity outside the smart device 104, such as by using the radar system of another smart device 104. When using direct communication, those radar systems are configured with each other to support bistatic radar sensing, utilize techniques such as triangulation to enable faster acquisition of objects of interest, and / or maintain tracking of objects of interest that become temporarily hidden but are detected by other radar systems. When using the radar communication module, the radar system can share information and improve performance in a wide variety of different environments and situations.

[0051] The data collection module can record data that is available to the system processor 218 for system development and integration. Exemplary data can include data provided by the transceiver 216, or data generated by the digital signal processing module 222, the radar data processing module 224, and / or the application programming interface engine 226. In some cases, the data collection module acquires the raw data provided by the transceiver 216 and does not perform additional functions to reduce computational costs.

[0052] The data visualization tool can provide a visual diagram of the data being collected using the data collection module. In this way, the data visualization tool can facilitate the developer of application 206 to view and understand the data generated by radar system 102. In an exemplary embodiment, the data visualization tool tags the data with labels that describe the environment or activity that the data contains. In another exemplary embodiment, the data visualization tool graphically shows a portion of the data generated by radar system 102, such as information representing a range-Doppler map or information representing the movement of an object. Other data visualization tools can illustrate the operational aspects of radar system 102, such as the field of view used by radar system 102, the radar transmission signal generated by radar system 102, or the antenna pattern of radar system 102.

[0053] The radar simulator can input virtual data into digital signal processing module 222, radar data processing module 224, and / or application programming interface engine 226. This virtual data can include data provided by an external source or simulation data generated by the radar simulator. In some embodiments, the radar simulator can generate simulation data by rendering a virtual environment with objects and / or clutter and analyzing the interaction of virtual radar waves within that virtual environment. The radar simulator can also inject noise to help algorithm developers and integration test engineers understand the impact that noise sources can have on the performance of radar system 102. By using the radar simulator, the developer of application 206 can determine the feasibility of using radar system 102 for a particular use case. The radar application programming interface 210 is further described with respect to FIG. 3-1.

[0054] FIG. 3-1 is a diagram showing an exemplary radar application programming interface 210. In some cases, the radar application programming interface 210 may include a plurality of application programming interfaces that enable the application 206 to communicate with different operating levels of the radar system 102. The plurality of application programming interfaces may represent multiple layers or levels of the radar application programming interface 210. As an example, the radar application programming interface 210 may include a use case application programming interface 302 (use case API 302), a machine learning (ML) and algorithm application programming interface 304 (ML and algorithm API 304), a digital signal processing (DSP) application programming interface 306 (DSP API 306), a hardware abstraction application programming interface 308 (hardware abstraction API 308), or some combination thereof.

[0055] The radar application programming interface 210 can be described with respect to a plurality of layers 310, each layer being associated with a particular operating level of the radar system 102 as further described with respect to FIG. 3-2. Generally, the upper layers of the radar application programming interface 210 are designed to enable engineers with less radar experience and knowledge to easily interact with higher operating levels of the radar system 102. The lower layers of the radar application programming interface 210 are designed to enable engineers with more radar experience and knowledge to control lower operating levels of the radar system 102.

[0056] The use case application programming interface 302 represents the first layer 310-1 (or upper layer) of the radar application programming interface 210. The use case application programming interface 302 enables non-radar technicians to interface with the radar system 102 without knowing the low-level details regarding the operation of the radar system 102 and / or the capabilities of the radar system 102, from a hardware or software perspective. The use case application programming interface 320 can standardize the interaction between the application 206 and the radar system 102 for a variety of use cases, including gesture recognition, presence detection, biometric recognition, navigation assistance, and / or health management.

[0057] The machine learning and algorithm application programming interface 304 represents the second layer 310-2 (or upper middle layer) of the radar application programming interface 210. Using the machine learning and algorithm application programming interface 304, a technician with some radar knowledge can build and train a machine learning model within the radar system 102. Additionally, the machine learning and algorithm application programming interface 304 can enable those technicians to customize the core algorithms that allow the radar system 102 to analyze high-level radar data (e.g., pre-processed radar data).

[0058] The digital signal processing application programming interface 306 represents the third layer 310-3 (e.g., the lower middle layer) of the radar application programming interface 210. The digital signal processing application programming interface 306 can define a library of functions that the radar system 102 can execute to process raw data. These functions can perform operations such as spectrogram generation, Doppler calculation, and execution of digital beamforming. Using the digital signal processing application programming interface 306, a technician with some radar knowledge can customize the digital signal processing performed by the radar system 102.

[0059] The hardware abstraction application programming interface 308 represents the fourth layer 310-4 (e.g., the lower layer) of the radar application programming interface 308. The hardware abstraction application programming interface 308 can summarize the details of the hardware implementation of the radar system 102 and enable a standardized output independent of the hardware of the radar system 102. Using the hardware abstraction application programming interface 308, a technician with radar knowledge can customize the operation and configuration of the hardware within the radar system 102, including the antenna array 214 and / or the transceiver 216.

[0060] In some cases, radar system 102 can operate according to one or more of a wide variety of available configurations 312. In some cases, these configurations 312 are derived from the limitations of radar system 102 and / or the limitations of smart device 104. Some limitations are considered fixed limitations that are unlikely to change over time. For example, fixed limitations can be based on the limitations of the hardware associated with radar system 102 (e.g., associated with antenna array 214, transceiver 216, and / or system processor 218). Exemplary fixed limitations include the available frequency band, bandwidth, transmission power level, antenna configuration, and / or duplex configuration of radar system 102. Other limitations are considered dynamic limitations that can change over time. Exemplary dynamic limitations include the amount of available power (e.g., the battery level of smart device 104), the amount of available memory (e.g., the size of system medium 220 or computer-readable medium 204), and / or the amount of processing power (e.g., the processing power of system processor 218 or computer processor 202).

[0061] The available configurations 312 represent the operating configurations of radar system 102 for the generation of radar signals, the transmission of radar signals, the reception of reflected radar signals, and / or the processing of reflected radar signals. In some cases, the available configurations 312 specify adjustable characteristics of the radar signals, such as the carrier frequency, bandwidth, radar waveform (e.g., type of modulation), and / or transmission power level. The available configurations 312 can also include the hardware configuration of radar system 102, the software configuration of radar system 102, the radar sensing performance metrics of radar system 102, the available resources of radar system 102 and / or smart device 104, or some combination thereof. The available configurations 312 can include at least one default configuration.

[0062] Exemplary hardware configurations include a single antenna for transmission and reception, an antenna configuration such as a phased array for transmission and / or reception, or MIMO operation. Another exemplary hardware configuration includes a duplex configuration such as a half-duplex configuration for implementing a pulsed Doppler radar or a full-duplex configuration for implementing a frequency-modulated continuous wave radar.

[0063] Exemplary software configurations may include a radar signal processing configuration. This radar signal processing configuration specifies signal processing techniques that can be used by the radar system 102 to describe explicit information about an object. Some radar signal processing configurations can be adjusted to reduce complexity and memory usage. For example, a first radar signal processing configuration performs a Fourier transform (e.g., a fast Fourier transform (FFT)) and uses a detection threshold algorithm. Using these techniques, the first radar signal processing configuration can detect an object and measure the distance to the object. Other radar signal processing configurations can be made more complex to reduce false detections and improve accuracy, and may use more memory. For example, a second radar signal processing configuration may include a clutter tracker for monitoring clutter, a tracking algorithm for improving the likelihood of object detection and measurement accuracy, and / or a digital beamformer for measuring one or more angles to an object.

[0064] In one aspect, the radar application programming interface 210 may enable an engineer to configure the radar system 102 according to one of the available configurations 312. In another aspect, the radar application programming interface 210 may enable an engineer to refer to the available configurations 312 and create a new custom configuration based on the available configurations 312.

[0065] During operation, application 206 sends request 314 to radar application programming interface 210. Request 314 can specify a particular operating configuration of radar system 102 or request specific information from radar system 102. Exemplary types of information can include available configurations 312, the performance capabilities of radar system 102, specific types of data being collected by radar system 102, or a list of error reports. Request 314 can apply to use case application programming interface 302, machine learning and algorithm application programming interface 304, digital signal processing application programming interface 306, and / or hardware abstraction application programming interface 308. Some requests 314 can reset or restart radar system 102.

[0066] Radar application programming interface 210 may map request 314 to a configuration 316 of radar system 102 (e.g., one of the available configurations 312). In some cases, radar application programming interface 210 can receive multiple requests 314 from application 206 or a wide variety of different applications. Radar system 102 can process those requests 314 together and determine an appropriate configuration 316 of radar system 102 that satisfies the multiple requests 314. In this way, radar application programming interface 210 enables radar system 102 to efficiently provide radar sensing for a wide variety of different applications 206 or use cases. Radar system 102 receives this configuration 316 and operates accordingly.

[0067] Radar system 102 transmits radar data 318 to radar application programming interface 210. The radar data 318 can indicate whether a particular gesture has been detected or whether the data collected by the radar system 102 (e.g., raw digital samples of received signals, or data representing a range-Doppler map) is included. In response to the receipt of the radar data 318, the radar application programming interface 210 can format the radar data 318 and provide a response 320 to the application 206. In some cases, the response 320 can include the radar data 318. The response 320 can also include an error report associated with a false positive. Using this information, the application 206 can make appropriate adjustments to the request 314 to reduce the false positive. Various application programming interfaces can be associated with different operating levels of the radar system 102, as further described with respect to FIG. 3-2.

[0068] FIG. 3-2 shows an exemplary relationship between multiple layers 310-1 to 310-4 of the radar application programming interface 210 and the operating levels 322-1 to 322-4 of the radar system 102. In the illustrated configuration, the radar system 102 is associated with multiple levels 322 (or operating levels). Generally, the upper levels 322 include modules that operate on higher-level radar data (e.g., preprocessed data). During reception, these modules can be executed towards the end of the reception process to generate the radar data 318 requested by the application 206. The lower levels 322 can include modules that operate on lower-level radar data (e.g., raw radar data). During reception, these modules can be executed towards the start of the reception process (or at least, before the modules associated with the upper levels 322). The lower levels 322 can also involve the hardware configuration of the radar system 102.

[0069] As shown in FIG. 3-2, the radar data processing module 224 is associated with a first level 322-1 (e.g., a first operating level) and a second level 322-2 (e.g., a second operating level) of the radar system 102. In particular, the detection module 232 is associated with the first level 322-1, and the object tracker 230 is associated with the second level 322-2. The digital signal processing module 222 is associated with a third level 322-3 (e.g., a third operating level) of the radar system 102. The hardware 324 of the radar system 102, including the antenna array 214 and the transceiver 216, is associated with a fourth level 322-4.

[0070] Each layer 310 of the radar application programming interface 210 can be associated with one or more levels 322 of the radar system 102. In particular, each layer 310 of the radar application programming interface 210 can communicate with the corresponding level 322 of the radar system 102 either directly or indirectly via the application programming interface engine 226. Through these communications, each layer 310 of the radar application programming interface 210 can manage the operating parameters and / or modules associated with the corresponding level 322 of the radar system 102. The radar application programming interface 210 can also extract information from the corresponding level 322 of the radar system 102 and / or customize the inputs and / or outputs of the modules associated with the corresponding level 322.

[0071] In this example, the use case application programming interface 302 is associated with the detection module 232. Using this relationship, the use case application programming interface 302 can customize the operation of the radar system 102 for gesture recognition, presence detection, biometric recognition, collision avoidance, health management, etc. In particular, the use case application programming interface 302 can specify different states of the detection module 232, as further described with respect to FIG. 10.

[0072] The machine learning and algorithm application programming interface 304 is associated with the object tracker 230. Using this relationship, the machine learning and algorithm application programming interface 304 can customize the operation of the object tracker 230 and can specify the output format of the object tracker 230.

[0073] The digital signal processing application programming interface 306 is associated with the digital signal processing module 222 (or the hardware abstraction module 228). Using this relationship, the digital signal processing application programming interface 306 can customize the digital signal processing operations of the radar system 102, which may include clutter removal, noise cancellation, detection threshold processing, Doppler filtering, interferometry, and / or digital beamforming. The digital signal processing application programming interface 306 can also update the rate by specifying features such as the integration time.

[0074] The Hardware Abstraction Application Programming Interface 308 is associated with the hardware 324. Using this relationship, the Hardware Abstraction Application Programming Interface 308 can adjust the operation of the antenna array 214 and / or the transceiver 216. For example, the Hardware Abstraction Application Programming Interface 308 can specify waveform parameters of the transmitted radar signal such as transmission power, frequency, bandwidth, type of modulation (e.g., frequency or phase modulation). The Hardware Abstraction Application Programming Interface 308 can also specify radiation pattern (e.g., direction of main lobe and / or beam width), pulse width, pulse repetition frequency (PRF) (e.g., inter-pulse period (IPP)), number of pulses, duty cycle, or polarization (e.g., horizontal polarization, vertical polarization, and / or circular polarization), etc.

[0075] In some cases, the Hardware Abstraction Application Programming Interface 308 enables an external party to specify higher-level operating modes of the radar system 102, such as detection range, and map it to one or more operating parameters such as transmission power. As another example, the Hardware Abstraction Application Programming Interface 308 enables a user to specify resolution (e.g., range, Doppler, and / or angle) and map it to one or more operating parameters (e.g., bandwidth, integration time, or number of active antenna elements). The operation of the Radar Application Programming Interface 120 and the radar system 102 for supporting gesture recognition is further described with respect to FIGS. 4-10.

[0076] FIG. 4 shows an exemplary request 314 that is generated by application 206 for gesture recognition and provided to radar application programming interface 210. In this example, radar application programming interface 210 may represent use case application programming interface 302. By radar application programming interface 210, application 206 can customize the operation of radar system 102 for gesture recognition. Thus, application 206 can control the types of gestures that radar system 102 can detect and the false positive rate of radar system 102 (e.g., the ratio at which radar system 102 incorrectly recognizes other movements as gestures).

[0077] For gesture recognition, application 206 sends request 314 to radar application programming interface 210. Request 314 includes a pattern recognition sequence 402 that specifies a sequence of the behavioral states of an object when the object makes a gesture. A gesture progresses through at least two states. Each state can describe one or more characteristics (or features) of the object making the gesture that can be measured or determined using radar sensing. For example, each state can describe the relative or absolute position of the object making the gesture, the movement associated with the object making the gesture (e.g., range rate, velocity, acceleration, or any derivatives thereof), the orientation or radar cross section of the object making the gesture, the composition of the object making the gesture, and / or some combination of these.

[0078] Each pattern recognition sequence 402 includes at least a start state that describes the start behavior or start feature of the gesture and an end state that describes the end behavior or end feature of the gesture. Depending on the complexity of the gesture, some pattern recognition sequences 402 may also include one or more intermediate states that describe one or more behaviors or features of the gesture that occur between the start state and the end state. Using the pattern recognition sequence 402, the application 206 can define the gesture in a way that differentiates it from other types of movements or gestures. In this way, the application 206 can control the sensitivity of the radar system 102 for detecting a specific gesture and the ability of the radar system 102 to ignore other movements not associated with that gesture.

[0079] By increasing the sensitivity, the radar system 102 can detect gestures performed by a wide variety of different users with different variations. For example, some users may perform gestures using different speeds or different degrees of subtlety or exaggeration. Increasing the sensitivity can also improve the ease of use for physically disabled users to perform gestures. However, increasing the sensitivity can also cause the radar system 102 to generate false positives. For example, the radar system 102 may incidentally detect a gesture based on a user performing other movements not related to the gesture, such as vacuuming, walking with the smart device 104, walking a dog adjacent to the smart device 104, folding clothes adjacent to the smart device 104, making the bed adjacent to the smart device 104, washing dishes adjacent to the smart device 104, or rearranging an object in proximity to the smart device 104. By defining the pattern recognition sequence 402, the application 206 can control the sensitivity level of the radar system 102 and manage the false positive rate of the radar system 102.

[0080] The pattern recognition sequence 402 includes states 404-1, 404-2, ··· 404-P, where P represents a positive integer. Each of the states 404-1 to 404-P defines the behavior of an object between parts of a gesture. The individual states 404-1 to 404-P can include at least one criterion 406 and can optionally pass one or more outputs 408. The criterion 406 can include one or more entry criteria, one or more exit criteria, or some combination thereof. The entry criteria represent the criteria for entering the states 404-1 to 404-P, and the exit criteria represent the criteria for exiting the states 404-1 to 404-P. Each criterion 406 can specify a threshold, a conditional comparison (e.g., greater than, less than, or equal to), a logical operation (e.g., AND, OR, or NOT), a range of valid values, or a delta change (e.g., displacement).

[0081] Exemplary criteria 406 include a position criterion 410, a motion criterion 412, a physical characteristic criterion 414, and a duration criterion 416. The position criterion 410 includes criteria associated with the position of the object executing the gesture. For example, the position criterion 410 can include a range criterion 418 (e.g., a distance criterion or a straight-line distance criterion) or an angle criterion 420 (e.g., an azimuth criterion and / or an elevation angle criterion).

[0082] The motion criterion 412 includes criteria associated with the motion of the object executing the gesture. For example, the motion criterion 412 can include a range rate criterion 422 or a velocity criterion 424. As another example, the motion criterion 412 can include an acceleration or jerk criterion (not shown). The range rate criterion 422 can be based on a Doppler-determined range rate or a range-determined range rate.

[0083] The physical characteristic criterion 414 includes criteria associated with the physical characteristics of the object performing the gesture. For example, the physical characteristic criterion 414 may include a radar cross section (RCS) criterion 426 (RCS criterion 426) that describes the behavior of the radar cross section of the object. Additionally or alternatively, the physical characteristic criterion 414 may include a material composition criterion 428 that describes the composition of the object (e.g., human tissue, metal, wood, or cloth). Although not shown, this physical characteristic criterion 414 may also include criteria associated with the size or dimensions of the object.

[0084] The duration criterion 416 specifies a duration during which at least one other criterion is satisfied. This duration may be specified with respect to a time interval during which the object is detected by the radar system 102 or the number of radar frames (e.g., instances).

[0085] The output 408 enables the current state 404 to pass information to the next state 404 within the pattern recognition sequence 402. Exemplary outputs 408 may include position information (e.g., range or angle), motion information (e.g., Doppler frequency, range rate, or velocity), or physical information (e.g., size, radar cross section, material composition). By receiving the output 408, the next state 404 can evaluate whether the behavior of the object remains relatively similar to the previous state or changes throughout the gesture.

[0086] Generally, the pattern recognition sequence 402 can include any number of states 404-1 to 404-P, whereby the pattern recognition sequence 402 can define simple or complex gestures. Simple gestures, such as a hand extension gesture, may have a smaller number of states 404 compared to more complex gestures, such as a reciprocating motion gesture or sign language. The criteria 406 for each state 404-1 to 404-P allow for strict or loose tolerances, which can affect the sensitivity and false positive rate of the radar system 102 for detecting gestures. An exemplary pattern recognition sequence 402 is further described with respect to various gestures in FIGS. 5-1 to 7-2.

[0087] FIG. 5-1 is a diagram showing an exemplary pattern recognition sequence 402-1 associated with an object 500 performing a reciprocating motion gesture 502. To perform this reciprocating motion gesture 502, the object 500 moves towards the smart device 104 and away from the smart device 104 at a substantially constant angle with respect to the radar system 102. In FIG. 5-1, the object 500 is shown as a human hand. However, other types of objects, including inanimate objects, can alternatively be used.

[0088] In this example, the pattern recognition sequence 402-1 includes states 404-1, 404-2, and 404-3. State 404-1 includes an angle criterion 420, a range rate criterion 422-1, and a duration criterion 416-1 that characterize the behavior of the object 500 when the object 500 approaches the smart device 104. State 404-2 includes an angle criterion 420 and a range rate criterion 422-2 that characterize the behavior of the object 500 when it decelerates at a short distance position from the smart device 104. State 404-3 includes an angle criterion 420, a range rate criterion 422-3, and a duration criterion 416-2 that characterize the behavior of the object 500 when the object 500 moves away from the smart device 104.

[0089] The criteria for states 404-1 to 404-3 can be similar to the case of the angular criterion 420, for example, and can be different for the case of the range rate criteria 422-1, 422-2, and 422-3. To reduce false positives, states 404-1 to 404-3 can include other criteria. For example, states 404-1 to 404-3 can include the radar cross-section criterion 426 for distinguishing between the reciprocating motion gesture 502 performed by a finger pointing in a direction perpendicular to the movement of the hand or another reciprocating motion gesture 502 performed with the fingers bent in a fist. As another example, states 404-1 to 404-3 can include the material composition criterion 428 for distinguishing between the reciprocating motion gesture 502 performed with the hand and the reciprocating motion gesture 502 performed with a stylus. The criteria 406 associated with states 404-1 to 404-3 are further described with respect to FIG. 5-2.

[0090] FIG. 5-2 is a diagram showing an exemplary behavior of an object 500 executing the reciprocating motion gesture 502. The graph 504 illustrates the range rate of the object 500 over the duration of the reciprocating motion gesture 502. The range rate can be determined based on the measured Doppler frequency of the object 500 or the change in the measured range (e.g., distance). In this case, the range rate changes from a negative value to a positive value. The graph 506 illustrates the angle of the object 500 over the duration of the reciprocating motion gesture 502. In this case, the angle remains relatively constant, which is a characteristic of the reciprocating motion gesture 502. For this example, the criteria 406 for states 404-1 to 404-3 are considered to represent exit criteria.

[0091] At time T0, the object 500 has a range rate smaller than the range rate reference 422-1 of state 404-1. The object 500 also has an angle that remains within the margin specified by the angle reference 420 of state 404-1. In this case, the object 500 can be at any angle as long as the angle does not change significantly (e.g., does not change more than the specified margin). The object 500 satisfies the range rate reference 422-1 and the angle reference 420 for the duration specified by the duration reference 416-1. Thereby, the first part of the reciprocating gesture 502 is characterized by state 404-1.

[0092] Between times T1 and T2, the range rate of the object 500 is between the thresholds specified by the range rate reference 422-2 of state 404-2. These thresholds enable the range rate reference 422-2 to capture signs of a change in the range rate (e.g., going from negative to positive). This indicates that the object 500 changes from the direction of traveling towards the smart device 104 to the direction of traveling away from the smart device 104. Also, the angle of the object 500 remains within the angle reference 420. Thus, the second part of the reciprocating gesture 502 is characterized by state 404-2.

[0093] At time T3, the range rate of the object 500 is greater than the range rate reference 422-3 of state 404-3. Also, the angle of the object 500 remains within the angle reference 420. The object satisfies the range rate reference 422-3 and the angle reference 420 for the duration specified by the duration reference 416-2. Thus, the third part of the reciprocating gesture 502 is characterized by state 404-3.

[0094] By recognizing an object that meets the criteria of states 404-1, 404-2, and 404-3 in sequence, the radar system 102 can recognize the reciprocating motion gesture 502, and the radar application programming interface 210 can send a response 320 indicating that the reciprocating motion gesture 502 has occurred to the application 206. The application 206 can perform an action in response to receiving the response 320. For example, the application 206 can stop a timer, click a button, activate a selected option, or pause a media.

[0095] Some applications 206 may add additional criteria 406 within one or more of the states 404-1 to 404-3, or may tighten the tolerance of the criteria 406 to reduce false positives. For example, the application 206 can add a range criterion 418 to state 404-2 to require that the object 500 be within the shortest distance from the smart device 104. As another example, the application 206 can add a radar cross-section criterion 426 or a material composition criterion 428 to states 404-1 to 404-3 to distinguish the reciprocating motion gesture 502 performed by the user from turning the bedsheet inside out when the user makes the bed next to the smart device 104. As yet another example, the range rate criteria 422-1 and 422-3 can be appropriately set to enable the detection of the reciprocating motion gesture 502, and detections associated with a sweeper having a lower speed range rate that cannot meet the range rate criterion 422-1 or 422-3 can be rejected.

[0096] FIG. 6-1 is a diagram showing an exemplary pattern recognition sequence 402-2 associated with an object 500 that executes a swipe gesture 602. To perform the swipe gesture 602, the object 500 moves across the smart device 104 along a relatively straight path. This straight path can be oriented along any direction for the full swipe gesture. Alternatively, this straight path can be oriented along a specific direction associated with a specific direction swipe gesture (e.g., a direction from top to bottom, from bottom to top, from left to right, or from right to left). In FIG. 6-1, the object 500 is shown as a human hand. However, other types of objects, including inanimate objects, can alternatively be used.

[0097] In this example, the pattern recognition sequence 402-2 includes states 404-1 and 404-2. State 404-1 includes an angle criterion 420-1, a speed criterion 424, and a duration criterion 416-1 that characterize the behavior of the object 500 when the object 500 starts to swipe across the smart device 104. State 404-2 includes an angle criterion 420-2, a speed criterion 424, and a duration criterion 416-2 that characterize the behavior of the object 500 when the object 500 continues to swipe across the smart device 104.

[0098] The criteria 406 across states 404-1 and 404-2 can be the same, such as in the case of the speed criterion 424, or different, such as in the case of the angle criteria 420-1 and 420-2. The criteria 406 associated with states 404-1 to 404-2 are further described with respect to FIG. 6-2.

[0099] FIG. 6-2 is a diagram showing an exemplary behavior of an object 500 that executes a swipe gesture 602. Graph 604 illustrates the speed of the object 500 over the duration of the swipe gesture 602. This speed can be determined based on the measured change in position of the object 500. In this case, the speed remains relatively constant. Graph 606 illustrates the angle of the object 500 over the duration of the swipe gesture 602. In this case, the angle changes over time, which is a characteristic of the swipe gesture 602. For this example, it is considered that the criteria 406 for states 404-1 and 404-2 represent exit criteria.

[0100] Between times T0 and T1, the speed of the object 500 exceeds the threshold specified by the speed criterion 424 of state 404-1. Also, the angle of the object 500 changes by the amount specified by the angle criterion 420-1. The object 500 meets the speed criterion 424 and the angle criterion 420-1 for the duration specified by the duration criterion 416-1. Thereby, the first part of the swipe gesture 602 is characterized by state 404-1.

[0101] Between times T1 and T2, the speed of the object 500 remains above the threshold specified by the speed criterion 424. Also, the angle of the object 500 changes by the amount specified by the angle criterion 420-2. The object 500 meets the speed criterion 424 and the angle criterion 420-1 for the duration specified by the duration criterion 416-2. Thereby, the second part of the swipe gesture 602 is characterized by state 404-2.

[0102] By recognizing an object that meets the criteria of states 404-1 and 404-2 in sequence, radar system 102 can recognize swipe gesture 602, and radar application programming interface 210 can send a response 320 indicating that swipe gesture 602 has occurred to application 206. Application 206 can perform an action in response to receiving response 320. For example, application 206 can scroll through content or play the next song in a playlist.

[0103] Other embodiments of pattern recognition sequence 402-2 may include an angular criterion 420 that supports a swipe gesture 602 along a specific direction. To recognize a left-to-right swipe gesture 602, the angular criterion 420-1 of state 404-1 can identify whether object 500 appears on the left side of smart device 104. Also, the angular criterion 420-2 of state 404-2 can identify whether object 500 moves to the right side of smart device 104.

[0104] Some applications 206 may add additional criteria 406 within one or more of states 404-1 and 404-2, or may tighten the tolerance range of criteria 406 to reduce false positives. For example, application 206 can add a material composition criterion 428 to states 404-1 and 404-2 to distinguish between a swipe gesture 602 performed by the user and the user moving an inanimate object (such as a cup or paper) across smart device 104.

[0105] FIG. 7-1 is a diagram showing an exemplary pattern recognition sequence 402-3 associated with an object 500 that executes a gesture 702 of extending a hand. To perform the gesture 702 of extending a hand, the object 500 moves along a relatively straight path toward the smart device 104. In this example, the pattern recognition sequence 402-3 includes states 404-1 and 404-2. State 404-1 includes a range criterion 418-1, a range rate criterion 422, and a duration criterion 416-1 that characterize the behavior of the object 500 when it starts to extend its hand toward the smart device 104. State 404-2 includes a range criterion 418-2, a range rate criterion 422, and a duration criterion 416-2 that characterize the behavior of the object 500 when it continues to extend its hand toward the smart device 104.

[0106] The criteria 406 across states 404-1 and 404-2 can be the same, such as in the case of the range rate criterion 422 for example, or can be different, such as in the case of the range criteria 418-1 and 418-2 for example. The criteria 406 associated with states 404-1 and 404-2 are further described with respect to FIG. 7-2.

[0107] FIG. 7-2 is a diagram showing an exemplary behavior of an object 500 that executes a gesture 702 of extending a hand. Graph 704 illustrates the absolute value of the range rate of the object 500 over the duration of the gesture 702 of extending a hand. The range rate can be determined based on the measured Doppler frequency of the object 500 or the change in the measured range (e.g., distance). In this case, the range rate remains relatively constant. Graph 706 illustrates the range (e.g., distance) of the object 500 over the duration of the gesture 702 of extending a hand. In this case, the range decreases over time, which is a characteristic of the gesture 702 of extending a hand. For this example, the criteria for states 404-1 and 404-2 are considered to represent exit criteria.

[0108] Between times T0 and T1, the absolute value of the range rate of object 500 exceeds the threshold value specified by the range rate criterion 422 of state 404-1. Also, the range of object 500 is less than range criterion 418-1. Object 500 satisfies range rate criterion 422 and range criterion 418-1 for the duration specified by duration criterion 416-1. Thereby, the first part of the gesture 702 of stretching the hand is characterized by state 404-1.

[0109] Between times T1 and T2, the absolute value of the range rate of object 500 exceeds the threshold value specified by the range rate criterion 422. Also, the range of object 500 is less than range criterion 418-2. Object 500 satisfies range rate criterion 422 and range criterion 418-2 for the duration specified by duration criterion 416-2. Thereby, the second part of the gesture 702 of stretching the hand is characterized by state 404-2.

[0110] By recognizing an object that satisfies the criteria 406 of states 404-1 and 404-2 in order, the radar system 102 can recognize the gesture 702 of stretching the hand, and the radar application programming interface 210 can send a response 320 indicating that the gesture 702 of stretching the hand has occurred to the application 206. The application 206 can perform an action in response to receiving the response 320. For example, the application 206 can activate a sensor, start face authentication, or turn on a display. The operation of the radar system 102 is further described with respect to FIGS. 8-10.

[0111] FIG. 8 is a diagram showing an exemplary antenna array 214 and an exemplary transceiver 216 of the radar system 102. In the illustrated configuration, the transceiver 216 includes a transmitter 802 and a receiver 804. The transmitter 802 includes at least one voltage controlled oscillator 806 and at least one power amplifier 808. The receiver 804 includes at least two receive channels 810-1 to 810-M, where M is a positive integer greater than 1. Each receive channel 810-1 to 810-M includes at least one low noise amplifier 812, at least one mixer 814, at least one filter 816, and at least one analog-to-digital converter 818. The antenna array 214 includes at least one transmit antenna element 820 and at least two receive antenna elements 822-1 to 822-M. The transmit antenna element 820 is coupled to the transmitter 802. The receive antenna elements 822-1 to 822-M are respectively coupled to the receive channels 810-1 to 810-M. Other embodiments of the radar system 102 may include a plurality of transmit antenna elements 820 and / or a single receive antenna element 822.

[0112] During transmission, the voltage controlled oscillator 806 generates a frequency modulated radar signal 824 at a radio frequency. The power amplifier 808 amplifies the frequency modulated radar signal 824 for transmission via the transmit antenna element 820. The transmitted frequency modulated radar signal 824 is represented by a radar transmission signal 826 that may include a plurality of chirps. As an example, the radar transmission signal 826 includes 16 chirps, and these chirps may be transmitted as continuous bursts or as pulses spaced in time. The duration of each chirp can be, for example, on the order of tens or thousands of microseconds (e.g., between approximately 30 microseconds (μs) and 5 milliseconds (ms)).

[0113] The individual frequencies of the chirp can increase or decrease over time. The radar system 102 can linearly increase and linearly decrease the frequency of the chirp over time using two ramps of a period (e.g., triangular frequency modulation). With these two ramps of a period, the radar system 102 can measure the Doppler frequency shift caused by the movement of the object 500. Generally, the transmission characteristics of the chirp (e.g., bandwidth, center frequency, duration, and transmission power) can be adjusted to achieve a specific detection range, range resolution, or Doppler sensitivity for detecting one or more characteristics of the object 500. In some cases, the characteristics of the chirp are adjusted by the application 206 using the radar application programming interface 210. For example, the radar application programming interface engine 226 can optionally configure the transmitter 802 to generate a specific type of radar signal according to the radar application programming interface 210.

[0114] During reception, each receive antenna element 822-1 to 822-M receives radar receive signals 828-1 to 828-M representing delayed versions of the radar transmit signal 826. The amount of delay is proportional to the distance between the antenna array 214 of the radar system 102 and the object 500. In particular, this delay represents the sum of the time required for the radar transmit signal 826 to propagate from the radar system 102 to the object 500 and the time required for the radar receive signals 828-1 to 828-M to propagate from the object 500 to the radar system 102. Generally, the relative phase difference between the radar receive signals 828-1 to 828-M is due to the difference in the positions of the receive antenna elements 822-1 to 822-M. When the object 500 is moving, the radar receive signals 828-1 to 828-M are frequency-shifted with respect to the radar transmit signal 826 due to the Doppler effect. Similar to the radar transmit signal 826, the radar receive signals 828- to 828-M consist of one or more of the chirps.

[0115] Within each receiving channel 810-1 to 810-M, the low-noise amplifier 812 amplifies the radar received signal 828, and the mixer 814 mixes the amplified radar received signal 828 with the frequency-modulated radar signal 824. In particular, the mixer performs a beating operation to convert the radar received signal 828 to a low frequency and demodulate it in order to generate the beat signal 830.

[0116] The frequency of the beat signal 830 represents the frequency difference between the frequency-modulated radar signal 824 and the radar received signal 828, which is proportional to the straight-line distance (slant range). Although not shown, the beat signal 830 may include multiple frequencies and represents reflections from different objects or parts of objects within the external environment. In some cases, these different objects move at different speeds, move in different directions, or are positioned at different straight-line distances relative to the radar system 102.

[0117] The filter 816 filters the beat signal 830, and the analog-to-digital converter 818 digitizes the filtered beat signal 830. Each of the receiving channels 810-1 to 810-M generates digital beat signals 832-1 to 832-M, respectively, and the generated signals are supplied to the system processor 218 for processing. The receiving channels 810-1 to 810-M of the transceiver 216 are coupled to the system processor 218 as shown in FIG. 9.

[0118] FIG. 9 shows an exemplary operation of the hardware abstraction module 228, the object tracker 230, the detection module 232, the application programming interface engine 226, and the radar application programming interface 210. The system processor 218 is connected to the receive channels 810-1 through 810-M and can also communicate with the radar application programming interface 210. In the illustrated configuration, the system processor 218 implements the hardware abstraction module 228, the object tracker 230, the detection module 232, and the application programming interface engine 226. Although not shown, the hardware abstraction module 228, the object tracker 230, the detection module 232, and / or the application programming interface engine 226 can alternatively be implemented by the computer processor 202.

[0119] In some embodiments, the application programming interface engine 226 can handle communication between the system processor 218 and the radar application programming interface 210. For example, the application programming interface engine 226 can receive the pattern recognition sequence 402 from the radar application programming interface 210 and pass the pattern recognition sequence 402 to the detection module 232 or configure the detection module 232 according to the pattern recognition sequence 402. The application programming interface engine 226 can also pass the radar data 318 from the detection module 232 to the radar application programming interface 210. In some cases, the application programming interface engine 226 formats the radar data 318 in the manner specified by the radar application programming interface 210. In other embodiments, the radar application programming interface 210 and the detection module 232 can communicate without an interface connection through the application programming interface engine 226. In this case, the radar application programming interface 210 passes the pattern recognition sequence 402 to the detection module 232, and the detection module 232 passes the radar data 318 to the radar application programming interface 210.

[0120] In this example, the hardware abstraction module 228 receives digital beat signals 832-1 through 832-M from receive channels 810-1 through 810-M. The digital beat signals 832-1 through 832-M represent raw or unprocessed complex data. The hardware abstraction module 228 performs one or more operations to generate complex radar data 900 based on the digital beat signals 832-1 through 832-M. The complex radar data 900 includes both magnitude and phase information (e.g., in-phase and quadrature phase components). In some embodiments, the complex radar data 900 includes information representing a range-Doppler map for each receive channel 810-1 through 810-M. In other embodiments, the complex radar data 900 includes angle information. This angle information may be implicit within the complex radar data 900, such as within a plurality of range-Doppler maps. Alternatively, the system processor 218 or the object tracker 230 performs digital beamforming to explicitly provide angle information in a form such as information representing a four-dimensional range-Doppler-azimuth-elevation map. Other forms of complex radar data 900 are possible. For example, the complex radar data 900 may include complex interferometry data for each receive channel 810-1 through 810-M. This complex interferometry data is an orthogonal representation of the range-Doppler map. In yet another example, the complex radar data 900 includes a frequency domain representation of the digital beat signals 832-1 through 832-M for an active radar frame. Sometimes, the complex radar data 900 may include a combination of any of the above examples. For example, the complex radar data 900 may include magnitude information associated with a range-Doppler map and complex interferometry data. Although not shown, other embodiments of the radar system 102 may directly provide the digital beat signals 832-1 through 832-M to the application programming interface engine 226.

[0121] The object tracker 230 analyzes the complex radar data 900 and identifies individual objects within the complex radar data 900. Over time, the object tracker 230 compiles information about those objects, including position information, motion information, physical characteristic information, or some combination thereof. In this example, the object tracker 230 generates radar timelines 902-1 through 902-O for objects 500-1 through 500-O, where O represents a positive integer. In some cases, the object tracker 230 generates a single radar timeline 902 associated with the closest moving object 500. This can accelerate the execution of the application programming interface engine 226.

[0122] The detection module 232 receives the radar timelines 902-1 through 902-O from the object tracker 230 and receives the pattern recognition sequence 402 from the radar application programming interface 210. The detection module 232 recognizes the pattern recognition sequence 402 within at least one of the radar timelines 902-1 through 902-O and transmits the radar data 318 to the radar application programming interface 210. Based on the radar data 318, the radar application programming interface 210 generates a response 320, which is provided to the application 206. The operation of the detection module 232 is further described with respect to FIG. 10.

[0123] FIG. 10 is a diagram showing an exemplary scheme implemented by the detection module 232. For gesture recognition, the detection module 232 implements a state machine based on states 404-1 to 404-P specified by the pattern recognition sequence 402. The detection module 232 receives a radar timeline 902 associated with an object 500. The radar timeline 902 includes a list of events 1000-1 to 1000-A, where A represents a positive integer. Each event 1000 may include a timestamp 1002, position information 1004, motion information 1006, physical characteristic information 1008, or some combination thereof. The position information 1004, motion information 1006, and physical characteristic information 1008 may be based on measurement data, smoothed data, and / or predicted data.

[0124] When the criteria 406 for each state are satisfied by the radar timeline 902, the detection module 232 transitions through states 404-1 to 404-P. In response to the completion state 404-P, the detection module 232 sends the radar data 318 to the radar application programming interface 210 to cause the radar application programming interface 210 to send its response 320 to the application 206. This process is executable for a plurality of radar timelines 902-1 to 902-O. In some cases, these operations may be executed in parallel when the object tracker 230 compiles the radar timelines 902-1 to 902-O.

[0125] Depending on the situation, other movements not associated with gestures can be observed by the radar system 102. These other movements can include vacuuming, walking, walking a dog, folding clothes, making the bed, and / or rearranging objects. When these other movements are observed by the radar system 102, the detection module 232 can determine, based on the pattern recognition sequence 402, that these other movements are not associated with gestures. It is considered that states 404-1 to 404-P are defined to distinguish between gestures and those other movements. In particular, states 404-1 to 404-P describe one or more characteristics of gestures that are different from other movements. When other movements occur, the detection module 232 determines that the criteria 406 for at least one of the states 404-1 to 404-P are not met. In this way, the radar system 102 determines that the other movements are not associated with gestures. The radar system 102 does not necessarily need to differentiate between or identify each of these other movements. Optionally, some radar systems 102 can utilize an additional pattern recognition sequence 404 that describes the characteristics of these other movements. In this way, the radar system 102 can explicitly recognize one or more of these other movements.

[0126] At times, the radar system 102 can operate according to multiple modes or support multiple applications 206 during the same time interval. In this case, the system processor 218 can execute multiple instances of the detection module 232 that can operate in parallel. In this case, each instance of the detection module 232 can be configured by a corresponding application 206 that uses the radar application programming interface 210. For example, one instance of the detection module 232 can be configured for gesture recognition by a first application 206, and another instance of the detection module 232 can be configured for health management by a second application 206. Also, a third instance of the detection module 232 can be configured differently for gesture recognition by a third application 206. Exemplary method FIG. 11 is a diagram illustrating an exemplary method 1100 executed by the radar system 102 using the radar application programming interface 210. The method 1100 is shown as a set of operations (or actions) to be performed, but the operations are not necessarily limited to the order or combination shown herein. Further, any one or more of the operations can be repeated, combined, rearranged, or linked to provide various additional and / or alternative ways. In a portion of the following description, reference may be made to the entities shown in detail in the environment 100-1 to 100-6 of FIG. 1 and FIGS. 2-1, 2-2, 3-1, or 4, but those references are for illustrative purposes only. The present technology is not limited to performance by one or more entities operating on one device.

[0127] At 1102, a first request from an application of the smart device is received via a first layer of a plurality of layers of a radar application programming interface of the smart device. The first request includes a pattern recognition sequence that specifies a sequence of states associated with a gesture. The plurality of layers of the radar application programming interface are associated with different operating levels of the radar system. For example, as shown in FIG. 3-1, the radar system 102 receives a first request 314 from an application 206 of the smart device 104 via a first layer 310-1 of a radar application programming interface 210 of the smart device 104. The first request 314 includes a pattern recognition sequence 402 that specifies a sequence of states 404-1 to 404-P associated with a gesture, as shown in FIG. 4. Exemplary gestures may include a reciprocating motion gesture 502, a swipe gesture 602, an extending hand gesture 702, or any of the gestures described above with respect to FIG. 1. As shown in FIG. 3-2, the plurality of layers 310 of the radar application programming interface 210 are associated with different operating levels 322 of the radar system 102. The first layer 310-1 of the radar application programming interface 210 may correspond to a use case application programming interface 302.

[0128] At 1104, a second request from the application of the smart device is received via the second layer of the plurality of layers of the radar application programming interface. The second request includes the detection range of the radar system for detecting gestures. For example, the radar system 102 receives the second request 314 from the application 206 via the fourth layer 310-4 of the radar application programming interface 210. The second request 314 includes the detection range of the radar system 102 for detecting gestures. The fourth layer 310-4 of the radar application programming interface 210 may correspond to the hardware abstraction application programming interface 308.

[0129] At 1106, the hardware configuration of the radar system is modified based on the detection range. For example, the application programming interface engine 226 modifies the hardware configuration (e.g., the hardware 324) of the radar system 102 to achieve the specified detection range. In particular, the application programming interface engine 226 adjusts the transmission power, the number of pulses, the number of active antenna elements in the antenna array 214, and / or the beamforming pattern of the radar system 102 to achieve the specified detection range. The hardware configuration may include one of the available configurations 312.

[0130] At 1108, radar signals are transmitted and received using the hardware configuration. At least a portion of the radar signals is reflected by an object making a gesture. For example, as shown in FIG. 8, the radar system 102 transmits a radar transmission signal 826 and receives radar reception signals 828-1 to 828-M. At least a portion of the radar transmission signal 826 is reflected by an object 500 making a gesture. The object 500 may include inanimate objects such as the user's accessory device or a stylus.

[0131] At 1110, a gesture is recognized based on a pattern recognition sequence and a received radar signal. For example, radar system 102 recognizes a gesture based on pattern recognition sequence 402 and radar received signals 828-1 through 828-M. In particular, object tracker 230 compiles a radar timeline 902 of object 500. Radar timeline 902 describes the behavior of object 500 over time. This behavior may include position information 1004, motion information 1006, and / or physical characteristic information 1008. Detection module 232 recognizes pattern recognition sequence 402 within radar timeline 902 of object 500. In particular, detection module 232 implements a state machine and, in sequence, recognizes pattern recognition sequence 402 in response to radar timeline 902 that meets criteria 406 associated with states 404-1 through 404-P.

[0132] At 1112, a response is sent to the application via a radar application programming interface. This response indicates the occurrence of a gesture. For example, radar application programming interface 210 sends response 320 indicating the occurrence of a gesture to application 206. Exemplary Computing System FIG. 12 is a diagram showing various components of an exemplary computing system 1200 that can be implemented as any type of client, server, and / or computing device, as described with reference to FIGS. 2-1 and 2-2 above, to implement radar application programming interface 210.

[0133] Computing system 1200 includes a communication device 1202 that enables wired and / or wireless communication of device data 1204 (e.g., received data, data being received, data for which a broadcast is scheduled, or data packets of data). The communication device 1202 or the computing system 1200 may include one or more radar systems 102. The device data 1204 or other device content may include device configuration settings, media content stored on the device, and / or information associated with a user of the device. The media content stored in the computing system 1200 may include any kind of audio, video, and / or image data. The computing system 1200 includes one or more data inputs 1206 through which any kind of data, media content, and / or input, such as human speech, user-selectable input (explicit or implicit), messages, music, television media content, recorded video media content, or any other kind of audio, video, and / or image data received from any content and / or data source, can be received.

[0134] The computing system 1200 also includes a communication interface 1208 that may be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any kind of network interface, a modem, and any other kind of communication interface. The communication interface 1208 provides a connection and / or communication link between the computing system 1200 and a communication network, thereby enabling other electronic devices, computing devices, and communication devices to communicate data with the computing system 1200.

[0135] Computing system 1200 includes one or more processors 1210 (e.g., any of a microprocessor, a controller, etc.), and the processor processes various computer-executable instructions for controlling the operation of computing system 1200. Alternatively or additionally, computing system 1200 may be implemented with any one or combination of hardware, firmware, or fixed logic circuitry implemented in connection with processing and control circuitry 1212 shown generally. Although not shown, computing system 1200 may include a system bus or data transfer system that couples the various components within the device. The system bus may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a Universal Serial Bus, and / or a processor or local bus utilizing any of a variety of bus architectures.

[0136] Computing system 1200 also includes a computer-readable medium 1214, such as one or more memory devices that enable persistent and / or non-transitory data storage (i.e., symmetric to mere signal transmission), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of read only memory (ROM), flash memory, EPROM, EEPROM, etc.), and disk storage devices. The disk storage device may be implemented as any type of magnetic or optical storage device, such as a hard disk drive, a recordable and / or rewritable compact disk (CD), any type of digital versatile disk (DVD), etc. Computing system 1200 may also include a mass storage media device (storage media) 1216.

[0137] The computer-readable medium 1214 provides a data storage mechanism for storing device data 1204, as well as various device applications 1218 and any other kind of information and / or data related to the operational aspects of the computing system 1200. For example, the operating system 1220 is maintained as a computer application by the computer-readable medium 1214 and can be executed on the processor 1210. The device applications 1218 can include a device manager, such as any form of control application, software application, signal processing and control module, code specific to a particular device, and a hardware abstraction layer for a particular device.

[0138] The device applications 1218 also include any system components, engines, or managers for implementing the radar application programming interface 210. In this example, the device applications 1218 include the application 206 of FIG. 2-2, the radar application programming interface 210, and the application programming interface engine 226. End It should be understood that the techniques using the radar application programming interface and the devices including the radar application programming interface are described in a language specialized for features and / or methods, but the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary embodiments of the radar application programming interface.

[0139] Some examples are shown below. Example 1: A method executed by a radar system of a smart device, comprising Receiving a first request from an application of a smart device via a first layer of a plurality of layers of a radar application programming interface of the smart device. The first request includes a pattern recognition sequence specifying a sequence of states associated with a gesture, and the plurality of layers of the radar application programming interface are associated with different operating levels of the radar system.

[0140] The method further includes receiving a second request from an application of the smart device via a second layer of the plurality of layers of the radar application programming interface. The second request includes a detection range of the radar system for detecting a gesture.

[0141] The method includes modifying the hardware configuration of the radar system based on the detection range, and further includes transmitting and receiving radar signals using the hardware configuration. At least a portion of the radar signals is reflected by a moving object.

[0142] The method includes recognizing a gesture based on the pattern recognition sequence and the received radar signals, and in response to the recognition, transmitting a response indicating the occurrence of the gesture to the application via the radar application programming interface. and further includes.

[0143] Example 2: The method according to Example 1, where the different operating levels of the radar system are a first operating level associated with a radar data processing module of the radar system, and a second operating level associated with the operation of a transceiver of the radar system.

[0144] The first layer of the plurality of layers of the radar application programming interface is associated with a first operating level.

[0145] The second layer of the plurality of layers of the radar application programming interface is associated with a second operating level.

[0146] Example 3: The method described in Example 1, further comprising receiving a third request from an application of a smart device via a third layer of the plurality of layers of the radar application programming interface. The third request includes an implementation chain that specifies a sequence of modules associated with digital signal processing or data processing.

[0147] The method further comprises modifying the software configuration of the radar system based on the implementation chain.

[0148] Example 4: The method described in any of the preceding examples, wherein the sequence of states includes at least two states. The at least two states are a start state that describes at least one start feature of a gesture, and an end state that describes at least one end feature of a gesture.

[0149] The at least one start feature and the at least one end feature each describe a characteristic of an object that performs a gesture that can be determined using radar sensing.

[0150] Example 5: The method described in any of the preceding examples, wherein each state within the sequence of states includes at least one criterion associated with a gesture.

[0151] The sequence of states includes a first state. The first state includes a first criterion.

[0152] The first criterion includes at least one of an entry criterion for entering the first state or an exit criterion for exiting the first state.

[0153] Example 6: The method according to Example 5, wherein the entry criterion or the exit criterion is a position criterion, a motion criterion, a signal characteristic criterion, or at least one of a duration criterion.

[0154] Example 7: The method according to any of the preceding examples, wherein recognizing a gesture includes compiling a radar timeline of an object based on a received radar signal. The radar timeline describes the behavior of the object over time.

[0155] Recognizing a gesture further includes detecting a pattern recognition sequence within the radar timeline.

[0156] Example 8: The method according to Example 7, wherein the radar timeline includes a plurality of events. Each event includes a timestamp, position information regarding the object, motion information regarding the object, or at least one of physical characteristic information regarding the object.

[0157] Example 9: The method according to Example 7 or Example 8, wherein the gesture includes a reciprocating motion gesture.

[0158] Detecting the pattern recognition sequence includes detecting an object moving towards the radar system along a certain angle at an absolute value of a range rate exceeding a first threshold value for a first duration according to a first state in a sequence of states; and Detecting an object that changes direction from moving towards the radar system to moving away from the radar system according to a second state in the sequence of states; Detecting an object moving away from the radar system along the certain angle at an absolute value of a range rate exceeding a second threshold for a second duration according to a third state in the sequence of states.

[0159] Example 10: The method according to Example 7 or Example 8, wherein the gesture includes a swipe gesture.

[0160] Detecting a pattern recognition sequence includes detecting an object that changes an angle at a speed exceeding a threshold for a first duration according to a first state in the sequence of states; detecting an object that changes an angle along the same direction as the first state at a speed exceeding a threshold for a second duration according to a second state in the sequence of states; and

[0161] Example 11: The method according to Example 10, wherein the swipe gesture includes a directional swipe gesture.

[0162] Detecting an object that changes an angle further includes detecting an object that changes an angle along a direction associated with the directional swipe gesture.

[0163] Example 12: The method according to Example 7 or Example 8, wherein the gesture includes an arm - stretching gesture.

[0164] Detecting a pattern recognition sequence includes detecting an object that moves towards the radar system and has a range less than a first threshold for a first duration according to a first state in the sequence of states; Moving towards the radar system according to the second state in the sequence of states and detecting an object having a range less than a second threshold for a second duration including

[0165] Example 13: A method according to any of the preceding examples, The object makes other movements in addition to the gesture.

[0166] The method includes determining, based on a pattern recognition sequence, that the other movements are not associated with the gesture.

[0167] Example 14: A method according to Example 13, wherein the other movements are a person vacuuming, a person walking with a smart device, a person walking a dog next to a smart device, a person folding clothes next to a smart device, a person making a bed next to a smart device, or a person rearranging an object in proximity to a smart device associated with at least one of them.

[0168] Example 15: A method according to any of the preceding examples, Modifying the hardware configuration includes adjusting the transmission power, adjusting the number of pulses, adjusting the number of active antenna elements in the antenna array, and / or adjusting the beamforming pattern of the radar system to achieve the specified detection range.

[0169] Example 16: An apparatus comprising a radar system configured to execute any one of the methods of Examples 1 to 15.

[0170] Example 17: A computer-readable storage medium including instructions that, in response to execution by a processor, cause a radar system to execute any one of the methods of Examples 1 to 15.

Claims

1. A method performed by a radar system of a smart device, comprising: receiving, via a first layer of a plurality of layers of the radar application programming interface of the smart device, a first request from an application of the smart device, wherein the first request includes a pattern recognition sequence specifying a sequence of states associated with a gesture, and the plurality of layers of the radar application programming interface are associated with different operating levels of the radar system, the method further comprising: receiving, via a second layer of the plurality of layers of the radar application programming interface, a second request from the application of the smart device, wherein the second request includes a detection range of the radar system for detecting the gesture, the method further comprising: modifying the hardware configuration of the radar system based on the detection range; and transmitting and receiving radar signals using the hardware configuration, wherein at least a portion of the radar signals is reflected by an object performing the gesture, the method further comprising: recognizing, based on the pattern recognition sequence and the received radar signals, the gesture specified by the pattern recognition sequence included in the first request by the application; and transmitting, in response to the recognition, a response indicating the occurrence of the gesture to the application via the radar application programming interface. A method as further described above.

2. The different operating levels of the radar system include: a first operating level associated with a radar data processing module of the radar system; and a second operating level associated with a transceiver of the radar system, wherein the first layer of the plurality of layers of the radar application programming interface is associated with the first operating level, and the second layer of the plurality of layers of the radar application programming interface is associated with the second operating level. The method according to claim 1.

3. The method further includes receiving, via a third layer of the plurality of layers of the radar application programming interface, a third request from the application of the smart device, The third request includes an implementation chain that specifies a sequence of modules associated with digital signal processing or data processing, The method is The method according to claim 1, further comprising modifying the software configuration of the radar system based on the implementation chain. **Claim 4** The sequence of states includes at least two states, and the at least two states are A start state that describes at least one start feature of the gesture, and An end state that describes at least one end feature of the gesture, The method according to claim 1, wherein the at least one start feature and the at least one end feature each describe a characteristic of the object performing the gesture that can be determined using radar sensing. **Claim 5** Each state within the sequence of states includes at least one criterion associated with the gesture, The sequence of states includes a first state, The first state includes a first criterion, The first criterion is An entry criterion for entering the first state, or The method according to claim 1, including at least one of an exit criterion for exiting the first state. **Claim 6** The entry criterion or the exit criterion is A position criterion, A motion criterion, A signal characteristic criterion, or The method according to claim 5, including at least one of a duration criterion. **Claim 7** Recognizing the gesture includes Compiling a radar timeline of the object based on the received radar signal, the radar timeline describing the behavior of the object over time, The method according to claim 1, including detecting a pattern recognition sequence within the radar timeline. **Claim 8** The radar timeline includes a plurality of events, and each event is A timestamp, Position information regarding the object, Motion information regarding the object, or The method according to claim 7, including at least one of physical characteristic information regarding the object. **Claim 9** The gesture includes a reciprocating motion gesture, Detecting the pattern recognition sequence is ​ ​ Detecting the object moving towards the radar system along a certain angle at an absolute value of the range rate exceeding a first threshold for a first duration according to a first state of the sequence of the states; Detecting the object changing its direction from moving towards the radar system to moving away from the radar system according to a second state of the sequence of the states; Detecting the object moving away from the radar system along the certain angle at the absolute value of the range rate exceeding a second threshold for a second duration according to a third state of the sequence of the states comprising The method according to claim 7.

10. The gesture includes a swipe gesture, Detecting the pattern recognition sequence includes detecting the object that changes an angle at a speed exceeding a threshold for a first duration according to a first state of the sequence of the states; detecting the object that changes the angle along the same direction as the first state at the speed exceeding the threshold for a second duration according to a second state of the sequence of the states comprising The method according to claim 7.

11. The swipe gesture includes a directional swipe gesture, The method according to claim 10, wherein detecting the object that changes the angle further includes detecting the object that changes the angle along a direction associated with the directional swipe gesture.

12. The gesture includes an arm - stretching gesture, Detecting the pattern recognition sequence includes detecting the object that moves towards the radar system and has a range less than a first threshold for a first duration according to a first state of the sequence of the states; detecting the object that moves towards the radar system and has a range less than a second threshold for a second duration according to a second state of the sequence of the states comprising The method according to claim 7.

13. The object makes other movements in addition to the gesture, The method according to claim 1, further comprising determining, based on the pattern recognition sequence, that the other movements are not associated with the gesture.

14. The other movements are a person doing housework The person walking with the smart device, The person walking a dog next to the smart device, The person folding clothes next to the smart device, The person making the bed next to the smart device, or The person rearranging an object in the vicinity of the smart device associated with at least one of The method according to claim 13.

15. Modifying the hardware configuration includes adjusting the transmission power, adjusting the number of pulses, adjusting the number of active antenna elements in the antenna array, and / or adjusting the beamforming pattern of the radar system to achieve the specified detection range, The method according to claim 1.

16. An apparatus comprising a radar system configured to execute any one of the methods according to any one of claims 1 to 15.

17. A program for causing the radar system to execute any one of the methods according to claims 1 to 15 in response to execution by a processor.

Images