Radar-Based Detection of Hand Gestures via a Communication Device
By reusing communication circuits in communication devices for radar-based gesture detection and combining machine learning algorithms to evaluate data, the problems of gesture detection uncertainty and response delay in handheld devices are solved, and efficient and reliable gesture recognition function is achieved.
Patent Information
- Application Number
- JP2024534167
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-12-09
- Publication Date
- 2025-04-28
- Estimated Expiration
- 2041-12-09
AI Technical Summary
When introducing gesture recognition function into handheld devices, the prior art faces problems such as uncertainty in gesture detection and response delays resulting in angle and distance changes, especially when the device size, power consumption and cost are limited, it is difficult to adapt effectively.
Reliable gesture recognition is achieved by reusing the communication circuits of the communication device, and receiving reflected signals by aggregating individual communication signal receivers, and evaluating measurement data in convolutional neural networks (CNNs) or other machine learning algorithms.
Improves the reliability and response speed of gesture detection, suitable for handheld devices with strict size, power consumption and cost limitations, while reducing the need for dedicated radar circuit expansion of the device.
Smart Images

Figure 0007669594000001 
Figure 0007669594000002 
Figure 0007669594000003
Abstract
Description
[Technical field]
[0001] A communications device and corresponding method of operation provides gesture recognition that uses gesture recognition algorithms to recognize defined gestures and reuses communications circuitry of the device for radar-based detection of hand gestures by a user of the device. [Background technology]
[0002] Incorporating gesture recognition capabilities into handheld devices presents several challenges. Such devices may be held or positioned at different angles, which changes the perspective and distance between the device sensor used for gesture recognition and the user's hand. The varying perspective and distance deteriorates the ability to detect gestures with the kind of reliability and responsiveness required for a satisfactory user experience. Other issues are more mundane, such as the difficulty of adapting gesture sensing systems to devices that have severe constraints on size, power consumption, and cost.
[0003] Known approaches to gesture recognition include the use of vision-based recognition or the use of radar-based recognition. While both techniques have attendant advantages, the use of either entails complications in the context of smaller handheld devices for at least some of the reasons noted above. For example, a common approach to radar-based gesture recognition in handheld communication devices involves the addition of dedicated radar circuitry. See Shahzad Ahmed, Karam Dad Kallu, Sarfaraz Ahmed, and Sung Ho Cho. "Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review," Remote Sens. 2021, 13(3), 527.
[0004] Achieving a suitable solution for hand gesture detection using radar requires the use of high signal frequencies, e.g., at or above 60 GHz. Detection distances are generally not far in the context of handheld devices. Furthermore, in a pulsed radar context, the short detection distances typical for handheld devices, e.g., on the order of 10 centimeters in some scenarios, require the use of short radar pulses, e.g., 100 picoseconds or less pulses. Complications involve the reception and processing of return reflections corresponding to the emitted radar pulses, as a result of the wide bandwidth required to receive such signals, and further due to the complexities associated with reliably recognizing hand gestures from the reflected signals. Summary of the Invention
[0005] A communications device and corresponding method of operation provides gesture recognition based on reusing communications circuitry of the device for radar-based detection of hand gestures by a user of the device. One advantageous aspect of the operation in one or more embodiments is the use of an aggregation of individual communications signal receivers of the device to receive reflected signals corresponding to radar pulses emitted from the device. Another advantageous aspect of the operation in one or more embodiments is the use of a gesture recognition algorithm, such as a convolutional neural network (CNN) or other machine learning (ML) algorithm, to evaluate measurement data obtained from the reflected signals for reliable recognition of a defined gesture.
[0006] An exemplary embodiment includes a method of operation by a communications device. A method step or action includes performing gesture detection by transmitting a plurality of radar pulses from an antenna of the communications device and receiving a reflected signal corresponding to a respective one of the radar pulses. Each reflected signal has a signal bandwidth greater than a receiver bandwidth of an individual communications signal receiver of the communications device. Each reflected signal is received using an aggregation of individual communications signal receivers, with each individual communications signal receiver in the aggregation tuned to receive signal components of the reflected signal that are within a respective frequency bin of the reflected signal and outputting a corresponding bin signal. Further method actions include the device generating measurement data by performing measurements on respective bin signals obtained for the plurality of radar pulses, evaluating the measurement data via a gesture recognition algorithm trained to detect a plurality of defined hand gestures, and in response to the gesture recognition algorithm detecting one of the plurality of defined hand gestures, initiating a corresponding control action at the communications device.
[0007] Another exemplary embodiment includes a communications device including a communications circuit and a processing circuit. The processing circuit is configured to control the communications circuit to transmit a plurality of radar pulses from an antenna of the communications device and to receive a reflected signal corresponding to a respective one of the radar pulses. Each reflected signal has a signal bandwidth greater than a receiver bandwidth of an individual communications signal receiver of the communications device. Each reflected signal is received using an aggregation of the individual communications signal receivers, with each individual communications signal receiver in the aggregation tuned to receive a signal component of the reflected signal that is within a respective frequency bin of the reflected signal and output a corresponding bin signal. The processing circuit is further configured to generate measurement data by performing measurements on respective bin signals obtained for the plurality of radar pulses, evaluate the measurement data via a gesture recognition algorithm trained to detect a plurality of defined hand gestures, and initiate a corresponding control action at the communications device in response to the gesture recognition algorithm detecting one of the plurality of defined hand gestures.
[0008] Of course, the present invention is not limited to the above features and advantages, and indeed those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a block diagram of an embodiment of a communication device configured for gesture recognition shown in the context of a wireless communication network. [Diagram 2] 2 is a block diagram of details for communication circuitry of a communication device, according to one embodiment. [Diagram 3] 1 is a plot of an example waveform for a reflected signal corresponding to an emitted radar pulse, with respect to spectrum and bandwidth, and an advantageous subdivision of the reflected signal into frequency bins, according to one embodiment; [Figure 4]FIG. 2 is a signal processing and data flow block diagram for reflected signal reception, processing, and evaluation by a communications device according to one embodiment. [Figure 5A] 1 is a diagram of a data structure containing measurement data obtained for I and Q components of a received reflected signal, according to one embodiment. [Figure 5B] 1 is a diagram of a data structure containing measurement data obtained for I and Q components of a received reflected signal, according to one embodiment. [Figure 6] FIG. 2 is a block diagram of an exemplary circuit configuration for implementation of a gesture recognition system in a communication device, according to one embodiment. [Figure 7] 4 is a logic flow diagram of a method of operation by a communications device to perform gestured recognition, according to one embodiment. [Figure 8] FIG. 2 is a block diagram of another embodiment of a communication device configured for gesture recognition. [Figure 9] FIG. 13 is a block diagram of yet another embodiment of a communication device configured for gesture recognition. [Figure 10] 4 is a logic flow diagram of a method of operation by a communications device to perform gesture recognition, according to another embodiment. [Figure 11] 1 is a diagram of example hand gestures, such as those that may be represented in a defined set of hand gestures that are recognizable by a communication device. [Figure 12] FIG. 2 is a block diagram of one embodiment of signal processing and data flow for processing measurement data obtained from received reflected signals for gesture recognition. [Figure 13] FIG. 2 is a block diagram of one embodiment of a communications signal receiver and corresponding circuitry that supports use of the receiver for receiving communications signals and reuse of the receiver for receiving reflected signals. [Figure 14] FIG. 2 is a block diagram of one embodiment of a circuit in a communications device for generating and transmitting radar pulses. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] 1 illustrates a communications device 10 according to one or more embodiments. Device 10 includes communications circuitry 12, processing circuitry 14, and storage 16. Processing circuitry 14 is configured to cooperate with communications circuitry 12 to transmit communications signals 18 and receive communications signals 20, such as those that may be exchanged between device 10 and a wireless communications network 22.
[0011] An exemplary network 22 includes a wireless communication network, such as a fifth generation (5G) New Radio (NR) network that operates according to specifications promulgated by the 3rd Generation Partnership Project (3GPP). The network 22 includes a Radio Access Network (RAN) 24 with one or more radio network nodes 26, such as base stations or other types of access points. A Core Network (CN) 28 includes one or more CN nodes 30 that provide authentication and mobility management functions for connecting the device 10 to the network 22, as well as communicatively coupling the device 10 to one or more external networks 32, such as the Internet.
[0012] The communications circuitry 12 includes radio frequency (RF) transceiver circuitry, e.g., circuitry configured to transmit RF signals generated from baseband information signals containing data to be transmitted, and configured to output baseband information signals containing data obtained from received RF signals. In this regard, the communications circuitry 12 may be understood as comprising analog RF circuitry along with at least some baseband processing circuitry and operating as a wireless modem. The processing circuitry 14, in turn, generates or otherwise provides baseband information signals for transmission via the communications circuitry 12, and processes or otherwise handles baseband information signals obtained from received RF signals.
[0013] In one or more embodiments, processing circuitry 14 comprises one or more microcontrollers or other digital processors specially adapted based on the execution of computer program instructions stored on a computer-readable medium, such as that provided by storage 16. Figure 1 suggests such an arrangement by illustrating storage 16 as storing one or more computer programs 32, shown in the figure as "CP(s)," as well as storing one or more items of configuration data 34, i.e., "CFG.data," in the figure.
[0014] A point of interest herein is the implementation of gesture recognition system 40 via processing circuitry 14 and communications circuitry 12. The reuse of communications circuitry 12 for gesture recognition, and certain aspects of that reuse, represents one of several advantages of the gesture recognition system, which allows a user of device 10 to interact with device 10 based on device 10 detecting and responding to hand gestures by the user. Another advantageous aspect is the implementation of gesture recognition algorithms that provide reliable recognition of a user's hand gestures from measurement data obtained from radar-based "imaging" of the user's hands.
[0015] 1 is a power supply 42. In at least one embodiment, power supply 42 is battery-based, which is suitable for implementation of device 10 as a handheld or otherwise mobile device, such as a smartphone or other personal computing or media device with communications capabilities.
[0016] 2 illustrates an exemplary embodiment of the communications circuitry 12 as including at least one communications signal transmitter 50 and multiple communications signal receivers 52. Each communications signal receiver 52 comprises a receiver chain or receive signal processing path, each path corresponding to a respective one of multiple antennas 54 of the device 10. The antennas 54 may be elements of an antenna array used for receive beamforming by the device 10. In receive beamforming, an incoming RF signal impinges on each of the antennas 54, and a signal for each antenna is fed to the antenna's respective signal processing path, i.e., to a respective communications signal receiver 52. Thus, the device 10 has multiple antenna-specific versions of the same received signal, where the multiple signals may each be weighted according to a beamforming vector and coherently combined. Beamforming may be performed in either the analog or digital domain.
[0017] The detection of the hand gesture is based on the device 10 emitting a radar pulse 60 and reusing multiple communication signal receivers 52 of the device 10 for receiving a reflected signal 62 corresponding to the emitted radar pulse 60. The transmission of the radar pulse 60 relies on the re-use of one or more communication signal transmitters 50 of the device 10 or involves the use of a dedicated pulse transmitter.
[0018] However, the transmitted radar pulses 60, and the corresponding reflected signals, have signal bandwidths that exceed the receiver bandwidths of the individual communication signal receivers 52. One advantageous aspect of the gesture detection operations disclosed herein is that device 10 compensates for that fact by tuning each one of the communication signal receivers 52 used for reflected signal reception to a different spectral segment, or frequency band, of the reflected signal 62. That is, for reflected signal reception, each communication signal receiver 52 is tuned for reception of reflected signal content in a respective segment of the spectrum nominally spanned by the reflected signal 62.
[0019] 3 shows an exemplary reflected signal 62 having a center frequency of 77 gigahertz (GHz) and a (main lobe) signal bandwidth 64 that spans the electromagnetic spectrum from about 73 GHz to about 81 GHz. The gesture recognition system 40 effectively divides the spectrum of the reflected signal 62 into a number of frequency bins 66, with each frequency bin 66 representing the receiver bandwidth of a respective one of the individual communication signal receivers 52 used in the aggregation to receive the reflected signal 62. Aggregating the communication signal receivers 52 in this manner involves adjusting each receiver 52 in the aggregation such that its receiver bandwidth 68 is located in a respective one of the frequency bins 66.
[0020] 4 illustrates an example signal processing or data flow in the context of using an aggregated receiver, i.e., an aggregation of individually tuned receivers, to receive a reflected signal 62 corresponding to a radar pulse 60 emitted by device 10. FIG. 4 illustrates the reflected signal 62 impinging on multiple antennas 70 of device 10, resulting in a signal 70 for each antenna. Receive processing of the signal 70 for each antenna may include, for example, filtering, amplification, downconversion, and digitization, resulting in a corresponding bin signal 72. Each bin signal 72 corresponds to a respective one of the communication signal receivers 52 used in the aggregation to receive the reflected signal 62. Thus, each bin signal 72 corresponds to a signal component of the reflected signal 62 that falls within a frequency bin for which the corresponding receiver 52 is tuned.
[0021] Circuitry in device 10 performs measurements on the bin signals 72, resulting in a corresponding signal measurement 74 for each bin signal 72. An exemplary measurement made on each reflected signal 62 is a signal power measurement reflecting the strength of the bin signal 72 at each measurement sample time within a sampling window referenced to the emission of the corresponding radar pulse 60.
[0022] The length of the sampling window corresponds to the detection range of device 10; i.e., the window defines the interval after emission of a radar pulse 60 during which device 10 monitors a corresponding reflected signal 62. Thus, a reflected signal 62 received in response to the emission of a given radar pulse 60 should be understood as potentially including multiple reflections, depending on the local scattering of the radar pulse 60. Another way to consider each reflected signal 62 is that each reflected signal 62 may be a composite of multiple return reflections, depending on the number, location, and nature of local objects illuminated by the radar pulse 60.
[0023] The signal measurements 74 provided by each receive processing path include, for example, signal strength measurements taken at a prescribed sampling rate. Thus, for each emitted radar pulse 60, the gesture detection system 40 obtains multiple measurement sets, one measurement set for each receive processing path, i.e., each measurement set for each frequency bin of the reflected signal 62 being measured, and each measurement set includes signal measurements taken for the reflected signal 62 within the portion of the overall signal spectrum occupied by the respective frequency bin. In gesture detection based on emitting a series of radar pulses 60, i.e., coherent bursts of radar pulses 60, and collecting a measurement set for each radar pulse 60, the gesture recognition system 40 accumulates measurement data 76 for the set of radar pulses 60. Thus, the measurement data 76 includes or is derived from signal measurements 74 made at prescribed sampling times for each frequency bin of each corresponding reflected signal 62 for all radar pulses 60 in a given set of radar pulses 60.
[0024] The measurement data evaluation process evaluates the measurement data 76 according to a gesture recognition algorithm (GRA) 78 and produces a detection output 80 at least in those instances in which the GRA 78 determines that the measurement data 76 is characteristic of a gesture within a defined set of gestures recognizable by the gesture recognition system 40.
[0025] 5A and 5B illustrate measurement data 76, according to one embodiment. The bin signals 72 provided by each communication signal receiver 52 for a given reflected signal 62 include an in-phase (I) and quadrature (Q) signal pair, and thus the signal measurements 74 obtained for each frequency bin of the reflected signal 62 include a joint set of measurements for the I and Q signal pairs that comprise the bin signal 72. Aggregating the per-bin measurements made for each radar pulse 60 on a set of radar pulses 60 results in a two-dimensional data set that can be understood as a type of “radar image.” FIG. 5A illustrates exemplary I component data 76A included in one such data set, and FIG. 5B illustrates exemplary Q component data 76B included in the same data set.
[0026] Thus, an exemplary format of the measurement data 76 as a radar image comprises a two-dimensional (2D) data set. One dimension or axis of the data set is defined by a transmission window during which multiple radar pulses 60 are emitted, e.g., this dimension is indexed by a pulse count: radar pulse 1, radar pulse 2, etc. The other dimension or axis of the data set is defined by a sampling window during which the communications device 10 receives a reflected signal 62 corresponding to one of the emitted radar pulses 60, i.e., this dimension is indexed by a sample count. The dimension indexed by the radar pulse count may be referred to as the “slow time” dimension, and the dimension indexed by the sample count may be referred to as the “fast time” dimension, or, more specifically, the “fast frequency bin” dimension, since a reflected signal sample is taken per bin.
[0027] Generally, each reflected signal 62 is represented as a set of signal measurements for each bin 66, i.e., a measurement at sample time 1, a measurement at sample time 2, etc., up to the maximum number of samples taken for each reflected signal. In other words, detecting a gesture involves emitting multiple radar pulses 60 and measuring a corresponding reflected signal 62 for each radar pulse 60. Each reflected signal 62 is represented by a sample value taken at each of multiple sample times. At each sample time, a sample value (measurement) is taken for each bin signal 72. Using the exemplary details shown in FIGS. 5A and 5B , measurement data 76 in one or more embodiments includes I and Q signal strength measurements obtained for a set of 50 radar pulses 60 based on 16 frequency bins 66 into which the bandwidth 68 of the reflected signal 62 is to be subdivided.
[0028] Combining the measurement data 76 as a radar image may provide advantageous detection processing in some embodiments, although other data configurations are contemplated for input to the GRA 78. In one example, the measurement data 76 may include bin-by-bin / pulse-by-pulse measurements that are continuously fed to the GRA 78, which reconstructs or combines the data as needed for gesture recognition processing.
[0029] At least one embodiment of the gesture recognition system 40 is based on the implementation of the GRA 78 as a machine learning based detector. A particular example involves the implementation of the GRA 78 as a convolutional neural network (CNN) trained to recognize individual gestures among a defined set of gestures. One or more particular embodiments of the GRA 78 as a CNN-based gesture detector rely on the formation of measurement data 76 in a radar image format, such as that shown in Figures 5A and 5B. In the two-dimensional (2D) "image" format, each "pixel" comprises a signal measurement (e.g., intensity) for a particular frequency bin 66 at a particular sampling time for a particular radar pulse 60 / reflected signal 62.
[0030] The CNN comprises an input layer configured to receive measurement data 76 as input data, an output layer that outputs values indicative of gestures recognized from the measurement data 76, and several hidden layers between the input layer and the output layer. The hidden layers include, for example, convolutional layers, pooling layers, and fully connected layers that cooperate to detect patterns in the input data. Extensive exemplary information on CNN theory, structure, and operation is found in Michelucci, Umberto, Advanced Applied Deep Learning: Convolutional Neural Networks and Object Detection, Apress 2019.
[0031] There may be a defined set of gestures that are recognizable by gesture recognition system 40, and detection output 80 shown in FIG. 4 indicates when one of those gestures is detected. Of course, there may be particular gestures that are available at any particular time, for example, when particular software executing on device 10 responds to only some gestures, e.g., a subset of the universe of gestures that are recognizable by gesture recognition system 40. Similarly, there may be some subset of gestures that are always available, e.g., those gestures that relate to operating system level control of device 10 by the user.
[0032] 4 continues with initiation of a control action by gesture recognition system 40. For example, gesture recognition system 40 sends a message or other signaling 82 identifying the particular gesture that has been recognized. Such a message may flow to a higher level software application executing on device 10 via an application programming interface (API) provided by the operating system or execution environment of device 10.
[0033] A "gesture recognition system 40" as understood in the context of this disclosure comprises a processing circuit and an RF circuit. A particular feature is that the RF circuit includes a communication signal receiver 52 of the device 10, temporarily reconfigured from communication use, for use in receiving and processing a reflected signal 62 for gesture detection. Further elements of the RF circuit in at least one embodiment include, for example, a control switch and a corresponding frequency multiplier, which are used to generate radar pulses 60 at a signal frequency that is a multiple of one or more frequencies used for the communication signals, and to process the corresponding reflected signal 62. For example, the device 10 transmits and / or receives communication signals in the 30 GHz range, but the radar pulses 60 have a frequency at some multiple of 30 GHz, e.g., 2 x 30 GHz = 60 GHz.
[0034] The gesture recognition system 40 also includes control circuitry required to control the reuse of the communication circuitry 12 for gesture recognition, e.g., to multiplex the communication circuitry 12 or at least the involved components of the communication circuitry 12 between communication and gesture recognition uses. The gesture recognition system 40 further includes signal processing, e.g., associated with making signal measurements or at least compiling measurement data 76, and processing circuitry used to implement the GRA 78. In at least one embodiment, some or all of such processing circuitry is shared. For example, the processing circuitry 14 shown in FIG. 1 comprises one or more digital signal processors (DSPs), microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other digital processing circuits that provide baseband processing of incoming and outgoing communication signals, such circuits also configured to provide gesture recognition processing and communication circuitry control. In other embodiments, the gesture recognition system 40, or at least the digital processing elements of the gesture recognition system 40, are embodied in a separate integrated circuit (IC) or other assembly that may include neural processing resources dedicated to implementing the GRA 78.
[0035] 6 illustrates an exemplary configuration of gesture recognition system 40, with the understanding that one or more of the illustrated elements of the configuration may be stand-alone or otherwise dedicated to gesture recognition or may be shared with other functions. Control circuitry 90, e.g., a microprocessor executing stored program instructions, controls gesture recognition operations, such as sending control signals to communications circuitry 12 to control whether communications circuitry 12 operates in a communications mode or a gesture recognition mode.
[0036] The control circuitry 90 may also provide direct control of the radar pulse emission, for example, the control circuitry 90 may directly control a radar pulse circuit, not shown, for the emission of radar pulses 60. Further elements include a measurement circuitry 92 that may perform the measurements used to obtain the per-bin / per-pulse signal measurements 74, or the communications circuitry 12 may perform those measurements, with the measurement circuitry 92 comprising buffering or queuing logic used to input the entire measurement data 76 to an evaluation circuitry 94 that implements the GRA 78.
[0037] Any one or more of the control circuitry 90, the measurement circuitry 92, and the evaluation circuitry 94 may be functionally implemented in a microprocessor or other digital processor forming all or part of the processing circuitry 14 shown in FIG. 1, and such circuitry may perform other functions, such as baseband signal processing of the communication signals. Thus, in at least one embodiment, the gesture recognition system 40 may be considered as added functionality integrated or subsumed into the processing circuitry 14 and the communication circuitry 12. For example, the communication circuitry 12 includes one or more additional circuit elements required to temporarily reuse the communication signal receiver 52 for receiving and processing the reflected signal 62, as compared to the communication signal 20, for setting the appropriate receiving frequency and aligning each one of the communication signal receivers 52 with a respective one of the frequency bins 66.
[0038] However, at least a portion of gesture recognition system 40 may comprise dedicated circuitry used for gesture recognition, and optionally one or more other machine learning (ML) or artificial intelligence (AI) functions, implemented in device 10. For example, processing circuitry 14 in one or more embodiments includes a graphics processing unit (GPU) or neural processor (NP) that implements gesture recognition algorithms 78.
[0039] Regardless of how gesture recognition system 40 is implemented, FIG. 7 illustrates one embodiment comprising a method 700 of operation by communication device 10. Method 700 includes: - transmitting a plurality of radar pulses 60 from the antenna 54 of the communication device 10 (block 702); - receiving a reflected signal 62 corresponding to each one of the radar pulses 60 (block 704), where each reflected signal 62 has a signal bandwidth 64 greater than a receiver bandwidth 68 of an individual communication signal receiver 52 of the device 10, where each reflected signal 62 is received using an aggregation of the individual communication signal receivers 52, where each individual communication signal receiver 52 in the aggregation is tuned to receive signal components of the reflected signal 62 that are within a respective frequency bin 66 of the reflected signal 62, and outputting a corresponding bin signal 72; - generating measurement data 76 by performing measurements on each bin signal 72 acquired for the radar pulse 60 (block 706); - evaluating the measurement data 76 through a gesture recognition algorithm 78 trained to detect a plurality of defined hand gestures (block 708); - initiating a corresponding control action in the device 10 in response to the gesture recognition algorithm 78 detecting one of a plurality of defined hand gestures (block 710); performing gesture detection by performing
[0040] Transmitting radar pulses 60 includes, for example, transmitting radar pulses 60 using a transmit frequency that is a multiple of the transmit frequency used by device 10 to transmit communication signals 18 and using the same multiple to set a mixer frequency used for frequency downconversion by each communication signal receiver 52. In one or more embodiments, the multiple is an integer multiple, such as "2," which means that the signal frequency of radar pulses 60 / reflected signals 62 is twice the signal frequency of communication signals 20 received by communication signal receiver 52 when radar pulses 60 / reflected signals 62 are used for communication signal reception rather than gesture recognition.
[0041] Correspondingly, to perform gesture detection, the method 700 according to such an embodiment includes activating a frequency multiplier circuit. When active, the frequency multiplier circuit multiplies a local oscillator (LO) signal according to a multiple. The LO signal controls the transmit and receive frequencies of the communication device 10. An exemplary configuration uses a communication signal frequency in the 30 GHz range and a radar signal frequency in the 60 GHz range. In an embodiment in which the communication signal frequency is the same as the radar signal frequency, the frequency multiplier circuit is omitted from the device 10 / gesture recognition system 40.
[0042] Tuning the individual communication signal receivers 52 for aggregated use in receiving reflected signals 62 includes, for example, incrementally offsetting the mixer frequencies used by each one of the individual communication signal receivers 52, the incrementally offset mixer frequencies corresponding to respective frequency bins 66. That is, each of the communication signal receivers 52 used in the aggregation for reflected signal reception is adjusted such that the receiver bandwidth 68 of that communication signal receiver 52 is aligned with the spectral frequencies spanned by each one of the frequency bins 66.
[0043] In one or more implementations of communications circuitry 12, for each received reflected signal 62, corresponding bin signals 72 each include an in-phase (I) signal and a quadrature (Q) signal. Signal measurements made on bin signals 72 include, for example, signal power measurements, and thus measurement data 76 includes a reflection strength map. Each value in the map indicates the reflected signal strength for a particular one of the radar pulses 60 in a particular frequency bin 66, at a particular receive sampling time for that particular radar pulse 60. In support of such operation, each communications signal receiver 52 includes, for example, an analog-to-digital converter (ADC) that samples an associated bin signal 72 at regular intervals, where the sampling is triggered by the emission of a radar pulse 60, and where the total number of samples taken defines a sampling window.
[0044] In one or more embodiments, generating the measurement data 76 includes forming a radar image as a two-dimensional data set having one dimension defined by a transmission window through which the multiple radar pulses 60 are transmitted and the other dimension defined by a sampling window used to receive each returned signal 62. The individual values in the two-dimensional data set may be understood as radar image "pixels." In at least one such embodiment, each radar image pixel is a signal strength measurement taken for a particular frequency bin 66 and for a particular radar pulse 60, at a particular sampling time within the sampling window, for the multiple radar pulses 60.
[0045] According to at least one embodiment, the gesture recognition algorithm 78 includes a convolutional neural network (CNN) trained using representative measurement data 76 characteristic of each of the defined hand gestures. Other machine learning implementations may be used for implementation of the gesture recognition algorithm 78. Regardless of the implementation details of the gesture recognition algorithm 78, there may be an index value or other identifier assigned to each hand gesture in the defined set of hand gestures to be recognized by the device 10, and the gesture recognition algorithm 78 may indicate detection of the defined hand gesture by outputting an index or identifier value assigned to the detected gesture.
[0046] 4 may be an index or identifier value that identifies a particular gesture detected by device 10 from among a predefined set of hand gestures. Regardless of the details of the instruction 80, initiating a corresponding control action in response to recognizing the hand gesture includes outputting a message or signaling 82 that provides the instruction 80 of the detected hand gesture for use by a software application running on a host processor of communication device 10.
[0047] In at least one embodiment, the method 700 includes performing gesture detection in a time-multiplexed configuration that uses the communication circuitry 12 of the communication device 10 during one or more first times to communicate and uses the communication circuitry 12 during one or more second times to perform gesture detection. The communication circuitry 12 involved in the reuse includes at least the individual communication signal receivers 52 operated in aggregation for receiving and processing the reflected signals 62. An example of a time-multiplexed operation of the communication circuitry 12 between gesture recognition and communication operations involves one or more software applications running on the device 10. The software application(s) uses one or more communication services that involve communicating with a remote device or server via the communication circuitry 12 and includes one or more gesture input functions where a control action is taken in response to detection of a particular hand gesture by a user of the device 10.
[0048] A corresponding example comprises a communications device 10 including communications circuitry 12 and processing circuitry 14 configured to control communications circuitry 12 to (a) transmit a plurality of radar pulses 60 from an antenna 54 of communications device 10, and (b) receive a reflected signal 62 corresponding to a respective one of the radar pulses 60, each reflected signal 62 having a signal bandwidth 64 greater than a receiver bandwidth 68 of an individual communications signal receiver 52 of communications device 10. Each reflected signal 62 is received using an aggregation of individual communications signal receivers 52. Each individual communications signal receiver 52 in the aggregation is tuned to receive signal components of the reflected signal 62 that are within a respective frequency bin of the reflected signal 62 and outputs a corresponding bin signal 72.
[0049] The processing circuitry 14 is further configured to (c) generate measurement data 76 by performing measurements on each bin signal 72 obtained for the multiple radar pulses 60, (d) evaluate the measurement data 76 via a gesture recognition algorithm 78 trained to detect a plurality of defined hand gestures, and (e) initiate a corresponding control action in the communication device 10 in response to the gesture recognition algorithm 78 detecting one of the plurality of defined hand gestures.
[0050] The radar pulses 60 have a signal frequency that is, for example, a multiple of the signal frequency used by the communication device 10 to transmit the communication signal 18. In such a case, the processing circuitry 14 uses the same multiple to set the mixer frequency used for frequency downconversion by the individual communication signal receivers 52. "Setting" the mixer frequency in this sense includes, in at least one embodiment, outputting a control signal or signals that switch or otherwise activate circuitry associated with the communication signal receiver 52 that changes the mixing frequency used by the communication signal receiver 52 to tune the communication signal receiver 52 to a receiving frequency that corresponds to a frequency bin 66 that logically subdivides the spectrum occupied by the reflected signal 62. Such circuitry may be considered to belong to the gesture recognition system 40.
[0051] As a particular example, in at least one embodiment, to perform gesture detection, processing circuitry 14 is configured to activate a frequency multiplier circuit that, when active, multiplies the LO signal according to a multiple, the LO signal controlling the transmit and receive frequencies of communication device 10. Such a configuration is advantageous in implementations where the signal frequency to be used for radar pulses 60 exceeds the signal frequency or frequency band used for communication signals. Correspondingly, in implementations of device 10 where the communication signal frequency and the targeted radar signal frequency are the same, the frequency multiplier circuit is not required.
[0052] However, the gesture recognition system 40 will still include circuitry used to create offset mixer frequencies for the aggregated individual communication signal receivers 52 for receiving and processing the reflected signal 62 to "set" the receiver bandwidth 68 of each such communication signal receiver 52 to a respective one of the frequency bins 66. That is, to subdivide the spectral band occupied by the reflected signal 62 into sub-bands and to use each one of the aggregated communication signal receivers 52 to receive signal components within the sub-bands, i.e., respective ones of the frequency bins 66, the individual communication signal receivers 52 are adjusted according to the incrementally offset mixer frequencies used by each one of the individual communication signal receivers 52, where the incrementally offset mixer frequencies correspond to respective frequency bins 66.
[0053] In at least one embodiment of device 10, for each reflected signal 62 received by device 10, corresponding bin signals 72 each include an in-phase I signal and a quadrature Q signal. Measurements made on bin signals 72 include signal power measurements as measurements 74 described above (e.g., I signal power measurements and Q signal power measurements). This approach results in measurement data 76 including a reflection strength map, with each value in the map indicating the reflected signal strength for a particular one of the radar pulses 60 and at a particular one of the frequency bins 66 at a particular receive sampling time.
[0054] Processing circuitry 14 in at least one embodiment is configured to generate measurement data 76 as a radar image including a two-dimensional data set having one dimension defined by a transmit window through which multiple radar pulses 60 are transmitted and the other dimension defined by a sampling window used to receive each returned signal 62. Each value in the two-dimensional data set is a radar image pixel, with each radar image pixel being a signal strength measurement taken for a particular frequency bin 66 and for a particular radar pulse 60, at a particular sampling time within the sampling window, for the multiple radar pulses 60.
[0055] The processing circuitry 14 in at least one embodiment is configured to control the communications device 10 to perform gesture detection in a time multiplexed configuration that uses the communications circuitry 12 of the communications device 10 during one or more first times to communicate and uses the communications circuitry 12 during one or more second times to perform gesture recognition. In response to recognizing a gesture, to initiate a corresponding control action, the processing circuitry 14 in one or more embodiments is configured to output a message or signaling 82 providing an indication 80 of a detected hand gesture for use by a software application running on a host processor of the communications device 10.
[0056] 8 shows details of another example of a communications device 10, with a wireless transceiver 100 representing an example of the communications circuitry 12 introduced in FIG. 1 and a baseband processor 102 representing an example of the processing circuitry 14 introduced in FIG. 1. According to this configuration, in addition to being configured for communications signal transmission and reception, the wireless transceiver 100 is configured for radar signal transmission and reflected signal reception, e.g., the wireless transceiver 100 includes circuitry for generating / emitting radar pulses 60 and the wireless transceiver 100 includes circuitry for operating multiple individual communications signal receivers 52 of the wireless transceiver 100 in an aggregated manner, described herein for receiving and processing reflected signals 62 corresponding to the emitted radar pulses 60.
[0057] The baseband processor 102 controls the wireless transceiver 100 for communication signal operations and for gesture detection operations. In addition to providing baseband processing for communication transmission and reception, the baseband processor 102 controls the time multiplexing (sharing) of the communication signal receiver 52 between communication signal use and gestured recognition use. In at least one embodiment, the baseband processor 102 performs gesture recognition in response to signaling from a host processor 104, which provides an execution environment 106, e.g., a runtime environment, for running one or more software applications 108. Such applications 108 may request gesture recognition and cause signaling to proceed from the host processor 104 to the baseband processor 102, or may otherwise receive gesture-related signaling, e.g., an indication of a recognized gesture, based on such signaling flowing from the baseband processor 102 to the host processor 104.
[0058] An exemplary implementation of the baseband processor 102 includes one or more microprocessors or other programmatically configured digital circuitry specifically adapted to perform the gesture recognition operations described herein based on the execution of a computer program stored on a computer-readable medium. The storage 16 shown in FIG. 8 and in FIG. 1 is an example of such a medium, where the storage 16 includes one or more types of memory or storage devices, such as, for example, SRAM, DRAM, flash, EEPROM, solid-state disk (SSD), or any mixture thereof. The storage 16 provides, for example, volatile, working memory for program execution and data processing, along with non-volatile memory for longer-term storage of computer programs and / or configuration data.
[0059] 8 also includes a user interface 110 that includes a touch screen, such as may be used to provide control input or display output data associated with the execution of the software application(s) 108. In at least one embodiment, device 10 uses gesture recognition as an alternative or additional means of user interaction.
[0060] 9 illustrates another exemplary implementation of processing elements associated with gesture recognition system 40, such as may be functionally realized by processing circuitry 14 shown in FIG. 1 or by baseband processor 102 shown in FIG. 7 based on execution of computer program instructions stored in memory, for example. Here, the processing elements may be referred to as “processing units” or “processing modules” or simply “units” or “modules” and comprise logic configured to perform control and processing operations attributed to gesture recognition system 40. The set of modules 900 includes a control module 902, a measurement module 904, and an evaluation module 906.
[0061] The control module 902 controls or otherwise initiates the transmission of a plurality of radar pulses 60 from the antenna 54 of the communication device 10 and the corresponding reception of the reflected signals 62 corresponding to respective ones of the radar pulses 60. The measurement module 904 generates the measurement data 76 by performing measurements on respective bin signals 72 obtained for the plurality of radar pulses 60, or at least by receiving and organizing such measurements from measurement circuitry implemented in the communication circuit 12 / wireless transceiver 100. The evaluation module 906 implements the gesture recognition algorithm 78, and the control module 902 initiates a corresponding control action in the communication device 10 in response to the gesture recognition algorithm 78 detecting one of a plurality of defined hand gestures.
[0062] With the above exemplary embodiments in mind, an exemplary communications device, such as a user equipment (UE) or other handheld electronic device, includes millimeter wave (mmW) communications circuitry. In particular, the device includes multiple mmW receivers, i.e., receive signal processing chains, such as for beamforming and / or improved signal reception, and some or all of such receivers are temporarily reused to receive reflected signals corresponding to radar pulses 60 emitted by device 10 to detect hand gestures by a user of device 10. Reusing may involve, for example, retuning the receiver such that the receiver is configured for reception of a signal frequency associated with the emitted radar pulses 60, if the communication signal frequency differs from the radar signal frequency. Such retuning may be done, for example, by multiplying an LO signal, possibly used by the receiver for communication signal reception.
[0063] The frequency of the offset LO for each receiver is set in such a way that the downconverted signal from each receiver is from a different frequency bin 66 of the received reflected signal 62. Thus, in aggregation, all frequency information of the received reflected signal 62 is captured by the multiple receivers used to receive that information. The downconverted signal output from each receiver, e.g., the bin signal provided by the receiver, includes, e.g., the I and Q components of the received reflected signal.
[0064] The downconverted I and Q signals from each receiver, representing respective frequency bins 66 of the received reflected signal 62, are integrated within the sampling time of the receiver's ADC. The corresponding power of the integrated I and Q signals is measured via the receiver's ADC. Such measurements form a data set, sometimes referred to as a radar image, as previously described herein. The images acquired for the defined set of hand gestures serve as training inputs for a gesture recognition algorithm based on machine learning (ML). For example, multiple radar images corresponding to each gesture in the defined set of gestures are used to train a CNN, which operates as the gesture recognition algorithm.
[0065] Aspects of such operation include the following: Using receivers in multi-antenna devices, e.g. in beamforming arrays; Tuning each receiver to a different frequency bin of the reflected signal by offsetting the LO frequency applied to each of the receivers; Measuring the I and Q signals output by each receiver by integrating over a sampling time for each receiver's ADC; Carrying out calibration for the path and frequency correction of the measurement data for the I and Q signals; Performing continuous measurements of each gesture movement using slow time data (successive pulses in multiple radar pulses); Producing low speed time vs. high speed frequency image data based on the measured I and Q data for each frequency bin; Use of ML such as CNN to recognize gestures, and Translating available fast-time / slow-time images for gesture detection into fast-frequency / slow-time images for CNN training, i.e., high-resolution radar sensing data in the time domain, characteristic for each gesture in a defined set of gestures, can be translated into the frequency domain as training data for the CNN, such that the CNN recognizes the fast-frequency / slow-time images acquired later by the device during live radar sensing.
[0066] Figure 10 illustrates an exemplary embodiment including a method 1000 implemented by a communications device, such as communications device 10 of Figure 1. The illustrated details assume that the radar signal frequency is twice the communications signal frequency, i.e., 2x, and thus "2xLO" designates an LO that is twice the frequency of the LO frequency used by the participating communications signal receiver 52 for downconversion of the communications signal.
[0067] The method 1000 includes transmitting (block 1002) a pulsed signal at a gestural radar frequency with a beamwidth using a single transmitter or multiple transmitters of the device 10. The method 1000 continues by the device 10 receiving (block 1004) a corresponding reflected signal via several antennas 54 of the device 10, each antenna path including a corresponding low noise amplifier (LNA).
[0068] Each receiver downconverts the portion of the reflected signal it receives, i.e., the portion of the reflected signal in the respective frequency bin for which it is tuned, based on the offset frequency used to set the mixing frequency used by the receiver for downconversion (block 1006). Method 1000 continues by each receiver filtering its bin signals with a low-pass filter to eliminate or at least attenuate unwanted signal components (block 1008) and digitizing the bin signals at a sampling frequency used for reflected signal processing, which may be the same or different from the sampling frequency used for communication signal processing (block 1010). Band-pass filters may be used in place of low-pass filters in one or more implementations, for example, depending on the system implementation and the need for clock noise suppression.
[0069] The method 1000 may also include performing alignment and calibration of the bin signals, i.e., the taken digital samples of the bin signals, across multiple communication signal receivers 52 used to receive and process the reflected signals (block 1012). In this regard, it should be appreciated that the bin signals should be aligned in time such that the sampling is coherent across the bin signals.
[0070] Processing continues with the creation and processing of a radar image formed from the bin signal measurements (block 1014). This image may be referred to as a fast frequency versus slow time image to indicate that one dimension of the image refers to the fast sampling occurring across frequency bins for each received return signal (i.e., samples within a sampling window) versus slower time represented by the succession of radar pulse transmissions (i.e., the transmission window). Processing further continues with the use of ML for gesture identification (block 1016).
[0071] FIG. 11 illustrates an example of a prescribed hand gesture, which may be referred to as an "okay" gesture. FIG. 12 illustrates the processing and data configurations associated with recognizing such a gesture. In particular, FIG. 12 illustrates the use of a trained neural network (NN) kernel to recognize the gesture. The trained NN kernel includes, for example, a set of activations based on training data.
[0072] 13 shows example details for a communication signal receiver 52, where there are "N" such receivers, where N is a positive integer greater than 1. Each receiver 52 is associated with a respective antenna 54, which may be an antenna element in an array or panel used for beamforming reception by device 10.
[0073] The receive signal processing path or chain provided by each receiver 52 includes an LNA 120, a first mixer 122, a second mixer 124, a low pass filter 130, an ADC 126, and a calibration / matching circuit 128. FIG. 13 assumes that gesture detection uses a signal frequency that is twice the frequency of the communication signal frequency. Thus, the first mixer 122 is used in a 1xLO mode for communication signal processing and in a 2xLO mode for reflected signal processing, e.g., the reflected signal frequency is 2x the frequency of the communication signal frequency. The processing circuit 14 may output a 1x / 2x control signal to control whether the mixer 122 uses the 1x or 2x mixing frequency.
[0074] The second mixer 124 is active for reflected signal processing and can again be activated / deactivated via control signaling from the processing circuitry 14. The second mixer 124 adds an offset frequency that tunes the receiver 52 to the respective frequency bin 66 of the spectrum occupied by the reflected signal 62. See FIG. 3. A low pass filter 130 before the ADC 126 controls the input frequency to the ADC 126, i.e. unwanted frequencies are suppressed to prevent aliasing problems. The ADC 126 produces digitized samples that make up the digitized bin signal 72, and the calibration / matching circuitry 128 provides time alignment of the signals / samples across the receiver path. The figure shows that the matched / digitized bin signal 72 is fed to the processing circuitry 14.
[0075] The LO 136 operates in 1x or 2x mode in response to a control signal and feeds the first mixer 122. Similarly, a set of offset LOs 138 provide respective offset frequency signals to corresponding second mixers 124 for frequency bin offsetting. The LOs 138 are active or inactive in response to activation control signaling.
[0076] FIG. 14 illustrates one embodiment of a communication device 10 with respect to radar pulse generation and emission. In an exemplary configuration, the communication device 10 includes a dedicated radar pulse transmitter 140 responsive to generator control signaling, which in one or more embodiments is provided by the processing circuitry 14. A switch 142 controls whether the antenna 54 of the device 10 is coupled to the radar pulse transmitter 140 or to one of the communication signal transmitters 50 of the device 10. The switch control signaling output from the processing circuitry 14 controls the switch 142 to switch between a communication mode and a gesture recognition mode, in one or more embodiments. The switch 142 may also provide for switching the antenna 54 from a receive mode and a transmit mode for half-duplex operation in the communication mode, or one or more additional switches not shown may provide such switching.
[0077] In particular, modifications and other embodiments of the disclosed invention(s) will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. It is therefore to be understood that the invention(s) is not to be limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of this disclosure. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.
Claims
1. A method (700) of operation by a communications device (10), said method (700) comprising: transmitting (702) a plurality of radar pulses (60) from an antenna (54) of the communication device (10); receiving (704) a reflected signal (62) corresponding to each of the radar pulses (60); each reflected signal (62) having a signal bandwidth (64) greater than a receiver bandwidth (68) of an individual communication signal receiver (52) of the communication device (10), and each reflected signal (62) is received using an aggregation of the individual communication signal receivers (52); each individual communication signal receiver (52) in the aggregation is tuned to receive signal components of the reflected signal (62) that are within a respective frequency bin (66) of the reflected signal (62) and outputs a corresponding bin signal (72); Receiving (704) a reflected signal (62); generating measurement data by performing measurements on the respective bin signals acquired for the plurality of radar pulses; evaluating (708) the measurement data (76) through a gesture recognition algorithm (78) trained to detect a plurality of defined hand gestures; Initiating (710) a corresponding control action in the communication device (10) in response to the gesture recognition algorithm (78) detecting one of the plurality of defined hand gestures; 4. The method of claim 3, further comprising: performing gesture detection by:
2. 2. The method of claim 1, wherein transmitting the radar pulse includes transmitting the radar pulse using a signal frequency that is a multiple of a signal frequency used by the communications device to transmit a communications signal, the method including using the same multiple to set a mixer frequency used for frequency downconversion by the individual communications signal receivers.
3. 3. The method of claim 2, wherein the method includes activating a frequency multiplier circuit that, when active, multiplies a local oscillator (LO) signal according to the multiple to perform gesture detection, the LO signal controlling a transmit and receive frequency of the communication device.
4. 4. The method (700) of claim 1, wherein adjusting the individual communication signal receivers (52) includes incrementally offsetting a mixer frequency used by each one of the individual communication signal receivers (52), the incrementally offset mixer frequency corresponding to the respective frequency bin (66).
5. 5. The method of claim 1, wherein for each reflected signal, the corresponding bin signal (72) each includes an in-phase (I) signal and a quadrature (Q) signal, and the measurements made on the bin signals (72) include signal power measurements (74), such that the measurement data (76) includes a reflection strength map, with each value in the map indicating a reflected signal strength for a particular one of the radar pulses (60) and at a particular one of the frequency bins (66) at a particular receive sampling time.
6. 6. The method (700) of claim 1, wherein generating the measurement data (76) includes forming a radar image as a two-dimensional data set having one dimension defined by a transmission window through which the multiple radar pulses (60) are transmitted and another dimension defined by a sampling window used to receive each returned signal (62).
7. 7. The method (700) of claim 6, wherein each value in the two-dimensional data set is a radar image pixel, each radar image pixel being a signal strength measurement taken for a particular frequency bin (66) and for a particular radar pulse (60) at a particular sampling time within the sampling window for the plurality of radar pulses (60).
8. 8. The method (700) of any one of claims 1 to 7, wherein initiating the corresponding control action comprises providing a message or signaling (82) including an indication (80) of the detected hand gesture for use by a software application running on a host processor of the communication device (10).
9. 9. The method (700) of any one of claims 1 to 8, wherein the gesture recognition algorithm (78) comprises a convolutional neural network (CNN) trained using representative measurement data (76) characteristic of each of the defined hand gestures.
10. 10. The method of claim 1, further comprising: performing the gesture detection in a time multiplexed configuration using a communication circuit (12) of the communication device (10) during one or more first times to communicate and using the communication circuit (12) during one or more second times to perform gesture recognition, the communication circuit (12) including the individual communication signal receivers (52).
11. A communication circuit (12); A processing circuit (14) A communication device (10) comprising: transmitting a plurality of radar pulses (60) from an antenna (54) of the communication device (10); receiving a reflected signal (62) corresponding to each of said radar pulses (60); each reflected signal (62) having a signal bandwidth (64) greater than a receiver bandwidth (68) of an individual communication signal receiver (52) of the communication device (10), and each reflected signal (62) is received using an aggregation of the individual communication signal receivers (52); each individual communication signal receiver (52) in the aggregation is tuned to receive signal components of the reflected signal (62) that are within a respective frequency bin (66) of the reflected signal (62) and outputs a corresponding bin signal (72); receiving a reflected signal (62); configured to control the communication circuitry (12) to The processing circuit (14) generating measurement data (76) by performing measurements on the respective bin signals (72) acquired for the plurality of radar pulses (60); evaluating the measurement data (76) through a gesture recognition algorithm (78) trained to detect a plurality of defined hand gestures; initiating a corresponding control action in the communication device in response to the gesture recognition algorithm detecting one of the plurality of defined hand gestures; The communication device (10), further configured to:
12. 12. The communication device (10) of claim 11, wherein the radar pulses (60) have a signal frequency that is a multiple of a signal frequency used by the communication device (10) to transmit communication signals (18), and the processing circuitry (14) uses the same multiple to set mixer frequencies used for frequency downconversion by the individual communication signal receivers (52).
13. 13. The communication device (10) of claim 12, wherein the processing circuit (14) is configured to activate a frequency multiplier circuit that, when active, multiplies a local oscillator (LO) signal according to the multiple, the LO signal controlling a transmit and receive frequency of the communication device (10) to perform gesture detection.
14. 14. The communication device (10) of claim 11, wherein the individual communication signal receivers (52) are adjusted according to incrementally offset mixer frequencies used by respective ones of the individual communication signal receivers (52), the incrementally offset mixer frequencies corresponding to the respective frequency bins (66).
15. 15. The communications device of claim 11, wherein for each reflected signal, the corresponding bin signal (72) each includes an in-phase (I) signal and a quadrature (Q) signal, and the measurements made on the bin signals (72) include signal power measurements (74), such that the measurement data (76) includes a reflection strength map, with each value in the map indicating a reflected signal strength for a particular one of the radar pulses (60) and at a particular one of the frequency bins (66) at a particular receive sampling time.
16. 16. The communication device (10) of claim 11, wherein the processing circuitry (14) is configured to generate the measurement data (76) as a radar image comprising a two-dimensional data set having one dimension defined by a transmission window during which the plurality of radar pulses (60) are transmitted and another dimension defined by a sampling window used to receive each reflected signal (62).
17. 17. The communications device (10) of claim 16, wherein each value in the two-dimensional data set is a radar image pixel, each radar image pixel being a signal strength measurement (74) taken for a particular frequency bin (66) and for a particular radar pulse (60) at a particular sampling time within the sampling window for the plurality of radar pulses (60).
18. 18. The communication device (10) of claim 11, wherein the processing circuitry (14) is configured to output a message or signaling (82) providing an indication (80) of the detected hand gesture for use by a software application running on a host processor of the communication device (10) to initiate the corresponding control action.
19. 19. The communication device (10) of claim 11, wherein the gesture recognition algorithm (78) comprises a convolutional neural network (CNN) trained using representative measurement data (76) characteristic of each of the defined hand gestures.
20. 20. The communication device (10) of claim 11, wherein the processing circuitry (14) is configured to control the communication device (10) to perform gesture detection in a time multiplexed configuration using a communication circuitry (12) of the communication device (10) during one or more first times to communicate and using the communication circuitry (12) during one or more second times to perform gesture recognition, the communication circuitry (12) including the individual communication signal receivers (52).
21. 21. The communication device (10) of any one of claims 11 to 20, wherein the communication circuitry (12) is configured to transmit radar pulses (60) from two or more antennas (54) of the communication device (10) using one or more communication signal transmitters (50) or a dedicated radar pulse transmitter (140).
Citation Information
Patent Citations
Smartphone with radar system, system and method
JP2021504771A
Band Stitching Electronic Circuits and Techniques
US20140105256A1
Gesture recognition using 3D mm-wave radar
US20200097092A1
Methods for gesture recognition and control
US20200356178A1
Gesture detection in interspersed radar and network traffic signals
US20210034160A1