A method, system, and electronic device for recognizing spatial gestures.

Abstract

This invention provides a spatial gesture recognition method, system, and electronic device. The method includes: acquiring pulse signals transmitted by a second electronic device through an antenna array of a first electronic device; wherein the pose of the second electronic device changes with the gesture; calculating the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal to obtain a differential phase change sequence of the antenna pair; performing spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture. By extracting the in-phase and quadrature components of the pulse signal transmission between electronic devices and calculating the instantaneous phase difference between each antenna to form a phase difference sequence, the spatial gesture is further determined. Simultaneously, by obtaining a pure differential phase change sequence related only to the gesture movement through differential operations, the accuracy of gesture recognition is significantly improved.

Description

Technical Field

[0001] This application relates to the fields of wireless communication and intelligent sensing technology, and to a spatial gesture recognition method, recognition system, and electronic device. Background Technology

[0002] Ultra-wideband (UWB) technology has a wide range of applications due to its extremely high nanosecond-level time resolution and strong anti-multipath interference capability. It can be mainly used to determine the distance and pose between multiple electronic devices by measuring the transmitted pulse signals. For example, positioning can be achieved by measuring the time of flight (ToF) of the signal.

[0003] Ultra-wideband (UWB) technology primarily involves pulse signals, including Channel State Information (CSI) or Channel Impulse Response (CIR). Further utilizing these pulse signals transmitted between electronic devices to recognize pose and displacement changes (e.g., recognizing spatial gestures of a user wearing a wearable device by its displacement relative to an electronic screen) would be a highly promising low-cost, low-power, and privacy-preserving solution. However, accurately extracting and reconstructing signal changes related to spatial gestures from these pulse signals remains a significant technical challenge in the field of wireless sensing. Summary of the Invention

[0004] This invention provides a spatial gesture recognition method to solve the technical problems of high accuracy and low cost.

[0005] A first aspect of the present invention provides a method for recognizing spatial gestures, the method comprising: acquiring a pulse signal transmitted by a second electronic device through an antenna array of a first electronic device; wherein the pose of the second electronic device changes with the gesture; calculating the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal to obtain a differential phase change sequence of the antenna pair; performing spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture.

[0006] In one embodiment of the present invention, the differential phase change sequence of the antenna pair is obtained by calculating the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal. This includes: determining the original channel state information corresponding to the pulse signal of each antenna in the antenna pair at each sampling time; determining the channel impulse response of each original channel state information; obtaining the in-phase component and quadrature component of the channel impulse response; determining the absolute phase of each antenna based on the in-phase component and quadrature component; calculating the instantaneous phase difference between the antenna pairs; and obtaining the differential phase change sequence of the antenna pair for the detection time period corresponding to each sampling time.

[0007] In one embodiment of the present invention, obtaining the in-phase and quadrature components of the channel impulse response includes: obtaining multiple channel state information matrices corresponding to the channel impulse response; performing a fast inverse Fourier transform on each channel state information matrix to obtain multiple time-domain channel impulse responses; performing a modulus-square operation on the multiple time-domain channel impulse responses to obtain multiple single-sample power delay spectra; performing mean-square processing on the multiple single-sample power delay spectra to obtain a specified power delay spectrum, and extracting the corresponding in-phase and quadrature components based on the specified power delay spectrum.

[0008] In one embodiment of the present invention, the average processing of multiple single-sample power delay spectra to obtain a specified power delay spectrum further includes: performing sliding window processing on the specified power delay spectrum with a sliding window of a preset length, calculating the sum of power of multiple sampling points in each window, and determining the maximum sum of power; determining all sampling points in the sliding window corresponding to the maximum sum of power as target sampling points and determining all sampling points other than the target sampling points as noise sampling points; calculating the average power of all noise sampling points, and subtracting the average power from the power of the target sampling points respectively, thereby obtaining the denoised specified power delay spectrum.

[0009] In one embodiment of the present invention, spatial gesture matching is performed on differential phase change sequences to determine the gesture category of the spatial gesture corresponding to the pose change of the second electronic device relative to the first electronic device. This includes: performing sliding window processing on at least one differential phase change sequence, extracting time-domain features, frequency-domain features, and spatiotemporal features corresponding to the differential phase change sequence in each sliding window to form a feature vector, which is used to describe the pose change of the second electronic device relative to the first electronic device; and inputting the feature vector into a pre-trained gesture classification model to determine the gesture category and confidence level corresponding to the pose change.

[0010] In one embodiment of the present invention, the method further includes: when the confidence level of the gesture category is greater than a preset threshold and the gesture is identified as the same gesture category for more than a preset number of consecutive times, the gesture category is output.

[0011] In one embodiment of the present invention, the in-phase component and the quadrature component are determined by the following formula: ,in, The antenna index of the antenna array is represented. This represents the in-phase component of the i-th antenna at sampling time t. Indicates the first The orthogonal components of the antenna at sampling time t, where j is the imaginary unit. Complex form of the orthogonal components of in-phase components.

[0012] In one embodiment of the present invention, the absolute phase of the antenna is determined by the following formula: .

[0013] A second aspect of the present invention provides a spatial gesture recognition system, comprising: a data acquisition module, a phase difference calculation module, and a spatial gesture determination module, wherein the data acquisition module is used to acquire pulse signals transmitted by a second electronic device through an antenna array of a first electronic device; wherein the pose of the second electronic device changes with the gesture; the phase difference calculation module is used to calculate the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal, thereby obtaining a differential phase change sequence of the antenna pair; and the spatial gesture determination module is used to perform spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture.

[0014] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the spatial gesture recognition method of the first aspect.

[0015] The beneficial effects of this invention are as follows: By extracting the in-phase and quadrature components of pulse signal transmission between electronic devices and calculating the instantaneous phase difference between each antenna to form a phase difference sequence, spatial gestures can be further determined, resulting in a significant improvement in sensitivity compared to traditional ToF ranging methods. Simultaneously, differential operations automatically cancel common-mode errors such as transmitter phase noise, chip internal phase-locked loop jitter, temperature drift, and Gaussian white noise, obtaining a pure differential phase change sequence solely related to gesture movement, significantly improving the accuracy of gesture recognition. Furthermore, it eliminates the need for high-cost, high-power millimeter-wave radar and cameras, thereby achieving low-cost, low-power perception while improving privacy protection performance. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1 This is a schematic diagram of the hardware architecture of a spatial gesture recognition system provided in an embodiment of the present invention; Figure 2 This is a schematic flowchart of a spatial gesture recognition method provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of a method for recognizing spatial gestures performed by a first electronic device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the phase difference calculation method involved in the spatial gesture recognition method provided in one embodiment of the present invention; Figure 5 This is a schematic diagram of model training and application for spatial gesture recognition provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of a spatial gesture recognition system provided in one embodiment of the present invention; Figure 7 This is a structural diagram of an electronic device for a spatial gesture recognition method provided in one embodiment of the present invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0020] In the description of the embodiments of this application, in order to enable those skilled in the art to establish a clear and unambiguous technical understanding, the proprietary technical terms, technical names, and technical concepts involved in this application are first uniformly defined and their functional behavior is linked to. These definitions remain unique and consistent throughout the specific implementation.

[0021] Channel state information, specifically referring in this application to diagnostic data in the physical layer used to characterize the attributes of ultra-wideband communication links, is a complex matrix calculated by the baseband processing unit of the ultra-wideband transceiver configured in the electronic device after receiving the preamble of the physical layer protocol data unit (PPDU) using a channel estimation algorithm. Each element in this complex matrix corresponds to a specific subcarrier or delay unit, and its value reflects the amplitude attenuation and phase shift of the pulse signal (electromagnetic wave signal) on that path. If there are pose and / or displacement changes between the two electronic devices transmitting and receiving the pulse signal, it will cause dynamic fluctuations in the corresponding elements of the complex matrix.

[0022] In this application, channel impulse response specifically refers to the impulse response function of an ultra-wideband channel in the time domain. Physically, it is represented as a signal sequence over a time period, output by the ultra-wideband transceiver after correlation accumulation and demodulation of the received pulse signals. The horizontal axis of this signal sequence represents time, and the vertical axis represents complex amplitude (including the real part, in-phase component, and imaginary part).

[0023] The Power Delay Profile (PDP), in this application, specifically refers to the power spectrum used to describe the distribution of ultra-wideband signal energy in the time domain (delay axis). It is obtained by averaging the squared modulus of the time-domain channel impulse response (i.e., the sum of the squares of the real and imaginary parts) through multiple samples. Each spectral peak in the PDP corresponds to the energy of a scattering path. Its physical significance lies in transforming the complex complex channel impulse response into an intuitive energy-delay distribution, thereby facilitating the high-precision location of the boundaries between direct and scattering paths.

[0024] In this application, the in-phase / quadrature (I / Q) components specifically refer to the real part (in-phase component I) and imaginary part (quadrature component Q) of the complex channel impulse response obtained after quadrature demodulation. In hardware implementation, the high-frequency radio frequency signal is multiplied by the local in-phase carrier and quadrature carrier respectively by the quadrature mixer inside the ultra-wideband transceiver, and then output after low-pass filtering and analog-to-digital conversion. The I / Q components can completely and clearly describe the amplitude and phase changes of the pulse signal caused by pose and displacement changes between two electronic devices.

[0025] Ultra-wideband (UWB) technology offers extremely high temporal resolution due to its massive bandwidth (greater than 500MHz) and nanosecond-level pulse widths. However, UWB distance sensing solutions primarily rely on macroscopic characteristics such as the amplitude of the echo signal or time-of-flight (ToF). Since ToF distance resolution is limited by the physical bandwidth of the pulse, its limiting resolution is typically on the centimeter level. This makes it insensitive to changes in the electromagnetic field caused by minute pose and / or displacement changes between two electronic devices. For example, when a user performs spatial gestures (such as swiping up, down, left, or right), the ToF value hardly changes, making this solution unable to recognize fine spatial gestures. Furthermore, this solution requires continuous transmission of high-frequency pulses and wide-range distance sweeps, resulting in extremely high system power consumption, making it difficult to operate continuously in mobile terminals.

[0026] This application provides a spatial gesture recognition method, aiming to fundamentally solve the aforementioned technical defects. The method can be based on the phase of ultra-wideband channel state information and includes the following steps: acquiring the original channel state information generated when the ultra-wideband antenna array synchronously receives ultra-wideband pulse signals (which can be referred to as pulse signals transmitted by ultra-wideband tag devices), the original channel state information including the channel impulse response; reading the in-phase component and quadrature component of the direct path in the channel impulse response through the firmware register; calculating the instantaneous phase difference between the first antenna and the second antenna in the ultra-wideband antenna array for the same direct path based on the in-phase component and the quadrature component to eliminate common-mode error and obtain a differential phase change sequence; and extracting features from the differential phase change sequence in continuous time, and inputting the extracted feature vector into a preset classifier for gesture template matching to output the gesture category.

[0027] As can be seen, the core idea of ​​the spatial gesture recognition method provided in this application is as follows: Utilizing an existing ultra-wideband antenna array on an electronic device to synchronously receive pulse signals continuously transmitted by a UWB tag (such as a wristband, key, or base station), the original I / Q components of the direct path of the channel impulse response are directly read through the firmware register, and the instantaneous phase difference between the antennas is calculated. Since the direct path is completely separated from the scattering path in the time domain, interference from the scattering path can be completely eliminated. Simultaneously, through phase differential calculation between the antennas, common-mode errors such as transmitter phase noise, chip internal phase-locked loop jitter, temperature drift, and Gaussian white noise are canceled, thereby obtaining an extremely pure differential phase change sequence that is only related to gesture movement. Because phase changes are highly sensitive to sub-millimeter displacement, this application achieves sub-millimeter-level fine spatial gesture recognition without increasing hardware cost and power consumption, and completely avoids the privacy leakage risk caused by camera-assisted recognition.

[0028] In order to enable those skilled in the art to fully and accurately reproduce the technical solutions of this application, the various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0029] See Figure 1 The diagram illustrates a hardware architecture schematic of a spatial gesture recognition system 10 provided in an optional embodiment of this application. This system 10 is based on ultra-wideband channel state information phase and can be integrated into electronic devices (first electronic devices) such as smartphones, smart central control screens, and smart wearable devices.

[0030] Specifically, the system may include, in terms of hardware, an ultra-wideband antenna array 101 (UWB antenna array), an ultra-wideband transceiver 102 (UWB transceiver), a digital baseband processor 103, and an application processor 104.

[0031] An ultra-wideband antenna array 101 is used to synchronously and in real-time receive ultra-wideband pulse signals in space, the pulse signals originating from an ultra-wideband tag device (second electronic device). An ultra-wideband transceiver 102, connected to the ultra-wideband antenna array 101, is used to perform low-noise amplification, down-conversion mixing, and analog-to-digital conversion on the received pulse signals, outputting in-phase / quadrature component data of the channel impulse response. A digital baseband processor 103, connected to the ultra-wideband transceiver 102, is used to execute the spatial gesture recognition method of this embodiment, including: CSI extraction (pulse signal), phase difference calculation (instantaneous phase difference), temporal feature extraction (features corresponding to the differential phase change sequence), and outputting gesture categories through a gesture classifier. An application processor 104, connected to the digital baseband processor 103, is used to receive gesture categories and execute specific instruction outputs according to the spatial gesture mapping, i.e., execute corresponding control instructions.

[0032] refer to Figure 2 , Figure 2 The diagram illustrates the overall flow of a spatial gesture recognition method provided in an optional embodiment of this application, which specifically includes the following steps.

[0033] Step S201: Acquire the pulse signal sent by the second electronic device through the antenna array of the first electronic device; wherein, the pose of the second electronic device changes with the gesture; Step S202: Calculate the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal to obtain the differential phase change sequence of the antenna pair; Step S203: Perform spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture.

[0034] An optional embodiment of this application provides a schematic diagram of the overall process of a spatial gesture recognition method. This method is applied to a first electronic device (mobile phone), particularly the digital baseband processor 103 of the first electronic device, and specifically includes the following steps (the flowchart can be referred to). Figure 3 (Specific step numbers are not shown).

[0035] Step S301: Obtain the pulse signal transmitted by the ultra-wideband tag device synchronously received by the ultra-wideband antenna array 101, and determine the original channel state information, which includes the channel impulse response.

[0036] In this embodiment, the ultra-wideband tag device (which authenticates with and communicates with the electronic device, such as a smart bracelet, smartwatch, smart key, or other smart terminal worn by the user) continuously transmits a pulse signal conforming to the IEEE 802.15.4 standard at a preset frequency (e.g., 100Hz). An ultra-wideband antenna array 101 mounted on the first electronic device synchronously receives this pulse signal. An ultra-wideband transceiver 102 (e.g., a Qualcomm WCN7881 chip, or a compatible chip such as Qorvo or NXP) outputs a channel impulse response while performing signal synchronization and ranging. A digital baseband processor 103 acquires this channel impulse response in real time.

[0037] Step S302: Read the in-phase / quadrature components of the direct path in the channel impulse response through the firmware register.

[0038] In this embodiment, to accurately capture the subtle movements of the ultra-wideband tag device, compared to traditional ToF ranging data, the in-phase / quadrature components (i.e., raw I / Q data) of the direct path (i.e., the first path) in the channel impulse response are directly read from the firmware register. These in-phase / quadrature components are denoted in complex form: Where i represents Figure 1 The antenna index of the ultra-wideband antenna array 101 shown (for example, in this embodiment, i=1,2,3 can correspond to three antennas). This represents the in-phase component of the i-th antenna at sampling time t. Let j represent the orthogonal component of the i-th antenna at sampling time t, where j is the imaginary unit.

[0039] Step S303: Based on the in-phase / quadrature components, calculate the instantaneous phase difference between the first antenna and the second antenna in the ultra-wideband antenna array 101 for the same direct path, so as to eliminate common-mode error and obtain differential phase change sequence.

[0040] In this embodiment, the differential phase change sequence can be a sequence of instantaneous phase differences between antennas at each sampling moment within the detection time period. (See reference...) Figure 4 , Figure 4 The physical and mathematical principles of phase differential calculation are illustrated. First, the absolute phase of a single antenna is calculated based on the read in-phase / quadrature components: Furthermore, the instantaneous phase difference between the first antenna and the second antenna is calculated: Similarly, if the antenna array includes a third antenna, then the instantaneous phase difference between the first and third antennas is calculated: .

[0041] In terms of physical mechanism, ultra-wideband signals have extremely high center frequencies (e.g., Channel 9 has a center frequency of 7.9872 GHz and a wavelength of λ≈37.5 mm). When the signal propagates in space, its phase change and the change in propagation path length satisfy the following physical relationship: When a user performs a spatial gesture, it can cause a second electronic device to change its posture in space, interfering with the electromagnetic field. This results in a minute change of 1 mm in the propagation path length of the pulse signal, causing a significant fluctuation of approximately 9.6° in the instantaneous phase difference. As can be seen, this embodiment improves the sensitivity of spatial motion sensing by extracting the in-phase / quadrature components of the direct path and calculating the instantaneous phase difference, enabling precise capture of small hand movements.

[0042] In addition, the absolute phase measured by a single antenna The error is mixed with a large amount of hardware and environmental common-mode error, and its mathematical model can be expressed as:

[0043] in, The true phase caused by the gesture. The phase noise at the transmitter source end, This refers to the phase jitter of the phase-locked loop in the receiver chip. The phase trend caused by temperature drift. This represents channel thermal noise. Since transmitter noise, chip jitter, and temperature drift affect all antennas in the antenna array simultaneously and equally, the instantaneous phase difference between the antennas is calculated as follows: Mathematically, the common-mode error term is completely subtracted, leaving only pure differential phase change sequence containing differential phase information closely related to the multidimensional trajectory of the spatial gesture. Within the detection time period, the corresponding sampling times... The sequence can form a differential phase change sequence. It can be understood that the process described in step S303 can be run within the UWB chip.

[0044] Step S304: Extract features from the differential phase change sequence within a continuous time period, input the extracted features into a preset gesture classifier for gesture template matching, and output the gesture category.

[0045] In this embodiment, the differential phase change sequence is processed to extract multi-dimensional spatiotemporal features (e.g., differential phase information of multiple sliding window durations is extracted from the detection time period through sliding windows), and input into a preset gesture classifier. The gesture classifier uses forward inference to match the extracted features with preset gesture templates such as "swipe left", "swipe right", "press", and "rotate", and finally outputs the gesture category.

[0046] In some optional embodiments, the physical topology and operating frequency band of the ultra-wideband antenna array 101 are optimized to maximize the sensitivity of phase changes to subtle hand movements and ensure the uniqueness of angle measurements. Specifically, the ultra-wideband antenna array 101 consists of a one-dimensional uniform linear array of 3 to 4 antennas. The physical spacing between adjacent antennas is strictly designed to be λ / 2, where λ is the wavelength of the pulse signal propagating in air. The operating frequency band of the ultra-wideband antenna array 101 is selected as Channel 9 in the ultra-wideband standard, with a center frequency of 7.9872 GHz. The corresponding electromagnetic wave wavelength λ is approximately 37.5 mm, therefore the center distance between adjacent antennas is designed to be 18.75 mm.

[0047] In contrast, if the antenna spacing is greater than λ / 2, spatial aliasing will occur during signal angle of arrival calculation, leading to multi-valued ambiguity between the phase difference and the angle of arrival, resulting in severe distortion in the reconstruction of the gesture trajectory. If the antenna spacing is too small, the electromagnetic coupling between the antennas will be drastically enhanced, leading to deterioration of channel isolation and distortion of phase measurement. This embodiment limits the antenna spacing to half a wavelength and operates in the high-frequency band Channel 9. Because the wavelength of high-frequency signals is short, even minute spatial displacements of the hand can cause more drastic phase changes. This gives the system extremely high sensitivity to sub-millimeter-level hand micro-movements, while avoiding spatial aliasing and improving the uniqueness of angle measurement.

[0048] In some optional embodiments, to accurately and cleanly extract the in-phase / quadrature components of the direct path from complex indoor multipath transmission environments, this method deeply optimizes the acquisition and processing flow of the original channel state information. Specifically, acquiring the original channel state information and extracting the in-phase / quadrature components of the direct path includes: First, the digital baseband processor 103 acquires multiple channel state information matrices of the ultra-wideband antenna array 101 in the antenna domain. Next, it performs an inverse fast Fourier transform (IFFT) on each channel state information matrix to convert the frequency-domain channel state information to the time domain, thus obtaining multiple time-domain channel impulse responses. Then, it performs modulo-square operations on the multiple time-domain channel impulse responses to obtain multiple single-sample power delay spectra. Finally, it performs mean-square processing on the multiple single-sample power delay spectra to eliminate instantaneous fading and thermal noise interference from the physical channel, obtaining a specified power delay spectrum.

[0049] In contrast, if phase extraction of channel state information is performed directly in the frequency domain, the multipath interference will severely distort the phase information, leading to a significant decrease in gesture recognition accuracy, because the frequency domain signal is a superposition of the direct path signal and all multipath reflection signals (such as reflections from walls and furniture). However, the embodiments of this application convert the signal to the time domain using IFFT, where the direct path and the scattering path are completely separated at different time delays. By constructing a specified power delay spectrum, the time delay position of the direct path energy can be clearly located, providing a high signal-to-noise ratio data foundation for the subsequent accurate extraction of the in-phase / quadrature components of the direct path, which are unaffected by multipath interference.

[0050] In some optional embodiments, to obtain a more accurate specified power delay spectrum, embodiments of this application may also employ a sliding window denoising algorithm. Specifically, the sliding window denoising algorithm may include: A sliding window of preset length is used to process the specified power delay spectrum, and the sum of power of multiple sampling points within the sliding window is calculated during the sliding process to determine the maximum sum of power. All sampling points within the sliding window corresponding to the maximum sum of power are determined as target sampling points, while all sampling points outside the target sampling points are determined as noise sampling points.

[0051] Next, the average power of all noise sampling points is calculated, and the power of the target sampling point is subtracted from the average power to obtain the specified power delay spectrum after denoising.

[0052] In contrast, traditional fixed-threshold denoising methods cannot adapt to time-varying background noise and are prone to mistakenly filtering out weak gesture reflection signals as noise. However, the embodiments of this application adaptively determine the noise floor through sliding windows and accurately remove strong direct path energy that does not carry gesture dynamic information, thereby significantly improving the signal-to-noise ratio of weak multipath scattering signals caused by gesture movement and greatly enhancing the ability to capture spatial gestures.

[0053] In some optional embodiments, to further improve the robustness of spatial gesture recognition in complex dynamic environments, this method introduces a multi-dimensional feature fusion mechanism. Specifically, the spatial gesture recognition method further includes: constructing a non-line-of-sight feature matrix based on the actual power sequences corresponding to multiple consecutive information packets, and using the non-line-of-sight feature matrix to assist in spatial gesture recognition.

[0054] Specifically, actual power sequences corresponding to multiple consecutive UWB packets (e.g., 500 consecutive packets with a sampling interval of 1 ms) are collected. These actual power sequences are arranged in chronological order to construct a non-line-of-sight feature matrix of size 500×L, where L is the number of sampling points. This non-line-of-sight feature matrix completely records the dynamic evolution of the scattered path energy in space over time during gesture movement.

[0055] In contrast, if gesture recognition relies solely on the instantaneous phase difference between antennas, the phase difference change is extremely subtle when the gesture trajectory is within the angle measurement blind zone of the antenna array (e.g., the direction of movement is strictly perpendicular to the antenna normal), easily leading to missed detections or misjudgments. However, the non-line-of-sight feature matrix constructed in this embodiment reflects the perturbation of the spatial multipath energy field by hand movement, and it is physically complementary to the phase difference feature. By using this non-line-of-sight feature matrix as an auxiliary feature and fusing it with the differential phase change sequence, the recognition blind zone of a single phase feature can be effectively filled, further improving the overall recognition accuracy of the system under various complex gesture trajectories.

[0056] In some optional embodiments, to eliminate hardware non-ideals and interference from strong indoor static clutter on weak gesture signals, the extracted in-phase / quadrature components must undergo rigorous preprocessing before calculating the instantaneous phase difference. Specifically, the spatial gesture recognition method further includes the following steps before calculating the instantaneous phase difference: Channel consistency calibration is performed on the in-phase / quadrature components to eliminate hardware amplitude and phase deviations between different receiving channels in the ultra-wideband antenna array 101. Specifically, pre-stored calibration coefficients are invoked to perform complex multiplication compensation on the in-phase / quadrature components of each receiving channel, ensuring that the amplitude and phase outputs of each channel are completely consistent when there are no obstructions.

[0057] Next, a second-order moving target indication (MTI) filter is used to filter the calibrated in-phase / quadrature components to remove the DC component and static clutter interference caused by static environmental reflections.

[0058] In contrast, without channel consistency calibration, the inherent phase difference between channels will be directly superimposed on the measured phase, resulting in a serious systematic deviation in angle calculation. Without MTI filtering, the static echo energy reflected by stationary objects indoors (such as walls, tables, and chairs) is usually 2 to 3 orders of magnitude greater than the dynamic echo energy reflected by the hand, which will saturate the receiver and drown out the weak gesture signal. However, the embodiments of this application use a second-order MTI filter, which constructs an extremely wide suppression stopband at the zero Doppler frequency (static), attenuating static clutter and DC components by more than 25 dB, while completely preserving the dynamic Doppler component caused by the gesture movement, thereby achieving stable extraction of sub-millimeter level hand micro-motion signals.

[0059] In some optional embodiments, to ensure the continuity of the phase difference sequence and eliminate the influence of the user's tilt when holding the receiving device (such as a mobile phone) on the accuracy of spatial gesture recognition, embodiments of this application may also provide a phase unwinding and dynamic attitude compensation algorithm. Specifically, after calculating the instantaneous phase difference between each pair of antennas, the method further includes: The instantaneous phase difference is normalized to the range of [-180°, 180°] to eliminate phase entanglement. Since the absolute phase output of the arctangent function is limited to the range of [-π, π], the instantaneous phase difference obtained by direct subtraction may exceed this range and produce a 2π phase jump (i.e., phase entanglement). Therefore, the digital baseband processor 103 normalizes the calculated instantaneous phase difference to the range of [-180°, 180°] (i.e., the range of [-π, π] radians) to eliminate phase jumps and ensure the continuity and smoothness of the phase difference sequence.

[0060] Simultaneously, the angle between the receiving device (where the digital baseband processor 103 is located) and the vertical direction is acquired. Based on the instantaneous phase difference and the angle, the instantaneous phase difference is corrected to determine the actual azimuth angle of the ultra-wideband tag device relative to the receiving device. Specifically, the digital baseband processor 103 acquires the current angle between the receiving device and the vertical direction in real time through the device's built-in attitude sensor (such as a gravity sensor or gyroscope). Based on this angle, a corresponding azimuth correction function is selected from a preset function library to dynamically correct the measured instantaneous phase difference.

[0061] In contrast, without posture correction, when a user holds the phone at a 30-degree angle, the actual direction of the gesture in space will deviate from the relative angle of the antenna array, causing the classifier to misidentify "swiping right" as "swiping diagonally upwards." However, the embodiments of this application eliminate the angle projection deviation caused by the holding posture by obtaining the tilt angle and compensating for the phase difference in real time, accurately calculating the actual azimuth angle of the ultra-wideband tag device relative to the device, so that gesture recognition can maintain high accuracy at any tilt angle of the device.

[0062] In some optional embodiments, to transform the phase difference time series into a high-dimensional feature vector that can be efficiently recognized by the classifier, this method designs a multi-dimensional spatiotemporal feature extraction algorithm. Specifically, features are extracted from the differential phase change sequence over continuous time, such as... Figure 5 As shown, it includes: the training phase and the application phase.

[0063] The training phase includes: collecting gesture samples of various spatial gestures, extracting the phase features corresponding to the gesture samples, labeling the gesture categories with the phase features, training the model using the phase features, and saving the model parameters. The application phase includes processing the differential phase change sequence over N consecutive sampling times using a sliding window, where N is an integer greater than 1 (in this embodiment, N=500 frames, corresponding to a continuous 500ms observation window, a sampling interval of 1ms, and a sliding window length of 100). The N sampling times can be called the detection time interval. To ensure the continuity of features, a 50% overlap rate is set between adjacent sliding windows. Typically, the execution time of a spatial gesture is approximately 0.5s. This embodiment can also configure multiple detection time intervals to ensure that complete and accurate spatial gestures can be detected within the execution time of various spatial gestures.

[0064] Within each window, the following multi-dimensional features are extracted: First, temporal features, including calculating the mean (reflecting the overall trend of the gesture movement), variance (reflecting the intensity of the gesture movement), and zero-crossing rate (reflecting the reciprocating frequency of the gesture movement) of the phase difference sequence; second, frequency domain features, by performing a Fast Fourier Transform (FFT) on the sequence within the window to calculate the spectral energy distribution to capture the micro-Doppler characteristics of the gesture movement; and finally, spatiotemporal features, by performing a polynomial fitting on the phase difference trajectory to extract the fitting coefficients to characterize the geometric trajectory of the gesture in space.

[0065] In contrast, if the original phase difference sequence is directly input into the classifier, the network is extremely difficult to converge and prone to overfitting due to noise and differences in gesture speed. However, the embodiments of this application extract multi-dimensional features in the time domain, frequency domain, and spatiotemporal domain to construct a highly discriminative feature vector, which significantly reduces the dimensionality of the input data, improves the training convergence speed of the gesture classifier, and enhances the recognition accuracy.

[0066] In some optional embodiments, to prevent accidental triggering of the system due to unconscious hand movements or body shaking during daily use of the device or other non-gesture operations, this method designs a dual-threshold anti-mistouch determination mechanism. Specifically, after outputting the gesture category, the method further includes performing anti-mistouch determination, which includes: Determine whether the confidence level of the currently output gesture category is greater than the first threshold (set to 90% in this embodiment), and whether it is identified as the same gesture M times consecutively (M=3 in this embodiment), where M is an integer greater than 1; If the determination result is yes, the current gesture is confirmed to be valid, and the action mapping instruction corresponding to the gesture category is executed (such as controlling the music player to switch songs and adjust the volume, or controlling the temperature and windows of the smart cockpit); if the determination result is no, no instruction is output, and the system returns to the signal receiving stage to continue real-time monitoring.

[0067] In contrast, if a single-frame threshold-free triggering mechanism is used, the user's micro-movements of the hand when speaking, drinking water, or adjusting the grip posture are easily misidentified as gesture commands, leading to frequent malfunctions of the device. However, the embodiments of this application reduce the false triggering rate of the system to an extremely low level through the dual constraints of confidence and consecutive frame count, greatly improving the user experience.

[0068] In some optional embodiments, to ensure consistency of mass-produced equipment and that all factory-shipped equipment can be perfectly adapted to the gesture classifier pre-trained in the laboratory, this method designs a production line phase calibration and online compensation mechanism, which can be further referenced. Figure 5 .

[0069] The classifier can be a lightweight recurrent neural network (such as LSTM, with a model size of approximately 50KB) or a dynamic time warping (DTW) classifier. The method also includes phase calibration before the device leaves the factory. Phase calibration involves: in an anechoic chamber, using a standard signal source to transmit a signal from a fixed angle, measuring the inherent phase of the receiving link of each antenna in the ultra-wideband antenna array 101, and storing the instantaneous phase difference between antenna pairs in non-volatile memory (e.g., NV memory) so that the inherent phase difference can be subtracted from the measured instantaneous phase difference during online algorithm execution to eliminate systematic errors caused by hardware manufacturing tolerances.

[0070] In contrast, without production line calibration, the phase difference output of different devices under the same gesture will deviate by several to tens of degrees, resulting in a significant drop in the recognition rate of the pre-trained neural network model, and even requiring separate training of the model for each device; however, the embodiments of this application achieve a high degree of consistency in the phase output of mass-produced devices through "offline calibration + online subtraction", ensuring the feasibility of large-scale mass production of the gesture recognition system.

[0071] An optional embodiment of this application provides a spatial gesture recognition system, such as... Figure 6As shown, it includes: a data acquisition module, a phase difference calculation module, and a spatial gesture determination module. The data acquisition module is used to acquire pulse signals sent by a second electronic device through the antenna array of a first electronic device; wherein the pose of the second electronic device changes with the gesture; the phase difference calculation module calculates the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal to obtain the differential phase change sequence of the antenna pair; the spatial gesture determination module performs spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture.

[0072] An optional embodiment of this application provides an electronic device 70, which physically includes a processor 702 and a memory 701. The memory 701 is a non-volatile storage medium used to store processor-executable computer programs. The processor 702 is configured to implement the spatial gesture recognition method as described in any of the foregoing method embodiments when the computer program is invoked and executed.

[0073] Specifically, the processor 702 controls the RF front-end to synchronously receive pulse signals transmitted by the ultra-wideband tag device, reads the in-phase / quadrature components of the channel impulse response direct path, calculates the instantaneous phase difference between antennas to eliminate common-mode errors, extracts timing features, and runs a pre-trained lightweight LSTM neural network model. Finally, it outputs a valid gesture category to control the operation of the electronic device. In contrast, traditional electronic devices mainly rely on touchscreens or physical buttons for human-computer interaction, which becomes extremely inconvenient when the user's hands are occupied. This electronic device, by running a spatial gesture recognition program in the processor 702, allows users to achieve contactless and precise control of the device simply by making simple gestures in front of it, greatly expanding the application scenarios of the electronic device.

[0074] In some optional embodiments, the specific product form of the electronic device is defined. Specifically, the electronic device is a smartphone, a smart control screen, or a smart wearable device (such as a smartwatch or smart glasses). Taking a smartphone as an example, an ultra-wideband antenna array is deployed above the phone screen. When the phone is placed on a table, the user can perform operations such as changing songs, turning pages, or muting simply by waving their hand above the phone, without having to pick it up, greatly improving the convenience of daily use. Taking a smart control screen as an example, it is often deployed on the living room wall of a smart home or the control panel of a smart cockpit. Users can make a rotating gesture within 1 meter of the control screen to accurately adjust the indoor air conditioning temperature or audio volume, realizing a highly technological non-contact spatial interaction.

[0075] In some optional embodiments, to address the issue of driver distraction and safety hazards caused by looking down at the central control screen while driving, this electronic device is specifically applied to the vehicle's smart cockpit. Specifically, the electronic device is used in a vehicle, with an ultra-wideband antenna array mounted on the steering column below the steering wheel. This array is used to extract the phase difference characteristics of the channel state information from the driver's hand micro-movements, enabling contactless control of the in-vehicle equipment.

[0076] When a driver needs to adjust in-vehicle equipment (such as adjusting the air conditioning temperature, changing the radio, or answering a phone call) while driving, they do not need to take their eyes off the road or reach for the central control screen. They only need to make hand gestures in the space to the right of the steering wheel. The ultra-wideband antenna array 101 on the steering column extracts the phase difference characteristics of the channel state information caused by the driver's micro-movements in real time. The digital baseband processor 103 quickly identifies the specific gesture and converts it into control commands, which are then sent to the in-vehicle equipment for execution. In contrast, if the gesture radar is placed above the center console, the driver needs to raise their arm a large distance to operate it, which can easily cause arm fatigue, and the large range of limb movements can still distract the driver. However, in this embodiment, the antenna array is placed on the steering column, and precise control can be achieved with only a small movement of the driver's wrist (micro-gesture). The hand does not need to leave the steering wheel too far, ensuring excellent interactive convenience while maximizing driving safety.

[0077] An optional embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the spatial gesture recognition method as described in any of the foregoing method embodiments. The computer-readable storage medium is preferably a non-volatile memory to ensure the physical rigidity and storage stability of the program.

[0078] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for recognizing spatial gestures, characterized in that, The method includes: The pulse signal sent by the second electronic device is acquired through the antenna array of the first electronic device; wherein the pose of the second electronic device changes with the gesture. The instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time is calculated based on the pulse signal to obtain the differential phase change sequence of the antenna pair; Spatial gesture matching is performed on the differential phase change sequence to determine the gesture category of the spatial gesture.

2. The method according to claim 1, characterized in that, The step of calculating the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal to obtain the differential phase change sequence of the antenna pair includes: Determine the original channel state information corresponding to the pulse signal of each antenna in the antenna pair at each sampling time; Determine the channel impulse response for each of the original channel state information; The in-phase and quadrature components of the channel impulse response are obtained. The absolute phase of each antenna is determined based on the in-phase and quadrature components. The instantaneous phase difference between the antenna pairs is calculated to obtain the differential phase change sequence of the antenna pairs during the detection time period corresponding to each sampling time.

3. The method according to claim 2, characterized in that, The step of obtaining the in-phase and quadrature components of the channel impulse response includes: Obtain multiple channel state information matrices corresponding to the channel impulse response; A fast inverse Fourier transform is performed on each channel state information matrix to obtain multiple time-domain channel impulse responses; The multiple time-domain channel impulse responses are subjected to modulo-square operation to obtain multiple single-sample power delay spectra. The power delay spectra of multiple single samples are averaged to obtain the specified power delay spectrum, and the corresponding in-phase and quadrature components are extracted based on the specified power delay spectrum.

4. The method according to claim 3, characterized in that, The step of averaging multiple single-sample power delay spectra to obtain a specified power delay spectrum further includes: The specified power delay spectrum is processed by a sliding window of a preset length, the power sum of multiple sampling points in each window is calculated, and the maximum power sum is determined. All sampling points within the sliding window corresponding to the maximum power are determined as target sampling points, and all sampling points outside the target sampling points are determined as noise sampling points; Calculate the average power of all noise sampling points, and subtract this average power from the power of the target sampling point to obtain the specified power delay spectrum after denoising.

5. The method according to claim 1, characterized in that, The step of performing spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture corresponding to the pose change of the second electronic device relative to the first electronic device includes: A sliding window process is performed on at least one of the differential phase change sequences. In each sliding window, the time-domain features, frequency-domain features, and spatiotemporal features corresponding to the differential phase change sequence are extracted to form a feature vector. The feature vector is used to describe the pose change of the second electronic device relative to the first electronic device. The feature vector is input into a pre-trained gesture classification model to determine the gesture category and confidence level corresponding to the pose change.

6. The method according to claim 5, characterized in that, Also includes: If the confidence level of the gesture category is greater than a preset threshold, and the gesture is identified as the same gesture category for more than a preset number of consecutive times, the gesture category is output.

7. The method according to claim 2, characterized in that, The in-phase and quadrature components are determined by the following formula: ,in, This indicates the antenna index of the antenna array. Indicates the first The root antenna at the sampling time In-phase components, Indicates the first The root antenna at the sampling time orthogonal components, The imaginary unit, Complex form of the orthogonal components of in-phase components.

8. The method according to claim 7, characterized in that, The absolute phase of the antenna is determined by the following formula: .

9. A spatial gesture recognition system, characterized in that, include: The module consists of an acquisition module, a phase difference calculation module, and a spatial gesture determination module. The acquisition module is used to acquire the pulse signal sent by the second electronic device through the antenna array of the first electronic device; wherein the pose of the second electronic device changes with the gesture; The phase difference calculation module is used to calculate the instantaneous phase difference of at least one antenna pair in the antenna array at each sampling time based on the pulse signal, and obtain the differential phase change sequence of the antenna pair; The spatial gesture determination module is used to perform spatial gesture matching on the differential phase change sequence to determine the gesture category of the spatial gesture.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the spatial gesture recognition method as described in any one of claims 1 to 8.

Images