A system and method for gesture control in free space based on a capacitive wave array

Abstract

The application discloses a kind of overhand gesture control system and method based on capacitive wave array, it is related to human-computer interaction technical field, including electric field establishment: after being powered on on capacitive wave array, transmitting electrode emits specific frequency and phase alternating voltage to space, establishes quasi electrostatic field above array;Initialization calibration step: measure the background capacitance value of all sensing electrodes in no finger state, establish initial baseline, set transmitting power and sampling rate to low-power polling mode;Partition dynamic wake-up: the capacitive wave array is divided into several independent sub-regions, and a sentinel electrode is arranged in each sub-region;The present application is programmable by multi-frequency gradient electric field, near-field high-frequency low-power, far-field low-frequency high-power, so that the detection height is expanded from traditional capacitance to 0~200mm, while using over-complete dictionary sparse matching and OMP algorithm, the horizontal positioning accuracy is better than 2mm, solve the problem of traditional single-frequency electric field detection height limitation, large positioning error.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a gesture control system and method based on a capacitor wave array. Background Technology

[0002] In recent years, human-computer interaction technology has evolved from physical buttons to touch screens. With the development of application scenarios such as smart homes, in-vehicle displays, and consumer electronics, air gesture interaction, as a natural and intuitive interaction method, has gradually attracted attention.

[0003] Traditional capacitive gesture sensing solutions rely on single-frequency electric field amplitude detection, which typically has a detection height of less than 20cm and a positioning error of more than 5cm, failing to meet the needs of medium- and long-distance interaction. In addition, this type of solution only uses a single modal feature, making it susceptible to interference from changes in environmental dielectrics and the approach of non-human objects, resulting in a high false trigger rate. After static interference occurs, global recalibration is required, leading to a poor user experience.

[0004] Optical / infrared gesture sensing solutions are greatly affected by light and occlusion, and their performance drops significantly in bright light or dark environments; the recognition accuracy is less than 70% when the user is wearing gloves; at the same time, such solutions require additional optical sensors and computing chips, resulting in high BOM (Bill of Materials) costs and high power consumption, making them difficult to adapt to low-power embedded scenarios.

[0005] Existing deep learning-based gesture recognition solutions are resource-intensive, require the deployment of high-performance processors, and are incompatible with general-purpose microcontrollers at the Cortex-M4F level. Custom gestures require the re-collection of a large number of samples to train the model, resulting in long adaptation cycles and high implementation costs.

[0006] Therefore, this application proposes a remote gesture control system and method based on a capacitor wave array. Summary of the Invention

[0007] To address the aforementioned technical problems, a capacitive wave array-based air gesture control system and method are provided. This technical solution solves the problems of high power consumption, high false touch rate, long adaptation cycle, and high cost associated with single-mode operation.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for air gesture control based on a capacitor wave array, comprising: Step 1. Electric field establishment steps: After the capacitor wave array is powered on, the transmitting electrode emits an AC voltage of a specific frequency and phase into space, establishing a quasi-electrostatic field above the array; Step 2. Initialization calibration steps: Measure the background capacitance value of all sensing electrodes in a fingerless state to establish an initial baseline, and set the transmit power and sampling rate to low-power polling mode; Step 3. Dynamic wake-up step: Divide the capacitor wave array into several independent sub-regions, and set a sentinel electrode in each sub-region; monitor the capacitance change rate of the sentinel electrode using the adjacent electrode differential measurement method; when the capacitance change rate of any sentinel electrode exceeds a preset threshold, wake up the corresponding sub-region and its adjacent sub-regions, and increase the transmission power and sampling rate to the normal working level. Step 4. 3D positioning steps: Obtain the capacitance change values ​​of all sensing electrodes in the current frame, subtract the dynamically updated background baseline to obtain the observation vector; perform sparse matching between the observation vector and the pre-stored overcomplete dictionary, use the orthogonal matching pursuit algorithm to solve the sparse coefficients, and extract the 3D coordinates of the finger from the sparse coefficients. Step 5. Trajectory tracking steps: Connect the three-dimensional coordinates of consecutive frames to form a motion trajectory, use a low-pass filter to smooth the trajectory, and divide the trajectory into motion segments and pause segments according to the trajectory temporal characteristics; Step 6. Gesture recognition steps: Classify each motion segment based on a finite state machine, and determine the gesture based on the field area where the finger is currently located using an adaptive threshold for the field area; Step 7. Power consumption revert step: If all fingers leave the detection area for more than a set time, the system will switch back to low power polling mode.

[0009] Preferably, in Step 1, different excitation frequencies and transmission powers are dynamically matched according to the field area where the finger is located at the height of the array: the field area is divided into: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm; The near-field uses a higher excitation frequency in the range of 100kHz to 2MHz, and the transmit voltage is 1 to 2Vpp; The mid-field uses a medium excitation frequency in the range of 100kHz to 500kHz, and the transmission voltage is 2 to 3Vpp; The far field uses a lower excitation frequency below 100kHz, and the transmit voltage is 3~5Vpp.

[0010] Preferably, in Step 3, in the low-power polling mode, the transmit voltage is 1Vpp, the duty cycle is 10%, and the sensing channel sampling rate is 10Hz; under normal operating conditions, the transmit voltage is 3~5Vpp, the sensing channel sampling rate of the woken sub-region is 100Hz, and the sensing channel sampling rate of the adjacent sub-region is 50Hz; if a sub-region does not detect a valid disturbance within 500ms and there is no activity in the adjacent sub-region, it is reduced back to low-power mode.

[0011] Preferably, the method for constructing the overcomplete dictionary includes: Discretize the three-dimensional space above the array into grid points, and set the X direction... One point, Y-direction setting Each point, set in the Z direction. A standard probe is moved sequentially to each grid point using a robotic arm, recording the capacitance changes of all sensing channels to form an atomic vector. Each atomic vector is then L2 normalized to obtain a dictionary matrix D of size [value missing]. Where C is the number of sensing channels. The total number of atoms in the dictionary; The execution steps of the orthogonal matching pursuit algorithm include: Initialize residual Support set , where y is the observation vector of the current frame; Calculate the inner product vector of all atoms and residuals. Find the atom index with the largest absolute value of the inner product. Add support set ; Compute the least-squares projection over the subspace spanned by the current support set: Update residuals If the number of iterations reaches the preset sparsity K or the residual L2 norm, If the iteration stops, then stop. Methods for extracting the 3D coordinates of a finger from sparse coefficients include: Atoms with coefficients whose absolute values ​​are less than a preset proportion of the maximum coefficient are set to zero; on the 3D mesh, non-zero coefficient points connected by 26 neighborhoods are grouped into a cluster; using the coefficient values ​​as weights, the 3D coordinate weighted centroid of each cluster is calculated: in The physical coordinates of grid point j , This corresponds to the sparsity coefficient value.

[0012] Preferably, the field area employs an adaptive threshold configuration, including: Motor initiation threshold : Near field 15mm / s, mid field 25mm / s, far field 40mm / s; pause detection threshold : 10mm / s near the field, 15mm / s mid-field, and 25mm / s far from the field.

[0013] Preferably, based on the obtained three-dimensional coordinates of the finger, a finite state machine is used for trajectory segmentation and gesture recognition, including idle state, motion tracking state, and state to be recognized: The idle state: when the speed exceeds the motion initiation threshold, it enters the motion tracking state; The motion tracking state: when the speed is lower than the pause threshold and the duration exceeds the first preset value, it enters the identification state; The state to be identified: the gesture type is identified based on the stored motion segment features, or it is identified as a double-click when a new motion segment is detected within a second preset time.

[0014] Preferably, in Step 6, the gesture recognition rules based on geometric features include: Click recognition: Single finger mode, movement duration less than 200ms, obvious downward pressure and return in the Z-axis direction, horizontal displacement less than the preset value, and the speed profile shows the characteristics of first accelerating and then decelerating. Double-click recognition: Both consecutive motion segments meet the characteristics of a single click, the pause time between the two segments is within 100~400ms, and the spatial center distance is less than 15mm; Slide recognition: The speed remains stable during motion, the trajectory curvature is less than a preset threshold, and horizontal or vertical sliding is distinguished based on the X-axis and Y-axis displacement ratio; Rotation recognition: Two-finger mode, the change in the distance between two fingers is less than a preset ratio, and the cumulative angle change exceeds 120°; Zoom recognition: Two-finger mode, the midpoint position remains basically unchanged even when the distance between the two fingers changes beyond the preset ratio.

[0015] Preferably, multimodal feature fusion is used for anti-interference, including synchronous acquisition of amplitude features, phase difference features and electrode gradient features, and combined with human body-specific dielectric frequency response features for judgment; the baseline dynamic update strategy includes: periodically or when an abnormal signal is detected, recalibrating the baseline to eliminate baseline drift caused by temperature and humidity changes.

[0016] A gesture control system based on a capacitive wave array includes a capacitive wave array comprising a transmitting electrode, a sensing electrode, and a shielding layer; the transmitting and sensing electrodes are arranged alternately to form a mosaic checkerboard structure; an AC voltage is applied to the transmitting electrode to establish a quasi-electrostatic field in space; the shielding layer is located on the back and edge of the array to suppress external radiation and interference from the external environment. Signal acquisition unit: High-speed acquisition of capacitance change signals on all sensing electrodes, converting weak electric field disturbances into digital quantities; employing differential measurement of adjacent electrodes to suppress common-mode interference; Signal processing and gesture recognition unit: includes an embedded processor that runs positioning and gesture recognition algorithms; the embedded processor is configured to perform the method steps of any one of claims 1 to 8.

[0017] Preferably, the distance between adjacent centers of the emitting electrodes is 10mm to 15mm; The sensing electrode is located in the gap between adjacent emitting electrodes, with a gap distance of 0.5mm to 1mm; The emitting electrode and the sensing electrode are both fabricated on the top layer of the same PCB. The shielding layer is a single piece of copper foil at the bottom of a double-layer PCB, or conductive cloth or copper foil tape bonded to the back of an FPC. The shielding layer is connected to the system signal ground via multiple grounding points; The signal processing and gesture recognition unit divides the detection space into field areas based on the height of the finger from the capacitor wave array. The field areas are divided into: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm. The signal processing and gesture recognition unit further includes: a dictionary storage module for storing a pre-built, incomplete dictionary D; The sparse matching module is used to implement the orthogonal matching pursuit algorithm to perform sparse decomposition on the observation vector; The trajectory management module is used to maintain the trajectory ID of each finger and record the historical position sequence; The state machine module is used to implement finite state machine logic and output gesture recognition results. Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention uses a multi-frequency gradient electric field programmable generation, with high frequency and low power in the near field and low frequency and high power in the far field, which expands the detection height from <20cm in the traditional capacitor scheme to 0~200mm. At the same time, it adopts an overcomplete dictionary sparse matching and OMP algorithm, and the lateral positioning accuracy is better than 2mm, which solves the problems of limited detection height and large positioning error in the traditional single-frequency electric field.

[0018] 2. By simultaneously acquiring three features—amplitude, phase difference, and electrode gradient—through multimodal feature fusion, and combining them with the human body's specific dielectric frequency response features, we can distinguish between real fingers and static interference objects. With the local baseline dynamic update strategy, there is no need for global recalibration when interference occurs, which solves the shortcomings of conventional capacitor solutions that are prone to false triggering in single mode and require global recalibration for static interference.

[0019] 3. By using a partitioned dynamic wake-up mechanism, the sub-regions in areas without fingers are kept in low-power polling mode (1Vpp / 10% duty cycle / 10Hz), waking up only the sub-regions near the fingers to the working mode; the gesture ends and the sub-regions are automatically switched back, which greatly reduces standby power consumption and solves the problem of continuous high power consumption and unsuitability for embedded scenarios in optical solutions.

[0020] 4. The gesture recognition adopts a rule scheme of finite state machine + field area adaptive threshold. The algorithm is fully compatible with Cortex-M4F level microcontrollers. Custom gestures only need to modify the rule parameters without retraining the model, which solves the defects of high resource consumption and long adaptation cycle of deep learning solutions. Attached Figure Description

[0021] Figure 1 The flowchart shows the air gesture control method based on capacitor wave array. Figure 2 A schematic diagram of the capacitor wave array and sub-region division; Figure 3 Flowchart of overcomplete dictionary construction and OMP algorithm execution; Figure 4 This refers to the state transitions of a finite state machine. Detailed Implementation

[0022] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0023] Reference Figure 1 As shown, a gesture control method based on a capacitor wave array includes: Step 1. Electric field establishment step: After the capacitor wave array is powered on, the transmitting electrode emits an AC voltage of a specific frequency and phase into space to establish a quasi-electrostatic field above the array; When an alternating voltage is applied to the emitting electrodes, an alternating electric field is formed around them. Multiple emitting electrodes are independently controllable, allowing for dynamic adjustment of the spatial shape of the electric field, forming an invisible "capacitive wave net" covering the area above the array. Human fingers are primarily composed of water and electrolytes, with a dielectric constant much higher than that of air. When a finger enters the electric field region, it alters the capacitive coupling relationship of the local electric field, equivalent to inserting a high-dielectric-constant object into the electric field, causing a change in the capacitance value measured on the inductive electrodes. The closer the finger is and the larger its size, the stronger the disturbance.

[0024] Step 2. Initialization calibration steps: Measure the background capacitance value of all sensing electrodes in a fingerless state to establish an initial baseline, and set the transmit power and sampling rate to low-power polling mode; After the system is powered on, in a fingerless state, the background capacitance values ​​of all sensing channels are measured to establish an initial baseline. A pre-stored electric field feature library and gesture classification rules are loaded. The capacitance wave array continuously emits an electric field at low power and low frequency. All sensing electrodes take turns monitoring capacitance changes at a low sampling rate to determine if they exceed a threshold, thus indicating the presence of a finger.

[0025] Step 3. Dynamic wake-up step: Divide the capacitor wave array into several independent sub-regions, and set a sentinel electrode in each sub-region; monitor the capacitance change rate of the sentinel electrode using the adjacent electrode differential measurement method; when the capacitance change rate of any sentinel electrode exceeds a preset threshold, wake up the corresponding sub-region and its adjacent sub-regions, and increase the transmission power and sampling rate to the normal working level. The influence range of finger disturbance is limited (the radius is about 1.5 to 2 times the electrode spacing), affecting only a few surrounding sensing electrodes. Therefore, the array can be divided into independent physical sub-regions, and the electric fields of each sub-region do not interfere with each other or the interference is negligible. The transmitting and sensing electrodes are multiplexed, requiring time-division driving and sampling. The sub-regions correspond to independent transmitting channel groups and sensing channel groups, which can realize parallel wake-up or time-division fast scanning. The transmitting voltage amplitude and sampling frame rate can be adjusted independently between sub-regions. Sub-regions far from the finger can reduce the transmitting power or even turn off the transmitting power, reducing the total power consumption and external electromagnetic radiation.

[0026] Step 4. 3D positioning steps: Obtain the capacitance change values ​​of all sensing electrodes in the current frame, subtract the dynamically updated background baseline to obtain the observation vector; perform sparse matching between the observation vector and the pre-stored overcomplete dictionary, use the orthogonal matching pursuit algorithm to solve the sparse coefficients, and extract the 3D coordinates of the finger from the sparse coefficients. For each finger's perturbation component, it is matched against a pre-stored electric field feature library to calculate the finger's three-dimensional coordinates (X, Y, height Z) above the array. Localization is achieved using pre-calculated feature library matching instead of real-time solution of the electric field equations, as the latter is computationally intensive and ill-posed. The pre-built library transforms the inverse problem into a search problem, and combined with a fast nearest neighbor search algorithm, millisecond-level response times can be achieved on low-computing-power platforms.

[0027] Step 5. Trajectory tracking steps: Connect the three-dimensional coordinates of consecutive frames to form a motion trajectory, use a low-pass filter to smooth the trajectory, and divide the trajectory into motion segments and pause segments according to the trajectory temporal characteristics; A capacitive wave array outputs the 3D coordinates and confidence level of each finger at a fixed frame rate. The position coordinates of multiple consecutive frames are connected to form a trajectory. Gesture type is identified based on the temporal characteristics of the trajectory (movement direction, speed changes, dwell time, etc.).

[0028] Step 6. Gesture recognition steps: Classify each motion segment based on a finite state machine, and determine the gesture based on the field area where the finger is currently located using an adaptive threshold for the field area; Gesture recognition employs a finite state machine plus rules: for typical gestures (clicks, swipes, etc.), a trajectory-based rule classifier is used, which has low computational cost, strong interpretability, and is easy to debug. Complex gestures can be upgraded to a lightweight neural network, but the basic version maintains low resource consumption.

[0029] Step 7. Power consumption revert step: If all fingers leave the detection area for more than a set time, the system will switch back to low power polling mode, retaining only the basic electric field emission.

[0030] like Figure 1 As shown, in Step 1, different excitation frequencies and transmission powers are dynamically matched according to the field area where the finger is located at the height of the array: The near field uses a higher excitation frequency and a lower transmission power; the mid-field uses a medium excitation frequency and a medium transmission power. The far field uses a lower excitation frequency and a higher transmit power.

[0031] This multi-frequency gradient electric field programmable generation mechanism enables a single hardware unit to be compatible with full-range gesture detection. The height of the finger from the array plane directly affects the intensity of the electric field perturbation, spatial resolution, and effective motion characteristics. By using different excitation parameters in different field regions, power consumption can be optimized while ensuring detection performance.

[0032] like Figure 1 As shown, in Step 3, in the low-power polling mode, the transmit voltage is 1Vpp, the duty cycle is 10%, and the sensing channel sampling rate is 10Hz; under normal operating conditions, the transmit voltage is 3~5Vpp, the sensing channel sampling rate of the woken sub-region is 100Hz, and the sensing channel sampling rate of the adjacent sub-region is 50Hz; if a sub-region does not detect a valid disturbance within 500ms and there is no activity in the adjacent sub-region, it is reduced back to low-power mode.

[0033] Sentinel electrode monitoring uses a central sensing electrode within each sub-region as a "sentinel" to monitor its capacitance change rate. If the change rate of a certain sentinel channel exceeds a threshold (e.g., change > 5% per second), then that sub-region and its adjacent sub-regions (Manhattan distance ≤ 1) are activated.

[0034] Taking a 120mm×80mm array with a 12mm electrode spacing as an example, the array is physically divided into 4×4=16 sub-regions. Each sub-region contains: 4 emitting electrodes (2×2 grid) and 9 sensing electrodes (3×3 grid) within the corresponding area. In actual division, one sensing electrode overlaps between adjacent sub-regions to ensure that at least 2 sub-regions can sense simultaneously when a finger is at the boundary, avoiding loss.

[0035] The array is divided into 4 segments in the X direction (0~120mm), each segment is 30mm; the array is divided into 4 segments in the Y direction (0~80mm), each segment is 20mm; forming 16 rectangular sub-regions, numbered (0,0) to (3,3).

[0036] like Figure 4 As shown, the method for constructing the overcomplete dictionary includes: Discretize the three-dimensional space above the array into grid points, and set the X direction... One point, Y-direction setting Each point, set in the Z direction. A standard probe is moved sequentially to each grid point using a robotic arm, recording the capacitance changes of all sensing channels to form an atomic vector. Each atomic vector is then L2 normalized to obtain a dictionary matrix D of size [value missing]. Where C is the number of sensing channels. The total number of atoms in the dictionary; Specific parameter examples: 32 points in the X direction (corresponding to an array width of 120mm), 32 points in the Y direction (corresponding to a length of 80mm), 10 layers in the Z direction (covering 0~150mm), total number of dictionary atoms. .

[0037] During data acquisition, a robotic arm is used to sequentially move a standard finger (or a simulated probe equivalent to the dielectric constant) to each grid point, recording the capacitance change values ​​of all sensing channels (assuming a total number of channels of C, typically 128), forming a C-dimensional feature vector. This process is repeated multiple times for each grid point, and the average is taken to obtain the atomic vector at that location.

[0038] Each atom is L2 normalized (divided by its own modulus) so that the support matching strength is independent in subsequent inner product operations.

[0039] The execution steps of the orthogonal matching pursuit algorithm include: Initialize residual Support set , where y is the observation vector of the current frame; Calculate the inner product vector of all atoms and residuals. Find the atom index with the largest absolute value of the inner product. Add support set ; Compute the least-squares projection over the subspace spanned by the current support set: Update residuals If the number of iterations reaches the preset sparsity K or the residual L2 norm, If the iteration stops, then stop.

[0040] The residual norm 2 stopping condition ε ranges from 0.05 to 0.2 (dimensionless), with a preferred value of ε = 0.1. This value is based on the observation vector y after L2 normalization of the capacitance changes of all sensing channels.

[0041] The value of ε should be slightly higher than the L2 norm of the system background noise. Actual measurements show that, without a finger, the mean L2 norm of the normalized capacitance change noise for all channels is 0.03~0.05. Setting ε=0.1 ensures that the effective signal is decomposed while avoiding overfitting the algorithm to noise, keeping the computational complexity within an acceptable range (typically convergence is achieved in K≤6 iterations).

[0042] Algorithm termination condition: If the maximum number of iterations K is reached (usually...) The maximum hand index or the residual L2 norm is less than a preset threshold. If the iteration stops, then stop.

[0043] Methods for extracting the 3D coordinates of a finger from sparse coefficients include: Atoms with coefficients whose absolute values ​​are less than a preset proportion of the maximum coefficient are set to zero; on the 3D mesh, non-zero coefficient points connected by 26 neighborhoods are grouped into a cluster; using the coefficient values ​​as weights, the 3D coordinate weighted centroid of each cluster is calculated: in The physical coordinates of grid point j , This corresponds to the sparsity coefficient value.

[0044] The specific steps include: thresholding, setting all coefficients whose absolute value is less than a threshold (e.g., 20% of the maximum coefficient) to zero; spatial clustering, grouping non-zero coefficient points connected by 26 neighborhoods into a cluster on a 3D grid, with each cluster corresponding to a finger; and centroid calculation, using the coefficient values ​​as weights to calculate a weighted average of the 3D coordinates for each cluster.

[0045] Computational complexity analysis: , , For example: the inner product in each iteration is approximately Multiply and add again; reduce by 3 iterations. The least squares projection involves inverting small matrices, which can be ignored. Floating-point operations on a 100MHz Cortex-M7 take approximately a few milliseconds; using fixed-point operations or pre-dividing the dictionary into blocks can reduce this to less than 1ms, meeting the requirements of 50~100fps.

[0046] In this embodiment of the application, the field area adopts an adaptive threshold configuration, including: Motor initiation threshold : Near field 15mm / s, mid field 25mm / s, far field 40mm / s; pause detection threshold : 10mm / s near the field, 15mm / s mid-field, and 25mm / s far from the field.

[0047] The values ​​of the motion initiation threshold Vstart and the pause detection threshold Vpause in different field areas are determined based on the following criteria: Near-field region: Since the finger is close to the array, the capacitive coupling signal is strong (signal-to-noise ratio > 30dB), allowing for precise detection of even the slightest finger movement. Furthermore, the near-field region is often used for fine manipulation (such as clicking), requiring high sensitivity. Therefore, Vstart = 15mm / s and Vpause = 10mm / s are set to capture slow pressing and micro-movements.

[0048] Mid-range: Signal strength is moderate; slight finger tremors (approximately 5-10 mm / s) will be amplified by noise. To balance sensitivity and anti-shake performance, the threshold is set to 1.5 to 1.7 times that of the near-field range, i.e., Vstart = 25 mm / s and Vpause = 15 mm / s.

[0049] In the far field region, the signal-to-noise ratio drops below 15dB, and natural physiological tremors of the fingers (approximately 15-20mm / s) dominate. To avoid frequent erroneous state machine switching due to tremors, the threshold needs to be significantly increased. Referring to the stable velocity statistics of a human arm naturally suspended in the air (>30mm / s in 95% of cases), Vstart=40mm / s and Vpause=25mm / s are set to ensure that only clear, conscious movements are recognized.

[0050] These thresholds are set in accordance with the principle of combining "signal strength-to-noise ratio" and "human movement physiological characteristics".

[0051] Regarding the field division, the detection space is divided into three fields based on the height of the finger from the capacitive wave array: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm.

[0052] Characteristics of each field area: Near field (0~30mm): Strong signal, disturbances concentrated on a few sensing electrodes directly below the finger, lateral resolution less than 2mm, height resolution less than 1mm, suitable for detecting single click, double click, long press, swipe, and other gestures. A relatively high excitation frequency in the range of 100kHz~2MHz is used, with a transmit voltage of 1~2Vpp.

[0053] Mid-range (30~100mm): Moderate signal, disturbance spreads to multiple surrounding sensing electrodes, lateral resolution 25mm, height resolution 24mm, suitable for detecting gestures such as swiping (page turning), rotating, and zooming. Uses a moderate excitation frequency in the range of 100kHz~500kHz, with a transmit voltage of 2~3Vpp.

[0054] Far field (100~200mm): weak signal, large but small perturbation range, decreased signal-to-noise ratio, lateral resolution 5~10mm, height resolution 5~10mm, suitable for detecting gestures such as hand gestures to wake up, directional waving, and presence. A lower excitation frequency below 100kHz is used, with a transmit voltage of 3~5Vpp.

[0055] The parameter configuration is shown in Table 1: Table 1 Parameter Configuration Table Implementation method: In the trajectory analysis module, the corresponding threshold is obtained by looking up a table based on the current field area, without changing the algorithm logic.

[0056] like Figure 4 As shown, based on the obtained three-dimensional coordinates of the finger, a finite state machine is used for trajectory segmentation and gesture recognition, including idle state, motion tracking state, and state to be recognized: The idle state: when the speed exceeds the motion initiation threshold, it enters the motion tracking state; The motion tracking state: when the speed is lower than the pause threshold and the duration exceeds the first preset value, it enters the identification state; The state to be identified: the gesture type is identified based on the stored motion segment features, or it is identified as a double-click when a new motion segment is detected within a second preset time.

[0057] For trajectory smoothing, the capacitive wave array outputs the 3D coordinates of each finger at a fixed frame rate (typically 50-100fps). A first-order low-pass filter is used to smooth the jitter while preserving the dynamic characteristics of fast movements. in It is a coefficient less than 1. For quick gestures (such as swipes), it can be switched to adaptive mode. (The faster the exercise) The larger the value, the less the lag.

[0058] Calculation of speed: Velocity vector: rate: Acceleration: Instantaneous velocity is calculated using the two most recent frames. Then, a first-order low-pass filter is used to obtain the smoothed speed. .

[0059] Segmentation of the trajectory: when If the duration exceeds a set value, it is marked as a pause point. Pause points are commonly used as start and end markers for gestures. From the first speed exceeding the threshold... Starting from the point, until the speed drops below the threshold and pauses for more than [time period missing] End, extract one motion segment.

[0060] A finite state machine is used for trajectory segmentation and gesture recognition, including idle state, motion tracking state, and state to be recognized: The system includes three states: Idle, Tracking, and Penning.

[0061] Idle state: When the speed exceeds the motion initiation threshold, it enters motion tracking state.

[0062] Motion tracking status: When the speed is lower than the pause threshold and the duration exceeds the first preset value, it enters the waiting state; if the movement continues for more than the maximum duration, it will be forcibly terminated and enter the waiting state.

[0063] Pending recognition status: The gesture type is identified based on the stored motion segment features, or it is identified as a double-tap when a new motion segment is detected within a second preset time.

[0064] State transition example: IDLE → TRACKING, when Record the start time and start position for 3 consecutive frames; TRACKING→PENDING, when Record the end time and end position continuously for 80ms; PENDING→IDLE, after After 250ms, it will automatically exit or trigger gesture recognition and exit; TRACKING→IDLE, if the movement continues for more than 2 seconds, it will force exit and be recognized as "drag" or invalid.

[0065] Motion segment feature calculation: Triggered when entering the PENDING state, it calculates the horizontal displacement from the cached historical trajectory. Vertical displacement (Negative pressure); Exercise duration Peak speed Velocity profile (rise time, fall time); trajectory curvature (path length / straight-line distance).

[0066] Multi-touch processing: Each finger independently constructs a trajectory, and then the relative geometric quantities are calculated: coordinates of the midpoint between two fingers. The distance between two fingers The angle between the line connecting the two fingers and the X-axis .

[0067] In this embodiment of the application, in Step 6, the gesture recognition rules based on geometric features include: Click recognition: Single finger mode, movement duration less than 200ms, obvious downward pressure and return in the Z-axis direction, horizontal displacement less than the preset value, and the speed profile shows the characteristics of first accelerating and then decelerating. Specific parameters: Single finger, movement duration <200ms; noticeable downward pressure in the Z-axis direction ( And return (difference between endpoint Z and starting point Z < 5mm); the horizontal displacement is very small ( The speed profile is a rapid downward press followed by an almost immediate return (pause <20ms), which is identified as a light touch.

[0068] The calculation is based on the muscle movement time (approximately 80-150ms) of a typical human "click" action. The Z-axis downward and returning motion is used to distinguish it from a "slide" (no return motion). A horizontal displacement less than a preset value (e.g., 10mm) ensures the movement is primarily vertical, preventing misidentification as a slide.

[0069] Double-click recognition: Both consecutive motion segments meet the characteristics of a single click, the pause time between the two segments is within 100~400ms, and the spatial center distance is less than 15mm; Double-click identification parameters: two consecutive motion segments, each segment conforming to the "single click" characteristic; the pause time between the two segments is within 100~400ms; the spatial center distance between the two segments is <15mm.

[0070] Slide recognition: The speed remains stable during motion, the trajectory curvature is less than a preset threshold, and horizontal or vertical sliding is distinguished based on the X-axis and Y-axis displacement ratio; Specific parameters for sliding recognition: Velocity remains stable during motion (small acceleration); motion trajectory curvature <1.3; direction angle judgment: if... For horizontal sliding (further subdivided into left / right); if For vertical sliding (up / down); otherwise, for diagonal sliding.

[0071] Axial motion ratio calculation: like It is a horizontal slide; It slides vertically; otherwise, it slides diagonally.

[0072] Rotation recognition: Two-finger mode, the change in the distance between two fingers is less than a preset ratio, and the cumulative angle change exceeds 120°; Rotation recognition parameters: Two-finger mode, with minimal change in the distance between the two fingers (<15%); features include a cumulative angle change >120° and an angle between the endpoints >30°; output is clockwise or counterclockwise rotation. A finger-to-finger distance change < a preset percentage (e.g., 15%) ensures the user intends to rotate rather than zoom. A cumulative angle change >120° is based on actual interaction testing, where angle changes less than 90° are often perceived by users as "fine-tuning" rather than a clear rotation gesture.

[0073] Zoom recognition: Two-finger mode, the midpoint position remains basically unchanged even when the distance between the two fingers changes beyond the preset ratio.

[0074] Zoom recognition specific parameters: two-finger mode, and the distance between the two fingers changes significantly ( The characteristics are that the midpoint is almost stationary, and the spacing increases or decreases monotonically; the output is to enlarge (pull apart) or shrink (pinch together).

[0075] A change in the distance between two fingers greater than a preset ratio (e.g., 20%) ensures that it is recognized as intentional scaling. A relatively constant midpoint position (e.g., a drift of <10mm) is used to distinguish between scaling and two-finger swipes.

[0076] Long press recognition: If a single finger remains still for more than 500ms, and the Z-height remains unchanged or slightly downwards without producing obvious horizontal movement, it is recognized as a long press.

[0077] Customizable compound gestures: such as drawing circles, V-shapes, etc., with optional upgrades to lightweight HMM or DTW matching.

[0078] In this embodiment, multimodal feature fusion is used to resist interference, including synchronously acquiring amplitude features, phase difference features and electrode gradient features, and combining them with human-specific dielectric frequency response features for judgment; the baseline dynamic update strategy includes: periodically or when an abnormal signal is detected, recalibrating the baseline to eliminate baseline drift caused by temperature and humidity changes.

[0079] In this embodiment, multimodal feature acquisition is performed, and the following three types of features are acquired simultaneously: amplitude feature, which is the absolute value of the change in capacitance of the sensing electrode; phase difference feature, which is the phase difference of the measurement signals of adjacent electrodes; and electrode gradient feature, which is the gradient direction and magnitude of the capacitance change between adjacent electrodes.

[0080] These three types of features reflect the characteristics of finger perturbation from different perspectives. Feature-level fusion can improve the reliability and anti-interference ability of detection.

[0081] The human body possesses unique dielectric frequency response characteristics. Human fingers, primarily composed of water and electrolytes, exhibit distinctive dielectric frequency response properties. When a finger enters an electric field region, the dielectric response varies at different frequencies. This characteristic distinguishes it from the dielectric response of common objects (such as metals and plastics), thus enabling the screening of human targets.

[0082] The definition of the specific dielectric frequency response characteristics of the human body is as follows: Under alternating electric fields of different frequencies, the complex permittivity of human tissues (mainly muscles and blood) exhibits specific relaxation characteristics (β dispersion, mainly occurring in the 100kHz~100MHz frequency band). This characteristic is represented by an N-dimensional eigenvector: F=[ΔC(f1),ΔC(f2),...,ΔC(fN)], where ΔC(fi) is the normalized value of the combined capacitance change of all sensing channels at frequency fi. Typically, f1=100kHz, f2=500kHz, and f3=2MHz are chosen.

[0083] The binary classification model uses a lightweight support vector machine, and its structure is as follows: Input layer: Receives the above-mentioned 3D dielectric frequency response feature vector F.

[0084] Kernel function: Radial basis function , where γ is the kernel parameter, with a value of 0.5.

[0085] Output layer: Outputs a category label L∈{0,1}, where 0 represents “non-human interference” and 1 represents “human gesture”.

[0086] Training and Decision-Making: During production, this SVM model was trained offline by collecting over 10,000 sets of human gesture data and an equal number of common interference object data (metal, water droplets, plastic boards, etc.). In actual operation, the system extracts a feature vector F for each frame and inputs it into the model. Only when the output is 1 is the current signal considered a valid gesture input for subsequent localization and recognition processes. This model has extremely low computational cost and can run during the embedded processor's idle time or each time it is woken up, effectively filtering out false wake-ups.

[0087] By combining the unique dielectric frequency response characteristics of the human body, interference from non-human objects can be eliminated, thus improving the system's anti-interference capability.

[0088] Baseline dynamic update strategy By employing differential measurement with adjacent electrodes, common-mode suppression of power frequency interference and radio frequency noise in the environment is achieved.

[0089] The baseline dynamic update strategy includes: recalibrating the baseline periodically or when an abnormal signal is detected, and eliminating baseline drift caused by changes in temperature and humidity.

[0090] This strategy ensures that weak capacitive disturbances can still be detected in complex electromagnetic environments, and static interference does not require global recalibration.

[0091] The present invention also provides a gesture control system based on a capacitive wave array, comprising a capacitive wave array including a transmitting electrode, a sensing electrode, and a shielding layer; the transmitting electrode and the sensing electrode are arranged alternately to form a mosaic checkerboard structure; an AC voltage is applied to the transmitting electrode to establish a quasi-electrostatic field in space; the shielding layer is located on the back and edge of the array to suppress external radiation and external environmental interference; The emitting and sensing electrodes are arranged alternately to form a mosaic checkerboard structure: the entire array is arranged at equal intervals along the X and Y directions; the emitting electrodes are located on the grid nodes, and the center distance (i.e., spacing) between adjacent emitting electrodes is 10mm~15mm; the sensing electrodes are located in the gaps between the emitting electrodes, that is, one rhombus sensing electrode is placed at each of the four adjacent positions (up, down, left, and right) of each rhombus emitting electrode; the emitting and sensing electrodes are distributed alternately to form a "mosaic" checkerboard, with each sensing electrode surrounded by four emitting electrodes, and vice versa.

[0092] An alternating voltage is applied to the emitting electrode to establish a quasi-electrostatic field in space.

[0093] The shielding layer is located on the back and edges of the array to suppress external radiation and interference from the external environment.

[0094] Signal acquisition unit: High-speed acquisition of capacitance change signals on all sensing electrodes, converting weak electric field disturbances into digital quantities; employing differential measurement of adjacent electrodes to suppress common-mode interference; Signal processing and gesture recognition unit: includes an embedded processor that runs positioning and gesture recognition algorithms; the embedded processor is configured to perform the method steps of any one of claims 1 to 8.

[0095] The distance between adjacent centers of the emitting electrodes is 10mm to 15mm; The sensing electrode is located in the gap between adjacent emitting electrodes, with a gap distance of 0.5mm to 1mm; To avoid direct short circuits while maintaining strong capacitive coupling, both the emitting and inductive electrodes are fabricated on the top layer of the same PCB, ensuring that the electric field is primarily emitted upwards into space.

[0096] The shielding layer is a single piece of copper foil at the bottom of a double-layer PCB, or conductive cloth or copper foil tape bonded to the back of an FPC. The shielding layer is connected to the system signal ground through multiple grounding points. The specific connection method is to drill a via every 20-30mm on the copper foil on the back of the array PCB and directly connect it to the ground network of the top layer (electrode layer). This ground network is connected to the reference ground of the analog front-end circuit at a single point, and then connected to the digital ground and power ground in a star configuration on the motherboard.

[0097] The signal processing and gesture recognition unit divides the detection space into field areas based on the height of the finger from the capacitor wave array. The field areas are divided into: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm. The signal processing and gesture recognition unit also includes: a dictionary storage module, used to store a pre-built, incomplete dictionary D. The dictionary can be stored in blocks (by spatial region) to facilitate fast local search without loading the entire dictionary into memory. The sparse matching module is used to implement the orthogonal matching tracking algorithm to perform sparse decomposition on the observation vector. Each frame obtains a C-dimensional observation vector (the real-time value after differencing all sensing channels with the baseline). The goal is to find a set of sparse coefficients (M-dimensional, mostly 0, with non-zero elements corresponding to finger positions) such that... ,and The value of each non-zero coefficient is proportional to the perturbation intensity of the finger at that position; The trajectory management module maintains the trajectory ID of each finger and records its historical position sequence. It maintains a sliding window of length N (N=1530 frames, corresponding to 0.3-0.6 seconds) to store the historical position of each finger. Nearest neighbor matching or Kalman filtering is used to associate the finger detected in the current frame with the trajectory of the previous frame, maintaining the finger ID. If a finger is not matched for N consecutive frames (e.g., 3 frames), it is considered to have disappeared; if a newly appearing finger is more than a threshold away from all existing trajectories, a new trajectory is created. The state machine module is used to implement finite state machine logic and output gesture recognition results. It identifies the gesture type based on the stored motion segment features and outputs gesture commands such as single click, double click, swipe, rotate, and zoom.

[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for air gesture control based on a capacitor wave array, characterized in that, include: Step 1. Electric field establishment steps: After the capacitor wave array is powered on, the transmitting electrode emits an AC voltage of a specific frequency and phase into space, establishing a quasi-electrostatic field above the array; Step 2. Initialization calibration steps: Measure the background capacitance value of all sensing electrodes in a fingerless state to establish an initial baseline, and set the transmit power and sampling rate to low-power polling mode; Step 3. Dynamic wake-up step: Divide the capacitor wave array into several independent sub-regions, and set a sentinel electrode in each sub-region; monitor the capacitance change rate of the sentinel electrode using the adjacent electrode differential measurement method; when the capacitance change rate of any sentinel electrode exceeds a preset threshold, wake up the corresponding sub-region and its adjacent sub-regions, and increase the transmission power and sampling rate to the normal working level. Step 4. 3D positioning steps: Obtain the capacitance change values ​​of all sensing electrodes in the current frame, subtract the dynamically updated background baseline to obtain the observation vector; perform sparse matching between the observation vector and the pre-stored overcomplete dictionary, use the orthogonal matching pursuit algorithm to solve the sparse coefficients, and extract the 3D coordinates of the finger from the sparse coefficients. Step 5. Trajectory tracking steps: Connect the three-dimensional coordinates of consecutive frames to form a motion trajectory, use a low-pass filter to smooth the trajectory, and divide the trajectory into motion segments and pause segments according to the trajectory temporal characteristics; Step 6. Gesture recognition steps: Classify each motion segment based on a finite state machine, and determine the gesture based on the field area where the finger is currently located using an adaptive threshold for the field area; Step 7. Power consumption revert step: If all fingers leave the detection area for more than a set time, the system will switch back to low power polling mode.

2. The air gesture control method based on a capacitor wave array according to claim 1, characterized in that: In Step 1, different excitation frequencies and transmission powers are dynamically matched according to the field area where the finger is located at the height of the array: the field area is divided into: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm. The near-field uses a higher excitation frequency in the range of 100kHz to 2MHz, and the transmit voltage is 1 to 2Vpp; The mid-field uses a medium excitation frequency in the range of 100kHz to 500kHz, and the transmission voltage is 2 to 3Vpp; The far field uses a lower excitation frequency below 100kHz, and the transmit voltage is 3~5Vpp.

3. The air gesture control method based on a capacitor wave array according to claim 1, characterized in that: In Step 3, in the low-power polling mode, the transmit voltage is 1Vpp, the duty cycle is 10%, and the sensing channel sampling rate is 10Hz; under normal operating conditions, the transmit voltage is 3~5Vpp, the sensing channel sampling rate of the woken sub-region is 100Hz, and the sensing channel sampling rate of the adjacent sub-region is 50Hz; if a sub-region does not detect a valid disturbance within 500ms and there is no activity in the adjacent sub-region, it is reduced back to low-power mode.

4. The air gesture control method based on a capacitor wave array according to claim 3, characterized in that: The method for constructing the overcomplete dictionary includes: Discretize the three-dimensional space above the array into grid points, and set the X direction... One point, Y-direction setting Each point, set in the Z direction. A standard probe is moved sequentially to each grid point using a robotic arm, recording the capacitance changes of all sensing channels to form an atomic vector. Each atomic vector is then L2 normalized to obtain a dictionary matrix D of size [value missing]. Where C is the number of sensing channels. The total number of atoms in the dictionary; The execution steps of the orthogonal matching pursuit algorithm include: Initialize residual Support set , where y is the observation vector of the current frame; Calculate the inner product vector of all atoms and residuals. Find the atom index with the largest absolute value of the inner product. Add support set ; Compute the least-squares projection over the subspace spanned by the current support set: Update residuals If the number of iterations reaches the preset sparsity K or the residual L2 norm, If the iteration stops, then stop. The residual L2 norm ε ranges from 0.05 to 0.2; Methods for extracting the 3D coordinates of a finger from sparse coefficients include: Atoms with coefficients whose absolute values ​​are less than a preset proportion of the maximum coefficient are set to zero; on the 3D mesh, non-zero coefficient points connected by 26 neighborhoods are grouped into a cluster; using the coefficient values ​​as weights, the 3D coordinate weighted centroid of each cluster is calculated: in The physical coordinates of grid point j , This corresponds to the sparsity coefficient value.

5. The air gesture control method based on a capacitor wave array according to claim 2, characterized in that: The field area adopts an adaptive threshold configuration, including: Motor initiation threshold : Near field 15mm / s, mid field 25mm / s, far field 40mm / s; pause detection threshold : 10mm / s near the field, 15mm / s mid-field, and 25mm / s far from the field.

6. The air gesture control method based on a capacitor wave array according to claim 4, characterized in that: Based on the obtained three-dimensional coordinates of the finger, a finite state machine is used for trajectory segmentation and gesture recognition, including idle state, motion tracking state, and state to be recognized: The idle state: when the speed exceeds the motion initiation threshold, it enters the motion tracking state; The motion tracking state: when the speed is lower than the pause threshold and the duration exceeds the first preset value, it enters the identification state; The state to be identified: the gesture type is identified based on the stored motion segment features, or it is identified as a double-click when a new motion segment is detected within a second preset time.

7. The air gesture control method based on a capacitor wave array according to claim 6, characterized in that: In Step 6, the gesture recognition rules based on geometric features include: Click recognition: Single finger mode, movement duration less than 200ms, obvious downward pressure and return in the Z-axis direction, horizontal displacement less than the preset value, and the speed profile shows the characteristics of first accelerating and then decelerating. Double-click recognition: Both consecutive motion segments meet the characteristics of a single click, the pause time between the two segments is within 100~400ms, and the spatial center distance is less than 15mm; Slide recognition: The speed remains stable during motion, the trajectory curvature is less than a preset threshold, and horizontal or vertical sliding is distinguished based on the X-axis and Y-axis displacement ratio; Rotation recognition: Two-finger mode, the change in the distance between two fingers is less than a preset ratio, and the cumulative angle change exceeds 120°; Zoom recognition: Two-finger mode, the midpoint position remains basically unchanged even when the distance between the two fingers changes beyond the preset ratio.

8. The air gesture control method based on a capacitor wave array according to claim 6, characterized in that: Multimodal feature fusion is used to resist interference, including synchronous acquisition of amplitude features, phase difference features and electrode gradient features, and combined with human body-specific dielectric frequency response features for judgment; The baseline dynamic update strategy includes: periodically or when an abnormal signal is detected, recalibrating the baseline to eliminate baseline drift caused by changes in temperature and humidity.

9. A gesture control system based on a capacitor wave array, characterized in that: Including Capacitive wave array: includes emitting electrodes, sensing electrodes, and a shielding layer; the emitting electrodes and sensing electrodes are arranged alternately to form a mosaic checkerboard structure; an AC voltage is applied to the emitting electrodes to establish a quasi-electrostatic field in space; the shielding layer is located on the back and edge of the array to suppress external radiation and external environmental interference; Signal acquisition unit: High-speed acquisition of capacitance change signals on all sensing electrodes, converting weak electric field disturbances into digital quantities; employing differential measurement of adjacent electrodes to suppress common-mode interference; Signal processing and gesture recognition unit: includes an embedded processor that runs positioning and gesture recognition algorithms; the embedded processor is configured to perform the method steps of any one of claims 1 to 8.

10. A gesture control system based on a capacitor wave array according to claim 9, characterized in that: The distance between adjacent centers of the emitting electrodes is 10mm to 15mm; The sensing electrode is located in the gap between adjacent emitting electrodes, with a gap distance of 0.5mm to 1mm; The emitting electrode and the sensing electrode are both fabricated on the top layer of the same PCB. The shielding layer is a single piece of copper foil at the bottom of a double-layer PCB, or conductive cloth or copper foil tape bonded to the back of an FPC. The shielding layer is connected to the system signal ground via multiple grounding points; The signal processing and gesture recognition unit divides the detection space into field areas based on the height of the finger from the capacitor wave array. The field areas are divided into: near field height 0~30mm, mid field height 30~100mm, and far field height 100~200mm. The signal processing and gesture recognition unit further includes: a dictionary storage module for storing a pre-built, incomplete dictionary D; The sparse matching module is used to implement the orthogonal matching pursuit algorithm to perform sparse decomposition on the observation vector; The trajectory management module is used to maintain the trajectory ID of each finger and record the historical position sequence; The state machine module is used to implement finite state machine logic and output gesture recognition results.

Images