Gesture tracking method and system based on acoustic signals

Abstract

This invention proposes a gesture tracking method and system based on acoustic signals. It fuses distance, angle, and velocity information from signals reflected from multiple targets using a triangular chirped signal model, analyzes the signals through a joint estimation algorithm, and achieves signal separation through signal cancellation and reconstruction to obtain multi-dimensional parameters for each target. This invention breaks through the limitations of motion tracking using acoustic signals, enabling continuous real-time tracking of multiple targets simultaneously with centimeter-level accuracy.

Description

Technical Field

[0001] This invention relates to the field of wireless sensing technology, specifically to a gesture tracking method and system based on acoustic signals. Background Technology

[0002] As the Internet of Things (IoT) continues to penetrate various industries, and the level of industry informatization and networking continues to improve, smart devices and their applications have become an indispensable part of people's lives. In the current era of ubiquitous connectivity, smart devices such as smartwatches, smartphones, smart cars, and smart homes have become essential elements of daily life. The rapid development of these devices and applications has fundamentally changed people's lives, which has also greatly stimulated research into human-computer interaction (HCI) technology. People hope to better understand and apply these smart devices through various technologies to achieve more efficient and convenient HCI interaction.

[0003] As an intersection of the Internet of Things (IoT) and artificial intelligence (AI), wireless sensing and identification technology is a crucial component of human-computer interaction (HCI) technology. Compared to traditional sensor-based sensing and identification, the biggest advantage of wireless sensing and identification technology lies in its device-free and contactless nature. It eliminates the need for dedicated sensors and wired connections when sensing and identifying people and the environment, and users do not need to wear any devices. Sensing is achieved using wireless signals, such as sound, light, and radio frequency (RF) signals. Taking RF signals as an example, after being emitted by a transmitter, the signal propagates through space via various paths, including direct transmission, reflection, and scattering. These signals, after reaching the receiver through different paths, superimpose to form a composite signal carrying spatial information. Wireless sensing and identification technology analyzes the changes in the signal during propagation to obtain distance, speed, and angle information, enabling high-precision positioning, gesture capture, motion recognition, and detection and tracking of passive objects. This allows for a wide range of new services, such as scene perception and environmental reconstruction. After years of development, the fundamental and verification work on wireless sensing and identification technologies has been completed, and applications have been achieved in many fields. For example, in the medical field, wireless sensing technology can detect patients' vital signs and enable remote monitoring and diagnosis; in smart home scenarios, it enables home security monitoring and control and management of smart home devices such as smart TVs and smart curtains. The development of wireless sensing and identification technologies has propelled human perception of the physical world into a new stage of comprehensive intelligence, and has also created more possibilities for the integration of human-computer interaction and intelligence.

[0004] With the increasing prevalence of smart terminals such as mobile phones, speakers, and wearable devices, intelligent sensing technologies based on various sensing media have attracted widespread attention from researchers. For non-contact tracking and identification, acoustic signals have inherent advantages in terms of sensing granularity and tracking accuracy among different types of sensing media due to their stronger sensitivity and adaptability to environmental changes. Furthermore, acoustic signals have a lower propagation speed in the air compared to other wireless signals, such as WiFi and RFID. Acoustic sensing can also utilize ubiquitous audio devices without competing for radio resources with other household devices, and recent efforts have improved acoustic sensing accuracy to the millimeter level. By transmitting and receiving signals through speakers and microphones, acoustic signal-based intelligent identification has expanded to a wider range of applications, such as acoustic localization, non-contact motion tracking, gesture recognition and posture recognition, as well as the monitoring of important human physiological parameters (e.g., driver fatigue detection and respiratory activity).

[0005] Today, gesture control has become a research hotspot in the field of wireless sensing and interaction. This device-free technology allows users to control devices using actions or gestures, and is considered a natural, convenient, and promising human-computer interaction method, already widely used in many application scenarios. For example, in the smart home field, gesture tracking and recognition technology allows users to control smart home devices, such as turning them on and off, adjusting volume and brightness, without touching the devices. This is particularly useful when the user's hands are dirty, wet, or inconvenient to touch the devices. In the virtual reality field, gesture tracking and recognition technology allows users to manipulate objects in the virtual environment through gestures, providing a more natural interaction method for virtual reality games and training.

[0006] Existing gesture tracking and recognition methods are mainly divided into two categories: device-based and deviceless (non-contact tracking and recognition). Device-based tracking uses wearable sensors. However, wearable sensor-based methods are often cumbersome due to the inconvenience of wearing sensors. Deviceless gesture tracking and recognition can be broadly categorized into three types: vision-based, radio frequency (RF) signal-based, and acoustic signal-based. Vision-based methods are susceptible to poor lighting conditions or occlusion. RF-based methods require Wi-Fi or specialized equipment; furthermore, Wi-Fi measurements are too coarse-grained to provide fine accuracy or recognize small gestures. In recent years, some acoustic signal-based gesture recognition systems have been proposed. Compared to sensor-based, vision-based, and RF-based methods, acoustic signal-based methods have the advantage of not requiring good lighting conditions or any devices worn by the user, providing fine-grained gesture recognition by analyzing reflected signals from the hand. Summary of the Invention

[0007] The technical problem to be solved by this invention is how.

[0008] The present invention solves the above-mentioned technical problems through the following technical means:

[0009] The gesture tracking method based on acoustic signals includes the following steps:

[0010] The mixed signal calculation steps are as follows: The denoised triangular chirped signal is divided into an upper chirped signal and a lower chirped signal. The separated upper and lower chirped signals are orthogonally demodulated to obtain a preliminary mixed signal. The high-frequency part of the obtained preliminary mixed signal is filtered out to obtain the in-phase and quadrature parts of the signal. The in-phase and quadrature parts of the upper and lower chirped signals are mixed to obtain the upper chirped mixed signal and the lower chirped mixed signal. The mixed signal of multiple targets includes multiple upper chirped mixed signals and multiple lower chirped mixed signals. The flight time of the reflected signal can be obtained from the upper and lower chirped mixed signals.

[0011] The signal separation steps are as follows: First, eliminate noise from the chirped up and down mixed signals, and then search for distance information within the set parameter range. r Angle information and speed information v The optimal parameters are based on distance information. r Speed ​​information v The optimal parameters are then used to calculate the signal attenuation factor. When there are multiple targets, the strongest reflected signal in the mixed signal of the current multiple targets is searched each time through an iterative method. The distance, angle and velocity information parameters that match the signal are obtained, and then the signal attenuation factor of the reflected signal is calculated. After the iteration is completed, the parameters of different targets can be obtained.

[0012] Steps for target matching and tracking: using weighted l1-norm

[0013]

[0014] in Indicates in t The vector consisting of the estimated values ​​of four parameters at time t: signal attenuation factor, distance information, angle information, and velocity information, is represented as follows: , To normalize the differences in the four dimensions into a scale vector of the same scale, the parameter vector obtained by jointly estimating the two continuous signal periods with the smallest l1 distance is regarded as belonging to the same target.

[0015] Furthermore, the calculation process of the mixed signal in the mixed signal calculation step is as follows:

[0016] First, add the chirp signal. The 90° phase-shifted version of the transmitted signal is obtained by performing Hilbert transform on its own signal and the chirped signal respectively. Multiplying them yields a first preliminary mixed signal and a second preliminary mixed signal, respectively. The first and second preliminary mixed signals are then passed through a low-pass filter to obtain the in-phase components. and orthogonal part The in-phase and quadrature components are combined using a complex function to obtain the upper chirped mixed signal; the lower chirped mixed signal is obtained in the same way; the expression is:

[0017]

[0018]

[0019] In the formula, For chirped mixed signals, This is a chirped mixed signal.

[0020] Furthermore, the multi-target mixed signal is represented as follows: assuming there exists L For each target, the signal received by each microphone should be a composite signal formed by the superposition of the reflected signals from all targets. i When the signal sampling point is _____, the _____ signal sampling point is _____. k The mixed signal of all target reflections received by each microphone is represented as:

[0021]

[0022] in, α l , r l , v l They represent the first l The amplitude attenuation factor, distance information, and velocity information of the reflected signal from each target.

[0023] Furthermore, the calculation process for the signal attenuation factor of a single target in the signal separation step is as follows:

[0024] Suppose a triangular chirped signal contains 2N _________ sample points, where the upper chirped part and the lower chirped part each contain _________ sample points. N A sample, with... K There are several microphones; first consider the chirp component, for each microphone there is... N For each possible set of signal sample points, ( ) r , v Based on the signal's time of flight, the phase change is reconstructed, thus each microphone can reconstruct a phase change estimation vector. The size of the vector is NThe signal measurement value of each sample is multiplied by the conjugate of the signal phase change reconstructed according to the parameters to eliminate the phase shift generated by each sample in the distance and velocity dimensions, and all the results are added together to obtain the joint estimate of the upper chirped part. The same applies to the lower chirped part.

[0025]

[0026] The result of the joint estimator is obtained by summing the absolute values ​​of the joint estimates of the upper and lower chirped parts, which can be expressed as follows:

[0027]

[0028] Joint estimator The result will be in a set of optimal parameters Maximize the time, because at this point all signals are in phase; therefore, the problem of finding the optimal parameters can be expressed as:

[0029]

[0030] Finally, after obtaining two optimal parameters After obtaining the estimated values, signal attenuation is calculated using these estimates.

[0031] .

[0032] Furthermore, the method for multi-target signal separation is as follows:

[0033] First, the path and parameters corresponding to the strongest signal in the mixed signal are obtained based on the joint estimator. The strongest path signal is reconstructed based on the parameters obtained from the formula for chirped mixed signals. The reconstructed signal is then subtracted from the original mixed signal to obtain the remaining mixed signal. Then, the joint estimator is used again to estimate the strongest path signal from the remaining signal to perform joint estimation of the next target and obtain the parameter information of the next target. The joint estimation ends when the power of the remaining signal is less than a predefined threshold.

[0034] The present invention also provides a gesture tracking system based on acoustic signals, comprising:

[0035] The mixed signal calculation module: The denoised triangular chirped signal is divided into an upper chirped signal and a lower chirped signal. The separated upper and lower chirped signals are orthogonally demodulated to obtain a preliminary mixed signal. The high-frequency components of the preliminary mixed signal are filtered out, yielding the in-phase and quadrature components of the signal. The in-phase and quadrature components of the upper and lower chirped signals are then mixed to obtain the upper chirped mixed signal and the lower chirped mixed signal. The mixed signal for multiple targets includes multiple upper chirped mixed signals and multiple lower chirped mixed signals. The flight time of the reflected signal can be obtained from the upper and lower chirped mixed signals.

[0036] The signal separation module first eliminates noise from the chirped up-and-down mixed signals, and then searches for distance information within the set parameter range. r Angle information and speed information v The optimal parameters are based on distance information. r Speed ​​information v The optimal parameters are then used to calculate the signal attenuation factor. When there are multiple targets, the strongest reflected signal in the mixed signal of the current multiple targets is searched each time through an iterative method. The distance, angle and velocity information parameters that match the signal are obtained, and then the signal attenuation factor of the reflected signal is calculated. After the iteration is completed, the parameters of different targets can be obtained.

[0037] Target matching and tracking module: using weighted l1-norm

[0038]

[0039] in Indicates in t The vector consisting of the estimated values ​​of four parameters at time t: signal attenuation factor, distance information, angle information, and velocity information, is represented as follows: , To normalize the differences in the four dimensions into a scale vector of the same scale, the parameter vector obtained by jointly estimating the two continuous signal periods with the smallest l1 distance is regarded as belonging to the same target.

[0040] Furthermore, the calculation process of the mixed signal in the mixed signal calculation module is as follows:

[0041] First, add the chirp signal. The 90° phase-shifted version of the transmitted signal is obtained by performing Hilbert transform on its own signal and the chirped signal respectively. Multiplying them yields a first preliminary mixed signal and a second preliminary mixed signal, respectively. The first and second preliminary mixed signals are then passed through a low-pass filter to obtain the in-phase components. and orthogonal part The in-phase and quadrature components are combined using a complex function to obtain the upper chirped mixed signal; the lower chirped mixed signal is obtained in the same way; the expression is:

[0042]

[0043]

[0044] In the formula, For chirped mixed signals, This is a chirped mixed signal.

[0045] Furthermore, the multi-target mixed signal is represented as follows: assuming there exists L For each target, the signal received by each microphone should be a composite signal formed by the superposition of the reflected signals from all targets. i When the signal sampling point is _____, the _____ signal sampling point is _____. k The mixed signal of all target reflections received by each microphone is represented as:

[0046]

[0047] in, α l , r l , v l They represent the first l The amplitude attenuation factor, distance information, and velocity information of the reflected signal from each target.

[0048] Furthermore, the calculation process for the signal attenuation factor of a single target in the signal separation module is as follows:

[0049] Suppose a triangular chirped signal contains 2N _________ sample points, where the upper chirped part and the lower chirped part each contain _________ sample points. N A sample, with... K There are several microphones; first consider the chirp component, for each microphone there is... N For each possible set of signal sample points, ( ) r , v Based on the signal's time of flight, the phase change is reconstructed, thus each microphone can reconstruct a phase change estimation vector. The size of the vector is N The signal measurement value of each sample is multiplied by the conjugate of the signal phase change reconstructed according to the parameters to eliminate the phase shift generated by each sample in the distance and velocity dimensions, and all the results are added together to obtain the joint estimate of the upper chirped part. The same applies to the lower chirped part.

[0050]

[0051] The result of the joint estimator is obtained by summing the absolute values ​​of the joint estimates of the upper and lower chirped parts, which can be expressed as follows:

[0052]

[0053] Joint estimator The result will be in a set of optimal parameters Maximize the time, because at this point all signals are in phase; therefore, the problem of finding the optimal parameters can be expressed as:

[0054]

[0055] Finally, after obtaining two optimal parameters After obtaining the estimated values, signal attenuation is calculated using these estimates.

[0056] .

[0057] Furthermore, the method for multi-target signal separation is as follows:

[0058] First, the path and parameters corresponding to the strongest signal in the mixed signal are obtained based on the joint estimator. The strongest path signal is reconstructed based on the parameters obtained from the formula for chirped mixed signals. The reconstructed signal is then subtracted from the original mixed signal to obtain the remaining mixed signal. Then, the joint estimator is used again to estimate the strongest path signal from the remaining signal to perform joint estimation of the next target and obtain the parameter information of the next target. The joint estimation ends when the power of the remaining signal is less than a predefined threshold.

[0059] The advantages of this invention are:

[0060] This invention realizes a novel non-contact tracking method based on acoustic triangular chirped signals. It breaks through the limitations of motion tracking using acoustic signals, enabling continuous real-time tracking of multiple targets simultaneously with centimeter-level accuracy. The invention verifies its tracking performance through simulation and real-world scenarios, demonstrating its feasibility in practical applications and confirming its significant potential for deployment on smart devices. It provides a technical solution for a more natural and user-friendly human-computer interaction method from the perspective of acoustic wireless sensing. Specifically, this invention proposes a signal model based on triangular chirped signals to fuse distance, angle, and velocity information from signals reflected from multiple targets. A joint estimation algorithm is used to analyze the signals, and signal separation is achieved through signal cancellation and reconstruction to obtain multi-dimensional parameters for each target.

[0061] This invention proposes solutions to some problems related to the tracking accuracy of interference systems. For example, the alignment algorithm can effectively solve the start time error caused by the inherent limitations of hardware devices and software coding, and the background multipath subtraction can subtract the direct paths contained in the mixed reflection signal as well as most of the noise and other interference. Attached Figure Description

[0062] Figure 1 A flowchart of an acoustic signal-based gesture tracking method in an embodiment of the present invention;

[0063] Figure 2 The following is the calculation process for mixed signals in this embodiment of the invention (taking the above chirped mixed signal as an example, the calculation process for the following chirped mixed signal is the same).

[0064] Figure 3 This is the iterative process for joint estimation of multi-objective parameters in this embodiment of the invention;

[0065] Figure 4 This is a comparison of the signal reconstruction process in the embodiments of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Definitions:

[0068] Acoustic wireless sensing is a technology that uses sound waves to detect, locate, and identify objects. It obtains information about target objects by analyzing how sound waves propagate through space. Acoustic wireless sensing can be used in many fields, including wireless communication, security monitoring, and medical diagnostics.

[0069] FMCW (Frequency-Modulated Wave) is a commonly used wireless communication technology, primarily used for radar ranging and velocity measurement. In FMCW, the signal frequency is continuously modulated and changes linearly with time at a certain rate. Because single-string signals are highly periodic, one flight time corresponds to the same phase in multiple periods, resulting in ambiguity in range calculation. Compared to single-string signals, FMCW signals have unique advantages. Since the signal frequency of FMCW waves changes continuously, the distance and velocity of a target can be calculated by measuring the frequency and phase differences of the signals, achieving higher measurement accuracy and resolution. This characteristic also reduces interference from signals arriving via multiple paths. Each path has a different arrival time and generates different signal frequency differences, allowing the signal receiver to distinguish between multiple paths, thus improving signal reliability.

[0070] Time-of-flight (TOF) refers to the time it takes for a wireless signal to travel from the transmitter to the receiver. In radar, wireless communication, and positioning systems, accurately measuring TOF is crucial for determining target location and distance. By measuring TOF, the distance the signal travels in space can be calculated, which can then be used for applications such as positioning, ranging, and velocity measurement.

[0071] The angle of arrival (AOA) is the directional angle from which a signal travels from the transmitting antenna to the receiving antenna. In wireless communication systems, understanding the AOA helps optimize antenna layout and signal coverage, improving communication quality and system performance. The AOA is also an important parameter in radar and positioning systems, used to determine the azimuth angle of a target.

[0072] The Doppler effect refers to the phenomenon where the signal frequency changes when a signal source or receiver moves relative to itself or another source. When the source or receiver moves towards itself, the signal frequency increases; this is called the positive Doppler effect. When the source or receiver moves away from itself, the signal frequency decreases; this is called the negative Doppler effect. The Doppler effect has important applications in radar, wireless communication, and mobile communication systems, such as for speed measurement, frequency offset compensation, and speed estimation in mobile communications.

[0073] This embodiment provides a gesture tracking method based on acoustic signals, such as... Figure 1 As shown, the steps are as follows:

[0074] Mixed signal computation steps (i.e.) Figure 1 (Mathematical Signal Modeling Section): This implementation uses continuous triangular chirped signals. Each transmitted triangular chirped signal consists of an upper chirped portion with a linearly increasing signal frequency and a lower chirped portion with a linearly decreasing signal frequency. First, the signal received by the microphone is passed through a high-pass filter to remove some low-frequency noise. Based on the upper and lower chirped portions of the transmitted signal, the received signal is truncated at sampling points to form an upper chirped received signal and a lower chirped received signal. The separated upper and lower chirped signals are then quadrature demodulated separately. The resulting preliminary mixed signal is passed through a low-pass filter to filter out the high-frequency components, obtaining the in-phase and quadrature components of the signal. These components are then combined to obtain the mixed signal.

[0075] The joint estimation process for separating the signals involves the following steps: The joint estimation algorithm used in this implementation is designed based on the characteristics of triangular chirped signals. It can progressively separate reflected signals from mixed signals from multiple targets and estimate target parameters from these reflected signals. Before performing joint estimation on the mixed signals, the system first eliminates background multipath interference. This interference includes direct path interference from the loudspeaker to the microphone and reflection interference generated from the static environment. Then, a joint estimator is constructed to search for the optimal parameters for distance, angle, and velocity within the parameter range, and finally, signal attenuation is calculated. When there are multiple targets, the strongest reflected signal of the current signal is searched for each time using an iterative method to obtain the parameters that match that signal. The joint estimation steps are repeated with the remaining signals to obtain the parameters for different targets.

[0076] The steps of target matching and tracking: To continuously track moving targets and acquire their trajectories, the system needs to continuously perform joint estimation to obtain the target's position and motion parameters at different time points. When multiple targets need to be tracked, the system's joint estimation algorithm outputs a set of parameters for multiple targets. The key challenge lies in how to correlate the position and motion parameter estimates of the same target at different time stamps, especially when the trajectories of multiple targets overlap. This implementation addresses this issue by using parameters with more dimensions, because within a given time stamp, it is rare for multiple targets to have similar parameters across all dimensions. For example, it resolves the ambiguity caused by distance similarity in two-dimensional range space and angle of incidence (AoA).

[0077] The steps described above are described in detail below.

[0078] Without loss of generality, a dedicated Seeed Respeaker 6-Mic circular microphone array and a Lecoo Raspberry Pi 4B were set up in a typical open space. MATLAB software was run on a computer equipped with an AMD Ryzen 5 3550H processor and 16GB of RAM to process and analyze the data. In this implementation, the triangular chirp signal used by the system had a start frequency of 18kHz, a stop frequency of 22kHz, and a bandwidth of 4kHz. The upper chirp portion of the signal linearly increases in frequency from the start frequency to the stop frequency, lasting 0.04s, while the lower chirp portion linearly decreases in frequency from the stop frequency to the start frequency, also lasting 0.04s. Therefore, a complete signal cycle is 0.08s. By processing the collected sound signals, this implementation can track moving targets, verifying its effectiveness and feasibility.

[0079] Mixed-signal calculation steps: Figure 1 The part that establishes the mathematical signal model.

[0080] This implementation utilizes the unique properties of FMCW waves to measure signal flight time. Signal preprocessing is required before establishing the mathematical signal model. First, the signal received by the microphone is passed through a high-pass filter to remove some low-frequency noise. Both the transmitted and received signals are continuous triangular chirped signals, and subsequent processing involves processing and analyzing the upper and lower chirped components separately. Therefore, it is necessary to separate the upper and lower chirped components of the received signal. Based on the upper and lower chirped components of the transmitted signal, the received signal is divided into upper-chirped and lower-chirped received signals by sampling points.

[0081] The next step is a signal mixing process to obtain the signal flight time of a single target. For example... Figure 2Multiplying the upper and lower chirped portions of the transmitted signal by the corresponding portions of the received signal yields the mixed signal. Specifically, this can be achieved using trigonometric equations, for example: The mixed signal of the upper chirp signal and the lower chirp signal can be represented as follows:

[0082]

[0083]

[0084] in, f 0、 f 1. B , T、α、 These represent the start frequency, stop frequency, bandwidth, duration, amplitude attenuation due to signal reflection, and time of flight of the triangular chirped signal, respectively. f 1 is f 0 and B The sum of.

[0085] like Figure 2 As shown, the above chirping signals For example, first, the chirp signal is applied. The 90° phase-shifted version of the transmitted signal is obtained by performing Hilbert transform on its own signal and the chirped signal respectively. Multiplying them yields a first preliminary mixed signal and a second preliminary mixed signal, respectively. The first and second preliminary mixed signals are then passed through a low-pass filter to obtain the in-phase components. and orthogonal part The in-phase and quadrature components are combined using a complex function to obtain the upper-chirped mixed signal, represented as follows:

[0086]

[0087]

[0088] In the formula, For chirped mixed signals, This is a chirped mixed signal.

[0089] The flight time of the reflected signal can be analyzed from the above mixed signal, and then the distance, angle and speed information of the target can be extracted from the flight time by the step of jointly estimating the separated signal. Figure 2 The above chirped signal is used as an example to illustrate the calculation process of the chirped mixed signal. Similarly, it can be calculated based on... Figure 2 The method was used to calculate the chirped mixed signal.

[0090] The method for calculating the signal time of flight based on a single target described above can be extended to multiple targets. Assuming there exists... L For each target, the signal received by each microphone should be a composite signal formed by the superposition of the reflected signals from all targets. i When the signal sampling point is _____, the _____ signal sampling point is _____. k The mixed signal received by each microphone from all target reflections can be represented as:

[0091]

[0092] in α l , r l , v l They represent the first l The system tracking algorithm collects the signal attenuation factor, distance information, and velocity information of the reflected signals from each target. The ultimate goal is to separate the reflected signal of each individual target from the mixed signals reflected by all targets, and analyze the signals to obtain the corresponding multi-dimensional parameters of each target, thereby obtaining the target's spatial position and motion state information.

[0093] The steps for jointly estimating the separated signals are as follows:

[0094] Before performing joint estimation of the mixed signals, the system first eliminates background multipath interference. This interference includes direct path interference from the speaker to the microphone, as well as reflection interference generated from the static environment. Since the strength of the direct path signal is much higher than that of the various other signals arriving after reflection, the direct path is used as an anchor point to align the received signal.

[0095] For each passing the first i The sampling timestamp of the first k Each microphone samples a signal sample, and the signal can be determined by an attenuation factor. and phase transition We model it, where the phase transition of the signal is determined by the distance between the target and the microphone. r and the speed of the target's movement v Caused by, .in Representative at the i The sampling timestamp of the first k The total phase transition of the signal samples from each microphone and These represent the signal phase transitions caused by distance and velocity, respectively. When the distance and velocity parameters are correctly estimated, the phase variables are calculated based on the estimated parameters. Measured values ​​of actual phase transition They should be approximately equal.

[0096]

[0097] in and These represent the phase variables calculated based on the estimated distance and velocity parameters, respectively.

[0098] If we remove the phase changes based on these accurate estimates from the measurements at each sampling point... , such as equation

[0099]

[0100] in For conjugation operations, a phase value of 0 will be obtained (i.e. The signal is in phase, meaning that the remaining signal after removing the signal reconstructed from the estimated value is in phase. At this point, the signal strength reaches its maximum after superimposing the remaining signals from all sampling points. Based on this principle, distance and velocity parameters can be searched within a pre-set parameter search range. r , v The value of ) is the optimal parameter when the superimposed signal is at its maximum.

[0101] Suppose a triangular chirped signal contains 2N _________ sample points, where the upper chirped part and the lower chirped part each contain _________ sample points. N The experiment included [number] samples. K There are several microphones. First, consider the chirp component; for each microphone, there is... N For each possible set of signal sample points, ( ) r , v The system can reconstruct the phase change based on the signal's time of flight, therefore each microphone can reconstruct a phase change estimation vector. The size of the vector is N The signal measurement value of each sample is multiplied by the conjugate of the signal phase change reconstructed according to the parameters to eliminate the phase shift generated by each sample in the distance and velocity dimensions, and all the results are added together to obtain the joint estimate of the upper chirped part. The same applies to the lower chirped part.

[0102]

[0103] It's important to note that the result of the joint estimator is formed by adding the absolute values ​​of the joint estimates from the upper and lower chirped components. Therefore, a complete triangular chirped signal is required to obtain a set of optimal parameters. The result of the joint estimator, obtained by adding the absolute values ​​of the joint estimates from the upper and lower chirped components, can be expressed as follows:

[0104]

[0105] Joint estimator The result will be in a set of optimal parameters The optimal time is maximized because all signals are in phase at this point. Therefore, the problem of finding the optimal parameters can be expressed as:

[0106]

[0107] Finally, after obtaining two optimal parameters After obtaining the estimated values, the system uses these estimates to calculate signal attenuation.

[0108]

[0109] The single-objective joint estimation algorithm described above can be extended to multi-objective objectives. For example... Figure 3 As shown, the joint estimator first obtains the path and parameters corresponding to the strongest signal in the mixed signal, based on the chirped mixed signal. The strongest path signal is reconstructed from the obtained parameters, and the reconstructed signal is subtracted from the original mixed signal to obtain the remaining mixed signal. Then, the joint estimation algorithm is used again to estimate the strongest path signal from the remaining signal, and joint estimation is performed for the next target to obtain the parameter information of the next target. The joint estimation ends when the power of the remaining signal is less than a predefined threshold.

[0110] In the above formula and These represent theoretical and calculated values, respectively, for ease of description and differentiation. The same applies to other parameters.

[0111] Steps for target matching and tracking:

[0112] Since the target is in motion, its position and motion parameters change over different signal times. To continuously track the target and obtain its trajectory, the system needs to continuously perform joint estimation to obtain the target's position and motion parameters at different times. Within each time window, when multiple targets need to be tracked, the system's joint estimation algorithm outputs a parameter set for multiple targets. Correlating the estimated position and motion parameters of the same target at different timestamps becomes a major challenge for the system, especially when multiple targets have overlapping trajectories. The key idea behind the system's target parameter matching solution is that target motion is continuous (especially human motion). Due to the extremely short signal period (only 0.08s), the difference between estimated parameters obtained from two consecutive periods of the same target should be small. However, when the trajectories of two targets overlap, their parameters become similar, leading to tracking confusion. This implementation uses parameters from more dimensions to solve this problem because, within a given timestamp, few targets have similar parameters across all dimensions. For example, ambiguity caused by distance similarity can be resolved in a higher-dimensional space (i.e., two-dimensional range space and AoA). To quantify the overlap of different target trajectories, the system uses a weighted l1-norm.

[0113]

[0114] in Indicates in t The vector consisting of the estimated values ​​of the four-dimensional parameters (signal attenuation factor, distance information, angle information, and velocity information) at time t is represented as: , To normalize the differences in the four dimensions into a scale vector of the same scale, the parameter vectors obtained by jointly estimating the two consecutive signal periods with the smallest l1 distance are considered to belong to the same target.

[0115] This implementation achieves a novel non-contact tracking method based on acoustic triangular chirp signals. It breaks through the limitations of motion tracking using acoustic signals, enabling continuous real-time tracking of multiple targets simultaneously with centimeter-level accuracy. The implementation validates its tracking performance through simulation and real-world scenarios, demonstrating its feasibility in practical applications and confirming its significant potential for deployment on smart devices. It provides a technical solution for a more natural and user-friendly human-computer interaction method from the perspective of acoustic wireless sensing.

[0116] To implement this system, this implementation proposes a signal model based on triangular chirped signals to fuse distance, angle, and velocity information of signals reflected from multiple targets. The signal is analyzed through a joint estimation algorithm, and signal separation is achieved through signal cancellation and reconstruction to obtain multidimensional parameters of each target.

[0117] This implementation proposes solutions to some problems related to the tracking accuracy of interference systems. For example, the alignment algorithm can effectively solve the start time error caused by the inherent limitations of hardware devices and software coding, and the background multipath subtraction can subtract the direct path contained in the mixed reflection signal as well as most of the noise and other interference.

[0118] This implementation puts the system into practice on a Raspberry Pi equipped with speakers and a microphone array, and conducts comprehensive testing and evaluation of the system in terms of tracking accuracy, resolution, and factors affecting system performance.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gesture tracking method based on acoustic signals, characterized in that, Includes the following steps: Mixed signal calculation steps: The denoised triangular chirped signal is divided into an upper chirped signal and a lower chirped signal. The separated upper and lower chirped signals are quadrature demodulated to obtain a preliminary mixed signal. The high-frequency part of the obtained preliminary mixed signal is filtered out to obtain the in-phase part and the quadrature part of the signal. The in-phase part and the quadrature part of the upper and lower chirped signals are mixed to obtain the upper chirped mixed signal and the lower chirped mixed signal. The mixed signal of multiple targets includes multiple upper-chirped mixed signals and multiple lower-chirped mixed signals; the flight time of the reflected signal can be obtained from the upper and lower-chirped mixed signals; The signal separation steps are as follows: First, eliminate noise from the mixed signals of upper and lower chirps, and then search for distance information within the set parameter range. r Angle information and speed information v The optimal parameters are based on distance information. r Speed ​​information v The optimal parameters are then used to calculate the signal attenuation factor. When there are multiple targets, the strongest reflected signal in the mixed signal of the current multiple targets is searched each time through an iterative method. The distance, angle and velocity information parameters that match the signal are obtained, and then the signal attenuation factor of the reflected signal is calculated. After the iteration is completed, the parameters of different targets can be obtained. Steps for target matching and tracking: using weighted l1-norm in Indicates in t The vector consisting of the estimated values ​​of four parameters at time t: signal attenuation factor, distance information, angle information, and velocity information, is represented as follows: , To normalize the differences in the four dimensions into a scale vector of the same scale, the parameter vector obtained by jointly estimating the two continuous signal periods with the smallest l1 distance is regarded as belonging to the same target.

2. The gesture tracking method based on acoustic signals according to claim 1, characterized in that, The calculation process of the mixed signal in the mixed signal calculation step is as follows: First, add the chirp signal. The 90° phase-shifted version of the transmitted signal is obtained by performing Hilbert transform on its own signal and the chirped signal respectively. Multiplying them yields a first preliminary mixed signal and a second preliminary mixed signal, respectively. The first and second preliminary mixed signals are then passed through a low-pass filter to obtain the in-phase components. and orthogonal part The in-phase and quadrature components are combined using a complex function to obtain the upper chirped mixed signal. The lower chirped mixed signal is obtained using the same method; the expression is: In the formula, For chirped mixed signals, This is a chirped mixed signal.

3. The gesture tracking method based on acoustic signals according to claim 2, characterized in that, Multi-target mixed signals are represented as follows: assuming there exists L For each target, the signal received by each microphone should be a composite signal formed by the superposition of the reflected signals from all targets. i When the signal sampling point is _____, the _____ signal sampling point is _____. k The mixed signal of all target reflections received by each microphone is represented as: in, α l , r l , v l They represent the first l The amplitude attenuation factor, distance information, and velocity information of the reflected signal from each target.

4. The gesture tracking method based on acoustic signals according to claim 3, characterized in that, The calculation process for the signal attenuation factor of a single target in the signal separation step is as follows: Suppose a triangular chirped signal contains 2N 1000 sample points, of which the upper chirped portion and the lower chirped portion each contain 1000 sample points. N A sample, with... K There are several microphones; first consider the chirp component, for each microphone there is... N For each possible set of signal sample points, ( ) r , v Based on the signal's time of flight, the phase change is reconstructed, thus each microphone can reconstruct a phase change estimation vector. The size of the vector is N The signal measurement value of each sample is multiplied by the conjugate of the signal phase change reconstructed according to the parameters to eliminate the phase shift generated by each sample in the distance and velocity dimensions, and all the results are added together to obtain the joint estimate of the upper chirped part. The same applies to the lower chirped part. The result of the joint estimator is obtained by summing the absolute values ​​of the joint estimates of the upper and lower chirped parts, which can be expressed as follows: Joint estimator The result will be in a set of optimal parameters Maximize the time, because at this point all signals are in phase; therefore, the problem of finding the optimal parameters can be expressed as: Finally, after obtaining two optimal parameters After obtaining the estimated values, signal attenuation is calculated using these estimates. 。 5. The gesture tracking method based on acoustic signals according to claim 4, characterized in that, The method for multi-target signal separation is as follows: First, the path and parameters corresponding to the strongest signal in the mixed signal are obtained based on the joint estimator. The strongest path signal is reconstructed based on the parameters obtained from the formula for chirped mixed signals. The reconstructed signal is then subtracted from the original mixed signal to obtain the remaining mixed signal. Then, the joint estimator is used again to estimate the strongest path signal from the remaining signal to perform joint estimation of the next target and obtain the parameter information of the next target. The joint estimation ends when the power of the remaining signal is less than a predefined threshold.

6. A gesture tracking system based on acoustic signals, characterized in that, include: Mixed signal calculation module: The denoised triangular chirped signal is divided into an upper chirped signal and a lower chirped signal. The separated upper and lower chirped signals are quadrature demodulated to obtain a preliminary mixed signal. The high-frequency part of the obtained preliminary mixed signal is filtered out to obtain the in-phase part and the quadrature part of the signal. The in-phase part and the quadrature part of the upper and lower chirped signals are mixed to obtain the upper chirped mixed signal and the lower chirped mixed signal. The mixed signal of multiple targets includes multiple upper-chirped mixed signals and multiple lower-chirped mixed signals; the flight time of the reflected signal can be obtained from the upper and lower-chirped mixed signals; The signal separation module first eliminates noise from the mixed signals of upper and lower chirps, and then searches for distance information within the set parameter range. r Angle information and speed information v The optimal parameters are based on distance information. r Speed ​​information v The optimal parameters are then used to calculate the signal attenuation factor. When there are multiple targets, the strongest reflected signal in the mixed signal of the current multiple targets is searched each time through an iterative method. The distance, angle and velocity information parameters that match the signal are obtained, and then the signal attenuation factor of the reflected signal is calculated. After the iteration is completed, the parameters of different targets can be obtained. Target matching and tracking module: using weighted l1-norm in Indicates in t The vector consisting of the estimated values ​​of four parameters at time t: signal attenuation factor, distance information, angle information, and velocity information, is represented as follows: , To normalize the differences in the four dimensions into a scale vector of the same scale, the parameter vector obtained by jointly estimating the two continuous signal periods with the smallest l1 distance is regarded as belonging to the same target.

7. The gesture tracking system based on acoustic signals according to claim 6, characterized in that, The calculation process for the mixed signal in the mixed signal calculation module is as follows: First, add the chirp signal. The 90° phase-shifted version of the transmitted signal is obtained by performing Hilbert transform on its own signal and the chirped signal respectively. Multiplying them yields a first preliminary mixed signal and a second preliminary mixed signal, respectively. The first and second preliminary mixed signals are then passed through a low-pass filter to obtain the in-phase components. and orthogonal part The in-phase and quadrature components are combined using a complex function to obtain the upper chirped mixed signal; the lower chirped mixed signal is obtained in the same way; the expression is: In the formula, For chirped mixed signals, This is a chirped mixed signal.

8. The gesture tracking system based on acoustic signals according to claim 7, characterized in that, Multi-target mixed signals are represented as follows: assuming there exists L For each target, the signal received by each microphone should be a composite signal formed by the superposition of the reflected signals from all targets. i When the signal sampling point is _____, the _____ signal sampling point is _____. k The mixed signal of all target reflections received by each microphone is represented as: in, α l , r l , v l They represent the first l The amplitude attenuation factor, distance information, and velocity information of the reflected signal from each target.

9. The gesture tracking system based on acoustic signals according to claim 8, characterized in that, The calculation process for the signal attenuation factor of a single target in the signal separation module is as follows: Suppose a triangular chirped signal contains 2N 1000 sample points, of which the upper chirped portion and the lower chirped portion each contain 1000 sample points. N A sample, with... K There are several microphones; first consider the chirp component, for each microphone there is... N For each possible set of signal sample points, ( ) r , v Based on the signal's time of flight, the phase change is reconstructed, thus each microphone can reconstruct a phase change estimation vector. The size of the vector is N The signal measurement value of each sample is multiplied by the conjugate of the signal phase change reconstructed according to the parameters to eliminate the phase shift generated by each sample in the distance and velocity dimensions, and all the results are added together to obtain the joint estimate of the upper chirped part. The same applies to the lower chirped part. The result of the joint estimator is obtained by summing the absolute values ​​of the joint estimates of the upper and lower chirped parts, which can be expressed as follows: Joint estimator The result will be in a set of optimal parameters Maximize the time, because at this point all signals are in phase; therefore, the problem of finding the optimal parameters can be expressed as: Finally, after obtaining two optimal parameters After obtaining the estimated values, signal attenuation is calculated using these estimates. 。 10. The gesture tracking system based on acoustic signals according to claim 9, characterized in that, The method for multi-target signal separation is as follows: First, the path and parameters corresponding to the strongest signal in the mixed signal are obtained based on the joint estimator. The strongest path signal is reconstructed based on the parameters obtained from the formula for chirped mixed signals. The reconstructed signal is then subtracted from the original mixed signal to obtain the remaining mixed signal. Then, the joint estimator is used again to estimate the strongest path signal from the remaining signal to perform joint estimation of the next target and obtain the parameter information of the next target. The joint estimation ends when the power of the remaining signal is less than a predefined threshold.

Images