Multi-modal gait user identification method and system based on WiFi acoustics

Abstract

The invention provides a multi-modal gait user identification method and system based on WiFi acoustics. The method comprises the following steps: a user freely walks in an indoor equipment range; the method comprises the following steps: acquiring a WIFI signal by using commercial WIFI equipment, extracting a CSI feature from the WIFI signal, calculating a PLCR, and constructing a Fresnel zone elliptical model; a microphone array is used for collecting footstep sound, a footstep sound AoA sequence is calculated from sound signals, and a ray model is constructed; confirming a user moving track through multi-modal fusion; a trajectory and a PLCR spectrum are utilized. Mapping and deriving an independent velocity spectrum; and training a gait recognition model by using the independent velocity spectrum as a sample to realize user recognition. According to the invention, the technical problems of poor adaptability to complex scenes, high deployment difficulty, environment interference and poor robustness are solved.

Description

Technical Field

[0001] The present invention relates to the field of user identity recognition under wireless sensing technology, and in particular to a multi-modal gait user recognition method and system based on WiFi acoustics. Background Art

[0002] Wireless sensing technology is widely used in the field of smart Internet of Things. It sends and receives WiFi, RFID and sound signals through devices with signal receiving and sending functions such as routers, RFID readers, smart speakers, etc., and extracts fine-grained features associated with human behavior through signal processing technology. It can perceive and identify human behavior or location without users carrying sensor devices.

[0003] Gait recognition technology is a classification technology based on pattern recognition. Its main idea is that when a user walks, his or her potential biological characteristics are closely coupled with the corresponding gait. Moreover, based on the physiological structure and movement habits of the human body, the gait of the same user always tends to remain consistent, while the gaits generated by different users are different, which makes user identification possible.

[0004] With the rapid development of Internet of Things technology, intelligent applications based on wireless sensing technology have been widely deployed, and mature communication protocols and signal processing technologies provide reliable support for these services. WiFi sensing, as one of the core technologies of wireless sensing technology, is often used in human activity recognition (HAR) and positioning and tracking tasks. Its principle is that when WiFi signals encounter obstacles, the signals will undergo changes such as reflection and diffraction. Therefore, the position and posture changes caused by human activities will also affect the signal pattern, such as amplitude and phase. Since different types and frequencies of actions have different effects on the signal, human actions can be effectively identified and perceived by analyzing the characteristics and change patterns of the signal. In recent years, researchers have usually used channel state information (CSI) to analyze these changes. For example, the existing invention patent application document "A speed-independent gait recognition method based on WiFi signals" with publication number CN114757237A includes: extracting the CSI amplitude of WiFi that changes over time during a person's walking; preprocessing the CSI amplitude; determining whether there is a person walking in the environment, and extracting walking activity segments; converting the walking activity segments into time-frequency graphs of the same size; building a speed-independent gait recognition model based on DANN, which includes a feature extractor, an identity identifier and a speed identifier, the feature extractor is used to extract potential features from the input time-frequency graph, the identity identifier is used to predict the identity of the target under test using the features extracted by the feature extractor, and the speed identifier is used to predict the speed of the target under test using the features extracted by the feature extractor; training the speed-independent gait recognition model and outputting the identity of the target under test. And the existing invention patent application document "A gait recognition method based on wireless and video feature fusion" with publication number CN114783054A, the existing method includes: using video acquisition equipment to obtain pedestrian gait recognition video data, using a segmentation network to obtain a high-quality pedestrian profile sequence for the video frame, performing standardized cropping operations on the profile into a unified format, and then using a deep neural network based on time and space to obtain the video features of the pedestrian; using ordinary commercial wireless signal equipment, sending physical layer channel state information CSI data at one end and receiving it at the other end, denoising and normalizing the CSI data, and using a multi-scale convolutional neural network to extract the wireless features of the pedestrian from the preprocessed CSI data; finally, the extracted wireless and video features are fused to perform identity prediction. In the aforementioned prior art, CSI provides fine-grained physical layer information in signal propagation, such as propagation distance, power attenuation and scattering, etc., reflecting the combined impact of the environment and human body dynamics. When the user moves, the changes in the activities and positions of the limbs reflect their walking habits, such as stride, step frequency, arm swing habits, etc.These characteristics provide the potential for identifying different gaits, allowing the new generation of smart homes to provide more personalized and differentiated services based on different user identities.

[0005] Existing WiFi-based user identification work usually relies on extracting the Short-Time Fourier Transform (STFT) spectrum from CSI, and identifying users through the spectrum characteristics of different users when they walk. However, using only WiFi features can only obtain PLCR, but cannot obtain the true speed of the human body; at the same time, the spectrum characteristics are easily affected by multiple factors such as walking trajectory and direction, which means that the same user may produce different features when walking on different trajectories, resulting in the erroneous identification of these trajectories as different users, which cannot meet the needs of increasingly diverse and complex scenarios. Some subsequent works solve the problem of trajectory dependence of features by introducing additional orthogonal WiFi links to obtain the spectrum diagram in another direction, and mapping it from the orthogonal direction to the user's moving direction, but the strict requirements on the number of devices and deployment locations make it difficult to deploy the system in the home.

[0006] The widespread use of sound devices has also made people see the potential of acoustic perception. When users walk, they will continue to produce footsteps, which can also be used as an observation feature for user identification. In existing acoustic perception work, a circular array (hexagonal microphone) is usually used to receive sound signals, which has 6 microphones at different positions on the six corners. The typical size of these microphone arrays is usually small to adapt to common smart speaker products. The spacing between different microphones and the side length of the array are usually in the centimeter level. Relative to the distance between the array and the sound source, the array size can be ignored. Therefore, according to the far-field effect, we can regard the sound wave as a parallel wave, which ignores the amplitude difference between the sound signals received by each microphone and only considers the delay difference of each received signal. This delay difference provides the conditions for calculating the location of the sound source. On the other hand, the sound signal can also obtain the corresponding energy spectrum. Like the WiFi signal, the footstep spectrum can be used to obtain features such as step frequency and strength for identity distinction. However, pure audio is easily interfered by other factors, such as the material of the ground, the shoes worn, or the influence of environmental noise, and has poor robustness.

[0007] Given that WiFi gait features are difficult to overcome the dependence on walking trajectory and direction, while sound features are easily interfered with, this method aims to combine the advantages of the two and propose a multimodal gait recognition system based on WiFi acoustics, which can realize user gait recognition independent of trajectory and direction using only a single device link.

[0008] In the existing public document "Research on Passive Indoor Personnel Positioning Technology Based on WiFi", the estimation of the signal arrival angle is easily affected by environmental noise. As the distance between the target and the receiving end increases, a small signal arrival angle estimation error will often lead to a large loss in the final positioning accuracy, thereby seriously reducing the system performance. This system designs a dual-window AOA estimation algorithm based on time and space dimensions, which effectively guarantees the time complexity of the algorithm while reducing the environmental noise in the AOA estimation. The method of subtracting a small constant value α from the CSI amplitude of all sensors and adding a large constant value β to the CSI amplitude of the reference sensor can be used to eliminate the influence caused by DFS ambiguity. The signal arrival angle estimation value is obtained by the MUSIC algorithm. Due to the existence of the Doppler effect, it can be known that when the person to be measured moves in the indoor environment, the propagation path length of the human body reflection signal will change accordingly, which will cause the frequency of the received signal to shift to varying degrees compared with the frequency of the signal transmitted by the transmitting end. When the person to be measured moves towards the signal receiving end, the frequency of the WiFi received signal will increase compared with the original signal frequency sent by the transmitting end. Correspondingly, when the person to be tested is far away from the signal receiving end, the frequency of the WiFi received signal will be lower than the original signal frequency sent by the transmitting end. The aforementioned reference document records that the signal arrival angle estimation algorithm in this system does not depend on the Doppler frequency shift estimation value, but the reference document also mentions: d represents the distance from the target to the receiver, and its variation range can be obtained from the Doppler frequency shift estimation value. If the variation range is related to the sampling probability and angle solution, the error of the Doppler frequency shift may migrate to the angle estimation, resulting in cumulative error.

[0009] In summary, the existing technologies have technical problems such as poor adaptability to complex scenarios, difficulty in deployment, environmental interference and poor robustness. Summary of the invention

[0010] The technical problem to be solved by the present invention is: how to solve the technical problems of poor adaptability to complex scenarios, great deployment difficulty, environmental interference and poor robustness in the prior art.

[0011] The present invention adopts the following technical solutions to solve the above technical problems: A multimodal gait user recognition method based on WiFi acoustics includes:

[0012] S1, make the identified target move within the preset indoor equipment range, use the preset WiFi device to collect WiFi signals to extract CSI features, calculate the reflection path length change rate PLCR according to the CSI features, build a Fresnel zone ellipse model based on it, and process it to obtain PLCR features;

[0013] S2. Set up and use the microphone array to collect the footsteps of the identified target, calculate the footstep AoA sequence based on the footstep signal, build a ray model, model it according to the preset signal propagation mode and mathematical geometry theory, and obtain the theoretical sample offset value n corresponding to all angles θ sim , offset the actual measured samples by n shift The theoretical sample offset value n sim Matching is performed to obtain the closest angle value. By locating the sound source, the angle of arrival AoA of each footstep signal is extracted from the footstep signal to obtain a continuous angle sequence distributed along the time domain, and the original gait feature is converted into an AoA angle feature.

[0014] S3. Confirm the user's movement trajectory through multimodal fusion; combine PLCR features and AoA angle features to build a geometric model, and obtain the trajectory position sequence of the user's movement; construct a speed-angle matrix, and according to the relationship between speed and frequency, perform transformation operations on the speed-angle matrix and the frequency-angle matrix, and obtain the moving direction angle at each moment from the trajectory position sequence; use the moving direction angle, speed-angle matrix and frequency-angle matrix as indexes to match the speed in the PLCR speed spectrum, so as to transform the PLCR speed spectrum mapping into an independent gait speed spectrum;

[0015] S4. Obtain independent speed spectra of different identified targets, build a data-driven user identification model, build and train a lightweight neural network, and obtain a suitable gait recognition model to perform gait recognition and determine the identity of the identified target.

[0016] The present invention places the microphone and the WiFi transceiver at the same position to build a prototype of a smart speaker or a smart voice assistant. The speaker has the functions of sending and receiving WiFi signals and collecting sound with the microphone. The WiFi and the microphone are respectively used to collect the WiFi signals and the footsteps caused by the user walking. The corresponding feature extraction and fusion algorithms are designed to extract information such as the human body movement speed, angle, gait spectrum, etc., and finally the gait spectrum is mapped to the acoustic angle to obtain the gait features that are independent of the trajectory orientation, so as to achieve more robust user recognition.

[0017] The present invention can extract the user's gait features non-invasively without relying on wearable sensors or camera devices, avoiding the cumbersome equipment deployment requirements and effectively solving the privacy and lighting condition issues based on camera recognition. Accurate recognition effects can be achieved under a single link, improving the feasibility of deployment and application of non-invasive identity recognition technology in real scenarios.

[0018] In a more specific technical solution, S1 includes:

[0019] S11. Use the preset CSI extraction tool and the following logic to extract CSI from the WiFi signal:

[0020] H(f,t)=H s (f,t)+H d (f,t)=H s (f,t)+A(f,t)e -j2πf(t)

[0021] In the formula, H s (f,t) and H d (f, t) are respectively the static signal component and the dynamic signal component including the user's movement; S12, record the reflection path length at different times as d(t), and use the following logic to determine the Doppler frequency

[0022] Shift DFS:

[0023]

[0024] Among them, f D (t) is the Doppler frequency shift DFS, λ is the signal wavelength, and τ is the signal flight time;

[0025] S13. Determine the relationship between the reflection path length change rate PLCR and the Doppler frequency shift DFS using the following logic:

[0026]

[0027] S14. Using the following logic, integrate the reflection path length change rate PLCR over time to obtain the path length d(t):

[0028]

[0029] In a more specific technical solution, in S2, according to the preset geometric relationship, the following logic is used to express the different microphones m in the microphone array. i 、m j The time difference Δt of receiving the sound:

[0030]

[0031] Using the following logic, the sample offset n is expressed according to the time difference Δt shift :

[0032] n shift =Δt×fs

[0033] According to the sample offset n shift , using the following logic, the angle θ is obtained:

[0034]

[0035] Where L is the distance between the two microphones, c sound is the speed of sound, and fs is the acoustic sampling rate.

[0036] Specifically, in the process of solving the theoretical value of the present invention, the structure and size of the microphone array are known, the speed of sound propagation and the sampling rate of the microphone signal are also known. Therefore, for a sound signal transmitted from any angle in a two-dimensional plane, we can use a geometric formula to calculate the distance difference, that is, the time difference, of the signal reaching the six microphones; and then according to the signal sampling rate, we can further obtain the sample offset between the six signals;

[0037] In the actual value solving process of the present invention, in the actual footstep sound collection, we can solve the sample point offset between two signal sequences through cross-correlation.

[0038] According to the sample offset n shift , using the following cross-correlation function logic to express the cross-correlation relationship between microphone signals, and according to the microphone cross-correlation relationship, the discrete point sequence C[n shift ]:

[0039] C[n shift ]=∑(m i [n]·m j [n+n shift ]),

[0040] Using the discrete point sequence C[n shift ], indicating the signal sequence [n+n shift ] relative to the lag of the signal sequence [n].

[0041] Using the following logic, calculate the discrete point sequence C[n shift ] corresponds to the subscript of the peak value to find the signal sequence [n+n shift ] relative to the sample point offset value of the signal sequence [n]:

[0042]

[0043] Specifically, the present invention uses a hexagonal microphone array to collect footsteps signals. Since each microphone in the hexagonal microphone array is at a different position, the time at which the same sound reaches the six microphones is also different. The signals of the six channels thus obtained will have a slight offset, namely the sample offset in the present invention.

[0044] In a more specific technical solution, S2 includes:

[0045] S21, set the value range of the simulation angle θ′, bring the simulation angle into the preset geometric model, and use the following logic to obtain the simulation sample offset angle matrix nsim :

[0046]

[0047] Where L is the distance between the two microphones, c sound is the speed of sound, fs is the acoustic sampling rate;

[0048] A reference microphone is preset, and a time delay Δt of signals received by other microphones relative to a signal received by the reference microphone is calculated;

[0049] S22. Obtain the applicable theoretical sample offset that best matches the actual sample offset through similarity calculation, and use the simulation handover θ′ corresponding to the applicable theoretical sample offset as the sound signal arrival angle:

[0050]

[0051] For each step of the footstep signal, match the applicable theoretical sample offset Get the number of steps to align the angle sequence:

[0052] θ seq ={θ1,θ2,…,θ k}.

[0053] Specifically, since the traditional algorithm can only obtain integer n shift (Because the peak position subscript can only be an integer), but in actual situations, the sample point offset usually contains decimals, which will cause errors in the final angle of the solution. In view of the above situation, the present invention adopts a method of matching theoretical and actual sample point offsets, that is, modeling through signal propagation mode and mathematical geometry theory, and calculating the theoretical sample offset value n corresponding to all possible angles θ sim , and then offset the actual measured samples by n shift By matching it with the theoretical value, the closest angle value can be found, which enables a more accurate solution to the sample point offset.

[0054] In a more specific technical solution, in S21, each microphone is used as a reference microphone to expand the simulation sample offset angle matrix n sim According to the projection theorem, the following logic is used to process the theoretical sample offset

[0055]

[0056] In the formula, m i ,m j for different microphones i,j∈{1,2,3,4,5,6}, Δd is the additional flight distance of the signal, is the angle vector, is the distance vector between the two microphones.

[0057] The present invention can realize the extraction of gait features that are independent of trajectory and direction with limited equipment, thereby improving the user recognition accuracy in actual scenarios. The sound sequence of footsteps when the user is walking can be collected through a microphone array. When walking, the footsteps are discrete, that is, there will be a short silent segment between two adjacent steps. Compared with traditional sound source positioning, continuous sound sources are usually targeted. The application scenario of the present invention is that the continuously moving sound source is footsteps, which is different from the traditional research scenario. Therefore, the present invention regards the moving process as a combination of multiple static sound sources, and only calculates the steps with sound, while abandoning the silent parts between the steps. Therefore, the user's step is first determined by threshold detection or correlation matching. This can not only avoid the calculation interference caused by invalid information of silent segments, but also reduce the amount of calculation and improve the operation efficiency.

[0058] In a more specific technical solution, S3 includes:

[0059] S31, convert the obtained PLCR velocity spectrum into f mat (t) frequency spectrum; due to f mat The dimensions of are speed-angle-time, so through the β(t) we get, we can determine the time step and the corresponding angle, and then match the corresponding speed;

[0060] S32. By combining the velocity values ​​of each time step, the entire PLCR velocity spectrum is transformed into an independent velocity spectrum that is independent of trajectory and direction.

[0061] In a more specific technical solution, S31 includes:

[0062] S311. The user position is solved by combining the WiFi Fresnel ellipse and the angle sequence ray. The entire sensing area is divided into no less than 2 grids, and the coordinates of each grid are marked as (x, y). The Fresnel ellipse equation is constructed using the reflection path length d(t):

[0063]

[0064] According to the moving direction angle θ, construct the angle sequence ray equation:

[0065] y t =tanθ seq (t)·x t

[0066] S312, convert and process the Fresnel ellipse equation and the angle sequence ray equation to obtain the error calculation equation:

[0067]

[0068] S313, substitute the coordinates of each grid into the error calculation equation, perform maximum likelihood estimation, and obtain the user position (x t ,y t ):

[0069]

[0070] S314, in the process of solving gait characteristics, using polar coordinates v ρ 、v θ It represents the reflection path length change rate PLCR;

[0071] S315, obtain and construct a speed-angle matrix v according to the speed characteristics and angle characteristics mat According to the speed frequency formula, the speed-angle matrix v mat Converted into a frequency matrix to map the PLCR velocity spectrum using the following logic:

[0072]

[0073] The user position trajectory is obtained by processing, and the displacement direction angle β(t) is calculated based on two adjacent coordinate points.

[0074] Specifically, in the theoretical calculation of the present invention, the AOA of [0,360]° is substituted, and the theoretical sample offset corresponding to each angle is calculated respectively; the actual sample offset obtained by cross-correlation is matched with the theoretical value, and the AOA angle corresponding to the theoretical value when the two are closest is the required signal arrival angle.

[0075] In a more specific technical solution, in S314, v in the polar coordinates ρ Represents the velocity value, v θ Represents the speed direction:

[0076]

[0077] In a more specific technical solution, in S315, δ x and δ y is the directional coefficient determined by the location of the WiFi device:

[0078]

[0079] In the formula, Indicates the user's location. and Represent the transmitter and receiver locations respectively.

[0080] In a more specific technical solution, the multimodal gait user recognition system based on WiFi acoustics includes:

[0081] The PLCR feature processing module is used to make the identified target move within the preset indoor device range, collect WiFi signals using the preset WiFi device to extract the CSI feature, calculate the reflection path length change rate PLCR based on the CSI feature, and construct the Fresnel zone ellipse model based on it to obtain the PLCR feature;

[0082] The AoA angle feature processing module is used to set up and use the microphone array to collect the footsteps of the identified target, calculate the footstep AoA sequence based on the footstep signal, build a ray model, and model it according to the preset signal propagation mode and mathematical geometry theory to obtain the theoretical sample offset value n corresponding to all angles θ sim , offset the actual measured samples by n shift The theoretical sample offset value n sim Matching is performed to obtain the closest angle value. By locating the sound source, the angle of arrival AoA of each footstep signal is extracted from the footstep signal to obtain a continuous angle sequence distributed along the time domain, and the original gait feature is converted into an AoA angle feature.

[0083] The multimodal fusion module is used to confirm the user's movement trajectory through multimodal fusion; the PLCR feature and the AoA angle feature are combined to construct a geometric model, and the trajectory position sequence of the user's movement can be obtained; the speed-angle matrix is ​​constructed, and according to the relationship between speed and frequency, the speed-angle matrix and the frequency-angle matrix are converted to obtain the moving direction angle at each moment from the trajectory position sequence; the moving direction angle, the speed-angle matrix and the frequency-angle matrix are used as indexes to match the speed in the PLCR speed spectrum, so as to convert the PLCR speed spectrum mapping into an independent gait speed spectrum. The multimodal fusion module is connected with the PLCR feature processing module and the AoA angle feature processing module;

[0084] The gait recognition module is used to obtain the independent speed spectra of different identified targets, build a data-driven user identification model, build and train a lightweight neural network, and obtain a suitable gait recognition model to perform gait recognition and determine the identity of the identified target. The gait recognition module is connected to the multimodal fusion module.

[0085] Compared with the prior art, the present invention has the following advantages:

[0086] The present invention places the microphone and the WiFi transceiver at the same position to build a prototype of a smart speaker or a smart voice assistant. The speaker has the functions of sending and receiving WiFi signals and collecting sound with the microphone. The WiFi and the microphone are respectively used to collect the WiFi signals and the footsteps caused by the user walking. The corresponding feature extraction and fusion algorithms are designed to extract information such as the human body movement speed, angle, gait spectrum, etc., and finally the gait spectrum is mapped to the acoustic angle to obtain the gait features that are independent of the trajectory orientation, so as to achieve more robust user recognition.

[0087] The present invention can extract the user's gait features non-invasively without relying on wearable sensors or camera devices, avoiding the cumbersome equipment deployment requirements and effectively solving the privacy and lighting condition issues based on camera recognition. Accurate recognition effects can be achieved under a single link, improving the feasibility of deployment and application of non-invasive identity recognition technology in real scenarios.

[0088] The present invention adopts a method of matching theoretical and actual sample point offsets, that is, modeling is performed through signal propagation mode and mathematical geometry theory to calculate the theoretical sample offset values ​​n corresponding to all possible angles θ. sim , and then offset the actual measured samples by n shift By matching it with the theoretical value, the closest angle value can be found, which enables a more accurate solution to the sample point offset.

[0089] The present invention can realize the extraction of gait features that are independent of trajectory and direction with limited equipment, thereby improving the user recognition accuracy in actual scenarios. The sound sequence of footsteps when the user is walking can be collected through a microphone array. When walking, the footsteps are discrete, that is, there will be a short silent segment between two adjacent steps. Compared with traditional sound source positioning, continuous sound sources are usually targeted. The application scenario of the present invention is that the continuously moving sound source is footsteps, which is different from the traditional research scenario. Therefore, the present invention regards the moving process as a combination of multiple static sound sources, and only calculates the steps with sound, while abandoning the silent parts between the steps. Therefore, the user's step is first determined by threshold detection or correlation matching. This can not only avoid the calculation interference caused by invalid information of silent segments, but also reduce the amount of calculation and improve the operation efficiency.

[0090] The present invention solves the technical problems existing in the prior art, such as poor adaptability to complex scenarios, great deployment difficulty, environmental interference and poor robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is a schematic diagram of basic steps of a multi-modal gait user identification method based on WiFi acoustics according to Embodiment 1 of the present invention;

[0092] Figure 2 This is a schematic diagram of data stream processing of a multimodal gait user recognition method based on WiFi acoustics according to Embodiment 1 of the present invention;

[0093] Figure 3 Schematic diagram of specific steps of acoustic signal feature extraction according to Embodiment 1 of the present invention;

[0094] Figure 4 This is a schematic diagram of detecting footsteps through cross-correlation matching in Example 1 of the present invention;

[0095] Figure 5 This is a schematic diagram of different gait speed spectra resulting from different trajectories in the conventional method of Example 1 of the present invention;

[0096] Figure 6 A schematic diagram of the time delay of receiving a sound signal by the microphone array according to Embodiment 1 of the present invention;

[0097] Figure 7 This is a schematic diagram of the trajectory of the combined calculation of the Fresnel ellipse cluster and the angle ray according to Example 1 of the present invention;

[0098] Figure 8 Schematic diagram of specific steps for deriving independent speed spectrum gait features according to Embodiment 1 of the present invention;

[0099] Fig. 9 This is a diagram of the gait recognition neural network architecture of Example 1 of the present invention;

[0100] Fig.10 This is a schematic diagram of data flow processing for deriving gait features of independent speed spectrum according to Embodiment 1 of the present invention;

[0101] Fig.11 This is a schematic diagram of a user identification confusion matrix according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0102] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0103] Example 1

[0104] like Figure 1 and Figure 2 As shown, the multimodal gait user recognition method based on WiFi acoustics provided by the present invention includes the following basic steps:

[0105] S1, WiFi signal feature extraction;

[0106] In this embodiment, one WiFi transmitter and one receiver are deployed to construct a single-link WiFi signal collection device for collecting WiFi signals when a human body moves. After the signal data packets are collected, CSI information can be extracted from the original signal through the CSI tool. When a human body moves, the relative position and distance to the signal source continuously change, generating DFS. According to the communication principle, PLCR can be calculated and derived from the frequency information. This feature describes the activity of a human body during movement, and provides an information basis for the subsequent calculation of the user's gait characteristics.

[0107] In this embodiment, the CSI Tool toolbox can be used to easily obtain CSI from WiFi signals:

[0108] H(f,t)=H s (f,t)+H d (f,t)=H s (f,t)+A(f,t)e -j2πf(t) ,

[0109] Among them, H s (f,t) and H d (f, t) are the static signal component and the dynamic signal component including the user's movement. The Doppler frequency shift is caused by the change of the reflected signal path. We record the reflection path length at different times as d(t), and the Doppler frequency shift can be recorded as follows:

[0110]

[0111] Among them, f D (t) is DFS, λ is the signal wavelength, and τ is the signal flight time. PLCR is the signal path change rate, i.e. Therefore, according to the above formula, if we denote PLCR as r, it can be expressed as Therefore, the relationship between DFS and PLCR is as follows:

[0112]

[0113] Through the above derivation, we can know that the path length d is the integral of PLCR over time. In the background technology and solution, we mentioned that we need to calculate the user's movement trajectory. In the WiFi feature, we need to construct a Fresnel zone, which is a series of cofocal ellipses. According to the Fresnel ellipse and signal propagation principle, the signal propagation path length is required as the parameter of the ellipse. The path length d(t) can be expressed as follows:

[0114]

[0115] Fresnel zones refer to multiple equidistant ellipsoidal areas between the signal transmitter and the receiver. The signal transmitter and the receiver are located at the two foci of the ellipse respectively. When the target enters different Fresnel zones, the path lengths of the reflected signal and the direct signal will be different, resulting in changes in the superposition effect of the two at the receiving end. When an object passes through multiple Fresnel zones, the received signal strength will periodically increase and decrease, and this change can depict the user's activities across the Fresnel zone.

[0116] S2, acoustic signal feature extraction;

[0117] In this embodiment, a small microphone array is deployed to simulate an intelligent voice assistant to collect the footsteps of the user when walking; the signal arrival angle AoA of each footstep is extracted from the sound signal through sound source localization technology, and finally a continuous angle sequence distributed along the time domain is obtained. This angle information can be used to convert the original gait features into independent features that are independent of trajectory and direction.

[0118] The principle of acoustic positioning is to analyze the propagation of sound signals, including the flight time and arrival angle of the signal. The sound signal emitted from the transmitter (sound source) will form an acoustic arrival angle on the antenna array of the receiver (such as a microphone). This angle value reflects the azimuth of the sound source. Based on the speed distance calculation formula, the distance of the sound source can be further obtained, and combined with the angle, the specific location of the sound source can be solved.

[0119] In this embodiment, since the positions of the microphones on the array are different, the flight distances and times of the signals from the same sound source to reach different microphones are also different. These signals are presented in the form of one-dimensional time series data, and the data at each time point is a sample point. The time delay Δt caused by the extra flight distance Δd causes a slight sample point deviation in the sound signals captured by each microphone. We call this feature Sample Shift.

[0120] In this embodiment, according to the geometric relationship, different microphones m i and m j The time difference Δt of receiving the sound can be expressed by the following formula:

[0121]

[0122] Sample offset n shift It can be expressed as the following formula:

[0123] n shift =Δt×fs,

[0124] The angle θ can be solved by the following formula:

[0125]

[0126] Where L is the distance between the two microphones, c sound is the speed of sound (340 m / s in air), fs is the acoustic sampling rate, so as long as the sample offset n is obtained shift , then θ can be obtained.

[0127] In this embodiment, for the sound signal actually collected, for example, a generalized cross-correlation algorithm can be used. The generalized cross-correlation algorithm is widely used in signal delay solution problems. Specifically, in sound source localization, the target signal received by each microphone on the array comes from the same sound source, so the pattern characteristics such as the signal amplitude received by each microphone are very similar, and there is a strong correlation between the signals. The aforementioned cross-correlation method can calculate the similarity between two sequences and determine the time step difference between the two signals. Based on this phenomenon, the cross-correlation function can be used to calculate the peak value of the cross-correlation function of the two microphone signals to calculate the sample offset n shift For a pair of microphones m i and m j , let m i The signal consists of n samples, correspondingly, m j The received signal has a certain delay and sample offset, so it is expressed as n+n shift , the cross-correlation formula between two signals is as follows:

[0128] C[n shift ]=∑(m i [n]·m j [n+n shift ]),

[0129] In this embodiment, the above function can obtain a set of discrete point sequences C[n shift ] to represent the signal sequence [n+n shift ] relative to the lag of the signal sequence [n].

[0130] Therefore, calculate C[n shift ], we can find the signal sequence [n+n shift ] relative to the sample point offset value of the signal sequence [n]:

[0131]

[0132] like Figure 3 As shown, in this embodiment, the operation of extracting acoustic signal features also includes the following specific steps:

[0133] S21, simulation sample offset;

[0134] In this embodiment, the simulation angle θ′∈[1,360] is set (covering all possible angles in the two-dimensional space), and the simulation angle is brought into the previous geometric model to calculate the simulation sample offset angle matrix n sim :

[0135]

[0136] Where L is the distance between the two microphones, c sound is the speed of sound, and fs is the acoustic sampling rate. These values ​​are constants that can be obtained in advance. Since a time delay Δt is required, a reference microphone needs to be determined first, and then the signals received by other microphones relative to the received signals of the reference microphone are calculated.

[0137] like Figure 4 and Figure 5 As shown, in this embodiment, Figure 4 and Figure 5 Take the hexagonal ring microphone in as an example, n sim It should be a matrix of dimension A×6, where A represents the number of angles θ′ and 6 represents that each microphone is calculated once relative to the reference microphone, including the reference microphone itself.

[0138] In this embodiment, in order to further improve the resolution and accuracy, each microphone is used as a reference microphone. sim The dimension of will be expanded to A×6×6. The specific calculation method uses the projection theorem and is expressed as follows:

[0139]

[0140] Among them, m i ,m j for different microphones i,j∈{1,2,3,4,5,6}, Δd is the additional flight distance of the signal, is the angle vector, is the distance vector between the two microphones. In this way, each angle θ′ has a corresponding set of 6×6 theoretical sample offsets

[0141] S22, similarity measurement;

[0142] In this embodiment, the actual sample offset n is obtained by cross-correlation. shift And the theoretical sample offset n obtained by theoretical calculation sim After that, the theoretical sample offset value that best matches the measured sample offset can be found through similarity calculation. The θ′ corresponding to the sample offset with the highest similarity is the sound signal arrival angle to be solved:

[0143]

[0144] In this embodiment, taking footstep detection as an example, the signal generated by each step can be obtained as Finally, through this matching, we can get an angle sequence θ that is aligned with the number of step points. seq ={θ1,θ2,…,θ k}.

[0145] S3, multimodal fusion computing;

[0146] like Figure 6 As shown, in this embodiment, WiFi and acoustic signal features are combined for calculation. In the WiFi and acoustic feature calculation part, PLCR features and AoA angle features are obtained respectively. The two are combined to construct a geometric model, and the trajectory position sequence of the user's movement can be obtained; a speed-angle matrix is ​​constructed, and according to the relationship between speed and frequency, the speed-angle matrix and the frequency-angle matrix are transformed to obtain the moving direction angle at each moment from the user's trajectory position; the angle and the above matrix are used as indexes to match the speed in the PLCR speed spectrum, and the PLCR speed spectrum that originally represents the frequency of WiFi signal path changes is converted into an independent gait speed spectrum that is independent of trajectory and direction.

[0147] The path length change rate (PLCR) describes the rate at which the propagation path length of a WiFi signal changes after being reflected by a target. This rate of change is one of the causes of Doppler frequency shift (DFS). In indoor positioning scenarios, when a person moves relative to the signal transmission link, the length of the signal transmission path changes, causing Doppler frequency shift and PLCR. By analyzing the characteristics of the two, information related to the target's direction of movement and speed can be obtained.

[0148] like Figure 7 As shown, in the process of solving the user movement trajectory position in this embodiment, the user position is first solved by combining the WiFi Fresnel ellipse with the angle sequence ray. Specifically, the entire sensing area is divided into several grids, and the coordinates of each grid are marked as (x, y). Then d(t) is used to construct the Fresnel ellipse equation:

[0149]

[0150] Construct an angle sequence ray with θ as parameter:

[0151] y t =tanθ seq (t)·x t

[0152] Convert the two equations into error calculation equations:

[0153]

[0154] Finally, the coordinates of each grid in the area are substituted into the error calculation equation, and the user's position (x t ,y t ):

[0155]

[0156] In the gait feature solving process of this embodiment, for the convenience of calculation, PLCR is expressed in polar coordinates v ρ and v θ Indicates, representing the speed value and speed direction respectively:

[0157]

[0158] In this embodiment, a velocity-angle matrix v is established. mat , which is composed of velocity features with a granularity of [0m / s:0.02m / s:2m / s] and angle features with a granularity of [1ο:1°:360°], and the dimension of the entire matrix is ​​101*360. According to the velocity frequency formula, v mat The matrix is ​​converted into a frequency matrix to realize the mapping of PLCR velocity spectrum:

[0159]

[0160] Among them, δ x and δ y is the directional coefficient determined by the location of the WiFi device:

[0161]

[0162] in, Indicates the user location (user location), and Respectively represent the transmitter and receiver positions.

[0163] In the existing literature of the background technology, due to the phase ambiguity of amplitude 2, the search space of AOA is defined as [-90°, 90°]. This means that the located person can only move on one side of the receiving antenna array, which limits the deployment and user movement space in practical applications. The search space of the present invention is [0°, 360°], covering the entire two-dimensional plane.

[0164] Similarly, under a single WiFi link, the positioning performance of the technical solution recorded in the comparative document is lower than that of the present invention.

[0165] In this embodiment, we can get a matrix at each moment, so the entire f mat It is a tensor distributed along the time step with a dimension of 101*360*T (speed-angle-time), where T represents the time step of the entire user movement process, which is equal to the length of PLCR and AoA.

[0166] The user location trajectory is obtained from the above, and the direction angle of displacement is calculated based on two adjacent coordinate points. To distinguish it from the AoA in the previous article, we record this direction angle as β(t). In addition, the PLCR spectrum has been determined based on the WiFi stage.

[0167] like Figure 8 As shown, in this embodiment, during the multimodal fusion calculation process, the independent speed spectrum gait feature derivation process includes the following specific steps:

[0168] S31, convert the obtained PLCR velocity spectrum into f mat (t) frequency spectrum;

[0169] S32, due to f mat The dimensions of are speed-angle-time, so through the β(t) we get, we can determine the time step and the corresponding angle, and then match the corresponding speed;

[0170] S33, by combining the velocity values ​​of each time step together, the entire PLCR velocity spectrum is transformed into an independent velocity spectrum that is independent of trajectory and direction.

[0171] S4, user gait recognition;

[0172] In this embodiment, by acquiring speed spectra of different users, a data-driven user identification model is constructed, and a lightweight neural network is used to train the model to identify the user identity based on the input speed spectrum.

[0173] like Figures 9 to 11As shown, in this embodiment, a lightweight neural network is built for training and recognition. Different volunteer (user) identities are used as labels, and their respective gait speed curves are used as training samples of the neural network to train the model. The trained model can identify users based on the speed features calculated from the signal. Since the speed feature is a one-dimensional time series information, and this is a typical classification task, we use a combination of convolutional neural network (CNN) and recurrent neural network (RNN). The network includes but is not limited to: 2 convolutional layers for feature extraction, each layer adds a batch normalization layer, ReLU activation layer and a maximum pooling layer. In addition, it also includes an LSTM layer for capturing time information. LSTM is followed by a fully connected layer to learn the spatiotemporal features extracted by CNN and LSTM, and finally links a softmax layer to calculate and output category probabilities, and output the user label with the maximum probability.

[0174] Example 2

[0175] In this embodiment, the following experiments are conducted to verify the performance of the proposed method, including different gait categories and user identities, which proves the feasibility and robustness of the method.

[0176] In this embodiment, in an indoor scenario, two microcomputers equipped with Intel 5300 network cards and three antennas are deployed to simulate home WiFi devices, and the cost is similar to that of commercial WiFi routers. The two WiFis are used as transmitters and receivers to send and receive WiFi signals, with a signal frequency of 5.32Ghz and a bandwidth of 20MHz. In terms of acoustic signals, a Seeed Respeaker circular array with 6 small microphones is deployed. The Seeed Respeaker circular array can be set to, for example: a hexagonal microphone array, with microphones located at six corners, and a side length of 4.75cm for the array base, which conforms to the structure of most commercial smart speakers, such as Alibaba Tmall Genie and Amazon Echo. The microphone sampling rate is set to 48kHz, which covers the frequency range where footsteps may occur, and has a good trade-off between computing efficiency and computing time overhead. One of the WiFis is deployed together with the microphone array to form a prototype of the "intelligent voice assistant". The microphone array is operated by Raspberry Pi, and the WiFi device is connected to a laptop equipped with Intel i7-11800H CPU and 16G RAM through the SSH protocol of MobaXTerm, and the sending and receiving of signals are controlled by corresponding commands. Finally, MatlabR2023a is used for code writing and algorithm implementation.

[0177] In this embodiment, 12 volunteers of different heights, weights, genders and ages are recruited for data collection and system evaluation. Each volunteer needs to walk normally along several different trajectories and repeat a certain number of times. The collected data will be divided into training sets and test sets for training and testing user recognition network models.

[0178] In summary, the present invention places the microphone and the WiFi transceiver at the same position to build a prototype of a smart speaker or a smart voice assistant, which has the functions of sending and receiving WiFi signals and collecting sounds with the microphone. The WiFi and the microphone are respectively used to collect the WiFi signals and the footsteps caused by the user walking. The corresponding feature extraction and fusion algorithms are designed to extract information such as the human body movement speed, angle, gait spectrum, etc., and finally the gait spectrum is mapped to the acoustic angle to obtain the gait features that are independent of the trajectory orientation, so as to achieve more robust user recognition.

[0179] The present invention can extract the user's gait features non-invasively without relying on wearable sensors or camera devices, avoiding the cumbersome equipment deployment requirements and effectively solving the privacy and lighting condition issues based on camera recognition. Accurate recognition effects can be achieved under a single link, improving the feasibility of deployment and application of non-invasive identity recognition technology in real scenarios.

[0180] The present invention adopts a method of matching theoretical and actual sample point offsets, that is, modeling is performed through signal propagation mode and mathematical geometry theory to calculate the theoretical sample offset values ​​n corresponding to all possible angles θ. sim , and then offset the actual measured samples by n shift By matching it with the theoretical value, the closest angle value can be found, which enables a more accurate solution to the sample point offset.

[0181] The present invention can realize the extraction of gait features that are independent of trajectory and direction with limited equipment, thereby improving the user recognition accuracy in actual scenarios. The sound sequence of footsteps when the user is walking can be collected through a microphone array. When walking, the footsteps are discrete, that is, there will be a short silent segment between two adjacent steps. Compared with traditional sound source positioning, continuous sound sources are usually targeted. The application scenario of the present invention is that the continuously moving sound source is footsteps, which is different from the traditional research scenario. Therefore, the present invention regards the moving process as a combination of multiple static sound sources, and only calculates the steps with sound, while abandoning the silent parts between the steps. Therefore, the user's step is first determined by threshold detection or correlation matching. This can not only avoid the calculation interference caused by invalid information of silent segments, but also reduce the amount of calculation and improve the operation efficiency.

[0182] The present invention solves the technical problems existing in the prior art, such as poor adaptability to complex scenarios, great deployment difficulty, environmental interference and poor robustness.

[0183] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal gait user recognition method based on WiFi acoustics, characterized by: The method comprises: S1. The target to be identified moves within the range of a preset indoor device, and a preset WiFi device is used to collect WiFi signals to extract a CSI feature. The reflection path length change rate (PLCR) is calculated based on the CSI feature, and a Fresnel zone ellipse model is constructed based on the CSI feature to obtain a PLCR feature. S2. Set up and use a microphone array to collect the footsteps of the identified target, calculate the footstep AoA sequence based on the footstep signal, build a ray model, and model it according to the preset signal propagation mode and mathematical geometry theory to obtain the theoretical sample offset value n corresponding to all angles θ. sim , offset the actual measured samples by n shift The theoretical sample offset value n sim Perform matching to obtain the closest angle value, extract the angle of arrival (AoA) of each footstep from the footstep signal through sound source localization, and process it to obtain a continuous angle sequence distributed along the time domain, thereby converting the original gait feature into an AoA angle feature; S3. Confirm the user's movement trajectory through multimodal fusion; combine the PLCR features and the AoA angle features to construct a geometric model to obtain the trajectory position sequence of the user's movement; construct a speed-angle matrix, and perform a speed-angle matrix and frequency-angle matrix conversion operation based on the relationship between speed and frequency to obtain the movement direction angle at each moment from the trajectory position sequence; use the movement direction angle, speed-angle matrix, and frequency-angle matrix as indexes to match the speed in the PLCR speed spectrum to convert the PLCR speed spectrum mapping into an independent gait speed spectrum; S4. Obtain the independent speed spectra of different identified targets, build a data-driven user identification model, build and train a lightweight neural network, and obtain an applicable gait recognition model to perform gait recognition and determine the identity of the identified target.

2. The multimodal gait user identification method based on WiFi acoustics according to claim 1, characterized in that: Said S1 comprises: S11. Using a preset CSI extraction tool, extract CSI from the WiFi signal using the following logic: H(f,t)=H s (f,t)+H d (f,t)=H s (f,t)+A(f,t)e -j2πf(t) Where H s (f, t) and H d (f, t) are the static signal component and the dynamic signal component including the user's movement, respectively; S12, record the reflection path length at different times as d(t), and use the following logic to determine the Doppler frequency shift DFS: Among them, f D (t) is the Doppler frequency shift DFS, λ is the signal wavelength, and τ is the signal flight time; S13. Determine the relationship between the reflection path length change rate PLCR and the Doppler frequency shift DFS using the following logic: S14. Integrate the reflection path length change rate PLCR over time using the following logic to obtain the path length d(t):

3. The multimodal gait user identification method based on WiFi acoustics according to claim 1, characterized in that: In S2, according to the preset geometric relationship, the following logic is used to express the different microphones m in the microphone array: i 、m j The time difference Δt of receiving the sound: The following logic is used to express the sample offset n according to the time difference Δt: shift : n shift =Δt×fs According to the sample offset n shift , use the following logic to process and get the angle θ: Where L is the distance between the two microphones, c sound is the speed of sound, and fs is the acoustic sampling rate. According to the sample offset n shift , using the following cross-correlation function logic to express the cross-correlation relationship between microphone signals, according to the microphone cross-correlation relationship, the discrete point sequence C[n shift ]: C[n shift ]=∑(m i [n]·m j [n+n shift ]), Using the discrete point sequence C[n shift ], indicating the signal sequence [n+n shift ] relative to the lag of the signal sequence [n]. The discrete point sequence C[n shift ] corresponds to the subscript of the peak value to find the signal sequence [n+n shift ] relative to the sample point offset value of the signal sequence [n]:

4. The multimodal gait user identification method based on WiFi acoustics according to claim 1, characterized in that: The S2 includes: S21, set the value range of the simulation angle θ ′, bring the simulation angle into the preset geometric model, and use the following logic to obtain the simulation sample offset angle matrix n sim : Where L is the distance between the two microphones, c sound is the speed of sound, fs is the acoustic sampling rate; Preset a reference microphone, and calculate the time delay Δt of the signals received by the remaining microphones relative to the signal received by the reference microphone; S22. Obtain, by similarity calculation, an applicable theoretical sample offset that best matches the actual sample offset, and use the simulated handover θ′ corresponding to the applicable theoretical sample offset as the sound signal arrival angle: For each step of the footstep signal, the applicable theoretical sample offset is obtained by matching Get the number of steps to align the angle sequence: i seq ={θ1,θ2,...,θ k }。 5. The multimodal gait user identification method based on WiFi acoustics according to claim 4, characterized in that: In S21, each of the microphones is used as the reference microphone to expand the simulation sample offset angle matrix n sim According to the projection theorem, the following logic is used to process the theoretical sample offset Where m i , m j For different microphones i, j∈{1, 2, 3, 4, 5, 6}, Δd is the additional flight distance of the signal, is the angle vector, is the distance vector between the two microphones.

6. The multimodal gait user identification method based on WiFi acoustics according to claim 1, characterized in that: The S3 includes: S31, convert the obtained PLCR velocity spectrum into f mat (t) frequency spectrum; due to f mat The dimensions of are speed-angle-time, so through the β(t) we get, we can determine the time step and the corresponding angle, and then match the corresponding speed; S32. By combining the velocity values ​​of each time step, the entire PLCR velocity spectrum is converted into an independent velocity spectrum that is independent of trajectory and direction.

7. The multimodal gait user identification method based on WiFi acoustics according to claim 1, characterized in that: The S31 includes: S311. Determine the user location by combining the WiFi Fresnel ellipse and the angle sequence ray. The entire sensing area is divided into at least two grids, and the coordinates of each grid are labeled (x, y). The Fresnel ellipse equation is constructed using the reflection path length d(t): According to the moving direction angle θ, the angle sequence ray equation is constructed: y t =tanθ seq (t)·x t S312: Convert and process the Fresnel ellipse equation and the angle sequence ray equation to obtain an error calculation equation: S313, substituting the coordinates of each grid into the error calculation equation, performing maximum likelihood estimation, and obtaining the user position (x t ,y t ): S314, in the process of solving gait characteristics, using polar coordinates v ρ 、v θ The reflection path length change rate PLCR is expressed as: S315, obtain and construct the speed-angle matrix v according to the speed characteristics and angle characteristics mat According to the speed frequency formula, the speed-angle matrix v mat Converted into a frequency matrix to perform a mapping operation on the PLCR velocity spectrum using the following logic: The user position trajectory is obtained through processing, and the displacement direction angle β(t) is calculated based on two adjacent coordinate points.

8. The multimodal gait user identification method based on WiFi acoustics according to claim 7, characterized in that: In the S314, the v in the polar coordinates ρ Represents the velocity value, v θ Represents the direction of speed:

9. The multimodal gait user identification method based on WiFi acoustics according to claim 7, characterized in that: In the above S315, δ x and δ y is the directional coefficient determined by the location of the WiFi device: Where, Indicates the user's location, and Indicates the transmitter and receiver positions respectively.

10. A multimodal gait user recognition system based on WiFi acoustics, characterized by: The system comprises: A PLCR feature processing module is used to collect Wi-Fi signals from preset Wi-Fi devices while the target is moving within the range of a preset indoor device to extract CSI features. Based on the CSI features, the reflection path length change rate (PLCR) is calculated, and a Fresnel zone ellipse model is constructed to obtain the PLCR features. The AoA angle feature processing module is used to set up and use a microphone array to collect the footsteps of the identified target, calculate the footstep AoA sequence based on the footstep signal, build a ray model, and model it based on the preset signal propagation mode and mathematical geometry theory to obtain the theoretical sample offset value n corresponding to all angles θ. sim , offset the actual measured samples by n shift The theoretical sample offset value n sim Perform matching to obtain the closest angle value, extract the angle of arrival (AoA) of each footstep from the footstep signal through sound source localization, and process it to obtain a continuous angle sequence distributed along the time domain, thereby converting the original gait feature into an AoA angle feature; A multimodal fusion module is configured to confirm the user's movement trajectory through multimodal fusion; jointly construct a geometric model by combining the PLCR features and the AoA angle features to obtain a trajectory position sequence of the user's movement; construct a speed-angle matrix, and perform a speed-angle matrix and a frequency-angle matrix conversion operation based on the relationship between speed and frequency to obtain the movement direction angle at each moment from the trajectory position sequence; use the movement direction angle, speed-angle matrix, and frequency-angle matrix as indexes to match the speed in the PLCR speed spectrum to convert the PLCR speed spectrum mapping into an independent gait speed spectrum. The multimodal fusion module is connected to the PLCR feature processing module and the AoA angle feature processing module; A gait recognition module is used to obtain the independent speed spectra of different identified targets, build a data-driven user recognition model, build and train a lightweight neural network, and obtain an applicable gait recognition model to perform gait recognition and determine the identity of the identified target. The gait recognition module is connected to the multimodal fusion module.

Images