Fall detection method, system, terminal and medium suitable for accompanying action

Abstract

The application discloses a fall detection method and system suitable for accompanying action, a terminal and a medium, and comprises the following steps: S1, acquiring channel state information sample signal data of a user motion state through a measuring device, and performing data preprocessing on the signal data to obtain a perception signal capable of reflecting user action and keeping smooth and calm; S2, performing time-frequency analysis on the perception signal obtained after data preprocessing, and using a segmentation algorithm to find out the starting point and the ending point of the fall action, and generating a corresponding Doppler frequency shift spectrum; S3, inputting the Doppler frequency shift spectrum into a constructed fall recognition model to complete the judgment on the user motion state on the basis of eliminating the accompanying action interference. The application overcomes the shortcoming that the accuracy decreases when the traditional scheme identifies the accompanying action.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a fall detection method, system, terminal, and medium suitable for accompanying actions. Background Technology

[0002] Falls are the second leading cause of accidental death, resulting in 684,000 deaths globally each year. The vast majority of these victims are people over 60 years of age. Due to limited mobility, their lives are constantly threatened by falls. However, with the increasing aging of society, the healthcare of the elderly faces significant challenges. In this context, researchers are dedicated to designing an unmanned, automated fall detection system to provide timely medical assistance to the elderly in the event of a fall, reducing the cost and burden of manual care.

[0003] Fall detection schemes proposed in recent years can be broadly categorized into three types: 1) Wearable device-based methods; 2) Vision-based methods; and 3) Wireless signal-based methods. Wearable device-based methods use small inertial sensors worn by the user to determine their state. When a user falls, the sensors record the speed and direction of the movement. Based on the changes in these data during a fall, researchers can design corresponding detection algorithms. The advantage of this type of fall detection scheme is that it can obtain user behavior information most directly and effectively, making detection more reliable. Its disadvantage is the adverse effects of long-term wear on the elderly, specifically manifested in memory decline and skin infections. Vision-based methods detect potential fall behaviors through images or videos recorded by RGB or depth cameras. This is mainly due to the rapid development of computer vision technologies in recent years, such as object detection and semantic segmentation. The algorithm extracts key features related to fall behavior by determining the user's position in each frame and comparing changes between adjacent images. While vision-based schemes are also effective detection methods, long-term video surveillance raises concerns about privacy and security. Wireless signal-based methods utilize the reflection and refraction of ubiquitous electromagnetic waves and sound waves on the human body to sense user behavior. Its basic principle is similar to radar, except that the signal transmitter and receiver are usually located in different places. Compared to the previous two methods, wireless signal-based methods retain the non-contact characteristic while avoiding the privacy issues associated with cameras, thus attracting widespread attention from researchers. Moreover, wireless signal-based methods are more universal. They can utilize various existing communication signals, such as WiFi, radar, and sound signals. Currently, researchers have used wireless signals to implement various Human Activity Recognition (HAR) tasks, including but not limited to breathing detection, intrusion detection, gesture recognition, and user authentication.

[0004] In fall detection tasks, radar and commercial WiFi are the two most common devices, each with different application characteristics in action recognition. While radar-based solutions can provide researchers with detailed information about target location and velocity, their deployment is limited. Luis et al. used continuous wave radar (CW) to measure the Doppler Frequency Shift (DFS) of user actions. This is a two-dimensional feature image of time and frequency, which can decompose human motion according to different frequencies. Based on this image, researchers proposed feature descriptions such as EFC (Extremal Frequency Curve) and TFC (Torsor Frequency Curve), as well as action segmentation methods based on the Power Burst Curve (PBC). WiFi-based solutions mainly utilize Channel State Information (CSI) provided by the physical layer to perceive actions. Raw CSI data contains both amplitude and phase information; early fall detection schemes mined user behavioral features through amplitude signals. WiFall employs the Local Outlier Factor (LOF) algorithm to segment actions and designs seven statistical features related to amplitude. The model uses an SVM-based binary classifier, achieving approximately 87% fall detection accuracy. RT-Fall eliminates random errors in the original data by subtracting the phase between adjacent antennas, obtaining a signal more sensitive to fall behavior. Building upon WiFall, it introduces six statistical features derived from phase differences, achieving 93% accuracy using the same SVM model. FallDeFi performs time-frequency analysis on one-dimensional CSI data, using Short Time Fourier Transform (STFT) to process the amplitude signal, resulting in a two-dimensional feature image of time and frequency. By designing image features suitable for DFS and training an SVM model, FallDeFi enhances the detection system's adaptability to different types of falls, maintaining 89% accuracy across various scenarios. To further improve fall detection accuracy, DeFall proposes a template-matching-based fall detection scheme. It first uses a CSI-based velocity estimation model to learn behavioral patterns during falls, and then uses Dynamic Time Warping (DTW) to complete the matching. Fall experiments in different scenarios demonstrate that DeFall can achieve a recognition accuracy of 96%. While WiFi-based action recognition and fall detection have strong practical value, they still face many application challenges.Environmental changes affecting signal characteristics and the interconnections between different actions can all lead to a significant decrease in the accuracy of action recognition.

[0005] Existing work mainly divides action detection and recognition into two steps: 1) action segmentation; 2) action classification. A wireless transceiver continuously transmits signals during user activity, which are captured by the receiver and form several data sequences. Segmentation aims to determine which data in the sequence corresponds to the actual action, i.e., identifying the start and end times of the action. Classification aims to determine the specific content of these actions; it is responsible for extracting features from the data segments and building a recognition model, the output of which corresponds to different types of user actions. However, this recognition method relies on an important premise: each sample input into the model must contain only one action. This premise is broken when different types of actions occur consecutively. Taking fall detection as an example, although the action to be recognized is only falling, in daily life, falls often coexist with other actions, such as walking and then falling, or falling and then getting up. Since walking and getting up both accompany falls, they are called accompanying actions. In this case, a sample may consist of multiple independent actions, affecting traditional recognition models.

[0006] This problem mainly stems from two aspects: 1) the effectiveness of existing action segmentation algorithms is quite limited; 2) feature extraction for samples is global rather than local. Affected by user actions, the sampled signal at the receiving end will exhibit jitter in amplitude and phase, and the start and end times of this jitter are the action segmentation points. To determine the location of the segmentation point, researchers typically choose to calculate the first-order difference or standard deviation of the signal. When these measurements exceed a predetermined threshold, the action can be considered to have started; otherwise, it ends. The biggest drawback of this method is that the threshold needs to be determined empirically beforehand, and recalibration is required once the environment changes. Furthermore, when faced with accompanying actions, it can only identify the start point of the first action and the end point of the last action, but cannot find the time point of user behavior transition, thus it can only be segmented as a whole. The reason why accompanying actions affect the performance of classification models is that they break the one-to-one correspondence between samples in the feature space. The truly valuable content for recognition may only be a small fragment, while the rest becomes relative interference during global feature extraction. An even more serious problem is that accompanying actions can be arbitrarily combined, and their duration and change patterns are not fixed. For example, a user might fall after walking for a long time, or they might fall the instant they stand up. Global feature extraction might fail to classify these two scenarios due to their significant differences, even though they should both be classified as falls.

[0007] While several fall detection schemes based on WiFi wireless sensing have emerged in recent years, continuously improving their accuracy in fall detection, they all tend to underestimate the impact of accompanying movements, especially in cases involving prolonged walking followed by a fall. Therefore, designing a fall detection method applicable to accompanying movements is of great significance. Summary of the Invention

[0008] To overcome the drawback of decreased accuracy in traditional methods when recognizing accompanying movements, this invention provides a fall detection method, system, terminal, and medium suitable for recognizing accompanying movements.

[0009] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0010] In a first aspect, the present invention provides a fall detection method suitable for accompanying actions, comprising:

[0011] S1: Obtain channel state information containing user action characteristics through measurement equipment, and obtain a smooth and stable sensing signal through data preprocessing;

[0012] S2: Perform time-frequency analysis on the sensing signal obtained after data preprocessing, use segmentation algorithm to find the starting and ending points of the fall action, and generate a Doppler frequency shift spectrum that reflects the speed of human movement;

[0013] S3: Input the Doppler frequency shift spectrum into the completed fall recognition model to determine the user's motion state after eliminating the interference of accompanying actions.

[0014] Furthermore, the data preprocessing involves sequentially performing phase difference calculation, transition correction, and noise reduction on the signal data; the specific process of data preprocessing is as follows:

[0015] S11: Calculate the conjugate multiplication of the channel state information of two adjacent antennas and extract the phase difference signal of the two adjacent antennas;

[0016] S12: Perform a first-order difference on the phase difference signal to obtain the transition regions of π-transition and 2π-transition;

[0017] S13: Mark the transition type of each subcarrier to obtain matrix P, and sum each row of matrix P to obtain the number of votes for each subcarrier on the transition. Determine the π transition point f by majority vote.

[0018] S14: The π transition point divides the entire action sample into several segments, denoted as seg = {seg1, seg2, ..., seg...} pn+1}, pn represents the total number of π transition points. Take the segments with odd or even indices, and multiply the corresponding channel state information by their conjugates. The points are symmetrical about the origin to correct the π transition.

[0019] S15: Calculate the centroid of the complex points of the conjugate multiplication of all channel state information, rotate all data points of all subcarriers around the origin until their centroids are at the same angle to eliminate frequency domain jumps; continue to translate all complex points outward along the ray direction until all data points are in the first quadrant.

[0020] S16: Take the phase information obtained by conjugate multiplication of the adjusted channel state information, and eliminate the remaining 2π transition by unwinding; obtain the phase difference signal that has been restored to normal on each subcarrier; take the average of the phase difference signals on all subcarriers to obtain a smooth sensing signal after noise elimination.

[0021] Furthermore, the specific process of the time-frequency analysis is as follows:

[0022] S21: Perform a short-time Fourier transform on the sensing signal obtained after data preprocessing. The short-time Fourier transform is performed by truncating the signal in a sliding window, performing a fast Fourier transform on the phase difference signal within the window, and combining the results of the fast Fourier transform to obtain the Doppler frequency shift spectrum.

[0023] S22: Calculate the full-band energy burst curve PBC in the [5~90Hz] and [25~90Hz] frequency bands respectively. low and high-frequency energy burst curve PBC high If PBC high If the value is not zero, it indicates that there is a high-speed action in the sample; otherwise, discard it and record the PBC. low and PBC high The leftmost and rightmost endpoints of the non-zero energy are represented as [low1, low2] and [high1, high2], respectively; where [low1, high1] determines the left slope of energy and [high2, low2] determines the right slope of energy.

[0024] S23: Calculate the energy steps and corresponding energy noise in the intervals corresponding to the left and right slopes respectively, select the step closest to the noise level for segmentation, and finally determine the left and right endpoints of the action start and end;

[0025] S24: Extract the Doppler frequency shift spectrum between [5~90Hz] and smooth it. The resulting Doppler frequency shift spectrum can be represented as S′={W′1,W′2,...,W′ L}

[0026] Furthermore, the smoothing process in S24 is calculated using the following formula:

[0027]

[0028] Among them, Ssmooth This is the smoothed spectrum; t is the time label; f and k are both frequency labels; f h f is the highest frequency observed. l The lowest frequency of observation; S raw This is the original spectrogram.

[0029] Furthermore, the fall recognition model applicable to accompanying actions is a deep model based on RNN and attention mechanisms.

[0030] Furthermore, the specific steps of the fall recognition model applicable to accompanying actions are as follows:

[0031] S31: Extract the temporal features contained in the Doppler frequency shift spectrum through an RNN model and output the encoded feature image;

[0032] S32: Calculate the attention weights of the feature images through a weighted learning network to obtain the attention matrix for the feature vectors of a specific recognition task; where the specific recognition task refers to the fall recognition task and the walking recognition task.

[0033] S33: Multiply the attention matrix with the encoded feature image, and obtain the sentence vector representing the whole sample by weighted averaging;

[0034] S34: Feed the obtained sentence vectors to the task classifier to perform binary classification of falling and walking respectively; where the weight learning networks corresponding to falling and walking are independent of each other, and the generated sentence vectors are different;

[0035] S35: During network model training, backpropagation is performed using a weighted average of the losses from multiple task classifiers. The overall loss is denoted as Loss = r·Loss. fall +(1-r)·Loss walk Among them, Loss fall Loss is the loss of the fall classifier. walk Let r be the loss of the walking classifier, and r be the proportion of the loss of the fall classifier and the walking classifier.

[0036] Furthermore, the weight learning network in S32 consists of two additional linear layers without bias, and the additional layers are located at the output of the RNN encoder.

[0037] Secondly, the present invention provides a fall detection system suitable for accompanying actions, comprising:

[0038] Data acquisition and preprocessing module: used to acquire channel state information sample signal data of user motion state through measurement equipment, and to preprocess the signal data to obtain a perception signal that can reflect user action and remain smooth and calm;

[0039] Time-frequency analysis module: used to perform time-frequency analysis on the sensing signals obtained after data preprocessing, and use segmentation algorithms to find the start and end points of the fall action, and generate the corresponding Doppler frequency shift spectrum;

[0040] Model Recognition Status Module: This module is used to input the Doppler frequency shift spectrum into the constructed fall recognition model, and to determine the user's motion status after eliminating the interference of accompanying actions.

[0041] Thirdly, the present invention provides an electronic terminal comprising a processor and a memory, the memory storing a computer program, the processor invoking the computer program to execute the steps of the method described above.

[0042] Fourthly, the present invention provides a readable storage medium storing a computer program that, when invoked by a processor, performs the steps of the method described above.

[0043] Beneficial effects

[0044] This invention proposes a fall detection method, system, terminal, and medium suitable for accompanying actions. The method acquires channel state information (CSI) sample signal data of the user's motion state through a measurement device. By preprocessing the signal data, errors and noise in the original CSI data are removed, especially the transition errors commonly found in phase difference signals. Based on existing work, the action segmentation algorithm based on DFS spectral images is improved to identify the start and end points of the fall action and generate corresponding DFS spectral images. A fall recognition model suitable for accompanying actions is proposed, which solves the problem of unreasonable global feature extraction by introducing sentence embedding and attention mechanisms from natural language processing. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating a fall detection method applicable to accompanying actions provided by an embodiment of the present invention;

[0047] Figure 2 This is a phase difference signal jump correction provided in the embodiments of the present invention; wherein, Figure 2 (a) represents the original phase difference between two adjacent two lines; Figure 2 (b) is the first-order difference of the original phase difference; Figure 2(c) is the conjugate multiplication of the CSIs of two adjacent antennas; Figure 2 (d) is the result of averaging the phase difference signals on all subcarriers after the correction transition;

[0048] Figure 3 This is a time-frequency analysis of a fall action provided in an embodiment of the present invention; wherein, Figure 3 (a) is the original DFS spectral image of a standing fall; Figure 3 (b) is the smoothed DFS spectrum image; Figure 3 (c) Multi-level PBC for DFS of fall action; Figure 3 (d) represents the energy step near the starting position of the fall.

[0049] Figure 4 This is a schematic diagram of the overall structure of a fall recognition model applicable to accompanying actions provided in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of sentence embedding based on an attention mechanism provided in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the experimental setup for testing the fall detection system provided in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram illustrating the impact of accompanying actions on FallDeFi recognition performance, provided in an embodiment of the present invention.

[0053] Figure 8 This is a schematic diagram comparing the DFS spectrum of different actions provided in the embodiments of the present invention; wherein, Figure 8 (a) indicates that walking has stopped; Figure 8 (b) Falling while walking;

[0054] Figure 9 This is a schematic diagram illustrating the attention weights for different actions provided in an embodiment of the present invention; wherein, Figure 9 (a) indicates that walking has stopped; Figure 9 (b) Falling while walking;

[0055] Figure 10 This is a performance comparison diagram of different encoder models and different sentence embedding methods provided in the embodiments of the present invention; Figure 10 (a) Using LSTM / GRU as the encoder; Figure 10 (b) Using Transformer as the encoder;

[0056] Figure 11 This is a comparative schematic diagram provided by the embodiments of the present invention with previous work;

[0057] Figure 12This is a schematic diagram illustrating the impact of changing the loss ratio of different tasks in multi-task learning on accuracy, provided by an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Example 1

[0060] like Figure 1 As shown, this embodiment provides a fall detection method suitable for accompanying actions, including:

[0061] S1: Acquire channel state information (CSI) sample signal data of user motion state through measurement equipment, and perform data preprocessing on the signal data to obtain a perception signal that can reflect user actions and remain smooth and calm.

[0062] In this embodiment, the measurement device used is the new CSI measurement device Intel AX200 network card. The AX200 wireless network card has three antennas, each of which can acquire 57 subcarriers. In specific implementations, the measurement device can be selected according to actual needs, and no limitation is made here.

[0063] Furthermore, the data preprocessing involves sequentially performing phase difference calculation, transition correction, and noise reduction on the signal data; the specific process of data preprocessing is as follows:

[0064] S11: Calculate the conjugate multiplication of the CSI of two adjacent antennas, and extract the phase information of the result, i.e., the phase difference signal φ = {φ1, φ2, ..., φ3} of the two adjacent antennas. M}, where M represents the total number of subcarriers acquired; and vector This represents the phase difference on the m-th subcarrier, and N represents the total number of data packets, i.e., the duration of the sample.

[0065] This invention selects the phase difference of CSI as the basic sensing signal for two main reasons: 1) Compared to amplitude signals, phase difference is more sensitive to environmental changes and can detect a wider range of user falls; 2) When the AX200 is used as a receiver to measure CSI, its signal amplitude will produce random up-and-down jumps, which will cause serious noise interference, and no effective means of elimination has yet been found. Since the receiving chain of multiple antennas on the network card usually shares some hardware, they are also affected by errors from within the hardware in the same way. Typical errors, such as carrier frequency offset (CFO) and sampling frequency offset (SFO), can be eliminated by subtraction. However, experiments have shown that the signal after subtraction still has four serious jump phenomena: 1π jump, 2π jump, small-scale jump, and frequency domain jump.

[0066] S12: Perform a first-order difference on the phase difference signal to obtain the transition regions of π-jump and 2π-jump. Due to the existence of the transitions, the resulting signal will converge near kπ (k = 0, ±1, ±2, ...) and 2kπ (k = 0, ±1, ±2, ...). The former corresponds to the π-jump and the latter corresponds to the 2π-jump, which divide the signal into two parts.

[0067] like Figure 2 As shown in (a), 1π and 2π transitions coexist in the original phase difference, creating multiple copies of the original signal. Traditional dewinding is no longer applicable because the transition distance is not simply 2π, but kπ (k = 0, ±1, ±2, ...). Furthermore, the positions of these transitions are completely random. An important observation is that the frequency of 1π transitions is significantly lower than that of 2π transitions. If the two transitions are distinguished and labeled, and all odd-multiple-π transitions are corrected to even-multiple-π transitions, the transitions can be unified. Afterward, the original signal can be easily recovered through modulo 2π and dewinding. The purpose of first-order differential is to distinguish these two transitions. Figure 2 As shown in (b), all points with 2π transitions cluster around 2kπ (k = 0, ±1, ±2, ...) after differencing, while all points with 1π transitions cluster around 2(k+1)π. By setting a fixed-length window (2 rad for example) centered on these points, the transition type of each point can be easily identified.

[0068] S13: Mark the transition type of each subcarrier to obtain the matrix P = {p1, p2, ..., p...} M}, where M represents the total number of subcarriers acquired, p m，t=1 indicates that the phase difference result corresponding to this point falls within [kπ-1, kπ+1], and vice versa; m is the label of the m-th subcarrier; t represents the time label. The summation of each row of matrix P yields the number of votes for each subcarrier's transition; the π transition point is determined by the majority vote. In this embodiment, if the number of votes exceeds half the total number of subcarriers, it is recorded as the π transition point; otherwise, it is discarded.

[0069] During the experiment, the determination method based on a fixed-length window was prone to errors. Especially when the original signal jitter was severe, the signal difference could be larger than expected. However, the location of the π transition was the same for each subcarrier of the CSI. For the same user action, although the CSI change trends on different subcarriers were similar, the degree of noise impact on each path differed. Therefore, all 57 subcarriers were marked, and the number of π transitions at each time point was counted. When the number exceeded half the number of subcarriers (i.e., 29), the π transition at that point was confirmed.

[0070] S14: The π transition point divides the entire action sample into several segments, denoted as seg = {seg1, seg2, ..., seg...} pn+1}, pn represents the total number of π transition points, taking segments with odd or even indices, such as seg sel ={seg1, seg3, ..., seg 2k+1 The point obtained by multiplying the corresponding channel state information by its conjugate is symmetric about the origin, thus correcting the π jump.

[0071] like Figure 2 As shown, the breakpoint created by the π transition divides the original signal into several segments. The segments numbered even and odd differ by a distance of π. If the goal is simply to correct the π transition, then simply raising or lowering the even or odd segments by π would suffice. However, for better correction of frequency domain transitions, the conjugate multiplication of the CSI is chosen, which is essentially a complex combination of the amplitude product and the phase difference. Viewed from the complex plane, the π transition causes the complex points to become centrally symmetric about the origin, while the 2π transition has no effect. Given the marked positions, the segment with the smaller proportion of length in both even and odd segments is corrected by flipping them to the other side of the origin (negating the real and imaginary parts).

[0072] For a single antenna, the extracted CSI data can be represented as a matrix H(f, t); where f represents subcarriers of different frequencies and t represents the time stamp of the sampling point. Each data point in the matrix is ​​a complex number, its amplitude reflecting the energy of the received signal, while the phase angle is related to the signal propagation distance. Due to the multipath effect, H(f, t) is the result of the combined effect of multiple paths. In WiFi sensing, researchers often divide H(f, t) into a static component H0. s (f, t) and dynamic component H d The equation (f, t) consists of two parts: the former corresponds to stationary objects in the environment, and the latter corresponds to changing user actions. The conjugate multiplication is performed on H1(f, t) from both antennas. Perform element-wise multiplication. Here, H1(f, t) represents the channel state information measured by the first antenna; This represents the conjugate complex number of the channel state information obtained from the measurement of the second antenna. It is an effective way to observe changes in static and dynamic components, and its calculation formula is shown in equations (1) and (2).

[0073]

[0074]

[0075] Where a(f,t) represents the signal energy attenuation on the dynamic component, a1(f,t) and a2(f,t) represent the energy attenuation of the dynamic components of the first and second antennas, respectively, j corresponds to the complex number symbol, d(t) represents the propagation path length, d1(t) and d2(t) represent the lengths of the dynamic reflection paths on the first and second antennas, respectively, λ represents the signal wavelength, and H conj (f, t) represents the result of conjugate multiplication, H s，1 This represents the static component of the first antenna. This represents the conjugate complex number representing the static component of the second antenna. It can be seen that the result of the conjugate multiplication contains four components. When a line-of-sight (LOS) path is maintained between the signal transmitter and receiver, the static components can all be considered constants; the key to change lies in d(t). Based on previous research, as the user performs actions, the CSI conjugate multiplication will move in a circle or ellipse around a point in the complex plane. The center of the circle depends on the static component, while the radius and speed of motion depend on the action itself. However, when the dynamic component dominates, the circular motion is likely to cover the origin, causing the phase to jump across 2π, resulting in an unnatural transition, such as... Figure 2 As shown in (c), since the center position gradually drifts with the change of subcarrier frequency, some subcarriers will have jumps while others will not, which is called frequency domain jump.

[0076] S15: Calculate the centroid of the complex points of the conjugate multiplication of all channel state information, rotate all data points of all subcarriers around the origin until their centroids are at the same angle to eliminate frequency domain jumps; continue to translate all complex points outward along the ray direction until all data points are in the first quadrant.

[0077] The centroids of all complex points are calculated in the complex plane. Due to environmental disturbances, static components cannot remain completely stable in practical applications. Furthermore, the randomness of user limb movements makes it difficult to determine the center position of dynamic components, thus making it impossible to calculate the true phase change caused by the action. However, the speed of the user's movement is still reflected in the phase of the synthesized vector. The goal of the adjustment is to unify the subcarriers and eliminate the differences caused by frequency domain jumps. Therefore, the complex points are chosen to be far from the origin.

[0078] Rotate all complex points using the current centroid as a reference. Rotation in the complex plane corresponds to a phase shift (up / down). While it doesn't change the trend of the phase difference signal, it shifts the quadrant region where the complex points are located. In the experiment, the centroid of the complex points is uniformly rotated to 45°. If the data doesn't show a frequency jump at this point, directly calculating the phase angle can eliminate the previous 2π jump.

[0079] All complex points are translated outward along the ray direction. For data with frequency domain jumps, its coverage must include the origin. Based on the determined boundary, the complex points are moved outward along a 45° angle ray until all points are located in the first quadrant. The reason for translating along the ray direction is that this minimizes the distortion of the phase difference signal. The distance the complex points move depends on the farthest boundary; the farther the boundary, the longer the movement and the greater the distortion. This is done at the cost of effectively eliminating frequency domain jumps.

[0080] S16: Take the phase information obtained by conjugate multiplication of the adjusted channel state information, eliminate the remaining 2π transition by unwinding, and obtain the phase difference signal that has been restored to normal on each subcarrier; take the average of the phase difference signals on all subcarriers to obtain a smooth sensing signal after noise elimination.

[0081] Through the above steps, a relatively smooth phase difference signal can be obtained on 57 subcarriers. However, the small-scale transitions remain unresolved. Unlike π and 2π transitions, these small-scale transitions cause small fluctuations of varying degrees to each subcarrier signal, and the magnitude of these fluctuations changes linearly with the subcarrier frequency. The 29th subcarrier, i.e., the center frequency, is almost unaffected. In contrast, the phase difference of the 1st subcarrier increases, while that of the 57th subcarrier decreases. Utilizing this characteristic, averaging across the 57 subcarriers allows the transitions of the upper half of the subcarriers to cancel each other out with those of the lower half, as shown in the figure. Figure 2 As shown in (d), the signal at this point is not only free from the effects of the transition, but also achieves good noise reduction.

[0082] S2: Perform time-frequency analysis on the sensing signal obtained after data preprocessing, use segmentation algorithm to find the starting and ending points of the fall action, and generate a Doppler frequency shift spectrum (i.e., DFS spectrum image) that reflects the speed of human movement.

[0083] Furthermore, the specific process of the time-frequency analysis is as follows:

[0084] S21: Perform a short-time Fourier transform (STFT) on the sensing signal obtained after data preprocessing. STFT is obtained by truncating the signal in a sliding window, performing a fast Fourier transform (FFT) on the phase difference signal in the window, and combining the results of the FFT to obtain the Doppler frequency shift spectrum.

[0085] This embodiment uses STFT to perform time-frequency analysis on the preprocessed phase difference signal. As can be seen from formulas (1) and (2), the motion of the target under test will cause the complex points of the CSI after conjugate multiplication to move in a circle or ellipse on the complex plane, and the phase difference signal will also change accordingly. Within a fixed time, the faster the phase difference changes, the faster the reflection path length changes, and the greater the moving speed of the target under test. By performing FFT on the phase difference signal within the sliding window, STFT can obtain the frequency and intensity of motion changes within a certain time period. Combining the FFT results at several times gives the DFS spectrum image, which can also characterize the characteristics of motion frequency and intensity changing with time. In this embodiment, the window length is 512 and the step size is 1. For each segment of signal, the average is subtracted first and then the Hamming window function is applied, and zeros are filled up to 2048 before performing FFT. The final spectrum is represented as S = {W1, W2, ..., W...} T}, W T = [w1, w2, ..., w K ] T This represents the FFT result for each sliding window. K represents the number of all frequency points, and T represents all time points.

[0086] Figure 3(a) shows the original DFS spectrum image of a standing fall. It can be observed that the fall has significant high-frequency components. Compared to the micro-movements of the body before the fall (located on the left side of the image), the highest frequency of the fall can reach approximately 60-70 Hz. Further observation of the energy distribution within the fall reveals a dividing line that separates it into two parts with higher amplitude at the beginning and lower amplitude at the end, forming a cone-like outline. The higher amplitude of the first half corresponds to the acceleration of the body during the fall, while the lower amplitude of the second half corresponds to the deceleration and oscillation of the body after impact. There is a moment of rapid energy decay between the two, which is an important characteristic of the fall.

[0087] To extract features of a fall and achieve pattern recognition, the time-frequency analysis module performs two tasks on the DFS spectral image: 1) action detection and segmentation; 2) image preprocessing. The former determines the start and end points of the action based on the frequency and energy of the DFS, while the latter extracts the action segments from the original DFS spectral image and converts them into a form suitable for model processing.

[0088] S22: Using the PBC calculation formula (Formula (4)), the full-band energy burst curve PBC is obtained in the two frequency bands [5~90Hz] and [25~90Hz]. low and high-frequency energy burst curve PBC high If PBC high If the value is not zero, it indicates that there is a high-speed action in the sample; otherwise, discard it and record the PBC. low and PBC high The leftmost and rightmost endpoints of the non-zero energy are represented as [low1, low2] and [high1, high2], respectively; where [low1, high1] determines the left slope of energy and [high2, low2] determines the right slope of energy.

[0089] This embodiment employs a motion detection and segmentation scheme based on multi-level PBC (Power Burst Curve). PBC accumulates the energy within a specific frequency band of the DFS (Dynamic Free Spectrum Image) to reflect the characteristics of energy changes over time. If the original DFS spectrum image is denoted as S... raw (t, f), frequency band [f] l f h The corresponding PBC can be represented by equation (3); where f l f represents the highest frequency of observation. hThis represents the lowest observed frequency. Utilizing the fact that the human body's speed changes rapidly and its frequency energy is high during a fall, a simple detection and segmentation scheme is proposed: a certain energy threshold is pre-set. When the amplitude of the PBC exceeds this threshold, the system determines that the target has performed an action and records the time exceeding the threshold as the start point of the action; when the amplitude is below the threshold, the system determines that the target's action has ended and records the time below the threshold as the end point of the action.

[0090]

[0091] However, threshold-based segmentation methods are susceptible to noise and motion randomness, such as natural user body tremors. A fixed energy threshold cannot dynamically adapt to these changes, often resulting in discrete segments on the PBC (Programmable Component Block). The system struggles to discern the relationships between these segments, leading to increased segmentation errors. To improve motion detection and segmentation, three improvements are made to the traditional PBC-based threshold method:

[0092] Improvement 1: Utilize multi-level PBC to determine the occurrence and location of the action. Observing the DFS spectrum image reveals that interference from environmental noise and user body vibrations is mainly concentrated in the low-frequency region. When calculating the overall PBC for both low and high frequencies, this noise appears on both sides of the core action, interfering with the cutting. To address this, the lower boundary of the PBC frequency band is appropriately increased, and the energy burst curves in the [5–90Hz] and [25–90Hz] ranges are observed and denoted as PBCs respectively. low and PBC high .like Figure 3 As shown in (c), the full-band energy burst curve PBC is affected by noise interference. low There are two peaks, but only the latter truly corresponds to the fall motion; the high-frequency energy burst curve PBC high By shielding itself from low-frequency interference, it pinpoints the true location of the peak of the fall motion. Therefore, the PBC... high Consider it as a trigger; as long as the frequency component of the target action exceeds 25Hz, i.e., PBC... high If the energy is greater than 0, the system executes the segmentation process; otherwise, it is ignored by default. This serves to qualitatively filter out irrelevant minor actions. However, increasing the lower boundary of the frequency band leads to the loss of some spectral content, which is detrimental to model recognition. Therefore, in subsequent processing, we still extract the content of the entire frequency band as the model input, and the high-frequency band energy burst curve PBC... high It is only used to detect the occurrence of a fall.

[0093] S23: Calculate the energy steps and corresponding energy noise in the intervals corresponding to the left and right slopes respectively, select the step closest to the noise level for segmentation, and finally determine the left and right endpoints of the action start and end.

[0094] Improvement 2: Utilizing energy steps to find motion boundaries. While multi-level PBC achieves target motion detection and localization, it cannot pinpoint the exact moment of cut. To address this... Figure 3 The two peaks in (c) are separated, introducing an energy level. Energy levels are linearly selected between the minimum and maximum values ​​of the DFS. Each time an energy level is selected, the portion of the DFS below that level is set to 0. Then, the overall PBC of the image is calculated, and the left and right endpoints of consecutive segments with energy greater than 0 are recorded, i.e., the PBC. low and PBC high The portion greater than 0 defines two intervals in the time domain, with the latter always contained within the former. The non-overlapping portions of these two intervals correspond to the left and right slopes of the energy peak, respectively. The endpoints closest to the peaks within these intervals are then selected as the results, until all DFS energies are 0. As the energy level increases, the endpoint positions gradually move closer to the peak. A noteworthy characteristic is that the endpoint positions create a gentle step at locations where the DFS energy changes abruptly, while rapidly contracting at locations where the energy change is slow, such as... Figure 3 (d). This step-like change actually delineates the contour boundary of the DFS energy distribution, and each abrupt change is a candidate action cut-off point;

[0095] Improvement 3: Adaptive Noise Estimation. During the gradual shrinking towards the peak, multiple different energy steps may be obtained. To select the most suitable step for segmentation, an adaptive noise estimation method is introduced. The basic idea is to determine a noise estimation range [f] on the DFS spectral image. l f n ], f n The highest frequency of the noise is represented by 25Hz. Calculate the mean value u of the energy change over time at each frequency. f With variance σ f The average of these linear combinations of statistical characteristics is taken as the PBC threshold (see formula (4) for specific calculation). This noise estimate will be related to the PBC. low The intersections are obtained, and several intersection points are obtained. The distance PBC is taken. high The nearest reference point is the appropriate action cut-off point, and the energy step closest to that reference point is the one from which the action is cut off.

[0096]

[0097] S24: Extract the Doppler frequency shift spectrum between 5 and 90 Hz (the content between 25 and 90 Hz is used to better detect falls and shield against interference from low-frequency energy, but the high-frequency content is incomplete, so the entire spectrum between 5 and 90 Hz is used). Smooth the spectrum using the PBC calculation formula, and then use the PAA algorithm to reduce the image size for easier model reading. The resulting Doppler frequency shift spectrum can be represented as S′={W′1,W′2,...,W′ L}; where W′ L This represents the frequency domain features corresponding to each sliding window after smoothing, and L represents the length of the image after segmentation.

[0098] Building upon motion detection and segmentation, this embodiment further performs image preprocessing on the DFS spectral segment. The purpose of image preprocessing is data augmentation, aiming to filter out information that is not helpful for fall recognition and enhance the energy change characteristics during the fall process. For example... Figure 3 As shown in (a), the energy distribution of the original DFS is uneven, and there are several irregular depressions inside the fall action. These energy depressions hinder fall recognition, mainly for two reasons: 1) Experiments have shown that these depressions have strong randomness. Even if the action is the same, the location of the depression may be different. This means that it cannot reflect the intrinsic characteristics of the action well; 2) The rapid decay of energy upon contact with the ground is an important indicator of a fall, and the fluctuations caused by the depressions will seriously interfere with it as noise. In response, this embodiment proposes a smoothing scheme for the DFS spectrum, the specific representation of which is shown in formula (5).

[0099]

[0100] Among them, S smooth The spectrum is the smoothed Doppler frequency shift spectrum; t is the time label; f and k are both frequency labels; f h The highest frequency observed is 90 Hz; f l The lowest frequency to be observed, i.e., 5Hz; S raw S represents the original Doppler frequency shift spectrum.

[0101] S smooth (t, f) is essentially composed of multiple weighted PBCs stacked together, and each row corresponds to a frequency range of [f, f] in the original spectrum. hThe PBC (Pulse Curve) is defined as follows: From high to low frequencies, the frequency range of the PBC gradually increases, and the sum of its energies also gradually increases. During this process, the spectral profile remains unchanged, and the original dips are filled by the accumulated energy. However, the accumulation of energy causes the amplitude of the low-frequency components to always be too high, thus masking the high-frequency components of the fall motion. Therefore, a weight is applied to the frequency position of each PBC, with its magnitude decreasing linearly from high to low frequency. Figure 3 (b) shows Figure 3 (a) Results of spectrum smoothing. It can be seen that the smoothed low-frequency components are well suppressed, while the high-frequency components, due to their low energy, are not overly emphasized. The significantly enhanced regions are mainly concentrated in the mid-frequency range. The rise and fall of energy before and during ground contact are amplified, while the body's cushioning after ground contact is weakened.

[0102] S3: Input the Doppler frequency shift spectrum into the constructed fall recognition model to determine the user's motion state after eliminating the interference of accompanying actions.

[0103] This embodiment presents a fall recognition model for accompanying actions, which is a deep model based on sentence embedding and multi-task learning with an attention mechanism, designed to overcome the influence of accompanying actions on fall recognition. The model uses a smoothed DFS spectral image as input to determine the user's movement state, including whether they are walking and whether they have fallen. The model's structure is as follows... Figure 4 As shown, it consists of three modules: 1) sequence encoder; 2) sentence embedding module; 3) task classifier.

[0104] Furthermore, the fall recognition model applicable to accompanying actions is a deep model based on RNN and attention mechanisms.

[0105] Furthermore, the specific steps of the fall recognition model applicable to accompanying actions are as follows:

[0106] S31: Extract the temporal features contained in the Doppler frequency shift spectrum using an RNN model and output the encoded feature image. Specifically, encode the DFS spectrum image using an RNN model and output the feature image; specifically: encode the DFS spectrum image using an RNN model (LSTM / GRU), and the output feature image can be represented as E = {E1, E2, ..., E...} L}; where L represents the length of the image after cropping; It represents the frequency domain characteristics of each sliding window after encoding, d e Indicates the feature dimension of the output;

[0107] Previous work on DFS spectral images has largely employed CNN models because CNN convolution operations, based on two-dimensional pixels, excel at extracting and recognizing shape features hidden in the spectrum. However, the image size input to the CNN model must be fixed, requiring all CSI samples to have equal durations. In some applications, the duration of user actions is relatively similar, and interpolation can address the issue of inconsistent sample lengths. But for fall recognition with accompanying actions, the sample length is primarily determined by the accompanying action; different fall samples not only vary in length but also differ significantly from each other. Therefore, RNN models capable of handling variable-length data are used to extract features.

[0108] For a given CSI sample of length T, a DFS spectral image S of length T and width K is obtained through STFT, where the size of K depends on the desired frequency range (5–90 Hz). After action clipping and smoothing, the image length is shortened to L. If each column of the spectrum S is considered as a K-dimensional feature vector W... t = (w1, w2, ..., w K ) T Then the entire image can be viewed as a sequence S = (W1, W2, ..., Wn) consisting of L feature vectors. L This can be represented in this way because DFS itself has obvious temporal characteristics; each column of its FFT output corresponds one-to-one with the original signal. These feature vectors are sequentially input into the RNN model to obtain a sequence output E = {E1, E2, ..., E...} of the same length as the input DFS. L};in, It contains temporal characteristics of DFS energy changes, d e Indicates the feature dimension of the output.

[0109] S32: Calculate the attention weights of the feature images using a weighted learning network to obtain the attention matrix for the fall recognition task and the walking recognition task; the formula for calculating the attention matrix is ​​as follows:

[0110]

[0111] in, and These represent the internal weights of two linear layers in the weight learning network; the resulting attention matrix is ​​represented as A = (a1, a2, ..., a...). R ) T Where R represents the number of attention heads, and vector a r The weight corresponding to the r-th attention head.

[0112] S33: Multiply the attention matrix by the encoded feature image, and obtain the sentence vector representing the entire sample by weighted averaging, denoted as:

[0113] While RNN models solve the feature extraction problem for variable-length samples, the number of feature vectors they output still varies, which doesn't match the input requirements of subsequent classifier tasks. Therefore, a selection strategy or transformation method is needed to combine the multiple feature vectors corresponding to the original sequence into a single vector that best summarizes the fall information in the original sequence. In NLP, this task of recognizing multiple one-dimensional signals is called sequence classification, where the sequence itself corresponds to sentences in text, and the output feature vectors correspond to word vectors. The process of converting multiple word vectors in a sentence into a single vector suitable for a classifier is called sentence embedding, and this vector is also called a sentence vector.

[0114] The simplest and most commonly used sentence embedding method is to extract fixed word vectors, such as the first or last one. For RNN models (LSTM and GRU) that process input sequences strictly in chronological order, extracting the last one is the most reasonable. This is because the model has already fully read each word in the sequence before this point, and the output feature vector will contain all the information of the sequence. For models using attention mechanisms (Transformer), the extraction position can be arbitrary. Since each input word in the sequence participates in the attention calculation with the others, all word vectors carry the complete information of the sentence. Through training guidance, the model's attention will spontaneously focus on the extracted sentence vector, making it a suitable feature representation. In practical applications, researchers often add blank CLS markers at the beginning and end of sentences to indicate the extraction position of the sentence vector, to prevent the content of the word vectors themselves from affecting the sentence embedding. However, extraction-based sentence embedding methods discard other feature vectors output by the model. This incomplete utilization of information can miss key information in the sequence, especially when the duration of the accompanying action is much longer than the fall action; the truly valuable features may only account for a small portion of the sequence. To address this issue, an attention-based sentence embedding method was introduced.

[0115] like Figure 5 As shown, a weight learning network consisting of two unbiased linear layers is added to the output of the RNN encoder. The first linear layer is Kindin. a Its internal weights can be represented as a vector of length d. a A matrix W1 of width K. The second linear layer is d. a The input is R and the output is R. Its internal weight is represented as a vector with length R and width d. aThe matrix W2. Tanh is used as the activation function between the two linear layers, and the final output is normalized using softmax. The encoded sequence E = (E1, E2, ..., E...) is... L The matrix ) is represented as a matrix of length L and width K, which can be processed by a linear network to obtain a matrix. This matrix represents the learned attention, which can be expressed as A = (a1, a2, ..., a...). R ) T Where vector a r The sum is 1, recording the weight assigned to each word vector. R represents the number of attention heads, each attention vector being independent and focusing on different content in the sequence. At this point, multiplying the attention matrix by the encoding sequence and then averaging the results yields the sentence vector.

[0116] S34: Feed the obtained sentence vectors to the task classifier to perform binary classification of falling and walking respectively; where the weight learning networks corresponding to falling and walking are independent of each other, and the generated sentence vectors are different;

[0117] A multi-task learning model with two binary classifiers is employed. One classifier determines whether the input sequence sample contains a fall, while the other determines whether it contains a walking action. Multi-task learning offers two main benefits: 1) By using loss functions from two different tasks during backpropagation, the same samples can be reused, preventing overfitting and further enhancing feature extraction capabilities; 2) Although the two task classifiers share the same RNN encoder, they use two parallel weight learning networks. This means that, with the help of an attention mechanism, the two classifiers can extract different features from the encoded sequence according to the needs of their respective tasks. The fall classifier focuses its attention weights on the region where the fall occurred, while the walking classifier focuses on the walking region. When the fall action is concatenated with an accompanying action, the attention weights of the sentence embedding help to pinpoint the actual time of the fall in the sequence.

[0118] S35: During network model training, backpropagation is performed using a weighted average of the losses from multiple task classifiers. The overall loss is denoted as Loss = r·Loss. fall +(1-r)·Loss walk Among them, Loss fall Loss is the loss of the fall classifier. walk Let r be the loss of the walking classifier, and r be the proportion of the loss of the fall classifier and the walking classifier.

[0119] Specifically, when using two task classifiers, cross-entropy is used to calculate the classification loss function. Assume the loss of the fall classifier is Loss. fall The loss of the walking classifier is Loss walk The backpropagation loss is obtained by weighted averaging of the two parts, and its calculation formula is shown in (6). Among them, the parameter r is used to control the proportion of the two parts of the loss.

[0120] Loss = r·Loss fall +(1-r)·Loss walk (6)

[0121] To verify the effectiveness of this invention for fall detection methods accompanied by actions, in Figure 6 The experiment was conducted in the environment shown (a standard conference room, 9m long and 6m wide). The experiment consisted of two parts: CSI data acquisition and model training and testing. During the data acquisition phase, WiFi transceivers (two PCs, one equipped with an AX200 and the other with an IWL5300 network card; the type of transceiver can be chosen based on actual needs and is not limited) were placed at opposite ends of the conference table, with a line-of-sight (LOS) distance of 330m between them. Three fall test locations (numbered 1-6) were located on each side of the conference table, each 177cm vertically from the LOS and 162cm apart. Ensuring no obstructions between the transceivers, volunteers simulated falls at each of the six locations. For each fall, the IWL5300 was used as the transmitter, transmitting WiFi data packets at a frequency of 1kHz on a 5.5GHz channel, with a bandwidth of 20MHz for each transmission. The AX200, acting as the receiver, captured these data packets and converted them into raw CSI files. The above operations for controlling the network card to send and receive are completed with the help of the middleware software PicoScenes.

[0122] Four types of samples were collected to construct the dataset: 1) Stand-Fall falls while the user is standing still; 2) Walk-Fall falls while the user is walking; 3) In-situ Acts performed by the user in place; and 4) Walk-Stops performed by the user in daily life. In-situ Acts included: 1) standing up / sitting down from a chair; 2) bending down to pick up an object from the ground; 3) squatting and standing up; and 4) jumping upwards in place. Since Stand-Fall and In-situ Acts are performed without movement, their durations are short and relatively uniform, generally occurring within 3–5 seconds. In contrast, Walk-Fall and Walk-Stops are significantly longer and more random in duration. Walk-Fall is considered representative of accompanying actions because, although it consists of both walking and falling, it still appears as a complete and continuous action in terms of signal. The target fall and the accompanying walking are interconnected, causing significant interference with recognition. To examine the impact of interference and test the effectiveness of the model, Walk-Fall and Walk-Stop samples of varying durations were collected, ranging from 8 seconds to 18 seconds. The final total number of samples was 169, which were divided into training and testing datasets in a 7:3 ratio. The distribution of samples in each class is shown in the table below.

[0123]

[0124] During model training, eight deep neural networks were constructed based on different combinations of encoder and sentence embedding modules. For the encoder module, GRU and Transformer were chosen as the research subjects. The GRU model is relatively shallow, containing only two hidden layers, and the feature dimension of each layer's output is consistent with the width of the input DFS spectrum image (175 frequency points). For Transformer, since it only needs to extract features from the DFS spectrum image, the decoder module was omitted. In this embodiment, the remaining encoder module uses a 6-layer network stack, with 5 attention heads in each layer, a dropout parameter of 0.1, and a feature dimension of 512 for the feedforward network. Similar to GRU, the final feature image size encoded by Transformer is also consistent with the input DFS spectrum image.

[0125] For the sentence embedding module, four methods were selected, along with the previously introduced attention methods: 1) based on special markers; 2) based on max pooling; 3) a hybrid method based on special markers and max pooling; and 4) based on attention mechanisms. The special marker-based method adds a placeholder (CLS) without any semantic information to the beginning or end of the sequence. The training guides the model to focus information about the entire sequence on the CLS, and finally, only the CLS is extracted and sent to the downstream task classifier. The max pooling-based method takes the maximum value of each feature of the encoded sequence along the time dimension, combines them into a feature vector, and then sends it to the downstream task classifier. The hybrid method combines CLS and max pooling, passing it through linear and max pooling layers before DFS spectral image encoding. The resulting feature vector is treated as a CLS inserted at the beginning of the sequence and fed to the RNN model. The attention mechanism-based sentence embedding differs from the above three methods in that it generates different sentence vectors based on two attention matrices, which traditional sentence embedding schemes cannot do. Therefore, for the remaining three sentence embedding modules, the same sentence vector is sent to both task classifiers simultaneously.

[0126] Subsequent fall and walk classifiers are constructed using two fully connected layers. During training, stochastic gradient descent (SGD) is used as the optimizer, with a learning rate of 0.05. Due to the relatively small number of samples in the dataset, this embodiment sets 100 epochs for training the model, with each batch containing 10 samples. Since fall and walk detection are both binary classifications, the model performance is measured using accuracy (Accuracy), precision (Precision), recall (Recall), and F-measure (F1S), and their calculation formulas are shown in (7). Here, TP and TN represent correctly judged fall and non-fall samples, respectively, and FP and FN represent incorrectly judged fall and non-fall samples, respectively.

[0127]

[0128] like Figure 8 As shown, to illustrate the effectiveness of the attention-based sentence embedding scheme, the performance of the fall classifier was tested with the encoder fixed. Figure 9(a) When using GRU as the encoder, attention-based and max-pooling sentence embeddings are effective, achieving 93% and 90% recognition accuracy, respectively, on datasets with accompanying actions. Conversely, hybrid methods and sentence embeddings based on special markers fail to train successfully under the same settings, their loss functions are difficult to reduce, and they only achieve around 50% accuracy. Both of these approaches extract only one part of the encoded sequence as the sentence vector, and their poor performance suggests that other discarded encoded sequences may contain more important features related to falls than the end of the sequence. When samples are affected by accompanying actions, using only one output cannot reflect the presence of a fall. Figure 9 (b) When using Transformer as the encoder, the performance of the four sentence embeddings is similar. The attention-based scheme is the best, achieving 95.8% accuracy; the hybrid scheme is second best, achieving 94% accuracy; and the scheme based on max pooling and special markers is the worst, at 93% and 92% respectively. It can be seen that while considering sentence embeddings of all encoded sequences can improve performance, its impact is far less significant than that of GRU. Benefiting from Transformer's own attention mechanism, it forms effective feature representations through training, something the simpler GRU cannot achieve. However, compared to Transformer, traditional GRU has two advantages: 1) it requires far less time and space for training than Transformer, and has lower data requirements; 2) the output feature vectors maintain their causality, meaning each encoded vector is only related to the preceding input sequence and not to the content following the sequence. Therefore, combining this characteristic with sentence embeddings allows us to determine which part of the sequence the task classifier is biased towards through attention weights.

[0129] To observe the regions of interest for each task classifier, two samples were selected as examples. Figure 7 Two preprocessed DFS spectral images are shown: one for walking to a standstill and the other for walking to a fall. It can be seen that both actions exhibit energy spikes in their spectra, but the spike for walking to a standstill is located in the first half of the action, while the spike for walking to a fall is located at the end. The remaining portions correspond to walking, and there is no significant difference between the two samples. Figure 8These are the attention weights extracted from two images after passing through a GRU and sentence embedding module (averaged across different attention heads). It can be observed that the fall classifier is highly interested in the energy spikes of DFS, successfully capturing both peak regions. However, the weight corresponding to the energy spike at the end of the fall is significantly higher than that at the point where walking stops. For the walking classifier, the weight distribution is more dispersed, and all weights are basically at the same level. A key phenomenon is that the attention of the fall classifier is exactly the opposite of that of the walking classifier. In the area where the fall occurs, the walking classifier's attention shows a trough, while the fall classifier shows a peak. Therefore, the exact moment the fall actually occurred can be roughly inferred. Figure 8 (b) The intersection of the two attention curves is around 10.2s, which corresponds to the time when the fall occurs.

[0130] To further illustrate the effectiveness of this invention, it is compared with four previous fall detection works (FallDeFi, RT-Fall, and WiFall). FallDeFi is most similar to the solution presented in this invention, both using DFS spectral images as feature images, while RT-Fall and WiFall use CSI phase difference and amplitude signals, respectively. The above three works all identify the segmented whole motion signal, which can be affected by accompanying actions. Therefore, the Walk-Fall samples were removed, resulting in two datasets: one containing accompanying actions and one without. The performance of the four solutions was then tested separately. Figure 10 As shown in (b), after removing the Walk-Fall samples, all four schemes performed reasonably well. The scheme provided by this invention achieves 97% accuracy when using GRU as the encoder, while WiFall and FallDeFi achieve 96% and 93% accuracy respectively, and RT-Fall performs the least well, with only around 90% accuracy. Figure 10 As shown in (a), when the Walk-Fall sample is included, the performance of all schemes declines. The scheme of this invention is least affected, still maintaining 93%, while the performance of the other schemes declines significantly. Among them, FallDeFi drops to 86%, while WiFall and FallDeFi are only around 75%.

[0131] To understand the impact of accompanying actions on traditional recognition models, FallDeFi was used as the research object, and the recognition accuracy of four types of actions was examined on two datasets. Figure 11As shown, when Walk-Fall samples are not included in training, the model performs better than 90% on Stand-Fall, In-situ Act, and Walk-Stop, with Walk-Stop even reaching 100%. This is because Walk-Stop lasts significantly longer than falls and everyday actions that occur in place, making it easy to distinguish from the other two. In this case, the model's accuracy for Walk-Fall is only 30%. When Walk-Fall samples are included in training, while the recognition rate for Walk-Fall improves to 83%, the recognition rate for Walk-Stop drops to 75%. Meanwhile, Stand-Fall and In-situ Act are not significantly affected. This is because the durations of Walk-Stop and Walk-Fall are similar, with the difference being only a short fall at the end. The model can only distinguish based on the internal features of the signal, but the global feature extraction approach cannot accurately identify key features under prolonged walking interference, thus leading to recognition difficulties.

[0132] This invention uses two task classifiers to calculate the loss function, with their ratio r being a hyperparameter. To investigate its impact on model performance, GRU and Transformer encoders were tested separately. Figure 12 As shown, adjusting the loss ratio of the two task classifiers has no significant impact on model performance. With increasing r, the performance of the model using GRU remains relatively unchanged, but the performance of the model using Transformer as the encoder gradually decreases. Therefore, this embodiment selects 0.7 as the default value, making the loss of the fall classifier the dominant component.

[0133] Example 2

[0134] This invention provides a fall detection system suitable for accompanying movements, comprising:

[0135] Data acquisition and preprocessing module: Acquires channel state information sample signal data of user motion state through measurement equipment, and performs data preprocessing on the signal data to obtain a perception signal that can reflect user actions and remain smooth and calm;

[0136] Time-frequency analysis module: used to perform time-frequency analysis on the sensing signals obtained after data preprocessing, and use segmentation algorithms to find the start and end points of the fall action, and generate the corresponding Doppler frequency shift spectrum;

[0137] Model Recognition Status Module: This module is used to input the Doppler frequency shift spectrum into the constructed fall recognition model, and to determine the user's motion status after eliminating accompanying actions.

[0138] Example 3

[0139] The present invention provides an electronic terminal comprising a processor and a memory, wherein the memory stores a computer program, and the processor invokes the computer program to perform the steps of the method described above.

[0140] Example 4

[0141] The present invention provides a readable storage medium storing a computer program that, when invoked by a processor, performs the steps of the method described above.

[0142] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0143] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the controller described in any of the foregoing embodiments, such as the controller's hard drive or memory. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both the controller's internal storage unit and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0144] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0146] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A fall detection method suitable for accompanying motion, characterized by, include: S1: Obtain channel state information containing user action characteristics through measurement equipment, and obtain a smooth and stable sensing signal through data preprocessing; S2: Perform time-frequency analysis on the sensing signal obtained after data preprocessing, use segmentation algorithm to find the starting and ending points of the fall action, and generate a Doppler frequency shift spectrum that reflects the speed of human movement; S21: Perform a short-time Fourier transform on the sensing signal obtained after data preprocessing. The short-time Fourier transform is performed by truncating the signal through a sliding window, performing a fast Fourier transform on the phase difference signal within the window, and combining the results of the fast Fourier transform to obtain the Doppler frequency shift spectrum. S22: Calculate the full-band energy burst curve and the high-band energy burst curve in [5~90Hz] and [25~90Hz] respectively and high-band energy burst curve If is not zero, it means that there is a high-speed action in the sample, otherwise it is discarded; and record and The left and right endpoints of the energy in and are respectively represented as The left slope of the energy is determined The right slope of the energy is determined S23: Calculate the energy steps and corresponding energy noise in the intervals corresponding to the left and right slopes respectively, select the step closest to the noise level for segmentation, and finally determine the left and right endpoints of the action start and end; S24: Extract the Doppler frequency shift spectrum between [5~90Hz] and smooth it; The smoothing process in S24 is calculated using the following formula: ； in, The smoothed spectrum; For time tags; and All are frequency tags; The maximum frequency observed; The lowest frequency of observation; This is the original spectrum diagram; S3: Input the Doppler frequency shift spectrum into the completed fall recognition model to determine the user's motion state after eliminating the interference of accompanying actions.

2. The fall detection method applicable to accompanying movements according to claim 1, characterized in that, The specific process of data preprocessing is as follows: S11: Calculate the conjugate multiplication of the channel state information of two adjacent antennas and extract the phase difference signal of the two adjacent antennas; S12: Perform a first-order difference on the phase difference signal to obtain the transition regions of π-transition and 2π-transition; S13: Mark the transition type of each subcarrier to obtain the matrix. and the matrix Summing each row yields the vote count for each subcarrier pair's transition, and the π transition point is determined by the majority vote. S14: The transition point divides the entire action sample into several segments, denoted as: , represent The total number of transition points is calculated, and segments with odd or even indices are selected. The points obtained by multiplying the conjugate of their corresponding channel state information are symmetric about the origin. This process is then corrected. Jump; S15: Calculate the centroid of the complex points of the conjugate multiplication of all channel state information, rotate all data points of all subcarriers around the origin until their centroids are at the same angle to eliminate frequency domain jumps; continue to translate all complex points outward along the ray direction until all data points are in the first quadrant. S16: Take the phase information obtained by conjugate multiplication of the adjusted channel state information, and eliminate the remaining phase information by unwinding. The phase difference signal is obtained after the phase difference signal on each subcarrier is restored to normal. The phase difference signal on all subcarriers is averaged to obtain a smooth sensing signal after noise removal.

3. The fall detection method applicable to accompanying movements according to claim 1, characterized in that, The fall recognition model applicable to accompanying actions is a deep model based on RNN and attention mechanisms.

4. The fall detection method applicable to accompanying movements according to claim 3, characterized in that, The specific recognition steps of the fall recognition model applicable to accompanying actions are as follows: S31: Extract the temporal features contained in the Doppler frequency shift spectrum through an RNN model and output the encoded feature image; S32: Calculate the attention weights of the feature images through a weighted learning network to obtain the attention matrix for the fall recognition task and the walking recognition task; S33: Multiply the attention matrix with the encoded feature image and obtain the sentence vector representing the whole sample by weighted averaging; S34: Feed the obtained sentence vectors to the task classifier to perform binary classification of falling and walking respectively; where the weight learning networks corresponding to falling and walking are independent of each other, and the generated sentence vectors are different; S35: Backpropagation is performed during network model training using a weighted average of the losses from multiple task classifiers. The overall loss is denoted as... ;in, For the loss of the fall classifier, For the loss of the walking classifier, To control the proportion of loss between the fall classifier and the walking classifier.

5. The fall detection method applicable to accompanying movements according to claim 4, characterized in that, The weight learning network in S32 consists of two additional linear layers without bias, and the additional layers are located at the output of the RNN encoder.

6. A fall detection system suitable for accompanying actions, said system being used to perform the steps of the method according to any one of claims 1-5, characterized in that, include: Data acquisition and preprocessing module: used to acquire channel state information sample signal data of user motion state through measurement equipment, and to preprocess the signal data to obtain a perception signal that can reflect user action and remain smooth and calm; Time-frequency analysis module: used to perform time-frequency analysis on the sensing signals obtained after data preprocessing, and use segmentation algorithms to find the start and end points of the fall action, and generate the corresponding Doppler frequency shift spectrum; Model Recognition Status Module: This module is used to input the Doppler frequency shift spectrum into the constructed fall recognition model, and to determine the user's motion status after eliminating the interference of accompanying actions.

7. An electronic terminal, characterized in that: It includes a processor and a memory, the memory storing a computer program, the processor calling the computer program to perform the steps of the method according to any one of claims 1-5.

8. A readable storage medium, characterized in that: A computer program is stored, which, when invoked by a processor, performs the steps of the method according to any one of claims 1-5.

Images