Method for recognizing motion based on wireless signals, electronic device and computer program product

Abstract

The present disclosure provides a wireless signal-based action recognition method, an electronic device and a computer program product. The wireless signal-based action recognition method comprises: receiving wireless signals emitted by a signal emitting device, obtaining channel state information corresponding to a plurality of subcarriers through the wireless signals, the channel state information comprising channel gain amplitudes of the plurality of subcarriers; generating energy features based on the channel state information, the energy features being used to represent energy intensities of the plurality of subcarriers in a plurality of time frames, the values of the energy intensities being positively correlated with the values of the channel gain amplitudes; extracting a feature vector from the energy features through a feature extraction model, the feature vector being used to represent a spatiotemporal disturbance condition in a space; and if the spatiotemporal disturbance condition represented by the feature vector matches a target spatiotemporal disturbance condition when an object makes a specified action, it is determined that there is an object that is making the specified action.

Description

Technical Field

[0001] This disclosure specifically relates to a motion recognition method based on wireless signals, an electronic device, a readable storage medium, and a computer program product. Background Technology

[0002] When users are active in non-open spaces, it is sometimes necessary to recognize their behavior for safety or other reasons, so that accidents or dangerous actions can be detected in a timely manner. For example, when a user is walking indoors, they may fall due to their own health or the ground conditions; recognizing user movements can quickly detect this situation.

[0003] To detect falls promptly, common practices currently involve using contact-based devices to recognize user movements. These include wearable cameras worn on the neck, wrist, or other body parts to capture images and identify falls, or using accelerometers. However, contact-based devices must be worn constantly and require charging, leading to poor user compliance, inconvenience, and difficulty in integrating with existing residential infrastructure. Deploying them in age-friendly renovations is also challenging. Summary of the Invention

[0004] This disclosure provides a motion recognition method based on wireless signals, an electronic device, a readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a method for action recognition based on wireless signals is provided, comprising: The receiver receives wireless signals emitted by a signal transmitting device and obtains channel state information corresponding to multiple subcarriers through the wireless signals. The channel state information includes the channel gain amplitude of the multiple subcarriers. Energy features are generated based on the channel state information. These energy features represent the energy intensity of the multiple subcarriers in multiple time frames. The value of the energy intensity is positively correlated with the value of the channel gain amplitude. Feature vectors are extracted from the energy features using a feature extraction model. These feature vectors are used to represent the spatiotemporal perturbations in space. If the spatiotemporal perturbation represented by the feature vector matches the target spatiotemporal perturbation when the object performs the specified action, then it is determined that there exists an object performing the specified action.

[0006] According to some embodiments of this disclosure, before generating the energy characteristics, the method further includes: standardizing the channel state information by treating the plurality of subcarriers as a whole, specifically including: determining the mean and standard deviation of the channel gain amplitude of the plurality of subcarriers; and obtaining the standardized channel state information by subtracting the mean and dividing by the standard deviation.

[0007] According to some embodiments of this disclosure, before generating energy features based on the channel state information, one or more of a first denoising step and a second denoising step are performed; the first denoising step includes: obtaining the channel state information with background noise removed by determining the difference between the channel gain amplitude and an established static baseline, wherein the static baseline is used to represent the channel state of a static background environment in space; the second denoising step includes: obtaining the channel state information with background noise removed by determining the difference between the channel state information in a time window and existing sample channel state information by using a sliding window differential method.

[0008] According to some embodiments of this disclosure, the method further includes: updating the static baseline through the channel state information when an update condition is met, wherein the update condition includes: the maximum change between a specified number of consecutively extracted feature vectors is less than a change threshold.

[0009] According to some embodiments of this disclosure, the signal transmitting device has multiple antennas, and the multiple antennas correspond to multiple channel state information. The energy feature is a multi-dimensional feature formed by reconstructing the original one-dimensional channel state information sequence collected over time into a two-dimensional matrix by taking the time dimension as one axis and the subcarrier index as another axis, and further introducing the spatial channel corresponding to the multiple antennas as the third dimension.

[0010] According to some embodiments of this disclosure, the energy features are input into the feature extraction model in the form of an image, and the energy intensity is represented by a color value.

[0011] According to some embodiments of this disclosure, the feature extraction model includes a convolutional neural network and a recurrent neural network. Spatial features in the feature vector are extracted by the convolutional neural network, and temporal features in the feature vector are extracted by the recurrent neural network.

[0012] According to some embodiments of this disclosure, the designated action includes the action of a human falling down.

[0013] According to some embodiments of this disclosure, after determining that there is an object performing the specified action, the method further includes: comparing a prior feature vector that does not match the target spatiotemporal disturbance with an existing sample feature vector to identify the object's identity information, wherein the prior feature vector was generated earlier than the feature vector was generated, and the sample feature vector includes at least one object's feature vector that does not match the target spatiotemporal disturbance.

[0014] A second aspect of this disclosure provides an electronic device, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform the action recognition method described in any of the above embodiments.

[0015] A third aspect of this disclosure provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the action recognition method described in any of the above embodiments.

[0016] The fourth aspect of this disclosure provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the action recognition method described in any of the above embodiments. Attached Figure Description

[0017] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0018] Figure 1 The diagram illustrates an application scenario of the wireless signal-based motion recognition method according to some embodiments of the present disclosure.

[0019] Figures 2-5 A schematic diagram of the overall process of a wireless signal-based motion recognition method M100 according to some embodiments of the present disclosure is shown.

[0020] Figure 6 A schematic diagram of the overall process for updating a static baseline in a wireless signal-based motion recognition method M100 according to some embodiments of the present disclosure is shown.

[0021] Figure 7 A schematic diagram of the overall flow of a wireless signal-based motion recognition method M100 according to other embodiments of the present disclosure is shown.

[0022] Figure 8 This is a schematic block diagram of a wireless signal-based motion recognition device according to one embodiment of the present disclosure.

[0023] Figure 9 This is a flowchart of a wireless signal-based motion recognition device according to one embodiment of the present disclosure.

[0024] Figure 10 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure. Detailed Implementation

[0025] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0026] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] Unless otherwise stated, the exemplary implementations / embodiments shown are to be understood as providing exemplary features of various details that provide ways in which the technical concepts of this disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of various implementations / embodiments may be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of this disclosure.

[0028] The terminology used herein is for the purpose of describing particular embodiments and is not restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0029] In recent years, my country's aging population has accelerated, with many elderly people living in older housing. A large number of these older homes lack accessibility features, hindering home-based aging, making age-friendly renovations an urgent need. Current home renovations for age-friendliness primarily focus on aesthetics and functional upgrades, such as environmentally friendly materials and smart home technology, but insufficient consideration is given to fall prevention and real-time monitoring for the elderly.

[0030] For example, elderly users may slip and fall in enclosed or semi-enclosed environments such as indoors due to slippery floors or pre-existing medical conditions. To address this issue, wearable contact devices can be used for body monitoring. For instance, a camera can be worn on the body, and the collected images can be analyzed to determine if a fall has occurred. Another example is wearing an accelerometer, which calculates the detected acceleration values ​​to determine if a fall has taken place.

[0031] However, these contact-based devices all require batteries to operate, necessitating frequent charging. Furthermore, the wearing experience can be inconvenient, especially for the elderly and patients. Some contact-based devices also require active user intervention to trigger alarms; for example, pressing a button after a fall can notify caregivers, but this can be difficult, delaying rescue efforts and leading to poor user compliance. Monitoring via wearable cameras relies on good lighting, and there are potential risks of privacy data leaks. Therefore, this disclosure proposes a motion recognition method based on wireless signals.

[0032] The method disclosed herein can utilize existing residential infrastructure such as widely deployed Wi-Fi routers without additional wiring or damage to renovations. It can be deployed in conjunction with age-friendly physical modifications such as non-slip floors and handrails to provide continuous and accurate fall monitoring.

[0033] Figure 1 The diagram illustrates an application scenario of a wireless signal-based motion recognition method according to some embodiments of the present disclosure. In this application scenario, a signal transmitting device and a motion recognition device may be installed in space H. In this disclosure, both the signal transmitting device T and the motion recognition device R include at least one processor and at least one memory.

[0034] For example, the signal transmitting device T can be a wireless access point, a router, or other electronic device capable of transmitting wireless signals, and the motion recognition device R can be a home terminal device equipped with a wireless network card, or it can be installed in an electronic device with wireless signal receiving capabilities, such as a router. The motion recognition device R receives the signal emitted by the signal transmitting device T through the wireless network card. By analyzing the received signal, it determines whether the user's action is a fall action based on the changes in the user's behavior within the space represented by the waveform changes, thereby determining whether a user has fallen. Figure 1 The dashed line in the diagram represents the signal wave emitted by signal transmitting device T, which passes through the location of user U. Figure 1 The shape and structure of the signal transmitting device T and the motion recognition device R shown should not be construed as a limitation on the scope of protection of this disclosure.

[0035] In this disclosure, "terminal device" can be different types of electronic devices, such as mobile phones, tablets, or laptops.

[0036] Figure 2 A schematic diagram illustrating the overall flow of a wireless signal-based motion recognition method M100 according to some embodiments of this disclosure is shown. Figure 2 The method shown includes steps S110, S130, S150, and S170. This method can be executed by a motion recognition device, which can be installed independently in a non-open space or integrated into electronic devices such as routers.

[0037] S110: Receive the wireless signal emitted by the signal transmitting device and obtain channel state information corresponding to multiple subcarriers through the wireless signal. The channel state information describes the complex channel response on the multiple subcarriers, including the channel gain amplitude of each subcarrier; in this embodiment, the channel gain amplitude is used to construct energy characteristics.

[0038] In an indoor environment, the signal transmitting device can be an existing electronic device, such as a router or a wireless access point (AP) that can transmit wireless signals. Accordingly, the wireless signal emitted by the signal transmitting device can be a Wi-Fi network signal.

[0039] For example Figure 1 Within space H, a signal transmitting device T for emitting Wi-Fi signals has been installed, along with a home computer capable of wireless internet access. The computer's wireless network card can receive wireless signals, thus allowing the computer to function as a motion recognition device. By running a specific driver on the computer, it can collect Wi-Fi signals in real time. Alternatively, the motion recognition device can be integrated into electronic devices such as wireless routers. Routers themselves have wireless signal transmission and reception capabilities, allowing the motion recognition device to acquire the signals received by the router and perform motion recognition using its own hardware resources or by leveraging the router's hardware resources.

[0040] It is understandable that receiving wireless signals is a passive sensing action, so the action recognition device does not need to establish a wireless communication connection with the signal transmitting device to perform the action recognition method.

[0041] By performing channel estimation on the acquired wireless signals, channel state information (CSI) for multiple subcarriers can be obtained. CSI describes the wireless propagation environment between the transmitter and receiver. Different scenarios—such as an empty room, a stationary person, or someone moving around—indicate different environmental conditions during signal transmission, causing fluctuations or disturbances and thus different CSI values. CSI can be estimated from the actual received wireless signals, known transmitted signals from devices, and noise. Estimation methods can include least squares estimation and minimum mean square error estimation (MMSE). Configuring multi-antenna synchronous acquisition and clock calibration ensures the accuracy of data timing.

[0042] The channel gain amplitude in the channel state information is used to indicate whether the amplitude of the wireless signal is attenuated or enhanced after passing through the channel.

[0043] In some embodiments of this disclosure, the channel state information may further include phase information to describe the phase shift that occurs during signal propagation. The channel state information obtained in this step may be referred to as the original CSI sequence.

[0044] The overall frequency range in which signal transmitting equipment operates is a frequency band, such as 2.4 GHz or 5 GHz. A channel is a specific frequency range within a frequency band used for actual data transmission; for example, 2.4 GHz may be divided into multiple channels. A channel can contain multiple subcarriers, all of which transmit data simultaneously. A subcarrier is a narrow-band, mutually orthogonal frequency component that divides a wideband channel. Each component independently carries a portion of data. The channel state information obtained in this step corresponds to multiple subcarriers, thus yielding the CSI information for multiple subcarriers.

[0045] S130, generate energy features based on channel state information. These energy features represent the energy intensity of multiple subcarriers across multiple time frames, and the energy intensity value is positively correlated with the channel gain amplitude.

[0046] Energy features can be multi-dimensional, encompassing time, subcarrier, and energy intensity dimensions. The time dimension corresponds to the specific acquisition time of the wireless signal, and the energy feature contains a specified number of time frames. For example, if the specified number is 100, the energy feature contains signal data from 100 time frames. The subcarrier dimension represents the subcarrier's number or frequency, uniquely identifying each subcarrier. The energy intensity dimension corresponds to the energy intensity value of each subcarrier. The energy intensity value can be the square of the channel gain amplitude, which is the square of the magnitude of the channel gain signal. Therefore, the two are positively correlated; the larger the channel gain amplitude, the higher the energy intensity value. Generating energy features is equivalent to reconstructing CSI information into energy features.

[0047] Assuming channel state information is collected at a frequency of F Hz, and there are N subcarriers, then data at tF time points can be obtained by continuously collecting data for t seconds. Each time point corresponds to N amplitude values. At this time, the energy feature can be a tFN matrix, where the horizontal axis corresponds to the time dimension and the vertical axis corresponds to the subcarrier dimension. Each element in the matrix is ​​the energy intensity value of the subcarrier at the corresponding time.

[0048] When generating energy features, the generation can be performed at fixed intervals. Assuming the wireless signal acquisition frequency is 1000Hz (channel state information is acquired every 0.01 seconds), and the energy feature generation frequency is 100Hz, generating energy features based on the CSI information of the most recent 100 time frames, this is equivalent to generating energy features every 0.1 seconds using the CSI information acquired in the most recent second. In adjacent energy feature generation, most of the CSI information used is repeated, with only the data acquired in the most recent 0.1 seconds being new data.

[0049] S150 extracts feature vectors from energy features using a feature extraction model. These feature vectors represent spatiotemporal perturbations within the space.

[0050] The feature extraction model can be a pre-trained neural network model, and the feature vectors are high-level feature vectors. When someone moves in space, the spatiotemporal disturbances represented in the feature vectors formed by the collected wireless signals will also be different. When someone is dancing or making some fast-moving movements, the energy intensity of some subcarriers will be higher at the corresponding moment, and the signal disturbance reflected in the spatiotemporal disturbances will be stronger. By using energy features, the signal disturbances under temporal and spatial characteristics can be extracted, which is to obtain the feature vectors.

[0051] Therefore, by using the spatiotemporal perturbation represented by the feature vector, it is possible to indicate whether the user is performing certain specific actions at the current moment. The current moment can refer to the moment corresponding to the latest time frame contained in the energy feature. For example, if the energy feature contains signal data under 100 time frames, and the 100 time frames are t1 to t100 respectively, then the current moment can be the moment corresponding to t100.

[0052] S170, if the spatiotemporal perturbation represented by the feature vector matches the target spatiotemporal perturbation when the object performs the specified action, then it is determined that there is an object performing the specified action.

[0053] This step is used to detect a specified action. Feature vectors can be input into the action recognition model, which then identifies whether spatiotemporal disturbances indicate that someone is performing the specified action. The specified action can include the action of a person falling.

[0054] The target spatiotemporal perturbation is used to represent the dynamic perturbation features generated when someone is performing a specified action. If the spatiotemporal perturbation represented by the current feature vector is the same as or similar to the target spatiotemporal perturbation, it indicates that someone has performed the specified action. The action recognition model can specifically be a binary classification model, which derives one of two mutually exclusive categories: the category belonging to the specified action and the category not belonging to the specified action.

[0055] Assuming the designated action is falling, if someone falls, the wireless signal received during the fall will be disturbed by the falling action, making the spatiotemporal disturbance in the formed feature vector have some identifiable features. Thus, it is possible to determine whether someone is falling or has already fallen through the feature vector.

[0056] Understandably, after confirming the existence of an object performing a specified action, an alarm message can be sent to a designated device. This alarm function can be implemented by triggering a local service or a cloud service. The designated device can include one or more of the following: a family member's terminal device, a caregiver's terminal device, or a service device from a medical institution. The alarm message can include the time and location of the fall. The fall time can be the "current moment" mentioned above, and since the installation location of the motion recognition device is known, the fall location can be the installation location. Additionally, a voice announcement can be triggered, such as by activating a buzzer or a voice device.

[0057] The motion recognition method based on wireless signals proposed according to the embodiments of this disclosure uses non-contact wireless signal detection to identify user actions, and can continuously provide motion recognition function without charging, thus solving the problem of frequent charging faced by users wearing contact devices.

[0058] Moreover, there are no security risks such as data leakage, and it is not affected by ambient light. It solves the privacy leakage problem faced by users wearing cameras and the problem of not being able to recognize actions due to lighting conditions, resulting in better user compliance and experience.

[0059] At the same time, the wireless signal can provide full coverage within the space, with no blind spots in the detection area. It can identify specified actions such as falling in a timely and accurate manner, and will not fail to capture the action process accurately due to obstruction by furniture, appliances or other equipment in the space.

[0060] In addition, if there are already electronic devices with wireless communication capabilities installed in the space, these devices can be used to perform motion recognition, providing continuous and accurate motion recognition services without damaging the decoration effect. This method is low-cost and quick and easy.

[0061] Furthermore, when identifying actions by extracting features, this method introduces a multi-dimensional structure and integrates frequency and time domain information. This allows for the consideration of correlations between subcarriers, treating all subcarriers as a whole for data processing. It utilizes the time and amplitude variations of all subcarriers, employing multi-dimensional vectors instead of one-dimensional arrays for action recognition, thus avoiding the loss of important information due to insufficient features. Existing methods often treat CSI amplitude sequences as one-dimensional or two-dimensional time series processed independently by subcarrier, lacking unified modeling of cross-subcarrier correlations and multi-antenna spatial dimensions, resulting in limited feature representation capabilities. This solution overcomes these shortcomings. When the specified action is a fall, it can promptly and accurately detect user falls, enabling immediate alarm activation or notification of relevant personnel for rescue. It is expected that in practical applications, this will significantly reduce injuries and accidents caused by undetected falls, improve the safety of elderly residents, and reduce the burden on family members and caregivers.

[0062] Figure 3 A schematic flowchart of the action recognition method M100 based on wireless signals, according to some other embodiments of this disclosure, is shown. (See also...) Figure 3 Before generating energy characteristics (step S130), step S120 can be performed first.

[0063] S120 standardizes the channel state information by treating multiple subcarriers as a whole.

[0064] Since the magnitude of CSI amplitude varies depending on the device and distance, standardization can suppress the scale difference in channel gain amplitude, compress the dynamic range, and make subsequent processing more stable. Standardizing the entire subcarrier as a whole preserves the complete information across subcarriers, thus retaining the energy characteristics of disturbances, such as gait disturbances, to facilitate the identification of falls. Compared to processing each subcarrier individually, this implementation avoids disrupting the relative energy relationship between subcarriers (e.g., high-frequency subcarriers are more sensitive to rapid movements), thereby preventing wideband and weakly disturbed signals such as gait from being misjudged as noise and filtered out. Existing methods often treat the CSI amplitude sequence as a one-dimensional or two-dimensional time series processed independently by subcarrier, lacking unified modeling of cross-subcarrier correlation and multi-antenna spatial dimensions, resulting in limited feature representation capabilities. This scheme, by standardizing multiple subcarriers as a whole, preserves and utilizes this cross-subcarrier correlation.

[0065] Figure 4 A schematic flowchart of the action recognition method M100 based on wireless signals, according to some other embodiments of this disclosure, is shown. (See also...) Figure 4 In step S120, the method for standardizing the channel state information may include the following steps S121 and S122.

[0066] S121, treating multiple subcarriers as a whole, determines the mean and standard deviation of the channel gain amplitude of the multiple subcarriers.

[0067] S122, by subtracting the mean and dividing by the standard deviation, we obtain the standardized channel state information.

[0068] Before standardization, random noise can be removed using methods such as low-pass filtering. Random noise can include thermal noise and environmental electromagnetic interference noise, mainly including transient noise caused by sudden interference. The source of random noise can include electronic devices, and its characteristics can be high frequency, short duration, and irregular. By standardizing the channel gain amplitude of each subcarrier, random noise can be removed, filtering can be achieved, and random noise can be avoided from being identified as human motion information.

[0069] During global standardization, multiple subcarriers at multiple time points are treated as a whole. The mean value of the subcarriers within the window is uniformly subtracted from the CSI amplitude data of all subcarriers and all time points to remove the DC component and eliminate hardware drift. Then, the standardization is completed by dividing by the global standard deviation, which is calculated by dividing by the standard deviation of data from all subcarriers and all time points.

[0070] Understandably, since the action recognition process is continuous, constantly receiving new wireless signals and generating new energy features, it can be performed using a time window approach. Standardization and energy feature generation are both based on data within the window. By sliding the time window at fixed intervals, some data within the window is removed and newly acquired data is added, followed by further standardization and subsequent energy feature generation.

[0071] Standardization allows for scaling and noise suppression of the original amplitude while preserving the energy characteristics of gait disturbances. High-frequency subcarriers, being sensitive to rapid movements, naturally exhibit larger disturbance amplitudes, making their characteristics more pronounced after standardization. Meanwhile, the weak but stable gait characteristics of low-frequency subcarriers are not obscured.

[0072] Figure 5 A schematic flowchart of the action recognition method M100 based on wireless signals, according to some other embodiments of this disclosure, is shown. (See also...) Figure 5 Before generating energy features based on channel state information (step S130), one or more of the first denoising step S123 and the second denoising step S124 can be executed first.

[0073] The first denoising step S123 and the second denoising step S124 calculate and remove the static background trend generated by the spatial environment layout using different methods. The static background trend can include extremely slow changes caused by factors such as temperature drift, furniture movement, wall dampness, and equipment aging. These changes are characterized by low frequency, long duration, non-stationarity, and overlap with user activities. By removing background trend noise, the energy feature generated in step S130 only reflects CSI changes caused by user actions. These two steps can be performed individually or sequentially according to their step numbers.

[0074] The first denoising step S123 may include: obtaining channel state information with background noise removed by determining the difference between the channel gain amplitude and the established static baseline. The static baseline represents the channel state of the static background environment in space.

[0075] A static background environment can represent a space where the layout has not changed. For example, the tables, chairs, and appliances in the room have not been moved. This makes the reflections caused by fixed objects in the space (such as walls and furniture) relatively stable, so that it can be represented by establishing a baseline.

[0076] A baseline is a reference model established to represent the static channel characteristics of an environment in the absence of targeted activity. Subsequent observations are compared with this baseline to detect anomalies or activity. Baseline modeling separates static background from dynamic activity, isolating CSI changes caused by human actions from the static environment and avoiding misinterpretation of slow environmental changes (such as temperature and humidity) as activity. The baseline can be updated periodically to adapt to slow environmental drift.

[0077] The baseline is pre-established, for example, by collecting CSI information multiple times and taking the average value when the space only contains the background environment (i.e., no one in the environment) as the current static baseline. After starting to execute the action recognition method of this disclosure, the baseline data is subtracted from the collected CSI information to obtain CSI information without background noise.

[0078] The second denoising step S124 may include: determining the difference between the channel state information in the time window and the existing sample channel state information by means of sliding window differential, so as to obtain the channel state information with background noise removed.

[0079] Sliding window differencing calculates the difference between the current CSI information and the sample CSI information within a time window, thereby highlighting dynamic changes and suppressing static components. The sample CSI information can be the CSI from the previous time window, meaning the current CSI information is subtracted from the previous CSI information; in this case, sliding window differencing is a first-order differencing. Alternatively, the sample CSI information can be the mean of the CSI information from one or more previous time windows, meaning the current CSI information is subtracted from the mean of the CSI from multiple previous windows; in this case, sliding window differencing is a forward window mean differencing. Time-varying channel disturbances caused by human movement and gestures are amplified after differencing, thus highlighting dynamic events related to human activity. It can also filter out static CSI offsets caused by stable reflections from fixed objects in the environment (such as walls and furniture), reducing the sensitivity of the action recognition device to the deployment environment.

[0080] It is understandable that steps S123 and S124 can be used in combination. First, static multipath in the environment is removed through step S123, and then fine dynamic disturbance features are extracted through sliding window differential.

[0081] Furthermore, since noise and obstruction may cause some time window data to be missing, after completing the first denoising step S123 or the second denoising step S124, the channel state information can be interpolated first. Matrix completion is performed using the correlation between adjacent subcarriers and time windows to reconstruct missing values ​​and ensure the continuity of energy characteristics. Combined with energy normalization, robustness under different environments is enhanced.

[0082] Interpolation methods can be neighborhood-based, such as using the average of the values ​​in the preceding and following time frames for a time frame with a missing energy intensity value, or using the average of the values ​​in the preceding and following subcarriers for that time frame, as the missing value for that subcarrier. Alternatively, interpolation can be low-rank matrix completion; if amplitude values ​​are missing at multiple consecutive locations, matrix completion can be triggered to restore the complete matrix.

[0083] Figure 6 A schematic diagram illustrating the overall process of updating a static baseline in the wireless signal-based action recognition method M100 according to some embodiments of this disclosure is shown. (See also...) Figure 6 The action recognition method M100 may also include step S180. Figure 6 Step S170 is not shown in the diagram.

[0084] S180, under the condition that the update is met, the static baseline is updated using channel state information. The update condition includes: the maximum change between a specified number of consecutively extracted feature vectors is less than a change threshold.

[0085] Because the environment within a space can undergo slow or permanent changes—for example, furniture may be moved or the motion recognition device itself may be displaced—updating the static baseline ensures that important information related to human activity is not lost when removing background noise.

[0086] Updating the static baseline can be automatic. Assuming the spatial environment layout remains unchanged, in an unmanned state, the feature vector at the current time step will not show significant changes compared to the feature vector at the previous time step. The specified number can be a large value.

[0087] When 10,000 action recognitions are performed consecutively, it's equivalent to generating 10,000 energy features and extracting 10,000 feature vectors. These feature vectors may contain minor fluctuations and changes. If the difference between two feature vectors with the largest difference does not reach a change threshold, it indicates that the largest change during this period is still very small. This suggests that no one was present in the environment during this time, and the current environment is considered normal. Therefore, a new static baseline can be generated using the latest channel state information to replace the original baseline, or the average of these 10,000 channel state information points can be used to generate a new static baseline to replace the original baseline.

[0088] If someone enters the room, the resulting signal disturbance causes the feature vector to change beyond a threshold. Therefore, the baseline is not updated when someone is inside. If no one is inside and an object falls from a height, the feature vector change exceeds the threshold. From this point onward, if no one enters the room after 10,000 action recognitions, the room layout after the object falls will be used as the basis for a new baseline. Then, if someone enters the room, puts the object back in its place, and leaves, from this point onward, if no one enters the room after 10,000 action recognitions, the room layout after the object is reset will again be used as the basis for a new baseline, at which point the baseline will revert to the baseline before the object fell.

[0089] Signal transmitting equipment can have multiple antennas. Multiple antennas correspond to multiple channel state information.

[0090] Using time as one axis and subcarrier index as another, the original time-acquired one-dimensional CSI sequence is reconstructed into a two-dimensional matrix. Furthermore, spatial channels corresponding to multiple antennas are introduced as a third dimension, forming a multi-dimensional energy feature map. In this scenario, the energy feature incorporates the spatial channels corresponding to the multiple antennas as antenna dimensions to represent the energy intensity in the antenna, subcarrier, and time dimensions.

[0091] When a wireless access point acting as a signal transmitting device has multiple antennas, and the wireless network card in the motion recognition device is also equipped with multiple antennas, the antenna dimension can be considered a new dimension in the multidimensional structure introduced in this disclosure. Each pair of transceiver antennas generates a set of independent CSI data (containing all subcarriers). The signal propagation path is different, and the angle, distance, and obstruction encountered by the human body also vary. Human activity will also cause different disturbances to the wireless signal along different spatial paths. Therefore, multiple antennas provide spatial diversity, enabling a more comprehensive capture of the dynamic characteristics of human movement in three-dimensional space.

[0092] In the case of multiple antennas, the generated energy feature adds an antenna dimension, with each pair of antennas corresponding to a tF. The matrix is ​​N, where F is the sampling frequency, N is the number of subcarriers, and t is the window duration. The elements in the matrix are the energy intensity values.

[0093] Energy features can be input into the feature extraction model in the form of images. Energy intensity can be represented by color values. Without introducing the antenna dimension, the X-axis of the energy feature map is the time axis, and the Y-axis is the subcarrier axis. Pixels in the image are determined by the energy intensity value at their corresponding locations; for example, the higher the color value of a pixel, the higher the energy intensity value. In the case of multiple antennas, the generated energy features can form P two-dimensional energy maps, where P is the number of antennas. In addition to the X and Y axes, there is also a Z-axis representing the antenna index (spatial channel).

[0094] In multi-antenna scenarios, in addition to combining time-domain and frequency-domain information, spatial diversity is also incorporated. Multipath can bring stronger robustness, richer spatial features of gait, which is conducive to distinguishing users and can even perceive the spatial orientation and position of actions.

[0095] Understandably, the channel state information standardized through step S120 can be stored in CSV (Comma-Separated Values) format. CSV format can be used to store structured numerical matrices, where each row corresponds to a time point and each column corresponds to the standardized amplitude value of a subcarrier (DC removed, globally normalized), in order to generate an energy feature map.

[0096] In step S150, the feature extraction model may include a convolutional neural network and a recurrent neural network. Specifically, spatial features in the feature vector are extracted using a convolutional neural network, while temporal features in the feature vector are extracted using a recurrent neural network.

[0097] Convolutional neural networks and recurrent neural networks can be part of a "hybrid temporal-spatial model," which is part of the CNN-LSTM architecture.

[0098] When extracting spatial features, each frame's energy feature map can be input into a convolutional neural network (CNN) (which may contain several convolutional and pooling layers), outputting a compressed spatial feature vector (e.g., 128-dimensional). Spatial features can reflect the distribution of wireless signal disturbances in the subcarrier and antenna dimensions. For example, when a person falls, high-frequency subcarriers may be disturbed first, and low-frequency subcarriers may respond later, forming a specific and identifiable "spectral fingerprint." For example, when a person approaches an antenna, the disturbance to that antenna's signal is stronger, and different falling directions will lead to differences in the disturbance intensity of different antennas, forming identifiable "spatial orientation features."

[0099] When extracting temporal features, spatial features from multiple consecutive frames can be sorted by time and input into a recurrent neural network LSTM. Temporal features can reflect the dynamic changes of human activity over time, such as the process of "standing → losing balance → falling".

[0100] Furthermore, the feature extraction model can be pruned using lightweight compression algorithms and the Balloon mechanism, allowing for a reduction in model size while maintaining high recognition accuracy. This lightweight design and feature enhancement reduce model size and training time while ensuring accuracy, improving the practical deployment capability of the solution. This enables the action recognition device to be integrated into home routers or embedded devices for long-term wireless sensing and monitoring.

[0101] Network pruning can be achieved by removing redundant convolutional channels in a convolutional neural network through channel pruning and resetting unimportant connection weights to zero through weight sparsification.

[0102] The Balloon mechanism refers to pre-training a large supernet, which is a large, redundant neural network structure containing multiple subnetworks. This supernet contains far more channels than actually needed, allowing for efficient sampling or pruning of lightweight quantum models of various sizes and structures. By evaluating the importance of each channel during training, unimportant channels and layers are progressively pruned, and the remaining parts are fine-tuned to obtain a lightweight model.

[0103] In step S170, a binary classification model can be used to determine whether the spatiotemporal disturbance represented by the feature vector corresponds to the target spatiotemporal disturbance. The binary classification model can be combined with indicators such as action duration and acceleration features to make the judgment.

[0104] Figure 7 A schematic flowchart of the action recognition method M100 based on wireless signals, according to some other embodiments of this disclosure, is shown. (See also...) Figure 7 After determining that there is an object performing the specified action (step S170), step S190 can be executed to automatically identify the identity of the object performing the specified action, such as identifying who fell down.

[0105] S190, compare the prior feature vectors that do not match the target spatiotemporal perturbation with the existing sample feature vectors to identify the object's identity information. The prior feature vectors are generated earlier than the existing feature vectors, and the sample feature vectors include at least one object's feature vector that does not match the target spatiotemporal perturbation.

[0106] Taking a fall as an example, if step S170 determines that someone fell at the first moment, the first moment is the moment corresponding to the latest time frame t100 among the 100 time frames included in the energy feature. Since action recognition is continuous, before the first moment, it is also determined whether someone has fallen by receiving wireless signals, generating energy features, and extracting feature vectors. In the few seconds before the fall begins, the extracted feature vectors do not match the target's spatiotemporal disturbances; for example, they may represent feature vectors indicating a normal walking state. Therefore, feature vectors representing normal walking before the fall can be used as prior feature vectors.

[0107] The sample feature vector can also represent a feature vector of normal walking, and can be generated by pre-collecting gait data from family members. For example, each family member walks normally individually, and a motion recognition method is performed while they are walking to record their feature vectors. Multiple feature vectors from the same person walking normally are clustered or averaged to obtain the sample feature vector (i.e., gait template). Since both the pre-collected feature vector and the sample feature vector are feature vectors formed during normal walking, gait identification can be performed by comparing the two.

[0108] Once a fall is detected, multiple feature vectors prior to the fall can be automatically acquired. These vectors are then pooled (e.g., by averaging) to obtain a gait vector representing the normal walking motion. Cosine similarity or Euclidean distance calculations are performed on the gait vector and the sample feature vectors, and the object with the highest similarity or closest Euclidean distance is identified as the person who fell. This allows for the identification of the person who fell, overcoming the limitation of not being able to identify individuals. Furthermore, it enables targeted alarms based on the faller's identity.

[0109] During the execution of the action recognition method, the collected data of the specified action and alarm feedback can be used to update the model parameters through transfer learning technology and feed them back to the feature extraction model. This allows the model to be fine-tuned so that it can adapt to different room layouts, furniture placements or router locations, ensuring the long-term stability of the feature extraction model in actual deployment.

[0110] This implementation method integrates contactless Wi-Fi fall detection and fall identification technology into age-friendly renovations. By installing necessary hardware during renovations or directly utilizing existing home routers, combined with modifications such as non-slip floors and handrails, continuous and accurate fall warnings can be provided without compromising the renovation effect.

[0111] Furthermore, as communication technology is developing towards "integrated communication and sensing," which means integrating communication, sensing, and computing functions within the same network, wireless devices can not only transmit data but also have the ability to sense the environment.

[0112] This disclosure utilizes Wi-Fi signals for fall detection and personnel identification, making full use of existing communication infrastructure to achieve home safety monitoring.

[0113] Based on any of the above embodiments, this disclosure also provides a motion recognition device based on wireless signals. Figure 8 This is a schematic block diagram of a wireless signal-based motion recognition device according to one embodiment of this disclosure. Figure 8 As shown, the motion recognition device includes: a signal acquisition module 110, an energy feature generation module 120, and a motion detection module 130.

[0114] The signal acquisition module 110 is used to receive wireless signals emitted by the signal transmitting device and obtain channel state information corresponding to multiple subcarriers through the wireless signals. The channel state information describes the complex channel response on multiple subcarriers, including the channel gain amplitude of each subcarrier; in this embodiment, the energy characteristics are mainly constructed using the channel gain amplitude.

[0115] The feature extraction module 120 is used to generate energy features based on channel state information, and extract feature vectors from the energy features through a feature extraction model. The energy features represent the energy intensity of multiple subcarriers over multiple time frames, and the energy intensity value is positively correlated with the channel gain amplitude. The feature vectors represent the spatiotemporal disturbances in space.

[0116] The action detection module 130 is used to determine that there is an object performing a specified action if the spatiotemporal perturbation represented by the feature vector matches the target spatiotemporal perturbation when the object performs the specified action.

[0117] The aforementioned motion recognition device can be in the form of computer software, and each module of the aforementioned motion recognition device can be implemented through computer software modules.

[0118] Figure 9 This is a flowchart of a wireless signal-based motion recognition device according to one embodiment of this disclosure. (See also...) Figure 9 In the data acquisition layer, the input of the WiFi data acquisition module is the wireless signal emitted by the wireless access point. By performing CSI signal acquisition and multi-antenna synchronization, the original CSI sequence is generated.

[0119] In the data processing layer, the energy feature map generation module takes the original CSI sequence as input and generates an energy feature map through filtering, denoising, and data reconstruction. The feature extraction module takes the energy feature map as input and generates feature vectors through a convolutional neural network (CNN) and a recurrent neural network (LSTM).

[0120] In the detection and recognition layer, the fall detection module takes a feature vector as input and uses a binary classification model to determine the fall state. If a fall is confirmed, the fall state is output. The identity recognition module takes the fall state as input and generates the fall victim's identity information through gait recognition.

[0121] In the response processing layer, the alarm module takes fall status and identity information as input, pushes alarm information to designated devices, and generates fall data. The update module takes fall data as input, generates model parameters through transfer learning, and outputs them to the feature extraction module for model fine-tuning.

[0122] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0123] The execution subject of the wireless signal-based action recognition method in the specific embodiments of this disclosure can be an electronic device such as a router or a computer.

[0124] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the wireless signal-based action recognition method of any of the embodiments described above.

[0125] Figure 10 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure.

[0126] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0127] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0128] The processor 1200 can be a central processing unit (CPU). The processor 1200 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0129] The memory 1300 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions of the computer program in the embodiments of this disclosure. The processor 1200 implements the action recognition method based on wireless signals by running the non-transitory software programs, instructions, and modules stored in the memory 1300.

[0130] The memory 1300 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created by the processor 1200, such as CSI information, energy characteristic maps, and eigenvectors. Furthermore, the memory 1300 may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1300 may optionally include memory remotely located relative to the processor 1200, and these remote memories may be connected to the processor 1200 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0131] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0132] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the processes or functions of this disclosure are performed wholly or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, network equipment, user equipment, core network equipment, OAM, or other programmable device.

[0133] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0134] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, apparatus, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0139] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0140] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A motion recognition method based on wireless signals, characterized in that, include: The receiver receives wireless signals emitted by a signal transmitting device and obtains channel state information corresponding to multiple subcarriers through the wireless signals. The channel state information includes the channel gain amplitude of the multiple subcarriers. Energy features are generated based on the channel state information. These energy features represent the energy intensity of the multiple subcarriers in multiple time frames. The value of the energy intensity is positively correlated with the value of the channel gain amplitude. Feature vectors are extracted from the energy features using a feature extraction model; these feature vectors represent spatiotemporal perturbations within the space. If the spatiotemporal perturbation represented by the feature vector matches the target spatiotemporal perturbation when the object performs the specified action, then it is determined that there exists an object performing the specified action.

2. The action recognition method based on wireless signals according to claim 1, characterized in that, Before generating the energy characteristics, the method further includes: standardizing the channel state information by treating the plurality of subcarriers as a whole, specifically including: determining the mean and standard deviation of the channel gain amplitude of the plurality of subcarriers; and obtaining the standardized channel state information by subtracting the mean and dividing by the standard deviation.

3. The action recognition method based on wireless signals according to claim 2, characterized in that, Before generating energy features based on the channel state information, one or more of the first denoising step and the second denoising step are executed first. The first denoising step includes: obtaining the channel state information after removing background noise by determining the difference between the channel gain amplitude and the established static baseline, wherein the static baseline is used to represent the channel state of the static background environment in space; The second denoising step includes: determining the difference between the channel state information in the time window and the existing sample channel state information by means of sliding window differential, so as to obtain the channel state information after removing background noise.

4. The action recognition method based on wireless signals according to claim 3, characterized in that, The method further includes: Under the condition that the update is met, the static baseline is updated using the channel state information. The update condition includes: the maximum change between a specified number of consecutively extracted feature vectors is less than a change threshold.

5. The action recognition method based on wireless signals according to claim 1, characterized in that, The signal transmitting device has multiple antennas, each corresponding to multiple channel state information. The energy feature is a multi-dimensional feature formed by reconstructing the original one-dimensional channel state information sequence collected over time into a two-dimensional matrix by using the time dimension as one axis and the subcarrier index as another axis, and further introducing the spatial channel corresponding to the multiple antennas as a third dimension.

6. The action recognition method based on wireless signals according to claim 1 or 5, characterized in that, The energy features are input into the feature extraction model in the form of an image, and the energy intensity is represented by color values.

7. The action recognition method based on wireless signals according to claim 1, characterized in that, The feature extraction model includes a convolutional neural network and a recurrent neural network. Spatial features in the feature vector are extracted by the convolutional neural network, and temporal features in the feature vector are extracted by the recurrent neural network.

8. The action recognition method based on wireless signals according to claim 7, characterized in that, The specified action includes the action of a human falling. Optionally, after determining that there is an object performing the specified action, the method further includes: comparing a prior feature vector that does not match the target spatiotemporal disturbance with an existing sample feature vector to identify the identity information of the object. The prior feature vector was generated earlier than the feature vector was generated. The sample feature vector includes at least one object's feature vector that does not match the target spatiotemporal disturbance.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the action recognition method according to any one of claims 1 to 8.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, is used to implement the action recognition method according to any one of claims 1 to 8.

Images