Anti-jumping system based on Wi-Fi signal and multiple sensing

Abstract

The invention discloses an anti-drop system based on a Wi-Fi signal and multiple sensors. The anti-drop system comprises a data acquisition module, a data processing module, a data identification module, a multi-modal data decision fusion module and a protection measure module. The data acquisition module is used for acquiring CSI data and physical sensing data; the data processing module is used for preprocessing the CSI data and the physical sensing data; the data identification module classifies and identifies the preprocessed CSI data and physical sensing data to obtain classification and identification results; and the multi-modal data decision fusion module performs decision-level fusion by using logic and operation according to the classification and recognition result to obtain a decision-making result, and starts different levels of protection measures according to the decision-making result. According to the invention, dangerous behavior actions possibly causing the jumping can be detected, corresponding protection measures can be started, early prevention and early response can be realized, and meanwhile, the method has the advantages of high accuracy and sensitive response.

Description

Technical Field

[0001] The present invention relates to the technical field of safety assurance, and in particular to an anti-jumping system based on Wi-Fi signals and multiple sensors. Background Art

[0002] In recent years, high-altitude jumping incidents have occurred frequently in sensitive places such as schools and hospitals, causing tragedies and losses to countless families and society. At present, in addition to safety training and the formulation of strict safety standards, most of the existing high-altitude prevention measures are to set up physical obstacles such as railings and power grids. However, this will not only block the user's field of vision, create a tense atmosphere and a sense of unfreedom; but there is also a risk of electric shock in the power grid, which may cause harm to the user. Therefore, real-time detection of user behavior can avoid the harm caused by setting up physical obstacles. Based on computer vision to monitor dangerous high-altitude jumping behavior, this method can capture the user's behavior at heights in real time. However, in some special scenarios, such as hospital wards, nursing homes, campuses, etc., setting up cameras may make users feel uncomfortable, and privacy needs are more sensitive and important. This image method has weak privacy. The device-based sensing method uses wearable sensors such as accelerometers and gyroscopes to identify human activities. However, this wearable sensor requires users to wear sensors, which is very inconvenient to move, lacks comfort for long-term use, and is not suitable for the living habits of most people. Summary of the invention

[0003] The purpose of the present invention is to provide an anti-jumping system based on Wi-Fi signals and multiple sensors. The present invention can detect dangerous behaviors that may lead to jumping from a building and initiate corresponding protective measures, thereby achieving early prevention and early response, and has the advantages of high accuracy and quick response.

[0004] The technical solution of the present invention: an anti-jumping system based on Wi-Fi signals and multiple sensors, including a data acquisition module, a data processing module, a data identification module, a multi-modal data decision fusion module and a protection measure module;

[0005] The data acquisition module is used to acquire CSI data and physical sensor data;

[0006] The data processing module is used to pre-process the CSI data and physical sensor data;

[0007] The data identification module classifies and identifies the preprocessed CSI data and physical sensor data to obtain classification and identification results;

[0008] The multimodal data decision fusion module uses a logical AND operation to perform decision-level fusion according to the classification and recognition results to obtain a decision result;

[0009] The protection measure module initiates protection measures of different levels according to the decision result.

[0010] In the above-mentioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data acquisition module, physical sensing data is acquired by infrared ranging sensors, ultrasonic sensors and pressure sensors; the infrared ranging sensor is used to quickly detect whether a human body is close to a dangerous position; the ultrasonic sensor is used to determine the actual distance between the human body and the dangerous position; the pressure sensor is set at the dangerous position to detect whether a human body applies pressure at the dangerous position.

[0011] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data processing module, the preprocessing of physical sensor data is to automatically segment according to the timestamp of the CSI signal, obtain data samples of the same duration, and finally standardize the data.

[0012] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data recognition module, a support vector machine is used to classify the pre-processed physical sensor data signal into two categories: whether the distance is dangerous and whether the human body is exerting pressure in a dangerous position.

[0013] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data acquisition module, the CSI data is acquired by a WiFi data collection system composed of a transmitter and a receiver; the number of antennas of the transmitter is 1, the number of antennas of the receiver is 3, the number of subcarriers is 30, the data sampling rate is set to 1000 Hz, the collection is 2s, and each CSI data is composed of a 90×2000 matrix.

[0014] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data processing module, the preprocessing of CSI data is to first normalize the amplitude of the CSI data:

[0015]

[0016] Among them, x normalization is the normalized data; μ is the mean of all sample data, and σ is the variance of all sample data.

[0017] The normalized data is then downsampled from the original 2000 time steps to 500 time steps and resized into a three-dimensional array (3, 30, 500).

[0018] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, in the data recognition module, a neural network is used to perform action recognition and classification on the pre-processed CSI data.

[0019] The aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, the neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer and a fully connected layer connected in sequence; wherein the first convolutional layer uses CSI data of size 3×30×500, accepts 3 input channels, outputs 16 channels, uses a 5x5 convolution kernel, uses a ReLu activation function, and the output size of this layer is 16×26×496; the first pooling layer and the second pooling layer both use the maximum pooling layer, downsample the feature map, and halve the size, the first The output size of the pooling layer is 16×13×248; the second convolutional layer accepts 16 input channels and outputs 32 channels, using a 5x5 convolution kernel. The output size of the second convolutional layer is 32×9×244, and the output size after the ReLu activation function and the second pooling layer is 32×4×122; the last fully connected layer contains two linear layers to achieve the mapping from the convolutional layer to the output category; the ReLU activation function is used after the first linear layer, and the output size of the last linear layer is num_classes, which indicates the number of categories finally classified by the model.

[0020] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, the neural network is also provided with a function for judging dangerous actions, which is used to judge whether the predicted value of the model action classification is dangerous; wherein the function judges the part of the predicted value less than the boundary value as dangerous, and the part greater than or equal to the boundary value as safe according to the set boundary value boundary.

[0021] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, the multimodal data decision fusion module uses logical AND operations to perform decision-level fusion according to the classification and recognition results to obtain the decision results, which are specifically:

[0022] The classification result of CSI data is represented as predict_csi, the classification result of distance data is represented as predict_d, and the prediction of pressure sensor data is predicted_p. The decision-level fusion is performed using logical AND operation:

[0023] The user behavior danger level of level 3 indicates a tendency to jump off the building. There are two situations: predict_d is classified as 1, predict_csi is classified as 1, and predict_p is classified as 0, which means that the user is too close to the window and the action is dangerous, but the specific action does not put pressure on the window; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 1, which means that the user is too close to the window and the action is safe, but it puts pressure on the window;

[0024] The behavior danger level is level 2, which means that the user is jumping off the building. There is one of the following situations: predict_d, predict_csi, and predict_p are all 1, that is, the user is doing dangerous actions in front of the window;

[0025] If the user triggers the second level and the protection module fails to provide protection, the behavior risk level is defined as level one;

[0026] The following behaviors are defined as safe: as long as predict_d is 0 in all cases, that is, the user is far away from the window and there is no risk of jumping off the building; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 0, that is, the user is just too close to the window but does not make the move of jumping off the building, that is, the behavior is safe.

[0027] In the aforementioned anti-jumping system based on Wi-Fi signals and multiple sensors, the protection measure module starts different levels of protection measures according to the decision result, specifically:

[0028] For the third level of dangerous behaviors, the protection measure module reminds the user through voice prompts, voice warnings or visual methods to make the user aware of the potential dangerous situation;

[0029] For the second level of dangerous behaviors, the protection measure module deploys physical protection measures to ensure the safety of the user;

[0030] For the first level of dangerous behavior, the protection measure module automatically triggers an alarm mechanism.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention adopts a Wi-Fi-based method for human motion recognition, avoiding the privacy infringement and other problems of computer vision methods, thereby avoiding the user's concerns about privacy leakage, and is suitable for dangerous motion recognition needs in environments and scenarios that require privacy protection. The present invention does not require users to wear sensors, avoids the discomfort of wearable sensors, greatly improves user comfort, and truly realizes non-invasive and senseless recognition. The dangerous motion recognition based on multimodal data of the present invention combines multiple advantages and has high application potential. It can play an important role in dangerous scenarios such as falling from heights, improve safety and avoid tragic events.

[0033] 2. The physical sensing data of the present invention is acquired by infrared ranging sensors, ultrasonic sensors and pressure sensors. The infrared ranging sensors can quickly detect whether the human body is close to the dangerous position, while the ultrasonic sensors can provide more accurate distance measurement, thereby determining the actual distance between the human body and the dangerous position. Since the infrared ranging sensor is very sensitive to the detection of approaching objects, but is easily affected by heat sources and light sources, the ultrasonic sensor is not sensitive, but the waveform generated by the ultrasonic transmitter head will produce measurement errors due to interference from other obstacles. Therefore, the present invention combines these two sensors to make up for each other's shortcomings and ensure that the human body position and distance can be effectively detected within the entire distance range. The pressure sensor of the present invention is set at a dangerous position, which can detect whether the human body applies pressure at the dangerous position, thereby making subsequent decision-making more convenient.

[0034] 3. Currently, the training data sets based on distance and CSI signals are very limited, especially the lack of data sets of human motion in dangerous environments at high altitudes indoors. Therefore, in order to enrich the data sets of human motion in dangerous environments, the present invention constructs a distance data acquisition mechanism using multiple sensors during data collection, thereby realizing data acquisition for model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the structure of the system of the present invention;

[0036] Figure 2 It is a schematic diagram of the indoor experimental environment;

[0037] Figure 3 It is a schematic diagram of the structure of a neural network. DETAILED DESCRIPTION

[0038] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.

[0039] Example: Anti-jumping system based on Wi-Fi signals and multiple sensors, such as Figure 1 As shown, it includes a data acquisition module, a data processing module, a data recognition module and a multimodal data decision fusion module;

[0040] The data acquisition module is used to acquire CSI data and physical sensor data;

[0041] The data processing module is used to pre-process the CSI data and physical sensor data;

[0042] The data identification module classifies and identifies the preprocessed CSI data and physical sensor data to obtain classification and identification results;

[0043] The multimodal data decision fusion module uses a logical AND operation to perform decision-level fusion according to the classification and recognition results to obtain a decision result;

[0044] In this embodiment, in the data acquisition module, physical sensing data is acquired by infrared ranging sensors, ultrasonic sensors and pressure sensors, wherein the infrared ranging sensors, ultrasonic sensors and pressure sensors realize data collection based on Arduino controller; the infrared ranging sensor is used to quickly detect whether the human body is close to the dangerous position; the ultrasonic sensor is used to determine the actual distance between the human body and the dangerous position; the pressure sensor is set at the dangerous position to detect whether the human body applies pressure at the dangerous position.

[0045] Infrared ranging sensors are used for human body sensing detection. They detect human bodies by detecting infrared rays around the human body. The human body has a constant body temperature, generally 37 degrees, so it emits infrared rays of a specific wavelength. The infrared ranging sensor detects this radiation and outputs a high-level signal, thereby detecting the human body. When the human body moves within the sensing range, its output remains at a high level until the person leaves, then the high level changes to a low level. This characteristic is used to detect whether the human body has entered the dangerous range.

[0046] Ultrasonic distance sensors are used to collect data from sensors in two ranges: dangerous and safe. The velocity (Velocity, V) of ultrasonic waves (sound) in the air is known to be 340 m / s. When encountering an obstacle, it will be reflected back along the original path. The distance from the location where the sound wave is emitted to the obstacle can be calculated according to formula (1).

[0047]

[0048] Among them, V is the speed of ultrasonic wave propagation in the air, which is known to be 340m / s, and T is the time it takes for the ultrasonic wave to travel back and forth to the obstacle.

[0049] The pressure sensor is placed at a dangerous location, such as a window. When a person approaches the window and makes a dangerous move, pressure is applied to the pressure sensor in order to maintain body balance. This determines the actual distance between the person and the dangerous location and whether a dangerous move is being made, providing another layer of safety verification for the anti-jump system.

[0050] In this embodiment, data collection is performed in front of the windows of the laboratory, and the experimental site is divided into a dangerous range (within 30 cm from the window) and a safe range (about 50 cm from the window, actually 60-70 cm). Figure 2As shown. Infrared, ultrasonic distance sensors and pressure sensors are fixed on the window, and each volunteer faces three sensors. The sensor data sampling rate is set to 10Hz. The sampling rate is set moderately to preserve the effective distance to the greatest extent without increasing the burden of data analysis. The definition of the dangerous distance range comes from actual measurements. If a person is within 20cm of the window, the hand can support the window sill, and the body can easily flip out. If a person is within 30cm of the window, the hand can support the window sill, but the difficulty of flipping out increases slightly. If a person is within 40cm of the window, the hand can support the window, but the difficulty of flipping out increases significantly. From a common sense perspective, people will not climb over the window at this distance.

[0051] In this embodiment, in the data acquisition module, CSI data is acquired by a WiFi data collection system consisting of a transmitter and a receiver. The WiFi data collection system is built using Linux CSI Tool, where both the receiver and the transmitter are a laptop computer Thinkpad X250 with an Intel5300 network card; the number of antennas of the transmitter is 1, the number of antennas of the receiver is 3, the number of subcarriers is 30, the data sampling rate is set to 1000 Hz, the collection is 2s, and each CSI data is composed of a 90×2000 matrix.

[0052] During the CSI data collection process, the actions performed by the volunteers included 3 dangerous actions (left leg lift, right leg lift, jump) and 3 safe actions (standstill, left hand lift, and right hand lift). Each action was performed 50 times in the dangerous and safe ranges, for a total of 600 times. When the volunteers performed dangerous actions in the dangerous range, they supported their hands on the pressure sensors on the window to maintain balance. In the end, 600 CSI data packets were obtained for each action at two positions, for a total of 3,600 data packets of training data.

[0053] In this embodiment, in the data processing module, the preprocessing of the physical sensing data is to automatically segment according to the timestamp of the CSI signal, obtain data samples of the same duration, and finally perform standardization processing on the data.

[0054] As for CSI data, since the collected original CSI data packet is a complex matrix, which contains the amplitude information and phase information of CSI, the amplitude of CSI is obtained by performing a modulo operation on the complex matrix, and the phase of the complex matrix is ​​the phase of CSI. The present invention uses the amplitude for subsequent operations, which is more stable than the phase and does not require complicated denoising operations.

[0055] First, the amplitude of the CSI data is normalized (using z-score standardization):

[0056]

[0057] Among them, x normalization is the normalized data; μ is the mean of all sample data, and σ is the variance of all sample data.

[0058] The normalized data is then downsampled (referring to the process of reducing the frequency of the original data to reduce the amount of data. It is usually used to eliminate redundant information and reduce computational complexity). In this embodiment, the CSI data is sampled at a fixed interval, and one data out of every four data is retained, thereby downsampling the original 2000 time steps to 500 time steps, and readjusting it into a three-dimensional array (3, 30, 500).

[0059] In this embodiment, in the data recognition module, a support vector machine is used to classify the preprocessed physical sensor data signal into two categories: whether the distance is dangerous and whether the human body is exerting pressure in a dangerous position. The support vector machine (SVM) is a two-classification model, the purpose of which is to find a hyperplane to segment the samples. The principle of segmentation is to maximize the interval, which is ultimately converted into a convex quadratic programming problem to solve.

[0060] The preprocessed distance and pressure data sets are divided into training sets and test sets, with the training set division ratio being 80%. The distance and pressure data represented by each action are collected for a total of 2s, with a sampling rate of 10Hz. After the SVM model is used, the classification of whether the 20 data in 2s are dangerous is obtained. The hard voting method is used, and the most votes are used to determine whether the distance and pressure data corresponding to each action are dangerous. The SVM model is trained using the training set, and the optimal hyperparameter combination is found through cross-validation and grid search CV. The optimization process can be summarized as follows:

[0061] Define the parameter range. The parameter range to be tuned is defined in the parameters variable: kernel function type (Kernel), regularization parameter (C), and coefficient (Gamma) of the RBF kernel function. The C parameter of SVM indicates the degree of penalty for error cases. The larger the C, the greater the penalty, which is prone to overfitting the model. The smaller the C, the more prone to underfitting the model. The Gamma coefficient controls the scaling of the data after projection to a higher dimension. It is a parameter of the kernel functions RBF, poly, and sigmoid. coef0 is the kernel function constant value, which is only available for the poly and sigmoid kernel functions. Degree is the highest degree of the selected polynomial when the kernel function is specified as poly. The default is a cubic polynomial. See Table 1 for specific parameters:

[0062] Kernel Function parameter linearkernel C RBFkernel C. gamma polykernel C, gamma, coef0, degree (default) sigmoidkernel C, gamma, coef0

[0063] Table 1

[0064] Use Grid Search CV to perform grid search, and use 5-fold cross validation to evaluate the performance of different parameter combinations, using accuracy as the evaluation indicator. Finally, train and obtain the best performing classifier, parameter combination, and accuracy. Through the above parameter optimization, the best SVM model is trained through the training set, and the trained SVM model is evaluated using the test set.

[0065] In this embodiment, in the data recognition module, a LeNet neural network model is implemented by using the PyTorch library to extract all feature information of CSI and complete action recognition and classification. The LeNet neural network model can extract spatial features from the input CSI data, and the fully connected layer can help achieve the final multi-category classification task. Figure 3 As shown in FIG. 1 , the neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer and a fully connected layer which are sequentially connected; the first convolutional layer uses CSI data of size 3×30×500, accepts 3 input channels, outputs 16 channels, uses a 5x5 convolution kernel, and uses a ReLu activation function. The output size of this layer is 16×26×496; the first pooling layer and the second pooling layer both use a maximum pooling layer to downsample the feature map and halve the size. The output size of the first pooling layer is 16× 13×248; the second convolutional layer accepts 16 input channels and outputs 32 channels, using a 5x5 convolution kernel. The output size of the second convolutional layer is 32×9×244, and the output size after the ReLu activation function and the second pooling layer is 32×4×122; the last fully connected layer contains two linear layers to achieve the mapping from the convolutional layer to the output category; the ReLU activation function is used after the first linear layer, and the output size of the last linear layer is num_classes, which indicates the number of categories finally classified by the model.

[0066] The neural network is also provided with a function for judging dangerous actions, which is used to judge whether the predicted value of the model action classification is dangerous; wherein the function judges the part of the predicted value less than the boundary value as dangerous (1) and the part greater than or equal to the boundary value as safe (0) according to the set boundary value boundary. The boundary value is generated according to the action label, and the six action labels of the present invention are 0-left leg raising, 1-right leg raising, 2-jumping, 3-standing still, 4-left hand raising, 5-right hand raising, and the boundary value is set to 3. The prediction result is simply and clearly converted into a binary classification result for the prediction and judgment of dangerous actions.

[0067] In this embodiment, the LeNet neural network model and the support vector machine (SVM) are used to learn the characteristics of the CSI signal and the sensor data, respectively, and the three classification results are obtained for decision-level fusion. The multimodal data decision fusion module uses a logical AND operation to perform decision-level fusion according to the classification and recognition results to obtain a decision result, which is specifically:

[0068] The classification result of CSI data is represented as predict_csi, the classification result of distance data is represented as predict_d, and the prediction of pressure sensor data is predicted_p. The decision-level fusion is performed using logical AND operation:

[0069] The user behavior danger level of level 3 indicates a tendency to jump off the building. There are two situations: predict_d is classified as 1, predict_csi is classified as 1, and predict_p is classified as 0, which means that the user is too close to the window and the action is dangerous, but the specific action does not put pressure on the window; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 1, which means that the user is too close to the window and the action is safe, but it puts pressure on the window;

[0070] The behavior danger level is level 2, which means that the user is jumping off the building. There is one of the following situations: predict_d, predict_csi, and predict_p are all 1, that is, the user is doing dangerous actions in front of the window;

[0071] If the user triggers the second level and the protection module fails to provide protection, the behavior risk level is defined as level one;

[0072] The following behaviors are defined as safe: as long as predict_d is 0 in all cases, that is, the user is far away from the window and there is no risk of jumping off the building; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 0, that is, the user is just too close to the window but does not make the move of jumping off the building, that is, the behavior is safe.

[0073] This decision-level fusion method combines the characteristics of high performance and low latency, and effectively solves the problem of rapid identification and response to dangerous actions in dangerous scenarios in real life.

[0074] Furthermore, the system of the present invention further comprises the protection measure module, which starts protection measures of different levels according to the decision result, specifically:

[0075] For the third level of dangerous behavior, the system will remind users through voice prompts or voice warnings to make users aware of potential dangerous situations. In some cases, visual prompts can be provided through smart lights or LED displays to enhance users' perception of dangerous behaviors.

[0076] For the second level of dangerous behavior, the system deploys physical protection measures to ensure the safety of users. If the system detects that the user is performing dangerous actions near the window, it will automatically close the window to avoid accidents. Notify family members or caregivers in time for manual intervention. An intelligent interception device is set on the lower side of the window. When the system identifies dangerous behavior, the interception device can be automatically popped up to ensure the personal safety of the user after continuing the dangerous action.

[0077] For the first level of dangerous behavior, the system will automatically trigger the alarm mechanism and automatically call the 120 emergency center so that they can quickly dispatch rescue personnel to the scene; automatically notify emergency contacts, such as family members or caregivers, to inform them of the emergency and provide necessary information so that they can assist in the rescue.

[0078] In summary, the present invention adopts a Wi-Fi-based method for human motion recognition, avoiding the privacy infringement and other problems of computer vision methods, thereby avoiding the user's concerns about privacy leakage, and is suitable for dangerous motion recognition needs in environments and scenarios that require privacy protection. The present invention does not require users to wear sensors, avoids the discomfort of wearable sensors, greatly improves user comfort, and truly realizes non-invasive and senseless recognition. The dangerous motion recognition based on multimodal data of the present invention combines multiple advantages and has high application potential. It can play an important role in dangerous scenarios such as falling from heights, improve safety and avoid the occurrence of tragic events.

Claims

1. Anti-jumping system based on Wi-Fi signals and multiple sensors, characterized by: It includes data acquisition module, data processing module, data identification module, multimodal data decision fusion module and protection measures module; The data acquisition module is used to acquire CSI data and physical sensor data; The data processing module is used to pre-process the CSI data and physical sensor data; The data identification module classifies and identifies the preprocessed CSI data and physical sensor data to obtain classification and identification results; The multimodal data decision fusion module uses a logical AND operation to perform decision-level fusion according to the classification and recognition results to obtain a decision result; The protection measure module initiates protection measures of different levels according to the decision result.

2. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 1 is characterized in that: In the data acquisition module, the physical sensing data is acquired by an infrared distance sensor, an ultrasonic sensor and a pressure sensor; the infrared distance sensor is used to quickly detect whether a human body is close to a dangerous position; the ultrasonic sensor is used to determine the actual distance between the human body and the dangerous position; the pressure sensor is set at the dangerous position to detect whether the human body applies pressure at the dangerous position; In the data processing module, the preprocessing of the physical sensor data is to automatically segment a variety of physical sensor data according to the timestamp of the CSI signal, obtain data samples of the same duration, and finally perform standardization processing on the data.

3. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 2 is characterized in that: In the data recognition module, a support vector machine is used to perform binary classification on the pre-processed physical sensor data as to whether the distance is dangerous and whether the human body is exerting pressure at a dangerous position.

4. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 2 is characterized in that: In the data acquisition module, CSI data is acquired by a WiFi data collection system consisting of a transmitter and a receiver; the number of antennas of the transmitter is 1, the number of antennas of the receiver is 3, the number of subcarriers is 30, the data sampling rate is set to 1000 Hz, the collection is 2s, and each CSI data is composed of a 90×2000 matrix.

5. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 4 is characterized in that: In the data processing module, the preprocessing of the CSI data is to first normalize the amplitude of the CSI data: Among them, x normalization is the normalized data; μ is the mean of all sample data, and σ is the variance of all sample data. The normalized data is then downsampled from the original 2000 time steps to 500 time steps and resized into a three-dimensional array (3, 30, 500).

6. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 5 is characterized in that: In the data recognition module, a neural network is used to perform action recognition and classification on the pre-processed CSI data.

7. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 6 is characterized in that: The neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer and a fully connected layer which are sequentially connected; the first convolutional layer uses CSI data of size 3×30×500, accepts 3 input channels, outputs 16 channels, uses a 5x5 convolution kernel, and uses a ReLu activation function. The output size of this layer is 16×26×496; the first pooling layer and the second pooling layer both use a maximum pooling layer to downsample the feature map and halve the size. The output size of the first pooling layer is 16×1 3×248; the second convolutional layer accepts 16 input channels and outputs 32 channels, using a 5x5 convolution kernel. The output size of the second convolutional layer is 32×9×244, and the output size after the ReLu activation function and the second pooling layer is 32×4×122; the last fully connected layer contains two linear layers to achieve the mapping from the convolutional layer to the output category; the ReLU activation function is used after the first linear layer, and the output size of the last linear layer is num_classes, which indicates the number of categories of the model action classification.

8. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 7 is characterized in that: The neural network is also provided with a function for judging dangerous actions, which is used to judge whether the predicted value of the model action classification is dangerous; wherein the function judges the part of the predicted value that is less than the boundary value as dangerous, and the part that is greater than or equal to the boundary value as safe according to the set boundary value boundary.

9. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 6 is characterized in that: The multimodal data decision fusion module uses logical AND operations to perform decision-level fusion according to the classification and recognition results to obtain a decision result, which is specifically: The classification result of CSI data is represented as predict_csi, the classification result of distance data is represented as predict_d, and the prediction of pressure sensor data is predicted_p. The decision-level fusion is performed using logical AND operation: The user behavior danger level of level 3 indicates a tendency to jump off the building. There are two situations: predict_d is classified as 1, predict_csi is classified as 1, and predict_p is classified as 0, which means that the user is too close to the window and the action is dangerous, but the specific action does not put pressure on the window; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 1, which means that the user is too close to the window and the action is safe, but it puts pressure on the window; The behavior danger level is level 2, which means that the user is jumping off the building. There is one of the following situations: predict_d, predict_csi, and predict_p are all 1, that is, the user is doing dangerous actions in front of the window; If the user triggers the second level and the protection module fails to provide protection, the behavior risk level is defined as level one; The following behaviors are defined as safe: as long as predict_d is 0 in all cases, that is, the user is far away from the window and there is no risk of jumping off the building; predict_d is classified as 1, predict_csi is classified as 0, and predict_p is classified as 0, that is, the user is just too close to the window but does not make the move of jumping off the building, that is, the behavior is safe.

10. The anti-jumping system based on Wi-Fi signals and multiple sensors according to claim 9, characterized in that: The protection measure module starts protection measures of different levels according to the decision result, specifically: For the third level of dangerous behaviors, the protection measure module reminds the user through voice prompts, voice warnings or visual methods to make the user aware of the potential dangerous situation; For the second level of dangerous behaviors, the protection measure module deploys physical protection measures to ensure the safety of the user; For the first level of dangerous behavior, the protection measure module automatically triggers an alarm mechanism.

Images