A method for cross-domain joint recognition of fitness actions and quantities based on WiFi

Abstract

The application discloses a method for cross-domain joint identification of bodybuilding actions and quantities based on WiFi, constructs a serial structure of a space attention mechanism, a channel attention mechanism and a ResNet34 network connected with each other, allocates greater weights to key features of bodybuilding actions and quantities contained in CSI data by using the double attention mechanisms, and then identifies the bodybuilding actions and quantities respectively by using a prototype network of the parallel structure. Experiments prove that the application has high identification precision of the bodybuilding actions and quantities, and can obtain ideal cross-domain identification effects without using or only using a small amount of samples in a cross-domain environment, so that the application can provide a more intelligent, accurate and convenient bodybuilding solution for users.

Description

Technical Field

[0001] This invention relates to the field of wireless sensing technology, and in particular to a WiFi-based method for cross-domain joint recognition of fitness movements and quantities. Background Technology

[0002] In an era of increasing health awareness, exercise has become a way for people to improve their physical condition and ensure a healthy life. Standardized exercise movements and the number of movements per unit time directly affect the fitness effect. Currently, most people choose to exercise on their own, but due to the lack of effective supervision and guidance from professionals, the exercise effect is often poor, and there is even a risk of injury. To address this, artificial intelligence technology has been introduced into the field of fitness monitoring for intelligent recognition and counting of human exercise movements, providing a practical technical solution for intelligent fitness. However, existing computer vision technology has drawbacks such as high cost, leakage of personal privacy, and high requirements for lighting and environment; wearable devices also have the problem of high cost and are limited by charging and wearing devices. Some researchers have adopted WiFi-based wireless sensing technology to recognize exercise movements, which has the following advantages: it does not require users to wear additional sensors or devices, and does not restrict the user's freedom of movement; it can capture subtle movement details and changes, providing more accurate recognition of movements and quantities; it is not affected by environmental factors such as light, smoke, and obstacles, and has lower requirements for the application environment. However, it can only achieve good performance in its respective experimental scenarios, and cannot achieve ideal recognition accuracy in other cross-domain scenarios different from the experimental scenarios. Summary of the Invention

[0003] The present invention aims to solve the aforementioned technical problems existing in the prior art by providing a WiFi-based method for cross-domain joint recognition of fitness movements and quantities.

[0004] The technical solution of this invention is: a WiFi-based method for cross-domain joint recognition of fitness movements and quantities, characterized by the following method:

[0005] Step 1: Collect and extract CSI data from the receiving end as follows:

[0006]

[0007] Where: H s (f k ,t) represents the CSI static component of the k-th subcarrier, A d (f k ,t) and d d (t) represents the amplitude, phase, and path length of the CSI dynamic component of the k-th subcarrier, respectively. It represents a time-varying random phase shift;

[0008] Step 2. Calculate the CSI data ratio of two different antennas on the same WiFi network interface card, using the following formula:

[0009]

[0010] In the formula: H1(f k ,t) and H2(f k ,t) represent the CSI data of the kth subcarrier of the two receiving antennas, and Δd represents the length difference of the reflection path of the dynamic component of the CSI data of the two receiving antennas;

[0011] Extract H q (f k Phase information Q of t): Q = angle(H q (f k ,t));

[0012] In the formula: angle() represents extracting H q (f k (t) is a function of phase information;

[0013] Construct the phase information Q into an image P: P = imagesc(Q);

[0014] In the formula: imagesc() represents a function that maps phase information to a 3-channel RGB image;

[0015] Step 3: Input the image P into the following spatial attention mechanism weight function:

[0016] Map1(P) = σ(f 7×7 (BN(P)));

[0017] In the formula: σ() represents the Sigmoid function, f 7×7 () represents a convolution operation with a 7×7 kernel, BN() represents a batch normalization operation, and Map1(P) is a matrix with the same dimension as P, where each element corresponds to the weight of a pixel in P.

[0018] Then, multiply Map1(P) element-wise with the RGB image P to obtain the reconstructed image;

[0019] Step 4: For the obtained reconstructed image, use the ResNet34 network to extract features and obtain feature map F. Then, use the channel attention mechanism to assign different weights to different channels, as shown in the function below:

[0020] Map2(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))),

[0021] In the formula: AvgPool() and MaxPool() are the average pooling and max pooling functions, respectively; MLP() is the multilayer perceptron function; and Map2() is then element-wise multiplied with the feature map F to obtain the reconstructed feature map Z. test ;

[0022] Step 5: Construct two sets of prototype networks with parallel structures to reconstruct the feature map Z. test The input is fed into the prototype network to identify fitness movements and the number of movements, respectively.

[0023] The two sets of prototype networks that construct the parallel structure are obtained by reconstructing feature maps from N training samples according to steps 1-4 during network training. And for each sample, there is a class label a, a = 1, 2, ..., A, with a total of A classes. Calculate the class label for all training samples in each class a. The arithmetic mean is used to obtain the prototype vector μ of prototype network category a. a ;

[0024] The reconstructed feature map Z test The input is fed into the prototype network to identify fitness movements and the number of movements, respectively, and the reconstructed feature map Z is calculated. test With each prototype vector μ a The Euclidean distance is calculated, and then the predicted probability of the test sample is calculated using the Softmax function, as shown in the following formula:

[0025]

[0026] In the formula: d(Z) test ,μ a ) represents Z test and μ a Euclidean distance;

[0027] Comparing all the obtained predicted probabilities, Z test It belongs to the category with the highest predicted probability.

[0028] This invention constructs a spatial attention mechanism, a channel attention mechanism, and a serial structure of a ResNet34 network connecting the two. This dual attention mechanism assigns greater weight to key features of fitness movements and quantities contained in CSI data. Then, a prototype network with a constructed parallel structure is used to identify the fitness movements and quantities separately. Experiments demonstrate that this invention not only has high accuracy in recognizing fitness movements and quantities, but also achieves ideal cross-domain recognition results even with little or no samples in cross-domain environments, providing users with a more intelligent, accurate, and convenient fitness solution. Attached Figure Description

[0029] Figure 1 This is a network structure block diagram of an embodiment of the present invention.

[0030] Figure 2 This is a flowchart of an embodiment of the present invention.

[0031] Figure 3 This is a bar chart showing the average in-domain recognition accuracy in an embodiment of the present invention.

[0032] Figure 4 This is a bar chart showing the average cross-domain recognition accuracy of an embodiment of the present invention.

[0033] Figure 5 This is a bar chart showing the average cross-domain recognition accuracy with small sample assistance according to an embodiment of the present invention. Detailed Implementation

[0034] This embodiment of the invention uses two computers equipped with Intel 5300 network cards and Ubuntu 12.04 operating systems. One computer is connected to one antenna as a transmitter, and the other computer is connected to three antennas as a receiver. The antennas are both 0.5 meters above the ground, and the distance between the transmitting and receiving devices is 3 meters. The WiFi operating frequency is set to 2.4 GHz, the subcarrier bandwidth is 20 MHz, and the device packet transmission frequency is 1000 Hz. The experimental system transmits data through three channels, each with 30 subcarriers.

[0035] In this embodiment of the invention, three indoor experimental environments were selected. In each experimental environment, three volunteers performed six fitness movements. Each movement was repeated 1-3 times. CSI data was collected for 4 seconds for each sample. Fifty samples were collected for each movement from each person as a sample set, including training samples and test samples.

[0036] The present invention provides a WiFi-based method for cross-domain joint recognition of fitness movements and quantities, the network structure of which is shown in the following diagram. Figure 1 As shown, the feature extraction module has a spatial attention mechanism, a channel attention mechanism, and a serial structure of a ResNet34 network connecting the two; the prototype network module consists of two sets of prototype networks in parallel structure, namely the quantity prototype and the action prototype, followed by a distance metric and a Softmax function, respectively.

[0037] The process is as follows Figure 2 As shown: First, both the training and test samples are processed using steps 1-4 to obtain the reconstructed feature maps, as detailed below:

[0038] Step 1: Collect and extract CSI data from the receiving end as follows:

[0039]

[0040] Where: Hs (f k ,t) represents the CSI static component of the k-th subcarrier, A d (f k ,t) and d d (t) represents the amplitude, phase, and path length of the CSI dynamic component of the k-th subcarrier, respectively. It represents a time-varying random phase shift;

[0041] Step 2. Calculate the CSI data ratio of two different antennas on the same WiFi network interface card to eliminate random offset, using the following formula:

[0042]

[0043] In the formula: H1(f k ,t) and H2(f k ,t) represent the CSI data of the kth subcarrier of the two receiving antennas, and Δd represents the length difference of the reflection path of the dynamic component of the CSI data of the two receiving antennas;

[0044] Extract H q (f k Phase information Q of t): Q = angle(H q (f k ,t));

[0045] In the formula: angle() represents extracting H q (f k (t) is a function of phase information;

[0046] Construct the phase information Q into a 3×224×224 3-channel RGB image P: P = imagesc(Q);

[0047] In the formula: imagesc() represents a function that maps phase information to a 3-channel RGB image;

[0048] Step 3: Input the image P into the following spatial attention mechanism weight function:

[0049] Map1(P) = σ(f 7×7 (BN(P)));

[0050] In the formula: σ() represents the Sigmoid function, f 7×7 () represents a convolution operation with a 7×7 kernel, BN() represents a batch normalization operation, and Map1(P) is a matrix with the same dimension as P, where each element corresponds to the weight of a pixel in P.

[0051] Then, multiply Map1(P) element-wise with the RGB image P to obtain the reconstructed image;

[0052] Step 4: For the obtained reconstructed image, a ResNet34 network is used to extract features, resulting in a feature map F of size 512×7×7. Feature map F contains 512 channels, each channel being a 7×7 matrix. Then, a channel attention mechanism is used to assign different weights to different channels, as shown in the following function:

[0053] Map2(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))),

[0054] In the formula: AvgPool() and MaxPool() are the average pooling and max pooling functions, respectively, MLP() is the multilayer perceptron function, and Map2() is then multiplied element-wise with the feature map F to obtain the reconstructed feature map;

[0055] The reconstructed feature map of the training samples is represented as And the category label a (a = 1, 2, "A") for each sample, with 6 categories for fitness movements and 3 categories for the number of fitness sessions;

[0056] The reconstructed feature map of the test sample is represented as follows:

[0057] Step 5: Construct two sets of prototype networks with parallel structures, where, during network training, the computational power of all training samples for each class a is calculated. The arithmetic mean is used to obtain the prototype vector μ of prototype network category a. a ;

[0058] Reconstruct feature maps from test samples The input is fed into the prototype network to identify fitness movements and the number of movements, specifically by calculating the number of movements for each test sample. With each prototype vector μ a The Euclidean distance is calculated, and then the predicted probability of the test sample is calculated using the Softmax function, as shown in the following formula:

[0059]

[0060] In the formula: express and μ a Euclidean distance, calculation After determining the predicted probability for each category, compare all the obtained predicted probabilities. It belongs to the category with the highest predicted probability.

[0061] In this embodiment of the invention, the first 80% of the data for each experimental scenario is selected as the training sample, and the last 20% is selected as the test sample. The average recognition accuracy of the three scenarios is calculated, and the average intra-domain recognition accuracy is shown in the bar chart below. Figure 3 As shown in the figure, the horizontal axis representing "Action," "Quantity," and "BOTH" indicates the average in-domain recognition accuracy when identifying only actions, only quantities, and both using the dataset, respectively. Figure 3 It can be seen that the average recognition accuracy within the domain for "action" and "quantity" is higher than that for "BOTH", but the average recognition accuracy within all domains is above 94%, indicating that the present invention has a high average recognition accuracy within the domain.

[0062] In this embodiment of the invention, all data from two experimental scenarios are selected as training samples, and data from a third experimental scenario are used as test samples. The average cross-domain recognition accuracy is shown in the bar chart below. Figure 4 As shown. From Figure 4 It can be seen that the cross-domain average recognition accuracy rates for "action" and "BOTH" are 83.21% and 82.22%, respectively, which are 13.66% and 12.22% lower than the intra-domain average recognition accuracy rates. In addition, the cross-domain average recognition accuracy rate for "quantity" is 97.14%, achieving very ideal performance.

[0063] In this embodiment of the invention, all data from two experimental scenarios and 0, 5, 10, and 15 samples from a third scenario were selected as training samples, and the remaining samples from the third experimental scenario were used as test samples. The experimental results are as follows: Figure 5 As shown. From Figure 5 It can be seen that the average cross-domain recognition accuracy of "Action" and "BOTH" with small sample assistance increases with the number of auxiliary samples. Specifically, with 5, 10, and 15 auxiliary samples, the average cross-domain recognition accuracy of "Action" increased by 7.17%, 13.33%, and 14.12%, respectively; while the average cross-domain recognition accuracy of "BOTH" increased by 3.71%, 11.52%, and 12.39%, respectively. These experimental results demonstrate that the performance of the present invention is significantly improved by using small sample assistance.

Claims

1. A WiFi-based method for cross-domain joint recognition of fitness movements and quantities, characterized in that... Follow these steps: Step 1: Collect and extract CSI data from the receiving end as follows: Where: H s (f k ,t) represents the CSI static component of the k-th subcarrier, A d (f k ,t) and d d (t) represents the amplitude, phase, and path length of the CSI dynamic component of the k-th subcarrier, respectively. It represents a time-varying random phase shift; Step 2. Calculate the CSI data ratio of two different antennas on the same WiFi network interface card, using the following formula: In the formula: H1(f k ,t) and H2(f k ,t) represent the CSI data of the kth subcarrier of the two receiving antennas, and Δd represents the length difference of the reflection path of the dynamic component of the CSI data of the two receiving antennas; Extract H q (f k Phase information Q of t): Q = angle(H q (f k ,t)); In the formula: angle() represents extracting H q (f k (t) is a function of phase information; Construct the phase information Q into an image P: P = imagesc(Q); In the formula: imagesc() represents a function that maps phase information to a 3-channel RGB image; Step 3: Input the image P into the following spatial attention mechanism weight function: Map1(P)1σ(f 7×7 (BN(P))) In the formula: σ() represents the Sigmoid function, f 7×7 () represents a convolution operation with a 7×7 kernel, BN() represents a batch normalization operation, and Map1(P) is a matrix with the same dimension as P, where each element corresponds to the weight of a pixel in P. Then, multiply Map1(P) element-wise with the RGB image P to obtain the reconstructed image; Step 4: For the obtained reconstructed image, use the ResNet34 network to extract features and obtain feature map F. Then, use the channel attention mechanism to assign different weights to different channels, as shown in the function below: Map2(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))), In the formula: AvgPool() and MaxPool() are the average pooling and max pooling functions, respectively; MLP() is the multilayer perceptron function; and Map2() is then element-wise multiplied with the feature map F to obtain the reconstructed feature map Z. test ; Step 5: Construct two sets of prototype networks with parallel structures to reconstruct the feature map Z. test The input is fed into the prototype network to identify fitness movements and the number of movements, respectively. The two sets of prototype networks that construct the parallel structure are obtained by reconstructing feature maps from N training samples according to steps 1-4 during network training. Given n = 1, 2, ..., N, and the class label a, a = 1, 2, ..., A for each sample, calculate the class label for all training samples of each class a. The arithmetic mean is used to obtain the prototype vector μ of prototype network category a. a ; The reconstructed feature map Z test The input is fed into the prototype network to identify fitness movements and the number of movements, respectively, and the reconstructed feature map Z is calculated. test With each prototype vector μ a The Euclidean distance is calculated, and then the predicted probability of the test sample is calculated using the Softmax function, as shown in the following formula: In the formula: d(Z) test ,μ a ) represents Z test and μ a Euclidean distance; Comparing all the obtained predicted probabilities, Z test It belongs to the category with the highest predicted probability.

Images