WiFi signal human body posture estimation algorithm based on mixed attention mechanism

Abstract

According to the WiFi signal human body posture estimation algorithm based on the mixed attention mechanism, WiFi channel state information data is used as input through an artificial intelligence means, and indoor human body skeleton information is predicted. According to the method, the WiFi signals are used as input data, the problems that a traditional picture-based method is weak in light, obstructs and invades personal privacy and the like are solved, corresponding functions can be achieved only by using the WiFi household router as hardware needed by the method, the pressure of equipment cost is reduced, and the method has the potential of large-scale deployment. Besides, through a channel and an attention mechanism, the characteristics of the acquired WiFi channel information data are more emphasized, so that the human body posture information can be predicted more accurately.

Description

Technical Field

[0001] The present invention relates to the field of human body posture estimation, and in particular to an algorithm for human body posture estimation based on a hybrid attention mechanism for WiFi signals. Background Art

[0002] The human posture estimation technology based on WiFi signals combines the progress of IoT technology and deep learning to provide an innovative solution for indoor security management. With the rapid development of IoT technology, people's demand for indoor security monitoring and management is increasing. Traditional monitoring methods have many limitations, such as privacy issues in camera monitoring, high cost and difficult maintenance of sensor deployment. WiFi signals, as a popular and privacy-friendly communication method, provide new possibilities for indoor monitoring.

[0003] The background of this technology comes from the increasing demand for indoor safety management. People's attention to indoor safety is constantly increasing, and the requirements for safety monitoring and management in various places are becoming increasingly stringent. Human posture estimation technology based on WiFi signals can monitor the location and movement trajectory of people in real time. Combined with technologies such as deep learning, it can accurately identify human posture and behavior, thereby realizing early warning and monitoring of unsafe behaviors.

[0004] This technology has great application potential. It can be used not only for security management in public places such as hospitals and office buildings, but also for the care of the elderly and children, and can even be extended to safety monitoring in industrial production. By using WiFi signals to estimate human posture, a low-cost, light-free and non-obstructive indoor security monitoring solution can be implemented, providing more reliable protection for people's lives and work. Summary of the invention

[0005] The purpose of this invention is to propose a pedestrian video description algorithm based on a temporal attention mechanism to address the lack of research on pedestrian video description and the incompatibility between the temporal nature of video description and attention.

[0006] The technical solution of the present invention is:

[0007] The present invention provides a pedestrian video description algorithm based on a temporal attention mechanism, the algorithm comprising the following steps:

[0008] Step 1: Input the feature data of the human posture estimation data set based on the CSI signal into the upsampling network, and input the sample number N, carrier number S, signal transmitter antenna number N1, and signal collector antenna number N2 of the feature data as parameters into the input layer of the model; perform data expansion and convolution of the spatial dimension and convolution of the channel dimension on the input feature data in the upsampling layer;

[0009] Step 2: Input the processed raw data into a stacked multi-scale convolutional network, where each multi-scale convolutional network includes convolutional networks with multiple convolution kernel sizes, and adjacent multi-scale convolutional networks are connected using a downsampling network to extract data features in the spatial dimension;

[0010] Step 3: Introduce the spatial self-attention module, assign spatial attention weights to the extracted feature vectors, and generate deep feature vectors with human posture information;

[0011] Step 4: Input the feature vector into the stacked deep residual convolutional network. Each deep residual convolutional network includes multiple continuously stacked convolutional networks, and the input is added to the output of the last convolutional network using a residual connection based on spatial attention. Adjacent deep residual convolutional networks are linked using an upsampling network. The output of the last deep residual convolutional network is connected to the first multi-scale convolutional network using a residual connection operation.

[0012] Step 5: For the feature vector containing human posture information, use the global adaptive pooling layer and convolutional network to convert the channel and spatial resolution, convert it to the target resolution, and output two-dimensional human posture information.

[0013] According to the WiFi signal human posture estimation algorithm based on the hybrid attention mechanism, it is characterized by the stacked multi-scale convolutional network in step 2, wherein the multi-scale convolutional network is specifically:

[0014] Step 2.1: Multi-scale convolutional network, with convolution kernels of multiple resolution sizes, for the same input data (c in , H in , w in ) extracts feature information of different scales and ensures that each intermediate convolutional layer outputs X1, X2, ..., X n The resolution of the convolutional layers is the same, and the padding and stride of each convolutional layer need to be adjusted accordingly.

[0015] X1=conv1(c in , H in , w in )

[0016] X2=conv2(c in , H in , w in )

[0017] …

[0018] X n =conv n (c in , H in , win )

[0019] Step 2.2: Connect the data output by the intermediate convolutional layer according to the channel dimension, and fuse the features of each scale to obtain the corresponding data.

[0020] X=cat(X1,X2,...,X n )

[0021] Step 2.3: Finally, the convolution layer conv, batch normalization function BacthNorm, and activation function Active are used to perform channel and spatial convolution on the data output by the intermediate convolution layer.

[0022] (c out , H out , W out )=Active(BatchNorm(conv(X)))

[0023] Step 2.4: Adjacent multi-scale convolutional networks are connected using downsampling networks. The deep network data is the feature extraction of the shallow network data. The downsampling network includes the maximum pooling layer MaxPool and the convolution network conv with a convolution kernel size of 1x1. 1×1 combination.

[0024] layer i+1 =conv 1×1 (MaxPool(layer i ))

[0025] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the spatial self-attention module in step 3, and the implementation of the spatial self-attention module is specifically as follows:

[0026] Step 3.1: Convert the input feature vector to a resolution format, and convert the first and second dimensions to an N×1 text sequence format.

[0027] W=Trans w×H→N×1 (Feature)

[0028] Step 3.2: Position-encode the N×1 feature sequence and add it to the feature vector of the original text sequence.

[0029] P(loc,2i)=sin(loc / 10000 2i / d )

[0030] P(loc,2i+1)=cos(loc / 10000 2i / d )

[0031] X=P+W

[0032] Step 3.3: Use multiple layers of self-attention mechanism to set specific weights, giving higher weights to eigenvalues ​​with greater effects and smaller weights to eigenvalues ​​with smaller effects.

[0033] X = Transformer1(X)

[0034] …

[0035] X = Transformer n (X)

[0036] Step 3.4: Use resolution conversion to convert the N×1 text sequence format into the first and second dimensional resolutions of the input feature vector to ensure the uniformity of input and output resolutions and obtain the feature vector with attention weight adjustment.

[0037] Res=Trans N×1→W×H (X)

[0038] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the deep residual convolutional network in step 4, and the implementation of the deep residual convolutional network is specifically as follows:

[0039] Step 4.1: Use multiple continuously stacked convolutional networks to restore features, and use residual connections to link the convolutional network output data and input data.

[0040] X1=conv(X)

[0041] X2=conv(X1)

[0042] …

[0043] X n =conv(X n-1 )

[0044] Step 4.2: Use the spatial attention mechanism in the residual connection to extract the more important channel features in the input data and add them to the output data of the stacked convolutional network.

[0045]

[0046] Res=X+X n

[0047] Among them, GAP is the global average pooling function, sig is the Sigmoid function that generates a value between 0 and 1. is the dot product function.

[0048] Step 4.3: Adjacent continuously stacked convolutional networks are connected using an upsampling network. The shallow network data restores the features of the deep network data. The upsampling network includes a bilinear interpolation function.

[0049] layer i+1 =upsample(layer i )

[0050] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized in that the processing output of human posture information in step 5, the implementation of the processing network of human posture information is specifically as follows:

[0051] Use the global adaptive pooling layer to control the first and second dimensional resolution of the input data to the specified number of human skeleton joints, and then use a convolution layer with a convolution kernel of 1×1. Through the convolution operation on the channel, the number of channels of the output data is made to be the number of dimensions of the predicted human posture estimation.

[0052] Res = conv 1×1 (AdaptiveAvg(X))

[0053] (x i ,y i )=Res

[0054] Beneficial effects of the present invention:

[0055] The advantages of the present invention over the prior art are: the present invention processes WiFi signals by means of artificial intelligence to obtain indoor human posture information, solving the problems of weak light, obstruction, and invasion of personal privacy in traditional methods, and the price of the signal transceiver is relatively low, which reduces the cost pressure. In addition, the present invention pays more attention to the inherent characteristics of the collected WiFi channel information data through the channel and attention mechanism, so that human posture information can be predicted more accurately.

[0056] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0058] Figure 1 This is a flowchart of the WiFi signal human posture estimation algorithm based on the hybrid attention mechanism.

[0059] Figure 2This is a network model diagram for WiFi signal human posture estimation based on the hybrid attention mechanism.

[0060] Figure 3 It is a model diagram of multi-scale convolutional network and deep residual convolutional network. DETAILED DESCRIPTION

[0061] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0062] like Figure 1 As shown, the present invention provides a WiFi signal human posture estimation algorithm based on a hybrid attention mechanism, and the method comprises the following steps:

[0063] Step 1: Input the feature data of the human posture estimation data set based on the CSI signal into the upsampling network, and input the sample number N, carrier number S, signal transmitter antenna number N1, and signal collector antenna number N2 of the feature data as parameters into the input layer of the model; perform data expansion and convolution of the spatial dimension and convolution of the channel dimension on the input feature data in the upsampling layer;

[0064] Step 2: Input the processed raw data into a stacked multi-scale convolutional network, where each multi-scale convolutional network includes convolutional networks with multiple convolution kernel sizes, and adjacent multi-scale convolutional networks are connected using a downsampling network to extract data features in the spatial dimension;

[0065] Step 3: Introduce the spatial self-attention module, assign spatial attention weights to the extracted feature vectors, and generate deep feature vectors with human posture information;

[0066] Step 4: Input the feature vector into the stacked deep residual convolutional network. Each deep residual convolutional network includes multiple continuously stacked convolutional networks, and the input is added to the output of the last convolutional network using a residual connection based on spatial attention. Adjacent deep residual convolutional networks are linked using an upsampling network. The output of the last deep residual convolutional network is connected to the first multi-scale convolutional network using a residual connection operation.

[0067] Step 5: For the feature vector containing human posture information, use the global adaptive pooling layer and convolutional network to convert the channel and spatial resolution, convert it to the target resolution, and output two-dimensional human posture information.

[0068] According to the WiFi signal human posture estimation algorithm based on the hybrid attention mechanism, it is characterized by the stacked multi-scale convolutional network in step 2, wherein the multi-scale convolutional network is specifically:

[0069] Step 2.1: Multi-scale convolutional network, with convolution kernels of multiple resolution sizes, for the same input data (c in , H in ,w in ) extracts feature information of different scales and ensures that each intermediate convolutional layer outputs X1, X2,..., X n The resolution of the convolutional layers is the same, and the padding and stride of each convolutional layer need to be adjusted accordingly.

[0070] X1=conv1(c in , H in , w in )

[0071] X2=conv2(c in , H in , w in )

[0072] …

[0073] X n =conv n (c in , H in , w in )

[0074] Step 2.2: Connect the data output by the intermediate convolutional layer according to the channel dimension, and fuse the features of each scale to obtain the corresponding data.

[0075] X=cat(X1,X2,...,X n )

[0076] Step 2.3: Finally, the convolution layer conv, batch normalization function BacthNorm, and activation function Active are used to perform channel and spatial convolution on the data output by the intermediate convolution layer.

[0077] (c out , H out , W out )=Active(BatchNorm(conv(X)))

[0078] Step 2.4: Adjacent multi-scale convolutional networks are connected using downsampling networks. The deep network data is the feature extraction of the shallow network data. The downsampling network includes the maximum pooling layer MaxPool and the convolution network conv with a convolution kernel size of 1x1. 1×1 combination.

[0079] layer i+1 =conv 1×1 (MaxPool(layer i ))

[0080] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the spatial self-attention module in step 3, and the implementation of the spatial self-attention module is specifically as follows:

[0081] Step 3.1: Convert the input feature vector to a resolution format, and convert the first and second dimensions to an N×1 text sequence format.

[0082] W=Trans W×H→N×1 (Feature)

[0083] Step 3.2: Position-encode the N×1 feature sequence and add it to the feature vector of the original text sequence.

[0084] P(loc,2i)=sin(loc / 10000 2i / d )

[0085] P(loc,2i+1)=cos(loc / 10000 2i / d )

[0086] X=P+W

[0087] Step 3.3: Use multiple layers of self-attention mechanism to set specific weights, giving higher weights to eigenvalues ​​with greater effects and smaller weights to eigenvalues ​​with smaller effects.

[0088] X = Transformer1(X)

[0089] …

[0090] X = Transformer n (X)

[0091] Step 3.4: Use resolution conversion to convert the N×1 text sequence format into the first and second dimensional resolutions of the input feature vector to ensure the uniformity of input and output resolutions and obtain the feature vector with attention weight adjustment.

[0092] Res=Trans N×1→W×H (X)

[0093] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the deep residual convolutional network in step 4, and the implementation of the deep residual convolutional network is specifically as follows:

[0094] Step 4.1: Use multiple continuously stacked convolutional networks to restore features, and use residual connections to link the convolutional network output data and input data.

[0095] X1=conv(X)

[0096] X2=conv(X1)

[0097] …

[0098] X n =conv(X n-1 )

[0099] Step 4.2: Use the spatial attention mechanism in the residual connection to extract the more important channel features in the input data and add them to the output data of the stacked convolutional network.

[0100]

[0101] Res=X+X n

[0102] Among them, GAP is the global average pooling function, sig is the Sigmoid function that generates a value between 0 and 1. is the dot product function.

[0103] Step 4.3: Adjacent continuously stacked convolutional networks are connected using an upsampling network. The shallow network data restores the features of the deep network data. The upsampling network includes a bilinear interpolation function.

[0104] layer i+1 =upsample(layer i )

[0105] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized in that the processing output of human posture information in step 5, the implementation of the processing network of human posture information is specifically as follows:

[0106] (c out , H out , w out )=Active(BatchNorm(conv(X)))

[0107] Step 2.4: Adjacent multi-scale convolutional networks are connected using downsampling networks. The deep network data is the feature extraction of the shallow network data. The downsampling network includes the maximum pooling layer MaxPool and the convolution network conv with a convolution kernel size of 1x1. 1×1 combination.

[0108] layeri+1 =conv 1×1 (MaxPool(layer i ))

[0109] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the spatial self-attention module in step 3, and the implementation of the spatial self-attention module is specifically as follows:

[0110] Step 3.1: Convert the input feature vector to a resolution format, and convert the first and second dimensions to an N×1 text sequence format.

[0111] w=Trans w×H→N×1 (Feature)

[0112] Step 3.2: Position-encode the N×1 feature sequence and add it to the feature vector of the original text sequence.

[0113] P(loc,2i)=sin(loc / 10000 2i / d )

[0114] P(loc,2i+1)=cos(lic / 10000 2i / d )

[0115] X=P+w

[0116] Step 3.3: Use multiple layers of self-attention mechanism to set specific weights, giving higher weights to eigenvalues ​​with greater effects and smaller weights to eigenvalues ​​with smaller effects.

[0117] X = Transformer1(X)

[0118] …

[0119] X = Transformer n (X)

[0120] Step 3.4: Use resolution conversion to convert the N×1 text sequence format into the first and second dimensional resolutions of the input feature vector to ensure the uniformity of input and output resolutions and obtain the feature vector with attention weight adjustment.

[0121] Res=Trans N×1→w×H (X)

[0122] According to the human posture estimation algorithm based on WiFi signal of hybrid attention mechanism, it is characterized by the deep residual convolutional network in step 4, and the implementation of the deep residual convolutional network is specifically as follows:

[0123] Step 4.1: Use multiple continuously stacked convolutional networks to restore features, and use residual connections to link the convolutional network output data and input data.

[0124] X1=conv(X)

[0125] Use the global adaptive pooling layer to control the first and second dimensional resolution of the input data to the specified number of human skeleton joints, and then use a convolution layer with a convolution kernel of 1×1. Through the convolution operation on the channel, the number of channels of the output data is made to be the number of dimensions of the predicted human posture estimation.

[0126] Res = conv 1×1 (AdaptiveAvg(x))

[0127] (x i ,y i )=Res

[0128] The advantages of the present invention over the prior art are: the present invention processes WiFi signals by means of artificial intelligence to obtain indoor human posture information, solving the problems of weak light, obstruction, and invasion of personal privacy in traditional methods, and the price of the signal transceiver is relatively low, which reduces the cost pressure. In addition, the present invention pays more attention to the inherent characteristics of the collected WiFi channel information data through the channel and attention mechanism, so that human posture information can be predicted more accurately.

[0129] Other features and advantages of the present invention will be described in detail in the following detailed description.

[0130] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A WiFi signal human posture estimation algorithm based on hybrid attention mechanism, characterized in that: The method comprises the following steps: Step 1: Input the feature data of the human posture estimation data set based on the CSI signal into the upsampling network, and input the sample number N, carrier number S, signal transmitter antenna number N1, and signal collector antenna number N2 of the feature data as parameters into the input layer of the model; perform data expansion and convolution of the spatial dimension and convolution of the channel dimension on the input feature data in the upsampling layer; Step 2: Input the processed raw data into a stacked multi-scale convolutional network, where each multi-scale convolutional network includes convolutional networks with multiple convolution kernel sizes, and adjacent multi-scale convolutional networks are connected using a downsampling network to extract data features in the spatial dimension; Step 3: Introduce the spatial self-attention module, assign spatial attention weights to the extracted feature vectors, and generate deep feature vectors with human posture information; Step 4: Input the feature vector into the stacked deep residual convolutional network. Each deep residual convolutional network includes multiple continuously stacked convolutional networks, and the input is added to the output of the last convolutional network using a residual connection based on spatial attention. Adjacent deep residual convolutional networks are linked using an upsampling network. The output of the last deep residual convolutional network is connected to the first multi-scale convolutional network using a residual connection operation. Step 5: For the feature vector containing human posture information, use the global adaptive pooling layer and convolutional network to convert the channel and spatial resolution, convert it to the target resolution, and output two-dimensional human posture information.

2. The WiFi signal human posture estimation algorithm based on hybrid attention mechanism according to claim 1 is characterized in that The stacked multi-scale convolutional network in step 2, wherein the multi-scale convolutional network is specifically: Step 2.1: Multi-scale convolutional network, with convolution kernels of multiple resolution sizes, for the same input data (c in , H in , w in ) extracts feature information of different scales and ensures that each intermediate convolutional layer outputs X1, X2, ..., X n The resolution of the convolutional layers is the same, and the padding and stride of each convolutional layer need to be adjusted accordingly. X1=conv1(c in ,H in ,w in ) X2=conv2(c in ,H in ,w in ) ... X n =conv n (c in ,H in ,w in ) Step 2.2: Connect the data output by the intermediate convolutional layer according to the channel dimension, and fuse the features of each scale to obtain the corresponding data. X=cat(X1,X2,...,X n ) Step 2.3: Finally, the convolution layer corv, batch normalization function BacthNorm, and activation function Active are used to perform channel and spatial convolution on the data output by the intermediate convolution layer. (c out ,H out ,w out )=Active(BatchNorm(conv(X))) Step 2.4: Adjacent multi-scale convolutional networks are connected using downsampling networks. The deep network data is the feature extraction of the shallow network data. The downsampling network includes the maximum pooling layer MaxPool and the convolution network conv with a convolution kernel size of 1x1. 1×1 combination. layer i+1 =conv 1×1 (MaxPool(layer i ))。 3. The human posture estimation algorithm based on WiFi signal of hybrid attention mechanism according to claim 1 is characterized in that The spatial self-attention module in step 3 is implemented as follows: Step 3.1: Convert the input feature vector to a resolution format, and convert the first and second dimensions to an N×1 text sequence format. w=Trans w×H→N×1 (Feature) Step 3.2: Position-encode the N×1 feature sequence and add it to the feature vector of the original text sequence. P(loc,2i)=sin(loc / 10000 2i / d ) P(loc,2i+1)=cos(loc / 10000 2i / d ) X=P+w Step 3.3: Use multiple layers of self-attention mechanism to set specific weights, giving higher weights to eigenvalues ​​with greater effects and smaller weights to eigenvalues ​​with smaller effects. X = Transformer1(X) ... X=Transformer n (X) Step 3.4: Use resolution conversion to convert the N×1 text sequence format into the first and second dimensional resolutions of the input feature vector to ensure the uniformity of input and output resolutions and obtain the feature vector with attention weight adjustment. Res=Trans N×1→w×H (X)。 4. The human posture estimation algorithm based on WiFi signal of hybrid attention mechanism according to claim 1 is characterized in that The deep residual convolutional network in step 4 is implemented as follows: Step 4.1: Use multiple continuously stacked convolutional networks to restore features, and use residual connections to link the convolutional network output data and input data. X1=conv(X) X2=conv(X1) ... X n =conv(X n-1 ) Step 4.2: Use the spatial attention mechanism in the residual connection to extract the more important channel features in the input data and add them to the output data of the stacked convolutional network. Res=X+X n Among them, GAP is the global average pooling function, sig is the Sigmoid function that generates a value between 0 and 1. is the dot product function. Step 4.3: Adjacent continuously stacked convolutional networks are connected using an upsampling network. The shallow network data restores the features of the deep network data. The upsampling network includes a bilinear interpolation function. layer i+1 =upsample(layer i )。 5. The human posture estimation algorithm based on WiFi signal of hybrid attention mechanism according to claim 1 is characterized in that The processing output of the human body posture information in step 5 and the implementation of the processing network of the human body posture information are specifically as follows: Use the global adaptive pooling layer to control the first and second dimensional resolution of the input data to the specified number of human skeleton joints, and then use a convolution layer with a convolution kernel of 1×1. Through the convolution operation on the channel, the number of channels of the output data is made to be the number of dimensions of the predicted human posture estimation. Res=conv 1×1 (AdaptiveAvg(X)) (x i ,and i )=Res。

Images