Wi-Fi visual perception method based on Markov transition field

Abstract

This invention discloses a Wi-Fi visualization perception method based on Markov transition fields, primarily addressing the existing issues of poor channel state information (CSI) feature extraction and low recognition accuracy. Its implementation involves: utilizing Markov state transition characteristics to characterize the temporal variations of Wi-Fi signals and transforming CSI data into CSI images; adaptively correcting the CSI images based on a channel attention mechanism to focus on the channel feature information of the CSI images and synthesize RGB three-channel CSI images; dividing the CSI images into training and test sets according to a set ratio; constructing a Visual Transformer (ViT) network model and training it using the training set; and inputting the test set into the trained ViT network model to obtain gesture recognition results. This invention can effectively extract channel features from CSI images, improve the accuracy of indoor human gesture recognition, and can be used for visualizing and recognizing dynamic human gestures indoors.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and in particular relates to a Wi-Fi visualization perception method, which can be used for visualizing and recognizing dynamic human gestures indoors. Background Art

[0002] With the development of intelligent sensing technology, traditional sensing technologies such as visual perception, Bluetooth positioning, and Zigbee have limitations. For example, visual perception technology has high requirements for lighting conditions, shooting conditions, and device performance, and the captured images may threaten the privacy and security of users. In addition, Bluetooth positioning and Zigbee technologies have poor stability and require the deployment of additional hardware, making them difficult to implement in practice.

[0003] During transmission, Wi-Fi signals are subject to reflection, refraction, and scattering due to the movement of people and other objects, causing signal variations. Wi-Fi-based wireless sensing technology utilizes conventional communication equipment to capture the propagation characteristics of Wi-Fi signals in wireless channels. Combined with data analysis algorithms, it enables intelligent sensing applications such as intrusion detection and human behavior recognition. Due to Wi-Fi's wide coverage, low cost, and ease of deployment, Wi-Fi sensing has become an ideal research target in the field of intelligent sensing.

[0004] According to the IEEE802.11 standard, Wi-Fi signals use OFDM systems with multiple orthogonal subcarriers. The channel state information (CSI) describes the path loss, signal attenuation, and other characteristics of Wi-Fi signals along the transmission path. For a single-transmitter-multiple-receiver system, CSI is a dimension of N. Rx ×N c ×N p A three-dimensional complex tensor of , where N Rx is the number of receiving antennas, N c is the number of subcarriers, N p The number of packets is complex, so extracting valid information is necessary for subsequent processing. In recent years, researchers both domestically and internationally have proposed extracting CSI amplitude information while ignoring its phase information. Because CSI amplitude reflects rich channel characteristics, this approach can effectively reduce the dimensionality of the data, facilitating the development of subsequent perception algorithms. Other studies have also suggested that extracting valid data can also be accomplished by preprocessing CSI phase information, such as deconvolution. However, CSI phase noise is significant, resulting in poor stability and versatility in perception algorithms based on CSI phase.

[0005] Existing Wi-Fi sensing algorithms can be primarily categorized into two types: model-based and learning-based. Model-based approaches leverage physical models that describe Wi-Fi signal propagation to explain the underlying mechanisms of Wi-Fi sensing. While this approach achieves high accuracy in simple sensing scenarios, in complex scenarios, physical models struggle to accurately capture the dynamic changes in signals, resulting in poor sensing performance. Learning-based approaches feed large amounts of data samples into machine learning or deep learning models, leveraging the models' learning capabilities to understand the underlying properties of different data samples. This approach utilizes large amounts of data for training, eliminating the need for specialized communications expertise. The trained models exhibit strong scalability and generalization capabilities.

[0006] Learning-based perception methods can be categorized as machine learning-based and deep learning-based. Machine learning-based methods first require extracting statistical features from the data. Common algorithms for feature extraction include principal component analysis (PCA), fast Fourier transform (FFT), and inverse fast Fourier transform (IFFT). For example, in the paper "ActiviTune: A Multi-stage System for Activity Recognition of Passive Entities from AmbientFM-Radio Signals," Shuyu et al. used FFT to extract features and then employed support vector machines (SVM), k-nearest neighbor (KNN), and decision trees (DT) to classify five different activities. However, CSI data is high-dimensional and complex, making it difficult for feature extraction algorithms like PCA and FFT to extract complete feature information related to the time, frequency, and spatial domains. The performance of machine learning-based methods is highly dependent on appropriate feature selection. If the feature dimensionality is insufficient or the feature selection is inappropriate, the model may not learn enough information to make accurate predictions.

[0007] In the field of deep learning, image processing algorithms are relatively mature, with the mainstream approach being based on convolutional neural networks (CNNs), which can effectively extract features from images. For example, Qirong proposed a gesture recognition method based on deep transfer learning in the paper "Deep transfer learning for gesture recognition with WiFi signals." This method expresses the CSI as an image matrix and uses a deep CNN model and a fine-tuned CNN model to extract high-level features from the image, thereby achieving gesture recognition. Although CNN models have been widely used for image feature extraction, they lack flexibility and have a relatively fixed structure. For each pixel in the image, they only focus on the values ​​of its neighboring pixels, making them difficult to process complex and irregular input data, resulting in a decrease in recognition accuracy. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of the above methods and propose a Wi-Fi visual perception method based on Markov transition field to improve the CSI feature extraction effect and enhance the gesture recognition accuracy.

[0009] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0010] (1) Collecting channel state information (CSI) data for different gestures in an experimental environment

[0011] (2) Based on the amplitude of the CSI, the Markov transition field image corresponding to the gesture is generated:

[0012] 2a) Calculate the amplitude of CSI

[0013] 2b) According to the amplitude of CSI Define the state tensor of each subcarrier at different times

[0014] 2c) According to the state tensor Calculate the transition probability tensor

[0015] 2d) Transition probability tensor Computing Markov transition fields Generate CSI image M i:: :

[0016] 2d1) Define the Markov transition field: N p is the number of CSI data packets, N c Indicates the number of subcarriers of the Wi-Fi signal, N m =3×N c ; Among them, m ijk is the state transition probability of the i-th Markov chain at time j and time k, let u = ceil (i / N c ), v = mod (i / N c ), then m ijk Expressed as:

[0017] m ijk =w ij′k′ |j′=s uvj ,k′=s uvk ;

[0018] 2d2) According to the tensor decomposition principle, Horizontal slicing is performed to generate CSI images, namely the Markov transition field Mi:: :

[0019]

[0020] Among them, the value range of j is [1,N p ], the value range of k is [1,N p ];

[0021] (3) Modify the CSI image according to the channel attention mechanism CAM to obtain the training set and test set:

[0022] 3a) The three-dimensional Markov field Perform global maximum pooling and global average pooling of spatial dimensions respectively, and obtain two dimensions of N m ×1×1 feature map;

[0023] 3b) The feature maps are fed into a shared multi-layer perceptron (MLP) to learn and calculate the channel attention weight matrix.

[0024] 3c) According to the weight matrix M c Select the channel with the largest weight to generate three modified Markov transfer fields, corresponding to the R channel, G channel, and B channel of the image, to synthesize the pseudo-color image

[0025] 3d) Divide the pseudo-color image x into a training set and a test set in a ratio of 3:1;

[0026] (4) Construct a ViT network model consisting of a preprocessing module, a vector mapping layer, a coding layer, and a multi-layer perceptron classification layer connected in sequence;

[0027] (5) Input the training set into the ViT network model and train it using back propagation to obtain a trained ViT network model;

[0028] (6) Input the test set into the trained ViT network model to obtain the gesture classification results.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] First, the present invention uses the channel attention mechanism (CAM) to correct the channel state information (CSI) image, which can focus on the channel feature information of the CSI image, thereby achieving accurate extraction of CSI features;

[0031] Secondly, the present invention uses the Markov state transition characteristic to characterize the time-varying characteristics of Wi-Fi signals and transforms CSI data into CSI images, thereby improving the accuracy of gesture recognition.

[0032] The simulation results show that the present invention uses the ViT network after 30 rounds of training to recognize the gestures of the tested person with an accuracy rate of 89.17%. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the overall flow chart for realizing the present invention;

[0034] Figure 2 are CSI image samples corresponding to different gestures in the present invention;

[0035] Figure 3 This is a sub-flow chart for correcting the CSI image in the present invention;

[0036] Figure 4 is a diagram of the ViT network structure constructed in the present invention;

[0037] Figure 5 3 is a comparison chart of the results of using the ViT network model trained by the present invention to identify the training set and the test set respectively. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0039] It should be noted that the step numbers in the specification and claims of the present invention are only for the purpose of clearly describing the embodiments of the present invention and facilitating understanding, and the order of the step numbers is not limited.

[0040] Reference Figure 1 The implementation steps of this example include the following:

[0041] Step 1: Collect channel state information (CSI) data of different gestures in an experimental environment.

[0042] This example uses CSI gesture data from the Widar 3.0 dataset, and the test gestures are: push / pull, sweep, applause, and slide. The specific implementation is as follows:

[0043] 1.1) Place a Wi-Fi transmitter and three receivers in an open area indoors, and mark the transmitter as Tx and the receiver as Rx i ,i=1,2,3;

[0044] 1.2) Use the transmitter and each receiver to form a Wi-Fi channel, generating a total of 3 independent Wi-Fi channels, and set each Wi-Fi signal to contain N c orthogonal subcarriers to form a test environment;

[0045] 1.3) The person being tested stands in the above test environment and performs dynamic gestures such as push / pull, sweep, clap, and slide. Three receivers collect data simultaneously, and a total of N data is collected. p CSI data packets are collected and labeled with the test environment, the person being tested, and the gesture type;

[0046] 1.4) The three receivers contain different spatial dimension information, so the CSI data packets collected by the three receivers are merged to obtain a dimension of 3×N c ×N p Three-dimensional complex CSI data samples

[0047]

[0048] Among them, h ijk =x ijk +jy ijk Represents Rx i The channel state information CSI of the j-th subcarrier at time k, x ijk represents the real part of CSI, y ijk Represents the imaginary part of CSI. The value range of i is {1,2,3}, and the value range of j is [1,N p ], the value range of k is [1,N p ];

[0049] Step 2: Based on the amplitude of the CSI, generate the Markov transition field image corresponding to the gesture.

[0050] Since the amplitude of CSI contains rich channel characteristics, the amplitude of CSI can be calculated instead of the original CSI data to reduce the dimension of the original CSI data.

[0051] 2.1) Calculate the amplitude of CSI, that is, calculate the channel state information CSI data Element h ijk The modulus value Get the amplitude tensor of CSI

[0052]

[0053] 2.2) According to the amplitude of CSI Define the state tensor of each subcarrier at different times

[0054] 2.2.1) Extract amplitude based on tensor decomposition principle The third mode fiber gets vector A ij: :

[0055]

[0056] Among them, vector A ij: Receiver Rx i The amplitude of the jth subcarrier; a ijk Indicates the receiver Rx i The amplitude of the channel state information CSI of the j-th subcarrier at time k;

[0057] Since Wi-Fi signals use OFDM, and different subcarriers are orthogonal, statistics can be collected on different subcarriers at different times. This is achieved by performing the following steps 2.2.2) to 2.2.5):

[0058] 2.2.2) Vector A ij: Element a in ijk Sort by amplitude and divide equally into N s bins, each containing N p / N s elements, the value set of the bin number is:

[0059] 2.2.3) According to the situation that the subcarrier amplitudes in different bins are quite different, vector A is ij: Middle a ijk The serial number of the bin is used as the state s of the subcarrier ijk , to replace the CSI amplitude with discrete states to further reduce the dimension of the data. ijk Rx i Status information of the j-th subcarrier at time k;

[0060] 2.2.4) The state information s of the subcarrier at different times ijk Perform linear combination to get Rx i The state vector Si of the j-th subcarrier j: :

[0061] Si j: =[si j1 ,si j2 ,…,s ijk ,…,s ijNp ] T

[0062] The Si j: Rx iStatus information of the j-th subcarrier at different times;

[0063] 2.2.5) Rx i Chinese N c The state vector Si of the subcarrier j: Combine to get the state tensor of CSI Should Each element in corresponds to CSI data one by one, representing the state of all subcarriers collected by the receiver at each moment;

[0064] 2.3) According to the state tensor Calculate the transition probability tensor

[0065] 2.3.1) Based on the transition probability value [0, 1], define the state transition probability value set of the channel state information CSI

[0066] 2.3.2) Based on the independent characteristics of the state vectors of different subcarriers, the state vector Si in each subcarrier is used j: Calculate the state transition frequency ni of each subcarrier at consecutive moments j'k' :

[0067] n ij'k' =count(s ijk =j',si j(k+1) =k')

[0068] in,

[0069] 2.3.3) Si j: The state transition frequency n i'j'k' The state transition probability w of the state vector is approximated ij'k' , and N s ×N s w ij'k' Combine and get the transition probability matrix W i:: :

[0070]

[0071] 2.3.4) Collect N data from all receivers m =3×N c The transfer probability matrix W of subcarriers i:: Combined into a transition probability tensor:

[0072] 2.4) According to the transition probability tensor Computing Markov transition fields To generate CSI image Mi:: :

[0073] 2.4.1) Define Markov transition field By N m It is composed of a two-dimensional Markov field, which is expressed as:

[0074]

[0075] Where m ijk represents the state transition probability of the i-th Markov chain at time j and time k, and let u = ceil (i / N c ), v = mod (i / N c ), then m ijk Expressed as:

[0076] m ijk =w ij'k' |j'=s uvj ,k'=s uvk ;

[0077] 2.4.2) According to the tensor decomposition principle, Perform horizontal slicing to generate CSI image M i:: :

[0078]

[0079] In this step, different gestures generate corresponding CSI image samples. Figure 2 As shown, where:

[0080] Figure 2 (a) CSI image sample corresponding to push / pull gesture;

[0081] Figure 2 (b) shows the CSI image sample corresponding to the gesture sweep;

[0082] Figure 2 (c) An example of a CSI image corresponding to the applause gesture;

[0083] Figure 2 (d) shows the CSI image sample corresponding to the gesture sliding.

[0084] Step 3: Modify the CSI image according to the channel attention mechanism CAM to obtain the training set and test set.

[0085] Since the CSI image obtained in step 2 has not been subjected to noise reduction processing, the feature information of the image will be submerged by high-frequency noise. Therefore, CAM is used to focus on the channel information of the image to achieve accurate extraction of CSI features.

[0086] Reference Figure 3, the implementation of this step to correct the CSI image is as follows:

[0087] 3.1) The three-dimensional Markov field Perform global maximum pooling and global average pooling of spatial dimensions respectively, and obtain two dimensions of N m ×1×1 feature map;

[0088] 3.2) The feature maps are fed into a shared multi-layer perceptron MLP to learn and calculate the channel attention weight matrix

[0089] 3.2.1) Set the dimensionality reduction coefficient of MLP to r;

[0090] 3.2.2) Set the number of neurons in the first layer of MLP to N m / r, the number of neurons in the second layer is N m , the activation function is the linear rectification function Relu;

[0091] 3.2.3) Input the feature maps obtained by global maximum pooling and global average pooling into the multi-layer perceptron MLP respectively, and obtain two feature maps with dimensions N m ×1×1 output feature map;

[0092] 3.2.4) Set the two dimensions to N m The output feature maps of ×1×1 are added and mapped by the logistic activation function Sigmoid to obtain the channel attention weight matrix It is used for Different channels are assigned different weights. The larger the channel weight, the richer the CSI feature information contained in the channel.

[0093] 3.3) According to the channel attention weight matrix M c , from each receiver Rx i The channel with the largest weight is selected The Markov transfer field in the image is used to obtain three modified Markov transfer fields corresponding to the R channel, G channel, and B channel of the image, and then these three modified Markov transfer fields are synthesized into a pseudo-color image.

[0094] 3.4) Divide the pseudo-color image x into a training set and a test set in a ratio of 3:1.

[0095] Step 4: Build the Visual Transformers ViT network model.

[0096] 4.1) Establish a preprocessing module consisting of an image cropping submodule and a normalization submodule. The image cropping submodule is used to randomly crop the image, and the normalization submodule is used to normalize each channel of the image with a mean of 0.5 and a variance of 0.5.

[0097] 4.2) Establish a vector mapping layer consisting of a convolution layer and a mapping layer. This convolution layer has 768 convolution kernels, a kernel size of 16*16, and a step size of 16, and is used to perform convolution operations on the image. This mapping layer is used to splice the image block information map and the position information map to obtain the image mapping data.

[0098] 4.3) Establish an encoding layer consisting of two residual modules, two normalization modules, two random dropout modules, a multi-head attention module, and a multi-layer perceptron module. The structural relationship is as follows:

[0099] First residual module → first layer normalization module → multi-head attention module → first random deactivation module → second residual module → second layer normalization module → multi-layer perceptron module → second random deactivation module; the normalization module is used to connect the mapping data, the residual module is used to introduce residual connection, the multi-head attention module and multi-layer perceptron module are used to learn mapping data and position information, and the random deactivation module is used to randomly deactivate the mapping data of the image;

[0100] 4.4) Construct a multi-layer perceptron classification layer consisting of a data slicing layer and a fully connected layer. The data slicing layer is used to extract the category information of the image block, and the fully connected layer is used to output the predicted probability of the gesture category.

[0101] 4.5) Connect the preprocessing module, vector mapping layer, encoding layer and multi-layer perceptron classification layer in sequence to obtain the ViT network structure diagram, as shown in Figure 4 shown.

[0102] Step 5: Train the ViT network model to obtain a trained ViT network model.

[0103] 5.1) Set the optimizer to stochastic gradient descent (SGD), set the learning rate to 0.001, the momentum factor to 0.9, and the weight decay to 5e-5.

[0104] 5.2) Input the training set into the ViT network model to obtain the predicted probability distribution function for each data sample, and select the category corresponding to the maximum predicted probability in the predicted probability distribution function as the predicted category;

[0105] 5.3) Use the cross entropy loss function to calculate the loss value between the predicted category and the true category. This loss value describes the difference between the predicted category and the true category.

[0106] 5.4) Input the loss value into the back-propagation algorithm to update the network parameters;

[0107] 5.5) Repeat steps 5.2)-5.4) until 30 rounds of training are completed to obtain the trained ViT network model.

[0108] Step 6: Input the test set into the trained ViT network model to obtain the gesture recognition results.

[0109] The effect of the present invention can be further illustrated by the following simulation experiments:

[0110] 1. Simulation Data

[0111] Set 2250 training samples and 750 test samples.

[0112] The recognition network is the ViT network model trained by the method of the present invention.

[0113] 2. Simulation content

[0114] Under the above conditions, the training samples and test samples are input into the trained ViT network model respectively to obtain their respective gesture recognition accuracies. The results are as follows: Figure 5 shown.

[0115] from Figure 5 As can be seen, the trained ViT model achieved an accuracy of 92.71% in the training set and 89.17% in the test set. This 3.54% difference is due to the fact that the trained ViT model is trained on the training set, and its network parameters gradually adjust to the training set data. Therefore, when the model is tested on the training set, its recognition accuracy is higher; however, for the unlearned test set data, the recognition accuracy is slightly lower than that of the training set. Simulation results show that the present invention achieves high gesture recognition accuracy for both learned and unlearned data.

Claims

1. A Wi-Fi visualization perception method based on Markov transition field, characterized in that: The steps include: (1) Collecting channel state information (CSI) data for different gestures in an experimental environment (2) Based on the amplitude of the CSI, the Markov transition field image corresponding to the gesture is generated: 2a) Calculate the amplitude of CSI 2b) According to the amplitude of CSI Define the state tensor of each subcarrier at different times 2c) According to the state tensor Calculate the transition probability tensor 2d) According to the transition probability tensor Computing Markov transition fields Generate CSI image M i:: : 2d1) Define the Markov transition field: N p is the number of CSI data packets, N c Indicates the number of subcarriers of the Wi-Fi signal, N m =3×N c ; Among them, m ijk is the state transition probability of the i-th Markov chain at time j and time k, let u = ceil (i / N c ), v = mod (i / N c ), then m ijk Expressed as: m ijk =w ij'k' |j'=s uvj ,k'=s uvk ; 2d2) According to the tensor decomposition principle, Perform horizontal slicing to generate CSI image M i:: : Among them, the value range of j is [1,N p ], the value range of k is [1,N p ]; (3) Modify the CSI image according to the channel attention mechanism CAM to obtain the training set and test set: 3a) The three-dimensional Markov field Perform global maximum pooling and global average pooling of spatial dimensions respectively, and obtain two dimensions of N m ×1×1 feature map; 3b) The feature maps are fed into a shared multi-layer perceptron (MLP) to learn and calculate the channel attention weight matrix. 3c) According to the weight matrix M c Select the channel with the largest weight to generate three modified Markov transfer fields, corresponding to the R channel, G channel, and B channel of the image, to synthesize the pseudo-color image 3d) Divide the pseudo-color image x into a training set and a test set in a ratio of 3:1; (4) Construct a ViT network model consisting of a preprocessing module, a vector mapping layer, a coding layer, and a multi-layer perceptron classification layer connected in sequence; (5) Input the training set into the ViT network model and train it using back propagation to obtain a trained ViT network model; (6) Input the test set into the trained ViT network model to obtain the gesture classification results.

2. The method according to claim 1, characterized in that Step (1) Collect channel state information (CSI) data of different gestures in an experimental environment The implementation is as follows: 1a) Place a Wi-Fi transmitter and three receivers in an open area indoors, and mark the transmitter as Tx and the receiver as Rx. i ,i=1,2,3; 1b) Use the transmitter and each receiver to form a Wi-Fi channel, and set each Wi-Fi signal to contain N c orthogonal subcarriers to form a test environment; 1c) The tester performs dynamic gestures in the test environment, and three receivers simultaneously collect N p CSI data packets; 1d) Combine the data packets collected by the three receivers to obtain the CSI data samples corresponding to each gesture Among them, h ijk =x ijk +jy ijk Represents Rx i The channel state information CSI of the j-th subcarrier at time k, x ijk represents the real part of CSI, y ijk Represents the imaginary part of CSI.

3. The method according to claim 1, characterized in that Step (2a) calculates the amplitude of CSI, which is used to calculate the channel state information CSI data. Element h ijk The modulus value Get the amplitude tensor of CSI Among them, x ijk represents the real part of CSI, y ijk Represents the imaginary part of CSI.

4. The method according to claim 1, wherein Step (2b) is based on the amplitude of CSI Define the state tensor of each subcarrier at different times The implementation is as follows: 2b1) Extract the amplitude based on the tensor decomposition principle The third mode fiber gets Rx i The amplitude vector A of the jth subcarrier ij: : Among them, a ijk Indicates the receiver Rx i The amplitude of the channel state information CSI of the j-th subcarrier at time k; 2b2) Vector A ij: Element a in ijk Sort by amplitude and divide equally into N s bins, each containing N p / N s elements, the value set of the bin number is 2b3) Vector A ij: Middle a ijk The bin number is the state of the subcarrier s ijk ; 2b4) s ijk Linear combination, get Rx i The state vector S of the j-th subcarrier ij: : 2b5) Rx i N c The state vector S of the subcarrier ij : Combined into state tensor 5. The method according to claim 1, wherein Step (2c) is based on the state tensor Calculate the transition probability tensor The implementation is as follows: 2c1) Define the state transition probability value set of channel state information CSI 2c2) According to S ij: Calculate the state transition frequency of each subcarrier at consecutive moments; 2c3) Approximate the state transition frequency to the state transition probability and obtain the receiver Rx i The transition probability matrix W of the j-th subcarrier i:: : 2c4) N m The transition probability matrix W i:: Combined into a transition probability tensor 6. The method according to claim 1, characterized in that In step 3a), the three-dimensional Markov field Perform global maximum pooling and global average pooling in the spatial dimension respectively, and implement them as follows: 3a1) According to the tensor decomposition principle, the three-dimensional Markov field Perform horizontal slicing to generate N m Channel state information CSI image M i:: , that is, the two-dimensional Markov transition field M i:: ; 3a2) From each M i:: Select the largest numerical output and get the dimension N m ×1×1 feature map; 3a3) From each M i:: Solve the mean output of the elements and get the dimension N m ×1×1 feature map.

7. The method according to claim 1, characterized in that In step 3b), the feature maps are fed into a shared multi-layer perceptron MLP to learn and calculate the channel attention weight matrix M c , implemented as follows: 3b1) Set the dimensionality reduction coefficient to r; 3b2) Set the number of neurons in the first layer of the multilayer perceptron MLP to N m / r, the number of neurons in the second layer is N m , the activation function is the linear rectification function Relu; 3b3) Input the feature maps obtained by global maximum pooling and global average pooling into the multi-layer perceptron MLP respectively, and obtain two feature maps of dimension N m ×1×1 output feature map; 3b4) Set the two dimensions to N m The output feature maps of ×1×1 are added and mapped by the logistic activation function Sigmoid to obtain the channel attention weight matrix 8. The method according to claim 1, characterized in that In step 3c), according to the weight matrix M c The channel with the largest weight is selected to generate three modified Markov transfer fields, which are based on the channel attention weight matrix M c , from each receiver Rx i Select the channel with the maximum weight The Markov transfer field in is used to obtain three modified Markov transfer fields corresponding to the R channel, G channel, and B channel of the image respectively.

9. The method according to claim 1, characterized in that The structural parameters of each layer of the ViT network model constructed in step (4) are as follows: The preprocessing module is composed of an image cropping submodule and a normalization submodule; the image cropping submodule is used to randomly crop the image; the normalization submodule is used to perform a normalization operation on each channel of the image with a mean of 0.5 and a variance of 0.5; The vector mapping layer is composed of a convolution layer and a mapping layer. The convolution layer has 768 convolution kernels, a kernel size of 16*16, and a step size of 16, and is used to perform convolution operations on the image. The mapping layer is used to splice the image block information map and the position information map to obtain the image mapping data. The encoding layer is composed of a first residual module, a first layer normalization module, a multi-head attention module, a first random deactivation module, a second residual module, a second layer normalization module, a multi-layer perceptron module, and a second random deactivation module, which are cascaded in sequence; the normalization module is used to connect the mapping data; the residual module is used to introduce residual connection; the multi-head attention module and the multi-layer perceptron module are used to learn the mapping data and position information; the random deactivation module is used to randomly deactivate the mapping data of the image; The multi-layer perceptron classification layer is composed of a data slicing layer and a fully connected layer; the data slicing layer is used to extract category information of image blocks; and the fully connected layer is used to output the predicted probability of the gesture category.

10. The method according to claim 1, characterized in that In step (5), the training set is input into the ViT network model and trained using backpropagation as follows: 5a) Set the optimizer to stochastic gradient descent (SGD), with a learning rate of 0.001, a momentum factor of 0.9, and a weight decay of 5e-5. 5b) Input the training set into the ViT network model and select the category corresponding to the maximum prediction probability as the predicted category; 5c) Use the cross entropy loss function to calculate the loss between the predicted category and the true category; 5d) Input the loss value into the back-propagation algorithm to update the network parameters; 5e) Repeat steps 5b)-5d) until 30 rounds of training are completed to obtain the trained ViT network model.

Images