A method for multi-person 3D posture estimation based on wireless signals

Abstract

A multi-person three-dimensional posture estimation method based on wireless signals comprises the following steps: step 1, using a WiFi device to collect channel state information CSI signals, and using a camera to shoot corresponding videos for supervision; step 2, processing the video through AlphaPose, outputting a human target frame and human key point processing to generate a heat map and a target frame diagram; step 3, preprocessing the CSI signal; step 4, matching the preprocessed CSI signal with a video frame, and inputting five CSI data into a CSI-2D network for training; step 5, processing the CSI signal in the heat map and target frame diagram output by the CSI-2D model, and regressing the two-dimensional posture of each human body; step 6, each human body two-dimensional posture constitutes a video frame input into a 2D-3D model and combined with two-dimensional coordinates to generate a multi-person three-dimensional posture. The present invention has low cost and high precision.

Description

Technical Field

[0001] The invention belongs to the technical field of wireless communications and relates to a method for multi-person three-dimensional posture estimation based on wireless channel state information. Background Art

[0002] In terms of human posture estimation, camera-based posture estimation methods, including two-dimensional and even three-dimensional posture estimation, are already quite mature. However, traditional sensor technology such as cameras is obviously limited by lighting, occlusion and background, and there are privacy issues.

[0003] There is great potential for sensing the human body based on wireless signals. In recent years, wireless sensing has also achieved some results in human perception, including behavior recognition, breathing detection, target positioning, crowd counting, etc. There are two basic methods of human sensing based on wireless signals: one is device-based, requiring people to wear or carry devices / sensors; the other is device-free, which uses sensing elements located in the environment to monitor human actions without requiring humans to carry any devices or sensors. Although device-based methods are usually accurate, they are not practical or convenient in many important real-life scenarios, such as requiring the elderly or people with dementia to carry devices at all times. Device-free human sensing provides obvious advantages for these scenarios.

[0004] Many research results have shown that wireless signals can perform two-dimensional and three-dimensional pose estimation tasks well under the supervision of video models. Zhao et al. proposed RF-Pose, a deep learning architecture for human pose estimation based on a codec. Its input signal is an antenna array that transmits a continuous frequency modulated wave (FMCW). Although this signal can obtain the distance between the object and the signal compared to commercial WiFi, it is expensive and cannot be popularized. Wang et al. first used CSI signals collected by commercial WiFi to estimate human poses and obtained human masks, two-dimensional joint points, and joint effective area fractions (PAFs). Guo et al. also used deep learning to obtain human skeletons from multi-directional CSI signals. The previous methods only obtain two-dimensional poses. Jiang et al. proposed a method to obtain three-dimensional poses from CSI signals. By using the VICON system (a pose capture camera) as a marker to train the CSI model, the direct end-to-end learning of the three-dimensional pose model has a high learning pressure and low learning accuracy, and is still limited to the pose movement in place.

[0005] In summary, the existing problems of the current technology are as follows: Human body posture estimation based on CSI mostly stays in two dimensions, and three-dimensional posture estimation is only performed in a stationary state and is limited to a single person, and it cannot achieve good accuracy. In addition, the interference of CSI signals on the environment is relatively large, and solving environmental problems is also a difficulty. The difficulty of solving the above problems: how to train a fine-grained three-dimensional human body posture model from CSI data; how to achieve posture estimation in the case of multiple people; how to eliminate the influence of environmental noise. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides a method for multi-person three-dimensional posture estimation based on wireless signals with low cost and high accuracy.

[0007] The technical solution adopted by the present invention to solve the technical problem is:

[0008] A method for multi-person three-dimensional posture estimation based on wireless signals comprises the following steps:

[0009] Step 1: Use the WiFi device to collect the channel state information CSI signal, and use the camera to shoot the corresponding video for supervision;

[0010] Step 2: Process the video with AlphaPose, output the human target frame and human key points to generate a heat map and target frame map as annotations for wireless signal training;

[0011] Step 3: preprocess the collected signals, including eliminating the phase offset between the two antennas, removing abnormal points and environmental noise of the CSI signal, and removing the static DC component;

[0012] Step 4: Match the preprocessed CSI signal with the video frame, and input five CSI data into the CSI-2D network as one segment for training;

[0013] Step 5: Process the CSI signal in the heat map and target frame diagram output by the CSI-2D model to regress the two-dimensional posture of each human body;

[0014] Step 6: Each video frame composed of two-dimensional posture of the human body is input into the 2D-3D model and combined with the two-dimensional coordinates to generate three-dimensional postures of multiple people.

[0015] Further, the process of step one is as follows:

[0016] 1.1 Arrange transceiver equipment and cameras in a conference room environment, use two laptops equipped with Intel5300 network cards to conduct data transceiver experiments, use 6 directional antennas, divide 3 antennas into a group, and the antenna distance between the same group of antennas is 20cm, forming a device similar to a WiFi router, one group as a transmitter (T), and the other group as a receiver (R); use the WiFi device to transmit at a frequency of 100Hz, 30 subcarrier signals of different frequencies can obtain the signal attenuation and phase change of signals of different frequencies to understand the information of different scales of the propagation path, and the receiving end receives 3×3×30 channel state information CSI signals that have been reflected and penetrated by the target. ;

[0017] 1.2 Use the camera to shoot the corresponding video. The process is as follows: use a monocular RGB camera to record 20FPS video frames. Four volunteers participate in data collection. Data of multiple people and single person are collected respectively. The actions include waving, clapping, walking, kicking, squatting, jumping, punching and shaking hands. The timestamp is saved to correspond to the CSI signal part.

[0018] Furthermore, in step 2, the process of generating a heat map and a target frame map from the AlphaPose output is as follows: the video frame is input into the AlphaPose model to generate 17 human key points and human target frame position coordinates, the generated key point coordinates are processed by Gaussian blur to generate a heat map tensor; the generated target frame coordinates are multi-scale transformed, and 4 scale target frames are used to place these multi-scale transformed target frame coordinates on multiple maps to generate tensors. The two tensors will be used as annotations for the CSI-2D model to supervise its learning.

[0019] Furthermore, in step three, the data preprocessing process of the collected data is as follows;

[0020] 3.1 To eliminate the phase offset between the two antennas, the conjugate multiplication method is used to eliminate the phase offset:

[0021]

[0022] Where H1(f,t) is the channel state information of antenna 1, is the conjugate of the channel state information of antenna 2, H 1,S (f) and H 2,S (f) are the static path parts, K and L are the multipath numbers, α l (f,t) is the amplitude attenuation function on the l path, is the Doppler frequency shift function;

[0023] 3.2 Eliminate environmental noise;

[0024] First, the Hample outlier filter is used to remove the most obvious outliers in the original signal. After the outliers are removed, a high-performance filter is used to filter out the high-frequency noise. The nonlinear wavelet transform soft threshold method is used.

[0025]

[0026] When the absolute value of the wavelet coefficient w is greater than or equal to a given threshold thr, the absolute value of the wavelet coefficient w is subtracted from the threshold and multiplied by the sgn function, and when the wavelet coefficient w is less than the given threshold thr, the wavelet coefficient is set to 0.

[0027] The threshold selection criterion uses the heuristic threshold principle, that is, when the signal-to-noise ratio is very small, the unbiased likelihood estimation principle is used for denoising, and when the signal-to-noise ratio is large, the fixed threshold method is used for denoising;

[0028] 3.3 Remove static DC components;

[0029] The static DC component is very large, but does not contain the required human motion information. The signal mean is subtracted to eliminate the static variable.

[0030] In the step 4, the preprocessed CSI signal is matched with the video frame, and the process is: the video frame is 20FPS, the CSI signal transmission frequency is 100Hz, and the timestamp is recorded when the video and CSI signal are collected, and the video and CSI signal are matched.

[0031] In step 5, the CSI-2D model training process is as follows:

[0032] Use a learning rate of 0.06, train for 40 rounds, divide the dataset into training set: validation set = 2:8, use the Adam optimizer as the optimizer, and set the offset to 0.9;

[0033] The training goal of the CSI-2D model is to reduce the difference between the key point heat map predicted by the vision-based network model and the calculated loss L is composed of multiple losses:

[0034] L=λ1L box +λ2L JHMs ;

[0035] Among them, the two-class loss is used to calculate the window loss L box , use L2 loss (square loss) multiplied by the element weight to calculate the joint position error L JHMs , and set the window loss weight coefficient λ1 to 0.1, and the joint position error loss weight coefficient λ2 to 1;

[0036] L JHMs The calculation formula is as follows:

[0037]

[0038] In the formula That is the MSE loss formula, It represents the video label data matrix. represents the matrix generated by the model, For element weights, the attention mechanism is implemented by using Matthew weights;

[0039] Where Π(·) is The function value is +1 when The function value is -1;

[0040] The input CSI tensor is first upsampled, then passed through a residual block and U-Nets, and the output is passed through a pooling layer. The box and JHMs are output in the same network model, so the output after pooling passes through two convolutional layers and a BN layer to output the representation tensor. The output multi-scale box map and heat map are processed to extract the two-dimensional key points of each person. The operation is as follows:

[0041] The human target boxes segmented at each scale in the multi-scale box map are evaluated, and the best one is evaluated as the final target detection box for each human body. Then, the heat map and the single human target box map are multiplied to extract the heat map of a single human body, and finally the key point coordinates are regressed using the argmax function:

[0042]

[0043] Among them, j represents the number of key points, p represents the number of people, It means the coordinate position of the pth person at the jth key point is regressed. represents the heat map of the jth key point, Represents the target detection block diagram of the p-th person.

[0044] In step six, the process of generating multi-person 3D coordinates from multi-person 2D coordinates is as follows: normalizing the single-person 2D coordinate sequence, inputting it into the 2D-3D model, generating single-person 3D coordinates, combining the single-person 2D coordinates and the single-person 3D coordinates, and finally generating a multi-person 3D human skeleton in the same 3D coordinate system.

[0045] The present invention proposes a new method for multi-person 3D posture estimation using CSI signals. The signal is subjected to data preprocessing, and wireless network 2D posture and human target frame joint training is performed under the supervision of a video 2D posture estimation deep learning model. The output 2D posture is combined with the video frame through a pre-trained 3D posture model to output a movable 3D posture skeleton.

[0046] The use of WiFi signals to estimate human posture will be well applied in the market, including activity detection and life assistance for patients or the elderly with limited mobility. The use of WiFi signals can avoid some privacy issues; the human body can be applied to games, animations, and AR in a cluttered environment; in addition, the human body can be detected in a dim environment. It can meet the requirements of low cost and high precision.

[0047] The beneficial effects of the present invention are mainly manifested in:

[0048] 1. Use commercial WiFi to achieve 2D and 3D multi-person pose estimation. This method is low-cost, can be achieved with only WiFi equipment, and has high accuracy.

[0049] 2. The present invention uses a multi-scale target frame to solve the technical problem that the key points of one person occupy another space when multiple people interact, thereby improving the accuracy of multiple people interacting during the multi-target segmentation process;

[0050] 3. In terms of generating 3D postures, the present invention combines the 3D part with the 2D joint points, positions and video frames to smooth the actions, thereby improving the accuracy of the model and reducing the pressure of directly performing end-to-end learning models;

[0051] 4. The present invention abandons the key point affinity fields (PAFs) in two-dimensional posture recognition, and instead improves the real-time performance of the model by combining the human target frame; the present invention collects a data set including 8 actions: waving, clapping, walking, kicking, shaking hands, squatting, jumping, and punching, with four volunteers participating, and is divided into single-person and multi-person video and CSI data. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a method for three-dimensional posture estimation based on channel state information provided by an embodiment of the present invention;

[0053] Figure 2 is a flow chart of preprocessing of an original CSI signal provided by an embodiment of the present invention;

[0054] Figure 3 This is the network structure diagram of the CSI-2D module. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with the accompanying drawings.

[0056] Reference Figure 1 to Figure 3 , a method for multi-person three-dimensional posture estimation based on wireless signals, comprising the following steps:

[0057] Step 1: Use the WiFi device to collect the channel state information CSI signal, and use the camera to shoot the corresponding video for supervision;

[0058] Step 2: Process the video with AlphaPose, output the human target frame and human key points to generate a heat map and target frame map as annotations for wireless signal training;

[0059] Step 3: preprocess the collected signals, including eliminating the phase offset between the two antennas, removing abnormal points and environmental noise of the CSI signal, and removing the static DC component;

[0060] Step 4: Match the preprocessed CSI signal with the video frame, and input five CSI data into the CSI-2D network as one segment for training;

[0061] Step 5: Process the CSI signal in the heat map and target frame diagram output by the CSI-2D model to regress the two-dimensional posture of each human body;

[0062] Step 6: Each video frame composed of two-dimensional posture of the human body is input into the 2D-3D model and combined with the two-dimensional coordinates to generate three-dimensional postures of multiple people.

[0063] like Figure 1 As shown, the method for multi-person three-dimensional posture estimation based on wireless channel state information provided by the embodiment of the present invention includes the following steps

[0064] Step 1: Use the WiFi device to collect the channel state information CSI signal. The process is as follows:

[0065] The WiFi device uses a transmission frequency of 100 Hz and 30 subcarrier signals of different frequencies to obtain the signal attenuation and phase change of signals of different frequencies to understand the information of different scales of the propagation path. The receiving end receives 3×3×30 channel state information (CSI) signals that are reflected and penetrated by the target.

[0066] Use the camera to shoot the corresponding video. The process is:

[0067] A monocular RGB camera was used to record 20FPS video frames. Four volunteers participated in data collection, including multi-person and single-person data. The actions performed included waving, clapping, walking, kicking, squatting, jumping, punching, shaking hands, etc. The timestamps were saved to correspond to the CSI signal part.

[0068] Step 2: Generate a heat map and target frame map from the AlphaPose output. The process is as follows:

[0069] The video frame is input into the AlphaPose model to generate 17 human key points and human target frame position coordinates. The generated key point coordinates are processed by Gaussian blur to generate a heat map tensor; the generated target frame coordinates are transformed by multi-scale. The present invention uses 4-scale target frames. These multi-scale transformed target frame coordinates are placed on multiple maps to generate sheets. The two tensors will be used as annotations for the CSI-2D model to supervise its learning.

[0070] Step three, preprocess the collected data, the process is as follows;

[0071] 3.1 To eliminate the phase offset between the two antennas, the conjugate multiplication method is used to eliminate the phase offset:

[0072]

[0073] Where H1(f,t) is the channel state information of antenna 1, is the conjugate of the channel state information of antenna 2, H 1,S (f) and H 2,S (f) are the static path parts, K and L are the multipath numbers, α l (f,t) is the amplitude attenuation function on the l path, is the Doppler frequency shift function;

[0074] 3.2 Eliminate environmental noise;

[0075] First, use the Hample outlier filter to remove the most obvious outliers in the original signal. After the outliers are removed, use a high-performance filter to filter out high-frequency noise. Use the soft threshold method based on nonlinear wavelet transform.

[0076]

[0077] Among them, when the absolute value of the wavelet coefficient w is greater than or equal to the given threshold thr, the absolute value of the wavelet coefficient w is subtracted from the threshold and multiplied by the sgn function, and when the wavelet coefficient w is less than the given threshold thr, the wavelet coefficient is set to 0;

[0078] The threshold selection criterion uses the heuristic threshold principle, that is, when the signal-to-noise ratio is very small, the unbiased likelihood estimation principle is used for denoising, and when the signal-to-noise ratio is large, the fixed threshold method is used for denoising;

[0079] 3.3 Remove static DC components;

[0080] The static DC component is very large, but does not contain the required human motion information. The signal mean can be subtracted to eliminate the static variable.

[0081] Step 4: Match the pre-processed CSI signal with the video frame. The process is as follows:

[0082] The video frame is 20FPS, and the CSI signal transmission frequency is 100Hz. The timestamps are recorded when the video and CSI signals are collected, and the video and CSI signals are matched;

[0083] Step 5: CSI-2D model training process. The process is as follows:

[0084] Use a learning rate of 0.06, train for 40 rounds, divide the dataset into training set: validation set = 2:8, use the Adam optimizer as the optimizer, and set the offset to 0.9;

[0085] The training goal of the CSI-2D model is to reduce the difference between the key point heat map predicted by the vision-based network model and the calculated loss L is composed of multiple losses:

[0086] L=λ1L box +λ2L JHMs ;

[0087] Among them, the two-class loss is used to calculate the window loss L box , use L2 loss (square loss) multiplied by the element weight to calculate the joint position error L JHMs , and set the window loss weight coefficient λ1 to 0.1, and the joint position error loss weight coefficient λ2 to 1;

[0088] L JHMs The calculation formula is as follows:

[0089]

[0090] In the formula

[0091] In the formula That is the MSE loss formula, It represents the video label data matrix. represents the matrix generated by the model, For element weights, the attention mechanism is implemented by using Matthew weights;

[0092]

[0093] Where Π(·) is The function value is +1 when The value of the function is -1.

[0094] The input CSI tensor is first upsampled, then passed through a residual block and U-Nets, and the output is passed through a pooling layer. The box and JHMs are output in the same network model, so the output after pooling passes through two convolutional layers and a BN layer to output the tensor. The output multi-scale box map and heat map are processed to extract the two-dimensional key points of each person. The operation is as follows:

[0095] The human target boxes segmented at each scale in the multi-scale box map are evaluated, and the best one is evaluated as the final target detection box for each human body. Then, the heat map and the single human target box map are multiplied to extract the heat map of a single human body, and finally the key point coordinates are regressed using the argmax function:

[0096]

[0097] Among them, j represents the number of key points, p represents the number of people, It means the coordinate position of the pth person at the jth key point is regressed. Represents the heat map of the jth key point, Represents the target detection frame diagram of the pth person;

[0098] Step 6: Generate multi-person 3D coordinates from multi-person 2D coordinates. The process is as follows:

[0099] The single-person 2D coordinate sequence is normalized and input into the 2D-3D model to generate a single-person 3D coordinate. The single-person 2D coordinate and the single-person 3D coordinate are combined to finally generate a multi-person 3D human skeleton in the same 3D coordinate system.

[0100] The experimental results of the present invention are described in detail below in conjunction with the experiments:

[0101] 1. Experimental environment and data collection: The transceiver equipment and camera were arranged in a 10m*15m conference room environment. Two laptops equipped with Intel5300 network cards were used for data transceiver experiments. Six directional antennas were used. Three antennas were divided into a group to form a device similar to a WiFi router. One group was used as a transmitter (T) and the other group was used as a receiver (R). The distance between the transceiver antennas was 6m. The transmission signal was recorded with an orthogonal frequency division multiplexing signal (OFDM) WiFi signal centered at 5GHz that complies with the 802.11n WiFi communication standard. There are 30 subcarriers and the transmission frequency is 100Hz. The 30 subcarrier signals of different frequencies can obtain the signal attenuation and phase change of signals of different frequencies to understand the information of different scales of the propagation path. The receiving end receives the 3×3×30 channel state information (CSI) signal that is reflected and penetrated by the target. While collecting the CSI signal, a monocular RGB camera is used at the receiving end to record 20FPS video frames.

[0102] Four volunteers participated in data collection. The volunteers, both multiple and single, performed actions including waving, clapping, walking, kicking, squatting, jumping, punching, shaking hands, etc. between the transmitting and receiving antennas, and saved timestamps so that the video frame time corresponds to the CSI signal collection time.

[0103] 2. Data processing: The *.dat file obtained by the CSI acquisition tool is preprocessed in the MATLAB data processing tool, including: eliminating the phase offset between the two antennas; using the Hample outlier filter to remove the most obvious outliers of the original signal; removing noise based on the nonlinear wavelet transform soft threshold method; and removing the static DC component.

[0104] 3. Model training: The collected CSI data was made into a dataset, and the dataset was divided into training set: validation set = 2:8. The Adam optimizer with β1 = 0.9, β2 = 0.999 was used in the training. k = 1, b = 1 was used in calculating the MW weights. The networks were trained for 40 rounds.

[0105] In model evaluation, the present invention uses two-dimensional and three-dimensional joint points generated by a video model as true values ​​to evaluate the entire model.

[0106] Table 1 shows the error (P-MPJPE) results of each key point of this model on the multi-person dataset after alignment with the true value in translation, rotation and scale. The lower the P-MPJPE, the better;

[0107]

[0108] Table 1

[0109] The sequence of joint points represents: 1: middle hip; 2: left hip; 3: left knee; 4: left ankle; 5: right hip; 6: right knee; 7: right ankle; 8: spine; 9: chest; 10: neck; 11: head; 12: left shoulder; 13: left elbow; 14: left wrist; 15: right shoulder; 16: right elbow; 17: right wrist.

[0110] It can be seen from Table 1 that the overall P-MPJPE of the method of the present invention can reach 30.4 mm, which is 7.5 mm higher than 37.9 mm of WiPose.

[0111] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A method for multi-person three-dimensional posture estimation based on wireless signals, characterized in that: The method comprises the following steps: Step 1: Use the WiFi device to collect the channel state information CSI signal, and use the camera to shoot the corresponding video for supervision; Step 2: Process the video with AlphaPose, output the human target frame and human key points to generate a heat map and target frame map as annotations for wireless signal training; Step 3: preprocess the collected signals, including eliminating the phase offset between the two antennas, removing abnormal points and environmental noise of the CSI signal, and removing the static DC component; Step 4: Match the preprocessed CSI signal with the video frame, and input five CSI data into the CSI-2D network as one segment for training; Step 5: Process the CSI signal in the heat map and target frame diagram output by the CSI-2D model to regress the two-dimensional posture of each human body; Step 6: Each human body 2D posture constitutes a video frame, which is input into the 2D-3D model and combined with the 2D coordinates to generate a multi-person 3D posture; In the step 2, the process of generating a heat map and a target frame map from the AlphaPose output is as follows: the video frame is input into the AlphaPose model to generate 17 human key points and human target frame position coordinates, and the generated key point coordinates are subjected to Gaussian blur processing to generate a heat map tensor; the generated target frame coordinates are subjected to multi-scale transformation, and 4-scale target frames are used to place these multi-scale transformed target frame coordinates on multiple maps to generate tensors, and the two tensors will be used as annotations of the CSI-2D model to supervise its learning; In the step 3, the data preprocessing process of the collected data is as follows: 3.1 To eliminate the phase offset between the two antennas, the conjugate multiplication method is used to eliminate the phase offset: Where H1(f,t) is the channel state information of antenna 1, is the conjugate of the channel state information of antenna 2, H 1,S (f) and H 2,S (f) are the static path parts, K and L are the multipath numbers, α l (f,t) is the amplitude attenuation function on the l path, is the Doppler frequency shift function; 3.2 Eliminate environmental noise; First, the Hample outlier filter is used to remove the most obvious outliers in the original signal. After the outliers are removed, a high-performance nonlinear wavelet transform soft threshold filter is used to filter out high-frequency noise. Among them, when the absolute value of the wavelet coefficient w is greater than or equal to the given threshold thr, the absolute value of the wavelet coefficient w is subtracted from the threshold and multiplied by the sgn function, and when the wavelet coefficient w is less than the given threshold thr, the wavelet coefficient is set to 0; The selection criteria use the heuristic threshold principle, that is, when the signal-to-noise ratio is very small, the unbiased likelihood estimation principle is used for denoising, and when the signal-to-noise ratio is large, the fixed threshold method is used for denoising; 3.3 Remove static DC components; The static DC component is very large, but does not contain the required human motion information. The signal mean is subtracted to eliminate the static variable. In step 5, the CSI-2D model training process is as follows: The learning rate was 0.06, and the training was performed for 40 rounds. The dataset was divided into a training set: validation set = 2:

8. The optimizer used was the Adam optimizer, and the offset was set to 0.

9. The training goal of the CSI-2D model is to reduce the difference between the key point heat map predicted by the vision-based network model and the calculated loss L is composed of multiple losses: L=λ1L box +λ2L JHMs ; Among them, the two-class loss is used to calculate the window loss L box , use L2 loss multiplied by element weight to calculate the joint position error L JHMs , and set the window loss weight coefficient λ1 to 0.1, and the joint position error loss weight coefficient λ2 to 1; L JHMs The calculation formula is as follows: In the formula That is the MSE loss formula, It represents the video label data matrix. represents the matrix generated by the model, For element weights, the attention mechanism is implemented by using Matthew weights; Where Π(·) is The function value is +1 when The function value is -1; The input CSI tensor is first upsampled, then passed through a residual block and U-Nets, and the output is passed through a pooling layer. The box and JHMs are output in the same network model, so the output after pooling passes through two convolutional layers and a BN layer to output the representation tensor. The output multi-scale box map and heat map are processed to extract the two-dimensional key points of each person. The operation is as follows: The human target boxes segmented at each scale in the multi-scale box map are evaluated, and the best one is evaluated as the final target detection box for each human body. Then the heat map and the single human target box map are multiplied to extract the heat map of a single human body, and finally the key point coordinates are regressed using the argmax function: Among them, j represents the number of key points, p represents the number of people, It means the coordinate position of the pth person at the jth key point is regressed. Represents the heat map of the jth key point, Represents the target detection block diagram of the p-th person.

2. A method for multi-person three-dimensional posture estimation based on wireless signals as claimed in claim 1, characterized in that: The process of step one is as follows: 1.1 Arrange transceiver equipment and cameras in a conference room environment, use two laptops equipped with Intel5300 network cards to conduct data transmission and reception experiments, use 6 directional antennas, divide 3 antennas into a group, the antenna distance between the same group of antennas is 20cm, form a device similar to a WiFi router, one group is used as a transmitter (T), and the other group is used as a receiver (R); the WiFi device uses a transmission frequency of 100Hz, and 30 subcarrier signals of different frequencies can obtain the signal attenuation and phase change of signals of different frequencies to understand the information of different scales of the propagation path. The receiving end receives 3×3×30 channel state information CSI signals that are reflected and penetrated by the target; 1.2 Use the camera to shoot the corresponding video. The process is as follows: use a monocular RGB camera to record 20FPS video frames. Four volunteers participate in data collection. Data of multiple people and single person are collected respectively. The actions include waving, clapping, walking, kicking, squatting, jumping, punching and shaking hands. The timestamp is saved to correspond to the CSI signal part.

3. A method for multi-person three-dimensional posture estimation based on wireless signals as claimed in claim 1 or 2, characterized in that: In the step 4, the preprocessed CSI signal is matched with the video frame, and the process is: the video frame is 20FPS, the CSI signal transmission frequency is 100Hz, and the timestamp is recorded when the video and CSI signal are collected, and the video and CSI signal are matched.

4. A method for multi-person three-dimensional posture estimation based on wireless signals as claimed in claim 1 or 2, characterized in that: In step six, the process of generating multi-person 3D coordinates from multi-person 2D coordinates is as follows: normalizing the single-person 2D coordinate sequence, inputting it into the 2D-3D model, generating single-person 3D coordinates, combining the single-person 2D coordinates and the single-person 3D coordinates, and finally generating a multi-person 3D human skeleton in the same 3D coordinate system.

Images