A human posture estimation network model and estimation method based on thermal imaging

Abstract

The present invention relates to a human posture estimation network model and estimation method, and more specifically to a human posture estimation network model and estimation method based on thermal imaging. These methods are designed to address the shortcomings of HRNet-based human posture estimation methods, such as being overly complex, having an overly deep model, being suitable for extracting fine features but unsuitable for thermal imaging that only displays contour information, and having low computational efficiency and slow inference speed. The human posture estimation network model based on thermal imaging includes a Start module, a Transition module, and a Stage module. The present invention introduces thermal imaging into the field of human posture estimation, addressing issues such as illumination effects and complex backgrounds. Because thermal imaging only displays the contours of the human body, it also addresses the issue of personal privacy disclosure.

Description

Technical Field

[0001] The present invention relates to a human body posture estimation network model and an estimation method, and in particular to a human body posture estimation network model and an estimation method based on thermal imaging. Background Art

[0002] 2D single-person human pose estimation is a classic computer vision problem with widespread applications in many fields, including pedestrian prediction during vehicle driving and motion recognition for the elderly, as well as human motion effects and human simulation modeling in short video applications on the internet. The goal of 2D single-person human pose estimation is to determine the locations of key points of the human skeleton in an image.

[0003] 2D single-person human pose estimation is typically studied based on RGB images. The RGB color model is an industry standard that produces a variety of colors by varying and superimposing the three color channels: red (R), green (G), and blue (B). RGB images can only clearly display human pose information under certain lighting conditions, thus limiting the application scenarios of human pose estimation.

[0004] Currently, 2D single-person human pose estimation methods mainly include regression-based and detection-based methods. Regression-based methods use the coordinates of key points as the final coordinates that the network needs to regress, directly obtaining the position information of each key point. Detection-based methods generate a heat map for each joint point and locate the joint point as the point with the maximum value in the heat map. This method better preserves spatial information and has higher performance, thus dominating the field of 2D single-person human pose estimation.

[0005] Most detection-based human pose estimation methods primarily employ network frameworks that scale from high-resolution to low-resolution and vice versa. For example, High-Resolution Net (HRNet) connects high-resolution and low-resolution networks in parallel, maintaining high resolution throughout the entire process and making the predicted thermal maps more accurate. However, HRNet is overly complex and deep, making it suitable for extracting fine features but not for thermal imaging that only displays contour information. Furthermore, HRNet suffers from low computational efficiency and slow inference speed. Summary of the Invention

[0006] The purpose of the present invention is to solve the shortcomings of the human body posture estimation method based on HRNet, such as being too complex, having a deep model, being suitable for extracting fine features, not being suitable for thermal imaging that only displays contour information, as well as having low computational efficiency and slow inference speed, and to provide a human body posture estimation network model and estimation method based on thermal imaging.

[0007] In order to solve the deficiencies of the above-mentioned prior art, the present invention provides the following technical solutions:

[0008] A human posture estimation network model based on thermal imaging is special in that it includes a Start module, a Transition1 module, a Stage2 module, a Transition2 module, a Stage3 module, a Transition3 module, and a Stage4 module, the output and input of which are connected in sequence;

[0009] The Start module is used to perform the first step of feature extraction on the thermal imaging data of the human body posture to be estimated. The Start module includes a convolutional layer, a BN layer, a ReLU layer and four Bottleneck blocks whose output and input are connected in sequence;

[0010] The Transition1 module, the Transition2 module, and the Transition3 module constitute a Transition module; the Transition1 module includes two parallel sub-networks, and the resolution of the second sub-network is half of the resolution of the first sub-network; the Transition2 module includes three parallel sub-networks, and the resolution of the N+1th sub-network is half of the resolution of the Nth sub-network; N=1, 2; the Transition3 module includes four parallel sub-networks, and the resolution of the M+1th sub-network is half of the resolution of the Mth sub-network; M=1, 2, 3;

[0011] The Stage2 module, Stage3 module, and Stage4 module constitute the Stage module; the Stage2 module, Stage3 module, and Stage4 module are respectively used to introduce exchange units between the sub-networks in the Transition1 module, Transition2 module, and Transition3 module, so that each sub-network repeatedly receives information from other sub-networks; the Stage2 module, Stage3 module, and Stage4 module are all composed of BasicBlock blocks, convolutional layers, BN layers, ReLU layers, and upsampling layers.

[0012] Furthermore, each of the Bottleneck blocks is the same as the building block of ResNet-50, consisting of three convolutional layers, and the convolution kernel sizes of the three convolutional layers are 1*1, 3*3 and 1*1 respectively.

[0013] Furthermore, the BasicBlock block is the same as the building block of ResNet-34, and is composed of two convolutional layers with a convolution kernel size of 5*5.

[0014] At the same time, the present invention provides a method for estimating human posture based on thermal imaging, which is special in that it includes the following steps:

[0015] Step 1: Construct a 2D single-person human posture thermal imaging dataset;

[0016] Step 1.1: Use the acquisition device to collect thermal imaging data of a single person's body posture of multiple acquisition objects according to the preset acquisition action;

[0017] Step 1.2, thermal imaging data calibration:

[0018] Process the thermal imaging data obtained in step 1.1 by extracting frames from the video, so that the number of thermal imaging data for each preset acquisition action is 4000-5000. Randomly divide the processed thermal imaging data into training, validation, and test sets in a ratio of 3:1:1 for the number of acquisition objects, and label the data set to which the thermal imaging data belongs.

[0019] Step 2: Preprocessing of training set data;

[0020] Perform data augmentation on each thermal imaging data in the training set and crop its image size to (256, 256);

[0021] Step 3: Thermal imaging feature extraction;

[0022] Step 3.1, inputting the training set thermal imaging data obtained in step 2 into the human posture estimation network model based on thermal imaging for training, thereby obtaining a trained human posture estimation network model based on thermal imaging;

[0023] Step 3.2: Perform data augmentation on the validation set data or the test set data, crop the image size to (256, 256), input the trained thermal imaging-based human posture estimation network model obtained in step 3.1, and output the corresponding human posture estimation result, thus completing the thermal imaging feature extraction.

[0024] Furthermore, the step 3.1 is specifically as follows: inputting the training set thermal imaging data obtained in step 2 into the thermal imaging-based human posture estimation network model and training the thermal imaging-based human posture estimation network model using a gradient descent method;

[0025] Step 3.1.1. Input the training set thermal imaging data obtained in step 2 into the Start module for the first step of feature extraction, so that the feature dimension of each thermal imaging data is converted from (batch, 256, 256, 3) to (batch, 128, 128, 256).

[0026] Step 3.1.2: Input the features of each thermal imaging data obtained in step 3.1.1 into the Transition1 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 256) to (batch, 128, 128, 32) and (batch, 64, 64, 64).

[0027] Step 3.1.3: Input the features of each thermal imaging data obtained in step 3.1.2 into the Stage2 module, so that the two sub-networks of the Transition1 module can exchange information with each other;

[0028] Step 3.1.4: Input the features of each thermal imaging data obtained in step 3.1.3 into the Transition2 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32) and (batch, 64, 64, 64) to (batch, 128, 128, 32), (batch, 64, 64, 64) and (batch, 32, 32, 128);

[0029] Step 3.1.5: Input the features of each thermal imaging data obtained in step 3.1.4 into the Stage3 module, so that the three sub-networks of the Transition2 module can exchange information with each other;

[0030] Step 3.1.6: Input the features of each thermal imaging data obtained in step 3.1.5 into the Transition3 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32), (batch, 64, 64, 64), and (batch, 32, 32, 128) to (batch, 128, 128, 32), (batch, 64, 64, 64), (batch, 32, 32, 128), and (batch, 16, 16, 256).

[0031] Step 3.1.7: Input the features of each thermal imaging data obtained in step 3.1.6 into the Stage4 module so that the four sub-networks of the Transition3 module exchange information with each other and then fuse into a high-resolution network to output the human body posture estimation results of each thermal imaging data.

[0032] Furthermore, in step 2 and step 3.2, the data enhancement is specifically as follows: for each thermal imaging data in the training set, data enhancement is performed by random rotation [-30°, 30°], random scaling [0.85, 1.45], random horizontal flipping, and random translation [-10, 10].

[0033] Furthermore, in step 1.1, the acquisition devices are a SenseThunder-E Thermal Camera and a SensePass Pro panel; the distance between the acquisition object and the acquisition device is 4 to 5 meters to ensure that the complete human posture of the acquisition object is collected and privacy is better protected; the preset acquisition actions include some or all of standing upright, walking upright, squatting upright, waving upright, standing sideways, and walking sideways.

[0034] Furthermore, in step 1.2, the labeling of the data set to which the thermal imaging data belongs is specifically: labeling the data set to which the thermal imaging data belongs according to the labeling format of the MPII data set using labelme labeling software.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The present invention provides a human posture estimation network model based on thermal imaging, including a Start module, a Transition module and a Stage module. The present invention introduces thermal imaging into the field of human posture estimation, solves problems such as illumination influence and complex background, and solves the problem of personal privacy leakage because thermal imaging only displays the outline information of the human body.

[0037] (2) The present invention proposes a human posture estimation network model based on thermal imaging, which greatly reduces the number of parameters and computational complexity of the model, improves the inference speed of the model, and makes it possible to deploy it on edge devices; specifically, compared with Pose ResNet50, the PCKh@0.5 of the present invention is improved by 2.42 points, the model size is about one-tenth of that of Pose ResNet50, the number of parameters is reduced by about one-half, and the FPS is improved by 3; compared with Hourglass and its extended network Yang et al., the number of parameters of the present invention is about one-eighth of that of the Hourglass network, the model size is about one-eighth of the Hourglass network, and the FPS is doubled compared with the Hourglass network; although the PCKh@0.5 of the present invention is slightly lower than HRNet-W32, the number of parameters is about one-ninth of that of HRNet-W32, the model size is about one-tenth of that of HRNet-W32, and the FPS is doubled; compared with HRNet-W48, the PCKh@0.5 of the present invention is improved by 0.9 points, and the FPS is doubled.

[0038] (3) The present invention provides a human posture estimation network model based on thermal imaging, which improves the accuracy of the model by always maintaining high resolution.

[0039] (4) The present invention provides a method for human posture estimation based on thermal imaging, which is used to solve the problems of light intensity, complex background, privacy leakage and slow reasoning speed that appear in the current research on human posture estimation based on RGB images. By introducing thermal imaging into the field of human posture estimation, a lightweight, high-resolution network is constructed, which solves the influence of environmental factors such as light and background on the detection of human key points, protects personal privacy, improves the reasoning speed, and makes it possible to deploy the model on edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic structural diagram of a human posture estimation network model based on thermal imaging according to the present invention;

[0041] Figure 2 This is a schematic diagram of the structure of the Start module in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of the Transition module in an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of the structure of the Stage 2 module in an embodiment of the present invention;

[0044] Figure 5 Schematic diagram of the structure of the Stage 2 module in an embodiment of the present invention;

[0045] Figure 6 A schematic flow chart of a method for estimating a human body posture based on thermal imaging according to the present invention;

[0046] Figure 7 This is a schematic diagram of the person standing upright in the preset collection action in step 1.1 of an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of the stepping and walking upright in the preset collection action in step 1.1 of the embodiment of the present invention;

[0048] Figure 9 This is a schematic diagram of squatting upright in the preset collection action in step 1.1 of an embodiment of the present invention;

[0049] Figure 10 This is a schematic diagram of the preset collection action in step 1.1 of the embodiment of the present invention, showing a standing upright waving hand;

[0050] Figure 11 This is a schematic diagram of standing sideways in the preset collection action in step 1.1 of an embodiment of the present invention;

[0051] Figure 12 This is a schematic diagram of the sideways stepping in the preset collection action in step 1.1 of an embodiment of the present invention;

[0052] Figure 13 This is a schematic diagram of the human body posture estimation result under the lighting environment obtained in step 3.1.7 of the embodiment of the present invention;

[0053] Figure 14 This is a schematic diagram of the human body posture estimation result in a dark environment obtained in step 3.1.7 of an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described below with reference to the accompanying drawings and exemplary embodiments.

[0055] Reference Figures 1 to 4 ,A human posture estimation network model based on thermal imaging, includes a Start module, a Transition1 module, a Stage2 module, a Transition2 module, a Stage3 module, a Transition3 module, and a Stage4 module whose output and input are sequentially connected.

[0056] Reference Figure 2 The Start module is used to perform the first step of feature extraction on the thermal imaging data of the human body posture to be estimated. The Start module includes a convolutional layer, a BN layer, a ReLU layer, and four Bottleneck blocks whose output and input are connected in sequence; each Bottleneck block is the same as the building block of ResNet-50, consisting of three convolutional layers, and the convolution kernel sizes of the three convolutional layers are 1*1, 3*3, and 1*1, respectively.

[0057] Reference Figure 3 , the Transition1 module, the Transition2 module, and the Transition3 module constitute a Transition module; the Transition1 module includes two parallel sub-networks, and the resolution of the second sub-network is half of the resolution of the first sub-network; the Transition2 module includes three parallel sub-networks, and the resolution of the N+1th sub-network is half of the resolution of the Nth sub-network; N=1,2; the Transition3 module includes four parallel sub-networks, and the resolution of the M+1th sub-network is half of the resolution of the Mth sub-network; M=1,2,3.

[0058] Reference Figure 4 、 Figure 5, the Stage2 module, Stage3 module, and Stage4 module constitute the Stage module; the Stage2 module, Stage3 module, and Stage4 module are respectively used to introduce exchange units between the sub-networks in the Transition1 module, Transition2 module, and Transition3 module, so that each sub-network repeatedly receives information from other sub-networks; the Stage2 module, Stage3 module, and Stage4 module are all composed of BasicBlock blocks, convolutional layers, BN layers, ReLU layers, and upsampling layers (Upsample); the BasicBlock block is the same as the building block of ResNet-34, and is composed of two convolutional layers with a convolution kernel size of 5*5.

[0059] Reference Figure 6 , a method for estimating human posture based on thermal imaging, comprising the following steps:

[0060] Step 1: Construct a 2D single-person human posture thermal imaging dataset;

[0061] Step 1.1: Use the SenseThunder-E Thermal Camera and the SensePass Pro panel to collect thermal imaging data of 60 subjects in both illuminated and dark environments. The distance between the subjects and the device is 4 meters to ensure that the subjects' complete postures are captured and privacy is better protected. The preset collection actions include standing upright, walking upright, squatting upright, waving upright, standing sideways, and walking sideways. Figures 7 to 12 As shown;

[0062] Step 1.2, thermal imaging data calibration:

[0063] The thermal imaging data obtained in step 1.1 were processed by video frame extraction, so that the number of thermal imaging data for each preset acquisition action was 4000-5000. The processed thermal imaging data were randomly divided into training, validation, and test sets according to the ratio of the number of acquisition objects of 3:1:1. The datasets to which the thermal imaging data belonged were annotated using the LabelMe annotation software according to the annotation format of the MPII dataset. The validation and test sets serve the same purpose.

[0064] Step 2: Preprocessing of training set data;

[0065] The thermal imaging data in the training set are augmented by random rotation [-30°, 30°], random scaling [0.85, 1.45], random horizontal flipping, and random translation [-10, 10], and the image size is cropped to (256, 256);

[0066] Step 3: Thermal imaging feature extraction;

[0067] Inputting the training set thermal imaging data obtained in step 2 into the thermal imaging-based human posture estimation network model, and training the thermal imaging-based human posture estimation network model using a gradient descent method to obtain a trained thermal imaging-based human posture estimation network model;

[0068] Step 3.1.1. Input the training set thermal imaging data obtained in step 2 into the Start module for the first step of feature extraction, so that the feature dimension of each thermal imaging data is converted from (batch, 256, 256, 3) to (batch, 128, 128, 256).

[0069] Step 3.1.2: Input the features of each thermal imaging data obtained in step 3.1.1 into the Transition1 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 256) to (batch, 128, 128, 32) and (batch, 64, 64, 64).

[0070] Step 3.1.3: Input the features of each thermal imaging data obtained in step 3.1.2 into the Stage2 module, so that the two sub-networks of the Transition1 module can exchange information with each other;

[0071] Step 3.1.4: Input the features of each thermal imaging data obtained in step 3.1.3 into the Transition2 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32) and (batch, 64, 64, 64) to (batch, 128, 128, 32), (batch, 64, 64, 64) and (batch, 32, 32, 128);

[0072] Step 3.1.5: Input the features of each thermal imaging data obtained in step 3.1.4 into the Stage3 module, so that the three sub-networks of the Transition2 module can exchange information with each other;

[0073] Step 3.1.6: Input the features of each thermal imaging data obtained in step 3.1.5 into the Transition3 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32), (batch, 64, 64, 64), and (batch, 32, 32, 128) to (batch, 128, 128, 32), (batch, 64, 64, 64), (batch, 32, 32, 128), and (batch, 16, 16, 256).

[0074] Step 3.1.7: Input the features of each thermal imaging data obtained in step 3.1.6 into the Stage4 module so that the four sub-networks of the Transition3 module exchange information with each other and fuse into a high-resolution network, outputting the human posture estimation results of each thermal imaging data. The human posture estimation results under the lighting environment are as follows: Figure 13 As shown in Figure 2, the human body posture estimation results in a dark environment are as follows: Figure 14 As shown;

[0075] Step 3.2: For each thermal imaging data of the validation set, data enhancement is performed by randomly rotating [-30°, 30°], randomly scaling [0.85, 1.45], randomly flipping horizontally, and randomly translating [-10, 10]. After the image size is cropped to (256, 256), the human posture estimation network model based on thermal imaging obtained in step 3.1 is input and the corresponding human posture estimation result is output, thus completing the thermal imaging feature extraction.

[0076] Based on the general evaluation indicators of human pose estimation models, the embodiment of the present invention is compared with existing human pose estimation methods, and the results are shown in Table 1. Table 1 shows that compared with Pose ResNet50, the embodiment of the present invention improves PCKh@0.5 by 2.42 points, the model size is about one-tenth of that of Pose ResNet50, the number of parameters is reduced by about one-half, and the FPS is improved by 3. Compared with Hourglass and its extended network Yang et al., the embodiment of the present invention has about one-eighth of the parameters of the Hourglass network, the model size is about one-eighth of the Hourglass network, and the FPS is doubled compared with the Hourglass network. Although PCKh@0.5 of the embodiment of the present invention is slightly lower than HRNet-W32, the number of parameters is about one-ninth of that of HRNet-W32, the model size is about one-tenth of HRNet-W32, and the FPS is doubled. Compared with HRNet-W48, the embodiment of the present invention improves PCKh@0.5 by 0.9 points and the FPS is doubled.

[0077] Table 1

[0078]

[0079] Size: size of the model; Params: number of parameters; GFLOPS: Giga Floating-point Operations Per Second, that is, 1 billion floating-point operations per second; Fps: number of frames transmitted per second; PCKh@0.5: If the distance between the predicted joint and the true joint is within 50% of the skull link, the detected joint is considered correct.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. For ordinary professional and technical personnel in this field, the specific technical solutions recorded in the above embodiments can be modified, or some of the technical features therein can be replaced by equivalents. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions protected by the present invention.

Claims

1. A human posture estimation network model based on thermal imaging, characterized by: It includes the Start module, Transition1 module, Stage2 module, Transition2 module, Stage3 module, Transition3 module, and Stage4 module, whose output and input are connected in sequence; The Start module is used to perform the first step of feature extraction on the thermal imaging data of the human body posture to be estimated. The Start module includes a convolutional layer, a BN layer, a ReLU layer and four Bottleneck blocks whose output and input are connected in sequence; The Transition1 module, the Transition2 module, and the Transition3 module constitute a Transition module; the Transition1 module includes two parallel sub-networks, and the resolution of the second sub-network is half of the resolution of the first sub-network; the Transition2 module includes three parallel sub-networks, and the resolution of the N+1th sub-network is half of the resolution of the Nth sub-network; N=1, 2; the Transition3 module includes four parallel sub-networks, and the resolution of the M+1th sub-network is half of the resolution of the Mth sub-network; M=1, 2, 3; The Stage2 module, Stage3 module, and Stage4 module constitute the Stage module; the Stage2 module, Stage3 module, and Stage4 module are respectively used to introduce exchange units between the sub-networks in the Transition1 module, Transition2 module, and Transition3 module, so that each sub-network repeatedly receives information from other sub-networks; the Stage2 module, Stage3 module, and Stage4 module are all composed of BasicBlock blocks, convolutional layers, BN layers, ReLU layers, and upsampling layers.

2. The human body posture estimation network model based on thermal imaging according to claim 1, characterized in that: Each of the Bottleneck blocks is the same as the building block of ResNet-50, consisting of three convolutional layers, and the convolution kernel sizes of the three convolutional layers are 1*1, 3*3 and 1*1 respectively.

3. The human body posture estimation network model based on thermal imaging according to claim 1, characterized in that: The BasicBlock block is the same as the building block of ResNet-34, and consists of two convolutional layers with a convolution kernel size of 5*5.

4. A method for estimating human posture based on thermal imaging, characterized in that: The following steps are involved: Step 1: Construct a 2D single-person human posture thermal imaging dataset; Step 1.1: Use the acquisition device to collect thermal imaging data of a single person's body posture of multiple acquisition objects according to the preset acquisition action; Step 1.2, thermal imaging data calibration: Process the thermal imaging data obtained in step 1.1 by extracting frames from the video, so that the number of thermal imaging data for each preset acquisition action is 4000-5000. Randomly divide the processed thermal imaging data into training, validation, and test sets in a ratio of 3:1:1 for the number of acquisition objects, and label the data set to which the thermal imaging data belongs. Step 2: Preprocessing of training set data; Perform data augmentation on each thermal imaging data in the training set and crop its image size to (256, 256); Step 3: Thermal imaging feature extraction; Step 3.1, inputting the training set thermal imaging data obtained in step 2 into the human posture estimation network model based on thermal imaging according to claim 1 for training to obtain a trained human posture estimation network model based on thermal imaging; Step 3.2: Perform data augmentation on the validation set data or the test set data, crop the image size to (256, 256), input the trained thermal imaging-based human posture estimation network model obtained in step 3.1, and output the corresponding human posture estimation result, thus completing the thermal imaging feature extraction.

5. The method for estimating human posture based on thermal imaging according to claim 4, characterized in that: The step 3.1 specifically comprises: inputting the training set thermal imaging data obtained in step 2 into the human body posture estimation network model based on thermal imaging according to claim 1, and training the human body posture estimation network model based on thermal imaging using a gradient descent method; Step 3.1.

1. Input the training set thermal imaging data obtained in step 2 into the Start module for the first step of feature extraction, so that the feature dimension of each thermal imaging data is converted from (batch, 256, 256, 3) to (batch, 128, 128, 256). Step 3.1.2: Input the features of each thermal imaging data obtained in step 3.1.1 into the Transition1 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 256) to (batch, 128, 128, 32) and (batch, 64, 64, 64). Step 3.1.3: Input the features of each thermal imaging data obtained in step 3.1.2 into the Stage2 module, so that the two sub-networks of the Transition1 module can exchange information with each other; Step 3.1.4: Input the features of each thermal imaging data obtained in step 3.1.3 into the Transition2 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32) and (batch, 64, 64, 64) to (batch, 128, 128, 32), (batch, 64, 64, 64) and (batch, 32, 32, 128); Step 3.1.5: Input the features of each thermal imaging data obtained in step 3.1.4 into the Stage3 module, so that the three sub-networks of the Transition2 module can exchange information with each other; Step 3.1.6: Input the features of each thermal imaging data obtained in step 3.1.5 into the Transition3 module, so that the feature dimensions of each thermal imaging data are transformed from (batch, 128, 128, 32), (batch, 64, 64, 64), and (batch, 32, 32, 128) to (batch, 128, 128, 32), (batch, 64, 64, 64), (batch, 32, 32, 128), and (batch, 16, 16, 256). Step 3.1.7: Input the features of each thermal imaging data obtained in step 3.1.6 into the Stage4 module so that the four sub-networks of the Transition3 module exchange information with each other and then fuse into a high-resolution network to output the human body posture estimation results of each thermal imaging data.

6. The method for estimating human posture based on thermal imaging according to claim 5, characterized in that: In step 2 and step 3.2, the data enhancement is specifically as follows: for each thermal imaging data in the training set, data enhancement is performed by random rotation [-30°, 30°], random scaling [0.85, 1.45], random horizontal flipping, and random translation [-10, 10].

7. A method for estimating human posture based on thermal imaging according to any one of claims 4 to 6, characterized in that: In step 1.1, the acquisition devices are a SenseThunder-E Thermal Camera and a SensePassPro panel. The distance between the acquisition object and the acquisition device is 4 to 5 meters to ensure that the complete human posture of the acquisition object is collected and privacy is better protected. The preset acquisition actions include some or all of the following: standing upright, walking upright, squatting upright, waving upright, standing sideways, and walking sideways.

8. The method for estimating human posture based on thermal imaging according to claim 7, wherein: In step 1.2, the step of labeling the data set to which the thermal imaging data belongs is specifically: labeling the data set to which the thermal imaging data belongs according to the labeling format of the MPII data set using labelme labeling software.

Images