Human body behavior recognition device and method based on infrared image

Abstract

The invention discloses a human body behavior recognition device and method based on an infrared image, and the method comprises the following steps: S1, collection of the infrared image: employing a single-chip microcomputer to drive an infrared sensor, enabling detected temperature data to form an 80 * 64 * n digital matrix, transmitting the 80 * 64 * n digital matrix to the single-chip microcomputer through an SPI, and storing the 80 * 64 * n digital matrix, then the temperature digital matrix is transmitted to a PC end in a UART communication mode; s2, preprocessing the infrared image; s3, making a human body behavior data set: designing and collecting infrared images of seven behavior actions of bending down, falling down, lying flat, standing, sitting still, standing-to-sitting and walking; s4, designing a behavior recognition algorithm; s5, behavior recognition algorithm training; and S6, identifying seven behavior actions. According to the method, the accuracy is higher, the recognition speed is higher, terminal deployment with limited computing resources is facilitated, and the actual application requirement is met.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and pattern recognition, and in particular to a human behavior recognition device and method based on infrared images. Background Art

[0002] Human behavior recognition is one of the core topics in the field of computer vision and pattern recognition, and its purpose is to identify human behavior in videos or images. Effective and accurate recognition of human behavior is the basis of many intelligent services. It has important research significance and wide application value in many fields such as intelligent monitoring, smart home, virtual reality, etc., and has attracted much attention from scholars at home and abroad.

[0003] Most of the research on traditional human behavior recognition focuses on visible light information. In visible light videos or images, the outline of the human body is relatively clear, and the pixel data is relatively rich, but the human behavior recognition device based on visible light information can only operate in the daytime with sufficient light and good vision, and loses its function in the night and other conditions with poor vision. In comparison, the infrared imaging system collects temperature data in the environment for imaging, and can work normally at night when the vision is poor, making up for the shortcomings based on visible light information. Therefore, the present invention focuses on the infrared imaging system for the collection of human behavior information. In addition, the infrared imaging system collects temperature data information in the space environment, avoiding the defect of the visible light imaging system exposing privacy. For example, in humid environments such as toilets and bathrooms, the possibility of dangerous behaviors such as falling is greater, and the infrared imaging system will not expose the detailed information of the human appearance, so the infrared imaging system has important application value and significance for the research on human behavior recognition. Summary of the invention

[0004] The purpose of the present invention is to provide a human behavior recognition device and method based on infrared images to solve the problems in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for human behavior recognition based on infrared images, comprising the following steps:

[0006] S1. Acquisition of infrared images:

[0007] The infrared sensor is driven by a single-chip microcomputer, and the detected temperature data is formed into a 80*64*n digital matrix and transmitted to the single-chip microcomputer for storage through SPI, where n is the total number of frames, and then the temperature digital matrix is ​​transmitted to the PC through UART communication;

[0008] S2. Preprocessing of infrared images:

[0009] The image stitching method is used to stitch 25 consecutive frames of images of an action into a large image in chronological order as the input of the system; secondly, the stitched infrared large image is denoised;

[0010] S3. Preparation of human behavior dataset:

[0011] Design and collect infrared images of seven kinds of actions: "bending", "falling down", "lying flat", "standing", "sitting still", "from standing to sitting", and "walking";

[0012] S4. Design of behavior recognition algorithm:

[0013] Based on ResNet18, we designed the attention residual network ResNet18c_split based on channel separation;

[0014] S5. Behavior recognition algorithm training:

[0015] Input the infrared human action dataset into the channel-separated attention residual network ResNet18c_split;

[0016] S6. Recognition of seven kinds of behaviors:

[0017] The divided test set is used to test the model after training convergence separately; after each infrared image is predicted by the algorithm, a one-dimensional feature vector containing seven elements will be generated. After being processed by the SoftMax function, the probability values ​​of the seven action categories can be obtained. The label category corresponding to the maximum value is the final prediction result.

[0018] Preferably, in the S1, the PIL function library in the Python language is used to convert the temperature digital matrix in the txt file into an infrared image for storage; first, the txt file is read to generate a two-dimensional digital matrix of 80*64*n, and its number of rows is traversed and then divided by 64 to obtain the total number of frames of the infrared image, and then the array is sliced ​​to obtain n 80*64 two-dimensional array small matrices, and the temperature data is normalized to the range of 0-255 by row, and finally the fromarry function is used to obtain n infrared grayscale images and save them.

[0019] Preferably, the denoising method in S2 is to traverse all pixel points in the image, and if the point is a noise point or a pixel is missing, the pixel value of the point is replaced by the average value of the eight surrounding pixel points.

[0020] Preferably, 754 samples are obtained for each action in S3, 62.3% of which are divided into a training set, 26.5% into a validation set, and 11.1% into a test set; the 470 samples in the training set are data enhanced by using the rotation angle and horizontal mirror flipping method to triple the number of samples, while the validation set and the test set are not enhanced; the total amount of all data in the processed data set is (470*3+200+84)*7=11858.

[0021] Preferably, in S4:

[0022] Using the idea of ​​channel separation, the output feature map of each residual layer of the residual network is separated into group A and group B with the same number of channels. Group A is sent to the next residual layer to continue feature extraction and residual calculation, while group B is adjusted by a simple 1*1 convolution to adjust the dimension of the feature map and extract shallow features. Then the two groups of feature maps are spliced ​​in the channel dimension to restore the number of channels. After the feature map is spliced, the channel shuffle operation is added to increase the information interaction of different channels. The calculation formula for channel separation between each layer is as follows:

[0023] Out_A,Out_B=Channel_split(input)

[0024] Output=Channel_shuffle(Contact[F(Out_A);Conv2d(Out_B)])

[0025] Among them, Channel_split represents channel separation, Out_A and Out_B represent the two sets of feature maps after separation, F represents the residual calculation of the next layer, Conv2d represents 1*1 convolution, Contact represents splicing in the channel dimension, and Channel_shuffle represents the channel shuffle operation;

[0026] The CBAM attention mechanism is introduced into the original residual block: the CBAM attention module is added to the branch that is sent to the next residual layer after channel separation to enhance the network's attention and improve the network's recognition accuracy of seven behavioral actions; the CBAM module contains two sub-modules: channel attention (Att_C) and spatial attention (Att_S); channel attention uses a multi-layer perceptron to generate different channel weight vectors, and then multiplies the weight vector with the original feature map to increase the importance of the effective channel feature map; the formula for channel attention is as follows:

[0027] Att_C=σ(MLP(GmaxPool(x))+MLP(GavgPool(x)))

[0028] Where x represents input, GmaxPool and GavgPool represent global maximum pooling and global average pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid activation function;

[0029] Spatial attention focuses on the position information of the human target in the feature map, which enables the network to more effectively extract the contour features of the human body shape; the formula for spatial attention is as follows:

[0030] Att_S=σ(Conv2d(Contact[Maxpool(x);Avgpool(x)]))

[0031] Where x represents input, MaxPool and AvgPool represent maximum pooling in the spatial direction and average pooling in the spatial direction respectively, Conv2d represents two-dimensional convolution, and σ represents the Sigmoid activation function;

[0032] Channel attention and spatial attention are connected in series to form the CBAM attention module, which focuses on important features and suppresses unnecessary features. The formula of the CBAM attention mechanism is as follows:

[0033]

[0034] Where input represents the input of the network, Att_C and Att_S represent channel attention and spatial attention respectively.

[0035] Preferably, the method of S5 is: firstly train the ResNet18c_split network with the training set and the validation set until it converges, save the optimal model weight parameters, and then test the optimal model with the test set to obtain the recognition accuracy and confusion matrix of each action, and then obtain the average recognition accuracy of the algorithm;

[0036] For each round of training, the training set is first used as the input of the network, the batch data is set to 32, the initial learning rate is set to 0.001, and the learning rate decays in a cosine function trend as the number of training rounds increases, and the lower limit of the decay is set to 0.0001; during the training process, the cross-Sang loss function is used to obtain the loss error between the output result of the model and the true label category, and the Adam optimizer is used to back-propagate the loss error to obtain the model weight parameter that minimizes the loss function; then the validation set is used as the input of the network to obtain the probability value of the model's prediction of the seven types of actions, and the maximum value is taken as the final prediction result. The number of the final prediction result and the true label category is recorded, and then the accuracy of each round of the model is obtained;

[0037] Train the model until it converges and save the weight parameters of the optimal model, and finally complete the training of ResNet18c_split.

[0038] Preferably, the single chip microcomputer model in S1 is STM32F103; the infrared image sensor is Heiman HTPA80*64R2L.

[0039] Preferably, the human behavior recognition results include: "bending", "falling down", "lying down", "standing", "sitting still", "from standing to sitting", and "walking".

[0040] A human behavior recognition device based on infrared images comprises an infrared sensor, a single-chip microcomputer and a host computer; the infrared sensor transmits data to the single-chip microcomputer via SPI, and the single-chip microcomputer is connected to the host computer via serial port communication.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. Use STM32F103 single-chip computer to drive infrared sensor to collect infrared images; since a single image only contains contour information and cannot well display the timing information of a certain action within a period of time, the present invention splices 25 consecutive frames of infrared images in an action sample into a large image in chronological order as the sample data of a single action, and performs corresponding denoising; secondly, simulate the actual environment to produce infrared image data sets of seven actions for training the ResNet18c_split network. Through the improvement of the original ResNet18 network, a human behavior recognition method with higher recognition accuracy, less calculation and parameter amount is finally realized, achieving accurate and rapid recognition of dangerous behaviors, providing a guarantee for the monitoring of behavioral safety, and has broad application prospects.

[0043] 2. In terms of human behavior information collection, the infrared thermal imaging sensor model Heiman HTPA80*64R2L is used to collect infrared images. This type of sensor can collect temperature data information in the environment without exposing privacy, can work normally at night, and is low cost.

[0044] 3. Select ResNet18 in the convolutional neural network as the basic network and improve it, separate the channels of the feature maps between its residual layers, and then introduce the efficient CBAM attention module to form a new channel-separated attention residual network ResNet18c_split, to achieve higher recognition accuracy and faster recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0046] Figure 1Schematic diagram of the infrared image acquisition system.

[0047] Figure 2 This is a txt file showing the temperature data collected by the sensor.

[0048] Figure 3 This is a display of a single infrared image (80*64) collected.

[0049] Figure 4 The following is a sample (400*320) of the seven actions after splicing.

[0050] Figure 5 This figure shows the denoising effect of infrared images.

[0051] Figure 6 This is the effect diagram of training set data enhancement.

[0052] Figure 7 Structural diagram of the human behavior dataset produced.

[0053] Figure 8 This is the structure diagram of the attention residual network ResNet18c_split based on channel separation.

[0054] Fig. 9 The following is a diagram showing the recognition effects of seven behavior categories.

[0055] Fig.10 This is a comparison curve of the training effects of ResNet18 and ResNet18c_split.

[0056] Fig.11 The following is a comparison of the recognition effects of ResNet18 and ResNet18c_split on seven actions.

[0057] Fig.12 This is the confusion matrix of ResNet18 and ResNet18c_split on the test set.

[0058] Fig.13 This is a comparison chart of the computational complexity and parameter amounts of ResNet18 and ResNet18c_split. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention.

[0060] See also Figure 1-13 In an embodiment of the present invention, a method for identifying human behavior based on infrared images comprises the following steps:

[0061] Step 1: Acquisition of a single infrared image:

[0062] The present invention uses the infrared thermal imaging sensor model HTPA80*64R2L produced by German Heimann Company. The sensor contains a thermopile inside, which can capture infrared radiation on the human body surface and collect temperature data. In addition, its resolution is low, and the collected single image is only 80*64 pixels in size, which will not expose human privacy.

[0063] The schematic diagram of the infrared image acquisition system is as follows: Figure 1 As shown in the figure, first use the STM32F103 microcontroller to drive the infrared sensor, and transfer the detected temperature data into a 80*64*n digital matrix through SPI to the microcontroller for storage, where n is the total number of frames. Then the microcontroller transfers the temperature digital matrix to the PC through UART communication, and finally Figure 2 txt file format shown.

[0064] In the process of infrared image conversion, the temperature digital matrix in txt is converted into infrared image storage using the PIL function library in Python language. First, read the txt file to generate a 80*64*n two-dimensional digital matrix, traverse its number of rows and divide it by 64 to get the total number of frames of the infrared image, then slice the array to get n 80*64 two-dimensional array small matrices, normalize the temperature data by row to the range of 0-255, and finally use the fromarry function to get n infrared grayscale images and save them. Figure 3 It is a display of a single 80*64 infrared image collected, and the action shown is standing.

[0065] Step 2: Preprocessing of infrared images:

[0066] The preprocessing of infrared images mainly includes two aspects: infrared image stitching and infrared image denoising.

[0067] Since a single infrared image cannot well display continuous actions within a period of time, the image stitching method is used to stitch 25 consecutive frames of an action into a large image in chronological order as the input of the system. First, each frame of the large image contains the contour appearance features of the action, that is, the spatial information of the action. Secondly, the stitched large image contains the continuous changes in the time domain process of an action, that is, the time information of the action, which allows the residual network to learn the order in which each frame occurs. Therefore, the stitched large image has the ability to express action information, and classification and recognition of the stitched large image can achieve the effect of behavioral action recognition. Figure 4 These are sample images of the seven actions after splicing.

[0068] Since the collected infrared images contain noise points and missing pixels, the spliced ​​infrared images are denoised by traversing all the pixels in the image. If the point is a noise point or a missing pixel, the pixel value of the point is replaced by the average value of the eight surrounding pixels. Figure 5 As shown in Figure 2, this method handles the effect of noise points well.

[0069] Step 3: Preparation of human behavior dataset:

[0070] The present invention designs and collects infrared images of seven kinds of behaviors: "Bend", "Fall_down", "Lie", "Stand", "Sit", "Stand to Sit", and "Walk". 754 samples are obtained for each behavior, of which 62.3% (470) are divided into a training set, 26.5% (200) are divided into a validation set, and 11.1% (84) are divided into a test set. In addition, in order to enhance the generalization and robustness of the system and prevent overfitting of the model, Figure 6 As shown in the figure, the data is enhanced by rotating the angle and flipping the horizontal mirror. It is worth noting that, considering the problem of data leakage and the authenticity of the experiment, only 470 samples of the training set are enhanced, so that the number of samples in the training set is tripled, while the validation set and the test set are not enhanced. The total amount of all data in the processed data set is (470*3+200+84)*7=11858, which meets the requirements of model training. The detailed composition of the human behavior data set is as follows: Figure 7 shown.

[0071] Step 4: Design of behavior recognition algorithm:

[0072] The ResNet18 network is a typical representative of the residual series network. It has a good recognition effect in visible light images. However, due to factors such as blurred contour information in infrared images, its recognition effect is poor. The network contains an initial convolutional layer, four residual layers (each residual layer contains two residual blocks), and a fully connected layer, totaling eighteen groups of network structures. The feature map is forwarded between its layers through residual blocks. This makes the ResNet18 network have a large number of parameters, cumbersome calculations, and slow inference speed. It may cause accidents such as falls to not be recognized and discovered by the system in time, so it is not suitable for actual use.

[0073] For the above two pain points, this paper designs a channel-separated attention residual network ResNet18c_split based on ResNet18. Its structure is as follows Figure 8 shown.

[0074] First of all, in response to the problem of the large number of ResNet18 network parameters, it was found through experiments that it is unnecessary and cumbersome for the original residual network to perform residual calculations on the feature maps of all channels between each residual layer. Therefore, the present invention adopts the idea of ​​channel separation to separate the output feature maps of each residual layer of the residual network into Group A and Group B with the same number of channels, where Group A is sent to the next residual layer to continue feature extraction and residual calculation, while Group B adjusts the dimension of the feature map and extracts shallow features through a simple 1*1 convolution, and then splices the two groups of feature maps in the channel dimension to restore the number of channels. Taking into account the problem that feature maps of different channels need to be fused, the operation of channel shuffling is added after the feature map is spliced ​​to increase the information interaction between different channels. The calculation formula for channel separation between each layer is as follows:

[0075] Out_A,Out_B=Channel_split(input)

[0076] Output=Channel_shuffle(Contact[F(Out_A);Conv2d(Out_B)])

[0077] Among them, Channel_split represents channel separation, Out_A and Out_B represent the two sets of feature maps after separation, F represents the residual calculation of the next layer, Conv2d represents 1*1 convolution, Contact represents splicing in the channel dimension, and Channel_shuffle represents the channel shuffle operation.

[0078] Since each layer of the improved ResNet18 network only performs residual calculations on feature maps with half the number of channels, the amount of calculations and parameters required for the model is greatly reduced, speeding up model reasoning.

[0079] Secondly, in order to solve the problem that the ResNet18 network cannot capture the contour information of infrared images well and the recognition accuracy is reduced after channel separation, the CBAM attention mechanism is introduced in the original residual block. The specific approach is to add a CBAM attention module to the branch that is sent to the next residual layer after channel separation to enhance the network's attention and improve the network's recognition accuracy for seven types of behavioral actions. The CBAM module contains two sub-modules: channel attention (Att_C) and spatial attention (Att_S); channel attention can use a multi-layer perceptron to generate different channel weight vectors, and then multiply the weight vector by the original feature map to increase the importance of the effective channel feature map. The formula for channel attention is as follows:

[0080] Att_C=σ(MLP(GmaxPool(x))+MLP(GavgPool(x)))

[0081] Where x represents input, GmaxPool and GavgPool represent global maximum pooling and global average pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid activation function.

[0082] Spatial attention focuses on the position information of the human target in the feature map, which enables the network to more effectively extract the contour features of the human body. The formula for spatial attention is as follows:

[0083] Att_S=σ(Conv2d(Contact[Maxpool(x);Avgpool(x)]))

[0084] Where x represents the input, MaxPool and AvgPool represent the maximum pooling in the spatial direction and the average pooling in the spatial direction respectively, Conv2d represents two-dimensional convolution, and σ represents the Sigmoid activation function.

[0085] Channel attention and spatial attention are connected in series to form the CBAM attention module, which focuses on important features, suppresses unnecessary features, and enhances the learning efficiency of the network. The formula of the CBAM attention mechanism is as follows:

[0086]

[0087] Where input represents the input of the network, Att_C and Att_S represent channel attention and spatial attention respectively.

[0088] Step 5: Behavior recognition algorithm training:

[0089] Use the infrared human behavior dataset produced in step 3 to train the channel-separated attention residual network ResNet18c_split. The specific method is to first train the ResNet18c_split network with the training set and the validation set until it converges, save the optimal model weight parameters, and then test the optimal model with the test set to obtain the recognition accuracy and confusion matrix of each action, and then obtain the average recognition accuracy of the algorithm.

[0090] For each round of training, the training set is first used as the input of the network, the batch data is set to 32, the initial learning rate is set to 0.001, and the learning rate decays in the trend of the cosine function as the number of training rounds increases, and the lower limit of the decay is set to 0.0001. During the training process, the cross-mulberry loss function is used to obtain the loss error between the output of the model and the true label category, and the Adam optimizer is used to back-propagate the loss error to obtain the model weight parameter that minimizes the loss function. Then the validation set is used as the input of the network to obtain the probability value of the model's prediction of the seven types of actions. The maximum value is taken as the final prediction result, and the number of final prediction results that are the same as the true label category is recorded, and then the accuracy of each round of the model is obtained.

[0091] The model is trained for 200 rounds until it converges and the weight parameters of the optimal model are saved, finally completing the training of ResNet18c_split.

[0092] Step 6: Identification of seven types of behaviors:

[0093] After completing the training of ResNet18c_split, test it separately using the test set. Fig. 9 The following are sample images of seven action categories. “class” indicates the category predicted by the model, and “prob” indicates the probability of the category. Fig. 9 The actual meaning of Figure (a) is that the action category of the infrared image sample is recognized by the model as bending (Bend), and the possibility of correct recognition is 99.9%.

[0094] A human behavior recognition device based on infrared images comprises an infrared sensor, a single-chip microcomputer and a host computer; the infrared sensor transmits data to the single-chip microcomputer via SPI, and the single-chip microcomputer is connected to the host computer via serial port communication.

[0095] Fig.10 and Fig.11The results show that compared with ResNet18, the improved ResNet18c_split not only reaches convergence faster, but also improves the best accuracy on the validation set by 1.5%, the accuracy on the test set by 3.74%, and reduces the number of prediction errors for 588 samples in the test set by 22, which shows the effectiveness of the improvements. Fig.12 The confusion matrix shown in the figure shows that the improved algorithm has improved recognition effects on actions such as falling, standing, lying down, and sitting still, especially the prediction accuracy of dangerous behaviors such as falling has increased by 1.19%, which once again proves the superiority of the method proposed in the present invention.

[0096] In order to make the improved algorithm suitable for actual terminal deployment, the present invention conducts a comparative analysis of the parameters (M) and computational complexity (Mac) of the new model ResNet18c_split and ResNet18. For the same input image of size 3*400*320, the comparison of the computational complexity and parameter complexity of the two models is as follows: Fig.13 Compared with ResNet18, the number of parameters and the amount of calculation of ResNet18c_split are reduced by 65.18% and 65.70% respectively, which shows that the present invention also performs well in lightweight calculation, is convenient for terminal deployment, and meets practical application requirements.

[0097] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for human behavior recognition based on infrared images, characterized in that: The following steps are involved: S1. Acquisition of infrared images: The infrared sensor is driven by a single-chip microcomputer, and the detected temperature data is formed into a 80*64*n digital matrix and transmitted to the single-chip microcomputer for storage through SPI, where n is the total number of frames, and then the temperature digital matrix is ​​transmitted to the PC through UART communication; S2. Preprocessing of infrared images: The image stitching method is used to stitch 25 consecutive frames of images of an action into a large image in chronological order as the input of the system; secondly, the stitched infrared large image is denoised; S3. Preparation of human behavior dataset: Design and collect infrared images of seven kinds of actions: "bending", "falling", "lying flat", "standing", "sitting still", "from standing to sitting", and "walking"; S4. Design of behavior recognition algorithm: Based on ResNet18, we designed the attention residual network ResNet18c_split based on channel separation; S5. Behavior recognition algorithm training: Input the infrared human action dataset into the channel-separated attention residual network ResNet18c_split; S6. Recognition of seven kinds of behaviors: The divided test set is used to test the model after training convergence separately; after each infrared image is predicted by the algorithm, a one-dimensional feature vector containing seven elements will be generated. After being processed by the SoftMax function, the probability values ​​of the seven action categories can be obtained. The label category corresponding to the maximum value is the final prediction result.

2. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: In the S1, the PIL function library in the Python language is used to convert the temperature digital matrix in the txt file into an infrared image for storage; first, the txt file is read to generate a 80*64*n two-dimensional digital matrix, and its number of rows is traversed and then divided by 64 to obtain the total number of frames of the infrared image, and then the array is sliced ​​to obtain n 80*64 two-dimensional array small matrices, and the temperature data is normalized to the range of 0-255 by row, and finally the fromarry function is used to obtain n infrared grayscale images and save them.

3. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: The denoising method in S2 is to traverse all pixel points in the image, and if the point is a noise point or a pixel is missing, the pixel value of the point is replaced by the average value of the eight surrounding pixel points.

4. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: 754 samples are obtained for each action in S3, of which 62.3% are divided into a training set, 26.5% are divided into a validation set, and 11.1% are divided into a test set; the 470 samples in the training set are enhanced by using the rotation angle and horizontal mirror flipping method to triple the number of samples, while the validation set and the test set are not enhanced; the total amount of all data in the processed data set is (470*3+200+84)*7=11858.

5. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: In S4: 4.

1. Adopting the idea of ​​channel separation, the output feature map of each residual layer of the residual network is separated into group A and group B with the same number of channels. Group A is sent to the next residual layer to continue feature extraction and residual calculation, while group B is adjusted by simple 1*1 convolution to adjust the dimension of the feature map and extract shallow features. Then the two groups of feature maps are spliced ​​in the channel dimension to restore the number of channels. After the feature map is spliced, the channel shuffle operation is added to increase the information interaction of different channels. The calculation formula for channel separation between each layer is as follows: Out_A,Out_B=Channel_split(input) Output=Channel_shuffle(Contact[F(Out_A);Conv2d(Out_B)]) Among them, Channel_split represents channel separation, Out_A and Out_B represent the two sets of feature maps after separation, F represents the residual calculation of the next layer, Conv2d represents 1*1 convolution, Contact represents splicing in the channel dimension, and Channel_shuffle represents the channel shuffle operation; 4.

2. Introducing the CBAM attention mechanism in the original residual block: Adding the CBAM attention module to the branch that is sent to the next residual layer after channel separation to enhance the network's attention and improve the network's recognition accuracy for seven behavioral actions; the CBAM module contains two sub-modules: channel attention (Att_C) and spatial attention (Att_S); channel attention uses a multi-layer perceptron to generate different channel weight vectors, and then multiplies the weight vector with the original feature map to increase the importance of the effective channel feature map; the formula for channel attention is as follows: Att_C=σ(MLP(GmaxPool(x))+MLP(GavgPool(x))) Where x represents input, GmaxPool and GavgPool represent global maximum pooling and global average pooling respectively, MLP represents multi-layer perceptron, and σ represents Sigmoid activation function; Spatial attention focuses on the position information of the human target in the feature map, which enables the network to more effectively extract the contour features of the human body shape; the formula for spatial attention is as follows: Att_S=σ(Conv2d(Contact[Maxpool(x);Avgpool(x)])) Where x represents input, MaxPool and AvgPool represent maximum pooling in the spatial direction and average pooling in the spatial direction respectively, Conv2d represents two-dimensional convolution, and σ represents the Sigmoid activation function; Channel attention and spatial attention are connected in series to form the CBAM attention module, which focuses on important features and suppresses unnecessary features. The formula of CBAM attention mechanism is as follows: Where input represents the input of the network, Att_C and Att_S represent channel attention and spatial attention respectively.

6. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: The method of S5 is: firstly train the ResNet18c_split network with the training set and the validation set until it converges, save the optimal model weight parameters, and then test the optimal model with the test set to obtain the recognition accuracy and confusion matrix of each action, and then obtain the average recognition accuracy of the algorithm; For each round of training, the training set is first used as the input of the network, the batch data is set to 32, the initial learning rate is set to 0.001, and the learning rate decays in a cosine function trend as the number of training rounds increases, and the lower limit of the decay is set to 0.0001; during the training process, the cross-Sang loss function is used to obtain the loss error between the output result of the model and the true label category, and the Adam optimizer is used to back-propagate the loss error to obtain the model weight parameter that minimizes the loss function; then the validation set is used as the input of the network to obtain the probability value of the model's prediction of the seven types of actions, and the maximum value is taken as the final prediction result. The number of the final prediction result and the true label category is recorded, and then the accuracy of each round of the model is obtained; Train the model until it converges and save the weight parameters of the optimal model, and finally complete the training of ResNet18c_split.

7. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: The single chip microcomputer model in S1 is STM32F103; the infrared image sensor is Heiman HTPA80*64R2L.

8. The method for human behavior recognition based on infrared images according to claim 1, characterized in that: The human behavior recognition results include: "bending", "falling down", "lying down", "standing", "sitting still", "from standing to sitting", and "walking".

9. A human behavior recognition device based on infrared images, characterized in that: It includes an infrared sensor, a single-chip microcomputer and a host computer; the infrared sensor transmits data to the single-chip microcomputer via SPI, and the single-chip microcomputer is connected with the host computer via serial port communication.

Images