Infrared pedestrian recognition model and recognition method based on attention module

Abstract

The invention discloses an infrared pedestrian recognition model and recognition method based on an attention module. The model comprises a backbone network, a feature fusion network and a detection head network, an SE channel attention module is introduced behind a pooling module of the backbone network, and a SimAM space attention module is introduced before data enters the feature fusion network. The SE module is used for realizing feature weighting and re-calibration in a channel dimension, and the SimAM module is used for enhancing pedestrian target features and weakening background interference in a space dimension. The model is suitable for infrared pedestrian recognition of an embedded platform, and high-precision detection and real-time recognition of a pedestrian target in a low-resolution infrared image are achieved.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing technology, specifically to an infrared pedestrian recognition model and method based on an attention module, applicable to scenarios such as security monitoring, autonomous driving, and nighttime rescue. Background Technology

[0002] In the field of pedestrian detection, traditional methods mainly rely on image data acquired by visible light cameras. However, in adverse environmental conditions such as low light at night and hazy weather, the image quality of visible light deteriorates significantly, leading to a substantial decrease in pedestrian detection accuracy. Although infrared thermal imaging technology can acquire images in low-light environments, the low contrast of infrared images and the blurring or missing texture features of pedestrian targets make it difficult for traditional detection algorithms based on manually designed features to effectively extract discriminative features, easily leading to false positives and false negatives.

[0003] Existing pedestrian detection methods mostly rely on manual feature extraction combined with classifiers for target recognition. This approach not only suffers from low feature extraction efficiency and limited generalization ability, but also struggles to adapt to complex and ever-changing real-world environments. In dynamic scenes, pedestrian targets in infrared images are easily confused with vehicles, animals, and other heat-generating objects, further reducing recognition accuracy.

[0004] In recent years, deep learning-based detection algorithms have been widely studied in the field of infrared pedestrian recognition. However, there are still shortcomings in the modeling of the importance of different feature channels, which prevents the model from fully focusing on the feature regions that are more critical for recognition. Therefore, how to effectively enhance the model's ability to focus on key features in infrared images by introducing an improved attention module, thereby improving the accuracy and robustness of infrared pedestrian recognition, has become an urgent technical problem to be solved in this field. Summary of the Invention

[0005] Technical issues

[0006] This invention is proposed to solve the above-mentioned problems, and its purpose is to provide an infrared pedestrian recognition model and recognition method based on an attention module.

[0007] Existing infrared pedestrian detection models are mostly based on general target detection networks (such as YOLOv5). However, due to problems such as blurred pedestrian edges and significant background thermal interference in low-resolution, low-contrast, and high-noise environments, traditional convolutional feature extraction structures struggle to accurately distinguish pedestrian targets from background regions, leading to decreased recognition accuracy and insufficient real-time performance. Especially on embedded edge computing platforms, the models suffer from high computational overhead and slow detection response, failing to meet the requirements of low power consumption and high timeliness in infrared scenarios.

[0008] Therefore, it is necessary to provide an improved infrared pedestrian recognition model at the structural and algorithmic levels to enhance the recognition accuracy and real-time detection performance of small infrared targets under limited computing resources.

[0009] Technical solution

[0010] According to an embodiment of the present invention, an infrared pedestrian recognition model based on an attention module is provided, characterized in that it includes:

[0011] Backbone: Used to extract multi-level semantic features from the input infrared image;

[0012] Feature Fusion Network (Neck): Used to fuse feature maps of different scales to extract contextual information;

[0013] Head network: Used to output the category and location coordinates of pedestrian targets;

[0014] The feature is that a channel attention module is introduced after the pooling module in the backbone network, and a spatial attention module is introduced before the data enters the feature fusion network.

[0015] The channel attention module is the SE channel attention module, which is used to perform feature weighting and recalibration in the channel dimension to highlight highly relevant channel features and suppress channel-level noise;

[0016] The spatial attention module is the SimAM spatial attention module, which is used to weight the feature map in the spatial dimension to enhance the pedestrian target area and reduce background interference.

[0017] The channel attention module uses 1×1 convolutional layers instead of fully connected layers to reduce dimensionality transformation overhead and improve the operating efficiency of the embedded platform while maintaining the channel attention learning function.

[0018] The spatial attention module introduces a background weakening weight coefficient K during the energy weight calculation process, where 0.5 ≤ K ≤ 0.8, to weaken the background noise response and enhance the salience of the pedestrian target area.

[0019] According to another embodiment of the present invention, an infrared pedestrian recognition method based on the above-described infrared pedestrian recognition model is provided, characterized by comprising the following steps:

[0020] Infrared image acquisition and preprocessing steps:

[0021] Infrared image data is acquired through an infrared sensor, and the infrared images are then processed by format conversion, grayscale equalization, and noise filtering.

[0022] Dataset creation and model training steps:

[0023] Infrared image samples containing pedestrian targets are collected, labeled, and training and validation sets are constructed.

[0024] During the training process, based on the YOLOv5 network structure, the above-mentioned infrared pedestrian recognition model is used for feature learning to generate the trained model file;

[0025] Model conversion steps:

[0026] The model files trained using PyTorch are converted into Rknn model files suitable for embedded platforms using the Rknn toolkit.

[0027] Deployment and real-time detection steps:

[0028] The RKNN model is deployed on an embedded platform to perform real-time detection and recognition of the input infrared images, and output detection results containing pedestrian target boxes and category information.

[0029] Output and display steps:

[0030] The detection results are filtered, scaled, denoised, and image optimized, and the final results are output to the terminal device screen in real time. Combined with warning icons or voice prompts, real-time monitoring and warning of pedestrian targets are achieved.

[0031] Beneficial effects

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] Significant structural improvements have been achieved: An SE channel attention module is introduced after the SPP module, and a SimAM spatial attention module is introduced before entering the feature fusion network, forming a "channel-space" two-layer attention structure. This structure can more accurately focus on pedestrian target features in infrared images, improving the model's feature extraction capability and recognition robustness.

[0034] Algorithm optimization and enhanced embedded adaptability: By optimizing the SE module structure by replacing fully connected layers with 1×1 convolutional layers, the number of parameters and computational load are reduced, significantly improving the parallel efficiency and real-time performance of the model on embedded platforms such as NPU and FPGA.

[0035] Enhanced noise resistance and low contrast characteristics: By introducing a background weakening weight coefficient K into the SimAM spatial attention module, weighted suppression of low-response background regions is achieved, enhancing the saliency features of small infrared targets and distant pedestrians, and improving the recognition accuracy of low-resolution infrared images.

[0036] The system performs exceptionally well: the model structure of this invention is clear, has a low number of parameters, is easy to deploy, and can achieve high frame rate (FPS) real-time detection in an infrared thermal imaging embedded platform. It also features high precision, low power consumption, and good practicality. Attached Figure Description

[0037] Figure 1 A flowchart illustrating an infrared pedestrian recognition method based on an attention module according to an embodiment of the present invention is shown.

[0038] Figure 2A The network structure of the original model is shown;

[0039] Figure 2B The network structure of the improved recognition model according to an embodiment of the present invention is shown;

[0040] Figure 3 A flowchart illustrating the recognition model according to an embodiment of the present invention is shown;

[0041] Figure 4 A flowchart illustrating the novel channel attention module algorithm according to an embodiment of the present invention is shown; and

[0042] Figure 5 A flowchart illustrating a novel spatial attention module algorithm according to an embodiment of the present invention is shown. Detailed Implementation

[0043] The embodiments of this disclosure will now be described with reference to the accompanying drawings. In the drawings, the same reference numerals are used for the same or corresponding structural elements. Furthermore, in the following description of the embodiments, repeated descriptions of the same or corresponding structural elements may be omitted. However, even if the description of structural elements is omitted, it does not mean that those structural elements are not included in a particular embodiment.

[0044] Although the process steps, method steps, and algorithms illustrated in the flowcharts of this specification are described in sequence, these processes, methods, and algorithms can be configured to operate in any suitable order. In other words, the steps of the processes, method steps, and algorithms described in the various embodiments of this disclosure do not need to be executed in the order described in this disclosure.

[0045] Furthermore, even if some steps are described as not being performed simultaneously, these partial steps may be performed simultaneously in other embodiments. Moreover, the examples of processes described with reference to the accompanying drawings do not imply that the illustrated processes exclude other variations and modifications thereof, and do not imply that any of the illustrated processes or their steps are essential to one or more embodiments of the various embodiments of this disclosure, nor do they imply that the illustrated processes are preferred.

[0046] The structure and method of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Figure 1 A flowchart illustrating an infrared pedestrian recognition method using an attention-based pedestrian recognition model according to an embodiment of the present invention is shown.

[0048] like Figure 1 As shown, the infrared pedestrian recognition method using an attention-based pedestrian recognition model proposed in this invention includes the following steps:

[0049] Infrared sensor:

[0050] As the system's input, it is used to acquire infrared image data, serving as the raw image source for pedestrian recognition.

[0051] Infrared image preprocessing:

[0052] The input infrared images undergo preprocessing operations such as format conversion, interface integration, and image enhancement to meet the input requirements of subsequent recognition models.

[0053] Embedded platform image channel integration and data acquisition:

[0054] The infrared image input signal is connected to the image acquisition channel of the embedded platform to realize real-time acquisition and transmission of image data and complete data format matching.

[0055] Pedestrian target data collection:

[0056] Infrared image data containing pedestrian targets are collected through real-world shooting, forming the data foundation for model training.

[0057] Dataset creation:

[0058] The collected pedestrian images were labeled using labeling software to generate training label files required for pedestrian detection, thus constructing a training dataset.

[0059] Specifically, in this embodiment of the invention, a large amount of image data containing pedestrian targets is first acquired for the application scenario. The image data is not limited to the gender, age, and physical characteristics of the pedestrians. Since this invention requires real-time and accurate acquisition and identification of pedestrian targets in infrared images, all acquired image data containing pedestrian targets must be labeled to ensure that the subsequent model can accurately identify pedestrians under different feature conditions.

[0060] Furthermore, the resolution of the image data can be set to 1:1, and the number of pixels can be an integer multiple of 32 to meet the size requirements of the deep learning model input. Regarding the amount of data, small-sample experiments should include at least 200 images to ensure basic training effectiveness; in practical applications, to obtain higher recognition accuracy, it is necessary to collect tens of thousands of image data points to provide sufficient data support for model training and optimization.

[0061] In an embodiment of the present invention, once the image data collection is complete, the dataset can be created. Specifically, the target recognition regions in the dataset are identified using the labelimg tool, and a saving operation is performed after the identification is completed, thereby generating a file containing the coordinate information of the recognition regions. This coordinate file is required during subsequent model training. The specific operation steps are as follows:

[0062] Step 1: Execute the command "pip install labelimg" to install the labelimg tool for creating labels.

[0063] Step 2: Open the labelimg tool and create a new folder named "label" in the image data directory "photo" to store the created label files.

[0064] Step 3: In the labelimg tool interface, open the folder containing the image and set the target directory for saving the label.

[0065] Step 4: Create a new folder named "num" and create subfolders within it; then copy the images to the "num / train / images" and "num / valid / images" directories respectively; at the same time, copy the generated label files (i.e., the txt files in the "photo / label" directory) to the "num / train / labels" and "num / valid / labels" directories respectively.

[0066] Step 5: Copy the file "coco128.yaml" from the "yolov5-5.0\data" directory to the "num" folder and rename it to "num.yaml".

[0067] Step 6: Modify the “num.yaml” file to adapt it to the training requirements of the dataset.

[0068] Step 7: At this point, the dataset is complete, and the "num" folder constitutes the dataset used in this invention.

[0069] Generate model files:

[0070] Two improved attention modules are introduced into the YOLOv5 network structure and embedded at specific network layer positions to obtain an optimized infrared pedestrian recognition model through machine training.

[0071] Specifically, in this embodiment of the invention, once the dataset is created, the model can be trained to generate a model file for target recognition. The trained model file is in ".pt" format, and the model corresponding to the training result can only recognize targets in scenes similar to the dataset. The model training process can be divided into two stages: setting up the training environment and training. Assuming the training environment has been set up, the specific training steps include:

[0072] Step 1: Modify the "yolov5-5.0 / train.py" file to suit the training requirements of the dataset.

[0073] Step 2: Modify the "yolov5-5.0 / models / yolov5s.yaml" file to configure the model structure and parameters.

[0074] Step 3: After completing the above file modifications, execute the command "python train.py" to start the training process.

[0075] Step 4: Wait for the training process to finish. Once training is complete, the system will indicate the storage location of the generated model file. You can use the "best.pt" file when needed.

[0076] Step 5: Test the model. Specifically, execute the command "python detect.py --source.. / a --weights .. / best.pt"; where "--source" specifies the directory where the test images are located, and "--weights" specifies the model file generated during training.

[0077] Step 6: Check the test results. When the recognition effect meets expectations, it means that the training model is successful.

[0078] In embodiments of the present invention, a pedestrian recognition model generation method based on an improved attention module is proposed to address the problem of significant differences in infrared image features. Since different feature channels in infrared images contribute differently to pedestrian recognition—some channels may be more sensitive to edge information while others are more sensitive to the center region of the heat source—it is necessary to optimize the feature extraction structure of the model to improve recognition accuracy and robustness.

[0079] To improve the model's recognition performance and computational accuracy, this embodiment introduces an attention module into the YOLOv5 network structure. Through repeated experiments and performance comparisons, a Squeeze-and-Excitation Networks (SE) channel attention module and a SimAM (Simple Attention Module) spatial attention module were added to the model, and their embedding order and combination method within the network structure were determined. Based on the different mechanisms of action of the attention modules and the characteristics of infrared images, structural modifications and parameter optimizations were performed, resulting in two new attention algorithm schemes. Retraining based on the improved model structure significantly enhances the feature extraction capability and real-time detection accuracy of the infrared pedestrian recognition model, while reducing computational overhead.

[0080] The SE and SimAM modules are designed to weight and filter feature information from both "channel" and "spatial" dimensions, respectively. Their combination forms a complementary relationship, effectively overcoming the limitations of a single attention module. Specifically, compared to the spatial attention structure in the CBAM (Convolutional Block Attention Module), the SimAM module has a more refined spatial filtering capability, accurately capturing weak pedestrian target signals in infrared images at the pixel level. While the SE module's modeling effect in the channel dimension is similar to that of the channel attention module in CBAM, in the combined structure of this invention, its performance in spatial information extraction is superior when working in conjunction with the SimAM module.

[0081] Furthermore, compared to attention modules such as GAM (Global Attention Mechanism) and SKNet (Selective Kernel Networks) which have complex structures and numerous hyperparameters, the SE+SimAM proposed in this invention has advantages such as clear structure, fewer parameters, and lower computational overhead. While ensuring recognition accuracy, the proposed structure is more suitable for deployment on edge computing devices, meeting the real-time requirements of infrared applications and significantly improving the accuracy of infrared small target or long-distance pedestrian detection.

[0082] In this embodiment, considering the computational resource limitations of the embedded platform and the characteristics of infrared thermal imaging images and videos, the optimal embedding order of the attention modules was determined through experimental comparison, and a new infrared pedestrian recognition model was formed based on this. Subsequently, structural modifications were made based on the SE channel attention module and the SimAM spatial attention module, respectively, and two new attention algorithm schemes were proposed and formed. The relevant improvement details will be further explained in subsequent embodiments.

[0083] Figure 2A The network structure of the original model is shown. Figure 2B The network structure of the improved new recognition model is shown.

[0084] like Figure 2A As shown, this is the network structure of the original YOLOv5 model in an embodiment of the present invention; Figure 2B As shown, this is the new recognition model network structure formed after introducing an improved attention module in an embodiment of the present invention.

[0085] Based on the original YOLOv5 network, this invention modifies the Backbone structure by introducing a channel attention module and a spatial attention module between the SPP module and the C3 module to enhance the network's ability to express infrared image features and its ability to focus on pedestrian targets.

[0086] The Backbone is a feature extraction network used to extract multi-level semantic features from the input infrared image; the Neck network is located between the Backbone and the Head network and is used to fuse feature map information at different scales to extract more complex contextual features; the Head is the detection output layer used to predict the target's category and location coordinates.

[0087] Specifically, Input is the image input layer; the Focus module spatially reorganizes the input image through slicing operations to retain more detailed information; the Conv module is the basic unit of the convolutional neural network, consisting of convolutional layers, batch normalization (BN) layers, and activation functions; the C3 module consists of multiple Conv blocks, used to extract deeper features; the SPP (Spatial Pyramid Pooling Module) module adopts a multi-scale pooling kernel structure to enhance the receptive field; the Concat module concatenates feature maps from different layers along the channel dimension, thereby enriching the feature representation; the Detect module is the detection output layer, containing three detection channels, used to predict targets of different scales.

[0088] In the original YOLOv5 architecture, the feature extraction and detection parts were not optimized for the low contrast and noise characteristics of infrared images, resulting in limited detection accuracy in low-resolution infrared scenes. Therefore, this invention enhances the architecture by improving the attention module:

[0089] First, considering the application characteristics of low-resolution infrared thermal imaging, a combined structure of "channel attention first, then spatial attention" is proposed, with an attention module added at the P5 layer (1 / 32 scale) after reducing the high-resolution feature map. Experiments show that this embedding method has higher computational efficiency and can significantly improve the model's recognition ability while ensuring real-time performance.

[0090] Secondly, the SPP module performs multi-scale pooling and fusion of the infrared feature map of the P5 layer (1 / 32 scale) by setting multi-size pooling kernels (5×5, 9×9, 13×13). Its core function is to solve the scale adaptation problem in the infrared scene and provide rich contextual information for the subsequent channel attention module.

[0091] Next, the SE channel attention module sorts the importance of the 1024-dimensional feature channels output by SPP and automatically assigns weights, thereby highlighting highly relevant pedestrian features and suppressing channel-level noise, effectively addressing the issues of channel redundancy and low signal-to-noise ratio in infrared images.

[0092] Subsequently, the C3 module performs deep fusion on the selected high-quality features and further refines the infrared feature purification through a residual-free structural design, thereby enhancing the model's sensitivity to fine-grained thermal targets.

[0093] Finally, the SimAM spatial attention module further weights features in the spatial dimension, enabling it to locate key pedestrian regions within a pixel-level range and achieve precise spatial focusing. The SimAM spatial attention module possesses a more refined pixel-level filtering capability than CBAM spatial attention, effectively extracting weak target signals from complex backgrounds.

[0094] This improved configuration, while maintaining a clear model structure and low parameter count, balances the recognition performance and computational efficiency of the attention module, significantly enhancing the accuracy and robustness of infrared small target and long-distance pedestrian detection. Compared to attention modules with complex structures and numerous hyperparameters, such as GAM and SKNet, the proposed combination of the SE channel attention module and the SimAM spatial attention module is more suitable for deployment in embedded edge computing devices, meeting the real-time and energy consumption requirements of infrared pedestrian recognition.

[0095] Figure 3 A flowchart of an identification model according to an embodiment of the present invention is shown.

[0096] like Figure 3 As shown, the infrared pedestrian recognition model based on an improved attention module proposed in this embodiment of the invention includes the following main steps:

[0097] Input image:

[0098] Images acquired by the infrared thermal imaging module are input into the model as the raw data source for pedestrian recognition.

[0099] Focus module:

[0100] The input image is sliced ​​and convolutionally processed to achieve feature recombination and information compression, with an output size of P1 / 2 and 64 channels. This module's function is to preserve subtle thermal features in the infrared image, providing a foundation for subsequent convolutional layers to extract deeper features.

[0101] Conv convolutional layer (layer 1):

[0102] Downsampling is achieved using a convolution operation with a stride of 2, resulting in an output size of P2 / 4 and 128 channels, which are used to extract basic edge and texture features.

[0103] C3 Module (Level 1):

[0104] Perform three Bottleneck operations to output a 128-channel feature map, which is used to enhance the expression of local features.

[0105] Conv convolutional layer (layer 2):

[0106] Convolutional downsampling with a stride of 2 is used to extract mid-level semantic features. The output size is P3 / 8 and the number of channels is 256.

[0107] C3 module (Level 2):

[0108] Perform 9 Bottleneck operations to output a 256-channel feature map to capture mesoscale features such as pedestrian contours and poses.

[0109] Conv convolutional layer (layer 3):

[0110] Convolutional downsampling with a stride of 2 is used, resulting in an output size of P4 / 16 and 512 channels.

[0111] C3 module (Level 3):

[0112] Perform 9 Bottleneck operations to output a 512-channel feature map, which is used to extract higher-level semantic features.

[0113] Conv convolutional layer (4th layer):

[0114] Convolutional downsampling with a stride of 2 is used, with an output size of P5 / 32 and 1024 channels, providing input for the deep feature extraction stage.

[0115] SPP module (Space Pyramid Pooling Module):

[0116] Multi-scale fusion of feature maps from layer P5 is performed using multi-scale pooling kernels (5×5, 9×9, 13×13). The core function of this module is to expand the receptive field and supplement multi-scale contextual information in infrared images, thereby improving the robustness of recognizing pedestrians at different distances and of different body sizes.

[0117] SE Channel Attention Module:

[0118] The 1024-dimensional feature channels output by the SPP module are recalibrated using a compression ratio of 16, and the weights of each channel are adaptively allocated to highlight the highly correlated channel features in the infrared image and suppress channel noise.

[0119] C3 module (Level 4):

[0120] The features output by the SE module are fused again through three Bottleneck operations, using a non-jump connection structure to improve the ability to distinguish subtle features in infrared images.

[0121] SimAM Spatial Attention Module:

[0122] Perform parameter-free recalibration of features in the spatial dimension.

[0123] Backbone output characteristics:

[0124] The output contains 1024 channels of high-level semantic features, completing the feature extraction stage and providing input for the subsequent object detection head module.

[0125] Output image:

[0126] The model maps the pedestrian bounding boxes and their category information to the original infrared image and outputs the recognition results, realizing real-time pedestrian detection and recognition in infrared scenes.

[0127] In embodiments of the present invention, the SE channel attention module is a channel attention mechanism. Its main function is to enhance the channel features of the input feature map by adaptively learning the importance weights of different channels without changing the spatial size of the input feature map. This module can effectively highlight highly correlated pedestrian target features in infrared images and suppress background channel noise, thereby improving feature representation ability and pedestrian detection accuracy.

[0128] The standard processing procedure for the SE channel attention module includes the following steps:

[0129] Convolutional feature extraction

[0130] Let the input feature map be X, and the output feature map be U after convolution. The relationship between the input and output is defined as follows:

[0131]

[0132] in,

[0133] X represents the input feature map, and U represents the output feature map after the convolution operation.

[0134] F tr This represents the convolution transformation function from the input feature map to the output feature map.

[0135] →: Function mapping symbol

[0136] R: Represents the real number field.

[0137] W: Width of the input feature map.

[0138] H: The height of the input feature map.

[0139] C: The number of channels in the input feature map.

[0140] The mapping process from X to U is a convolution calculation process, and its mathematical expression is as follows:

[0141]

[0142] in,

[0143] This represents the c-th channel of the output feature map.

[0144] This represents the c-th convolutional kernel.

[0145] This represents the s-th input covered by the current convolutional kernel.

[0146] This represents the sub-convolution kernel of the s-th channel of the c-th convolution kernel.

[0147] This indicates the convolution and the number of elements.

[0148] The asterisk (*) represents the convolution operator.

[0149] Through the above convolution operation, a feature map U with dimensions [H,W,C] can be obtained. This feature map contains spatial and semantic information after preliminary feature extraction.

[0150] Fsq operation (Squeeze phase)

[0151] The Fsq operation, or Global Average Pooling operation, is used to compress information globally along the channel dimension. Its calculation formula is as follows:

[0152]

[0153] in,

[0154] This represents the compressed features after average pooling.

[0155] This represents the feature map of the c-th channel of the input.

[0156] W: Input feature map The width.

[0157] H: Input feature map The height.

[0158] i,j: Input feature maps The index.

[0159] The results of this step reflect the statistical distribution of the features of each channel in the current layer on a global scale, thereby achieving global information aggregation.

[0160] Fex operation (Excitation phase)

[0161] The Fex operation is the channel weight generation stage, and its main function is to learn the importance weights of different channels through a two-layer fully connected network. Its calculation formula is as follows:

[0162]

[0163] in,

[0164] sigmoid: Represents an activation function that maps the input to the (0, 1) interval and is used to normalize the weights.

[0165] and : These are the weight matrices of the two fully connected layers.

[0166] ReLU: An activation function used to introduce non-linearity between fully connected layers.

[0167] Compressed features after average pooling.

[0168] The output s of this operation is a channel-dimensional weight vector, used to represent the weight of each channel.

[0169] Because this process can be trained end-to-end, the model can automatically learn the dependencies between channels during training.

[0170] Fscale operation (recalibration phase)

[0171] After obtaining the channel weight vector s, a channel weighting operation is performed on the original feature map U. The calculation formula is as follows:

[0172]

[0173] It is the feature map after the recalibration stage.

[0174] in, This represents a channel in u. This represents the attention weight of the channel.

[0175] Therefore, this operation enables adaptive scaling of channel features, allowing the network to focus on high-information feature channels in infrared images and reduce invalid or redundant channels.

[0176] This embodiment proposes an improved channel attention algorithm that can replace the traditional SE channel attention module. By replacing the fully connected layers (Linear layers) in the SE channel attention module with 1×1 convolutional layers, this algorithm achieves greater adaptability to the tensor flow of convolutional neural networks (CNNs) while maintaining mathematical equivalence. This improvement retains the channel attention learning function of the SE module while significantly reducing the dimensionality transformation overhead, thereby improving computational efficiency and real-time performance on embedded platforms.

[0177] The improved solution has the following technical advantages:

[0178] Maintaining equivalence: A 1×1 convolutional layer is mathematically equivalent to a fully connected layer and can achieve the same linear transformation function;

[0179] Reduced computation: 1×1 convolution avoids explicit dimension unrolling operations, reducing memory access and the number of parameters;

[0180] Hardware friendliness: Convolution operations have higher parallel efficiency on embedded platforms (such as NPU, FPGA);

[0181] Superior real-time performance: It is particularly suitable for scenarios with high requirements for response speed, such as infrared image target recognition and nighttime pedestrian detection.

[0182] In summary, this improved channel attention algorithm significantly enhances the computational efficiency and real-time performance of the model while maintaining feature representation capabilities, making it particularly suitable for embedded deployments in infrared scenarios.

[0183] Figure 4 A flowchart illustrating the novel channel attention module algorithm according to an embodiment of the present invention is shown.

[0184] like Figure 4 The diagram shows a flowchart of the novel channel attention module algorithm proposed in this invention. The main processing steps of this module include the following:

[0185] Input feature image:

[0186] The feature map obtained from the previous steps is input into the new channel attention module proposed in this embodiment.

[0187] Global average pooling and max pooling:

[0188] Global average pooling and max pooling operations are performed on the input feature map to extract global statistical features and retain salient responses. By combining these two pooling methods, both the average trend and extreme value information of the features can be captured simultaneously, providing a more comprehensive channel description for subsequent weight calculations.

[0189] In the traditional SE channel attention module, the pooling result is first input to the first fully connected layer (fully connected layer 1) to compress the channel dimension; then the output result is dimensionally transformed and nonlinearly mapped by the ReLU activation function; then it is input to the second fully connected layer (fully connected layer 2) to restore the channel dimension, thereby generating the channel-level weight response.

[0190] The novel channel attention algorithm pathway in this embodiment of the invention may include:

[0191] 1×1 Convolutional Layer 1: Used to compress and fuse information between channels while maintaining spatial resolution. This convolutional operation is mathematically equivalent to the first fully connected layer (Linear layer) of the SE module, but it does not require dimension unrolling, thus maintaining the continuity of the tensor structure and reducing computational and storage overhead.

[0192] ReLU activation function:

[0193] The output of the 1×1 convolutional layer 1 is input into the ReLU activation function to enhance feature representation through nonlinear mapping. This process can enhance the salient channel response related to pedestrian targets and suppress low-response background channel features, providing a more discriminative input for subsequent feature reconstruction.

[0194] 1×1 Convolutional Layer 2: The features after ReLU activation are input into a second 1×1 convolutional layer to restore the channel dimension and reconstruct the features. This convolutional layer is functionally equivalent to the second fully connected layer of the SE module, which can learn the dependencies between channels and perform reweighted modeling of important features.

[0195] Feature fusion: The features output from the 1×1 convolutional layer 2 are fused with the original input features to obtain the final channel-weighted features. The fused result can be further normalized using a sigmoid or other activation function to generate the final channel weight coefficients, achieving adaptive adjustment of the channel-level attention distribution.

[0196] Sigmoid activation and feature weighting:

[0197] The fused feature results are input into the Sigmoid activation function to obtain the weight coefficients for each channel. The normalization effect of the Sigmoid function ensures that the weight values ​​are distributed between 0 and 1, enabling dynamic adjustment of the features of each channel.

[0198] Output phase:

[0199] Finally, the weighted feature map is output to the subsequent infrared pedestrian recognition network. At this point, the response of the pedestrian target-related channel is significantly enhanced, while the background noise channel is effectively suppressed, thereby improving the model's ability to capture salient pedestrian features.

[0200] Through the above improvements, the new channel attention module of the present invention significantly reduces the number of model parameters and computational overhead while maintaining the feature enhancement capabilities of the original SE channel attention module; at the same time, it achieves higher frame rate (FPS) and lower latency in the embedded inference environment.

[0201] In infrared pedestrian recognition experiments, this structure demonstrated superior real-time detection performance and energy efficiency, effectively meeting the comprehensive performance requirements of infrared thermal imaging systems for high timeliness, high accuracy, and low power consumption.

[0202] In embodiments of the present invention, the SimAM spatial attention module is a spatial attention mechanism for convolutional neural networks. This module achieves adaptive enhancement in the spatial dimension by measuring and weighting the importance of each neuron in the feature map, thereby improving the localization accuracy and feature representation capability of pedestrian targets in infrared images.

[0203] The typical processing procedure of the SimAM spatial attention module is as follows:

[0204] Energy function definition

[0205] Neurons exhibiting spatial inhibition effects should be assigned higher importance. To identify these neurons, the SimAM module defines the following energy function by calculating the linear separability of a target neuron with other neurons in the same channel:

[0206]

[0207] in,

[0208] : Represents the energy function of the target neuron, used to measure the linear separability of the target neuron with other neurons in the same channel. The larger the value, the higher the importance of the neuron.

[0209] The weight parameters corresponding to the target neuron are used to weight the input features in the linear transformation.

[0210] : The bias parameter corresponding to the target neuron, used to shift the weighted features in the linear transformation.

[0211] The output of the neuron after linear transformation.

[0212] : The input features of the i-th other neuron in the same channel.

[0213] : Output of the target neuron.

[0214] : The classification threshold of the target neuron.

[0215] The total number of neurons within the same channel.

[0216] : The output of other neurons after linear transformation.

[0217] : Predicted values ​​from other neuron inputs.

[0218] Energy minimization and simplification

[0219] The goal of minimizing the above energy function is to train the linear separability of neuron t with other neurons in the same channel. For ease of calculation, binary labels are introduced and a regularization term is added to prevent overfitting. The formula for the energy function can be simplified to:

[0220]

[0221] in, This is the regularization coefficient.

[0222] Analytical solution of energy function

[0223] For each channel, the SimAM module calculates the corresponding energy function. Through analytical derivation, the energy function can be obtained as follows:

[0224]

[0225] in,

[0226] This represents the variance of the input features of all non-target neurons within the same channel.

[0227] This is the mean of the input features of all non-target neurons within the same channel.

[0228] Minimum energy and importance weight calculation

[0229] Based on the above analysis results, the importance weights of neurons can be determined through... Represented by the minimum energy function:

[0230]

[0231] This formula shows that the lower the energy, the greater the difference between neuron t and its neighborhood, and the higher its importance. SimAM achieves pixel-level spatial attention modeling through this energy-based weight allocation.

[0232] Feature enhancement and output

[0233] Based on the above definition of attention weights, spatial weighted enhancement is performed on the feature maps, and the calculation formula is as follows:

[0234]

[0235] in,

[0236] : Input feature map.

[0237] : Output feature map after spatial weighting enhancement.

[0238] sigmoid: An activation function that maps the input to the interval (0, 1).

[0239] E: The energy value of a neuron, which measures the importance of its spatial location.

[0240] Hadamard product: This means multiplying the spatial attention weights with the corresponding elements of the original feature map X to achieve spatial weighted enhancement of the feature map.

[0241] Through this weighted operation, the SimAM spatial attention module can enhance the response of the target region in the infrared image at the pixel level and suppress irrelevant background noise, thereby improving the overall feature representation capability of the model.

[0242] By employing the aforementioned SimAM spatial attention module, this invention enables spatial dimension attention weighting without introducing additional trainable parameters.

[0243] Compared to traditional CBAM spatial attention modules, the SimAM module offers higher pixel-level selection accuracy and stronger interpretability, effectively distinguishing pedestrians from background areas in infrared images. This module features a lightweight structure, low computational complexity, and high inference efficiency, making it particularly suitable for infrared pedestrian detection and recognition tasks in embedded platform environments.

[0244] In an embodiment of the present invention, in order to further improve the feature focusing capability of the infrared pedestrian recognition model in low-resolution scenes, the original SimAM spatial attention module is optimized and modified to address the problems of low contrast, small targets and noise interference. An improved spatial attention module algorithm that introduces a background weakening coefficient K is proposed.

[0245] The key problem with low-resolution infrared images is that pedestrian targets have blurred details, and the temperature difference between them and the background is easily masked by noise. In particular, the background area often contains a lot of thermal noise (such as temperature fluctuations of the background wall), which may be misidentified as target features in the unified difference calculation of the original SimAM mechanism.

[0246] To address the challenges of low contrast, small targets, occlusion, and noise interference in infrared pedestrian recognition, an improved scheme based on "amplifying pedestrian target features and suppressing background noise" is proposed. Specifically, a background weakening weight coefficient K is introduced into the SimAM spatial attention mechanism. By weakening irrelevant background information, the above-average pedestrian temperature features are amplified, effectively strengthening the most discriminative and significant temperature information in the infrared image. This improves the model's matching degree to task characteristics, making the network focus more on the heat source region representing the pedestrian, rather than the background portion that is not a task target.

[0247] Based on experimental results, the recommended range for the background weakening coefficient K is 0.5–0.8. When K is less than this range, pedestrian feature information in the infrared image may be significantly lost; when K is greater than 0.8, temperature information of pedestrians and background noise may be mixed.

[0248] Figure 5 A flowchart illustrating a novel spatial attention module algorithm according to an embodiment of the present invention is shown.

[0249] The following is combined Figure 5 This paper provides a detailed description of the infrared pedestrian recognition method based on the improved SimAM spatial attention module proposed in this invention. It should be understood that the embodiments of this invention are only for illustrating the technical solutions of this invention and are not intended to limit the invention.

[0250] like Figure 5 The diagram shown is a flowchart of the improved spatial attention module algorithm proposed in this invention. This module mainly includes the following steps:

[0251] Input feature image:

[0252] The image processed in the previous steps is input into the improved SimAM spatial attention module.

[0253] Calculate the mean of the spatial dimension of the feature map:

[0254] The spatial dimension of the input feature image is averaged to obtain the global average value.

[0255] Calculate the squared deviation of the characteristic from the mean:

[0256] The squared deviation between each pixel value in the feature image and the spatial mean is calculated to characterize the degree of difference of each pixel relative to the global temperature distribution. The larger the squared deviation value, the higher the saliency of the pixel, which may correspond to a pedestrian target area.

[0257] Calculate the base values ​​for attention weights:

[0258] Based on the aforementioned squared deviation results, the initial basic value of the attention weight is calculated, which reflects the response intensity of each pixel in the spatial attention module.

[0259] Add background weakening factor:

[0260] A background weakening weight coefficient K (0.5≤K≤0.8) is introduced into the calculated attention weight base value. By applying this coefficient to the low response region, the background information is weakened by weighting, thereby effectively suppressing thermal noise interference in non-pedestrian areas and further highlighting pedestrian temperature characteristics that are higher than the mean.

[0261] When the K value is less than 0.5, the pedestrian region features may be excessively weakened, leading to a decrease in recognition performance; when the K value is greater than 0.8, the pedestrian and background temperature features may overlap, affecting the model's discrimination accuracy.

[0262] Sigmoid activation feature weighting:

[0263] The attention weights, after incorporating a background weakening coefficient, are input into a sigmoid activation function to perform non-linear weighting on the feature map. The smooth mapping provided by the sigmoid function confines the feature weights between 0 and 1, thereby enhancing the stability and continuity of the attention distribution.

[0264] Output characteristics:

[0265] The output feature map after Sigmoid activation serves as the final result of the improved spatial attention module. In the output features, the response in the pedestrian region is significantly enhanced, while noise in the background region is effectively suppressed, providing higher-quality input features for subsequent infrared pedestrian detection and classification.

[0266] Through the above steps, the improved spatial attention algorithm proposed in this embodiment can significantly suppress background thermal noise in low-resolution infrared images, improve the contrast between pedestrian targets and the background, strengthen the weak signals in the pedestrian edge region, and enable the model to have stronger spatial screening capabilities at the pixel level, thereby achieving high-precision recognition of pedestrians in low-contrast, small target and occluded conditions.

[0267] Reference Figure 1 After completing the steps described above to generate the model file, perform model conversion and subsequent processing steps.

[0268] Model conversion:

[0269] The model trained using PyTorch is converted into an rknn model file suitable for deployment on an embedded platform using the rknn toolkit.

[0270] Deploy the algorithm and perform real-time identification:

[0271] The transformed model is deployed to an embedded platform to detect and identify pedestrian targets in the input infrared image in real time, and an identification box is superimposed on the output image to display the detection results.

[0272] Target image optimization processing:

[0273] The recognition results are processed through image optimization techniques such as filtering, scaling, and denoising to improve the clarity and visual performance of the output image.

[0274] Real-time output of image results:

[0275] The recognition results are output in real time and displayed on the terminal device screen in combination with warning icons or voice prompts, enabling real-time monitoring and warning of pedestrian targets.

[0276] The above has been combined Figure 1 This paper fully describes an infrared pedestrian recognition method based on an improved attention module. In infrared thermal imaging applications, particularly for low-resolution infrared images and videos, and under limited platform computing resources, this invention, based on a conventional network model, integrates two attention modules for infrared image target recognition, forming a new network structure model. This model is structurally modified based on the SE channel attention module and the SimAM spatial attention module, proposing two schemes: a new channel attention algorithm and a spatial attention algorithm.

[0277] The proposed channel attention algorithm retains the channel feature learning function of the SE channel attention module while significantly reducing dimensionality transformation overhead, thereby improving hardware operating efficiency. This scheme is particularly suitable for embedded platforms with limited computing resources and high real-time requirements for infrared target recognition.

[0278] The spatial attention algorithm proposed in this invention introduces an image background weakening mechanism, effectively solving problems such as low contrast, small targets, occlusion, and noise interference in infrared pedestrian recognition. This scheme improves the contrast between the pedestrian target and the background, enhances the ability to recognize weak signals of pedestrian edge contours in low-resolution infrared images, and significantly improves the model's performance in detecting weak target signals at the pixel level.

[0279] Practical application verification shows that the solution of the present invention runs stably in a low-resolution infrared thermal imaging embedded platform, with high real-time performance and excellent recognition effect.

[0280] The scope of the claims of this invention is not limited to the specific embodiments described above. Various other embodiments, including modifications or alterations that can be made by those skilled in the art without departing from the spirit and intent of the invention as described in the claims, should also be included within the scope of the claims of this invention.

Claims

1. An infrared pedestrian recognition model based on an attention module, characterized in that, include: Backbone: Used to extract multi-level semantic features from the input infrared image; Feature Fusion Network (Neck): Used to fuse feature maps of different scales to extract contextual information; Head network: Used to output the category and location coordinates of pedestrian targets; The feature is that a channel attention module is introduced after the pooling module in the backbone network, and a spatial attention module is introduced before the data enters the feature fusion network. The channel attention module is the SE channel attention module, which is used to perform feature weighting and recalibration in the channel dimension to highlight highly relevant channel features and suppress channel-level noise; The spatial attention module is the SimAM spatial attention module, which is used to weight the feature map in the spatial dimension to enhance the pedestrian target area and reduce background interference.

2. The infrared pedestrian recognition model according to claim 1, characterized in that, The channel attention module uses 1×1 convolutional layers instead of fully connected layers to reduce dimensionality transformation overhead and improve the operating efficiency of the embedded platform while maintaining the channel attention learning function.

3. The infrared pedestrian recognition model according to claim 1, characterized in that, The spatial attention module introduces a background weakening weight coefficient K during the energy weight calculation process, where 0.5 ≤ K ≤ 0.8, to weaken the background noise response and enhance the salience of the pedestrian target area.

4. An electronic device, characterized in that, include: processor; Memory is used to store executable programs in a computer; When the computer-executable program is run on the processor, it performs the steps of the infrared pedestrian recognition model according to any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, the program, when executed by a processor, implementing the steps of the infrared pedestrian recognition model according to any one of claims 1 to 3.

6. An infrared pedestrian recognition method based on the infrared pedestrian recognition model according to any one of claims 1 to 3, characterized in that, Includes the following steps: Infrared image acquisition and preprocessing steps: Infrared image data is acquired through an infrared sensor, and the infrared images are then processed by format conversion, grayscale equalization, and noise filtering. Dataset creation and model training steps: Infrared image samples containing pedestrian targets are collected, labeled, and training and validation sets are constructed. During the training process, based on the YOLOv5 network structure, feature learning is performed using the infrared pedestrian recognition model described in any one of claims 1 to 3 to generate a trained model file; Model conversion steps: The model files trained using PyTorch are converted into Rknn model files suitable for embedded platforms using the Rknn toolkit. Deployment and real-time detection steps: The RKNN model is deployed on an embedded platform to perform real-time detection and recognition of the input infrared images, and output detection results containing pedestrian target boxes and category information. Output and display steps: The detection results are filtered, scaled, denoised, and image optimized, and the final results are output to the terminal device screen in real time. Combined with warning icons or voice prompts, real-time monitoring and warning of pedestrian targets are achieved.

7. An electronic device, characterized in that, include: processor; Memory is used to store executable programs in a computer; When the computer-executable program is run on the processor, it performs the steps of the infrared pedestrian recognition method according to any one of claims 6.

8. A computer-readable storage medium having a computer program stored thereon, the program, when executed by a processor, implementing the steps of the infrared pedestrian recognition method according to any one of claims 6.

Images