A street scene infrared target detection method

Abstract

The present case relates to a kind of street scene infrared target detection method, after training, the model is too complex and the effect of extracting target feature from complex background such as street scene is not good for existing target detection model, the feature information of specified target is obtained by inputting the real-time acquisition street scene infrared image into infrared target detector, and the target is positioned and identified.The infrared target detector adopts YOLOv5 network and parameter-free attention module SimAM to be combined, realizes each repeated part of C3 residual module in YOLOv5 network structure, parameter-free attention module SimAM is added in series behind each repeated part, while not increasing parameter quantity, the purpose of suppressing background noise and enhancing detection performance is realized.Furthermore, different dilated convolution module of expansion rate is added after each SPPF module maximum pooling operation, so that the model obtains flexible receptive field, and the performance of detecting the target with large size difference is enhanced.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and to a method for detecting designated targets on streets. This method can predict pedestrians more accurately and quickly, providing assistance for fields such as autonomous driving and drone detection. Background Technology

[0002] Unlike visible light imaging, infrared imaging technology does not require external ambient light sources and is unaffected by adverse weather conditions such as rain, snow, and wind. It boasts advantages such as all-weather imaging, strong anti-interference capabilities, long-distance imaging, and high imaging accuracy. Infrared target detection in street scenes is a crucial technology for autonomous driving, traffic monitoring, and security surveillance. Infrared images in these scenes suffer from low contrast, low signal-to-noise ratio (which can obscure and interfere with target detection), less detailed information, and more complex backgrounds (which can make target areas appear dark and uneven). These factors make it difficult for current infrared target detection algorithms for street scenes to extract deeper semantic information from images, resulting in poor detection performance.

[0003] In recent years, with the deepening of research in the field of computer vision, convolutional neural networks have been proven to have better feature extraction capabilities and are currently being used to solve many problems, such as image classification and super-resolution.

[0004] Traditional object detection algorithms can be broadly divided into three stages. First, sliding windows of different sizes are selected to scan the input image based on the size of the target object, generating candidate regions (ROIs). Since this process involves indiscriminate scanning of all regions, many redundant windows are generated, significantly impacting detection efficiency. Second, manually designed feature extractors are used to extract features from the candidate regions, including SIFT, Haar, and HOG. However, due to the diversity of target morphology and background illumination variations, the generalization ability of the extracted features is generally limited. Finally, Support Vector Machines (SVMs) or AdaBoost are used to determine whether an object exists in the region.

[0005] Although the above methods have achieved good detection performance in fields such as autonomous driving, the models trained by these methods are mostly too complex and cumbersome, and their performance is generally poor when detecting complex scenes such as street scenes. Summary of the Invention

[0006] To improve infrared target detection performance in complex street scenes and achieve real-time target detection, this solution proposes an infrared target detector, SimAM-YOL0, based on YOLOv5, to address the uniqueness of infrared images. To mitigate the large number of model parameters and complex background interference, a new feature extraction module, SimAM-C3, is proposed. Furthermore, to address the issue of large target scale differences in infrared images of street scenes, dilated convolution modules with different dilation rates are introduced to increase the model's receptive field and effectively improve the detection performance of targets with significant scale differences. The specific technical solutions are as follows.

[0007] Firstly, this case proposes an infrared target detection method for street scenes. The method inputs real-time acquired infrared images of the street scene into an infrared target detector. The infrared target detector extracts feature information of a specified target in the street scene and uses this feature information to locate and identify the target. The infrared target detector is implemented by combining a YOLOv5 network with a parameterless attention module SimAM. For the repeated part of the C3 residual module in the YOLOv5 network structure, a parameterless attention module SimAM is added in series after each repeated part, so as to suppress background noise and enhance detection performance without increasing the number of parameters.

[0008] As a further improvement to the above technical solution, the infrared target detector adds dilated spatial pyramid pooling fast (DSPPF) modules with different dilation rates after the max pooling operation of each SPPF module, so that the model can still ensure good detection performance when detecting targets with large size differences.

[0009] In one embodiment of the above technical solution, the dilation rates of the three dilated convolutional modules of each SPPF module from input to output are set to 3, 6, and 6, respectively.

[0010] In one embodiment of the above technical solution, the infrared target detector uses Mosaic data augmentation to increase the diversity of the dataset before training.

[0011] In one embodiment of the above technical solution, the value of Mosaic is set to 0.01.

[0012] In one embodiment of the above technical solution, the infrared target detector is trained with a batch size of 32, a learning rate of 0.01, and a momentum decay rate of 0.937.

[0013] In one embodiment of the above technical solution, the parameters of the infrared target detector are determined through the following steps:

[0014] The training dataset is input into the infrared target detector, the training results are recorded, the parameters are adjusted, and the parameter weights with the best detection performance are used as candidate weights.

[0015] The candidate weights of the infrared target detector were adjusted using the validation dataset, and the validation results were recorded.

[0016] Repeat the training and validation process until the weights that are globally optimal are obtained.

[0017] In one embodiment of the above technical solution, the infrared target detector is composed of a backbone, a neck, and a detection head;

[0018] The skeleton is composed of a first convolutional module, a second convolutional module, a first SimAM-C3 module, a third convolutional module, a second SimAM-C3 module, a fourth convolutional module, a third SimAM-C3 module, a fourth convolutional module, a third SimAM-C3 module, and a DSPPF module connected in series.

[0019] The neck is composed of the sixth convolution module, the first upsampling module, the first concat module, the fourth SimAM-C3 module, the seventh convolution module, the second upsampling module, the second concat module, the fifth SimAM-C3 module, the eighth convolution module, the third concat module, the sixth SimAM-C3 module, the ninth convolution module, the fourth concat module, and the seventh SimAM-C3 module connected in series.

[0020] The detection head comprises a first two-dimensional convolution, a second two-dimensional convolution, and a third two-dimensional convolution, which output images that are progressively larger.

[0021] Wherein: the output of the second SimAM-C3 module is also used as the input of the second ConCat module, the output of the third SimAM-C3 module is also used as the input of the first Concat module, and the output of the DSPPF module is the input of the sixth convolution module;

[0022] The output of the sixth convolutional module is also used as the input of the fourth concat module, the output of the seventh convolutional module is also used as the input of the third concat module, and the output of the second concat module is also used as the input of the fifth SimAM-C3 module.

[0023] The output of the fifth SimAM-C3 module is used as the input of the first two-dimensional convolution, the output of the sixth SimAM-C3 module is used as the input of the second two-dimensional convolution, and the output of the seventh SimAM-C3 module is used as the input of the third two-dimensional convolution.

[0024] Secondly, this case proposes a street scene infrared target detection device, including a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed according to any one of the methods of claims 1 to 8.

[0025] Thirdly, this case proposes an infrared target detector, which is implemented by combining a YOLOv5 network with a parameterless attention module SimAM. For the repeated part of the C3 residual module in the YOLOv5 network structure, a parameterless attention module SimAM is added in series after each repeated part. The infrared image of the street scene acquired in real time is input into the infrared target detector, which extracts the feature information of the specified target in the street scene and uses the feature information to locate and identify the target.

[0026] This case has the following technical effects:

[0027] This solution improves infrared target detection performance in complex scenarios such as street scenes. It overcomes interference from complex backgrounds and achieves real-time target detection without increasing the number of model parameters. Furthermore, it captures as much global feature information as possible of important features while preserving the original features of the original image and maintaining good detection performance. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 , one A schematic diagram of the SimAM-C3 module structure in one implementation method;

[0030] Figure 2 , one A comparative diagram of the SPPF module and DSPPF module in one implementation method;

[0031] Figure 3 A schematic diagram of the infrared target detector architecture for a street scene in one implementation method;

[0032] Figure 4 , one A schematic diagram illustrating the dynamic impact of different improvements obtained in various implementation methods on mAP (IoU = 0.5);

[0033] Figure 5 , oneIn one implementation, a heatmap comparison diagram of feature maps used to detect targets of different sizes in the detection head is drawn based on the confidence level.

[0034] Figures 6(a), 6(b), and 6(c) are schematic diagrams comparing the confidence loss, classification loss, and bounding box localization loss of different improvement methods on the FLIR validation set in one implementation method.

[0035] Figure 7 , one A comparative diagram of some detection results in one implementation method;

[0036] Figure 8 , one A schematic diagram of the process for acquiring infrared target detectors in a street scene in one implementation method. Detailed Implementation

[0037] To improve infrared target detection performance in complex street scenes and achieve real-time target detection, this invention proposes a novel infrared target detector, SimAM-YOLO, based on the YOLOv5 network structure, after analyzing the unique characteristics of infrared images. Addressing the challenges of large model parameters and complex background interference, a new feature extraction module, SimAM-C3, is introduced. Furthermore, to address the issue of significant target scale differences in infrared images during street scenes, dilated convolutions with varying dilation rates are introduced to increase the model's receptive field, effectively improving the detection performance for targets with large scale variations.

[0038] The following description, in conjunction with the accompanying drawings, clearly and completely describes how the technical solution of this application is implemented. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. The terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature.

[0039] (I) Infrared Target Detector

[0040] (1) Feature extraction module

[0041] Detection speed and accuracy are the key challenges in street scene object detection. Maintaining both accuracy and speed has always been a difficult problem. While past attention mechanisms can suppress background noise and improve detection performance, their addition increases the number of parameters, making it difficult to guarantee detection speed, as seen in GCNet.

[0042] In one implementation, the infrared target detector uses YOLOv5 as its basic network architecture and integrates a parameterless attention module, SimAM, on top of it to achieve the ability to suppress background noise and enhance detection performance without increasing the number of parameters.

[0043] Specifically, the parameterless attention module SimAM is integrated into the end of the repeating module in the residual block of the original feature extraction module C3 in YOLOv5. The location of the parameterless attention module is as follows: Figure 1 As shown, it is located after the last ordinary convolutional block in the repeating module, where Conv1, Conv2, Conv3, Conv4 and Conv5 all represent ordinary convolutional blocks, which contain three operations: standard convolution, normalization and activation function. SimAM represents the parameterless attention module.

[0044] The C3 module can fuse feature information from different scales. The SimAM module embeds 3D attention weights into the feature channels of the original features, achieving the goal of adjusting the weights of different channels. The SimAM module mainly undergoes the following processes in adjusting the feature channel weights:

[0045] (i) Obtain the size in the feature map space.

[0046] (ii) Calculate the variance between each pixel on the feature map and the mean.

[0047] (iii) Calculate the variance of each pixel in each channel based on the results obtained in the previous process.

[0048] (iv) Calculate the energy function corresponding to the feature map using variance, and determine the importance of the feature map based on the energy function. Then group the feature maps according to their importance.

[0049] (v) Fuse the original feature map with the features grouped according to importance to return the features that should be of interest on the feature map.

[0050] (2) Capture global feature information

[0051] In street scene object detection, the multi-scale problem has always been a key concern. How to adapt the model to targets with large scale differences is an urgent problem to be solved. During data acquisition, the camera angle and the distance between objects will affect the image size of the target to be detected. A good model should show good detection performance when detecting targets of different sizes.

[0052] In real-world detection scenarios, targets with significant scale differences often appear, such as large cars and small pedestrians. Therefore, designing models that can adapt to targets of different scales is crucial. For a model to adapt to targets with large scale differences, it needs to acquire a sufficiently large receptive field during detection. This allows the model to obtain global information and grasp the overall features of the target from a global perspective, rather than being limited to local information. Dilated convolution is a good solution for acquiring global feature information and understanding the entire target. Its biggest difference from ordinary convolution is that dilated convolution inserts zero values ​​into the convolution kernel according to a set dilation rate, which easily increases the model's receptive field. Furthermore, dilated convolution does not require image cropping or scaling to easily increase the receptive field, ensuring the originality of the image and the accuracy of training. The SPPF module in YOLOv5 is an example. Figure 2 As shown on the left, the SPFF module is an improvement upon the SPP module. It not only solves the image distortion problem and ensures scale invariance of image features, but also addresses the issue of repeated feature extraction, preventing overfitting and playing a crucial role. While the SPPF module guarantees scale invariance to some extent, it doesn't provide a sufficiently large receptive field or capture global target features. Therefore, the scale invariance guaranteed by the SPPF module is localized; it only ensures scale invariance for local feature information, which is quite limited and often fails to guarantee scale invariance for larger targets.

[0053] To address the aforementioned issues and ensure good detection performance for targets with varying sizes, one implementation proposes a dilated spatial pyramid pooling fast (DSPPF) module, such as... Figure 2As shown on the right. To obtain the largest possible receptive field at different scales, dilated convolutions with different dilation rates are added after each max pooling operation. By convolving the pooled feature maps, not only are the original features of the image preserved, but also the global feature information of important features is captured as much as possible, thus better ensuring the scale invariance of the model. In one implementation, setting the dilation rates of the dilated convolution blocks Dilated Conv1, Dilated Conv2, and Dilated Conv3 to 3, 6, and 6 respectively can maintain the best detection performance when detecting targets with large scale differences, while minimizing the number of parameters added and ensuring real-time performance.

[0054] (3) Overall Structure

[0055] In one implementation, an infrared target detector with residual block SimAM-C3 and DSPPF modules is constructed based on the YOLOv5 network architecture. Figure 3 As shown in the figure, Conv represents a regular convolutional block, which includes three operations: standard convolution, normalization, and activation function. Conv2d represents two-dimensional convolution. SimAM-YOLOs mainly consists of three parts: the backbone, the neck, and the detection head.

[0056] The infrared target detector consists of a backbone, a neck, and a detection head. The backbone is composed of a first convolutional module, a second convolutional module, a first SimAM-C3 module, a third convolutional module, a second SimAM-C3 module, a fourth convolutional module, a third SimAM-C3 module, a fourth convolutional module, a third SimAM-C3 module, and a DSPPF module connected in series. The neck is composed of a sixth convolutional module, a first upsampling module, a first concat module, a fourth SimAM-C3 module, a seventh convolutional module, a second upsampling module, a second concat module, a fifth SimAM-C3 module, an eighth convolutional module, a third concat module, a sixth SimAM-C3 module, a ninth convolutional module, a fourth concat module, and a seventh SimAM-C3 module connected in series. The detection head consists of a first two-dimensional convolutional module, a second two-dimensional convolutional module, and a third two-dimensional convolutional module, each outputting an image of progressively larger size. Specifically: the output of the second SimAM-C3 module is also used as the input of the second Concat module; the output of the third SimAM-C3 module is also used as the input of the first Concat module; the output of the DSPPF module is the input of the sixth convolution module; the output of the sixth convolution module is also used as the input of the fourth Concat module; the output of the seventh convolution module is also used as the input of the third Concat module; the output of the second Concat module is also used as the input of the fifth SimAM-C3 module; the output of the fifth SimAM-C3 module is used as the input of the first two-dimensional convolution; the output of the sixth SimAM-C3 module is used as the input of the second two-dimensional convolution; and the output of the seventh SimAM-C3 module is used as the input of the third two-dimensional convolution.

[0057] When real-time infrared images of street scenes are input into an infrared target detector, feature information of specified targets such as pedestrians or vehicles can be obtained, and the targets can be located and identified based on the obtained feature information.

[0058] (II) Training of Infrared Target Detectors

[0059] (1) Collecting datasets

[0060] The dataset uses the FLIR Infrared Pedestrian Dataset, released by FLIR Systems in July 2018, which provides labeled infrared images and unlabeled visible light images. The dataset was acquired using RGB and thermal imaging cameras mounted on vehicles, capturing images of vehicles and pedestrians on streets and highways in Santa Barbara, California, USA.

[0061] (2) Data cleaning

[0062] Because the dataset contains a large amount of data from various sources, much of it is contaminated. To make the dataset meet the requirements, the data was cleaned, which simply means selecting out contaminated or damaged images.

[0063] (3) Split the dataset

[0064] The dataset is divided into a training set and a validation set. The training set is primarily used to train the model and extract feature information about pedestrians or vehicles. The validation set is used to verify the model's detection performance. To enable the model to extract more feature information and improve its robustness, a larger number of images, approximately nine-tenths of the total dataset, can be allocated to the training set, which is significantly larger than the validation set.

[0065] (4) Initialize model parameters

[0066] To better adapt the model to the dataset, the model parameters were initialized.

[0067] During training, the input image size was set to 640x640, training was performed 300 times, the batch size was set to 32, the learning rate was set to 0.01, and the momentum decay rate was set to 0.937. This invention uses Mosaic data augmentation to increase the diversity of the dataset, and the Mosaic value is set to 1.0. During validation, the confidence threshold is set to 0.001. All experiments were conducted on a Tesla V100 device and based on the PyTorch framework.

[0068] (5) Determination of model parameters

[0069] The training set was input into the SimAM-YOLO object detection model, and the experimental results were recorded, focusing on the dynamic changes of the model during training. After each experiment, the experimental data was organized, and the dynamic changes and final results were compared. Parameters were repeatedly adjusted based on the changes, and experiments were repeated until the optimal results were obtained. The model weights with the best detection performance were then selected as candidate weights to validate the model.

[0070] The obtained candidate weights are used as validation weights to validate the validation dataset. The validation results are recorded, and the parameters are adjusted according to the validation results. The training and validation steps are repeated until the globally optimal weights are obtained.

[0071] Thus, through the above implementation process, the final infrared target detector can be obtained. In one embodiment, such as... Figure 8As shown, the globally optimal weights obtained are applied to embedded devices to assist in practical production and daily life. In another implementation, it is achieved using software plus necessary general-purpose hardware, or through dedicated hardware including application-specific integrated circuits (ASICs), dedicated CPUs, dedicated memory, and dedicated components. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the method of this disclosure, software implementation is more often a preferred implementation method.

[0072] (III) Simulation Experiment

[0073] (1) Simulation Environment

[0074] Tesla V100 was used as the simulation environment, and simulations were performed using the PyTorch framework.

[0075] (2) Simulation results

[0076] The detection performance of different detection methods on the FLIR dataset was collected, and YOLOv5 and the infrared target detector SimAM-YOLO proposed in this invention were added. The comparison results are shown in Table 1. SimAM-YOLO achieved the best detection performance under the same experimental conditions.

[0077] As shown in Table 1, the latest research reports (bold numbers indicate the best results) were obtained from the most recent work, and the detection results of YOLOv3-tiny, YOLOv5s, Strip-YOLOs, and SimAM-YOLOs were compared. SimAM-YOLOs ultimately showed the best overall performance (detection accuracy and detection speed). Compared to YOLOv5s, SimAM-YOLOs improved accuracy by 4.6%, recall by 0.7%, mAP (IoU=0.5) by 1.9%, and F1 score by 2.4%, with only a slight increase in inference time. This indicates that the improved algorithm can detect targets more accurately and comprehensively. Compared to Strip-YOLOs, while SimAM-YOLOs did not show a significant improvement in detection accuracy, it reduced inference time by 6.2 ms and FLOPs by 2 bytes, greatly improving the model's detection speed and enabling real-time detection.

[0078] Table 1

[0079]

[0080] (3) Simulation comparison

[0081] 1) The effectiveness of different improvement methods

[0082] To more objectively verify the effectiveness of each improvement method, this paper controls experimental variables and records the performance improvement of YOLOv5 detection performance brought by different improvement methods. Specifically, keeping the YOLOv5 model structure unchanged, only the C3 module is changed, and its performance change is observed. Then, the DSPPF module is added, and its impact on detection performance is observed. Table 2 shows the impact of different improvement methods on YOLOv5 detection results, and Table 3 shows the impact of different improvement methods on the accuracy and recall of target detection for each category.

[0083] (a) Effectiveness of SimAM-C3. To reduce the impact of complex background features on detection performance without increasing the number of model parameters and ensuring detection speed, this paper improves the C3 module with a parameterless attention module, proposing a new feature extraction module, SimAM-C3. Unlike previous attention structures, the parameterless attention module can easily obtain the three-dimensional feature weights of the feature map through simple calculations, achieving the purpose of suppressing the background and highlighting the target, without increasing the number of parameters. This is very important for ensuring the detection speed of the model. The first set of comparative experiments in Table 2 shows the comparison of detection performance between YOLOv5s and adding only SimAM-C3. It can be seen that after adding SimAM-C3 alone, the model's mAP (IoU=0.5) improved by 0.5%, and F1 improved by 1.2%. This proves that after adding SimAM-C3, the model focuses more on the feature information of the target itself, reduces the interference of other feature information, and significantly improves the accuracy of the model in locating the target. The first set of comparative experiments in Table 3 also confirms this conclusion. As shown in the first and second rows, after adding SimAM-C3, the model's accuracy and recall improved by 1.6% and 0.9%, respectively. This means that SimAM-C3 can enable the model to detect targets more accurately, significantly reducing the false positive and false negative rates. It is worth mentioning that since the parameterless attention module is very lightweight, it requires almost no increase in the number of parameters. Therefore, after adding the SimAM structure, the model's parameter count and inference time are almost the same as the original model.

[0084] (b) Effectiveness of the DSPPF Module. To increase the receptive field of the model and enable it to better detect targets with large scale differences, this paper proposes the DSPPF module by adding dilated convolutions with different dilation rates to the SPPF module. Dilated convolution is similar to ordinary convolution, except that some zero values ​​are inserted according to a certain rule, which allows the dilated convolution to have a larger receptive field. Therefore, the addition of dilated convolution not only does not affect the original function of the SPPF module, such as ensuring image integrity, but also significantly increases the model's receptive field. The second set of comparative experiments in Table 2 shows the comparison of detection performance between YOLOv5s and the model with only the DSPPF module added. It can be seen that when the DSPPF module is added alone, the model's mAP (IoU = 0.5) improves by 0.7% and F1 score improves by 1.7%. This proves that after improving the SPPF module, the model has a larger receptive field and still has good detection performance when facing targets with large scale differences. The second set of comparative experiments in Table 3 also confirms this conclusion. As can be seen from the first and third rows of the table, after adding the .DSPPF module, the accuracy and recall of the model improved by 2.4% and 1.1%, respectively.

[0085] Table 2

[0086]

[0087] Table 3

[0088]

[0089]

[0090] 2) Convergence speed analysis

[0091] Figure 4 The graphs show the dynamic changes in mAP (IoU = 0.5) after adding the SimAM-C3 and DSPPF modules with increasing training iterations. The graphs clearly show that the model's convergence speed and mAP value are significantly better than YOLOv5s after adding the SimAM-C3 and DSPPF modules. This is mainly because the addition of parameterless attention suppresses complex background information, allowing the model to focus more on target features and accelerating the extraction of important features. Secondly, the addition of dilated convolution also enables the model to better adapt to detection tasks of targets at different scales, reducing feature loss.

[0092] 3) Thermal diagram analysis

[0093] Heatmaps can more intuitively display the model's focus areas. Therefore, this invention uses Class Activation Mapping (CAM) based on confidence levels to create heatmaps of feature maps for detecting targets at different scales (small targets 8080, medium targets 4040, large targets 2020) in the SimAM-YOLOs model's detection head. For example... Figure 5 As shown in the figure, the visualization results of small, medium, and large target feature maps in the detection heads of YOLOv5s and SimAM-YOLOs are presented. From the first set of comparative experiments in the visualization results (first and second rows), it can be seen that when facing complex scenes, SimAM-YOLOs can extract more feature information (e.g., the bicycle on the right side of the figure). This is because the SimAM-C3 module adjusts the weights of the target feature channels through the energy function evolution formula, suppressing the interference of background information. Furthermore, from the second set of comparative experiments (third and fourth rows), it can be seen that when facing targets with large scale differences, SimAM-YOLOs can extract more feature information (e.g., pedestrians in the distance in the figure), reducing feature loss. This is because the addition of dilated convolution increases the model's receptive field, enabling the model to better capture the global features of targets with large scale differences, more accurately locate target boundaries, and maintain good detection performance.

[0094] 4) Loss Analysis

[0095] The loss function reflects the difference between the true and predicted values; a smaller loss value indicates better detection results, and the predicted value is closer to the true value. Figure 6 shows the changes in model loss of YOLOv5s after adding SimAm-C3 and DSPPF modules respectively. It is clearly visible from the figure that the improved object detection model SimAM-YOLOs has lower values ​​on the validation set than YOLOv5s in terms of confidence loss, classification loss, and bounding box localization loss. This confirms that SimAM-YOLO has better feature extraction and object detection capabilities, and its predicted values ​​are closer to the true values.

[0096] 5) Partial test results

[0097] Figure 7The image shows partial detection results of YOLOv5s and SimAM-YOLOs on the FLIR dataset. From the first and second rows, it's clear that in scenarios with relatively complex backgrounds, YOLOv5s misses many harder-to-detect targets (e.g., the bicycle on the right side of the first row and the pedestrian on the right side of the second row). SimAM-YOLOs, by incorporating the SimAM-C3 module and adjusting channel weights, focuses more on target features, improving its feature extraction capabilities. Therefore, SimAM-YOLOs detects these targets well. From the third, fourth, and fifth rows, it's evident that when detecting targets with large scale differences, YOLOv5s cannot accurately locate the entire target region, resulting in inaccurate bounding boxes (e.g., the larger pedestrian in the third row), reducing the model's mAP. SimAM-YOLOs, on the other hand, captures these targets relatively accurately. Furthermore, YOLOv5s misses many targets when detecting objects with large scale differences (such as the smaller pedestrian target in the fourth row and the larger pedestrian target in the fifth row), while SimAM-YOLOs detects them accurately. This is because SimAM-YOLOs incorporates dilated convolutions, increasing the model's receptive field. This not only allows the model to more comprehensively and accurately grasp the overall feature region of the target and more accurately define the target, but also significantly reduces the false negative rate.

[0098] In summary, when faced with scene detection tasks involving complex backgrounds and significant differences in target scale, SimAM-YOLOs, which incorporates a parameterless attention module and dilated convolution, can more accurately define target regions and significantly reduce the possibility of missed targets compared to YOLOv5s.

[0099] It should be noted that the terms "one embodiment," "another embodiment," and "embodiment" used in this specification refer to specific features, structures, or characteristics described in connection with that embodiment, which are included in at least one embodiment described in the general description of this application. The appearance of the same expression in multiple places in the specification does not necessarily refer to the same embodiment. Furthermore, when a specific feature, structure, or characteristic is described in connection with any embodiment, the intention is to suggest that implementing such a feature, structure, or characteristic in conjunction with other embodiments also falls within the scope of this invention.

[0100] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A method for detecting infrared targets in street scenes, characterized in that: The method inputs real-time acquired infrared images of the street scene into an infrared target detector, which extracts feature information of a specified target in the street scene and uses this feature information to locate and identify targets at multiple scales. The infrared target detector is implemented using a YOLOv5 network combined with a parameterless attention module SimAM. For the repeated portions of the C3 residual module in the YOLOv5 network structure, a parameterless attention module SimAM is added in series after each repeated portion to form a SimAM-C3 module. The final infrared target detector structure is as follows: The skeleton is composed of the first convolutional module, the second convolutional module, the first SimAM-C3 module, the third convolutional module, the second SimAM-C3 module, the fourth convolutional module, the third SimAM-C3 module, the fifth convolutional module, the fourth SimAM-C3 module, and the DSPPF module connected in series. The neck is composed of the sixth convolution module, the first upsampling module, the first concat module, the fifth SimAM-C3 module, the seventh convolution module, the second upsampling module, the second concat module, the sixth SimAM-C3 module, the eighth convolution module, the third concat module, the seventh SimAM-C3 module, the ninth convolution module, the fourth concat module, and the eighth SimAM-C3 module connected in series. The detection head consists of a first two-dimensional convolution, a second two-dimensional convolution, and a third two-dimensional convolution, which output images that are progressively larger. Specifically: the output of the second SimAM-C3 module is also used as the input of the second Concat module; the output of the third SimAM-C3 module is also used as the input of the first Concat module; the output of the DSPPF module is the input of the sixth convolution module; the output of the sixth convolution module is also used as the input of the fourth Concat module; the output of the seventh convolution module is also used as the input of the third Concat module; the output of the second Concat module is also used as the input of the sixth SimAM-C3 module; the output of the sixth SimAM-C3 module is used as the input of the first two-dimensional convolution; the output of the seventh SimAM-C3 module is used as the input of the second two-dimensional convolution; and the output of the eighth SimAM-C3 module is used as the input of the third two-dimensional convolution. The DSPPF module is constructed by adding dilated convolution modules with different dilation rates after the max pooling operation of the SPPF module.

2. The method according to claim 1, characterized in that, Three dilated convolutional modules with different dilation rates are added after the max pooling operation of the SPPF module. The dilation rates are set to 3, 6 and 6 respectively.

3. The method according to claim 1, characterized in that, The infrared target detector uses Mosaic data augmentation to increase the diversity of the dataset before training.

4. The method according to claim 3, characterized in that, The value of Mosaic is set to 0.

01.

5. The method according to claim 1, characterized in that, During training, the infrared target detector has its batch size set to 32, learning rate set to 0.01, and momentum decay rate set to 0.

937.

6. The method according to claim 1, characterized in that, The parameters of the infrared target detector are determined through the following steps: The training dataset is input into the infrared target detector, the training results are recorded, the parameters are adjusted, and the parameter weights with the best detection performance are used as candidate weights. The candidate weights of the infrared target detector were adjusted using the validation dataset, and the validation results were recorded. Repeat the training and validation process until the weights that are globally optimal are obtained.

7. A street scene infrared target detection device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed according to any one of claims 1 to 6.

8. An infrared target detector, characterized in that, The infrared target detector is implemented using a YOLOv5 network combined with a parameterless attention module SimAM. For the repeated portions of the C3 residual module in the YOLOv5 network structure, a parameterless attention module SimAM is added in series after each repeated portion to form a SimAM-C3 module. The final infrared target detector structure is as follows: The skeleton is composed of the first convolutional module, the second convolutional module, the first SimAM-C3 module, the third convolutional module, the second SimAM-C3 module, the fourth convolutional module, the third SimAM-C3 module, the fifth convolutional module, the fourth SimAM-C3 module, and the DSPPF module connected in series. The neck is composed of the sixth convolution module, the first upsampling module, the first concat module, the fifth SimAM-C3 module, the seventh convolution module, the second upsampling module, the second concat module, the sixth SimAM-C3 module, the eighth convolution module, the third concat module, the seventh SimAM-C3 module, the ninth convolution module, the fourth concat module, and the eighth SimAM-C3 module connected in series. The detection head consists of a first two-dimensional convolution, a second two-dimensional convolution, and a third two-dimensional convolution, which output images that are progressively larger. Specifically: the output of the second SimAM-C3 module is also used as the input of the second Concat module; the output of the third SimAM-C3 module is also used as the input of the first Concat module; the output of the DSPPF module is the input of the sixth convolution module; the output of the sixth convolution module is also used as the input of the fourth Concat module; the output of the seventh convolution module is also used as the input of the third Concat module; the output of the second Concat module is also used as the input of the sixth SimAM-C3 module; the output of the sixth SimAM-C3 module is used as the input of the first two-dimensional convolution; the output of the seventh SimAM-C3 module is used as the input of the second two-dimensional convolution; and the output of the eighth SimAM-C3 module is used as the input of the third two-dimensional convolution. The DSPPF module is constructed by adding dilated convolution modules with different dilation rates after the max pooling operation of the SPPF module. The infrared images of the street scene acquired in real time are input into the infrared target detector, which extracts the feature information of the specified target in the street scene and uses the feature information to locate and identify targets at multiple scales.

Images