Target detection method based on feature fusion of color camera and infrared thermal imager

Abstract

The invention discloses a target detection method based on feature fusion of a color camera and an infrared thermal imager. The target detection method comprises the following steps of: a, acquiring acolor data set through the color camera, and acquiring a thermal infrared data set through the infrared thermal imager; b, inputting the bimodal data set into a bimodal YOLOv3 neural network algorithm at the same time and extracting color features and temperature features of a target, fusing the features of the two modals on a certain layer of a backbone network through a fusion function and a 1*1 convolution block, and then selecting a fused feature map to continue feature extraction of the backbone network to obtain a fused extracted feature map; and c, inputting the fused extracted featuremap into a subsequent convolutional layer for target classification, and outputting a trained bimodal neural network algorithm model. According to the target detection method, temperature and color information is fused, fusion is carried out by adopting the bimodal neural backbone network algorithm, the fused information is input into the classification layer for target prediction, multiple feature information of the target is increased, and the target identification accuracy is improved.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a target detection method based on feature fusion of a color camera and an infrared thermal imager. Background technique [0002] In complex terrain environments such as changing backgrounds, occlusions by terrain and objects, and in natural environments with low visibility such as rainy days, smog, and darkness, the accuracy of using traditional target recognition solutions is low and cannot meet the needs of autonomous driving. This is because the color camera is particularly sensitive to light, and the neural network cannot extract the complete features of the target, so blindly improving the network algorithm cannot improve the accuracy of target recognition. In order to ensure that self-driving vehicles can timely and accurately perceive potential safety hazards in the road environment, and quickly take measures to avoid traffic accidents, multi-sensor joint ...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above problems, the present invention proposes a target detection method based on the feature fusion of a color camera and an infrared thermal imager. Aiming at the problem that the color camera cannot accurately identify the target in a special scene, the temperature information of the thermal infrared camera and the color of the color camera are fused. information, using a dual-channel neural network algorithm to learn target features separately, and fuse them in a certain layer of the dual-mode neural network algorithm, and then input them into the classification layer of the network algorithm for target prediction, which can increase multiple features of the target information to improve the accuracy of target recognition

Images