Multispectral pedestrian detection method based on feature fusion deep neural network

Abstract

The invention discloses a multispectral pedestrian detection method based on a feature fusion deep neural network. The multispectral pedestrian detection method comprises the following steps: step 1,respectively extracting feature information of multispectral images; 2, fusing the feature information to obtain a third tensor; 3, performing convolution operation on the third tensor to obtain a fourth tensor; 4, the Faster R-CNN network is improved to serve as a pedestrian detection model; and step 5, inputting the fourth tensor into the improved Faster R-CNN algorithm, and outputting a pedestrian detection result. The device is simple in structure and reasonable in design; feature information of the visible light image and the infrared image is fused to form complementation; the method comprises the following steps: improving a cross entropy loss function of an RCNN in a Faster R-CNN algorithm by adopting a local loss function; according to the method, the problem of imbalance of positive and negative samples is solved, difficult-to-classify and easy-to-classify samples are reasonably measured, a frame regression loss function of a Faster R-CNN algorithm is improved by adopting a KL loss function, and the loss of a bounding box regression device on a fuzzy bounding box is reduced.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a multispectral pedestrian detection method based on a feature fusion deep neural network. Background technique [0002] With the advent of the era of big data, computer vision has been widely used in all aspects of our lives. It is a subject that uses computers instead of human eyes to detect, identify, and track targets. Pedestrian detection based on computer vision is a very important part of autonomous driving and night driving applications. As an important branch of target detection, pedestrian detection is to detect pedestrians in images or videos. The purpose is to determine the position and size of pedestrians, which can be used for subsequent target trajectory analysis. It can reduce vehicle accidents and improve vehicle flow efficiency in regulating traffic. It plays a very important role in reducing energy consumption and emissions. [0003] The f...

AI Technical Summary

Problems solved by technology

[0003] Full-time pedestrian detection technology is a very important mode in vehicle vision systems, but most of the current domestic vehic

Images