Multi-modal image target detection method based on image fusion

Abstract

The invention relates to a multi-modal image target detection method based on image fusion, and the method comprises the steps: 1), making a multi-modal image data set through an infrared image and avisible light image which are collected in advance; 2) taking the preprocessed paired images as the input of a generation model G in the fusion model; generating a model G based on U-Net full convolutional network. A convolutional neural network based on a residual network is used as a generative network model structure and comprises a contraction process and an expansion process, a contraction path comprises a plurality of convolution plus ReLU active layer plus maximum pooling (Max Pooling) structures, the number of feature channels of each step of down-sampling is doubled, and a generated fusion image is output; the fused image is input into a discrimination network model in a fusion model; according to the change of a loss function in a training process, a learning rate training indexis adjusted according to the number of iterations, and through training, based on a self-owned multi-modal image data set, an image fusion model which retains infrared image thermal radiation characteristics and visible light image structural texture characteristics at the same time can be obtained.

Description

technical field [0001] The invention belongs to the fields of deep learning, computer vision and image fusion, and relates to a deep neural network-based infrared-visible multimodal image fusion model and a target detection method for a target detection model. Background technique [0002] In the natural environment, objects will radiate electromagnetic waves of different frequencies that cannot be seen by the human eye, which is called thermal radiation [1]. The infrared images taken by the infrared sensor can record the thermal radiation of different objects. Compared with Visible Image (VI) images, infrared (Infrared Image, IR) images have the following characteristics: they can reduce the influence of external environments such as sunlight and smoke [1]; they are sensitive to objects and areas with obvious infrared thermal characteristics. At present, target detection tasks in infrared images are widely used, including important applications in military, electric power,...

AI Technical Summary

Problems solved by technology

In this method, the image fusion network is used as the pre-order step of the target detection model, and a general target detection method suitable for infrared images is proposed. The fused image with infrared and visible light image features is detected by the target detection model based on deep neural network, so as to overcome the problem of lack of structural features of a single infrared sensor, which is of great significance to the improvement of detection results and practical engineering applications

Images