Target Recognition Method and System Based on Infrared Image and Visible Light Image Fusion

Abstract

The embodiments of this specification provide a target recognition method and system based on the fusion of infrared images and visible light images. Among them, the method includes: preprocessing the collected infrared images and visible light images; filtering the preprocessed infrared images and visible light images respectively through a high-pass filter to obtain feature images; performing discrete cosine transform and wavelet transform processing on the preprocessed infrared images and visible light images respectively to obtain detail images; fusing the feature images and detail images, and recognizing target information according to the obtained fused image. It solves the problem of the adverse effects brought by the environment on image acquisition and makes target recognition more accurate.

Description

Technical Field

[0001] This document relates to the technical field of target recognition, and in particular, to a target recognition method and system based on the fusion of infrared images and visible light images. Background Art

[0002] In the modern war environment, which is complex and changeable, when using unmanned equipment for intelligence reconnaissance in special operations, it often faces harsh environments such as camouflage, insufficient lighting, and low visibility. There are problems such as unclear target image capture and indistinct background differentiation. It is difficult for unmanned equipment to automatically interpret image information, and target recognition is prone to errors. There is an urgent need for reliable target recognition technology. Any object in nature with a temperature above absolute zero (-273.15 °C / 0 K) continuously emits infrared rays of different intensities and wavelengths. Using this characteristic, we can use infrared sensors for image acquisition work. Although infrared sensors have characteristics such as all-weather and strong environmental adaptability, their disadvantages such as weak perception of scene brightness and low resolution of the captured images limit their applications. Utilizing the characteristics of high resolution and strong environmental brightness perception of visible light sensors for image capture can effectively make up for the shortcomings of infrared sensors. Therefore, by fusing visible light and infrared sensors, the main information and detailed information of the target can be obtained well.

[0003] The fusion methods of visible light images and infrared images can be basically divided into two categories: region-based fusion methods and pixel-based fusion methods. Region-based fusion methods are achieved by image segmentation or by extracting the significant regions and regions of interest in the images. Currently, the commonly used fusion method is to use neural networks; Pixel-based fusion methods are usually easy to implement, generally based on transform domains, such as image pyramids, wavelet transforms, and contourlet transforms. The key of region-based fusion methods lies in the local significant region information of infrared images, and it is easy to ignore the detailed information of the images; Pixel-based fusion methods focus on extracting the detailed information of the images, and the expression of the significant region information of the images is not obvious enough. Summary of the Invention

[0004] The purpose of the present invention is to provide a target recognition method and system based on the fusion of infrared images and visible light images, aiming to solve the problem of the adverse effects of the environment on image acquisition and make target recognition more accurate.

[0005] One or more embodiments of this specification provide a target recognition method based on the fusion of infrared images and visible light images, including:

[0006] S1. Preprocess the collected infrared images and visible light images;

[0007] S2. Filter the preprocessed infrared image and visible light image respectively through a high-pass filter to obtain feature images;

[0008] S3. Perform discrete cosine transform and wavelet transform on the preprocessed infrared image and visible light image respectively to obtain detail images;

[0009] S4. Fuse the feature images and detail images, and identify target information based on the obtained fused image.

[0010] One or more embodiments of this specification provide an object recognition system based on the fusion of infrared images and visible light images, including:

[0011] Image preprocessing module: used to preprocess the collected infrared images and visible light images;

[0012] Feature extraction module: used to filter the preprocessed infrared image and visible light image respectively through a high-pass filter to obtain feature images;

[0013] Detail extraction module: used to perform discrete cosine transform and wavelet transform on the preprocessed infrared image and visible light image respectively to obtain detail images;

[0014] Fusion and recognition module: used to fuse the feature images and detail images, and identify target information based on the obtained fused image.

[0015] One or more embodiments of this specification provide an object recognition device based on the fusion of infrared images and visible light images, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above object recognition method are implemented.

[0016] Adopting the infrared and visible light image fusion method of the embodiments of the present invention enables the fused image to contain the information of the locally significant regions and detail information of the image; by using image decoding and encoding techniques, the speed of image fusion is faster and the real-time performance is better, and the real-time detection and recognition of image target information can be achieved.

[0017] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically described below. Brief Description of the Drawings

[0018] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the following will briefly introduce the drawings required for describing the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a flowchart of the target recognition method based on the fusion of infrared images and visible light images provided by an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the method for fusing infrared images and visible light images provided by an embodiment of the present invention;

[0021] Figure 3 It is a schematic diagram of the target recognition system based on the fusion of infrared images and visible light images provided by an embodiment of the present invention;

[0022] Figure 4 It is a schematic structural diagram of the target recognition device based on the fusion of infrared images and visible light images provided by an embodiment of the present invention. Detailed implementation manners

[0023] In order to enable those skilled in the art of this technology to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0024] Figure 1 It is a flowchart of the target recognition method based on the fusion of infrared images and visible light images provided by an embodiment of the present invention. According to an embodiment of the present invention, a target recognition method based on the fusion of infrared images and visible light images is provided, as Figure 1 shown. The target recognition method based on the fusion of infrared images and visible light images according to an embodiment of the present invention specifically includes:

[0025] S1. Preprocess the collected infrared images and visible light images.

[0026] When performing image acquisition, the relative positions of the visible light camera and the infrared camera remain unchanged. After the image acquisition is completed, if the sizes of the acquired visible light image and infrared image are different, it is necessary to first edit the visible light image and infrared image acquired by the visible light camera and the infrared camera into images with the same size through an image editor.

[0027] After the image size processing is completed, the visible light image and infrared image with the same size are respectively decomposed through two-dimensional discrete wavelet transform, and the low-frequency sub-band LL of the two images after the decomposition processing is further decomposed through two-level wavelet transform.

[0028] After the image decomposition processing is completed, discrete cosine transform is used to perform quantization compression processing on the two images after the two-level wavelet transform decomposition processing, removing the information ignored by the human visual system and reducing the amount of data in the images.

[0029] S2. The preprocessed infrared image and visible light image are respectively filtered through a high-pass filter to obtain feature images.

[0030] Specifically, the high-pass filter is used to filter the preprocessed infrared image and visible light image data, filtering out the low-frequency data and retaining the high-frequency data, and respectively obtaining the feature images of the infrared image and visible light image.

[0031] S3. The preprocessed infrared image and visible light image are respectively subjected to discrete cosine transform and wavelet transform processing to obtain detail images.

[0032] Specifically, first, the preprocessed image data are respectively subjected to inverse quantization processing, and the discrete cosine transform processing is respectively performed on the compressed infrared image and visible light image data, and they are decoded using the look-up table method to obtain the decoded infrared image and visible light image;

[0033] Then, the decoded infrared image and visible light image are subjected to wavelet transform to respectively obtain the detail images of the infrared image and visible light image.

[0034] S4. The feature images and detail images are fused, and the target information is recognized according to the obtained fused image.

[0035] Figure 2 It is a schematic diagram of the infrared image and visible light image fusion method provided by an embodiment of the present invention. As Figure 2 shown, the visible light image and infrared image are respectively processed to obtain their respective feature images and detail images. After the respective feature images and detail images are fused, the fused visible light image and infrared image are further fused to obtain the final fused image for target recognition.

[0036] After the feature image and the detail image are fused, the fused image is comprehensively evaluated and analyzed by an objective evaluation method to further determine the accuracy of target information recognition based on the fused image. Among them, the quality of the fused image can be quantitatively evaluated through evaluation systems such as image gradient information, peak signal-to-noise ratio (PSNR), structural similarity (SSIM), mutual information, and entropy (EN).

[0037] The target recognition method based on the fusion of infrared images and visible light images provided in this embodiment can be applied to mobile carriers such as unmanned vehicles and unmanned aerial vehicles.

[0038] Using the infrared and visible light image fusion method of the embodiment of the present invention enables the fused image to contain image local significant region information and image detail information; by using image decoding and encoding technologies, the speed of image fusion is faster and the real-time performance is better, and real-time detection and recognition of image target information can be achieved; it can be applied to a variety of mobile carriers and has a wide range of applications.

[0039] Figure 3 It is a schematic diagram of the target recognition system based on the fusion of infrared images and visible light images provided in the embodiment of the present invention. According to the embodiment of the present invention, a target recognition system based on the fusion of infrared images and visible light images is provided, as Figure 3 shown. The target recognition system based on the fusion of infrared images and visible light images according to the embodiment of the present invention specifically includes:

[0040] An image preprocessing module 30: configured to preprocess the collected infrared image and visible light image.

[0041] The image preprocessing module 30 is specifically configured to:

[0042] Edit the collected infrared image and visible light image into images with the same size through an image editor;

[0043] Perform decomposition processing on the infrared image and visible light image with the same size respectively through two-dimensional discrete wavelet transform, and perform secondary wavelet transform decomposition processing on the low-frequency subband LL of the decomposed image;

[0044] Perform quantization compression processing on the image after the secondary wavelet transform decomposition processing by using discrete cosine transform.

[0045] A feature extraction module 32: configured to filter the preprocessed infrared image and visible light image respectively through a high-pass filter to obtain a feature image.

[0046] The feature extraction module 32 is specifically configured to: filter the preprocessed infrared image and visible light image data through a high-pass filter, filter out low-frequency data, and retain high-frequency data to obtain a feature image.

[0047] Detail extraction module 34: It is used to perform discrete cosine transform and wavelet transform on the preprocessed infrared image and visible light image respectively to obtain detail images.

[0048] Specifically, the detail extraction module 34 is used for:

[0049] Perform inverse quantization on the preprocessed infrared image and visible light image data respectively through discrete cosine transform, and decode them using the look-up table method to form the decoded infrared image and visible light image;

[0050] Perform wavelet transform on the decoded infrared image and visible light image respectively to obtain detail images.

[0051] Fusion and recognition module 36: It is used to fuse the feature image and the detail image, and recognize the target information according to the obtained fused image.

[0052] The embodiment of the present invention is a system embodiment corresponding to the above method embodiment. The specific operations of each module can be understood with reference to the description of the method embodiment, and will not be elaborated here.

[0053] As Figure 4 shown, the embodiment of the present invention provides a target recognition device based on the fusion of infrared images and visible light images, including: a memory 40, a processor 42, and a computer program stored on the memory 40 and executable on the processor 42. When the computer program is executed by the processor 42, the following method steps are implemented:

[0054] S1. Preprocess the collected infrared image and visible light image.

[0055] When collecting images, the relative positions of the visible light camera and the infrared camera remain unchanged. After the image collection is completed, if the sizes of the collected visible light image and infrared image are different, it is necessary to first edit the visible light image and infrared image collected by the visible light camera and the infrared camera into images with the same size through an image editor.

[0056] After the image size processing is completed, decompose the visible light image and infrared image with the same size respectively through two-dimensional discrete wavelet transform, and perform a secondary wavelet transform decomposition on the low-frequency sub-band LL of the two decomposed images.

[0057] After the image decomposition processing is completed, perform quantization compression processing on the two images after the secondary wavelet transform decomposition through discrete cosine transform to remove the information ignored by the human visual system and reduce the amount of data in the images.

[0058] S2. Filter the preprocessed infrared image and visible light image respectively through a high-pass filter to obtain a feature image.

[0059] Specifically, a high-pass filter is used to filter the preprocessed infrared image and visible light image data, filtering out low-frequency data and retaining high-frequency data to respectively obtain the feature images of the infrared image and the visible light image.

[0060] S3. Respectively perform discrete cosine transform and wavelet transform on the preprocessed infrared image and visible light image to obtain detail images.

[0061] Specifically, first perform inverse quantization on the preprocessed image data respectively, and perform discrete cosine transform on the compressed infrared image and visible light image data respectively, and use the look-up table method to decode them to obtain the decoded infrared image and visible light image;

[0062] Then perform wavelet transform on the decoded infrared image and visible light image to respectively obtain the detail images of the infrared image and the visible light image.

[0063] S4. Fuse the feature images and detail images, and identify target information according to the obtained fused image.

[0064] After the feature images and detail images are fused, the fused image is comprehensively evaluated and analyzed by an objective evaluation method to further judge the accuracy of target information recognition based on the fused image; among them, the quality of the fused image can be quantitatively evaluated through evaluation systems such as image gradient information, peak signal-to-noise ratio PSNR, structural similarity SSIM, mutual information, and information entropy EN.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target recognition method based on the fusion of infrared images and visible light images, characterized in that, Including: S1. Preprocess the collected infrared images and visible light images; Step S1 specifically includes: Edit the collected infrared images and visible light images into images with the same size through an image editor; Perform decomposition processing on the infrared images and visible light images with the same size respectively through two-dimensional discrete wavelet transform, and perform secondary wavelet transform decomposition processing on the low-frequency subband LL of the decomposed images; Perform quantization compression processing on the images after the secondary wavelet transform decomposition processing by using discrete cosine transform; S2. Filter the preprocessed infrared images and visible light images respectively through a high-pass filter to obtain feature images; S3. Perform discrete cosine transform and wavelet transform processing on the preprocessed infrared images and visible light images respectively to obtain detail images; Step S3 specifically includes: Perform inverse quantization processing on the preprocessed infrared image and visible light image data respectively through discrete cosine transform, and decode them by using the look-up table method to form the decoded infrared image and visible light image; Perform wavelet transform on the decoded infrared image and visible light image respectively to obtain detail images; S4. Fuse the feature images and detail images, and identify the target information according to the obtained fused image.

2. The method according to claim 1, wherein Step S2 specifically includes: Filter the preprocessed infrared image and visible light image data through a high-pass filter, filter out the low-frequency data, and retain the high-frequency data to obtain feature images.

3. The method according to claim 1, wherein The method further includes Perform comprehensive evaluation and analysis on the fused image to judge the accuracy of image target information recognition; among them, the evaluation system for comprehensive evaluation and analysis includes: image gradient information, peak signal-to-noise ratio PSNR, structural similarity SSIM, mutual information, and information entropy EN.

4. An object recognition system based on the fusion of infrared images and visible light images, characterized in that, Including: Image preprocessing module: used to preprocess the collected infrared images and visible light images; The image preprocessing module is specifically used for: Edit the collected infrared images and visible light images into images with the same size through an image editor; Perform decomposition processing on the infrared images and visible light images with the same size respectively through two-dimensional discrete wavelet transform, and perform secondary wavelet transform decomposition processing on the low-frequency subband LL of the decomposed images; Perform quantization compression processing on the images after the secondary wavelet transform decomposition processing by using discrete cosine transform; Feature extraction module: used to filter the preprocessed infrared images and visible light images respectively through a high-pass filter to obtain feature images; Detail extraction module: used to perform discrete cosine transform and wavelet transform processing on the preprocessed infrared images and visible light images respectively to obtain detail images; The detail extraction module is specifically used for: Perform inverse quantization processing on the preprocessed infrared image and visible light image data respectively through discrete cosine transform, and decode them by using the look-up table method to form the decoded infrared image and visible light image; Perform wavelet transform on the decoded infrared image and visible light image respectively to obtain detail images; Fusion recognition module: used to fuse the feature images and detail images, and identify the target information according to the obtained fused image.

5. The system according to claim 4, characterized in that, The feature extraction module is specifically configured to: filter the preprocessed infrared image and visible light image data through a high-pass filter, filter out low-frequency data, retain high-frequency data, and obtain a feature image.

6. An object recognition device based on the fusion of infrared images and visible light images, characterized in that, It includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the target recognition method according to any one of claims 1 to 3 are implemented.

Images