Unlock instant, AI-driven research and patent intelligence for your innovation.

Image fusion method based on total variation deep learning and application and system thereof

A deep learning and image fusion technology, applied in the field of image fusion, can solve problems such as difficult to solve, blurred edges of fusion images, low edge texture evaluation indicators, etc., to achieve good robustness and consistency, good interpretability, and good image clear effect

Pending Publication Date: 2020-06-30
XIHUA UNIV
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, in the above two methods, the method based on deep neural network region recognition in the prior art has similar consistency problems to the fusion of salient region discrimination, and the fusion of network representation levels has the difficulty of intuitively explaining the fusion rules and methods; and Generally, the total variation solution method requires the model to be a convex optimization model, which greatly hinders the design and solution of the total variation model for image fusion, and also makes it difficult for this type of method to obtain better evaluation indicators in the practical application of image fusion. and visual effects, mainly manifested in the blurred edge of the fused image, and the low edge texture evaluation index
On the one hand, the existing total variational model that can be solved is difficult to meet the intuitive physical meaning of image fusion, and the design of a new total variational model is difficult to solve by traditional methods due to factors such as non-convexity.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image fusion method based on total variation deep learning and application and system thereof
  • Image fusion method based on total variation deep learning and application and system thereof
  • Image fusion method based on total variation deep learning and application and system thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0096] The objective function is optimized by formula (3) to perform image fusion processing on 12 groups of visible light and infrared data, and each group runs 10 times to calculate the average value and variance. The results of each evaluation index are shown in Table 1 (where avg, std represent the average Difference).

[0097] Table 1 Fusion effect evaluation of different fusion images

[0098]

[0099]

[0100] As can be seen from Table 1, the standard deviations of each index of the gained 12 groups of fusion images are very small, and most of them are less than 1‰ of the average value, which shows that the method of the present invention has little difference in the results of each operation, and it has better consistency, and the robustness of the algorithm is good.

Embodiment 2

[0102] The comparison and evaluation of different algorithms are carried out through the following process:

[0103] (1) Collect visible light image and infrared image data in different situations, and establish a data set, as shown in the attached image figure 2 shown;

[0104] (2) Based on all the data of the data set, the quality of the fusion method adopting different algorithms is evaluated as a whole;

[0105] (3) Select representative data, that is, a set of urban architectural scene data and a set of wild natural scene images, and compare their fusion effects.

[0106] Among them, the fusion methods adopted include FusionGAN, DenseFuse, DeepMSTFuse, SWTDCTSF, FeatureExtractFuse, MEGC, DeepFuse, GTF, NSCT, JSR fusion methods in the prior art and the method of the present invention.

[0107] On the verification data set, the results of the overall evaluation of different fusion methods are shown in Table 2 below:

[0108] Table 2 Fusion effect evaluation of different...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an image fusion method based on total variation deep learning and application and system thereof, and the fusion method comprises the steps: taking a source image and / or a pre-fused image feature value as input, taking an optimization target function obtained through a total variation model as a loss function, and obtaining a fused image through a convolutional neural network. The method can be used for fusion of the infrared image and the visible light image, is a universal image fusion method which is non-locally optimized and can be used for non-convex function optimization, has relatively high robustness, and is remarkably superior to a current advanced image fusion method in two aspects of objective evaluation indexes and a visual effect of a fused image.

Description

technical field [0001] The invention relates to the technical field of image fusion. Background technique [0002] Image fusion is to fuse the feature information of multiple source images of the same scene into a comprehensive image, which has good visual effects while containing rich information of the source images. Currently, image fusion methods can be classified into the following seven categories: multiscale transformation, sparse representation, saliency detection, subspace, neural network, variational and hybrid models. [0003] Among them, the method of hybrid model is to combine various types of image fusion methods, and make use of the advantages of various methods to make up for the shortcomings of a single method, thereby improving the quality of fusion images. Multi-scale transformation is currently the most mature and successful image fusion method, which mainly includes three steps: first, decompose and transform the source image into low-frequency and high...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T5/50G06T5/40
CPCG06T5/50G06T5/40G06T2207/20221G06T2207/20081G06T2207/20084
Inventor 谢春芝高志升
Owner XIHUA UNIV