A multi-source image fusion method and system based on salient region detection

Abstract

The application discloses a kind of multi-source image fusion method and system based on salient region detection, which comprises: the decomposition of background and edge to source image, obtain basic layer and first detail layer;Based on salient region detection and guided filter fusion model, the first detail layer is decomposed, to obtain salient detail layer and background detail layer;Using the fusion rule of modified Laplacian energy and local energy to the fusion of basic layer;Based on the method of taking the absolute value of big to the fusion of background detail layer;Combining fusion basic layer, salient detail layer and fused background detail layer, obtain final fused image.The system includes: first decomposition module, second decomposition module, first fusion module, second fusion module and final fusion module.By using the application, significant structure, edge information and energy information in source image can be effectively retained.The application can be widely applied in the field of image fusion.

Description

Technical Field

[0001] This invention relates to the field of image fusion, and in particular to a multi-source image fusion method and system based on salient region detection. Background Technology

[0002] Image fusion is the process of combining images from different sensors or images acquired by the same sensor at different times to obtain a fused image. This fused image contains useful information from all the source images to better describe the scene and prepare for subsequent image processing tasks such as image segmentation, object detection, and image recognition. Image fusion technology plays a vital role in major fields such as medical imaging, microscopic imaging, computer vision, and military applications.

[0003] Existing image fusion methods are mainly based on spatial domain methods, which process the source image directly at the pixel level. However, most spatial domain-based methods rely to some extent on complex and inefficient segmentation algorithms, which can introduce erroneous information into the fused image if not handled properly. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide a multi-source image fusion method and system based on salient region detection, which can effectively preserve salient structure, edge information, and energy information in the source images.

[0005] The first technical solution adopted in this invention is: a multi-source image fusion method based on salient region detection, comprising the following steps: The source image is decomposed into background and edges to obtain a base layer and a first detail layer; The first detail layer is decomposed based on a fusion model of salient region detection and guided filtering to obtain a salient detail layer and a background detail layer. The base layer is fused using the fusion rule of modified Laplace energy and local energy to obtain a fused base layer; The background detail layer is blended based on the method of taking the largest absolute value to obtain the blended background detail layer; The final merged image is obtained by combining the base layer, the salient detail layer, and the background detail layer.

[0006] Furthermore, the step of decomposing the source image into a base layer and a first detail layer specifically includes: Acquire multiple source images; Embed the least squares method into a bilateral filter; The source image is decomposed using a bilateral filter to obtain a base layer with background information and a first detail layer with edge texture.

[0007] Furthermore, the formula for decomposing the source image based on the bilateral filter is expressed as follows:

[0008] In the above formula, Indicates the input image. This represents the output image, where S represents the number of pixels. This represents the gradient of the output image. and These represent the gradients along the x and y axes of the input image smoothed using BLF, respectively. This represents the equilibrium parameter.

[0009] Furthermore, the step of decomposing the first detail layer into a salient detail layer and a background detail layer using the fusion model based on salient region detection and guided filtering specifically includes: The first detail layer is input into the fusion model of salient region detection and guided filtering. The first detail layer is processed based on the salient region detection of deformation smoothness constraint to obtain the salient information map; A threshold correction method is used to process the saliency map to obtain an initial weight map; The overlapping parts in the initial weight map are removed and processed by guided filtering to obtain the final weight map; The first detail layer is decomposed based on the final weight map to obtain the salient detail layer and the background detail layer.

[0010] Considering the influence of boundaries, we use guided filtering to process the saliency results to obtain a better visual effect for the fused image.

[0011] Furthermore, the step of processing the first detail layer to obtain a salient information map based on deformation smoothness constraints for salient region detection specifically includes: The first detail layer is segmented into superpixels and represented as a graph; A rough map is generated using a background seed and deformation smoothness manifold sorting propagation model. Construct an object graph using object proposals; By combining information from the rough map and the object map, a salient information map is generated.

[0012] This preferred step generates a coarse map to detect missing areas with low contrast to the background. Since the coarse map is often affected by background noise, an object map is constructed through object proposal. The object map has a better effect in highlighting objects and suppressing background noise.

[0013] Furthermore, the saliency information map is represented as follows: In the above formula, Represents a salient information graph. express transpose, Represents the balance parameters. Representing an object diagram, Represents a diagonal matrix. Represents a rough map. , and express The diagonal matrix and volume, Indicates by The calculated weight matrix.

[0014] Furthermore, the calculation method for the fused background detail layer is as follows: In the above formula, and This represents different background detail layers.

[0015] Furthermore, the step of combining the base fusion layer, the salient detail layer, and the background detail fusion layer to obtain the final fused image specifically includes: The salient detail layers are added together and blended to obtain the blended salient detail layer; The saliency detail layer and the background detail layer are added together and blended to obtain the blended detail layer; The detail layer and the base layer are added together to obtain the final merged image.

[0016] The second technical solution adopted in this invention is: a multi-source image fusion system based on salient region detection, comprising: The first decomposition module is used to decompose the background and edges of the source image to obtain the base layer and the first detail layer. The second decomposition module decomposes the first detail layer based on a fusion model of salient region detection and guided filtering to obtain a salient detail layer and a background detail layer. The first fusion module is used to fuse the base layer using the fusion rule of modified Laplace energy and local energy to obtain a fused base layer; The second fusion module fuses the background detail layer based on the method of taking the largest absolute value, thus obtaining a fused background detail layer; The final fusion module combines the base fusion layer, the salient detail layer, and the background detail fusion layer to obtain the final fused image.

[0017] The beneficial effects of the method and system of this invention are as follows: This invention provides a multi-source image fusion method based on least squares embedded bilateral filters and salient target detection. First, the source image is decomposed into a base layer containing background information and a detail layer containing edge textures using least squares embedded bilateral filters. For the fusion of the detail layer, the detail layer is considered as a superposition of salient regions and background information. A detail layer fusion model based on deformation-constrained salient target detection and guided filtering is designed, enabling the salient structure and edge information of the source image to be successfully transformed into the fused image. For the base layer obtained from the decomposition, a fusion rule of modified Laplacian energy and local energy is directly adopted. This method can effectively preserve the salient structure, edge, and energy information in the source image, achieving good fusion results in image fusion tasks. Attached Figure Description

[0018] Figure 1 This is a flowchart of the steps of a multi-source image fusion method based on salient region detection according to the present invention; Figure 2 This is a structural block diagram of a multi-source image fusion system based on salient region detection according to the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0020] like Figure 1 As shown, the present invention provides a multi-source image fusion method for salient region detection, the method comprising the following steps: S1. Obtain multiple first source images, and use the least squares method to embed a bilateral filter to decompose the source images into a base layer with background information and a detail layer with edge texture. For a given image g, the specific calculation of the output image u after passing through the bilateral filter is as follows: Where S and T represent pixels. Spatial kernel. and distance from the core It is a Gaussian kernel. Determine space support, Control the sensitivity to edges.

[0021] By embedding bilateral filter smoothing of image gradients into a least squares framework, effective edge-preserving smoothing is achieved. The specific details are as follows: Given an input image g, the output image u after passing through a least-squares embedded bilateral filter is: in and These represent the gradients along the x and y axes of the input image g, smoothed using BLF, respectively. This represents the equilibrium parameter.

[0022] Solving the least squares embedded bilateral filter model in the equation can be achieved as follows: in, and These are the Fast Fourier Transform (FFT) and Inverse Fast Fourier Transform (IFFT) operators. express The complex conjugate of . F(1) is the FFT of the δ function.

[0023] Each source image is first decomposed into a base layer containing background information and a detail layer containing edge texture. The specific details are as follows: in Represents pixel coordinates. It is the source image. and They are respectively composed of source images and The basic layer of decomposition. This represents a least-squares embedded bilateral filter used for smoothing the image. After obtaining the base layer, the detail layer can be directly extracted by subtracting the base layer from the source image.

[0024] S2. A fusion model based on deformation smoothness constraint-based salient region detection and guided filtering is used to decompose the detail layer into a salient detail layer and a background detail layer.

[0025] S201, a salient information map is obtained by using salient region detection based on deformation smoothness constraints.

[0026] Specifically, the salient region detection under deformation smoothness constraints mainly includes three steps: First, the image is segmented into superpixels and represented as a graph; second, a coarse map is generated using a background seed and a deformation smoothness manifold ordering propagation model. Furthermore, an object graph was constructed through object proposals. Finally, by combining the information from the coarse map and the object map, a saliency detection map is generated.

[0027] Generate saliency detection map The description is as follows: in This represents the object graph obtained from the bounding box; Represents a diagonal matrix. . This represents a rough map generated by a background seed and a deformation smoothness manifold sorting propagation model. and for The diagonal matrix and volume; It is by The calculated weight matrix, Generate a saliency detection map. The optimal solution is expressed as: Will Normalized to [0,1], assigned to the corresponding superpixel to generate a saliency detection map.

[0028] S202, a threshold correction method is used to process the salient information graph to obtain an initial weight graph for decomposing the detail layer.

[0029] A threshold correction method is used to process the saliency graph to obtain an initial weight graph for decomposing the detail layer. in This indicates that a DSC detection operation is performed on the image. For the threshold, and To be carried out respectively and The initial weight map obtained by performing DSC detection.

[0030] S203 removes the overlapping parts of the two initial weight maps and processes them using guided filtering to obtain the final weight map.

[0031] Will and overlapping parts Remove: in and This represents the saliency weight map after removing overlapping parts. Considering the influence of boundaries, we use guided filtering to process the saliency results to obtain a better visual effect for the fused map.

[0032] in Indicates a pilot filter. and This represents the weighted graph used to decompose the detail layer graph.

[0033] S204. The final weight map is used to process the detail layer to obtain the salient detail layer and the background detail layer.

[0034] use and right and By decomposing the layers, we can obtain salient detail layers and ordinary detail layers, as described below: in and express and The significant detail layer, and This represents the background detail layer.

[0035] S3. The base layer is fused using the fusion rule of modified Laplace energy and local energy.

[0036] For the basic layer The SML definition is as follows: For size The window, which is The modified Laplace operator (ML) is defined as follows: base layer The definition of LEN is as follows: The window size is .

[0037] Finally, the fusion of the base layer can be summarized as follows: S4. Use the method of taking the largest absolute value to merge the background detail layer of the image.

[0038] For blending background detail layers We obtain the result by taking the largest absolute value. S5. The background detail layer, salient detail layer and base layer of the fused image obtained in steps S3 and S4 are added together to obtain the fused image.

[0039] The fused image is obtained as follows: like Figure 2 As shown, a multi-source image fusion system based on salient region detection includes: The first decomposition module is used to decompose the background and edges of the source image to obtain the base layer and the first detail layer. The second decomposition module decomposes the first detail layer based on a fusion model of salient region detection and guided filtering to obtain a salient detail layer and a background detail layer. The first fusion module is used to fuse the base layer using the fusion rule of modified Laplace energy and local energy to obtain a fused base layer; The second fusion module fuses the background detail layer based on the method of taking the largest absolute value, thus obtaining a fused background detail layer; The final fusion module combines the base fusion layer, the salient detail layer, and the background detail fusion layer to obtain the final fused image.

[0040] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0041] A multi-source image fusion device based on salient region detection: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a multi-source image fusion method based on salient region detection as described above.

[0042] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0043] A storage medium storing processor-executable instructions, characterized in that: the processor-executable instructions, when executed by a processor, are used to implement a multi-source image fusion method based on salient region detection as described above.

[0044] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0045] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A multi-source image fusion method based on salient region detection, characterized in that, Includes the following steps: The source image is decomposed into background and edges to obtain a base layer and a first detail layer; The first detail layer is decomposed based on a fusion model of salient region detection and guided filtering to obtain a salient detail layer and a background detail layer. The base layer is fused using the fusion rule of modified Laplace energy and local energy to obtain a fused base layer; The background detail layer is blended based on the method of taking the largest absolute value to obtain the blended background detail layer; The final fused image is obtained by combining the base layer, the salient detail layer, and the background detail layer. The step of decomposing the first detail layer into a salient detail layer and a background detail layer using the fusion model based on salient region detection and guided filtering specifically includes: The first detail layer is input into the fusion model of salient region detection and guided filtering; The first detail layer is processed based on the salient region detection of deformation smoothness constraint to obtain the salient information map; A threshold correction method is used to process the saliency map to obtain an initial weight map; The overlapping parts in the initial weight map are removed and processed by guided filtering to obtain the final weight map; The first detail layer is decomposed based on the final weight map to obtain the salient detail layer and the background detail layer; The fusion process of the base layer is represented as follows: 。 2. The multi-source image fusion method based on salient region detection according to claim 1, characterized in that, The step of decomposing the source image into background and edges to obtain a base layer and a first detail layer specifically includes: Acquire multiple source images; Embed the least squares method into a bilateral filter; The source image is decomposed using a bilateral filter to obtain a base layer with background information and a first detail layer with edge texture.

3. The multi-source image fusion method based on salient region detection according to claim 2, characterized in that, The formula for decomposing the source image based on a bilateral filter is expressed as follows: In the above formula, Indicates the input image. This represents the output image, where S represents the number of pixels. This represents the gradient of the output image. and These represent the gradients along the x and y axes of the input image smoothed using BLF, respectively. This represents the equilibrium parameter.

4. The multi-source image fusion method based on salient region detection according to claim 2, characterized in that, The step of processing the first detail layer to obtain a salient information map based on deformation smoothness constraints for salient region detection specifically includes: The first detail layer is segmented into superpixels and represented as a graph; A rough map is generated using a background seed and deformation smoothness manifold sorting propagation model. Construct an object graph using object proposals; By combining information from the rough map and the object map, a salient information map is generated.

5. The multi-source image fusion method based on salient region detection according to claim 4, characterized in that, The salient information map is represented as follows: In the above formula, Represents a salient information graph. express transpose, Represents the balance parameters. Representing an object diagram, Represents a diagonal matrix. Represents a rough map. , and express The diagonal matrix and volume, Indicates by The calculated weight matrix.

6. The multi-source image fusion method based on salient region detection according to claim 4, characterized in that, The calculation method for the blended background detail layer is as follows: In the above formula, and This represents different background detail layers.

7. The multi-source image fusion method based on salient region detection according to claim 4, characterized in that, The step of combining the base layer, the salient detail layer, and the background detail layer to obtain the final fused image specifically includes: The salient detail layers are added together and blended to obtain the blended salient detail layer; The saliency detail layer and the background detail layer are added together and blended to obtain the blended detail layer; The detail layer and the base layer are added together to obtain the final merged image.

8. A multi-source image fusion system based on salient region detection, characterized in that, A method for performing a multi-source image fusion method based on salient region detection as described in claim 1 includes: The first decomposition module is used to decompose the background and edges of the source image to obtain the base layer and the first detail layer. The second decomposition module decomposes the first detail layer based on a fusion model of salient region detection and guided filtering to obtain a salient detail layer and a background detail layer. The first fusion module is used to fuse the base layer using the fusion rule of modified Laplace energy and local energy to obtain a fused base layer; The second fusion module fuses the background detail layer based on the method of taking the largest absolute value, thus obtaining a fused background detail layer; The final fusion module combines the base fusion layer, the salient detail layer, and the background detail fusion layer to obtain the final fused image.

Images