A multi-exposure image fusion method and system based on cross-layer random walk

Abstract

The application discloses a multi-exposure image fusion method and system based on cross-layer random walk, and the method comprises the following steps: acquiring multi-exposure images, establishing a cross-layer random walk model corresponding to the exposure images, wherein the cross-layer random walk model is a double-layer topological structure; obtaining optimal probability mapping corresponding to the exposure images based on the cross-layer random walk model; and obtaining a fusion image through weighted fusion according to the exposure images and the optimal probability mapping corresponding to the exposure images, so that the technical problem of low multi-exposure image fusion quality is solved, the details of the brightest / darkest area in the scene under different exposure conditions can be revealed, color distortion in the fusion image is avoided, and a high-quality fusion image is obtained.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of image processing, in particular to a multi-exposure image fusion method and system based on cross-layer random walk. BACKGROUND

[0002] Multi-exposure image fusion is one of the methods to generate high-quality images. By fusing image sequences under different exposures, an image with more information and more perceptual appeal than any input image can be obtained. However, the existing multi-exposure image fusion method will cause unnecessary information loss in the fusion process, often leading to a decrease in the visual fidelity of the fused image. In addition, images using a single light source can only capture scenes within a certain dynamic range, which will cause over-bright or over-dark areas in the captured area, such as backlight, indoor weak light, and industrial extremely dark environment, etc., resulting in poor fusion effect of the existing multi-exposure image fusion algorithm, producing high-light or low-dark areas.

[0003] At present, multi-exposure image fusion methods are mainly divided into two categories: pixel-based methods and transform domain-based methods. Among them, the pixel-based multi-exposure image method will produce noise in the fusion process and will amplify the high-light or low-dark area, usually requiring corresponding post-processing. While the transform domain-based multi-exposure image method will cause loss of image details during feature conversion. In summary, most of the existing multi-exposure fusion methods usually cause loss of details, underexposure and overexposure. In order to solve these problems, this paper proposes a cross-layer random walk model for multi-exposure image fusion. The model is designed as a double-layer graph topology with an attribute layer to achieve global optimization. In order to capture fine scene details under different exposure conditions, latent structural information is applied between layers. In addition, contrast information is introduced as a prior in the graph topology model to improve local contrast and obtain visually comfortable fusion fidelity. Therefore, a multi-exposure fusion framework is established based on the above steps. This framework not only restores high-quality scene details, but also prevents the effects of color and noise.

[0004] The patent with publication number CN 114187192 A discloses an image processing method based on multi-exposure fusion. An exposure fusion method based on YUV color channels is proposed for the invention, which removes noise in short-exposure images through short-exposure image noise estimation; converts the RGB color model into the YUV color model; synthesizes the values of the obtained Y channel and solves the high dynamic range image in combination with the YUV three channels; and finally converts to the RGB channel to obtain the final fused image.

[0005] However, in fact, this method will cause serious loss of details in the fusion process. This method can effectively remove noise by estimating noise, but this will lose the details and texture information of the image.

[0006] The patent with publication number CN 115272149 A discloses a high dynamic image fusion method based on an unsupervised deep learning network. The invention proposes a high dynamic image fusion method based on an unsupervised deep learning network. Through the design of an encoder, a decoder and a weight map, the network can realize unsupervised learning without labeled training set samples.

[0007] However, the fusion effect of this method for overexposed and underexposed areas is still poor, and it cannot eliminate high light and low light areas. Although this method does not require labeled training set samples, it still requires corresponding true value training data sets, which is poor in robustness for some extreme environments that cannot obtain true values. SUMMARY

[0008] The multi-exposure image fusion method and system based on cross-layer random walk provided by the present application solve the technical problem of low multi-exposure image fusion quality.

[0009] To solve the above technical problems, the multi-exposure image fusion method based on cross-layer random walk proposed by the present application comprises:

[0010] Obtain a multi-exposure image, which includes multiple exposure images.

[0011] Establish a cross-layer random walk model corresponding to the exposure image, which is a double-layer topological structure.

[0012] Based on the cross-layer random walk model, obtain the optimal probability mapping corresponding to the exposure image.

[0013] According to the exposure image and the optimal probability mapping corresponding to the exposure image, weighted fusion is performed to obtain a fusion image.

[0014] Further, establishing a cross-layer random walk model corresponding to the exposure image comprises:

[0015] Construct a double-layer topological structure containing an intensity layer and an attribute layer, which is specifically represented as:

[0016]

[0017] Wherein, Different layers in the double-layer topological structure are represented as, And The intensity layer node set and the attribute layer node set are represented as, respectively, α And β The edge set between the intensity layer node and the attribute layer node is represented as, The attribute set of the intensity layer node and the edge is represented as, The attribute set of the attribute layer node and the edge is represented as.

[0018] Construct the intra-layer transition matrix and inter-layer transition matrix for walking in a cross-layer random walk model.

[0019] The overall transition matrix is ​​obtained based on the intra-layer transition matrix and the inter-layer transition matrix.

[0020] Furthermore, constructing the inter-layer transition matrix for walking in the cross-layer random walk model includes:

[0021] Obtain the set of strong connections of the strength layer nodes. Specifically, the set of strong connections of the strength layer nodes refers to the set of edge weights between the strength layer node and its neighboring nodes that are greater than the average edge weights in the neighborhood of the strength layer node.

[0022] Based on the set of strong connections of the strength layer nodes, the information content of the strength layer nodes is obtained, specifically:

[0023]

[0024] in, Let represent the absolute number of subsets in the strongly connected set of the i-th strength layer node, where i = {1, ..., N}, and N represents the total number of strength layer nodes or attribute layer nodes. i Let P be a neighboring node of the i-th intensity layer node. ij} N×N This represents the intra-layer transition probability matrix.

[0025] The inter-layer edge weights are derived based on the information content of the strength layer nodes, specifically:

[0026]

[0027] in, Let e ​​represent the edge weight between the i-th intensity layer node and the j-th attribute layer node, where e is a non-zero constant.

[0028] The inter-layer edge weights are normalized to obtain the inter-layer transition matrix, specifically:

[0029]

[0030] in, This represents the inter-layer transition probability between the i-th intensity layer node and the j-th attribute layer node. Let d represent the inter-layer transition matrix, ξ be the probability that a random walker reaches a node in the strength layer and kills or stays in that node, and d be the probability that a random walker reaches a node in the strength layer and kills or stays in that node. αβ The degree matrix representing the relationship between the strength layer and the attribute layer is derived from the inter-layer edge weight matrix. The summation of the elements in the array is obtained.

[0031] Further, based on the intra-layer transition matrix and the inter-layer transition matrix, the specific formula of the overall transition matrix is obtained as follows:

[0032]

[0033] wherein Q represents the overall transition matrix of the double-layer topological structure, and represent the ith and jth intensity layer nodes respectively, and represent the ith and jth attribute layer nodes respectively, Δ is a killer node in the intensity layer, S k (k = 1,..., K) is a stay node in the intensity layer, K is the total number of stay nodes, ξ is the probability of a random walker reaching the killer node and the stay node in the intensity layer, d all = d α + d β + d αβ is a degree matrix corresponding to the double-layer topological structure, d α represents the degree matrix of the intensity layer, which is obtained by summing the elements in the edge weight matrix of the intensity layer, d β represents the degree matrix of the attribute layer, which is obtained by summing the elements in the edge weight matrix of the attribute layer.

[0034] Further, based on the cross-layer random walk model, obtaining the optimal probability mapping corresponding to the exposure image includes:

[0035] obtaining the edge weight matrix of the exposure image based on the contrast information, and the specific formula is as follows:

[0036]

[0037]

[0038] wherein Wlm represents the edge weight matrix of the mth exposure image based on the contrast information in the multi-exposure image, l m represents the contrast information graph node of the mth exposure image, represents the edge weight between the contrast information graph node of the mth exposure image and the intensity layer, I m represents the mth exposure image, c m represents the spatial contrast of the mth exposure image, |c m | represents taking the absolute value of c m , Erf(·) is a sigmoid function, which is monotonically increasing, is a Gaussian weighting function, σ1, σ2 are variances, and N1 = N*N.

[0039] The edge weight matrix of the exposure image based on contrast information is added as a label prior into the cross-layer random walk model to obtain a contrast overall transition matrix, specifically:

[0040]

[0041] Wherein, The contrast overall transition matrix is represented, The degree matrix d of the cross-layer random walk model with the label prior, l The degree matrix between the graph node of the exposure image based on contrast information and the intensity layer is represented, and is obtained by summing the elements in the edge weight matrix of the exposure image based on contrast information.

[0042] Based on the contrast overall transition matrix, a target function for calculating the optimal probability map is obtained.

[0043] The target function is optimized and solved to obtain the optimal probability mapping corresponding to the exposure image.

[0044] Further, the specific formula of the target function is:

[0045]

[0046] Wherein, The node is represented, The arrival probability of the arrival node in the intensity layer, The node is represented, The arrival probability of the arrival node in the attribute layer to the intensity layer, The indicator vector is represented, which indicates that when the selected node is the arrival node, at this time Otherwise k={1,...,K} is the index of the arrival node, μ is a penalty factor, t, m={1,...,M}.

[0047] Further, the specific formula of the fused image obtained by weighted fusion according to the exposure image and the optimal probability mapping corresponding to the exposure image is:

[0048]

[0049] Wherein, I * The final fused image is represented, I m The mth exposure image is represented, (r α ) m The optimal probability mapping corresponding to the mth exposure image is represented,

[0050] The multi-exposure image fusion system based on cross-layer random walk provided by the application comprises:

[0051] The memory, the processor and the computer program stored on the memory and capable of running on the processor, the processor implements the steps of the multi-exposure image fusion method based on cross-layer random walk provided by the application when executing the computer program.

[0052] The effects of the application specifically include:

[0053] (1) The application designs a cross-layer random walk model, which is composed of an intensity layer, an attribute layer and two walk strategies. The intensity layer and the attribute layer are both weighted undirected graphs. The former mainly captures the similarity of the intensity between node pairs. The latter mainly captures the difference between different attributes of node pairs and provides more information for the next step of the random walker. The walk strategy is composed of intra-layer walk strategy and inter-layer walk strategy. The intra-layer walk strategy captures node information to improve the compactness of the boundary, and the inter-layer walk strategy selects the next access node by considering the common neighbor information (the information amount of the current access node in the intensity layer).

[0054] (2) By designing the node information amount and the inter-layer transition probability matrix, the application can effectively preserve the detail information of the image. Specifically, by calculating the transition probability matrix, the cross-layer random walk model allocates higher probability to the neighbor nodes with high information amount and lower probability to the neighbor nodes with low information amount. In this way, the walk range of the cross-layer random walk model can be limited to the region with consistent intensity (the node domain with high information amount), thereby enhancing the boundary adhesion and detail preservation ability of the result.

[0055] (3) In order to eliminate underexposed and overexposed regions, the application designs an edge weight function of contrast information to maintain the consistency of image contrast. Specifically, the local contrast should be biased to provide fewer pixels with local brightness changes. In order to avoid rapid changes in local contrast, the application uses an error function to modulate the estimated contrast. In addition, a Gaussian weighting function reduces the exposure intensity to prevent the image from being affected by improper exposure.

[0056] (4) Based on the intra-layer transition probability matrix, the inter-layer transition probability matrix and the label prior, the application establishes a multi-exposure fusion framework. The framework models the multi-exposure fusion problem and converts it into a general energy optimization problem of calculating the membership probability vector. By minimizing the three indexes in the objective function, the optimal weight probability mapping of each exposure image is evaluated. Among the three indexes, two transition matrices assign high weights to the boundary structure with rapid gradient changes, effectively reducing the halo artifacts around the edges limited by reasonable structure protection. The label prior assigns high weights to the contrast with slow changes, effectively improving the scene contrast. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1Fig. 2 is a schematic diagram of a multi-exposure image fusion method based on cross-layer random walk of the second embodiment of the present application;

[0058] Figure 2 Fig. 3 is a schematic diagram of a double-layer random walk model of the second embodiment of the present application;

[0059] Figure 3 Fig. 4 is a schematic diagram of a double-layer random walk model based on label prior of the second embodiment of the present application;

[0060] Figure 4 Fig. 5 is a schematic diagram of a multi-exposure image fusion result under night scene of the second embodiment of the present application;

[0061] Figure 5 Fig. 6 is a structure block diagram of a multi-exposure image fusion system based on cross-layer random walk of the second embodiment of the present application.

[0062] Reference signs:

[0063] 10, memory; 20, processor. DETAILED DESCRIPTION

[0064] In order to facilitate the understanding of the present application, the present application will be described in more detail below in conjunction with the drawings and preferred embodiments, but the protection scope of the present application is not limited to the following specific embodiments.

[0065] The embodiments of the present application will be described in detail below in conjunction with the drawings, but the present application can be implemented in various different ways limited and covered by the claims.

[0066] Embodiment one

[0067] The multi-exposure image fusion method based on cross-layer random walk provided by the first embodiment of the present application comprises:

[0068] Step S101, a multi-exposure image is acquired, and the multi-exposure image comprises a plurality of exposure images.

[0069] Step S102, a cross-layer random walk model corresponding to the exposure image is established, and the cross-layer random walk model is a double-layer topological structure.

[0070] Step S103, an optimal probability mapping corresponding to the exposure image is obtained based on the cross-layer random walk model.

[0071] Step S104, a fusion image is obtained by weighted fusion according to the exposure image and the optimal probability mapping corresponding to the exposure image.

[0072] The multi-exposure image fusion method based on cross-layer random walk provided by the embodiment of the application solves the technical problem of low multi-exposure image fusion quality, can reveal the details of the brightest / darkest area in the scene under different exposure conditions, and avoids color distortion in the fused image, thereby obtaining a high-quality fused image.

[0073] Embodiment two

[0074] As shown in the following table, the multi-exposure image fusion method based on cross-layer random walk provided by the embodiment includes the following steps: Figure 1

[0075] S1: A cross-layer random walk model based on a double-layer graph topology is built, and the model includes: an intensity layer, an attribute layer, and construction of a walking strategy.

[0076] S2: Based on the cross-layer random walk model established in step S1, a layer-in transfer matrix and a layer-to-layer transfer matrix corresponding to the double-layer graph topology are designed. The fine scene details under different exposure conditions are captured by applying potential structural information between layers.

[0077] S3: Contrast information is added as prior knowledge to the cross-layer random walk model established in step S1, so as to improve the contrast of the fused image.

[0078] The specific implementation scheme is as follows:

[0079] S1: A double-layer random walk model is established:

[0080] The multi-exposure fusion method can be expressed as a probability synthesis process. A multi-exposure image sequence is defined as where M represents the number of input pictures, and each picture has N pixels. The fused image I * can be written as:

[0081]

[0082] where i is the pixel index of the image coordinate (x i ,y i ), and p represents the probability of pixel i in I M and

[0083] The embodiment first establishes a double-layer graph topology. Define as a double-layer topology graph with node alignment, where​ is a set of nodes, each of which is associated with an image attribute. is a set of edges between the nodes. represents different layers in the two-layer graph. Each edge is an ordered pair and is associated with a weight . is undirected, so there are and represents a set of attributes for each node and edge, where is the pixel intensity of the image, and is other attributes of the image, such as texture or structure, etc. The relationship between nodes in each layer is defined independently, and each node can be connected to other nodes through two types of links: intra-layer links and inter-layer links. Intra-layer links represent the connection between two nodes in the same layer, and inter-layer links represent the connection between two nodes in different layers. The degree matrix associated with each layer is d α represents the degree matrix of the intensity layer and d β represents the degree matrix of the attribute layer. represents the degree matrix between the intensity layer and the attribute layer, denotes the edge weight between the i-th intensity layer node and the j-th attribute layer node.

[0084] Secondly, the intensity layer, attribute layer and walk strategy in the two-layer topological graph are defined as follows:

[0085] Intensity layer: This layer is a weighted undirected graph, defined as graph It mainly captures the similarity of intensity between node pairs. In this layer, there are two special nodes: kill node Δ, stay node S k (k = {1,..., K}). The kill node mainly affects nodes with small degrees (usually the boundary of well-exposed regions). This reduces the case where the walk crosses well-exposed regions, which can improve the smoothness at the boundary. The stay node does not affect the walk. It is used to supplement the broken sub-Markov property at the boundary of overexposure and underexposure, such as Figure 2 .

[0086] Attribute layer: This layer is a weighted undirected graph, defined as graph It mainly captures the difference between different attributes of node pairs, and provides more information for the next step of the random walker starting from the node.

[0087] Walking strategy: The first walking strategy is the same layer walking. That is: the starting and target nodes are in the same layer, and do not need to cross layers directly to walk; the second strategy is cross-layer walking. That is: switching from the starting node to the corresponding node in a different layer, and then walking to the target node through multiple steps.

[0088] S2: Design of inter-layer and intra-layer transition matrix:

[0089] For the cross-layer random walk model, the most important thing is how to traverse the double-layer graph topology, that is, to make the next step node selection. The transition matrix of the cross-layer random walk model has two types: intra-layer transition probability matrix and inter-layer transition probability matrix. The intra-layer transition probability matrix P = {p ij} N×N The inter-layer transition probability matrix P = {p describes the link relationship between edges of different layers of vertices.

[0090] Intra-layer transition matrix: mainly composed of strength layer transition matrix and attribute layer transition matrix. The strength layer is a weighted undirected graph with two auxiliary nodes, and the structure layer is a weighted undirected graph without auxiliary nodes. The edge weight of the double layer is defined as:

[0091]

[0092] where and are the pixel colors at two nodes and in the Lab color space.

[0093] Two auxiliary nodes are added in the strength layer to improve the compactness of the boundary. Therefore, the new adjacency matrix is defined as:

[0094]

[0095] where ξ is the probability of the random walker reaching the killer node and the rest node in the strength layer. and represent the i-th and j-th strength layer nodes, and represent the i-th and j-th attribute layer nodes, Δ is the killer node in the strength layer, S k (k = 1,..., K) is the rest node in the strength layer, K is the total number of rest nodes, and ξ is the probability of the random walker reaching the killer node and the rest node in the strength layer.

[0096] After normalizing the adjacency matrix, the intra-layer transition probability matrix is obtained as follows:

[0097]

[0098] wherein, is the degree matrix of the attribute layer or the intensity layer.

[0099] Inter-layer transition matrix: refers to the probability of transition between the intensity layer and the attribute layer. In order to fuse the structural information between different images, the embodiment first defines a strong connection set of the intensity layer node, and the strong connection set of the intensity layer node specifically refers to a set of edge weight values between the intensity layer node and its adjacent nodes which are greater than the average value of the edge weight in the neighborhood of the intensity layer node.

[0100] Then, the information amount of the intensity layer node is obtained according to the strong connection set of the intensity layer node, and specifically is:

[0101]

[0102] wherein, represents the absolute value of the number of subsets in the strong connection set of the i-th intensity layer node, i={1,...,N}, N represents the total number of intensity layer nodes or attribute layer nodes, nn i is the neighbor node of the i-th intensity layer node, P={p ij} N×N represents the intra-layer transition probability matrix.

[0103] The inter-layer edge weight is derived according to the information amount of the intensity layer node, and specifically is:

[0104]

[0105] wherein, represents the edge weight between the i-th intensity layer node and the j-th attribute layer node, e is a non-zero constant; the inter-layer edge weight is normalized to obtain the inter-layer transition matrix, and specifically is:

[0106]

[0107] wherein, represents the inter-layer transition probability between the i-th intensity layer node and the j-th attribute layer node, represents the inter-layer transition matrix, ξ is the probability of the random walker reaching the killing node and the staying node in the intensity layer, d αβ represents the degree matrix between the intensity layer and the attribute layer, which is obtained by summing the elements in the inter-layer edge weight matrix .

[0108] Overall transition matrix: the overall transition probability of the double-layer topological structure can be obtained by comprehensively considering the inter-layer and intra-layer transition matrices.

[0109]

[0110] where Q represents the overall transition matrix of the two-layer topology, and represent the ith and jth intensity layer node, respectively, and represent the ith and jth attribute layer node, respectively, and Δ is the killer node in the intensity layer, S k (k = 1,..., K) is the rest node in the intensity layer, K is the total number of rest nodes, ξ is the probability of random walker reaching the killer node and rest node in the intensity layer, d all = d α + d β + d αβ is the degree matrix corresponding to the two-layer topology, d α represents the degree matrix of the intensity layer, which is obtained by summing the elements in the edge weight matrix of the intensity layer, d β represents the degree matrix of the attribute layer, which is obtained by summing the elements in the edge weight matrix of the attribute layer.

[0111] S3: Two-layer random walk based on label prior

[0112] In the multi-exposure fusion task, there are many potential contrast information hidden in a sequence of exposure images, but most multi-exposure fusion algorithms do not fully utilize these information, therefore, the contrast information of the image is added to the two-layer random walk model as a label prior, as shown in Figure 3 . The contrast information graph node of the mth exposure image is defined as l m , and {m = 1,..., M}. The embodiment based on the cross-layer random walk model obtains the optimal probability mapping corresponding to the exposure image, which specifically includes:

[0113] Obtain the edge weight matrix of the exposure image based on the contrast information, and the specific formula is:

[0114]

[0115]

[0116] wherein, represents the edge weight matrix of the mth exposure image based on the contrast information, l m represents the contrast information graph node of the mth exposure image, represents the edge weight between the contrast information graph node of the mth exposure image and the intensity layer, I m represents the mth exposure image, and cm representing the spatial contrast of the m-th exposure image, |c m | represents the contrast of c m taking absolute value, Erf(·) is sigmoid function, monotonically increasing, is a Gaussian weighted function, σ1, σ2 are variances, and N1=N*N. Specifically, c m = 0.298R m + 0.587G m + 0.115B m is the spatial contrast. The greater c m is, the better the pixel contrast is.

[0117] Therefore, these contrast information is added as label prior to the cross-layer random walk, and the final transition matrix is obtained:

[0118]

[0119] wherein, represents the contrast overall transition matrix, is the degree matrix corresponding to the cross-layer random walk model with label prior, d l represents the degree matrix between the exposure image based on contrast information graph nodes and intensity layer, which is obtained by summing the elements in the exposure image based on contrast information edge weight matrix.

[0120] The random walker walks on the graph , and let be the probability that the random walker reaches the stay node S k (k = {1,...,K}) in the intensity layer:

[0121]

[0122] Here is an indicator vector. In order to calculate the optimal probability mapping , equation (12) is rewritten as a general energy optimization for calculating membership probability vector . Therefore, the objective function is obtained as follows:

[0123]

[0124] wherein, is the arrival probability of node in the intensity layer to the stay node, is the arrival probability of node in the attribute layer to the stay node in the intensity layer, is an indicator vector, indicating that when the selected node is a stay node, at this time otherwise k = {1,..., K} is the index of the stopping node, t, m = {1,..., M}, μ is a penalty factor. Thus each node in the mth exposure image The final result of the walk can be obtained by minimizing the objective function (13):

[0125]

[0126] Write equation (13) in matrix form:

[0127]

[0128] Here and D αβ = [d αβ ] N×N , D l = [d l ] N×N . D ξ = diag(ξ,..., ξ), D ψ = e-D ξ . e is the N x N identity matrix, 1 is the N x 1 matrix with all elements equal to 1. T is the transpose of the matrix.

[0129] By solving equation (15), the final result can be obtained as:

[0130]

[0131] s.t. E = e-D ψ D -1 D β -D ψ P α -D ψ P αβ

[0132] Here and P αβ = μD -1 W αβ .

[0133] Finally, the optimal probability map r α for each exposure image is obtained by calculating equation (16): *

[0134]

[0135] where I m represents the mth exposure image, (r α ) m ​represents the optimal probability map corresponding to the mth exposure image,

[0136] The embodiment of the present application aims to provide a multi-exposure image fusion method based on cross-layer random walk. Firstly, a double-layer graph topology structure is established: intensity layer and attribute layer; secondly, the latent structure information is applied between layers and the inter-layer transition matrix is calculated, so as to enhance the structure preservation effect of the fused image. Finally, the contrast information is added as prior knowledge, and a multi-exposure image fusion framework is established. The framework converts the multi-exposure fusion problem into evaluating the global optimal weight probability map, and generates the final fusion image according to the global optimal weight probability map.

[0137] With reference to Figure 4 , Figure 4 The multi-exposure image fusion result under night scene using the embodiment two is shown in the figure. From Figure 4 It can be seen that the method of the embodiment can reveal the details of the brightest / darkest area in the scene under different exposure conditions, and avoid color distortion in the fused image, so as to obtain a high-quality fused image.

[0138] With reference to Figure 5 , the multi-exposure image fusion system based on cross-layer random walk proposed in the embodiment of the present application comprises a memory 10, a processor 20, and a computer program stored in the memory 10 and executable on the processor 20, wherein the processor 20 implements the steps of the multi-exposure image fusion method based on cross-layer random walk proposed in the embodiment when executing the computer program.

[0139] The specific working process and working principle of the multi-exposure image fusion system based on cross-layer random walk of the embodiment can refer to the working process and working principle of the multi-exposure image fusion method based on cross-layer random walk of the embodiment.

[0140] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-exposure image fusion method based on cross-layer random walk, characterized in that, The method comprises: obtaining a multi-exposure image, the multi-exposure image comprising a plurality of exposure images; establishing a cross-layer random walk model corresponding to the exposure image, the cross-layer random walk model being a double-layer topological structure, wherein establishing the cross-layer random walk model corresponding to the exposure image comprises: constructing a double-layer topological structure comprising an intensity layer and an attribute layer, the double-layer topological structure being specifically represented as: wherein, denotes different layers in a two-layer topology, and denote a set of strength layer nodes and a set of property layer nodes, respectively, α andε β denote a set of edges between strength layer nodes and property layer nodes, respectively, denotes a set of properties of strength layer nodes and edges, denotes a set of properties of property layer nodes and edges; constructing an intra-layer transition matrix and an inter-layer transition matrix for walking in the cross-layer random walk model, wherein constructing the inter-layer transition matrix for walking in the cross-layer random walk model comprises: obtaining a strong connection set of the intensity layer node, the strong connection set of the intensity layer node specifically referring to a set of edge weight values between the intensity layer node and its adjacent nodes being greater than the average value of edge weights in the neighborhood of the intensity layer node; obtaining the information amount of the intensity layer node according to the strong connection set of the intensity layer node, specifically as: wherein, represents the absolute value of the number of subsets in the strong connection set of the i-th intensity layer node, i = {1,..., N}, N represents the total number of intensity layer nodes or attribute layer nodes, nn i is the neighbor node of the i-th intensity layer node, P = {p ij} N×N represents the intra-layer transition probability matrix; deriving the inter-layer edge weight according to the information amount of the intensity layer node, specifically as: wherein, represents the edge weight between the ith intensity layer node and the jth attribute layer node, e being a non-zero constant; normalizing the inter-layer edge weight to obtain the inter-layer transition matrix, specifically as: wherein, denotes the inter-layer transition probability between the i-th intensity layer node and the j-th attribute layer node, denotes the inter-layer transition matrix, and ξ is the probability of a random walker reaching a kill node and a stay node in the intensity layer, d αβ denotes the degree matrix between the intensity layer and the attribute layer, and is obtained by summing the elements in the inter-layer edge weight matrix . obtaining an overall transition matrix based on the intra-layer transition matrix and the inter-layer transition matrix; obtaining an optimal probability mapping corresponding to the exposure image based on the cross-layer random walk model; weighting and fusing to obtain a fused image according to the exposure image and the optimal probability mapping corresponding to the exposure image.

2. The cross-layer random walk based multi-exposure image fusion method according to claim 1, wherein, The specific formula for obtaining the overall transition matrix based on the intra-layer transition matrix and the inter-layer transition matrix is: where Q denotes the overall transition matrix of the two-layer topology, and denote the ith and jth strength layer node, respectively, and denote the ith and jth attribute layer node, respectively, Δ is a killer node in the strength layer, S k (k = 1,..., K) is a stay node in the strength layer, K is the total number of stay nodes, ξ is the probability of a random walker reaching a killer node and a stay node in the strength layer, d all = d α + d β + d αβ is the degree matrix of the two-layer topology, d α denotes the degree matrix of the strength layer, which is obtained by summing the elements in the edge weight matrix of the strength layer, d β denotes the degree matrix of the attribute layer, which is obtained by summing the elements in the edge weight matrix of the attribute layer.

3. The cross-layer random walk based multi-exposure image fusion method according to claim 2, wherein, obtaining an optimal probability mapping corresponding to the exposure image based on the cross-layer random walk model comprises: obtaining an edge weight matrix of the exposure image based on contrast information, the specific formula being: wherein, represents an edge weight matrix of the mth exposure image in the multi-exposure image based on the contrast information, l m represents a contrast information graph node of the mth exposure image, represents an edge weight between the contrast information graph node of the mth exposure image and the intensity layer, I m represents the mth exposure image, c m represents a spatial contrast of the mth exposure image, |c m | represents a contrast of c m takes an absolute value, Erf(·) is a sigmoid function, monotonically increasing, is a Gaussian weighting function, σ1, σ2 are variances, and N1=N*N; adding the edge weight matrix of the exposure image based on contrast information as a label prior to the cross-layer random walk model to obtain a contrast overall transition matrix, specifically as: where Q represents a contrast overall transfer matrix, d all = d α + d β + d αβ + d l is a degree matrix corresponding to the cross-layer random walk model with label prior, d l represents a degree matrix between the graph nodes based on contrast information and the intensity layer of the exposure image, which is obtained by summing the elements in the edge weight matrix based on contrast information of the exposure image; obtaining a target function for calculating the optimal probability mapping based on the contrast overall transition matrix; optimizing and solving the target function to obtain the optimal probability mapping corresponding to the exposure image.

4. The cross-layer random walker based multi-exposure image fusion method of claim 3, wherein, The specific formula for the target function is: where r i α is the node is the arrival probability of the node i β is the node is the arrival probability of the node is the indicator vector, which indicates that the selected node is the staying node, at which time Otherwise k = {1,..., K} is the index of the staying node, μ is the penalty factor, t, m = {1,..., M}.

5. The cross-layer random walker based multi-exposure image fusion method of claim 4, wherein, The specific formula for weighting and fusing to obtain a fused image according to the exposure image and the optimal probability mapping corresponding to the exposure image is: where I * represents the final fused image, I m denotes the mthexposed image, (r α ) m represents the optimal probability map corresponding to the mthexposed image, r α = {r i α} N×N .

6. A multi-exposure image fusion system based on cross-layer random walk, the system comprising: a memory (10), a processor (20), and a computer program stored on the memory (10) and executable on the processor (20), wherein the processor (20) implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

Images