Image fusion method and device based on multiple decision diagram, electronic equipment and medium

Abstract

The application discloses a kind of based on multivariate decision graph image fusion method, device, electronic equipment and medium, for solving the problem that the multi-focus image fusion method based on the existing decision graph framework cannot be fused multiple images in single time.The method comprises: obtaining the image to be processed, extracting the feature to be fused from the image to be processed, and determining the multiple images to be fused corresponding to the image to be processed based on the feature to be fused;The multiple images to be fused are preliminarily processed, and the preliminary multivariate decision graph is obtained;The preliminary multivariate decision graph is classified based on the small area decision of the symbol distance field, and the secondary allocation multivariate decision graph is output;The boundary smoothing processing is carried out on the secondary allocation multivariate decision graph, the target multivariate decision graph is obtained, and the multiple images to be fused are weighted fused according to the target multivariate decision graph according to weight, and the target fusion image is obtained.

Description

Technical Field

[0001] This invention relates to the field of image fusion technology, and in particular to an image fusion method, apparatus, electronic device and medium based on multivariate decision graphs. Background Technology

[0002] With the rapid development of deep learning, more and more large datasets and models have emerged. However, in the field of computer vision, many datasets are obtained by taking pictures with optical cameras in reality. Due to the limited depth of field of current optical cameras, objects within the depth of field can be clearly displayed in the image, while objects outside the depth of field become blurry. This phenomenon leads to the current problem of difficulty in obtaining fully focused images, which means that the dataset used to train computer vision models may have some missing information. On the other hand, if we assume that in order to ensure that information is not lost due to the blurring of the depth of field, we focus on objects at different distances in each scene of the dataset separately, it will result in a very large amount of memory required to store these images.

[0003] Multi-focus image fusion is an image fusion technique that effectively extends the depth of field of view of an optical lens. This method primarily fuses information from multiple images into a single image through image processing, resulting in a theoretically clear image with full depth of field. This solves the problem of blurred areas in images captured by cameras due to their limited depth of field. The fused image contains clear targets from different focused images, resulting in a fully sharp image. Therefore, it provides a more accurate description of the scene and is more conducive to human visual recognition and computer analysis. Furthermore, since the size of the fused image is the same as the size of a single image to be fused, this method can save space in storing the dataset.

[0004] Currently, based on the framework classification of multi-focus image fusion, multi-focus image fusion technology can be mainly divided into three categories: two-stream framework based on the base layer and detail layer, decision graph-based framework, and end-to-end framework based on deep learning.

[0005] Two-stream frameworks based on base and detail layers can fuse an unlimited number of images, but suffer from fusion performance bias and low post-fusion metrics. Decision graph frameworks offer better fusion results and higher metrics, but can only accept two images as input, preventing simultaneous fusion of multiple images. Fusing multiple images requires iterative fusion using only two images. End-to-end deep learning frameworks have input limitations, such as restrictions on the number and resolution of input images. Therefore, current multi-focus image fusion techniques all suffer from the inability to fuse multiple images in a single operation. Summary of the Invention

[0006] This invention provides an image fusion method, apparatus, electronic device, and medium based on a multivariate decision graph, which solves or partially solves the technical problem that existing multi-focus image fusion methods based on decision graph frameworks cannot fuse multiple images in a single operation.

[0007] This invention provides an image fusion method based on multivariate decision graphs, the method comprising:

[0008] Acquire an image to be processed, extract features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused.

[0009] Preliminary multivariate decision processing is performed on the multiple images to be fused to obtain a preliminary multivariate decision map;

[0010] The preliminary multivariate decision graph is subjected to small-region decision classification based on the symbolic distance field, and a secondary allocation multivariate decision graph is output.

[0011] The secondary allocation multivariate decision graph is subjected to boundary smoothing processing to obtain a target multivariate decision graph. The multiple images to be fused are then weighted and fused according to the target multivariate decision graph to obtain a target fused image.

[0012] The present invention also provides an image fusion device based on a multivariate decision graph, comprising:

[0013] The image to be fused determination module is used to acquire the image to be processed, extract the features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused.

[0014] The preliminary multivariate decision processing module is used to perform preliminary multivariate decision processing on the multiple images to be fused to obtain a preliminary multivariate decision map.

[0015] The small-area decision classification module is used to perform small-area decision classification based on the symbolic distance field on the preliminary multivariate decision graph and output the secondary allocation multivariate decision graph.

[0016] The target fusion image generation module is used to perform boundary smoothing processing on the secondary allocation multivariate decision map to obtain a target multivariate decision map, and to perform weighted fusion of the multiple images to be fused according to the target multivariate decision map to obtain a target fusion image.

[0017] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is configured to execute the image fusion method based on the multivariate decision graph as described above, according to the instructions in the program code.

[0020] The present invention also provides a computer-readable storage medium for storing program code for executing the image fusion method based on multivariate decision graphs as described in any of the preceding claims.

[0021] As can be seen from the above technical solution, the present invention has the following advantages: It proposes an image fusion method based on a multivariate decision graph. First, the image to be processed is input, and features related to sharpness are extracted from it. Based on these features, multiple images to be fused corresponding to the image to be processed can be determined. Then, preliminary multivariate decision processing is performed on the multiple images to be fused to obtain a preliminary multivariate decision graph. Next, small-region decision classification based on the symbolic distance field is performed on the preliminary multivariate decision graph to output a secondary allocation multivariate decision graph. Then, boundary smoothing processing is performed on the secondary allocation multivariate decision graph to obtain a target multivariate decision graph. Based on the target multivariate decision graph, the multiple images to be fused are weighted and fused, and finally, the target fused image can be output. By adopting the technical solution provided by the present invention, the technical problem that existing multi-focus image fusion methods based on decision graph frameworks cannot perform multi-image fusion in a single operation is solved. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating the steps of an image fusion method based on a multivariate decision graph, as provided in this embodiment of the invention;

[0024] Figure 2 A schematic diagram illustrating a multivariate decision graph representation provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram illustrating another form of multivariate decision graph provided in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram illustrating potential misclassifications in decision graphs within existing related technologies.

[0027] Figure 5 This is a schematic diagram illustrating the separation of large and small regions in a multivariate decision graph provided in an embodiment of the present invention.

[0028] Figure 6 This is a schematic diagram of the overall processing framework of an image fusion method based on multivariate decision graphs provided in an embodiment of the present invention;

[0029] Figure 7 This invention provides a comparison image before and after image fusion using a publicly available dataset, as an embodiment of the invention.

[0030] Figure 8 This invention provides a comparison image before and after image fusion using a real-world shooting dataset, as an embodiment of the invention.

[0031] Figure 9 This invention provides a comparison image before and after image fusion using a virtual world dataset, as an embodiment of the invention.

[0032] Figure 10 This is a structural block diagram of an image fusion device based on a multivariate decision graph, provided in an embodiment of the present invention. Detailed Implementation

[0033] This invention provides an image fusion method, apparatus, electronic device, and medium based on a multivariate decision graph, which solves or partially solves the technical problem that existing multi-focus image fusion methods based on decision graph frameworks cannot fuse multiple images in a single operation.

[0034] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0035] As an example, for multi-focus image fusion processing, currently, based on the framework classification of multi-focus image fusion, multi-focus image fusion technology can be mainly divided into three categories: two-stream framework based on the base layer and detail layer, decision graph-based framework, and end-to-end framework based on deep learning.

[0036] Two-stream frameworks based on base and detail layers can fuse an unlimited number of images, but suffer from fusion bias and low post-fusion performance metrics. Decision graph frameworks offer better fusion results and higher performance metrics, but can only accept two images as input, preventing simultaneous fusion of multiple images. Fusing multiple images requires iterative fusion using only two images. End-to-end deep learning frameworks have input limitations, such as restrictions on the number and resolution of input images. Therefore, current multi-focus image fusion techniques generally cannot fuse multiple images in a single operation. Furthermore, most neural networks output images of a fixed size, requiring interpolation to obtain an image the same size as the input image. Thus, the size of the input image can significantly impact the final fusion result.

[0037] Taking a decision graph-based framework as an example, a decision graph is a binary representation of pixel classification. In terms of expressive power, it can only express two states. Semantically, it can be represented as two states, 0 and 1, where one state represents the foreground and the other represents the background.

[0038] To better illustrate this point, the following discussion demonstrates that decision maps cannot be directly used for fusion of three or more images.

[0039] Define a mapping f: A→B. If the mapping f is injective, then set A can be represented by the set B. Injective means that for all x, y∈A, and x≠y, f(x)≠f(y). For a decision graph of size M×N, by the definition of a decision graph, the number of elements N in the 0-1 decision graph set C1 of this size is... sum1 for:

[0040] N sum1 =2 M×N

[0041] When there are two or more multi-focus images to be fused, assuming the number is k, then the set C2 of the decision graph of size M×N contains N elements. sum2 for:

[0042] N sum2 =k M×N

[0043] Since k > 2, therefore N sum2 >N sum1 Therefore, for a mapping f1: C2 → C1, there is no case where f1 is injective, that is, set C2 cannot be represented by set C1. Thus, when the 0-1 decision graph is not modified, it cannot be directly used for the fusion of 3 or more images due to its limited expressive power.

[0044] Therefore, one of the core inventive points of this invention is: addressing the problem that current multi-focus image fusion methods based on decision graph frameworks cannot fuse multiple images in a single operation, a new image fusion method based on multivariate decision graphs is proposed. First, the image to be processed is input, and features related to sharpness are extracted. Based on these features, multiple images to be fused corresponding to the image to be processed are determined. Then, preliminary multivariate decision processing is performed on the multiple images to be fused to obtain a symbolically represented preliminary multivariate decision graph. Next, through various processing methods such as contour extraction algorithms, image moments, and symbolic distance functions, small-region decision classification based on the symbolic distance field is performed on the preliminary multivariate decision graph, outputting a secondary allocation multivariate decision graph. Then, a first-order guided filtering process is used to smooth the boundaries of the secondary allocation multivariate decision graph to obtain a target multivariate decision graph. Finally, the multiple images to be fused are weighted and fused according to the target multivariate decision graph, and the target fused image is output. By adopting the method provided in the embodiments of the present invention, a multi-focus image fusion method based on a decision graph framework can perform a single-time fusion of an unlimited number of images. Compared with iterative fusion of pairs of images, since only one weighted fusion is required, the fusion speed can be greatly accelerated. In addition, the problem of information loss after fusion can be avoided, the iteration error can be reduced, and the fusion accuracy can be improved.

[0045] Reference Figure 1 The diagram illustrates a flowchart of an image fusion method based on a multivariate decision graph provided by an embodiment of the present invention, which may specifically include the following steps:

[0046] Step 101: Obtain the image to be processed, extract the features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused.

[0047] First, the image to be processed can be acquired, and features related to sharpness can be extracted using common feature extraction methods in the field of image processing technology. For example, deep learning networks can be used for learning, or convolution, filtering and other operations can be used to obtain the features to be fused. Then, the decision graph framework can be combined to obtain the value of the sub-decision graph corresponding to each image to be fused in the multivariate decision graph based on the features to be fused (i.e., the decision value corresponding to the pixel in the image to be fused after classification using a binary representation, such as 0 or 1).

[0048] Step 102: Perform preliminary multivariate decision processing on the multiple images to be fused to obtain a preliminary multivariate decision map;

[0049] For the preliminary multivariate decision processing flow, firstly, the values ​​of the decision graph need to be modified. For images, if represented solely based on the decision graph, there are only two cases: 0 and 1. The modification of the decision graph values ​​proposed in this invention specifically refers to using a symbol as a representative for each image to be fused. In the specific implementation, preliminary multivariate decision processing is performed on multiple images to be fused to obtain a preliminary multivariate decision graph. This can be achieved by symbolically representing each image to be fused using the following formula:

[0050] p i,j ∈{P1,P2,…,P n-1}

[0051] Where, p i,j P represents the pixel value at x-axis i and y-axis j in the preliminary multivariate decision graph. k (0≤k≤n-1) is the symbol for the image to be fused, and n is the number of images to be fused.

[0052] For example, Figure 2 This is a schematic diagram illustrating a multivariate decision graph according to an embodiment of the present invention. As shown in the diagram, a preliminary multivariate decision graph obtained after initial multivariate decision processing corresponds to three images to be fused, each represented by a symbol: P0, P1, and P2. The region marked P0 corresponds to one image to be fused, and similarly, the regions marked P1 or P2 each correspond to one image to be fused. It should be noted that, for simplification, only the symbol representation regions corresponding to the three images to be fused are shown in the diagram. In reality, one image to be processed may correspond to multiple image regions to be fused, especially when it contains multiple image regions that need to be focused, resulting in more symbol representation regions. It is understood that the present invention does not impose any limitations on this.

[0053] In actual processing, if the decision graph is modified directly using the above method, then during subsequent smoothing operations, since the image to be fused is represented by a certain symbol, it is impossible to directly perform smoothing operations on the image to be fused, while the decision graph can be directly smoothed. Therefore, based on the above symbolic representation method, this embodiment of the invention proposes to use more storage space to obtain an equivalent expression. That is, for each image to be fused, a sub-decision graph with a changed meaning is set up. In the sub-decision graph, the meaning of 0 and 1 is changed to indicate whether the pixel at that position belongs to the image to be fused.

[0054] For example, Figure 3This is a schematic diagram illustrating another form of multivariate decision graph provided in an embodiment of the present invention. As can be seen from the figure, for the preliminary multivariate decision graph in the aforementioned embodiment, a sub-decision graph is set for each of P0, P1, and P2. Sub-decision graph ID0 corresponds to P0, sub-decision graph ID1 corresponds to P1, and sub-decision graph ID2 corresponds to P2. In each sub-decision graph, a 1 at the pixel position indicates that the pixel at that position belongs to the image to be fused, and a 0 at the pixel position indicates that the pixel at that position does not belong to the image to be fused.

[0055] To better illustrate this point, we will now demonstrate that multivariate decision graphs can be used for the fusion of multiple images.

[0056] When the number of images to be fused is greater than or equal to 2, assuming the number is k, then the set C3 of the multivariate decision graph of size M×N has N elements. sum3 for:

[0057] N sum3 =k M×N

[0058] Based on the previous examples, we can derive N sum3 =N sum2

[0059] Among them, setting Figure 3 The upper part is represented using the first method of multivariate decision graph representation. Figure 3 The lower half is represented using the second method of multivariate decision graph representation.

[0060] Define a mapping g(x, y, z).

[0061]

[0062] Where f(x, y) is the value at pixel position (x, y) in the multivariate decision graph.

[0063] And because f(x, y) = P z Since ifg(x,y,z) = 1, the first method of representing a multivariate decision graph can be transformed into the second method by mapping g(x,y,z), and the second method can be transformed into the first method by mapping f(x,y). Therefore, if storage space is not considered, the two representations are equivalent.

[0064] Furthermore, it is worth noting that the comparison object based on the multivariate decision graph is all images to be fused. Therefore, for any two corresponding pixels in the sub-decision graphs, the result of the AND operation of their pixel values ​​is 0.

[0065] Conventional decision maps only compare two images, which cannot guarantee that the AND operation of corresponding pixel values ​​in any two 0-1 decision maps will be 0. In other words, the AND operation of some corresponding pixel values ​​will result in 1. If these regions with a result of 1 are not processed in any way, these regions may be misclassified.

[0066] For example, Figure 4 This is a schematic diagram of possible misclassification in decision maps in existing related technologies. After classification, the value of the second pixel in the upper left corner of both sub-decision maps is 1 (in conjunction with the previous example, the correct value of the element at this position in the sub-decision map corresponding to P1 should be 0). Because the subsequent fusion stage uses a weighted fusion method, if both values ​​at this position are 1, the value of the fused image at this pixel is obtained by adding the values ​​of the two images to be fused at this position. The output value after fusion will be very large, thus causing the fused image to shine at this point.

[0067] In conclusion, multivariate decision graphs, due to their sufficient expressive power, can be used for the fusion of multiple images.

[0068] Step 103: Perform small-region decision classification based on the symbolic distance field on the preliminary multivariate decision graph, and output a secondary allocation multivariate decision graph;

[0069] The aforementioned steps demonstrate that the method proposed in this invention can transform an original single decision graph into a multi-dimensional decision graph. However, this approach presents certain challenges for small-area decision-making steps.

[0070] Among them, small regions refer to some particularly small regions (corresponding to regions with an area smaller than a certain preset area threshold) in a region that are different from the decision of the region. When the region is small enough, these small regions can be regarded as noise caused by misclassification. Therefore, it is necessary to use small region decision processing methods to reclassify these regions.

[0071] Currently, the mainstream small-area decision-making technologies include the following:

[0072] (1) Median filtering;

[0073] Median filtering can be used for small-area decision-making, and by selecting an appropriately sized filter kernel, small noise in the binarized image can be removed. However, this method is not suitable for multivariate decision graphs for the following reasons:

[0074] Assuming there exists a region R1 that is a small region in all decision graphs, the output value after applying median filtering to the pixels p(i,j) of the small region in the multivariate decision graph is:

[0075] MedianBlue(p(i,j))=0,if(i,j)∈R1

[0076] The value of that pixel in the final merged image will become:

[0077] p f (i,j)=0×p1(i,j)+0×p2(i,j)+…+0×p n (i, j) = 0

[0078] This causes the merged image to appear as a black dot at that pixel.

[0079] (2) Morphological methods, such as expansion operations and corrosion operations;

[0080] The formula for handling expansion operations is:

[0081] p new (i, j) = max(p pre (i1, j1))

[0082] Where, p new (i, j) represents the pixel at position (i, j) after the dilation operation, (i1, j1) represents the pixels within a neighborhood of position (i, j), and p pre (i1, j1) represents the pixel at position (i1, j1) before the dilation operation.

[0083] The processing formula for corrosion operations is as follows:

[0084] p new (i, j) = min(p) pre (i1, j1))

[0085] Wherein, p new (i, j) represents the pixel at position (i, j) after the erosion operation, and (i1, j1) represents the pixel within a neighborhood of position (i, j). pre (i1, j1) represents the pixel at position (i1, j1) before the erosion operation.

[0086] This shows that if morphological methods are used for small-area decision-making, it will lead to changes in the edges of large areas.

[0087] Assuming pixel (i, j) is located at the boundary of large region A and large region B, the following situations may exist:

[0088] p Anew (i, j) = 0, p Bnew (i, j) = 0

[0089] or

[0090] p Anew (i, j) = 1, p Bnew(i, j) = 1

[0091] Among them, P Anew (i, j) represents the pixel at position (i, j) after the erosion operation on large region A, p Bnew (i, j) represents the pixel at position (i, j) after the erosion operation is performed on the large region B.

[0092] This will lead to the following first situation:

[0093] p f (i,j)=0×p1(i,j)+0×p2(i,j)+…+0×p n (i, j) = 0

[0094] Or the following second scenario may occur:

[0095] p f (i,j)=1×p1(i,j)+1×p2(i,j)+…+0×p n (i, j)

[0096] The first scenario results in black spots during fusion, while the second scenario results in white spots or even fusion failure, because this fusion method may cause the final pixel value to be greater than 255.

[0097] (3) Flip the small area;

[0098] The method for flipping small regions is to reverse the values ​​of small regions in the decision graph (changing the value 0 to the value 1, or the value 1 to the value 0). Specifically, the method for flipping small regions involves first finding small regions in the decision graph, and then changing the values ​​of the small regions.

[0099] For a decision graph, the value of a pixel in a small region has only two possibilities: 0 or 1. Therefore, once the small region is determined, the value of the pixel position can be changed directly as follows:

[0100]

[0101] Where x is the value of a pixel in a small region, and f(x) is the flip operation. When x is 1, it is flipped to 0, and when x is 0, it is flipped to 1.

[0102] In this case, assuming there exists a region R1 that is a small region in all decision graphs, then we can conclude that:

[0103] negation(p(i,j))=0,if(i,j)∈R1

[0104] Here, negation(*) represents the logical NOT operation, i.e., the toggle operation.

[0105] The value of that pixel in the final merged image will become:

[0106] p f (i,j)=0×p1(i,j)+0×p2(i,j)+…+0×p n (i, j) = 0

[0107] This shows that the problem caused by flipping small areas is also the reason why black spots appear at the merged area.

[0108] As can be seen from the above analysis, for the multivariate decision graph proposed in the embodiments of the present invention, the three mainstream small-region decision methods all have defects. Among them, the median filtering and inverted small-region numerical methods cannot be applied because there is no information interaction between the small-region processing of multiple 0-1 decision graphs, and the morphological method cannot be applied because the operation does not have the property of edge preservation.

[0109] In view of the above, this invention proposes a processing method based on symbolic distance field to solve the problem of multi-image and multi-focus image fusion in small-area decision-making.

[0110] The signed distance function is a level set representation that stores the minimum distance from each n-dimensional point in n-dimensional space to a certain region object. The signed distance field can be seen as a container that stores the results of the signed distance function. Usually, positive and negative signs are used to distinguish whether the n-dimensional point is inside or outside the region object. For example, if the value of point p1 in the signed distance field of region S is -1, it means that the distance from point p1 to region S is 1 unit and point p1 is inside region S. If the value of point p2 in the signed distance field of region S is 3, it means that the distance from point p2 to region S is 3 units and point p2 is outside region S.

[0111] Furthermore, the processing method based on the symbolic distance field can be mainly divided into the following two steps:

[0112] (1) Find all the small areas, and then put all the small areas into a container;

[0113] In the field of data processing, a container is a broad concept that refers to a data structure or abstract concept that can hold other elements or objects. In this embodiment of the invention, a container can be regarded as a symbolic distance field used to store the results of the symbolic distance function.

[0114] (2) Classify each pixel in each small region.

[0115] Combining with step (1), after all the small regions are found, the small regions are divided with the large regions, so that all the small regions can be processed together. In this process, since all the small regions are processed together, there will be no problem of multiple decision graphs being processed repeatedly.

[0116] As can be seen from the foregoing analysis, the problem with morphological methods is that morphological operations do not have edge-preserving properties. Therefore, the embodiments of the present invention adopt the method of first calculating the distance using the symbolic distance field and then classifying the pixels. Since no operation is performed on the region edges during the processing, this processing method preserves the edges.

[0117] Based on the foregoing, the preliminary multivariate decision graph can be viewed as being composed of multiple sub-decision graphs, each corresponding to an image to be fused. Therefore, the process of performing small-region decision classification based on the symbolic distance field on the preliminary multivariate decision graph to output a secondary allocation multivariate decision graph can specifically include the following steps:

[0118] Step S01: Use the contour extraction algorithm to extract the boundaries of multiple sub-decision maps respectively, and determine all small regions in the preliminary multivariate decision map by the moments of the image. Then, based on the determined small regions, divide the small regions and large regions of the preliminary multivariate decision map, where the area of ​​the small region is smaller than the area of ​​the large region.

[0119] In this step, the contour extraction algorithm can be used to extract the boundaries of each sub-decision map first, and then the small regions in each sub-decision map (i.e., all the small regions corresponding to the initial multivariate decision map) can be found by the moments of the image, and then the small regions and large regions can be divided.

[0120] First, for contour extraction, contour extraction algorithms based on the OpenCV library (a cross-platform computer vision and machine learning software library) can be used to extract the contours of the region.

[0121] For example, the boundary tracking method can be used for contour extraction. The working principle of the boundary tracking method is to start from a boundary point in the image and then search for the next boundary point according to a preset discrimination criterion (such as single search distance and search direction) to track the target boundary.

[0122] In the boundary tracking method, the center pixel can have 8 tracking directions, and each direction can be assigned a direction number and offset. First, the image can be binarized; then, the pixels are traversed, and the bottom left pixel of the object is selected as the starting point, where the pixel value of the starting point is 0; when the starting point is found, the direction is recorded, and the next pixel is found along the scanning direction; the above process is repeated until the starting point is found again, and the loop ends.

[0123] As an optional embodiment, all small regions corresponding to the preliminary multivariate decision map are determined by the moments of the image. Then, based on the determined small regions, the small regions and large regions of the preliminary multivariate decision map are divided, which may include the following steps:

[0124] Step S011: First, calculate the moments of the sub-decision map corresponding to each image to be fused. The moment of the image represents the area of ​​the sub-decision map corresponding to the image to be fused in the preliminary multivariate decision map. The calculation formula is as follows:

[0125]

[0126] Where, m ij Let I(x,y) be the moment of the image. When i=0 and j=0, the moment of the image represents the area of ​​the region of the sub-decision map corresponding to the image to be fused. I(x,y) is the pixel value at the position corresponding to the horizontal coordinate x and the vertical coordinate y in the image to be fused. i Represents the pixel's horizontal coordinate x raised to the power of i, y j Represents the pixel's y-coordinate raised to the power of j;

[0127] Step S012: Determine the sub-decision map regions corresponding to the images to be fused whose area is smaller than a preset area threshold as small regions;

[0128] Based on the above calculation process of image moments, since the representation of the multivariate decision graph used in this embodiment is binary, when i = 0, j = 0, the image moment is the area of ​​the region. Therefore, for each 0-1 decision graph with a changed meaning, the area of ​​the region is m. 00 Regions smaller than a certain percentage of the image (e.g., less than 0.1%) can be considered small regions.

[0129] Step S013: Set a zero matrix of the same size as the image to be processed;

[0130] Next, you can create an additional zero matrix that is the same size as the image to be processed.

[0131] Step S014: XOR the values ​​corresponding to the pixel positions belonging to small regions in the preliminary multivariate decision map with 1 to remove all small regions in the preliminary multivariate decision map and obtain large regions; set all the values ​​corresponding to the pixel positions belonging to small regions in the zero matrix to 1, and then determine all regions with pixel position values ​​of 1 in the zero matrix as small regions.

[0132] In specific operations, for pixels belonging to small regions, the value of the corresponding pixel in the decision map of the second multivariate decision map representation can be XORed with 1. If the original pixel value is 0, then XORing 0 with 1 will result in 1. If the original pixel value is 1, then XORing 1 with 1 will result in 0. This is equivalent to performing an inversion operation. The purpose of XORing here is to remove small regions from large regions, which means that the image only saves the large regions.

[0133] Next, the value at the corresponding position of the zero matrix is ​​changed to 1, indicating that the pixel at that position belongs to a small region and needs to be reclassified. The purpose of XORing here is to make the zero matrix at the corresponding position of the small region become 1, which is equivalent to storing the information of the small region in this zero matrix.

[0134] For example, Figure 5 This is a schematic diagram of the separation of large and small regions in a multivariate decision graph provided in an embodiment of the present invention. The left side of the attached figure is a schematic diagram of an image that simultaneously includes large and small regions, the upper right side of the attached figure is a schematic diagram of the separated large region image, and the lower right side of the attached figure is a schematic diagram of the separated small region image.

[0135] Step S02: Calculate the symbolic distance field of each large region using the symbolic distance function, and reclassify each pixel in each small region based on the symbolic distance field;

[0136] Next, the symbolic distance function can be calculated for the large region of each sub-decision map to obtain the corresponding symbolic distance field, and then each pixel in the small region can be reclassified according to the symbolic distance field.

[0137] As an optional embodiment, calculating the symbolic distance field for each large region using the symbolic distance function and reclassifying each pixel in each small region based on the symbolic distance field may include the following steps:

[0138] Step S021: Calculate the symbolic distance function for each large region using the following formula to further determine the symbolic distance field of the corresponding large region in the image to be fused:

[0139]

[0140] Where f(P,v) denotes the signed distance function, P represents a large region, v represents a point in space, and d(·) denotes the distance metric function. This represents the boundary of a large region P.

[0141] For the remaining large region P in the second method of multivariate decision diagram representation w, w represents the large region number in the multivariate decision map corresponding to the w-th image to be fused. The corresponding symbolic distance function value and the symbolic distance field can be obtained through the above formula. For all pixel positions of the image, the value of the symbolic distance function needs to be calculated once, and then the calculation results are saved in a table.

[0142] Step S022: For each small region, compare the symbolic distance fields of each pixel belonging to the small region in different images to be fused, and reassign the pixels to the large region closest to the small region, so as to re-determine the pixel to the corresponding image to be fused with the smallest corresponding position value in the symbolic distance field.

[0143] Next, each pixel in the small region needs to be reclassified based on the symbolic distance field.

[0144] In practical applications, whether it is median filtering, morphological operations or flipping small regions, the final result is to reassign the pixels of the small region to the largest region closest to it. Therefore, by comparing the symbolic distance fields of each pixel belonging to the small region in different images to be fused, the pixel can be reassigned to the corresponding image to be fused with the smallest value at the corresponding position in the symbolic distance field.

[0145] By making decisions for each pixel in a small region based on the symbolic distance field of a large region, the problem of small region decision-making in multivariate decision graphs is effectively solved, so that the multi-focus fusion framework method based on multivariate decision graphs proposed in this invention has no non-hardware limitation on the number of images to be fused.

[0146] Step S03: Based on the small region reclassification results, output the secondary allocation multivariate decision map corresponding to the preliminary multivariate decision map.

[0147] Then, based on the small region classification results, a secondary allocation multivariate decision graph can be output.

[0148] Step 104: Perform boundary smoothing processing on the secondary allocation multivariate decision graph to obtain the target multivariate decision graph, and perform weighted fusion of the multiple images to be fused according to the target multivariate decision graph to obtain the target fused image.

[0149] The image to be fused can include multi-focus region images and non-focus region images. Multi-focus region images represent image areas requiring focused fusion, while non-focus region images can be understood as image areas not requiring focused fusion. Conversely, images that need to be occluded represent image areas that require focused fusion. For boundary processing, this embodiment of the invention uses the occluded non-focus region image as a guide image to perform a guided filter on the final decision map. This processing method has two main advantages: first, it preserves edges; second, it performs a smoothing-like process on the decision map boundaries.

[0150] In the specific implementation, the boundary smoothing process of the secondary allocation multivariate decision graph to obtain the target multivariate decision graph can be performed as follows: using a non-focused region image as a guide image, the boundary smoothing process of the secondary allocation multivariate decision graph is performed through a first-stage guided filtering process to obtain the target multivariate decision graph.

[0151] Furthermore, the multiple images to be fused are weighted and fused according to the target multivariate decision map to obtain the target fused image. Specifically, the target fused image can be obtained by weighting and fusing the multiple images to be fused according to the target multivariate decision map using the following formula:

[0152]

[0153] Where fusion(x,y) represents the value of the target fused image at pixel (x,y), D i (x,y) represents the value of the sub-decision map corresponding to the i-th image to be fused at pixel (x,y), P i (x,y) represents the value of the i-th image to be fused at pixel (x,y).

[0154] Since current image fusion metrics are all based on two images to be fused, they cannot be directly used to evaluate the merits of the image fusion method proposed in this invention. Therefore, in this embodiment of the invention, two metrics that can be extended to judge the quality of fusion of multiple images are used to evaluate the fusion effect of the target fused image.

[0155] Image fusion metrics are used to judge the quality of image fusion. The inputs are the images to be fused and the fused image, and the output is a numerical value. The magnitude of the value indicates the quality of the fusion. In this embodiment of the invention, both metrics are expressed as follows: the larger the value, the better the effect. The specific metrics used are as follows:

[0156] (1)Q MI

[0157] Mutual Information (MI) measures the similarity between two images, i.e., how much information the fused image has acquired from the original image. The greater the mutual information, the more information the fused image retains from the source image, and the better the quality.

[0158] Indicator Q MI The formula for expressing can be obtained through the following derivation:

[0159] MI(U,V)=H(U)+H(V)-H(U,V)

[0160]

[0161]

[0162]

[0163]

[0164] Where MI(*,*) represents the mutual information value between two images, H(*) represents the information entropy of the image, also known as marginal entropy, H(*,*) represents the joint information entropy, u corresponds to each value of U, p(u) represents the probability of U taking u alone, v corresponds to each value of V, p(v) represents the probability of V taking v alone, p(u,v) represents the probability of V taking v and U taking u simultaneously, and Q MI This indicates the information retained in the original images A and B in the fused image F.

[0165] Next, Q MI Extending the fusion metric to n images, we can obtain Q. MI The expression is as follows:

[0166]

[0167] (2)Q NCIE

[0168] NCIE (Nonlinear Correlation Information Entropy) defines the correlation between the fusion result and the source image.

[0169] Indicator Q NCIE The expression is as follows:

[0170]

[0171] Where, λ i This represents the eigenvalues ​​of the nonlinear correlation matrix.

[0172] When the indicator Q NCIEWhen merging multiple (≥3) images, the following can be obtained:

[0173]

[0174] When the number of images to be fused is n, the above formula becomes a calculation of the eigenvalues ​​of the order matrix.

[0175] In this embodiment of the invention, a novel image fusion method based on multivariate decision graphs is proposed. By adopting the method provided in this embodiment of the invention, a multi-focus image fusion method based on the decision graph framework can perform a single-time fusion of an unlimited number of images. Compared with iterative fusion of pairs of images, it only requires one weighted fusion, which greatly speeds up the fusion speed. In addition, it avoids the problem of information loss after fusion, reduces iteration error, and improves fusion accuracy.

[0176] For better illustration, refer to Figure 6 This diagram illustrates the overall processing framework of an image fusion method based on a multivariate decision graph provided by an embodiment of the present invention. It should be noted that this example only briefly describes the general processing flow of the image fusion method based on a multivariate decision graph. For detailed descriptions of each step, please refer to the relevant content in the foregoing embodiments, which will not be repeated here.

[0177] First, the image to be processed is input, and it is preprocessed by anomaly removal, interpolation, normalization and other data preprocessing. Then, feature extraction is performed to extract the features related to sharpness to be fused, and multiple images to be fused corresponding to the image to be processed are determined based on the features to be fused.

[0178] Then, preliminary multivariate decision processing is performed on multiple images to be fused to obtain a preliminary multivariate decision map with symbolic representation;

[0179] Next, through various processing methods such as contour extraction algorithm, image moments, and symbolic distance function, the preliminary multivariate decision map is classified into small regions based on the symbolic distance field, and a secondary allocation multivariate decision map is output.

[0180] Then, the secondary allocation multivariate decision map is smoothed by a first guided filtering process to obtain the target multivariate decision map. Based on the target multivariate decision map, multiple images to be fused are weighted and fused according to their weights, and finally the target fused image can be output.

[0181] For ease of understanding, the embodiments of the present invention are described below through specific examples.

[0182] Since there are very few publicly available datasets for multi-focus image fusion of multiple images, in order to verify the effectiveness of the method, this embodiment of the invention uses multiple datasets to evaluate the effect of the proposed fusion method. Specifically, the first dataset is a publicly available dataset on the Internet, the second dataset is a dataset of images taken by the author, and the third dataset is a dataset created by the author in a virtual world using a certain game engine. All datasets used in this embodiment of the invention are from legal sources.

[0183] The experimental results using publicly available datasets are compared in Table 1 below:

[0184]

[0185] Table 1: Experimental results of image fusion using public datasets

[0186] Among them, LP (Laplacian Pyramid), SR (Sparse Representations), RP (Rate-of-low-pass Pyramid), DTCWT (Dual-Tree Complex Wavelet Transform), CVT (Curvelet), NSCT (Nonsubsampled Contourlet), SwinFusion (a general image fusion framework), and GFDF (guided filter-based focus region detection for multi-focus image fusion) are all existing image fusion techniques. SwinFusion and GFDF methods cannot directly fuse more than three images, so an iterative fusion method is used for fusion.

[0187] For example, refer to Figure 7 The image shown is a comparison image before and after image fusion using a public dataset, provided by an embodiment of the present invention.

[0188] Given the limited number of images in the publicly available dataset, this embodiment of the invention also employs manual photography to obtain some images as part of the dataset, and uses this dataset to conduct image fusion tests. The specific test results are compared in Table 2 below:

[0189]

[0190] Table 2: Experimental results of image fusion using real-world shooting datasets

[0191] For example, refer to Figure 8 The image shown is a comparison before and after image fusion using a real-world shooting dataset, provided by an embodiment of the present invention.

[0192] Given that manually captured images require extensive data preprocessing to become datasets, and that acquiring large datasets necessitates significant manpower and resources, this embodiment of the invention also uses a large number of images from a scene within a game engine as a dataset. Image fusion tests were then conducted using this dataset, and the specific test results are shown in Table 3 below.

[0193]

[0194] Table 3: Experimental results of data fusion using a game engine dataset.

[0195] For example, refer to Figure 9 The image shown is a comparison image before and after image fusion using a virtual world dataset, provided by an embodiment of the present invention.

[0196] As can be seen from Tables 1 to 3, the image fusion method provided in this embodiment of the invention achieves better performance in terms of index Q compared to other image fusion methods. MI and indicator Q NCIE The maximum value indicates the best effect.

[0197] Furthermore, in addition to the specific examples described above, the embodiments of the present invention also conducted ablation experiments to demonstrate the effectiveness of the image fusion method based on multivariate decision graphs.

[0198] Using publicly available datasets and real-world shooting datasets as test datasets, the accuracy comparison results are shown in Table 4 below:

[0199] Q_MI Q_NCIE Do not use multi-factor decision diagram 1.345 0.782 Using multivariate decision graphs 1.361 0.783

[0200] Table 4: Results of Accuracy Comparison Experiment

[0201] Next, under the same conditions (using R7 5800H equipment, image size of 1024x768 pixels, image format of PNG, and program running environment of Python 3.8), the time consumption of fusing 7 multi-focus images with and without multivariate decision graphs was tested. The specific comparison results are shown in Table 5 below:

[0202] time consuming Do not use multi-factor decision diagram 1.398s Using multivariate decision graphs 0.753s

[0203] Table 5: Results of the speed comparison experiment

[0204] As can be seen from Tables 4 and 5, the image fusion method provided by the embodiments of the present invention not only verifies the feasibility of image fusion based on multivariate decision graphs, but also further proves that the image fusion method based on multivariate decision graphs has higher fusion accuracy, better fusion effect, and can greatly reduce time consumption and accelerate fusion speed.

[0205] Reference Figure 10 The diagram illustrates a structural block diagram of an image fusion device based on a multivariate decision graph according to an embodiment of the present invention, which may specifically include:

[0206] The image to be fused determination module 1001 is used to acquire an image to be processed, extract features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused.

[0207] The preliminary multi-dimensional decision processing module 1002 is used to perform preliminary multi-dimensional decision processing on the multiple images to be fused to obtain a preliminary multi-dimensional decision map.

[0208] Small region decision classification module 1003 is used to perform small region decision classification based on symbolic distance field on the preliminary multivariate decision graph and output a secondary allocation multivariate decision graph.

[0209] The target fusion image generation module 1004 is used to perform boundary smoothing processing on the secondary allocation multivariate decision map to obtain a target multivariate decision map, and to perform weighted fusion of the multiple images to be fused according to the target multivariate decision map to obtain a target fusion image.

[0210] In one optional embodiment, the preliminary multivariate decision processing module 1002 is specifically used for:

[0211] Each of the images to be fused is symbolically represented using the following formula to obtain a preliminary multivariate decision map:

[0212] p i,j ∈{P1,P2,…,P n-1}

[0213] Where, p i,j P represents the pixel value at x-axis i and y-axis j in the preliminary multivariate decision graph. k (0≤k≤n-1) is the symbol for the image to be fused, and n is the number of images to be fused.

[0214] In one optional embodiment, the preliminary multivariate decision map consists of multiple sub-decision maps, each sub-decision map corresponding to an image to be fused, and the small region decision classification module 1003 includes:

[0215] The preliminary multivariate decision graph region division module is used to extract the boundaries of the multiple sub-decision graphs using a contour extraction algorithm, and to determine all small regions in the preliminary multivariate decision graph by the moments of the image. Then, based on the determined small regions, the preliminary multivariate decision graph is divided into small regions and large regions, wherein the area of ​​the small region is smaller than the area of ​​the large region.

[0216] The symbolic distance field calculation module is used to calculate the symbolic distance field of each of the large regions using the symbolic distance function, and to reclassify each pixel in each small region based on the symbolic distance field.

[0217] The secondary allocation multivariate decision graph output module is used to output the secondary allocation multivariate decision graph corresponding to the preliminary multivariate decision graph based on the small region reclassification results.

[0218] In one optional embodiment, the preliminary multivariate decision graph region partitioning module includes:

[0219] The image moment calculation module is used to calculate the moments of the sub-decision map corresponding to each of the images to be fused. The moment of the image represents the area of ​​the sub-decision map corresponding to the image to be fused in the preliminary multivariate decision map. The calculation formula is as follows:

[0220]

[0221] Where, m ij Let I(x,y) be the moment of the image. When i=0 and j=0, the moment of the image represents the area of ​​the region of the sub-decision map corresponding to the image to be fused. I(x,y) is the pixel value at the position corresponding to the horizontal coordinate x and the vertical coordinate y in the image to be fused. i Represents the pixel's horizontal coordinate x raised to the power of i, y j Represents the pixel's y-coordinate raised to the power of j;

[0222] The small region determination module is used to determine the sub-decision map regions corresponding to the images to be fused that have a region area smaller than a preset area threshold as small regions;

[0223] A zero matrix setting module is used to set a zero matrix of the same size as the image to be processed.

[0224] The large region determination module is used to XOR the values ​​corresponding to the pixel positions belonging to the small regions in the preliminary multivariate decision map with 1 to remove all the small regions in the preliminary multivariate decision map and obtain the large region.

[0225] The small region determination module is used to set all the values ​​corresponding to the pixel positions belonging to the small region in the zero matrix to 1, and then determine all regions with pixel position values ​​of 1 in the zero matrix as small regions.

[0226] In one optional embodiment, the symbolic distance field calculation module includes:

[0227] The symbolic distance function calculation module is used to calculate the symbolic distance function of each of the large regions using the following formula, so as to further determine the symbolic distance field of the image to be fused corresponding to the large region:

[0228]

[0229] Where f(P,v) denotes the signed distance function, P represents a large region, v represents a point in space, and d(·) denotes the distance metric function. This represents the boundary of a large region P;

[0230] The small region pixel re-decision module is used to compare each pixel belonging to the small region with the symbolic distance field corresponding to different images to be fused for each small region, and reassign the pixel to the large region closest to the small region, so as to re-determine the pixel to the corresponding image to be fused with the smallest corresponding position value in the symbolic distance field.

[0231] In one optional embodiment, the image to be fused includes a multi-focus region image and a non-focus region image, and the target fusion image generation module 1004 includes:

[0232] The boundary smoothing module is used to use the non-focused region image as a guide image and perform boundary smoothing on the secondary allocation multivariate decision map through a first-stage guided filtering process to obtain the target multivariate decision map.

[0233] In one optional embodiment, the target fusion image generation module 1004 includes:

[0234] The weighted fusion module is used to perform weighted fusion of the multiple images to be fused according to the target multivariate decision map using the following formula to obtain the target fused image:

[0235]

[0236] Where fusion(x,y) represents the value of the target fused image at pixel (x,y), D i (x,y) represents the value of the sub-decision map corresponding to the i-th image to be fused at pixel (x,y), P i (x,y) represents the value of the i-th image to be fused at pixel (x,y).

[0237] As the device embodiment is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment above.

[0238] This invention also provides an electronic device, which includes a processor and a memory:

[0239] The memory is used to store program code and transfer the program code to the processor;

[0240] The processor is used to execute the image fusion method based on multivariate decision graphs according to the instructions in the program code of any embodiment of the present invention.

[0241] This invention also provides a computer-readable storage medium for storing program code for executing the image fusion method based on multivariate decision graphs according to any embodiment of this invention.

[0242] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0244] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0245] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0246] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0247] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image fusion method based on multivariate decision graphs, characterized in that, include: Acquire an image to be processed, extract features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused. Preliminary multivariate decision processing is performed on the multiple images to be fused to obtain a preliminary multivariate decision map; The preliminary multivariate decision graph is subjected to small-region decision classification based on the symbolic distance field, and a secondary allocation multivariate decision graph is output. The secondary allocation multivariate decision graph is subjected to boundary smoothing processing to obtain a target multivariate decision graph. The multiple images to be fused are then weighted and fused according to the target multivariate decision graph to obtain a target fused image.

2. The image fusion method based on multivariate decision graphs according to claim 1, characterized in that, The preliminary multivariate decision processing of the multiple images to be fused to obtain a preliminary multivariate decision map includes: Each of the images to be fused is symbolically represented using the following formula to obtain a preliminary multivariate decision map: ； in, The preliminary multivariate decision diagram is shown on the horizontal axis as... The vertical axis is The value of the pixel at that time. The symbol representing the images to be merged. , where n is the number of images to be fused.

3. The image fusion method based on multivariate decision graphs according to claim 2, characterized in that, The preliminary multivariate decision map consists of multiple sub-decision maps, each corresponding to an image to be fused. The step of performing small-region decision classification based on the symbolic distance field on the preliminary multivariate decision map to output a secondary allocation multivariate decision map includes: The boundaries of the multiple sub-decision maps are extracted using a contour extraction algorithm, and all small regions in the preliminary multivariate decision map are determined by the moments of the image. Then, based on the determined small regions, the small regions and large regions of the preliminary multivariate decision map are divided, wherein the area of ​​the small region is smaller than the area of ​​the large region. The symbolic distance field of each of the large regions is calculated using the symbolic distance function, and each pixel in each small region is reclassified based on the symbolic distance field. Based on the small-region reclassification results, the secondary allocation multivariate decision map corresponding to the preliminary multivariate decision map is output.

4. The image fusion method based on multivariate decision graphs according to claim 3, characterized in that, The process involves determining all small regions in the preliminary multivariate decision graph using the moments of the image, and then, based on these determined small regions, dividing the preliminary multivariate decision graph into small and large regions, including: Calculate the moments of the sub-decision map corresponding to each of the images to be fused. The moment of the image represents the area of ​​the region of the sub-decision map corresponding to the image to be fused in the preliminary multivariate decision map. The calculation formula is as follows: ； in, Let be the moments of the image. When i=0 and j=0, the moments of the image represent the area of ​​the region of the sub-decision map corresponding to the image to be fused. The image to be fused is located on the horizontal axis. The vertical axis is The value of the pixel at the corresponding position. Represents the x-coordinate of pixels power of i Represents the pixel ordinate power of j; Sub-decision map regions corresponding to images to be fused with a region area smaller than a preset area threshold are defined as small regions; Set a zero matrix that is the same size as the image to be processed; The values ​​corresponding to the pixel positions belonging to small regions in the preliminary multivariate decision graph are XORed with 1 to remove all small regions in the preliminary multivariate decision graph and obtain large regions. Set all pixel positions in the zero matrix that belong to the small region to 1, and then define all regions in the zero matrix with pixel position values ​​of 1 as small regions.

5. The image fusion method based on multivariate decision graphs according to claim 3, characterized in that, The step of calculating the symbolic distance field for each of the large regions using the symbolic distance function, and reclassifying each pixel in each small region based on the symbolic distance field, includes: The symbolic distance function for each of the large regions is calculated using the following formula to further determine the symbolic distance field of the corresponding large region in the image to be fused: ； in, Let P represent the signed distance function, P represent a large region, and v represent a point in space. A function that measures distance. This represents the boundary of a large region P; For each small region, the symbolic distance fields corresponding to each pixel belonging to the small region in different images to be fused are compared, and the pixels are reassigned to the large region closest to the small region, so that the pixels are re-determined to the corresponding image to be fused with the smallest corresponding position value in the symbolic distance field.

6. The image fusion method based on multivariate decision graphs according to any one of claims 3 to 5, characterized in that, The image to be fused includes multi-focus region images and non-focus region images. The step of performing boundary smoothing processing on the secondary allocation multivariate decision map to obtain the target multivariate decision map includes: Using the non-focused region image as a guide image, the secondary allocation multivariate decision map is smoothed through a first-stage guide filtering process to obtain the target multivariate decision map.

7. The image fusion method based on multivariate decision graphs according to any one of claims 3 to 5, characterized in that, The step of weighted fusing of the multiple images to be fused according to the target multivariate decision graph to obtain the target fused image includes: Based on the target multivariate decision map, the multiple images to be fused are weighted and fused according to the following formula to obtain the target fused image: ； in, Indicates the target fused image at the pixel level The value at that location, This represents the sub-decision map corresponding to the i-th image to be fused at pixel point. The value at that location, This indicates that the i-th image to be fused is located at pixel point. The value at that location.

8. An image fusion device based on multivariate decision graphs, characterized in that, include: The image to be fused determination module is used to acquire the image to be processed, extract the features to be fused from the image to be processed, and determine multiple images to be fused corresponding to the image to be processed based on the features to be fused. The preliminary multivariate decision processing module is used to perform preliminary multivariate decision processing on the multiple images to be fused to obtain a preliminary multivariate decision map. The small-area decision classification module is used to perform small-area decision classification based on the symbolic distance field on the preliminary multivariate decision graph and output the secondary allocation multivariate decision graph. The target fusion image generation module is used to perform boundary smoothing processing on the secondary allocation multivariate decision map to obtain a target multivariate decision map, and to perform weighted fusion of the multiple images to be fused according to the target multivariate decision map to obtain a target fusion image.

9. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the image fusion method based on any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the image fusion method based on multivariate decision graphs as described in any one of claims 1-7.

Images