Image fusion method and device, equipment and medium

Abstract

The embodiment of the invention provides an image fusion method and device, equipment and a medium, and the method comprises the steps: obtaining an infrared thermal imaging image and a visible light image at the same moment, and determining the corresponding gray information of the two images; determining an edge pixel point elimination threshold value corresponding to the two images according to the gray scale information; respectively carrying out edge detection on the two images to determine corresponding edge pixel points; screening the edge pixel points corresponding to the two images according to the edge pixel point elimination threshold values corresponding to the two images to obtain screened edge pixel points corresponding to the two images; and carrying out image fusion after registering the two images according to the screened edge pixel points. According to the embodiment of the invention, the edge pixel elimination threshold values of the infrared thermal imaging image and the visible light image are respectively determined according to the gray information of the infrared thermal imaging image and the visible light image, so that the edge pixel points are screened, and image registration and fusion are carried out based on the screened edge pixel points. The imaging effect and accuracy of image fusion are improved, and the quality of the final image is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and in particular to an image fusion method and an image fusion device. Background Art

[0002] Infrared and visible light fusion technology is to fuse infrared images and visible light images, overcoming the shortcomings of a single sensor. Infrared images can sense the thermal radiation of objects, while visible light images can sense the shape, color and other information of objects. Fusion of the two can make up for their respective shortcomings and improve image recognition and detection capabilities. However, in embedded platforms, in order to reduce product costs, high-performance computing platforms will not be used. Both infrared and visible light use low-cost cameras, which will result in poor thermal imaging effects. The outlines of people / objects in infrared thermal images are blurred, and sometimes it is even impossible to distinguish between people and other heat sources. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide an image fusion method, device, electronic device and storage medium that overcome the above problems or at least partially solve the above problems.

[0004] According to a first aspect of an embodiment of the present invention, there is provided an image fusion method, the method comprising:

[0005] Acquire an infrared thermal imaging image and a visible light image at the same time, and determine grayscale information of the infrared thermal imaging image and the visible light image respectively;

[0006] Determine a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and determine a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image;

[0007] Performing edge detection on the infrared thermal imaging image and the visible light image respectively, and determining a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image respectively according to the edge detection results;

[0008] According to the first edge pixel point rejection threshold, the first edge pixel points of the infrared thermal imaging image are screened to obtain third edge pixel points of the infrared thermal imaging image; and according to the second edge pixel point rejection threshold, the second edge pixel points of the visible light image are screened to obtain fourth edge pixel points of the visible light image;

[0009] According to the third edge pixel point of the infrared thermal imaging image and the fourth edge pixel point of the visible light image, the infrared thermal imaging image and the visible light image are registered to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image respectively;

[0010] Fusion is performed based on the registered image of the infrared thermal imaging image and the registered image of the visible light image.

[0011] Optionally, determining the grayscale information of the infrared thermal imaging image and the visible light image includes:

[0012] Determine the grayscale information corresponding to the infrared thermal imaging image and the visible light image under different edge pixel point rejection thresholds; the grayscale information includes the grayscale average value of the foreground layer, the grayscale average value of the background layer and the grayscale average value of the entire image.

[0013] Optionally, determining a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and determining a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image, includes:

[0014] Determine a first inter-class variance between a foreground layer and a background layer of the infrared thermal imaging image according to grayscale information corresponding to different edge pixel point rejection thresholds of the infrared thermal imaging image;

[0015] Solving the first inter-class variance, and taking the edge pixel point elimination threshold when the first inter-class variance is maximized as the first edge pixel point elimination threshold;

[0016] determining a second inter-class variance between a foreground layer and a background layer of the visible light image according to grayscale information corresponding to the visible light image under different edge pixel rejection thresholds;

[0017] The second inter-class variance is solved, and the edge pixel point elimination threshold when the second inter-class variance is maximized is used as the second edge pixel point elimination threshold.

[0018] Optionally, determining the first inter-class variance between the foreground layer and the background layer of the infrared thermal imaging image according to the grayscale information corresponding to the infrared thermal imaging image under different edge pixel rejection thresholds includes:

[0019] According to the grayscale average value of the foreground layer of the infrared thermal imaging image, the grayscale average value of the background layer and the grayscale average value of the entire image, the first between-class variance between the foreground layer and the background layer of the infrared thermal imaging image is determined according to the following formula;

[0020] σ 2 =Pf ×(M f -M) 2 +P b ×(M b -M) 2

[0021] Where σ is the between-class variance; P f is the probability that the total number of foreground pixels accounts for the total number of pixels; P b is the probability that the total number of background pixels accounts for the total number of pixels; M f is the average grayscale value of the foreground layer; M b is the average grayscale value of the background layer; M is the average value of the entire image.

[0022] Optionally, the first edge pixel points of the infrared thermal imaging image are screened according to the first edge pixel point rejection threshold to obtain third edge pixel points of the infrared thermal imaging image; and the second edge pixel points of the visible light image are screened according to the second edge pixel point rejection threshold to obtain fourth edge pixel points of the visible light image, including:

[0023] Eliminate first edge pixel points of the infrared thermal imaging image that are smaller than the first edge pixel point elimination threshold to obtain third edge pixel points of the infrared thermal imaging image;

[0024] The second edge pixel points of the visible light image that are smaller than the second edge pixel point elimination threshold are eliminated to obtain fourth edge pixel points of the infrared thermal imaging image.

[0025] Optionally, the fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image includes:

[0026] Performing edge detection on the registered image of the infrared thermal imaging image and the registered image of the visible light image respectively;

[0027] Determine, according to the edge detection result, a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0028] Determining a Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0029] According to the Euclidean distance, the registered image of the infrared thermal imaging image and the registered image of the visible light image are fused.

[0030] Optionally, the Euclidean distance includes a first Euclidean distance and a second Euclidean distance; and determining the Euclidean distance between the registered image of the infrared thermal imaging image and the registered image of the visible light image includes:

[0031] Determine a zero-order matrix and a first-order matrix of the registration image of the infrared thermal imaging image according to the fifth edge pixel point; determine a zero-order matrix and a first-order matrix of the registration image of the visible light image according to the sixth edge pixel point;

[0032] Determine a first Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the zero-order matrix of the registration image of the infrared thermal imaging image and the zero-order matrix of the registration image of the visible light image;

[0033] Determine a second Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the first-order matrix of the registration image of the infrared thermal imaging image and the first-order matrix of the registration image of the visible light image;

[0034] The step of fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the Euclidean distance comprises:

[0035] When the first Euclidean distance and the second Euclidean distance simultaneously satisfy a preset threshold condition, the registered image of the infrared thermal imaging image and the registered image of the visible light image are fused.

[0036] According to a first aspect of an embodiment of the present invention, there is provided an image fusion device, the device comprising:

[0037] An acquisition module, used to acquire an infrared thermal imaging image and a visible light image at the same time, and respectively determine grayscale information of the infrared thermal imaging image and the visible light image;

[0038] A determination module, configured to determine a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and to determine a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image;

[0039] A detection module, used to perform edge detection on the infrared thermal imaging image and the visible light image, and determine a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image according to the edge detection results;

[0040] a screening module, configured to screen the first edge pixel points of the infrared thermal imaging image according to the first edge pixel point rejection threshold to obtain the third edge pixel points of the infrared thermal imaging image; and to screen the second edge pixel points of the visible light image according to the second edge pixel point rejection threshold to obtain the fourth edge pixel points of the visible light image;

[0041] a registration module, configured to register the infrared thermal imaging image and the visible light image according to a third edge pixel point of the infrared thermal imaging image and a fourth edge pixel point of the visible light image, so as to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image respectively;

[0042] A fusion module is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image.

[0043] Optionally, the acquisition module includes:

[0044] The first determination submodule is used to determine the grayscale information corresponding to the infrared thermal imaging image and the visible light image under different edge pixel point rejection thresholds; the grayscale information includes the grayscale average of the foreground layer, the grayscale average of the background layer and the grayscale average of the entire image.

[0045] Optionally, the determining module includes:

[0046] A second determination submodule is used to determine a first inter-class variance between a foreground layer and a background layer of the infrared thermal imaging image according to grayscale information corresponding to the infrared thermal imaging image under different edge pixel point rejection thresholds;

[0047] A first solving submodule is used to solve the first inter-class variance, and use the edge pixel point elimination threshold when the first inter-class variance is maximized as the first edge pixel point elimination threshold;

[0048] A third determination submodule, configured to determine a second inter-class variance between a foreground layer and a background layer of the visible light image according to grayscale information corresponding to the visible light image under different edge pixel rejection thresholds;

[0049] The second solving submodule is used to solve the second inter-class variance, and use the edge pixel point elimination threshold when the second inter-class variance is maximized as the second edge pixel point elimination threshold.

[0050] Optionally, the second determining submodule includes:

[0051] A first determining unit is used to determine a first inter-class variance between the foreground layer and the background layer of the infrared thermal imaging image according to the following formula based on the grayscale average value of the foreground layer of the infrared thermal imaging image, the grayscale average value of the background layer and the grayscale average value of the entire image;

[0052] σ 2 =P f ×(M f -M) 2 +P b ×(M b -M) 2

[0053] Where σ is the between-class variance; P f is the probability that the total number of foreground pixels accounts for the total number of pixels; P b is the probability that the total number of background pixels accounts for the total number of pixels; M f is the average grayscale value of the foreground layer; M b is the average grayscale value of the background layer; M is the average value of the entire image.

[0054] Optionally, the screening module includes:

[0055] A first screening submodule, configured to remove first edge pixel points of the infrared thermal imaging image that are smaller than the first edge pixel removal threshold, to obtain third edge pixel points of the infrared thermal imaging image;

[0056] The second screening submodule is used to remove the second edge pixel points of the visible light image that are smaller than the second edge pixel point removal threshold to obtain fourth edge pixel points of the infrared thermal imaging image.

[0057] Optionally, the fusion module includes:

[0058] A first detection submodule, used for performing edge detection on the registered image of the infrared thermal imaging image and the registered image of the visible light image respectively;

[0059] a fourth determination submodule, configured to determine, according to the edge detection result, a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0060] a fifth determination submodule, configured to determine a Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0061] The first fusion submodule is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the Euclidean distance.

[0062] Optionally, the Euclidean distance includes a first Euclidean distance and a second Euclidean distance; and the fifth determination submodule includes:

[0063] A second determining unit is used to determine a zero-order matrix and a first-order matrix of the registration image of the infrared thermal imaging image according to the fifth edge pixel point; and to determine a zero-order matrix and a first-order matrix of the registration image of the visible light image according to the sixth edge pixel point;

[0064] A third determining unit, configured to determine a first Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a zero-order matrix of the registration image of the infrared thermal imaging image and a zero-order matrix of the registration image of the visible light image;

[0065] a fourth determining unit, configured to determine a second Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a first-order matrix of the registration image of the infrared thermal imaging image and a first-order matrix of the registration image of the visible light image;

[0066] The first fusion submodule includes:

[0067] The first fusion unit is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image when the first Euclidean distance and the second Euclidean distance simultaneously meet a preset threshold condition.

[0068] According to a third aspect of the present invention, an electronic device is provided, the cooking device comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the image fusion method as described in any one of the above items when executed by the processor.

[0069] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image fusion method as described in any one of the above items are implemented.

[0070] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0071] The embodiment of the present invention provides an image fusion method, which obtains an infrared thermal imaging image and a visible light image at the same time, and respectively determines the grayscale information corresponding to the two images; determines the edge pixel point elimination threshold corresponding to the two images according to the grayscale information; performs edge detection on the two images to determine the corresponding edge pixels; screens the edge pixels corresponding to the two images according to the edge pixel point elimination threshold corresponding to the two images to obtain the screened edge pixels corresponding to the two images; and performs image fusion after aligning the two images according to the screened edge pixels. The embodiment of the present invention determines the edge pixel point elimination threshold of the infrared thermal imaging image and the visible light image respectively through the grayscale information of the two images, and then screens the edge pixels, and performs image alignment and fusion based on the screened edge pixels. The imaging effect and accuracy of image fusion are improved, and the quality of the final image is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a flowchart of the steps of an image fusion method provided by an embodiment of the present invention;

[0073] Figure 2 It is a structural block diagram of an image fusion device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] One of the core concepts of the embodiment of the present invention is to determine the edge pixel removal thresholds of the infrared thermal imaging image and the visible light image respectively through the grayscale information of the infrared thermal imaging image and the visible light image, and then filter the edge pixels, and perform image registration and fusion based on the filtered edge pixels. This improves the imaging effect and accuracy of image fusion and ensures the quality of the final image.

[0076] Reference Figure 1 , shows a flowchart of the steps of an image fusion method provided by an embodiment of the present invention, and the method may specifically include the following steps:

[0077] Step 101, acquiring an infrared thermal imaging image and a visible light image at the same time, and determining grayscale information of the infrared thermal imaging image and the visible light image respectively;

[0078] After obtaining the infrared thermal imaging image and the visible light image at the same time, the resolution of the infrared thermal imaging image and the visible light image is unified, and the infrared thermal imaging image is denoised. Specifically, the resolution of the infrared thermal imaging image and the visible light image is 320×240, and the original data resolution of the infrared thermal imaging image is 80×62. It is enlarged to 320×240 using bilinear interpolation. However, the noise in the thermal imaging image will affect the final fusion effect. Therefore, it is necessary to use Gaussian filtering to smooth the infrared thermal imaging image and remove the noise. A 3×3 convolution kernel is used, and the convolution kernel is set to:

[0079]

[0080] When using software convolution operators to implement Gaussian filtering, it takes a long time, about 180ms, so a convolutional neural network is constructed to implement Gaussian filtering. This method does not require training. Only one 3×3 convolution layer is constructed using the Keras framework, and the weight is the above k. The weight is directly loaded into the convolution layer through the model setting weight in the Keras framework, and finally the entire model is saved. After that, the compiler is used to compile the model file into an operator stream supported by the chip for calculation, which takes about 13ms, greatly reducing the algorithm time consumption.

[0081] The visible light image and the denoised infrared thermal imaging image are converted into grayscale images, and the grayscale information of the infrared thermal imaging image and the visible light image is determined based on the grayscale images. The grayscale image reduces the complexity of color information and reduces the consumption of computing resources, which is especially suitable for the limited processing power of embedded platforms. The grayscale image can focus more on brightness and edge features, which is crucial for subsequent edge detection, registration and fusion.

[0082] Step 102, determining a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and determining a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image;

[0083] In order to optimize the quality of the two images before fusion, it is necessary to determine the inaccurate edge pixels of each image based on the grayscale information of the infrared thermal imaging image and the visible light image, ensuring that only high-quality edge information is involved in subsequent processing. This not only improves the accuracy of image registration, but also enhances the clarity and accuracy of the fused image, so as to improve the final multimodal image fusion effect.

[0084] Step 103, performing edge detection on the infrared thermal imaging image and the visible light image respectively, and determining a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image respectively according to the edge detection results;

[0085] The edge detection of infrared thermal imaging images and visible light images is performed using the Canny edge detection algorithm. The Canny edge detection algorithm is specifically as follows:

[0086] First, use the first-order finite difference to calculate the gradient of each pixel point. Then the two matrices of the partial derivatives of the image in the x and y directions can use the Sobel operator as the gradient operator, that is, the convolution kernels in the x and y directions are:

[0087]

[0088] Assume the preset image is:

[0089]

[0090] A 11 For example, A 11 The gradients in the x and y directions are:

[0091] G x =2A 12 -2A 10 +A 22 +A 02 -A 20 -A 00

[0092] G y =2A 21 -2A 01 -A 00 -A 02 +A 20 +A 22

[0093] Then A 11 The gradient magnitude and direction are:

[0094]

[0095] θ=arctan(G y / G x )

[0096] The larger the element value in the image gradient amplitude matrix within the 8-neighborhood of each pixel point, the larger the gradient value of the point in the image, and the point is an edge pixel point. If the gradient value of a certain pixel point is not the largest compared with the gradient values ​​of the two pixels before and after it in the gradient direction, then this pixel point is not an edge pixel point. The first edge pixel point in the infrared thermal imaging image and the second edge pixel point in the visible light image are determined respectively in the above manner.

[0097] Step 104: screening the first edge pixel points of the infrared thermal imaging image according to the first edge pixel point rejection threshold to obtain the third edge pixel points of the infrared thermal imaging image; and screening the second edge pixel points of the visible light image according to the second edge pixel point rejection threshold to obtain the fourth edge pixel points of the visible light image;

[0098] Setting the grayscale values ​​of non-edge pixels to 0 will result in a binary image. Such a detection result still contains many false edges caused by noise and other reasons. The original Canny edge detection algorithm uses dual threshold screening processing, and manual determination of the threshold requires continuous attempts. Therefore, the present application improves the threshold setting algorithm. Through the grayscale information of the infrared thermal imaging image and the visible light image, the first edge pixel point rejection threshold of the infrared thermal imaging image and the second edge pixel point rejection threshold of the visible light image are determined. The false edges are rejected according to the corresponding edge pixel point rejection threshold to obtain accurate edge pixels.

[0099] Step 105, registering the infrared thermal imaging image and the visible light image according to the third edge pixel point of the infrared thermal imaging image and the fourth edge pixel point of the visible light image, to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image respectively;

[0100] Image registration based on the third edge pixel of the infrared thermal imaging image and the fourth edge pixel of the visible light image is to ensure that the two images are aligned in space, thereby achieving accurate multimodal information fusion. By selecting high-quality edge pixels, the registration accuracy can be improved to ensure that the thermal features in the infrared image accurately correspond to the visual features in the visible light image. This step solves the problem of image misalignment caused by differences in sensor viewing angles, resolutions, or shooting times, allowing subsequent fusion processing to fully utilize the advantages of both modalities and provide richer and more accurate information.

[0101] Step 106 : fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image.

[0102] Infrared images can sense the thermal radiation of objects and are suitable for low-light or nighttime environments, while visible light images capture the color and shape information of objects and perform well in daytime or good lighting conditions. Registration ensures that the two images are accurately aligned in space, solving the problem of perspective, resolution or time differences between different sensors.

[0103] In one embodiment, step 101 may include the following sub-steps:

[0104] Sub-step S11, determining the grayscale information corresponding to the infrared thermal imaging image and the visible light image under different edge pixel point rejection thresholds; the grayscale information includes the grayscale average of the foreground layer, the grayscale average of the background layer and the grayscale average of the entire image.

[0105] Calculating the grayscale average of the foreground layer and the background layer can better distinguish the target object and the background in the image and improve the contrast of the image. The grayscale average of the entire image reflects the overall brightness distribution, which helps adjust the parameters of the fusion algorithm to make the fused image more natural and harmonious, avoiding overexposure or underexposure. By analyzing the grayscale information under different thresholds, noise or inaccurate edge pixels can be more accurately identified and removed. This ensures that high-quality edge information participates in subsequent registration and fusion, thereby improving the quality of the final image.

[0106] In one embodiment, step 102 may include the following sub-steps:

[0107] Sub-step S21, determining a first inter-class variance between a foreground layer and a background layer of the infrared thermal imaging image according to grayscale information corresponding to the infrared thermal imaging image under different edge pixel point rejection thresholds;

[0108] In one embodiment, the sub-step S21 may include the following sub-steps:

[0109] Sub-step S211, determining the first between-class variance between the foreground layer and the background layer of the infrared thermal imaging image according to the following formula based on the grayscale average value of the foreground layer of the infrared thermal imaging image, the grayscale average value of the background layer and the grayscale average value of the entire image;

[0110] σ 2 =P f ×(M f -M) 2 +P b ×(M b -M) 2

[0111] Where σ is the between-class variance; P f is the probability that the total number of foreground pixels accounts for the total number of pixels; P b is the probability that the total number of background pixels accounts for the total number of pixels; M f is the average grayscale value of the foreground layer; M b is the average grayscale value of the background layer; M is the average value of the entire image.

[0112] Determine P according to the following formula f and P b :

[0113]

[0114] Among them, T0 is the initial edge pixel removal threshold, which divides the image into the foreground layer f and the background layer b. The total number of pixels is N, and the number of foreground pixels is N f , the number of background pixels is N b , the total number of gray levels of the image is L, and the number of pixels in each gray level is N i ,

[0115] Determine M according to the following formula f 、M b and M:

[0116]

[0117] M=P f ×M f +P b ×M b

[0118] Among them, P i is the probability that the number of pixels of each gray level accounts for the total number of pixels, and as T0 takes different values, the corresponding P f , P b 、M f 、M b The values ​​of and M will be different, so the value of the inter-class variance σ will also be different. The T0 value that maximizes σ is used as the threshold for removing edge pixels;

[0119] According to the above method, the first inter-class variance of the infrared thermal imaging image is determined to determine the first edge pixel point rejection threshold, and the second inter-class variance of the visible light image is determined to determine the second edge pixel point rejection threshold.

[0120] Sub-step S22, solving the first inter-class variance, and taking the edge pixel point elimination threshold when the first inter-class variance is maximized as the first edge pixel point elimination threshold;

[0121] Sub-step S23, determining a second inter-class variance between a foreground layer and a background layer of the visible light image according to grayscale information corresponding to different edge pixel rejection thresholds of the visible light image;

[0122] Sub-step S24, solving the second inter-class variance, and taking the edge pixel point elimination threshold when the second inter-class variance is maximized as the second edge pixel point elimination threshold.

[0123] The grayscale information of the infrared image under different edge pixel rejection thresholds is analyzed to determine the first inter-class variance between the foreground layer and the background layer. The inter-class variance measures the degree of difference between the foreground and the background. A larger inter-class variance indicates a better classification effect. By solving the first inter-class variance and selecting the threshold value when the variance is maximized as the first edge pixel rejection threshold, it is possible to ensure that the most significant edge features are retained while removing noise and inaccurate edge information. This method improves the accuracy of edge detection and provides high-quality input for subsequent registration and fusion. Similarly, for visible light images, the second inter-class variance between the foreground layer and the background layer is calculated under different edge pixel rejection thresholds. Similarly, the second inter-class variance is solved and the threshold value when the variance is maximized is selected as the second edge pixel rejection threshold. This step ensures that important edges in visible light images are accurately identified and background noise is effectively suppressed.

[0124] In one embodiment, step 104 may include the following sub-steps:

[0125] Sub-step S31, removing first edge pixel points of the infrared thermal imaging image that are smaller than the first edge pixel point removal threshold, to obtain third edge pixel points of the infrared thermal imaging image;

[0126] Sub-step S32, removing the second edge pixel points of the visible light image that are smaller than the second edge pixel point removal threshold, to obtain fourth edge pixel points of the infrared thermal imaging image.

[0127] The infrared image edge pixels that are smaller than the first edge pixel elimination threshold are eliminated to obtain the third edge pixel. This step ensures that only significant and reliable edges are retained, reducing false detections caused by noise or low contrast. Similarly, the visible light image edge pixels that are smaller than the second edge pixel elimination threshold are eliminated to obtain the fourth edge pixel. Through this screening process, background noise and other interference can be effectively removed, retaining clear and accurate edge information. The screened third and fourth edge pixels provide a more reliable basis for subsequent image registration and fusion. High-quality edge information helps to improve the registration accuracy and ensure the spatial consistency of infrared and visible light images, thereby improving the quality and accuracy of the fused image.

[0128] In one embodiment, step 106 may include the following sub-steps:

[0129] Sub-step S41, performing edge detection on the registered image of the infrared thermal imaging image and the registered image of the visible light image respectively;

[0130] Sub-step S42, determining a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image according to the edge detection result;

[0131] Sub-step S43, determining the Euclidean distance between the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the fifth edge pixel point of the registered image of the infrared thermal imaging image and the sixth edge pixel point of the registered image of the visible light image;

[0132] Based on the edge detection results, the Euclidean distance between the fifth edge pixel and the sixth edge pixel is calculated. The Euclidean distance is used to measure the spatial similarity of two images and help evaluate their matching degree. According to the calculated Euclidean distance, decide whether and how to fuse the two images. A smaller Euclidean distance indicates that the image features are similar and suitable for fusion; a larger distance may indicate that the feature difference is too large and it is not suitable for direct fusion. In this way, the system can maximize the use of information from the two modalities while ensuring image quality. This method not only improves the accuracy of image registration, but also enhances the robustness and adaptability of the system. High-quality edge information and reasonable fusion strategies ensure the clarity and accuracy of the final fused image, which is particularly suitable for multimodal image processing tasks in complex environments, such as security monitoring, intelligent driving, etc.

[0133] In one embodiment, the sub-step S43 may include the following sub-steps:

[0134] Sub-step S431, determining a zero-order matrix and a first-order matrix of the registration image of the infrared thermal imaging image according to the fifth edge pixel point; determining a zero-order matrix and a first-order matrix of the registration image of the visible light image according to the sixth edge pixel point;

[0135] Sub-step S432, determining a first Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the zero-order matrix of the registration image of the infrared thermal imaging image and the zero-order matrix of the registration image of the visible light image;

[0136] Sub-step S433, determining a second Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the first-order matrix of the registration image of the infrared thermal imaging image and the first-order matrix of the registration image of the visible light image;

[0137] Specifically, the edge detection is performed on the registered image of the infrared thermal imaging image and the registered image of the visible light image by using the Canny edge detection algorithm to obtain the fifth edge pixel point and the sixth edge pixel point. Then, the contour moments, i.e., the zero-order moment and the first-order moment, are obtained for the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the fifth edge pixel point and the sixth edge pixel point, respectively, in the following manner:

[0138] Determine the p+q order geometric matrix m of the image f(i, j) whose registration image is M×N pq for:

[0139]

[0140] Among them, f(i, j) is the gray value of the image at (i, j). From the above formula, the zero-order matrix, that is, p=q=0, is:

[0141]

[0142] The first-order matrix is:

[0143]

[0144] When the image is a binary image, m 00 is the sum of the white areas in the image; m 10 is the cumulative sum of the x coordinates of the white pixels in the image, and m 01 is the cumulative sum of the y coordinates;

[0145] The first Euclidean distance is:

[0146]

[0147] The second Euclidean distance includes the second Euclidean distance in the x direction and the second Euclidean distance in the y direction, which are:

[0148] Sub-step S44: fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the Euclidean distance.

[0149] In one embodiment, the sub-step S44 may include the following sub-steps:

[0150] Sub-step S441, when the first Euclidean distance and the second Euclidean distance simultaneously meet a preset threshold condition, fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image.

[0151] When the first Euclidean distance and the second Euclidean distance are both less than the preset threshold conditions, the registered image of the infrared thermal imaging image and the registered image of the visible light image are fused. And because the infrared thermal imaging image is taken by a low-resolution camera, the imaging effect is poor and the edge detection effect is poor, so when the set threshold is met, the edge of the visible light is set as the fused edge, and the gray-white pixels are also mainly visible light.

[0152] The embodiment of the present invention provides an image fusion method, which obtains an infrared thermal imaging image and a visible light image at the same time, and respectively determines the grayscale information corresponding to the two images; determines the edge pixel point elimination threshold corresponding to the two images according to the grayscale information; performs edge detection on the two images to determine the corresponding edge pixels; screens the edge pixels corresponding to the two images according to the edge pixel point elimination threshold corresponding to the two images to obtain the screened edge pixels corresponding to the two images; and performs image fusion after aligning the two images according to the screened edge pixels. The embodiment of the present invention determines the edge pixel point elimination threshold of the infrared thermal imaging image and the visible light image respectively through the grayscale information of the two images, and then screens the edge pixels, and performs image alignment and fusion based on the screened edge pixels. The imaging effect and accuracy of image fusion are improved, and the quality of the final image is ensured.

[0153] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0154] Reference Figure 2 , shows a structural block diagram of an image fusion device provided by an embodiment of the present invention, which may specifically include the following modules:

[0155] An acquisition module 201 is used to acquire an infrared thermal imaging image and a visible light image at the same time, and respectively determine grayscale information of the infrared thermal imaging image and the visible light image;

[0156] A determination module 202 is used to determine a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and to determine a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image;

[0157] A detection module 203 is used to perform edge detection on the infrared thermal imaging image and the visible light image, and determine a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image according to the edge detection results;

[0158] A screening module 204 is used to screen the first edge pixel points of the infrared thermal imaging image according to the first edge pixel point rejection threshold to obtain the third edge pixel points of the infrared thermal imaging image; and to screen the second edge pixel points of the visible light image according to the second edge pixel point rejection threshold to obtain the fourth edge pixel points of the visible light image;

[0159] A registration module 205, configured to register the infrared thermal imaging image and the visible light image according to the third edge pixel point of the infrared thermal imaging image and the fourth edge pixel point of the visible light image, to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image, respectively;

[0160] The fusion module 206 is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image.

[0161] In one embodiment, the acquisition module includes:

[0162] The first determination submodule is used to determine the grayscale information corresponding to the infrared thermal imaging image and the visible light image under different edge pixel point rejection thresholds; the grayscale information includes the grayscale average of the foreground layer, the grayscale average of the background layer and the grayscale average of the entire image.

[0163] In one embodiment, the determining module includes:

[0164] A second determination submodule is used to determine a first inter-class variance between a foreground layer and a background layer of the infrared thermal imaging image according to grayscale information corresponding to the infrared thermal imaging image under different edge pixel point rejection thresholds;

[0165] A first solving submodule is used to solve the first inter-class variance, and use the edge pixel point elimination threshold when the first inter-class variance is maximized as the first edge pixel point elimination threshold;

[0166] A third determination submodule, configured to determine a second inter-class variance between a foreground layer and a background layer of the visible light image according to grayscale information corresponding to the visible light image under different edge pixel rejection thresholds;

[0167] The second solving submodule is used to solve the second inter-class variance, and use the edge pixel point elimination threshold when the second inter-class variance is maximized as the second edge pixel point elimination threshold.

[0168] In one embodiment, the second determining submodule includes:

[0169] A first determining unit is used to determine a first inter-class variance between the foreground layer and the background layer of the infrared thermal imaging image according to the following formula based on the grayscale average value of the foreground layer of the infrared thermal imaging image, the grayscale average value of the background layer and the grayscale average value of the entire image;

[0170] σ 2 =P f ×(M f -M) 2 +P b ×(M b -M) 2

[0171] Where σ is the between-class variance; P f is the probability that the total number of foreground pixels accounts for the total number of pixels; P b is the probability that the total number of background pixels accounts for the total number of pixels; M f is the average grayscale value of the foreground layer; M b is the average grayscale value of the background layer; M is the average value of the entire image.

[0172] In one embodiment, the screening module comprises:

[0173] A first screening submodule, configured to remove first edge pixel points of the infrared thermal imaging image that are smaller than the first edge pixel removal threshold, to obtain third edge pixel points of the infrared thermal imaging image;

[0174] The second screening submodule is used to remove the second edge pixel points of the visible light image that are smaller than the second edge pixel point removal threshold to obtain fourth edge pixel points of the infrared thermal imaging image.

[0175] In one embodiment, the fusion module includes:

[0176] A first detection submodule, used for performing edge detection on the registered image of the infrared thermal imaging image and the registered image of the visible light image respectively;

[0177] a fourth determination submodule, configured to determine, according to the edge detection result, a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0178] a fifth determination submodule, configured to determine a Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image;

[0179] The first fusion submodule is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the Euclidean distance.

[0180] In one embodiment, the Euclidean distance includes a first Euclidean distance and a second Euclidean distance; the fifth determination submodule includes:

[0181] A second determining unit is used to determine a zero-order matrix and a first-order matrix of the registration image of the infrared thermal imaging image according to the fifth edge pixel point; and to determine a zero-order matrix and a first-order matrix of the registration image of the visible light image according to the sixth edge pixel point;

[0182] A third determining unit, configured to determine a first Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a zero-order matrix of the registration image of the infrared thermal imaging image and a zero-order matrix of the registration image of the visible light image;

[0183] a fourth determining unit, configured to determine a second Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a first-order matrix of the registration image of the infrared thermal imaging image and a first-order matrix of the registration image of the visible light image;

[0184] The first fusion submodule includes:

[0185] The first fusion unit is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image when the first Euclidean distance and the second Euclidean distance simultaneously meet a preset threshold condition.

[0186] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0187] The embodiment of the present invention provides an image fusion method, which obtains an infrared thermal imaging image and a visible light image at the same time, and respectively determines the grayscale information corresponding to the two images; determines the edge pixel point elimination threshold corresponding to the two images according to the grayscale information; performs edge detection on the two images to determine the corresponding edge pixels; screens the edge pixels corresponding to the two images according to the edge pixel point elimination threshold corresponding to the two images to obtain the screened edge pixels corresponding to the two images; and performs image fusion after aligning the two images according to the screened edge pixels. The embodiment of the present invention determines the edge pixel point elimination threshold of the infrared thermal imaging image and the visible light image respectively through the grayscale information of the two images, and then screens the edge pixels, and performs image alignment and fusion based on the screened edge pixels. The imaging effect and accuracy of image fusion are improved, and the quality of the final image is ensured.

[0188] An embodiment of the present invention further provides an electronic device, including:

[0189] The present invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, each process of the above-mentioned image fusion method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0190] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image fusion method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0191] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0192] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0193] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0195] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0196] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0197] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0198] The above is a detailed introduction to an image fusion method and an image fusion device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An image fusion method, characterized in that: The method comprises: Acquire an infrared thermal imaging image and a visible light image at the same time, and determine grayscale information of the infrared thermal imaging image and the visible light image respectively; Determine a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and determine a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image; Performing edge detection on the infrared thermal imaging image and the visible light image respectively, and determining a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image respectively according to the edge detection results; According to the first edge pixel point rejection threshold, the first edge pixel points of the infrared thermal imaging image are screened to obtain third edge pixel points of the infrared thermal imaging image; and according to the second edge pixel point rejection threshold, the second edge pixel points of the visible light image are screened to obtain fourth edge pixel points of the visible light image; According to the third edge pixel point of the infrared thermal imaging image and the fourth edge pixel point of the visible light image, the infrared thermal imaging image and the visible light image are registered to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image respectively; Fusion is performed based on the registered image of the infrared thermal imaging image and the registered image of the visible light image.

2. The method according to claim 1, characterized in that The step of determining the grayscale information of the infrared thermal imaging image and the visible light image includes: Determine the grayscale information corresponding to the infrared thermal imaging image and the visible light image under different edge pixel point rejection thresholds; the grayscale information includes the grayscale average value of the foreground layer, the grayscale average value of the background layer and the grayscale average value of the entire image.

3. The method according to claim 2, characterized in that The step of determining a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and determining a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image, comprises: Determine a first inter-class variance between a foreground layer and a background layer of the infrared thermal imaging image according to grayscale information corresponding to different edge pixel point rejection thresholds of the infrared thermal imaging image; Solving the first inter-class variance, and taking the edge pixel point elimination threshold when the first inter-class variance is maximized as the first edge pixel point elimination threshold; determining a second inter-class variance between a foreground layer and a background layer of the visible light image according to grayscale information corresponding to the visible light image under different edge pixel rejection thresholds; The second inter-class variance is solved, and the edge pixel point elimination threshold when the second inter-class variance is maximized is used as the second edge pixel point elimination threshold.

4. The method according to claim 3, characterized in that The determining, according to the grayscale information corresponding to the infrared thermal imaging image under different edge pixel point rejection thresholds, a first inter-class variance between the foreground layer and the background layer of the infrared thermal imaging image comprises: According to the grayscale average value of the foreground layer of the infrared thermal imaging image, the grayscale average value of the background layer and the grayscale average value of the entire image, the first between-class variance between the foreground layer and the background layer of the infrared thermal imaging image is determined according to the following formula; σ 2 =P f ×(M f -M) 2 +P b ×(M b -M) 2 Where σ is the between-class variance; P f is the probability that the total number of foreground pixels accounts for the total number of pixels; P b is the probability that the total number of background pixels accounts for the total number of pixels; M f is the average grayscale value of the foreground layer; M b is the average grayscale value of the background layer; M is the average value of the entire image.

5. The method according to claim 1, characterized in that The first edge pixel points of the infrared thermal imaging image are screened according to the first edge pixel point rejection threshold to obtain the third edge pixel points of the infrared thermal imaging image; and the second edge pixel points of the visible light image are screened according to the second edge pixel point rejection threshold to obtain the fourth edge pixel points of the visible light image, including: Eliminate first edge pixel points of the infrared thermal imaging image that are smaller than the first edge pixel point elimination threshold to obtain third edge pixel points of the infrared thermal imaging image; The second edge pixel points of the visible light image that are smaller than the second edge pixel point elimination threshold are eliminated to obtain fourth edge pixel points of the infrared thermal imaging image.

6. The method according to claim 1, characterized in that The fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image comprises: Performing edge detection on the registered image of the infrared thermal imaging image and the registered image of the visible light image respectively; Determine, according to the edge detection result, a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image; Determining a Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to a fifth edge pixel point of the registration image of the infrared thermal imaging image and a sixth edge pixel point of the registration image of the visible light image; According to the Euclidean distance, the registered image of the infrared thermal imaging image and the registered image of the visible light image are fused.

7. The method according to claim 6, characterized in that The Euclidean distance includes a first Euclidean distance and a second Euclidean distance; and determining the Euclidean distance between the registered image of the infrared thermal imaging image and the registered image of the visible light image includes: Determine a zero-order matrix and a first-order matrix of the registration image of the infrared thermal imaging image according to the fifth edge pixel point; determine a zero-order matrix and a first-order matrix of the registration image of the visible light image according to the sixth edge pixel point; Determine a first Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the zero-order matrix of the registration image of the infrared thermal imaging image and the zero-order matrix of the registration image of the visible light image; Determine a second Euclidean distance between the registration image of the infrared thermal imaging image and the registration image of the visible light image according to the first-order matrix of the registration image of the infrared thermal imaging image and the first-order matrix of the registration image of the visible light image; The step of fusing the registered image of the infrared thermal imaging image and the registered image of the visible light image according to the Euclidean distance comprises: When the first Euclidean distance and the second Euclidean distance simultaneously satisfy a preset threshold condition, the registered image of the infrared thermal imaging image and the registered image of the visible light image are fused.

8. An image fusion device, characterized in that: The device comprises: An acquisition module, used to acquire an infrared thermal imaging image and a visible light image at the same time, and respectively determine grayscale information of the infrared thermal imaging image and the visible light image; A determination module, configured to determine a first edge pixel point rejection threshold of the infrared thermal imaging image according to the grayscale information of the infrared thermal imaging image, and to determine a second edge pixel point rejection threshold of the visible light image according to the grayscale information of the visible light image; A detection module, used to perform edge detection on the infrared thermal imaging image and the visible light image, and determine a first edge pixel point in the infrared thermal imaging image and a second edge pixel point in the visible light image according to the edge detection results; a screening module, configured to screen the first edge pixel points of the infrared thermal imaging image according to the first edge pixel point rejection threshold to obtain the third edge pixel points of the infrared thermal imaging image; and to screen the second edge pixel points of the visible light image according to the second edge pixel point rejection threshold to obtain the fourth edge pixel points of the visible light image; a registration module, configured to register the infrared thermal imaging image and the visible light image according to a third edge pixel point of the infrared thermal imaging image and a fourth edge pixel point of the visible light image, so as to obtain a registered image of the infrared thermal imaging image and a registered image of the visible light image respectively; A fusion module is used to fuse the registered image of the infrared thermal imaging image and the registered image of the visible light image.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the image fusion method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image fusion method according to any one of claims 1 to 7 are implemented.

Images