A Real-Time Image Fusion Method Based on Salient Detail Extraction and Parameter Optimization

Abstract

A real-time image fusion method based on salient detail extraction and parameter optimization, belonging to the technical field of image fusion, includes the following steps: S1: acquiring registered infrared and visible light images, processing them, and converting them into YUV components; S2: acquiring a visible light salient detail map and calculating its sharpness Cla; S3: fusing the visible light salient detail map with the Y component of the infrared image to obtain a target fused image, and calculating the quality index Qua value of the target fused image; S4: using an intelligent optimization algorithm to optimize a fixed value within a given range, finding the fixed value with the lowest fitness, repeating steps S2 to S3 to obtain the final target fused image, and converting it into RGB components. This invention extracts salient details from the visible light image to the maximum extent and fuses the salient details of the visible light image with the thermal infrared image; for a specific application scenario, it can significantly improve fusion efficiency and achieve real-time fusion.

Description

Technical Field

[0001] This invention discloses a real-time image fusion method based on salient detail extraction and parameter optimization, which relates to the technical field of image fusion. Background Technology

[0002] With the continuous development of sensors, the content and representation of sensor information for the same scene have become more diverse and complex. Taking infrared and visible light images as examples, in real-world applications, infrared images reflect the thermal characteristics of objects, while visible light images reflect the texture details of images. Their features are different but complementary. Image fusion, which comprehensively considers both types of features, can better solve real-world problems. However, current image fusion methods have two limitations: first, the fusion algorithms are complex and time-consuming, making them unsuitable for real-time applications; second, the fusion effect is poor, failing to adequately represent both types of feature information.

[0003] Chinese Patent Document CN110363732A discloses an image fusion method and apparatus. The method includes: converting a visible light image acquired by an image acquisition device into an RGB image; converting a near-infrared image acquired by the image acquisition device into a first brightness image; converting the RGB brightness image into a second brightness image; calculating near-infrared image fusion weights based on the first brightness image to obtain a fusion weight map of the near-infrared image; performing brightness fusion on the first brightness image and the second brightness image based on the fusion weight map to obtain a brightness fused image; and performing RGB fusion on the second brightness image, the brightness fused image, and the RGB image to obtain a fused RGB image. This method can improve the brightness of images in low-light environments, maintain the color information of the image, and improve the quality of the fused image.

[0004] Chinese patent document CN102298769A discloses a color fusion method for low-light night vision images and infrared images based on color transfer. This invention first linearly fuses visible light and infrared images to obtain the , , and components of a grayscale fused image. The , , and components of the grayscale fused image are then adjusted using the brightness and color distribution of a reference image. The adjusted image data is then inversely transformed from YUV space back to RGB space to obtain the final color fused image. The fused image obtained using this method has a similar brightness distribution to the color reference image, resulting in good visual effects.

[0005] Chinese Patent Document CN113936017A discloses an image processing method and apparatus. The method includes: acquiring an infrared image of a thermal target; acquiring a visible light image of the thermal target; acquiring an infrared contour image of the thermal target based on the infrared image; and acquiring a fused image based on the infrared contour image and the visible light image. The fused image acquired by the method has good quality.

[0006] Chinese patent document CN109255774A discloses an image fusion method, apparatus, and device, comprising: fusing infrared and visible light images to be fused using different preset fusion rules to generate a grayscale fused image; replacing the Y-channel data of a pseudo-color fused image generated by mapping the infrared and visible light images using the data of the grayscale fused image; further detecting moving targets in the infrared image; setting different transfer weights for the moving target region and the background region respectively; and performing color transfer on the pseudo-color fused image according to a preset reference image and the transfer weights to generate a target-enhanced true-color fused image. In other words, this invention replaces the Y-channel data of the pseudo-color fused image with the grayscale fused image obtained by fusing using different preset fusion rules, and performs color transfer on the moving target region and the background region using different transfer weights, which can effectively enhance the detail information in the fused image and improve the saliency of the image target.

[0007] Chinese patent document CN112241935A provides an image processing method, apparatus, device, and storage medium. The image processing method includes acquiring a first image and a second image in a first data format; both the first and second images contain a target object, and the target object is located at the same position in both images; the first image is a visible light image, and the second image is a near-infrared light image; processing the first and second images yields a third image and target location information of the target object; the third image integrates color information from the first image and grayscale information from the second image; extracting the region corresponding to the target location information from the second image yields a first region image in a first data format; and enhancing the first region image yields a fourth image for target object recognition. This method can provide more complete monitoring information while improving the accuracy of target recognition.

[0008] The aforementioned fusion methods are either too complex and time-consuming to achieve real-time fusion, or they fail to extract significant image details effectively, resulting in poor fusion quality. Therefore, there is an urgent need for a visible light and infrared image fusion method that balances performance and timeliness, ensuring real-time application requirements while significantly enhancing the extraction of significant details and reducing method complexity, thereby guaranteeing optimal fusion results. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a real-time image fusion method based on salient detail extraction and parameter optimization. Taking infrared and visible light images as examples, it can extract salient details from visible light images to the greatest extent, avoid background information interference, and fuse salient details from visible light images with thermal infrared images with minimal information loss, ensuring the fusion result and enabling observers to understand thermal infrared images more clearly.

[0010] The detailed technical solution of this invention is as follows:

[0011] A real-time image fusion method based on salient detail extraction and parameter optimization, characterized by comprising:

[0012] S1: Acquire the registered infrared and visible light images, process them, and convert them into YUV components;

[0013] S2: Initialize the constant value, obtain the visible light salient detail map based on the constant value and the visible light Y component, and calculate the sharpness Cla of the salient detail map:

[0014] S3: The visible light salient detail image and the infrared image Y component are fused together. The fusion result is used as the Y component, and the U and V components are the corresponding components of the infrared image to obtain the target fused image. The quality index Qua value of the target fused image is calculated.

[0015] S4: Use an intelligent optimization algorithm to find the optimal value for a given range. The intelligent optimization algorithm includes a fitness function, which is the negative sum of Cla and Qua. Find the value with the lowest fitness and repeat steps S2 to S3 to obtain the final target fused image, and convert it into RGB components.

[0016] According to a preferred embodiment of the present invention, the method for processing and converting the component into YUV components in step S1 is as follows:

[0017] The visible light and infrared images are preprocessed to denoise, resulting in denoised visible light and infrared images. Channel separation is then performed on the denoised visible light and infrared images to obtain their respective Y, U, and V components.

[0018] According to a preferred embodiment of the present invention, the noise reduction method employs mean filtering:

[0019]

[0020] In equation (1), Img_denoised is the image after denoising; Img is the image before denoising. This is the convolution operation; blur is the mean filter template; ksize is the convolution kernel size.

[0021] According to a preferred embodiment of the present invention, the method for performing channel separation on the denoised visible light image and infrared image to obtain their respective Y, U, and V components is as follows:

[0022] The denoised infrared and visible light images are converted from RGB components to YUV components.

[0023] According to a preferred embodiment of the present invention, the method for obtaining a visible light salient detail image based on the initialization setpoint and the infrared Y component in step S2 is as follows:

[0024] The initialization value is a float constant with a range of [0, 255]. The initialization value is a value randomly assigned using a random number function, such as 56.

[0025] The visible light salient detail map is obtained based on a constant value and the infrared Y component: the constant value is subtracted from the Y component of the denoised visible light image to obtain the visible light salient detail map.

[0026] According to a preferred embodiment of the present invention, the method for calculating the sharpness Cla of the salient detail map in step S2 is as follows:

[0027] The sharpness Cla is an image evaluation index, mainly used to assess the sharpness of detail information such as texture and outline in salient detail images, and is in numerical form;

[0028] According to a preferred embodiment of the present invention, the energy gradient method is used as the evaluation index for sharpness Cla:

[0029] The sum of the squared differences in grayscale values ​​between adjacent pixels in the X and Y directions of the image is used as the gradient value of each pixel. The gradient values ​​of all pixels are accumulated to obtain the sharpness evaluation function value.

[0030] F = ∑ X ∑ Y {[f(x+1, y)-f(x, y)] 2 +[f(x, y+1)-f(x, y)] 2} (2)

[0031] In equation (2), F is the energy gradient value, i.e., the sharpness Cla; f is the image pixel gradient calculation function; x and y are the horizontal and vertical coordinates of the image pixel points, respectively;

[0032] For example, given a range of values, such as [150, 255], an intelligent optimization algorithm is used to find the fixed value when the energy gradient value is the maximum. However, for the intelligent optimization algorithm to run, an initial value needs to be given first, which can be randomly selected and will not affect the subsequent optimization process. In the end, the higher the energy gradient value, the clearer the image details and the more obvious the texture contours, which is more conducive to subsequent image fusion.

[0033] According to a preferred embodiment of the present invention, the method for fusing the visible light salient detail image with the infrared image Y component in step S3 is as follows:

[0034] The fusion process utilizes a pixel-weighted average method to fuse salient details and the infrared Y component.

[0035] The pixel-weighted average method is as follows:

[0036] Img = 0.5 * Img1 + 0.5 * Img2 (3)

[0037] In equation (3), Img is the fused image; Img1 is the image to be fused 1; Img2 is the image to be fused 2; Img1 and Img2 correspond to the "visible light significant detail map" and the "infrared image Y component" respectively; 0.5 is the weight value.

[0038] According to a preferred embodiment of the present invention, the method for obtaining the target fused image in step S3 is as follows:

[0039] The fused image Img is assigned to the Y component of the target fused image, and the U and V components of the denoised infrared image are assigned to the U and V components of the target fused image respectively. The YUV components are then merged to obtain the target fused image.

[0040] According to a preferred embodiment of the present invention, in step S3, the method for calculating the quality index Qua is as follows:

[0041] The quality index Qua is an image quality evaluation index, mainly used to evaluate the quality of fused images, and is in numerical form;

[0042] Qua is a novel objective non-reference quality assessment metric for fused images.

[0043] The quality indicator Qua is Q. ab / f :

[0044] Get Q ab / f The algorithm uses local metrics to estimate the extent to which salient information from the input is represented in the fused image, Q. ab / f A higher value indicates better quality of the fused image. The calculation formula is as follows:

[0045]

[0046] In equation (4), M and N are the sizes of the target fused image; m and n are the image pixels; Q AF (n, m) and Q BF (n, m) represent the data representing the horizontal edge information retention and the data representing the vertical edge information retention, respectively; w A(n, m) and w B (n, m) is the edge intensity function of the target fused image.

[0047] According to a preferred embodiment of the present invention, the specific content of the optimization of the fixed value of the given interval range using the intelligent optimization algorithm in step S4 is as follows:

[0048] The fitness function is: taking the negative value of the sum of the sharpness Cla and the quality index Qua, in order to continuously reduce the fitness value and find the minimum value when optimizing a fixed value;

[0049] The intelligent optimization algorithm uses the golden sine algorithm to optimize the constant values ​​in the parameter optimization part of image fusion.

[0050] The advantages of the GoldSA algorithm are fast convergence speed, good robustness, ease of implementation, and low parameter adjustment and computational complexity. It primarily utilizes the relationship between the sine function and the unit circle in mathematics for iterative optimization. During the search position update process, GoldSA introduces the golden ratio to reduce the solution space, thereby searching for regions that may produce the optimal solution. This significantly improves search efficiency and achieves a good balance between "search" and "exploration."

[0051] The core of the GoldSA algorithm is to optimize position updates by randomly generating the positions of s individuals:

[0052]

[0053]

[0054] In equations (5) and (6), This represents the extreme value of the i-th individual (1, 2, 3, ..., s) in the k-th iteration of the D-dimensional individual space: Represents the flight speed in the k-th iteration;

[0055] In the (k+1)th iteration, the position of the i-th individual (1, 2, 3, ..., s) is updated as follows:

[0056]

[0057] In formula (7), This represents the extreme value of the i-th individual (1, 2, 3, ..., s) in the (k+1)-th iteration of the D-dimensional individual space; The flight speed represents the velocity in the k-th iteration: R1 is a random number between 0 and 2; R1 determines the distance moved in the next iteration. R2 is a random number between 0 and π; R2 determines the particle's flight speed in the next iteration. x1 and x2 are coefficients obtained through the golden ratio to narrow the search space and guide individuals closer to the optimal value; they are the golden ratio numbers.

[0058] x1=a+(1-τ)*b (8)

[0059] x2=a+τ*b (9)

[0060]

[0061] In equations (8), (9), and (10), a and b are the initial values ​​for searching the golden ratio: a = -τ and b = π; τ is the golden ratio.

[0062] According to a preferred embodiment of the present invention, the method for obtaining the final target fused image and converting it into RGB components in step S4 is as follows:

[0063] The target final fused image is the optimal fused image obtained by substituting the determined optimal values ​​into steps S2 and S3.

[0064] The conversion to RGB components refers to converting the fused image from YUV components to RGB components.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0066] This invention discloses a real-time image fusion method based on salient detail extraction and parameter optimization.

[0067] 1) A salient detail extraction method is proposed, which can accurately extract the texture and contour details of visible light images while avoiding excessive background interference and ensuring the visualization effect of the fused image.

[0068] 2) Using the sharpness of the detail image and the quality of the fused image as the fitness function, the intelligent optimization algorithm is used to optimize the fixed parameters, which can maximize the extraction of visible light detail information and ensure the image fusion effect. Attached Figure Description

[0069] Figure 1 This is a flowchart of a real-time image fusion method.

[0070] Figure 2 This is the original image of visible light.

[0071] Figure 3 yes Figure 2 The corresponding original infrared image.

[0072] Figure 4 yes Figure 2 The corresponding visible light salient detail image.

[0073] Figure 5 This is an image showing the effect of fusing visible light and infrared light in this invention. Detailed Implementation

[0074] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description, in conjunction with preferred embodiments, provides a detailed explanation of the specific implementation methods, structures, features, and effects of the present invention:

[0075] Example 1

[0076] like Figure 1 , Figure 2 As shown, a real-time image fusion method based on salient detail extraction and parameter optimization includes:

[0077] S1: Acquire the registered infrared and visible light images, process them, and convert them into YUV components;

[0078] The method for processing and converting YUV components in step S1 is as follows:

[0079] The visible light and infrared images are preprocessed to denoise, resulting in denoised visible light and infrared images. Channel separation is then performed on the denoised visible light and infrared images to obtain their respective Y, U, and V components.

[0080] The noise reduction method employs mean filtering.

[0081]

[0082] In equation (1), Img_denoised is the image after denoising; Img is the image before denoising. This is the convolution operation; blur is the mean filter template; ksize is the convolution kernel size.

[0083] The method for separating the channels of the denoised visible light image and infrared image to obtain their respective Y, U, and V components involves converting the denoised infrared image and visible light image from RGB components to YUV components.

[0084] Read the original visible light and infrared images. If they are in YUV format, no processing is needed; if they are in RGB format, convert RGB to YUV using the cvtColor function. Perform denoising preprocessing on the visible light and infrared images, using the mean filtering function blur to obtain the denoised visible light I_denoised and infrared image V_denoised. Perform channel separation on the denoised visible light V_denoised and infrared image I_denoised, and obtain their respective Y, U, and V components, V_Y, V_U, V_V, I_Y, I_U, and I_V, using the split function.

[0085] S2: Initialize the constant value, obtain the visible light salient detail map based on the constant value and the visible light Y component, and calculate the sharpness Cla of the salient detail map;

[0086] The method for initializing the setpoint as described in step S2, and obtaining a visible light salient detail image based on the setpoint and the infrared Y component:

[0087] The initialization value is a float constant with a range of [0, 255]. The initialization value is a value randomly assigned using a random number function, such as 56.

[0088] The method for obtaining a visible light salient detail map based on a constant value and the infrared Y component is as follows: subtract the Y component of the denoised visible light image from the constant value to obtain the visible light salient detail map.

[0089] Use the rand function to generate a random number, such as 56, assign it a given value, and then subtract the visible light Y component from the given value to obtain the visible light salient detail map. The formula is as follows:

[0090] V_marked=A-V_Y (11)

[0091] In equation (11), A is a constant, V_Y is the visible light Y component, V_marked is the visible light salient detail map, and finally the Cla value of the salient detail map is 0.5322 obtained by the energy gradient method.

[0092] S3: The visible light salient detail image and the infrared image Y component are fused together. The fusion result is used as the Y component, and the U and V components are the corresponding components of the infrared image to obtain the target fused image. The quality index Qua value of the target fused image is calculated.

[0093] The method described above for fusing visible light salient detail images with the Y component of infrared images:

[0094] The fusion process utilizes a pixel-weighted average method to fuse salient details and the infrared Y component.

[0095] The pixel-weighted average method is as follows:

[0096] Img = 0.5 * Img1 + 0.5 * Img2 (3)

[0097] In equation (3), Img is the fused image; Img1 is the image to be fused 1; Img2 is the image to be fused 2; Img1 and Img2 correspond to the "visible light salient detail map" and the "infrared image Y component" respectively; 0.5 is the weight value.

[0098] The method for obtaining the target fused image in step S3 is as follows:

[0099] The fused image Img is assigned to the Y component of the target fused image, and the U and V components of the denoised infrared image are assigned to the U and V components of the target fused image respectively. The YUV components are then merged to obtain the target fused image.

[0100] In step S3, the method for calculating the quality index Qua is as follows:

[0101] The quality index Qua is an image quality evaluation index, mainly used to evaluate the quality of fused images, and is in numerical form;

[0102] Qua is a novel objective non-reference quality assessment metric for fused images.

[0103] The quality indicator Qua is Q. ab / f ;

[0104] Get Q ab / f The algorithm uses local metrics to estimate the extent to which salient information from the input is represented in the fused image, Q. ab / f A higher value indicates better quality of the fused image. The calculation formula is as follows:

[0105]

[0106] In equation (4), M and N are the sizes of the target fused image; m and n are the image pixels; Q AF (n, m) and Q BF (n, m) represent the data representing the horizontal edge information retention and the data representing the vertical edge information retention, respectively; w A (n, m) and w B (n, m) is the edge intensity function of the target fused image.

[0107] The visible light salient detail map and the infrared image Y component are fused together, as shown below:

[0108] Img_fuse_Y=0.5*V_marked+0.5*I_Y (12)

[0109] In Equation (12), Img_fuse_Y is the fusion image of the visible light salient detail map and the Y component of the infrared image, and I_Y is the Y component of the infrared image. Img_fuse_Y is taken as the Y component of the target fused image, and I_U and I_V are the U and V components of the target fused image, respectively. The three components are merged by the merge function to obtain the target fused image, and the Qua value of the fused image is calculated to be 0.3685 by Qabf.

[0110] S4: Use an intelligent optimization algorithm to find the optimal value for a given range. The intelligent optimization algorithm includes a fitness function, which is the negative sum of Cla and Qua. Find the value with the lowest fitness and repeat steps S2 to S3 to obtain the final target fused image, and convert it into RGB components.

[0111] Example 2

[0112] As described in Example 1, a real-time image fusion method based on salient detail extraction and parameter optimization includes the method for calculating the sharpness Cla of the salient detail map in step S2:

[0113] The sharpness Cla is an image evaluation index, mainly used to assess the sharpness of detail information such as texture and outline in salient detail images, and is in numerical form;

[0114] The energy gradient method is used as the evaluation index for sharpness (Cla):

[0115] The sum of the squared differences in grayscale values ​​between adjacent pixels in the X and Y directions of the image is used as the gradient value of each pixel. The gradient values ​​of all pixels are accumulated to obtain the sharpness evaluation function value.

[0116] F = ∑ x ∑ Y {[f(x+1, y)-f(x, y)] 2 +[f(x, y+1)-f(x, y)] 2} (2)

[0117] In equation (2), F is the energy gradient value, i.e., the sharpness Cla; f is the image pixel gradient calculation function; x and y are the horizontal and vertical coordinates of the image pixel points, respectively;

[0118] For example, given a range of values, such as [150, 255], an intelligent optimization algorithm is used to find the fixed value when the energy gradient value is the maximum. However, for the intelligent optimization algorithm to run, an initial value needs to be given first, which can be randomly selected and will not affect the subsequent optimization process. In the end, the higher the energy gradient value, the clearer the image details and the more obvious the texture contours, which is more conducive to subsequent image fusion.

[0119] Example 3

[0120] As described in Example 1, a real-time image fusion method based on salient detail extraction and parameter optimization is provided.

[0121] The specific details of using the intelligent optimization algorithm to find the optimal value for a given interval range in step S4 are as follows:

[0122] The fitness function is: taking the negative value of the sum of the sharpness Cla and the quality index Qua, in order to continuously reduce the fitness value and find the minimum value when optimizing a fixed value;

[0123] The intelligent optimization algorithm uses the golden sine algorithm to optimize the constant values ​​in the parameter optimization part of image fusion.

[0124] The advantages of the GoldSA algorithm are fast convergence speed, good robustness, ease of implementation, and low parameter adjustment and computational complexity. It primarily utilizes the relationship between the sine function and the unit circle in mathematics for iterative optimization. During the search position update process, GoldSA introduces the golden ratio to reduce the solution space, thereby searching for regions that may produce the optimal solution. This significantly improves search efficiency and achieves a good balance between "search" and "exploration."

[0125] The core of the GoldSA algorithm is to optimize position updates by randomly generating the positions of s individuals:

[0126]

[0127]

[0128] In equations (5) and (6), This represents the extreme value of the i-th individual (1, 2, 3, ..., s) in the k-th iteration of the D-dimensional individual space: Represents the flight speed in the k-th iteration;

[0129] In the (k+1)th iteration, the position of the i-th individual (1, 2, 3, ..., s) is updated as follows:

[0130]

[0131] In formula (7), This represents the extreme value of the i-th individual (1, 2, 3, ..., s) in the (k+1)-th iteration of the D-dimensional individual space: The flight speed represents the k-th iteration: R1 is a random number between 0 and 2; R1 determines the distance moved in the next iteration; R2 is a random number between 0 and π; R2 determines the particle's flight speed in the next iteration; x1 and x2 are coefficients obtained through the golden ratio to narrow the search space and guide individuals closer to the optimal value, and are called the golden ratio numbers.

[0132] x1=a+(1-τ)*b (8)

[0133] x2=a+τ*b (9)

[0134]

[0135] In equations (8), (9), and (10), a and b are the initial values ​​for searching the golden ratio; a = -π and b = π; τ is the golden ratio.

[0136] The method described in step S4 for obtaining the final fused image of the target and converting it to RGB components:

[0137] The target final fused image is the optimal fused image obtained by substituting the determined optimal values ​​into steps S2 and S3.

[0138] The conversion to RGB components refers to converting the fused image from YUV components to RGB components.

[0139] In this embodiment, an intelligent optimization algorithm is used to optimize the fixed value. The fitness function is the negative sum of Cla and Qua. The fixed value with the lowest fitness is found. Steps S2 to S3 are repeated to obtain the final fused image of the target and convert it into RGB components.

[0140] Step 1: Based on the above fusion method, determine the objective function to be optimized, and select the following formula as the fitness objective function:

[0141] fitness=-(Cla+Qua) (13)

[0142] Step 2: Parameter settings: Set the population size of the GoldSA algorithm N = 100, the maximum number of iterations Max_iter = 30, the search range of the parameter to be optimized A [0, 255], and the algorithm termination condition is the maximum number of iterations; Set the initial values ​​a and b for the golden ratio search, a = -π, b = π: Calculate the initial golden ratio using equation (10);

[0143] Step 3: Update the golden ratio using equations (8) and (9), and update the individual position according to equation (7):

[0144] Step 4: Calculate the fitness values ​​of all individuals based on equation (13) and compare them with the fitness values ​​of the previous generation. If the current fitness value is better than the previous generation fitness value, then retain the current best individual position X. best Otherwise, retain the position X of the previous generation's best individual. best ;

[0145] Step 5: Determine if the iteration termination condition has been met. If so, output the final result, i.e., the optimal individual position X. best Otherwise, repeat steps 1-5.

[0146] Finally, the optimal individual position is obtained, i.e., the optimal value A = 200. Steps S2-S3 are repeated, resulting in a Cla value of 0.9147 and a Qua value of 0.6312. The final visible light salient detail image is shown below. Figure 4 As shown; target fused image, such as Figure 5 As shown;

[0147] Appendix Figure 4 The texture is clear and the details are significant, extracting the most significant details from the visible light image; by comparing with the original infrared image... Figure 3 Image fused with target Figure 5 It can be seen Figure 5 With more detailed textures and greater clarity, it effectively verifies that observers can more clearly understand thermal infrared images.

[0148] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A real-time image fusion method based on salient detail extraction and parameter optimization, characterized in that, The method comprises the following steps: S1: obtaining the completed registration of infrared images and visible light images, processing and converting into YUV components; S2: initializing a constant value, obtaining a visible light significant detail map based on the constant value and the visible light Y component, and calculating the definition Cla of the significant detail map; S3: fusing the visible light significant detail map and the infrared image Y component, taking the fusion result as the Y component, the U and V components being the corresponding components of the infrared image, to obtain a target fusion image, and calculating the quality index Qua of the target fusion image; S4: optimizing the constant value in a given interval range by using an intelligent optimization algorithm, the intelligent optimization algorithm comprising a fitness function, the fitness function being the negative value of the sum of Cla and Qua, finding the constant value when the fitness is lowest, repeating steps S2 to S3 to obtain a final target fusion image, and converting into RGB components; The method for initializing the constant value in step S2 and obtaining the visible light significant detail map based on the constant value and the infrared Y component: The initialized constant value is a float type constant; the initialized constant value is a random value given by using a random number function; The visible light significant detail map is obtained by subtracting the Y component of the denoised visible light image from the constant value; The intelligent optimization algorithm adopts the golden sine algorithm to optimize the constant value.

2. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 1, characterized in that, The method for processing and converting into YUV components in step S1 is as follows: The visible light image and the infrared image are preprocessed by denoising to obtain denoised visible light and infrared images; the denoised visible light image and the infrared image are separated into channels to obtain respective Y, U and V components.

3. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 2, characterized in that, The denoising method adopts the mean filter method: （1） In formula (1), is the denoised image; is the image before denoising; is a convolution operation; is a mean filter template; is a kernel size.

4. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 3, characterized in that, The method for separating the denoised visible light image and the infrared image into channels to obtain respective Y, U and V components: The denoised infrared image and the visible light image are converted from RGB components into YUV components.

5. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 1, characterized in that, The method for calculating the definition Cla of the significant detail map in step S2: The energy gradient method is adopted as the evaluation index of the definition Cla: The sum of squares of the differences in gray values of adjacent pixels in the X and Y directions of the image is taken as the gradient value of each pixel point, and the gradient values of all pixels are accumulated as the definition evaluation function value: （2） In formula (2), F is an energy gradient value, i.e., a definition of clarity Cla; f is an image pixel gradient calculation function; x , y are respectively horizontal and vertical coordinates of an image pixel.

6. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 1, characterized in that, The method for fusing the visible light significant detail map and the infrared image Y component in step S3: The fusion processing is performed by using the pixel average weighting method to fuse the significant detail and the infrared Y component; The pixel average weighting method is as follows: （3） In formula (3), Img is the fused image; Img 1 is the image to be fused 1; Img 2 is the image to be fused 2; Img 1 and Img 2 respectively correspond to "visible light salient detail map" and "infrared image Y component"; 0.5 is the weight value.

7. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 1, characterized in that, The method for obtaining the target fusion image in step S3 is as follows: The post-fusion image is obtained by fusing the infrared image and the visible image. Img The Y component of the target fusion image is given, and the U and V components of the infrared image after noise reduction are correspondingly given to the U and V components of the target fusion image. The YUV components are merged to obtain the target fusion image.

8. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 7, characterized in that, In step S3, the method for calculating the quality index Qua is as follows: The quality indicator Qua is ; The calculation formula is as follows: （4） In formula (4), M and N is the size of the target fusion image; m and n is the image pixel; and respectively represent data indicating edge information retention in the horizontal direction and data indicating edge information retention in the vertical direction; and is the edge intensity function of the target fusion image.

9. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 1, characterized in that, The specific content of the method for optimizing the constant value in a given interval range by using the intelligent optimization algorithm in step S4 is as follows: The golden sine algorithm randomly generates the positions of s individuals: （5） （6） in formula (5), formula (6), represents the kth iteration of the D-dimensional individual space, the extreme value of the individual; represents the flight speed of the kth iteration Then in the k+1 iteration, the individual position is updated as follows: the individual position is updated as follows: （7） In formula (7), In the (k+1)th iteration of the D-dimensional individual space, the... Individual extreme values; R represents the flight speed in the k-th iteration; R1 is a random number between 0 and 2; R2 is a random number between 0 and π; x 1 and x 2 is a coefficient obtained by golden section to reduce the search space to lead the individual to approach the optimal value, which is a golden section number: （8） （9） （10） In formula (8), (9), (10), a, b: golden section ratio search initial value; a=-π, b=π. Golden section ratio.

10. The real-time image fusion method based on salient detail extraction and parameter optimization according to claim 9, characterized in that, The method for obtaining the target final fusion image and converting into RGB components in step S4: The target final fusion image is the best fusion image obtained by substituting the determined optimal constant value into steps S2 and S3.

Images