Two-light image fusion methods, equipment and software products

Abstract

This invention discloses a dual-light image fusion method, device, and program product. The method includes: acquiring an infrared image and a visible light image of the same target scene; fusing the infrared image and the visible light image based on a first fusion parameter to obtain a first fused image; extracting a first high-frequency fused image based on the first fused image; fusing the infrared image and the visible light image based on a second fusion parameter to obtain a second fused image; and performing a fusion operation based on the first high-frequency fused image and the second fused image to obtain a target fused image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a dual-light image fusion method, computing device, and computer program product. Background Technology

[0002] Infrared and visible light images are captured by different imaging devices, each with its own distinct advantages and disadvantages. Visible light imaging's advantage lies in capturing information consistent with human vision, such as color, texture, and detail. Its disadvantage is its extreme susceptibility to lighting conditions: images become blurry or even fail in low light or adverse weather conditions such as at night, in heavy fog, or with dense smoke. It is also easily affected by strong light: direct sunlight can cause overexposure, obscuring details. Infrared imaging's advantage is based on the thermal radiation of objects, unaffected by visible light: object outlines remain clearly visible at night, in heavy fog, or with dense smoke; it is insensitive to strong light and does not suffer from overexposure. Its disadvantages include the inability to capture color and fine texture, resulting in images that are typically grayscale or pseudo-color with poor detail representation; and weaker ability to distinguish non-heat-generating objects (such as rocks or walls). Current technologies fuse the two types of images, but this is a simple one-time fusion, easily leading to the loss of detail and poor fusion quality. Summary of the Invention

[0003] To address the existing technical problems, this invention provides a dual-light image fusion method, computing device, and computer program product that can retain more high-frequency information and improve image fusion quality.

[0004] Firstly, a two-light image fusion method is provided, including:

[0005] Acquire infrared and visible light images of the same target scene;

[0006] Based on a first fusion parameter, the infrared image and the visible light image are fused to obtain a first fused image; based on the first fused image, a first high-frequency fused image is obtained; based on a second fusion parameter, the infrared image and the visible light image are fused to obtain a second fused image; based on the first high-frequency fused image and the second fused image, a fusion operation is performed to obtain a target fused image; or

[0007] The infrared image and the visible light image are fused to obtain a third fused image; a second high-frequency fused image is obtained based on the third fused image; a fusion operation is performed on the second high-frequency fused image and the infrared image to obtain a target fused image; or

[0008] Based on the visible light image, a visible light high-frequency image is obtained; the infrared image and the visible light image are fused to obtain a fourth fused image; based on the visible light high-frequency image and the fourth fused image, a fusion operation is performed to obtain a target fused image.

[0009] In a second aspect, a computing device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform a dual-light image fusion method as described in the first aspect of this application.

[0010] Thirdly, a computer program product is provided, comprising a computer program that, when executed, implements the dual-light image fusion method as described in the first aspect of this application.

[0011] This application obtains a high-frequency image based on a visible light image or an image fused with a visible light image and an infrared image. This high-frequency image can then be used to obtain the edges and textures in the image. The high-frequency image is then fused with an infrared image or with an image fused with a visible light image and an infrared image to obtain a target fused image. In this way, the target fused image retains the color information of visible light, so that the target fused image presents both natural colors and infrared thermal features, taking into account the human eye's color perception habits and the need for key target recognition. Attached Figure Description

[0012] Figure 1 This is an application environment diagram of a dual-light image fusion method in one embodiment;

[0013] Figure 2 This is a flowchart of a two-light image fusion method in one embodiment;

[0014] Figure 3 A comparison of the fusion effect after images were captured in a sample target scene are shown.

[0015] Figure 4 A comparison of the fusion effect after images were captured from the target scene in another example are fused together;

[0016] Figure 5 A comparison of the fusion effect after images were captured in another example of the target scene;

[0017] Figure 6 This is a flowchart of a two-light image fusion method in another embodiment;

[0018] Figure 7 This is a fusion effect diagram of the dual-light image fusion method in another embodiment;

[0019] Figure 8 This is a flowchart of a two-light image fusion method in another embodiment;

[0020] Figure 9 This is a fusion effect diagram of the dual-light image fusion method in another embodiment;

[0021] Figure 10 This is a schematic diagram of a dual-light image fusion device in one embodiment;

[0022] Figure 11 This is a schematic block diagram of a computing device provided in one embodiment. Detailed Implementation

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0025] In the following description, the expression “some embodiments” refers to a subset of all possible embodiments. However, it should be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0026] See Figure 1 This diagram illustrates the application environment of a dual-light fusion method in one embodiment. The environment includes an image acquisition device 12 and a computing device 10. The image acquisition device 12 acquires infrared and visible light images. The dual-light fusion method is applied in the computing device 10. The computing device 10 fuses the acquired infrared and visible light images to obtain a fused image. Figure 1 As shown in application scenario two, the image acquisition device 12 can exist independently of the computing device 10. As shown in application scenario one, the image acquisition device 12 can also be integrated into the computing device 10; that is, the computing device 10 can also include the image acquisition device 12, for example, the computing device 10 is an infrared imaging device. The image acquisition device 12 and the computing device 10 are communicatively connected, and the image acquisition device 12 can transmit data to the computing device 10.

[0027] The computing device 10 includes a processor 13 and a memory 14. The processor 13 is used to execute program instructions corresponding to the dual-light fusion method. The memory 14 is used to store the program and data corresponding to the implementation of the dual-light fusion method.

[0028] The computing device 10 includes, but is not limited to, handheld detection devices, non-handheld autonomously movable detection devices, and non-handheld, non-autonomous detection devices. Handheld detection devices include, but are not limited to, handheld imaging devices with infrared thermal imaging and / or visible light imaging capabilities. Non-handheld, autonomously movable detection devices include, but are not limited to, autonomously movable detection devices with infrared thermal imaging and / or visible light imaging capabilities. Non-handheld, non-autonomous detection devices include, but are not handheld and cannot be autonomously moved, devices with infrared thermal imaging and / or visible light imaging capabilities. The dual-light fusion method provided in this application embodiment can be applied to various complex motion-changing scenarios and fixed scenarios. The computing device 10 can be a server, computer equipment, or terminal equipment. The computing device 10 can be a device with video processing or image processing capabilities, an infrared imaging device, a monitoring device with image capture capabilities, an autonomous mobile device, etc. For example, autonomous mobile devices include, but are not limited to, vehicles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobile devices, airplanes, drones, ships, or robots, etc. The computing device 10 can also be fixedly installed on an electronic device on a certain device or at a fixed location in a fixed scene. Terminal devices include, but are not limited to, mobile phones, tablets, wearable electronic devices, etc.

[0029] The image acquisition device 12 includes at least a visible light image sensor and an infrared thermal imaging sensor. The visible light image sensor and infrared thermal imaging sensor in the image acquisition device 12 can also be combined with one or more sensors such as a millimeter-wave sensor, a lidar sensor, and a depth sensor. For example, the image acquisition device 12 is a handheld computing device or a gimbal to acquire clear image data. For example, the image acquisition device 12 is a dual-light camera.

[0030] The processor 13 can be one or more. When there are multiple processors 13, the multiple processors can be integrated on one chip or set independently on each chip.

[0031] The computing device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion attitude sensors. Environmental perception sensors include, but are not limited to, one or more of the following: brightness sensors, temperature sensors, suspended particulate matter sensors, etc. Motion attitude sensors include, but are not limited to, one or more of the following combinations: inertial measurement units (IMUs), velocity sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors, and gravity sensors, etc. The computing device 10 may use a high-resolution infrared thermal imager, such as a handheld or pan-tilt type, to acquire clear image data. It may also be equipped with ranging devices such as laser rangefinders or depth cameras for accurately measuring the physical distance between the acquisition device and the target. Similar to the image acquisition device 12, other sensor modules may not be included in the computing device 10 and may exist independently of it, but may be able to communicate with it.

[0032] The computing device 10 may also include a display terminal for displaying images.

[0033] Infrared and visible light images are captured by different imaging devices, each with its own distinct advantages and disadvantages. Visible light imaging's advantage lies in its ability to capture information consistent with human vision, such as color, texture, and detail. Its disadvantage is its extreme susceptibility to lighting conditions: images become blurry or even fail in low light or adverse weather conditions, such as at night, in heavy fog, or with dense smoke. It is also easily affected by strong light: direct sunlight can cause overexposure, making details invisible. Infrared imaging's advantage is based on the thermal radiation of objects, unaffected by visible light: object outlines remain clearly visible at night, in heavy fog, or with dense smoke; it is insensitive to strong light and has no overexposure issues. Its disadvantages include the inability to capture color and fine texture, resulting in images that are typically grayscale or pseudo-color with poor detail representation; and a weaker ability to distinguish non-heat-generating objects (such as rocks or walls).

[0034] Two-spectrum fusion aims to maximize strengths and minimize weaknesses. By fusing information from two different spectra, two-spectrum fusion can achieve a "1+1>2" effect:

[0035] Adaptability to complex environments: In scenarios such as nighttime and dense smoke, it retains the penetrability and anti-interference ability of infrared light while superimposing the details and colors of visible light, making the image clearer and the information more complete.

[0036] Robustness enhancement: Avoid information loss caused by the failure of a single spectrum (e.g., when visible light fails at night, infrared light serves as a supplement; when infrared light is not sensitive to cold targets, visible light provides details).

[0037] Target recognition accuracy: For example, in security monitoring, infrared light can quickly locate the outline of a human body in the dark, while visible light can identify clothing color and facial features. Combining these features can more accurately identify targets.

[0038] To effectively address the limitations in complex environments and improve the comprehensiveness and reliability of scene perception, this patent proposes a novel target fusion method. This method overcomes the limitations of traditional dual-light fusion technology in terms of scene dependence, eliminating the need to redesign or adjust fusion algorithm parameters for specific scenes (such as nighttime, dense smoke, strong light, etc.), and can robustly output fused images with both reliability and integrity in various complex environments.

[0039] Please see Figure 2 This is a flowchart illustrating a two-light image fusion method according to an embodiment of this application. The two-light image fusion method is applied in a computing device and includes the following steps:

[0040] S10. Acquire infrared and visible light images of the same target scene.

[0041] Since infrared and visible light images are captured by different image sensors, dual-light registration is performed first to ensure a good fusion result. Image registration methods can include hardware calibration to achieve optical axis alignment between the two lights, or registration based on image registration algorithms. Image registration algorithms include feature point-based registration, grayscale-based registration, deep learning-based registration, and offline registration and online correction based on calibration boards. The core is to accurately align the visible light image and the infrared light image in space, ensuring that the position, scale, and angle of the same target are consistent in both images, and that the resolution of the two-light images is consistent after dual-light registration.

[0042] In this embodiment, after acquiring the original infrared and visible light images using the image acquisition device, image registration can be performed first. Specifically, the original infrared image can be converted to the coordinates of the original visible light image, or vice versa, thereby obtaining the registered infrared and visible light images. When there is a deviation in image registration, compensation can be performed based on the configured registration compensation value to obtain more accurately registered infrared and visible light images.

[0043] S11. Obtain the first fusion parameter, and based on the first fusion parameter, fuse the infrared image and the visible light image to obtain the first fused image.

[0044] In this embodiment, the first fusion parameters include, but are not limited to, a first infrared weight and a first visible light weight. The first infrared weight is used to adjust the contribution weight of the infrared image during fusion. The first visible light weight is used to adjust the contribution weight of the visible light image during fusion. In this fusion process, the main goal is to increase the contribution weight of the visible light image, thus providing more color information, edge information, texture information, etc. The first fusion parameters can be stored as a parameter set, and can be determined based on experimental test results. These first fusion parameters are primarily designed to control the generation of the richest high-frequency information in various complex environments, addressing the shortcomings of individual spectra.

[0045] S12. Based on the first fused image, a first high-frequency fused image is obtained.

[0046] In this embodiment, the first high-frequency fused image refers to the high-frequency information in the first fused image. The first fused image includes low-frequency information and high-frequency information. The slowly changing portions of the low-frequency information image are large areas with smooth transitions in color and brightness. Low-frequency information includes, but is not limited to, the general outline of objects, smooth backgrounds (such as a wall, sky, or blurred backgrounds), and the overall average gray level or base brightness of the image. Only low-frequency information images will appear blurry and hazy, showing only general color blocks and shapes, losing all details. High-frequency information represents rapidly changing portions of the image, where intensity and color suddenly and drastically jump. High-frequency information includes, but is not limited to, edges (object boundaries, boundaries between different areas), textures (wood grain, textile fibers, hair strands), details (wrinkles on a person, veins in leaves, text on paper), and noise (noise and imperfections in the image). High-frequency information mainly consists of the superposition of edges and textures in the image.

[0047] S13. Based on the second fusion parameter, the infrared image and the visible light image are fused to obtain the second fused image.

[0048] In this embodiment, the second fusion parameters include a second infrared weight and a second visible light weight. The second infrared weight is used to adjust the contribution weight of the infrared image during fusion. The second visible light weight is used to adjust the contribution weight of the visible light image during fusion. In this fusion process, to further increase the contribution weight of the infrared image, high-frequency features are further compensated using infrared features. In harsh environments, such as rainy days with low visibility, the contribution weight of the infrared image can be increased in the second fused image to enhance more high-frequency information. The infrared and visible light images are then combined again to generate the second fused image. The second fusion parameters are stored as a parameter group. The second fusion parameters are also determined based on experimental test results. The first fusion parameter focuses on obtaining high-frequency information from the first fused image, while the second fusion parameter ensures that color information is retained while supplementing high frequencies. The second fusion parameter is mainly set for the display effect of the fused image; the retained color information makes the fusion result more consistent with human visual perception. Based on the first fusion parameter, the obtained high-frequency information is closer to the high-frequency information of visible light. If the quality of visible light is not high, the second fusion parameter can further compensate for the high-frequency information because the second fusion parameter is closer to the infrared high-frequency information. At the same time, the second fused image also retains the color information of visible light.

[0049] S14. Based on the first high-frequency fused image and the second fused image, perform a fusion operation to obtain the target fused image.

[0050] In this embodiment, since the first high-frequency fused image indicates high-frequency information, namely the high-frequency information in the visible light image and the infrared image, when performing the fusion operation, the first high-frequency fused image is superimposed on the second fused image, or the second fused image is superimposed on the first high-frequency fused image, so that the final fused image can fully retain both high-frequency information and color information.

[0051] like Figures 3-5 As shown, Figure 3 A comparison of the fusion effect after images were captured from a target scene in an example. Figure 4 A comparison of the fusion effect after images were captured in the target scene of another example were fused. Figure 5 This is a comparison image showing the fusion effect after images were captured in another example of a target scene. From Figure 4 It can be seen that the target fused image has richer high-frequency information and is clearer than the second fused image and the first high-frequency fused image. Therefore, the image obtained after superimposing and fusing the high-frequency layer with the second fused image is clearer. The final fused image is shown after performing pseudo-color mapping on the target fused image, and colors with visible light can be seen. Figure 5As shown, this is an image captured from a target scene. When visible light glare causes poor image quality, the high-frequency information extracted directly from the visible light image is almost nonexistent, while the high-frequency information obtained from the first fused image is increased. Therefore, more high-frequency information can be obtained from the first fused image.

[0052] In the above embodiments, according to the first fusion parameter, a fusion operation is performed on the registered infrared image and the registered visible light image to generate a first fused image. Based on the first fused image, a first high-frequency fused image is obtained. Because the first fused image retains both the high-frequency information of visible light and the high-frequency information of infrared light, it can overcome the problem that high-frequency information cannot be extracted when the image quality is low in certain scenarios in single light. Therefore, the dual-light fusion method provided in this application embodiment can achieve scene adaptation. According to the second fusion parameter, a fusion operation is performed on the registered infrared image and the registered visible light image to generate a second fused image. The first high-frequency fused image includes more high-frequency information. The first high-frequency fused image and the second fused image are fused to obtain a target fused image. In this way, the target fused image retains the color information of visible light, so that the target fused image presents both natural colors and infrared thermal features, taking into account the human eye's perception habits of color and the need for identification of key targets.

[0053] In some embodiments, the first fusion parameter includes a first infrared weight and a first visible light weight, and the step of fusing the infrared image and the visible light image based on the first fusion parameter to obtain a first fused image includes:

[0054] Obtain the first infrared weight and the first visible light weight;

[0055] Based on the first infrared weight and the first visible light weight, the infrared image and the visible light image are fused to obtain a first fused image, wherein the first infrared weight is less than the first visible light weight.

[0056] In this embodiment, during the first fusion process, the first visible light weight is greater than the first infrared weight. This increases the contribution of visible light features, providing more color, edge, and texture information. Based on the first infrared and first visible light weights, an image fusion algorithm is used to fuse the infrared image and the visible light image to obtain a first fused image. The image fusion algorithm includes, but is not limited to, pixel-level fusion and multi-scale decomposition-based fusion. For example, multiplying the first infrared weight by the infrared image and then multiplying the first visible light weight by the visible light image yields the first fused image. The first infrared and first visible light weights can be pre-configured.

[0057] Optionally, obtaining the first infrared weight and the first visible light weight includes:

[0058] The image quality of the visible light image is evaluated to obtain a visible light image quality value.

[0059] Based on the visible light image quality value, a first visible light weight corresponding to the visible light image quality value is determined;

[0060] The first infrared weight is calculated based on the first visible light weight corresponding to the visible light image quality value.

[0061] In this embodiment, the first visible light weight is based on a preset visible light weight. With visible light weighted dynamic parameters The calculated value uses a preset visible light weight. With visible light weighted dynamic parameters This combined approach involves pre-configuring a preset visible light weight, which serves as a pre-defined parameter, and then dynamically fine-tuning the first visible light weight based on the specific visible light image quality. The dynamic parameter for the visible light weight is determined based on the visible light image quality value. For example, the visible light image quality value can be evaluated using the standard deviation of the grayscale values ​​of the visible light image. Specifically:

[0062]

[0063]

[0064]

[0065] in This represents the first fused image. Represents a visible light image. Represents an infrared image. Indicates the first visible light weight. Indicates the first infrared weight. , These represent the first preset visible light image quality threshold and the second preset visible light image quality threshold, respectively. This represents the first visible light weighted dynamic parameter value. This indicates the visible light image quality value. This represents the value of the second visible light weighting dynamic parameter. This represents the preset parameters, where Less than Therefore, the higher the image quality indicated by the visible light image quality range, the larger the corresponding visible light weight dynamic parameter value. In this way, the visible light image can provide more visible light feature information during the first fusion process.

[0066] In the above embodiments, during the first fusion process, the first visible light weight is greater than the first infrared weight. This increases the contribution of visible light features, providing more color information, edge information, texture, detail information, etc., thus enriching the high-frequency information of the fused image and improving the quality of the fused image.

[0067] In some embodiments, obtaining the first high-frequency fused image based on the first fused image includes at least one of the following:

[0068] The high-frequency image, including high-frequency information, is extracted from the first fused image using a high-pass filter to obtain the first high-frequency fused image; or...

[0069] The low-frequency image containing low-frequency information in the first fused image is extracted using a low-pass filter. The first high-frequency fused image is obtained by subtracting the low-frequency image from the first fused image.

[0070] In this embodiment, a high-pass filter is a key tool for extracting details and edges in image processing. A high-pass filter allows high-frequency signals (rapidly changing parts, such as edges, textures, noise, and details) to pass through, while suppressing or blocking low-frequency signals (slowly changing parts, such as smooth color regions). Inputting the first fused image into the high-pass filter filters out low-frequency components and extracts the high-frequency image, resulting in a first high-frequency fused image. A low-pass filter allows low-frequency signals in the image to pass through, while suppressing or attenuating high-frequency signals. Inputting the first fused image into the low-pass filter filters out high-frequency signals, resulting in a low-frequency image. Subtracting the low-frequency image from the first fused image yields high-frequency information again, thus obtaining the first high-frequency fused image.

[0071] A first high-frequency fused image is generated based on the generated first fused image. This first high-frequency fused image can be obtained directly by extracting high-frequency information from the first fused image, or it can be obtained by first extracting low-frequency information from the first fused image and then subtracting it from the first fused image. Using a low-pass filter to obtain low-frequency information and then subtracting it from the first fused image yields the first high-frequency fused image. Therefore, both methods—directly obtaining high-frequency information and subtracting low-frequency information from the first fused image—achieve the same goal. Preferably, the indirect method of subtracting low-frequency information from the first fused image is chosen because directly extracting high-frequency information retains the high-frequency information but also amplifies high-frequency noise, while subtracting low-frequency information from the first fused image retains high-frequency information while suppressing high-frequency noise, resulting in a better visual effect. Obtaining the first high-frequency fused image from the first fused image can effectively overcome the limitations of a single spectrum in complex environments, providing rich high-frequency information even in harsh environments such as nighttime, glare, and dense smoke.

[0072] In the above embodiments, the first high-frequency fused image can be obtained from the first fused image in different ways. This is because the fused image retains both the high-frequency information of visible light and the high-frequency information of infrared light. This can overcome the problem that high-frequency information cannot be extracted when the image quality is low in certain scenarios in single light. Obtaining high-frequency information from the first fused image can obtain more high-frequency information from the image than obtaining high-frequency information from only the visible light image, which is conducive to improving the subsequent fusion quality.

[0073] In some embodiments, the second fusion parameter includes a second infrared weight and a second visible light weight, and the infrared image and the visible light image are fused based on the second fusion parameter to obtain a second fused image.

[0074] Obtain the second infrared weight and the second visible light weight;

[0075] Based on the second infrared weight and the second visible light weight, the infrared image and the visible light image are fused to obtain a second fused image, wherein the second infrared weight is greater than the second visible light weight.

[0076] In this embodiment, during the second fusion process, the second visible light weight is less than the second infrared weight. This is because while the first fusion parameters retain high-frequency information from both lights, the visible light weight is larger. However, the infrared weight is even greater in the second fusion process. Therefore, the two fusion processes complement each other, preserving the color information of the visible light while ensuring the fused image presents both natural colors and infrared thermal features, thus balancing human color perception habits and the need for identifying key targets. Based on the second infrared and second visible light weights, an image fusion algorithm is used to fuse the infrared and visible light images to obtain the second fused image. This image fusion algorithm includes, but is not limited to, pixel-level fusion and multi-scale decomposition-based fusion. For example, multiplying the second infrared weight by the infrared image and then multiplying the second visible light weight by the visible light image yields the second fused image. The second infrared and second visible light weights can be pre-configured.

[0077] Optionally, obtaining the second infrared weight and the second visible light weight includes:

[0078] The image quality of the infrared image is evaluated to obtain an infrared image quality value.

[0079] Based on the infrared image quality value, a second infrared weight corresponding to the infrared image quality value is determined;

[0080] The second visible light weight is calculated based on the second infrared weight corresponding to the infrared image quality value.

[0081] In this embodiment, the second infrared weight is based on the infrared base weight. With infrared dynamic weights The calculated value also uses the infrared base weights. With infrared dynamic weights The calculation combines two methods: first, a preset base infrared weight is established, and then the dynamic infrared weight is fine-tuned based on the specific infrared image quality value. For example, the infrared image quality value is the standard deviation of the infrared image grayscale values. As a criterion for evaluation.

[0082]

[0083]

[0084]

[0085] in Including the second fused image, Represents an infrared image. Represents a visible light image. Indicates the second infrared weight. Indicates the second visible light weight. Indicates the first infrared dynamic weight. This represents the second infrared dynamic weight. This represents the preset value. This indicates the quality value of the infrared image. This indicates the first preset infrared image quality value. This represents the second preset infrared image quality value, where Greater than When the image quality indicated by the infrared image quality range is higher, the corresponding infrared dynamic weight is greater. This can increase the infrared features and provide more high-frequency infrared information during the second fusion process.

[0086] In the above embodiments, during the second fusion process, the second infrared weight is greater than the first visible light weight. Although the first fusion process retains the high-frequency information of both lights, the visible light weight is relatively large. However, the infrared weight is large in the second fusion process. The infrared image provides high-frequency information supplement to the first fusion process, while retaining the color information of the visible light. This allows the fused image to present both natural colors and infrared thermal features, taking into account both the human eye's color perception habits and the need for key target recognition.

[0087] In some embodiments, the first high-frequency fused image may be further enhanced before being fused with the second fused image. Then, the step of performing a fusion operation based on the first high-frequency fused image and the second fused image to obtain the target fused image includes:

[0088] Obtain the first enhancement coefficient;

[0089] The first enhancement coefficient is multiplied by the pixel value of each pixel in the first high-frequency fusion image to obtain the first enhanced high-frequency image;

[0090] An addition operation is performed on the first enhanced high-frequency image and the second fused image to obtain the target fused image.

[0091] In this embodiment, the first enhancement coefficient is used to enhance the high-frequency signals in the first high-frequency fused image, making the high-frequency information more prominent. The first enhancement coefficient, a numerical value, is directly multiplied onto each pixel in the first high-frequency fused image, improving the comprehensiveness and reliability of scene perception. During the fusion operation, the first enhanced high-frequency image is superimposed on the second fused image, ensuring that the final fused image retains both high-frequency and color information sufficiently.

[0092] Optionally, the acquisition of enhancement coefficients includes at least one of the following:

[0093] Based on the enhancement coefficient control provided by the user interface, obtain the configured first enhancement coefficient;

[0094] The image quality of the first high-frequency fused image is evaluated to obtain the high-frequency image quality value of the first high-frequency fused image. Based on the high-frequency image quality value of the first high-frequency fused image, the enhancement coefficient corresponding to the high-frequency image quality value of the first high-frequency fused image is determined.

[0095] In this embodiment, an enhancement coefficient control can be configured on the user interface. For example, the enhancement coefficient control can be an input box, a radio button, etc. The required enhancement coefficient is input through the enhancement coefficient control. The high-frequency image quality value of the first high-frequency fused image can be evaluated based on gradient distribution. The image quality of the first high-frequency fused image can be automatically identified. The gradient distribution of edge pixels in the first high-frequency fused image can be statistically analyzed, and the high-frequency image quality value of the first high-frequency fused image can be calculated based on the gradient distribution. The high-frequency image quality value range to which the high-frequency image quality value of the first high-frequency fused image belongs can be determined based on the high-frequency image quality value of the first high-frequency fused image. The enhancement coefficient corresponding to the high-frequency image quality value range of the first high-frequency fused image is used as the enhancement coefficient corresponding to the high-frequency image quality value of the first high-frequency fused image. The high-frequency image quality value of the first high-frequency fused image includes, but is not limited to, the gradient mean and gradient standard deviation of edge pixels.

[0096] Multiple high-frequency image quality value ranges can be pre-configured, each corresponding to a different enhancement coefficient. The smaller the high-frequency image quality value indicated by the range, the worse the image quality, requiring the addition of high-frequency information and thus an enhancement coefficient. The larger the enhancement coefficient, the more dynamically and segmentally the enhancement coefficient can be adjusted according to the image quality. Represents a high-frequency fused image. This indicates enhancement of high-frequency images, where K1, K2, and K3 are preset parameters, and thr1, thr2, and thr3 are preset thresholds. This indicates the quality value of a high-frequency image. These represent the x-coordinate and y-coordinate of a pixel, respectively. Where K3 < K2 < K1.

[0097]

[0098] For example, the enhancement coefficient is 5 for a high-frequency image quality value range of (0,5], 2 for a high-frequency image quality value range of (5,10], and 1 for a high-frequency image quality value range of (10,15], and so on.

[0099] In the above embodiments, when the image quality is poor and the high-frequency information is not prominent, the high-frequency information in the first high-frequency fused image can be enhanced by the enhancement coefficient, thereby highlighting the high-frequency information and improving the image fusion quality.

[0100] Please see Figure 6 This is a flowchart of a two-light image fusion method according to another embodiment of this application. A two-light image fusion method is applied in a computing device, and the method includes the following steps:

[0101] S60. Acquire infrared and visible light images of the same target scene.

[0102] In this embodiment, this step is the same as S10, and will not be described again here.

[0103] S61. The infrared image and the visible light image are fused to obtain a third fused image.

[0104] In this embodiment, the fusion process in this step can be the same as the fusion process in step S11 or step S13, or it can use different infrared weights and visible light weights than the above two steps to perform dual-light fusion and obtain a third fused image.

[0105] S62. Based on the third fused image, a second high-frequency fused image is obtained.

[0106] In this embodiment, this step is similar to the process of obtaining a first high-frequency fused image based on the first fused image. Optionally, obtaining a second high-frequency fused image based on the third fused image includes at least one of the following:

[0107] The high-frequency image including high-frequency information in the third fused image is extracted using a high-pass filter to obtain the second high-frequency fused image; or, the low-frequency image including low-frequency information in the third fused image is extracted using a low-pass filter, and the difference between the third fused image and the low-frequency image in the third fused image is obtained to obtain the second high-frequency fused image.

[0108] A second high-frequency fused image is generated based on the generated third fused image. This second high-frequency fused image can be obtained directly by extracting high-frequency information from the third fused image, or by first extracting low-frequency information from the third fused image and then subtracting it from the third fused image. Using a low-pass filter to obtain low-frequency information and then subtracting it from the third fused image yields the second high-frequency fused image. Therefore, both methods—directly obtaining the high-frequency information from the third fused image and subtracting the low-frequency information from the third fused image—achieve the same goal. Preferably, the indirect method of subtracting the low-frequency information from the third fused image is chosen because directly extracting high-frequency information retains the high-frequency information but also amplifies high-frequency noise, while subtracting the low-frequency information from the third fused image retains high-frequency information while suppressing high-frequency noise, resulting in better visual effects. Obtaining the second high-frequency fused image from the third fused image effectively overcomes the limitations of a single spectrum in complex environments, providing rich high-frequency information even in harsh environments such as nighttime, glare, and dense smoke.

[0109] S63. Based on the second high-frequency fused image and the infrared image, perform a fusion operation to obtain the target fused image.

[0110] In this embodiment, since the second high-frequency fused image indicates high-frequency information, namely the high-frequency information in the visible light image and the infrared image, during the fusion operation, the second high-frequency fused image is superimposed on the infrared image, or the infrared image is superimposed on the second high-frequency fused image. This allows the final fused image to fully retain both high-frequency information and sufficient color and contour information. Figure 7 As shown, Figure 7 This is a fusion effect diagram of another two-light image fusion method.

[0111] Optionally, the step of performing a fusion operation based on the second high-frequency fused image and the infrared image to obtain the target fused image includes:

[0112] Obtain the second enhancement coefficient, multiply the second enhancement coefficient by the pixel value of each pixel in the second high-frequency fusion image to obtain the second enhanced high-frequency image, and perform an addition operation on the second enhanced high-frequency image and the infrared image to obtain the target fusion image.

[0113] In this embodiment, the second enhancement coefficient is used to enhance the high-frequency signals in the second high-frequency fused image, making the high-frequency information more prominent. The second enhancement coefficient, a numerical value, is directly multiplied onto each pixel in the second high-frequency fused image, improving the comprehensiveness and reliability of scene perception. During the fusion operation, the second enhanced high-frequency image is superimposed on the infrared image, ensuring that the final fused image retains both high-frequency and color information sufficiently.

[0114] Optionally, obtaining the second enhancement coefficient includes at least one of the following:

[0115] Based on the enhancement coefficient control provided in the user interface, obtain the configured second enhancement coefficient;

[0116] The image quality of the second high-frequency fused image is evaluated to obtain the high-frequency image quality value of the second high-frequency fused image. Based on the high-frequency image quality value of the second high-frequency fused image, the enhancement coefficient corresponding to the high-frequency image quality value of the second high-frequency fused image is determined.

[0117] In this embodiment, an enhancement coefficient control can be configured on the user interface. For example, the enhancement coefficient control can be an input box, radio button, etc. The required enhancement coefficient is input through the enhancement coefficient control. The high-frequency image quality value can be evaluated based on gradient distribution. The image quality of the second high-frequency fused image can be automatically identified. The gradient distribution of edge pixels in the second high-frequency fused image can be statistically analyzed, and the high-frequency image quality value of the second high-frequency fused image can be calculated based on the gradient distribution. The high-frequency image quality value range to which the high-frequency image quality value of the second high-frequency fused image belongs can be determined based on the high-frequency image quality value of the second high-frequency fused image. The enhancement coefficient corresponding to the high-frequency image quality value range of the second high-frequency fused image is used as the enhancement coefficient corresponding to the high-frequency image quality value of the second high-frequency fused image. The high-frequency image quality value of the second high-frequency fused image includes, but is not limited to, the gradient mean and gradient standard deviation of edge pixels. Multiple high-frequency image quality value ranges can be pre-configured, each corresponding to a different enhancement coefficient. This configuration method is the same as in the above embodiment and will not be repeated here.

[0118] In the above embodiments, a dual-light fusion operation is performed based on the infrared image and the visible light image to generate a third fused image. A second high-frequency fused image is then obtained based on the third fused image. Because the third fused image retains both the high-frequency information of visible light and infrared light, it overcomes the problem of not being able to extract high-frequency information in certain scenarios where the image quality is low under single-light conditions. Therefore, the dual-light fusion method provided in this application can achieve scene adaptation. The first high-frequency fused image includes more high-frequency information. The first high-frequency fused image is fused with the infrared image to obtain the target fused image. This target fused image simultaneously retains the color information of visible light, allowing it to present both natural colors and infrared thermal features, taking into account both human visual color perception habits and the need for identifying key targets. The third fused image already contains the texture of visible light and the contour information of infrared light. Therefore, the second high-frequency fused image obtained from it is an enhanced and richer set of high-frequency information. It contains both the fine texture of visible light and the contour details revealed by infrared light under harsh conditions that are invisible to visible light. Obtaining high-frequency information from the fused image is equivalent to using the stability of infrared information to filter the high-frequency information, resulting in a high-frequency image with a higher signal-to-noise ratio and a greater focus on key structural information in the real scene. In the secondary fusion process, the second high-frequency fused image is then fused with the infrared image. By injecting the second high-frequency fused image obtained from the fused image, the visual information content of the infrared image is greatly enriched, transforming it from a smooth, blocky hotspot into a more textured and easily identifiable target.

[0119] Please see Figure 8 This is a flowchart of a two-light image fusion method in another embodiment of this application. A two-light image fusion method is applied in a computing device, and the method includes the following steps:

[0120] S80. Acquire infrared and visible light images of the same target scene.

[0121] In this embodiment, this step is the same as S10, and will not be described again here.

[0122] S81. Based on the visible light image, obtain the visible light high-frequency image.

[0123] Optionally, obtaining the visible light high-frequency image from the visible light image includes:

[0124] The visible light high-frequency image is obtained by extracting a high-frequency image containing high-frequency information from the visible light image using a high-pass filter; or, the visible light high-frequency image is obtained by extracting a low-frequency image containing low-frequency information from the visible light image using a low-pass filter and subtracting the visible light image from the low-frequency image in the visible light image.

[0125] Generating a high-frequency visible light image from a visible light image can be achieved either by directly extracting high-frequency information from the visible light image or by first extracting low-frequency information from the visible light image and then subtracting it from the visible light image. Using a low-pass filter to obtain low-frequency information and then subtracting it from the visible light image yields the high-frequency image. Therefore, both methods—directly obtaining high-frequency information and subtracting low-frequency information from the visible light image—achieve the same goal. Preferably, the indirect method of subtracting low-frequency information from the visible light image is chosen because directly extracting high-frequency information retains the high-frequency information but also amplifies high-frequency noise, while subtracting low-frequency information from the visible light image retains high-frequency information while suppressing high-frequency noise, resulting in a better visual effect.

[0126] S82. The infrared image and the visible light image are fused to obtain the fourth fused image.

[0127] In this embodiment, the fusion process in this step can be the same as the fusion process in step S11 or step S13, or it can use different infrared weights and visible light weights than the above two steps to perform dual-light fusion and obtain a fourth fused image.

[0128] S83. Based on the visible light high-frequency image and the fourth fused image, perform a fusion operation to obtain the target fused image.

[0129] In this embodiment, since the visible light high-frequency image indicates high-frequency information, i.e., the high-frequency information in the visible light image, when performing the fusion operation, the visible light high-frequency image is superimposed on the fourth fused image, or the fourth fused image is superimposed on the visible light high-frequency image. This allows the final fused image to fully retain both high-frequency information and sufficient color and contour information. Figure 9 As shown, Figure 9 This is a fusion effect diagram of another two-light image fusion method.

[0130] The step of performing a fusion operation based on the visible light high-frequency image and the fourth fused image to obtain the target fused image includes:

[0131] A third enhancement coefficient is obtained, and the third enhancement coefficient is multiplied by the pixel value of each pixel in the visible light high-frequency image to obtain a third enhanced high-frequency image. An addition operation is performed on the third enhanced high-frequency image and the fourth fused image to obtain the target fused image.

[0132] In this embodiment, the third enhancement coefficient is used to enhance the high-frequency signals in the visible light high-frequency image, making the high-frequency information more prominent. The third enhancement coefficient, a numerical value, is directly multiplied onto each pixel in the visible light high-frequency image, improving the comprehensiveness and reliability of scene perception. During the fusion operation, the visible light high-frequency image is superimposed on the fourth fused image, ensuring that the final fused image retains both high-frequency and color information sufficiently.

[0133] Optionally, obtaining the third enhancement coefficient includes at least one of the following:

[0134] Based on the enhancement coefficient control provided by the user interface, obtain the configured third enhancement coefficient;

[0135] The image quality of the visible light high-frequency image is evaluated to obtain the high-frequency image quality value of the visible light high-frequency image. Based on the high-frequency image quality value of the visible light high-frequency image, an enhancement coefficient corresponding to the high-frequency image quality value of the visible light high-frequency image is determined.

[0136] In this embodiment, an enhancement coefficient control can be configured on the user interface. For example, the enhancement coefficient control can be an input box, radio button, etc. The required enhancement coefficient is input through the enhancement coefficient control. High-frequency image quality values ​​can be evaluated based on gradient distribution. The image quality of visible light high-frequency images can be automatically identified. The gradient distribution of edge pixels in the visible light high-frequency image can be statistically analyzed, and the high-frequency image quality value of the visible light high-frequency image can be calculated based on the gradient distribution. The high-frequency image quality value range to which the visible light high-frequency image quality value belongs can be determined based on the high-frequency image quality value. The enhancement coefficient corresponding to the high-frequency image quality value range of the visible light high-frequency image is used as the enhancement coefficient corresponding to the high-frequency image quality value of the visible light high-frequency image. The high-frequency image quality value of the visible light high-frequency image includes, but is not limited to, the gradient mean and gradient standard deviation of edge pixels. Multiple high-frequency image quality value ranges can be pre-configured, each corresponding to a different enhancement coefficient. This configuration method is the same as in the above embodiment and will not be repeated here.

[0137] In the above embodiments, a high-frequency visible light image is obtained based on the visible light image. Since the high-frequency visible light image retains the high-frequency information of visible light and includes more high-frequency information, the visible light image is fused with the infrared image to obtain a fourth fused image. The high-frequency visible light image is then fused with the fourth fused image. This overcomes the problem of not being able to extract high-frequency information in certain scenarios where the image quality is low in single-light conditions. Therefore, the dual-light fusion method provided in this application can achieve scene adaptation. The target fused image simultaneously retains the color information of visible light, allowing it to present both natural colors and infrared thermal features, taking into account both human visual perception habits of color and the need for key target identification. The operation of obtaining a high-frequency visible light image based on the visible light image actively searches for any potentially useful parts in the visible light image, obtaining edge or texture information. In the subsequent re-fusion process, this information is combined with the infrared thermal contour to generate an image that has both thermal features and some high-frequency visible light information. This balances the dynamic range of the entire image, and the final image retains the color information of visible light, which conforms to the long-term observation habits of the human eye, enabling the observer to quickly understand the scene. Infrared thermal signatures are cleverly integrated, allowing key targets with temperature differences from the background (such as lurking personnel, overheated equipment, and nocturnal animals) to be clearly highlighted. Color aids in localization and classification, while thermal signatures assist in identification and alerting. This significantly improves situational awareness and decision-making efficiency.

[0138] In some embodiments, the method further includes:

[0139] Acquire the initial infrared image and initial visible light image for initial registration;

[0140] Obtain the registration compensation value;

[0141] Based on the registration compensation value, an offset operation is performed on the initial infrared image and / or the initial visible light image to obtain the registered infrared image and visible light image.

[0142] In this embodiment, the initial infrared image and the initial visible light image are images that have undergone initial registration using an image registration method. However, since the image registration method may have registration errors, a registration compensation value control can be provided on the user interface, allowing the user to input a registration compensation value. The registration compensation value controls the pixel offset of the initial infrared image or the initial visible light image. Based on the registration compensation value, the pixels of the initial infrared image or the initial visible light image are moved, thereby aligning the initial infrared image and the initial visible light image to obtain the registered infrared image and visible light image.

[0143] In the above embodiments, by configuring registration compensation values, the initial infrared image and the initial visible light image of the initial registration are aligned and adjusted, thereby obtaining infrared and visible light images with higher registration accuracy, which facilitates subsequent improvement of image fusion quality.

[0144] In some embodiments, the method further includes:

[0145] Obtain the configured mapping relationship between pixel values ​​and color values;

[0146] Based on the mapping relationship, the target fused image is colored to obtain a colored image, and the colored image is displayed.

[0147] The mapping relationship can be represented as a color palette. For example, such as... Figure 4 The last image in the series shows the image after pseudo-coloring of the target fused image. Pseudo-color mapping of the target fused image is performed according to a custom color palette, which preserves the high-frequency information of the fused image and enhances the visibility of temperature information.

[0148] In one or more of the above embodiments, this application has at least the following features:

[0149] 1. To fully leverage the complementarity of infrared and visible light and utilize their combined advantages to overcome limitations in complex environments, the main approach is to first acquire a high-frequency information layer based on a first fused image. First fusion parameters determine how infrared and visible light are adapted and combined to generate the first fused image, which controls the generation of the richest high-frequency information. The acquired high-frequency information at this stage is the result of dual-spectrum complementarity, effectively addressing the shortcomings of individual spectra. Subsequently, the acquired high-frequency information is added to a second fused image. A second fusion parameter set determines how infrared and visible light are adapted and combined to generate the second fused image, which controls the generation of the final imaging effect and further enhances the first high-frequency fused image.

[0150] 2. Most existing dual-light fusion information sources extract only the information of interest from a single light source (infrared or visible light) or extract the information of interest from both light sources separately. However, this application generates a first fused image based on the first fusion parameters, obtains high-frequency information based on the first fused image, and generates a first high-frequency fused image. Combined with S11 and S12, it can truly achieve scene adaptation because the first fused image in S12 retains both the high-frequency information of visible light and the high-frequency information of infrared light. This can overcome the problem that high-frequency information cannot be obtained when the image quality is low in certain scenes with single light, and completely carry scene information. When visible light loses details due to low light or complex lighting conditions, infrared light can retain the object outline and thermal radiation characteristics. When infrared light becomes blurred due to the small temperature difference of the target, visible light can supplement the high-frequency information, accurately superimposing key information such as color, texture, and details into the first fused image.

[0151] 3. Most existing dual-light fusion or infrared image enhancement methods directly superimpose the extracted information onto the infrared image, multiply the extracted information by coefficients and add it to the infrared image, or directly add the information extracted from the two lights. However, this technology generates a second fused image based on the second fusion parameters. By combining the two fusion processes, the high-frequency information of the fused image can be further enhanced. Although the first fusion parameters retain the high-frequency information of the two lights, they are closer to visible light, while the second fused image is closer to infrared. Therefore, the two can complement each other again, while retaining the color information of visible light. This allows the fused image to present natural colors and contain infrared thermal features, taking into account the human eye's color perception habits and the need for key target recognition.

[0152] Existing dual-light fusion algorithms sometimes employ deep learning architectures, focusing on abstract image features. They emphasize structured scene modeling and spatial perception through feature analysis, primarily serving subsequent intelligent decision-making rather than visual fusion tasks. Other methods perform image fusion based on such abstract image features, but these features often create a strong disconnect with natural tones, resulting in visual jarring and inconsistent with visual aesthetics. This application, however, focuses on optimizing visual presentation by collaboratively integrating infrared and visible light pixel data to generate a fused image that conforms to human visual perception. Furthermore, most deep learning methods are difficult to implement on resource-constrained, low-power edge devices, failing to fully meet the platform's high real-time requirements. This application, however, is suitable for dual-light fusion scenarios without GPU acceleration and with high real-time requirements.

[0153] 4. This technology eliminates the dependence on specific scenarios. It does not require separate design of algorithm parameters or customized development modules for different environments such as nighttime or strong light. It can maintain stable and efficient performance in various scenarios, and truly achieves cross-scenario universality improvement.

[0154] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the dual-light image fusion method described in any embodiment of this application.

[0155] In the computer program product, the optional implementation form of the program module architecture for implementing the steps of the dual-light image fusion method can be a dual-light image fusion method apparatus. Please refer to [link to relevant documentation]. Figure 10 This application provides a dual-light image fusion method apparatus, comprising: an acquisition module 60 for acquiring an infrared image and a visible light image of the same target scene; a fusion module 61 for fusing the infrared image and the visible light image based on a first fusion parameter to obtain a first fused image; a determination module 62 for obtaining a first high-frequency fused image based on the first fused image; the fusion module 61 is further configured to fuse the infrared image and the visible light image based on a second fusion parameter to obtain a second fused image; the fusion module 61 is also configured to perform a fusion operation based on the first high-frequency fused image and the second fused image to obtain a target fused image; or

[0156] The fusion module 61 is further configured to fuse the infrared image and the visible light image to obtain a third fused image; the determination module 62 is further configured to obtain a second high-frequency fused image based on the third fused image; the fusion module 61 is further configured to perform a fusion operation based on the second high-frequency fused image and the infrared image to obtain a target fused image; or

[0157] The determining module 62 is used to obtain a visible light high-frequency image based on the visible light image; the fusion module 61 is also used to fuse the infrared image and the visible light image to obtain a fourth fused image; the fusion module 61 is also used to perform a fusion operation based on the visible light high-frequency image and the fourth fused image to obtain a target fused image.

[0158] Optionally, the first fusion parameters include a first infrared weight and a first visible light weight, and based on the first fusion parameters, the fusion module 61 is further configured to:

[0159] Obtain the first infrared weight and the first visible light weight;

[0160] Based on the first infrared weight and the first visible light weight, the infrared image and the visible light image are fused to obtain a first fused image, wherein the first infrared weight is less than the first visible light weight.

[0161] Optionally, the fusion module 61 is also used for:

[0162] The image quality of the visible light image is evaluated to obtain a visible light image quality value.

[0163] Based on the visible light image quality value, a first visible light weight corresponding to the visible light image quality value is determined;

[0164] The first infrared weight is calculated based on the first visible light weight corresponding to the visible light image quality value.

[0165] Optionally, the determining module 62 is also used for:

[0166] The high-frequency image, including high-frequency information, is extracted from the first fused image using a high-pass filter to obtain the first high-frequency fused image; or, the low-frequency image, including low-frequency information, is extracted from the first fused image using a low-pass filter, and the difference between the first fused image and the low-frequency image in the first fused image is taken to obtain the first high-frequency fused image; or

[0167] Optionally, the determining module 62 is also used for:

[0168] The high-frequency image, including high-frequency information, is extracted from the third fused image using a high-pass filter to obtain the second high-frequency fused image; or, the low-frequency image, including low-frequency information, is extracted from the third fused image using a low-pass filter, and the difference between the third fused image and the low-frequency image in the third fused image is taken to obtain the second high-frequency fused image; or

[0169] Optionally, the determining module 62 is also used for:

[0170] The visible light high-frequency image is obtained by extracting a high-frequency image containing high-frequency information from the visible light image using a high-pass filter; or, the visible light high-frequency image is obtained by extracting a low-frequency image containing low-frequency information from the visible light image using a low-pass filter and subtracting the visible light image from the low-frequency image in the visible light image.

[0171] Optionally, the fusion module 61 is also used for:

[0172] Obtain the second infrared weight and the second visible light weight;

[0173] Based on the second infrared weight and the second visible light weight, the infrared image and the visible light image are fused to obtain a second fused image, wherein the second infrared weight is greater than the second visible light weight.

[0174] Optionally, the fusion module 61 is also used for:

[0175] The image quality of the infrared image is evaluated to obtain an infrared image quality value.

[0176] Based on the infrared image quality value, a second infrared weight corresponding to the infrared image quality value is determined;

[0177] The second visible light weight is calculated based on the second infrared weight corresponding to the infrared image quality value.

[0178] Optionally, the fusion module 61 is also used for:

[0179] Obtain a first enhancement coefficient, multiply the first enhancement coefficient by the pixel value of each pixel in the first high-frequency fused image to obtain a first enhanced high-frequency image, and perform an addition operation on the first enhanced high-frequency image and the second fused image to obtain the target fused image; or

[0180] Optionally, the fusion module 61 is also used for:

[0181] Obtain the second enhancement coefficient, multiply the second enhancement coefficient by the pixel value of each pixel in the second high-frequency fusion image to obtain the second enhanced high-frequency image, and perform an addition operation on the second enhanced high-frequency image and the infrared image to obtain the target fusion image; or

[0182] Optionally, the fusion module 61 is also used for:

[0183] A third enhancement coefficient is obtained, and the third enhancement coefficient is multiplied by the pixel value of each pixel in the visible light high-frequency image to obtain a third enhanced high-frequency image. An addition operation is performed on the third enhanced high-frequency image and the fourth fused image to obtain the target fused image.

[0184] Optionally, the fusion module 61 is also used for:

[0185] Based on the enhancement coefficient control provided by the user interface, obtain the configured first enhancement coefficient;

[0186] The image quality of the first high-frequency fused image is evaluated to obtain a high-frequency image quality value. Based on the high-frequency image quality value of the first high-frequency fused image, an enhancement coefficient corresponding to the high-frequency image quality value of the first high-frequency fused image is determined; or

[0187] Optionally, the fusion module 61 is also used for:

[0188] Based on the enhancement coefficient control provided in the user interface, obtain the configured second enhancement coefficient;

[0189] The image quality of the second high-frequency fused image is evaluated to obtain a high-frequency image quality value. Based on the high-frequency image quality value of the second high-frequency fused image, an enhancement coefficient corresponding to the high-frequency image quality value of the second high-frequency fused image is determined; or

[0190] Optionally, the fusion module 61 is also used for:

[0191] Based on the enhancement coefficient control provided by the user interface, obtain the configured third enhancement coefficient;

[0192] The image quality of the visible light high-frequency image is evaluated to obtain the high-frequency image quality value of the visible light high-frequency image. Based on the high-frequency image quality value of the visible light high-frequency image, an enhancement coefficient corresponding to the high-frequency image quality value of the visible light high-frequency image is determined.

[0193] Optionally, module 60 is also used for:

[0194] Acquire the initial infrared image and initial visible light image for initial registration;

[0195] Obtain the registration compensation value;

[0196] Based on the registration compensation value, an offset operation is performed on the initial infrared image and / or the initial visible light image to obtain the registered infrared image and visible light image.

[0197] Optionally, the fusion module 61 is also used for:

[0198] Obtain the configured mapping relationship between pixel values ​​and color values;

[0199] Based on the mapping relationship, the target fused image is colored to obtain a colored image, and the colored image is displayed.

[0200] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the dual-light image fusion method apparatus includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0201] This application embodiment can, based on the above method, exemplarily divide the dual-light image fusion method apparatus into functional modules. For example, the dual-light image fusion method apparatus may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0202] like Figure 11 As shown, computing device 10 includes a processor 13, a memory 14, and a communication interface 15. The processor 13, memory 14, and communication interface 15 communicate via a bus. Computing device 10 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 10. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in the figure, but this does not indicate that there is only one bus or one type of bus. Bus 104 can include a path for transmitting information between various components of computing device 10 (e.g., memory 14, processor 13, communication interface 15). Processor 13 can include any one or more processors such as a central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0203] Memory 14 may include volatile memory, such as random access memory (RAM). Processor 13 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0204] The memory 14 stores executable program code, which the processor 13 executes to implement the functions of the aforementioned modules, thereby realizing the dual-light image fusion method. That is, the memory 14 stores instructions for executing the dual-light image fusion method. Alternatively, the memory 14 stores executable code, which the processor 13 executes to implement the functions of the aforementioned dual-light image fusion device, thereby realizing the dual-light image fusion method. That is, the memory 14 stores instructions for executing the dual-light image fusion method. The communication interface 15 uses a transceiver module, such as, but not limited to, a network interface card or transceiver, to enable communication between the computing device 10 and other devices or communication networks.

[0205] In another aspect, this application provides a computer-readable non-volatile storage medium storing a computer program. When the computer program is executed by a processor, it causes the processor to perform the steps of a dual-light image fusion method provided in any of the above embodiments of this application.

[0206] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of a dual-light image fusion method as described in any embodiment of this application.

[0207] Those skilled in the art will understand that all or part of the processes in the methods provided in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0208] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A two-light image fusion method, characterized in that, include: Acquire infrared and visible light images of the same target scene; Based on the first fusion parameter, the infrared image and the visible light image are fused to obtain the first fused image; Based on the first fused image, a first high-frequency fused image is obtained; based on the second fusion parameters, the infrared image and the visible light image are fused to obtain a second fused image; based on the first high-frequency fused image and the second fused image, a fusion operation is performed to obtain a target fused image; The first fusion parameter includes a first infrared weight and a first visible light weight. The step of fusing the infrared image and the visible light image based on the first fusion parameter to obtain a first fused image includes: obtaining the first infrared weight and the first visible light weight; fusing the infrared image and the visible light image based on the first infrared weight and the first visible light weight to obtain a first fused image, wherein the first infrared weight is less than the first visible light weight. The second fusion parameter includes a second infrared weight and a second visible light weight. The step of fusing the infrared image and the visible light image based on the second fusion parameter to obtain a second fused image includes: obtaining the second infrared weight and the second visible light weight; fusing the infrared image and the visible light image based on the second infrared weight and the second visible light weight to obtain a second fused image, wherein the second infrared weight is greater than the second visible light weight.

2. The dual-light image fusion method as described in claim 1, characterized in that, The process of obtaining the first infrared weight and the first visible light weight includes: The image quality of the visible light image is evaluated to obtain a visible light image quality value. Based on the visible light image quality value, a first visible light weight corresponding to the visible light image quality value is determined; The first infrared weight is calculated based on the first visible light weight corresponding to the visible light image quality value.

3. The dual-light image fusion method as described in claim 1, characterized in that, The first high-frequency fused image obtained based on the first fused image includes at least one of the following: The high-frequency image containing high-frequency information is extracted from the first fused image using a high-pass filter to obtain the first high-frequency fused image; or, the low-frequency image containing low-frequency information is extracted from the first fused image using a low-pass filter, and the first high-frequency fused image is obtained by subtracting the first fused image from the low-frequency image in the first fused image.

4. The dual-light image fusion method as described in claim 1, characterized in that, The process of obtaining the second infrared weight and the second visible light weight includes: The image quality of the infrared image is evaluated to obtain an infrared image quality value. Based on the infrared image quality value, a second infrared weight corresponding to the infrared image quality value is determined; The second visible light weight is calculated based on the second infrared weight corresponding to the infrared image quality value.

5. The dual-light image fusion method as described in claim 1, characterized in that, The step of performing a fusion operation based on the first high-frequency fused image and the second fused image to obtain the target fused image includes: A first enhancement coefficient is obtained, and the first enhancement coefficient is multiplied by the pixel value of each pixel in the first high-frequency fusion image to obtain a first enhanced high-frequency image. An addition operation is performed on the first enhanced high-frequency image and the second fusion image to obtain the target fusion image.

6. The dual-light image fusion method as described in claim 5, characterized in that, The process of obtaining the first enhancement coefficient includes: Based on the enhancement coefficient control provided by the user interface, obtain the configured first enhancement coefficient; The image quality of the first high-frequency fused image is evaluated to obtain the high-frequency image quality value of the first high-frequency fused image. Based on the high-frequency image quality value of the first high-frequency fused image, the enhancement coefficient corresponding to the high-frequency image quality value of the first high-frequency fused image is determined.

7. The dual-light image fusion method as described in claim 1, characterized in that, The method further includes: Acquire the initial infrared image and initial visible light image for initial registration; Obtain the registration compensation value; Based on the registration compensation value, an offset operation is performed on the initial infrared image and / or the initial visible light image to obtain the registered infrared image and visible light image.

8. The dual-light image fusion method as described in claim 1, characterized in that, The method further includes: Obtain the configured mapping relationship between pixel values ​​and color values; Based on the mapping relationship, the target fused image is colored to obtain a colored image, and the colored image is displayed.

9. A computing device, characterized in that, The system includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the dual-light image fusion method as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the dual-light image fusion method as described in any one of claims 1 to 8.

Images