Infrared image and visible light image fusion method and device, equipment and storage medium

Abstract

The invention provides an infrared image and visible light image fusion method and device, equipment and a storage medium, and the method comprises the steps: constructing a visual perception factor according with the visual characteristics of human eyes based on an infrared image and a visible light image, and carrying out the weighted fusion of the infrared image and the visible light image according to the visual perception factor, and obtaining an initial fusion image; performing brightness equalization processing on the initial fusion image according to the brightness reference value in the initial fusion image; performing detail feature extraction based on visual perception on the initial fusion image, and fusing the extracted detail features to the initial fusion image after brightness equalization to obtain a detail perception image; and performing weighted fusion on the detail perception image and the initial fusion image to obtain a target fusion image.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, apparatus, device, and storage medium for fusing infrared and visible light images. Background Technology

[0002] Infrared images have low resolution and poor visual quality, while visible light images are subject to high lighting conditions and are prone to loss of detail under low light conditions. Therefore, it is necessary to fuse infrared images with visible light images to improve the detail and recognizability of the images.

[0003] Currently, infrared and visible light image fusion technology plays an indispensable role in various application scenarios such as nighttime surveillance and security detection. Existing infrared and visible light image fusion methods generally employ salient region extraction and image enhancement algorithms to improve the detail representation of images. Among these, detail enhancement strategies such as Unsharp Masking (USM) are widely used. USM enhances the highlighting of image structures by optimizing the weight map. However, detail enhancement strategies typically rely on fixed parameter configurations or manually designed feature models, lacking dynamic response capabilities to different image scenes and failing to meet users' visual perception needs. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, device and storage medium for fusing infrared images and visible light images, which realizes the fusion of infrared images and visible light images, enhances the detail features of the images, improves the contrast of the images, and generates a fused image that is more suitable for human visual perception.

[0005] In a first aspect, embodiments of this application provide a method for fusing infrared images and visible light images, including:

[0006] Based on infrared and visible light images, a visual perception factor that conforms to the visual characteristics of the human eye is constructed. The infrared and visible light images are then weighted and fused according to the visual perception factor to obtain an initial fused image.

[0007] Based on the brightness reference value in the initial fused image, the initial fused image is subjected to brightness equalization processing;

[0008] The initial fused image is subjected to visual perception-based detail feature extraction, and the extracted detail features are fused into the initial fused image after brightness equalization to obtain a detail-perceived image;

[0009] The detailed-aware image and the initial fused image are weighted and fused to obtain the target fused image.

[0010] In one possible implementation, the step of constructing a visual perception factor that conforms to the characteristics of human vision based on infrared and visible light images, and then weighting and fusing the infrared and visible light images according to the visual perception factor to obtain an initial fused image, includes:

[0011] A first visual perception factor is calculated from the infrared image, and a second visual perception factor is calculated from the visible light image.

[0012] Calculate the first fusion weight based on the first visual perception factor and the second visual perception factor;

[0013] Based on the first fusion weight, the infrared image and the visible light image are linearly added together to obtain the initial fused image.

[0014] In one possible implementation, the step of performing brightness equalization processing on the initial fused image based on a brightness reference value in the initial fused image includes:

[0015] A brightness reference value is obtained based on the initial fused image;

[0016] The brightness reference value is subtracted from the pixel value of each point in the initial fused image to obtain the brightness-equalized initial fused image.

[0017] In one possible implementation, the brightness reference value is the average value of all pixels in the initial fused image.

[0018] In one possible implementation, the step of extracting visually perceptual detail features from the initial fused image and fusing the extracted detail features into the brightness-equalized initial fused image to obtain a detail-perceptual map includes:

[0019] The frequency of each gray level in the initial fused image is counted, and the initial response value of each pixel is calculated based on the gray levels.

[0020] The initial response value is processed to determine the final response value of each pixel, thus obtaining a visual perception response map;

[0021] The visual perception response map and the initial fusion image after brightness equalization are fused together, and the pixel values ​​of corresponding pixels are multiplied to obtain the pixel values ​​of each pixel in the detail perception map, thereby obtaining the detail perception map.

[0022] In one possible implementation, the weighted fusion of the detail-aware map and the initial fused image to obtain the target fused image includes:

[0023] The preset second fusion weight is multiplied by the value of each pixel in the detail-aware image, and the result is added to the corresponding pixel in the initial fusion image to obtain the pixel value of each pixel in the target fusion image, thereby obtaining the target fusion image.

[0024] In one possible implementation, the preset second fusion weight p has the following value range: 0 ≤ p ≤ 0.5.

[0025] Secondly, embodiments of this application provide an infrared image and visible light image fusion device, comprising:

[0026] The first image processing module is used to calculate a visual perception factor that conforms to the visual characteristics of the human eye based on an infrared image and a visible light image, and to perform weighted fusion of the infrared image and the visible light image according to the visual perception factor to obtain an initial fused image.

[0027] The second image processing module is used to perform brightness equalization processing on the initial fused image based on the brightness reference value in the initial fused image, extract detail features based on visual perception from the initial fused image, and fuse the extracted detail features into the brightness equalized initial fused image to obtain a detail-perceived image.

[0028] The image fusion module is used to perform weighted fusion of the initial fused image and the detail-aware image to obtain the target fused image.

[0029] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions implementing the steps of the method as described in the first aspect when executed by the processor.

[0030] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0031] Fifthly, embodiments of this application provide a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0032] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect.

[0033] In this embodiment, by constructing a visual perception factor to simulate the human eye's sensitivity to brightness, and by performing brightness equalization processing on the initial fused image and extracting details from the initial fused image, the information that the human eye is sensitive to in the initial fused image can be enhanced, highlighting the detailed features in the initial fused image, improving the contrast of the target fused image, thereby enhancing the detail expression ability of the target fused image, and making the target fused image more in line with the visual effect of the human eye.

[0034] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This illustration shows one of the flowcharts of an infrared image and visible light image fusion method according to an embodiment of this application;

[0037] Figure 2 This is a second schematic flowchart illustrating an embodiment of the infrared image and visible light image fusion method of this application;

[0038] Figure 3 The third schematic flowchart illustrates an embodiment of the infrared image and visible light image fusion method of this application.

[0039] Figure 4 A structural block diagram of an infrared image and visible light image fusion device according to an embodiment of this application is shown. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0041] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0042] The following description, in conjunction with the accompanying drawings, details a method for fusing infrared and visible light images provided in this application through specific embodiments and application scenarios. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0043] like Figure 1 As shown, this application provides a method for fusing infrared and visible light images, and the subject of this method can be an electronic device capable of image processing.

[0044] Step 101: Based on the infrared image and the visible light image, construct a visual perception factor that conforms to the visual characteristics of the human eye. Then, perform weighted fusion of the infrared image and the visible light image according to the visual perception factor to obtain an initial fused image.

[0045] Infrared images are formed by capturing the infrared radiation emitted by an object (wavelengths typically ranging from 0.75 μm to 1000 μm), while visible light images are formed by capturing the visible light reflected by an object (wavelengths approximately 380 nm to 750 nm).

[0046] Visual perception factor is a mathematical model that quantifies the human eye's sensitivity to image content. By simulating the human visual system's response characteristics to brightness, contrast, and structure, it dynamically labels the visual importance of different regions in an image. Specifically, electronic devices construct visual perception factors that conform to the subjective characteristics of visual sensitivity based on image type, in order to enhance the image's structural perception ability and the response intensity of salient regions at the visual cognitive level.

[0047] The electronic device calculates the visual perception factor value of each pixel based on the degree of deviation of each pixel in the visible light image and infrared image from the preset perception center, calculates the first fusion weight based on the value of the visual perception factor, and fuses the infrared image and the visible light image based on the first fusion weight to obtain an initial fused image.

[0048] Step 102: Perform brightness equalization processing on the initial fused image based on the brightness reference value in the initial fused image.

[0049] The luminance reference value represents the overall luminance reference of the initial fused image. Specifically, the electronic device calculates the luminance reference value of the initial fused image based on the overall luminance of the initial fused image, and subtracts the luminance reference value from the value of each pixel in the initial fused image to obtain the luminance-equalized initial fused image.

[0050] Step 103: Extract detail features based on visual perception from the initial fused image, and fuse the extracted detail features into the initial fused image after brightness equalization to obtain a detail-perceived image.

[0051] Visual perception-based detail feature extraction is a computational method that simulates the sensitivity of the human visual system to image details. By combining psychological models with image processing techniques, it adaptively extracts effective details from multimodal images that conform to the subjective perception of the human eye.

[0052] Specifically, the electronic device calculates the visual perception response values ​​corresponding to different gray levels based on the initial fused image, performs normalization and nonlinear processing on the obtained visual perception response values ​​to obtain the detail features extracted from the initial fused image, and then fuses the extracted detail features with the brightness equalized initial fused image to obtain the detail perception map.

[0053] Step 104: Weighted fusion of the detail-aware image and the initial fused image to obtain the target fused image.

[0054] Specifically, the electronic device fuses the initial fusion image obtained in step 101 with the detail-aware image based on a preset second fusion weight, and finally obtains the target fusion image.

[0055] The infrared and visible light image fusion method provided in the above embodiments simulates the human eye's sensitivity to brightness by constructing visual perception factors. By performing brightness equalization processing on the initial fused image and extracting details from the initial fused image, the information that the human eye is sensitive to in the initial fused image can be enhanced, highlighting the detailed features in the initial fused image, improving the contrast of the target fused image, thereby improving the detail expression ability of the target fused image and making the target fused image more in line with the visual effect of the human eye.

[0056] In the embodiments of this application, further, as Figure 2 As shown, step 101 can be achieved through steps 1011 to 1013:

[0057] Step 1011: Calculate the first visual perception factor from the infrared image and calculate the second visual perception factor from the visible light image.

[0058] Electronic devices construct a response model that conforms to the characteristics of visual subjective sensitivity based on the type of input image. This response model is used to improve the structural perception ability and salient region response intensity of the input image at the visual cognitive level, and to calculate and generate visual perception factors.

[0059] Specifically, the electronic device calculates the first visual perception factor based on the infrared image using a first formula. The first formula is:

[0060]

[0061] Among them, I r0 I represents the raw pixel intensity value of an infrared image. r1 u represents the first visual perception factor value of the corresponding pixel in the infrared image. i This represents the set first brightness perception center value, u i Satisfy u i >200, σ i The first-scale adjustment factor represents the control of the amplitude of the perceived response.

[0062] The electronic device calculates the second visual perception factor based on the second formula for visible light images. The second formula is:

[0063]

[0064] Among them, V s0 V represents the raw pixel intensity value of a visible light image. s1 u represents the second visual perception factor value of the corresponding pixel in a visible light image. v This represents the set second brightness perception center value, u v Satisfy u v <200, σ v This represents the second-scale adjustment factor that controls the amplitude of the perceived response.

[0065] u in the first formula i u in the second formula v σ in the first formula is used to simulate the human eye's preference for medium brightness areas. i σ in the second formula v σ is used to determine the rate at which the response decays after a pixel value deviates from the center value of the perceived brightness. i and σ v The relation satisfies: σ v <σ i .

[0066] Step 1012: Calculate the first fusion weight based on the first visual perception factor and the second visual perception factor.

[0067] Specifically, the electronic device utilizes the first visual perception factor I according to the third formula. r1 Second visual perception factor V s1 Calculate the first fusion weight. The third formula is:

[0068] w = V s1 2 / (V s1 2 +I r1 2 )

[0069] Where w represents the first fusion weight, V s1 I represents the second visual perception factor value. r1 This represents the first visual perception factor value.

[0070] Step 1013: Based on the first fusion weight, the infrared image and the visible light image are linearly added together to obtain the initial fused image.

[0071] Specifically, the electronic device fuses the infrared and visible light images based on the fourth formula and according to the calculated first fusion weights to obtain an initial fused image. The fourth formula is:

[0072] X = w * V s0 +(1-w)*I r0

[0073] Where X represents the initial fused image, w represents the image fusion weight matrix, and V s0 I represents the raw pixel intensity value of a visible light image. r0 This represents the raw pixel intensity value of the infrared image.

[0074] Thus, the electronic device calculates the first visual perception factor and the second visual perception factor using the first and second formulas, and then calculates the first fusion weight using the third formula. Based on the first fusion weight, the infrared image and the visible light image are fused to obtain the initial fused image. The electronic device, based on the first fusion weight calculated by the third formula, can adaptively enhance the proportion of the first and second perception factors in the initial fused image. When the first visual perception factor is higher than the pixel value of the original infrared image, the obtained initial fused image is biased towards the style of the visible light image; when the value of the first visual perception factor is lower, the style of the obtained initial fused image is more biased towards the infrared image.

[0075] In this embodiment of the application, step 102 can be further implemented by steps 1021 to 1022:

[0076] Step 1021: Obtain the brightness reference value based on the initial fused image.

[0077] Furthermore, the brightness baseline value is the average value of all pixels in the initial fused image.

[0078] The electronic device calculates the average pixel value of each pixel in the initial fused image, and the resulting value is the brightness reference value. The brightness reference value decomposes each pixel in the initial fused image into two parts: low-frequency background and high-frequency details. This avoids interference from the overall brightness in enhancing the details of the initial fused image and highlights the contrast of the parts of the initial fused image that conform to the visual effect of the human eye.

[0079] Step 1022: Subtract the brightness reference value from the pixel value of each point in the initial fused image to obtain the initial fused image after brightness equalization.

[0080] Specifically, the electronic device calculates the pixel value of each pixel in the initial fused image after brightness equalization based on the fifth formula, thereby obtaining the initial fused image after brightness equalization. The fifth formula is:

[0081] X h =X-μ x

[0082] Where X represents the pixel value of a pixel in the initial fused image, X h μ represents the pixel value of the corresponding pixel in the initial fused image after brightness equalization. x This indicates the reference value for brightness.

[0083] In this way, the electronic device subtracts the pixel values ​​of the initial fused image from the brightness reference value to obtain the brightness-equalized initial fused image. This allows it to extract areas in the initial fused image that deviate from the perceptual equilibrium state as potential detail areas, highlight visually salient areas in the initial fused image, and avoid over-enhancing relatively flat areas in the initial fused image. This makes the obtained brightness-equalized initial fused image more in line with the visual effect of the human eye and improves the recognizability of the initial fused image.

[0084] In the embodiments of this application, further, as Figure 3 As shown, step 103 can be achieved through steps 1031 to 1033:

[0085] Step 1031: Count the frequency of each gray level in the initial fused image, and calculate the initial response value of each pixel based on the gray level.

[0086] Gray levels are perceptual saliency quantifications assigned to different regions or gray values ​​in the initial fused image based on the characteristics of the human visual system. Specifically, the electronic device counts the frequency of each gray level in the initial fused image and calculates the initial response value corresponding to different gray levels according to the sixth formula. The sixth formula is:

[0087]

[0088] Where i represents the gray level of a pixel value, f(i) represents the frequency of occurrence of pixel values ​​with gray level i in the initial fused image, j represents the current gray level, and S0(j) represents the initial response value corresponding to the current gray level.

[0089] Step 1032: Process the initial response value to determine the final response value of each pixel, thereby obtaining the visual perception response map.

[0090] Specifically, the electronic device normalizes the initial response value and then performs nonlinear processing on the normalized initial response value according to the seventh formula to obtain the final response value of each pixel, thereby obtaining the visual perception response map. The seventh formula is:

[0091] S1=(S0 / max(S0))γ

[0092] Where S0 represents the initial response value after normalization, S1 represents the final response value, and γ represents the first parameter, which satisfies: 0≤γ≤1, and the default value of the first parameter γ is 0.5.

[0093] Step 1033: The visual perception response map and the initial fusion image after brightness equalization are fused to obtain the pixel values ​​of each pixel in the detail perception map, thereby obtaining the detail perception map.

[0094] Specifically, based on the eighth formula, the electronic device multiplies the pixel values ​​of the corresponding pixels in the initial fused image after brightness equalization and the visual perception response map to obtain an attention-driven detail perception map. The eighth formula is:

[0095] X sd =X h *S1

[0096] Among them, X h X represents the pixel value of the pixel in the initial fused image after brightness equalization, S1 represents the pixel value of the corresponding pixel in the visual perception response map, and X represents the pixel value of the pixel in the initial fused image after brightness equalization. sd This represents the pixel value of the corresponding pixel in the calculated detail-aware map.

[0097] In this way, the electronic device obtains a visual perception response map based on the initial fused image, and finally obtains a detail perception map based on the visual perception response map and the initial fused image after brightness equalization. The detail perception map can filter out details in the initial fused image that conform to the visual sensitivity of the human eye, which is convenient for further representation of the details in the initial fused image in the subsequent weighted fusion step.

[0098] In this embodiment of the application, step 104 is implemented by the perceptual clarity psychological enhancement algorithm, which specifically involves multiplying the preset second fusion weight in the electronic device with the value of each pixel in the detail perception map, adding the result to the corresponding pixel in the initial fusion image, obtaining the pixel value of each pixel in the target fusion image, and thus obtaining the target fusion image.

[0099] The preset second fusion weight in the electronic device is an adjustment parameter that controls the intensity of detail representation in the target fused image by the detail-aware map. Specifically, based on the ninth formula, the electronic device performs weighted fusion of the pixel values ​​of pixels in the initial fused image and the corresponding pixel values ​​in the detail-aware map using the preset second fusion weight to obtain the pixel values ​​of each pixel in the target fused image, thereby obtaining the target fused image. The ninth formula is:

[0100] X out =X + p*X sd

[0101] Where X represents the pixel value of a pixel in the initial fused image, and p represents the preset second fusion weight. sd X represents the pixel value of the corresponding pixel in the detail-aware image. out This represents the pixel value of the corresponding pixel in the obtained target fused image.

[0102] Furthermore, the preset range of values ​​for the second fusion weight p is: 0 ≤ p ≤ 0.5.

[0103] The preset second fusion weight in the electronic device determines the proportion of detail information extracted from the detail-aware map and added to the initial fused image, which is used to balance the degree of detail enhancement and the naturalness of the image. When the preset second fusion weight p is 0, the initial fused image is the target fused image, and the details in the detail-aware map are completely ignored; when the preset second fusion weight p is 0.5, the detail features in the detail-aware map are maximized in the target fused image.

[0104] By using the preset second fusion weight in the electronic device, the degree of detail features reflected in the target fused image can be accurately grasped. This can avoid the distortion of the target fused image caused by the over-enhancement of detail features, and also avoid the insufficient reflection of detail features, thereby improving the detail visibility and robustness of the target fused image.

[0105] Thus, by utilizing a perceptual clarity enhancement algorithm, electronic devices can improve the detail feature representation of the target fused image by weighting the detail-perceived image and the initial fused image with a preset second fusion weight. This preset second fusion weight can be adjusted within a certain range, allowing for better control over the detail features in the target fused image and significantly improving its robustness. Using the initial fused image as the main body of the target fused image ensures that the generated image closely resembles human visual perception, minimizing visual discomfort.

[0106] This application also provides an infrared image and visible light image fusion device. The infrared image and visible light image fusion device includes: a first image processing module, a second image processing module, and an image fusion module.

[0107] The first image processing module is used to construct visual perception factors that conform to the visual characteristics of the human eye based on infrared and visible light images, and to perform weighted fusion of infrared and visible light images according to the visual perception factors to obtain an initial fused image.

[0108] The second image processing module is used to perform brightness equalization processing on the initial fused image based on the brightness reference value in the initial fused image, extract detail features based on visual perception from the initial fused image, and fuse the extracted detail features into the brightness equalized initial fused image to obtain a detail-perceived image.

[0109] The image fusion module is used to perform weighted fusion of the initial fused image and the detail-aware image to obtain the target fused image.

[0110] In this way, by simulating the human eye's sensitivity to brightness through visual perception factors, and by performing brightness equalization processing and detail extraction on the initial fused image, the brightness of the initial fused image can be balanced, and the information that the human eye is sensitive to in the initial fused image can be enhanced, thereby highlighting the detailed features in the initial fused image, improving the contrast of the target fused image, and thus enhancing the detail expression ability of the target fused image, making the target fused image more in line with the visual effect of the human eye.

[0111] In some embodiments, the first image processing module is used to calculate a first visual perception factor from an infrared image and a second visual perception factor from a visible light image; calculate a first fusion weight based on the first and second visual perception factors; and linearly add the infrared image and the visible light image based on the first fusion weight to obtain an initial fused image.

[0112] In some embodiments, the second image processing module is used to obtain a brightness reference value based on the initial fused image; and to subtract the brightness reference value from the pixel value of each point in the initial fused image to obtain the initial fused image after brightness equalization.

[0113] In some embodiments, the brightness reference value in the second image processing module is the average value of all pixels in the initial fused image.

[0114] In some embodiments, the second image processing module is further configured to count the frequency of each gray level in the initial fused image, calculate the initial response value of each pixel by combining the gray levels; process the initial response value to determine the final response value of each pixel to obtain a visual perception response map; fuse the visual perception response map and the brightness equalized initial fused image, multiply the pixel values ​​of corresponding pixels in the fused image to obtain the pixel values ​​of each pixel in the detail perception map, and thus obtain the detail perception map.

[0115] In some embodiments, the image fusion module is used to multiply the preset second fusion weight by the value of each pixel in the detail-aware image, add the result to the corresponding pixel in the initial fusion image, obtain the pixel value of each pixel in the target fusion image, and thus obtain the target fusion image.

[0116] In some embodiments, the preset second fusion weight p in the image fusion module has a value range that satisfies: 0≤p≤0.5.

[0117] The infrared and visible light image fusion device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0118] The infrared image and visible light image fusion device provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0119] This application also provides an electronic device, such as... Figure 4 As shown, the electronic device 400 includes a processor 401 and a memory 402. The memory 402 stores a program or instruction that can run on the processor 401. When the program or instruction is executed by the processor 401, it implements the various steps of the above-described infrared image and visible light image fusion method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0120] The memory 402 can be used to store software programs and various data. The memory 402 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 402 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 402 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0121] Processor 401 may include one or more processing units; optionally, processor 401 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 401.

[0122] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described infrared image and visible light image fusion method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0123] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described infrared image and visible light image fusion method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0124] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0125] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the infrared image and visible light image fusion method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0127] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for fusing infrared and visible light images, characterized in that, include: Based on infrared and visible light images, a visual perception factor that conforms to the visual characteristics of the human eye is constructed. The infrared and visible light images are then weighted and fused according to the visual perception factor to obtain an initial fused image. Based on the brightness reference value in the initial fused image, the initial fused image is subjected to brightness equalization processing; The initial fused image is subjected to visual perception-based detail feature extraction, and the extracted detail features are fused into the initial fused image after brightness equalization to obtain a detail-perceived image; The detailed-aware image and the initial fused image are weighted and fused to obtain the target fused image.

2. The infrared image and visible light image fusion method according to claim 1, characterized in that, The process involves constructing visual perception factors that conform to the characteristics of human vision based on infrared and visible light images, and then weighting and fusing the infrared and visible light images according to these visual perception factors to obtain an initial fused image, including: A first visual perception factor is calculated from the infrared image, and a second visual perception factor is calculated from the visible light image. Calculate the first fusion weight based on the first visual perception factor and the second visual perception factor; Based on the first fusion weight, the infrared image and the visible light image are linearly added together to obtain the initial fused image.

3. The method for fusing infrared and visible light images according to claim 1, characterized in that, The step of performing brightness equalization processing on the initial fused image based on the brightness reference value in the initial fused image includes: A brightness reference value is obtained based on the initial fused image; The brightness reference value is subtracted from the pixel value of each point in the initial fused image to obtain the brightness-equalized initial fused image.

4. The method for fusing infrared and visible light images according to claim 1, characterized in that, The brightness reference value is the average value of all pixels in the initial fused image.

5. The infrared image and visible light image fusion method according to claim 1, characterized in that, The step of extracting visually perceptual detail features from the initial fused image and fusing the extracted detail features into the brightness-equalized initial fused image to obtain a detail-perceptual map includes: The frequency of each gray level in the initial fused image is counted, and the initial response value of each pixel is calculated based on the gray levels. The initial response value is processed to determine the final response value of each pixel, thus obtaining a visual perception response map; The visual perception response map and the initial fusion image after brightness equalization are fused together, and the pixel values ​​of corresponding pixels are multiplied to obtain the pixel values ​​of each pixel in the detail perception map, thereby obtaining the detail perception map.

6. The method for fusing infrared and visible light images according to claim 1, characterized in that, The weighted fusion of the detail-aware map and the initial fused image to obtain the target fused image includes: The preset second fusion weight is multiplied by the value of each pixel in the detail-aware image, and the result is added to the corresponding pixel in the initial fusion image to obtain the pixel value of each pixel in the target fusion image, thereby obtaining the target fusion image.

7. The infrared image and visible light image fusion method according to claim 6, characterized in that, The preset second fusion weight p has the following value range: 0≤p≤0.

5.

8. A device for fusing infrared and visible light images, characterized in that, include: The first image processing module is used to calculate a visual perception factor that conforms to the visual characteristics of the human eye based on an infrared image and a visible light image, and to perform weighted fusion of the infrared image and the visible light image according to the visual perception factor to obtain an initial fused image. The second image processing module is used to perform brightness equalization processing on the initial fused image based on the brightness reference value in the initial fused image, extract detail features based on visual perception from the initial fused image, and fuse the extracted detail features into the brightness equalized initial fused image to obtain a detail-perceived image. The image fusion module is used to perform weighted fusion of the initial fused image and the detail-aware image to obtain the target fused image.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that run on the processor, the program or instructions being executed by the processor to implement the steps of the infrared image and visible light image fusion method as described in any one of claims 1 to 7.

10. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the infrared image and visible light image fusion method as described in any one of claims 1 to 7.

Images