Infrared Image and Visible Light Image Fusion Method, Device, Electronic Device and Medium

Abstract

The present application relates to an infrared image and visible light image fusion method, device, electronic device and medium, which includes: obtaining an infrared image and a visible light image corresponding to a target area; performing multi-scale decomposition on the infrared image to obtain first image information, and performing multi-scale decomposition on the visible light image to obtain second image information; determining the clarity information of the visible light image; determining a first weight corresponding to the first image information and a second weight corresponding to the second feature fusion based on the clarity information; and performing fusion based on the first image information, the second image information, the first weight and the second weight to obtain a fused image. The present application has the effect of improving the richness and reliability of the fused image in different scenarios.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an infrared image and visible light image fusion method, device, electronic device and medium. Background Art

[0002] Due to the limitations of human vision, different sensors can be used to obtain information that cannot be obtained by human vision in different situations. Image fusion technology can synergistically utilize the image information of multiple sensors in the same scene and generate a fused image to obtain a more accurate, reliable and comprehensive scene description, which is also convenient for human visual perception or further processing and analysis by a computer.

[0003] Among them, the images obtained by infrared and visible light sensors are complementary in content. Specifically, the infrared image sensor works on the principle of thermal imaging and is less affected by darkness or bad weather, but the imaging is generally darker and there is no color information. The visible light image has rich spectral information and can retain more detail and texture information, but it requires a well-illuminated working environment. The fusion of the two can effectively improve the ability of the image to describe scene details and thermal targets, and obtain more detailed and accurate information, which has wide application value in fields such as military operations, target detection, and tracking.

[0004] Since the imaging effect of the shooting area varies with time and other dynamic factors in the scene, how to improve the richness and reliability of the information in the fused image throughout the whole time period is an urgent problem to be solved. Summary of the Invention

[0005] In order to improve the richness and reliability of the fused image in different scenes, the present application provides an infrared image and visible light image fusion method, device, electronic device and medium.

[0006] In a first aspect, the present application provides an infrared image and visible light image fusion method, adopting the following technical solution:

[0007] An infrared image and visible light image fusion method includes:

[0008] Obtain an infrared image and a visible light image corresponding to a target area;

[0009] Perform multi-scale decomposition on the infrared image to obtain first image information, and perform multi-scale decomposition on the visible light image to obtain second image information;

[0010] Determine the clarity information of the visible light image;

[0011] Based on the clarity information, determine a first weight corresponding to the first image information and a second weight corresponding to the second image information;

[0012] Fuse based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image.

[0013] By adopting the above technical solution, under normal lighting conditions or when there is no occlusion, the visible light image has more detailed texture parts and higher image contrast; when the environmental scene has a greater impact, the infrared light image has stronger adaptability and anti-interference ability, and can retain more useful information. Therefore, it is necessary to determine whether the scene in the monitoring area of the visible light image has changed according to the clarity information of the visible light image (such as whether there are people coming in and out, whether there is occlusion, etc.), and adaptively adjust the weight of the fusion between the first image information and the second image information under different interference scenarios to improve the quality of the fused image.

[0014] Optionally, determining the clarity information of the visible light image includes:

[0015] Determine the average gray information, contrast information, and information entropy corresponding to the visible light image;

[0016] Based on the average gray information, contrast information, information entropy, and a preset corresponding relationship, determine the clarity information of the visible light image.

[0017] By adopting the above technical solution, the clarity information of the visible light image is determined based on the average gray information, contrast information, and information entropy, and the clarity of the visible light image can be accurately determined through multiple parameters.

[0018] Optionally, determining the clarity information of the visible light image includes:

[0019] Extract the first edge information corresponding to the infrared image and the second edge information corresponding to the visible light image;

[0020] Compare the first edge information and the second edge information to determine the difference information; determine the clarity information based on the difference information.

[0021] Optionally, the step of fusing based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image includes:

[0022] The first image information includes the first low-frequency component and the first high-frequency component obtained by wavelet transform of the infrared image, and the second image information includes the second low-frequency component and the second high-frequency component obtained by wavelet transform of the visible light image;

[0023] The first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the second weight includes a second low-frequency coefficient corresponding to the second low-frequency component;

[0024] A low-frequency image is obtained based on the first low-frequency component, the second low-frequency component, the first low-frequency coefficient, and the second low-frequency coefficient, and a high-frequency image is obtained based on the first high-frequency component and the second high-frequency component;

[0025] An inverse wavelet transform is performed based on the low-frequency image and the high-frequency image to obtain the fused image.

[0026] Optionally, determining the first weight corresponding to the first image information and the second weight corresponding to the second image information based on the clarity information includes: determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship.

[0027] Optionally, determining the second low-frequency coefficient based on the clarity information and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship includes:

[0028] The second low-frequency coefficient is: (k + h) / 2,

[0029] The first low-frequency coefficient is: 1 - (k + h) / 2;

[0030] Where h is a noise compensation value, and both k and h are greater than 0 and less than 1.

[0031] By adopting the above technical solution, the interference of the fusion noise compensation value on the weight allocation can be eliminated, and the accuracy of the allocated weight can be effectively improved.

[0032] Optionally, determining the noise compensation value h includes:

[0033] Determining a noise signal based on the visible light image information obtained in a historical time period;

[0034] Determining a noise intensity threshold of the noise signal;

[0035] Determining the noise compensation value h based on the noise threshold.

[0036] In a second aspect, the present application provides an infrared image and visible light image fusion device, adopting the following technical solution:

[0037] An infrared image and visible light image fusion device includes:

[0038] An acquisition module for acquiring an infrared image and a visible light image corresponding to a target area;

[0039] A decomposition module for performing multi-scale decomposition on the infrared image to obtain first image information and performing multi-scale decomposition on the visible light image to obtain second image information;

[0040] A first analysis module for determining the clarity information of the visible light image;

[0041] A weight assignment module for determining a first weight corresponding to the first image information and a second weight corresponding to the second image information based on the clarity information;

[0042] An inverse transformation module for fusing based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image.

[0043] Optionally, when determining the clarity information of the visible light image, the first analysis module is specifically configured to:

[0044] Determine the average gray level information, contrast information, and information entropy corresponding to the visible light image;

[0045] Determine the clarity information of the visible light image based on the average gray level information, contrast information, information entropy, and a preset correspondence relationship.

[0046] Optionally, when determining the clarity information of the visible light image, the first analysis module is specifically configured to:

[0047] Extract first edge information corresponding to the infrared image and second edge information corresponding to the visible light image;

[0048] Compare the first edge information and the second edge information to determine difference information; determine the clarity information based on the difference information.

[0049] When the weight assignment module fuses based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image, it is specifically configured to:

[0050] The first image information includes a first low-frequency component and a first high-frequency component obtained by wavelet transform of the infrared image, and the second image information includes a second low-frequency component and a second high-frequency component obtained by wavelet transform of the visible light image;

[0051] The first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the second weight includes a second low-frequency coefficient corresponding to the second low-frequency component;

[0052] An infrared image is obtained based on the first low-frequency component, the second low-frequency component, the first low-frequency coefficient, and the second low-frequency coefficient, and a high-frequency image is obtained based on the first high-frequency component and the second high-frequency component;

[0053] Based on the infrared image and the high-frequency image, an inverse wavelet transform is performed to obtain the fused image.

[0054] Optionally, when the weight distribution module determines the first weight corresponding to the first image information and the second weight corresponding to the second image information based on the clarity information, it specifically includes: determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset distribution relationship.

[0055] Optionally, when the first analysis module determines the second low-frequency coefficient based on the clarity information and determines the first low-frequency coefficient based on the second low-frequency coefficient and a preset distribution relationship, it specifically includes:

[0056] The second low-frequency coefficient is: (k + h) / 2,

[0057] The first low-frequency coefficient is: 1 - (k + h) / 2;

[0058] Where h is a noise compensation value, and both k and h are greater than 0 and less than 1.

[0059] Optionally, when the first analysis module determines the noise compensation value h, it specifically includes:

[0060] Determining a noise signal based on visible light image information obtained in a historical time period;

[0061] Determining a noise intensity threshold of the noise signal;

[0062] Determining the noise compensation value h based on the noise threshold.

[0063] In a third aspect, the present application provides an electronic device, adopting the following technical solution:

[0064] An electronic device, the electronic device includes:

[0065] One or more processors;

[0066] A memory;

[0067] One or more applications, where one or more applications are stored in the memory and are configured to be executed by one or more processors, and one or more applications are configured to: execute the above infrared image and visible light image fusion method.

[0068] Fourthly, the present application provides a computer-readable storage medium, adopting the following technical solution:

[0069] A computer-readable storage medium, comprising: a computer program stored therein that can be loaded and executed by a processor to perform the above infrared image and visible light image fusion method.

[0070] In summary, the present application includes the following beneficial technical effects:

[0071] Environmental changes affect the clarity of visible light images. According to the clarity information of visible light images under different external environmental conditions, the weight of the fusion between the first image information and the second image information is adaptively adjusted. While ensuring the clarity of the fused image, more details are retained to improve the quality of the fused image. Description of the Drawings

[0072] Figure 1 is a schematic flowchart of the method according to an embodiment of the present application;

[0073] Figure 2 is a schematic diagram of the device according to an embodiment of the present application;

[0074] Figure 3 is a schematic diagram of the electronic device according to an embodiment of the present application. Detailed Embodiments

[0075] The following further describes the present application in detail with reference to the accompanying drawings.

[0076] After reading this specification, those skilled in the art can make modifications to this embodiment without creative contributions as needed, but as long as they are within the scope of the claims of the present application, they are protected by the Patent Law.

[0077] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments of the present application belong to the scope of protection of the present application.

[0078] In addition, the term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0079] An embodiment of the present application provides an infrared image and visible light image fusion method, which is executed by an electronic device. Referring to Figure 1 , the method includes:

[0080] Step S101: Obtain an infrared image and a visible light image corresponding to a target area.

[0081] For the embodiment of the present application, the target area is a specific monitoring scenario. For example: an area for monitoring external personnel in a certain area within a factory building, or an area for monitoring instruments and equipment within a factory building, and so on.

[0082] Specifically, the infrared image and the visible light image can be taken with the same focus, for example, both are left-focused, or both are right-focused, or the infrared image and the visible light image can be taken with different focuses, for example, the infrared image is left-focused and the visible light image is right-focused, etc. The embodiment of the present application does not limit this.

[0083] Step S102: Perform multi-scale decomposition on the infrared image to obtain first image information, and perform multi-scale decomposition on the visible light image to obtain second image information.

[0084] Specifically, according to the level of information representation, image fusion can be divided into three categories from the degree of information abstraction, from high to low: decision-level image fusion, feature-level image fusion, and pixel-level image fusion. Among them, pixel-level image fusion is based on the basic data level, feature-level image fusion is based on the intermediate level (such as shape, region, etc. information), and decision-level image fusion aims to make an optimal decision.

[0085] In order to enable the fused image to provide rich texture maps and more profoundly reflect the morphological characteristics of the object to be measured, pixel-level image fusion is adopted in the embodiment of the present application.

[0086] Common pixel-level image fusion algorithms are divided into two types: the first type is image fusion algorithms based on the spatial domain, and the second type is image fusion algorithms based on the transform domain. The embodiment of the present application adopts image fusion algorithms based on the transform domain.

[0087] The specific process is as follows: First, convert the image to the transform domain and fuse the transform domain coefficients of the image. In the embodiment of the present application, image fusion algorithms based on multi-resolution analysis are adopted, and common algorithms include wavelet transform, pyramid transform, Contourlet transform, Fourier transform, Ridgelet transform, Curvelet transform, Bandelet transform, and so on.

[0088] The transform coefficients generated after multi-scale decomposition will complete coefficient fusion under the guidance of the fusion rule. The fusion rule determines how to select and discard coefficients, which decides the amount of information from the original images contained in the fused image and plays an important role in the fusion effect. The multi-scale decomposition coefficients after fusion will generate a reconstructed image through inverse transformation to complete the entire fusion process.

[0089] For the embodiments of the present application, the first image information is the transform coefficients obtained after multi-scale decomposition of an infrared image, and the second image information is the transform coefficients obtained after multi-scale decomposition of a visible light image.

[0090] Step S103: Determine the clarity information of the visible light image.

[0091] Continuing with the above example, when monitoring the working conditions of personnel or equipment in a certain area of the factory building, the visible light image is greatly affected by the environment (such as lighting, occlusion by obstacles, etc.). There may be local occlusion (obstacles) in the visible light image, or the entire image may be unclear (such as strong light irradiation or in dark conditions). Providing the image obtained by fusing the infrared light image and the visible light image for the management personnel to view can make the image more adaptable to the human eye's observation range while retaining more useful information.

[0092] Specifically, under normal lighting conditions or when there is no occlusion, the visible light image has more detailed texture parts and higher image contrast; when the environmental scene has a greater impact, the infrared light image has stronger adaptability and anti-interference ability and can retain more useful information. Therefore, it is necessary to determine whether the scene of its monitoring area has changed (such as whether there are people entering or leaving, whether there is occlusion, etc.) according to the clarity information of the visible light image, and adaptively adjust the fusion weight between the first image information and the second image information under different interference scenarios to improve the quality of the fused image.

[0093] Step S104: Determine the first weight corresponding to the first image information and the second weight corresponding to the second image information based on the clarity information.

[0094] Specifically, multiple clarity thresholds are preset, which are 0.3, 0.5, 0.7, and 1.0 respectively. When the clarity information of the visible light image is lower than 0.3, the first weight value is greater than the second weight value and shows a first preset corresponding relationship. When the clarity information of the visible light image is lower than 0.3 and greater than 0.5, the first weight value is greater than the second weight value and shows a second preset corresponding relationship. When the clarity information of the visible light image is greater than 0.5 and less than 0.7, the first weight value is less than the second weight value and shows a third preset corresponding relationship. When the clarity information of the visible light image is greater than 0.7 and less than 1.0, the first weight value is less than the second weight value and shows a fourth preset corresponding relationship. Thus, based on the clarity information of the visible light image, the weight distribution for the fusion of the visible light image and the infrared light image in this scenario can be determined.

[0095] Step S105: Based on the first image information, the second image information, the first weight, and the first weight, perform fusion to obtain a fused image.

[0096] According to the clarity information of the visible light image, adjust the weights of the corresponding features of the visible light image and the weights of the corresponding features of the infrared light image according to the dynamic scene changes, so as to realize the adaptive fusion of the first image information and the second image information, and can provide a clearer fused image in different scenarios in an automatically adjustable manner, so that the fused image retains more details suitable for the human eye.

[0097] The embodiments of the present application do not limit the steps between step S102 and step S103. Figure 1 It is only a schematic illustration of one of the situations in a method for fusing infrared images and visible light images in the embodiments of the present application.

[0098] In a possible implementation manner of the embodiments of the present application, determining the clarity information of the visible light image includes: determining the average gray information, the contrast information, and the information entropy corresponding to the visible light image; based on the average gray information, the contrast information, and the information entropy and the preset corresponding relationship, determining the clarity information of the visible light image.

[0099] Specifically, the average gray of the image directly reflects the average brightness and darkness of the image pixels. For the same shooting object, for the images obtained under different light intensities, their average gray values show a good linear relationship within a certain range; the image contrast can measure the size of the gray contrast in an image. In a dark environment, the contrast of the visible light image is low. As the light intensity of the shooting environment increases, the contrast gradually increases. However, when the light intensity is too high and the visible light image shows an "overexposed" effect, the image contrast will also decrease accordingly; the information entropy of the image reflects the amount of information in the image. When the illumination of the imaging environment is too dark or too bright, the information entropy of the image will be smaller than that of the image taken under normal illumination.

[0100] For the embodiment of the present application, statistics are respectively performed on the three parameters of average grayscale information, contrast information, and information entropy, and then the clarity information is obtained according to the preset corresponding relationship. The preset corresponding relationship is:

[0101] Clarity information = α × average grayscale information + β × contrast information + γ × information entropy;

[0102] Among them, α, β, and γ are coefficients.

[0103] In addition to being affected by illumination, visible light images may also be affected by other factors, such as smoke, dust, haze, etc. If gas particles block the light reflected from the object to be measured to the image sensor, it may cause blurred imaging of the object to be measured, or even make the image sensor unable to obtain the complete outline. Infrared radiation can penetrate smoke, and the imaging effect is almost unaffected by gas particles. Therefore, the outline information of the object to be measured can be more completely reflected in the infrared image.

[0104] Under the influence of the above-mentioned external environment, there are large differences in the outlines of the photographed objects in the image, so whether the visible light image is clear can be determined based on the outline differences. Therefore, in a possible implementation method of the embodiment of the present application, the clarity information of the visible light image is determined, including: extracting first edge information corresponding to the infrared image and second edge information corresponding to the visible light image; comparing the first edge information and the second edge information to determine difference information; and determining clarity information based on the difference information.

[0105] Specifically, the foreground image is extracted from the infrared image by using an image segmentation algorithm, and then the edge is extracted from the foreground image. The purpose of foreground segmentation of the infrared image is to extract the object to be measured from it, and try to exclude the background interference of the background environment. The edge extraction algorithm is used to extract the edge of the visible light image. After the edge parts of the two images are extracted, the first edge information is compared with the second edge information, and the first edge information is used as a reference to determine the completeness of the second edge information corresponding to the visible light image.

[0106] The method of determining the definition information based on the difference information is: definition information=δ×difference information (where δ is a coefficient).

[0107] In a possible implementation manner of the embodiment of the present application, based on the first image information, the second image information, the first weight, and the first weight, a fused image is obtained, including: the first image information includes a first low-frequency component and a first high-frequency component obtained by wavelet transform of an infrared image, and the second image information includes a second low-frequency component and a second high-frequency component obtained by wavelet transform of a visible light image; the first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the first weight includes a second low-frequency coefficient corresponding to the second low-frequency component; a low-frequency image is obtained based on the first low-frequency component, the second low-frequency component, the first low-frequency coefficient, and the second low-frequency coefficient; a high-frequency image is obtained based on the first high-frequency component and the second high-frequency component; and an inverse wavelet transform is performed based on the low-frequency image and the high-frequency image to obtain the fused image.

[0108] Specifically, the image fusion method based on wavelet transform can be divided into two parts: wavelet transform and fusion rule. The role of wavelet transform is to extract the high-frequency and low-frequency information of the image, and then inverse transform the fused high-frequency and low-frequency information into an image. The fusion rule includes a low-frequency subband fusion rule and a high-frequency subband fusion rule. The fusion rule determines the high-frequency and low-frequency information of the fused image, thus determining the quality of the fused image. Among them, the low-frequency subband of the image contains the main energy components of the image and is closely related to the brightness and contrast of the image. The high-frequency subband of the image contains the detail information of the image and is related to the sharpness of the image.

[0109] For the embodiment of the present application, in order to retain more detail information in the image, an adaptive weight assignment is adopted for the low-frequency component in the wavelet transform.

[0110] Specifically, the image fusion process based on wavelet is to first perform two-dimensional wavelet decomposition on the images to be fused, respectively obtaining 1 low-frequency subband and 3 high-frequency subbands, and then use different fusion rules to process the information of these 4 subbands to obtain the fused low-frequency and high-frequency subband information. Finally, an inverse two-dimensional wavelet transform is used to obtain the fused image. LL is the low-frequency subband after wavelet decomposition of the image; HL, LH, and HH are the 3 high-frequency subbands after wavelet decomposition of the image.

[0111] Obtaining a high-frequency image based on the first high-frequency component and the second high-frequency component includes: fusing the high-frequency subbands based on the local average gradient criterion:

[0112]

[0113] Among them, represents the average gradient; f(i,j) represents the coefficient value at a certain point; and respectively represent the first-order gradients in the x-axis and y-axis directions.

[0114] In a possible implementation manner of the embodiment of the present application, determining a first weight corresponding to the first image information and a second weight corresponding to the second image information based on the clarity information includes: determining a first low-frequency coefficient based on the clarity information; determining a second low-frequency coefficient based on the first weight and a preset allocation relationship.

[0115] Specifically, the process of determining the first low-frequency coefficient based on the clarity information is as follows:

[0116] Let the clarity information be k, then the second low-frequency coefficient corresponding to the visible light image is: (k + h) / 2, where h is a noise compensation value, and both k and h are greater than 0 and less than 1.

[0117] The first low-frequency coefficient corresponding to the infrared image is: 1 - (k + h) / 2.

[0118] Specifically, h is a noise compensation value, and determining the noise compensation includes step S001 (not shown in the figure), step S002 (not shown in the figure), and step 003 (not shown in the figure):

[0119] Step S001: Determine the noise signal based on the visible light image information obtained in the historical time period. Specifically, it includes:

[0120] Step S110 (not shown in the figure): Use the farthest-first strategy to select K clustering centers as the current clustering centers, where K is a natural number;

[0121] Step S111 (not shown in the figure): Cluster all the axial frequency domain signals according to the current clustering centers, and cluster each axial frequency domain signal into the clustering cluster represented by the nearest clustering center;

[0122] Step S112 (not shown in the figure): Calculate the mean value of each current clustering cluster as the new clustering center;

[0123] Step S113 (not shown in the figure): Determine whether the new clustering center is the same as the previous clustering center. If so, execute step S114; if not, use the new clustering center as the current clustering center, and loop steps S112 to S113;

[0124] Step S114 (not shown in the figure): Calculate the distance between any two clustering centers among all the new clustering centers;

[0125] Step S115 (not shown in the figure): Determine whether the distance between any two clustering centers is greater than the set reference threshold. If so, extract and screen out the clusters whose distance between any two clustering centers is greater than the set reference threshold, and use the axial frequency domain signals corresponding to the screened-out clusters as the noise signals; if not, output information indicating that there is no noise signal.

[0126] Step S002: Determine the noise intensity threshold of the noise signal.

[0127] Step S003: Determine the noise compensation value based on the noise threshold.

[0128] Extract multiple visible light images obtained in the historical time period, extract all noise signals and determine the intensity threshold. Assume the intensity threshold is p, and the noise compensation value h = R * p (R is a coefficient).

[0129] The above embodiments introduce an infrared image and visible light image fusion method from the perspective of the method flow. The following embodiments introduce an infrared image and visible light image fusion device from the perspective of virtual modules or virtual units. For details, see the following embodiments.

[0130] An embodiment of the present application provides an infrared image and visible light image fusion device 100. Refer to Figure 2 , including:

[0131] An acquisition module 1001, configured to acquire an infrared image and a visible light image corresponding to a target area;

[0132] A decomposition module 1002, configured to perform multi-scale decomposition on the infrared image to obtain first image information, and perform multi-scale decomposition on the visible light image to obtain second image information;

[0133] A first analysis module 1003, configured to determine the clarity information of the visible light image;

[0134] A weight assignment module 1004, configured to determine a first weight corresponding to the first image information and a second weight corresponding to the second image information based on the clarity information;

[0135] An inverse transform module 1005, configured to perform fusion based on the first image information, the second image information, the first weight, and the first weight to obtain a fused image.

[0136] In a possible implementation manner of an embodiment of the present application, when the first analysis module 1003 determines the clarity information of the visible light image, it is specifically configured to:

[0137] Determine the average gray level information, contrast information, and information entropy corresponding to the visible light image;

[0138] Determine the clarity information of the visible light image based on the average gray level information, contrast information, information entropy, and a preset correspondence relationship.

[0139] In a possible implementation manner of an embodiment of the present application, when the first analysis module 1003 determines the clarity information of the visible light image, it is specifically configured to:

[0140] Extract the first edge information corresponding to the infrared image and the second edge information corresponding to the visible light image;

[0141] Compare the first edge information and the second edge information to determine the difference information; determine the clarity information based on the difference information.

[0142] When the weight allocation module 1004 performs fusion based on the first image information, the second image information, the first weight, and the first weight to obtain the fused image, it is specifically used for:

[0143] The first image information includes a first low-frequency component and a first high-frequency component obtained by wavelet transform of the infrared image, and the second image information includes a second low-frequency component and a second high-frequency component obtained by wavelet transform of the visible light image;

[0144] The first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the second weight includes a second low-frequency coefficient corresponding to the second low-frequency component;

[0145] Based on the first low-frequency component, the second low-frequency component, the first low-frequency coefficient, and the second low-frequency coefficient, a low-frequency image is obtained, and a high-frequency image is obtained based on the first high-frequency component and the second high-frequency component;

[0146] Based on the low-frequency image and the high-frequency image, an inverse wavelet transform is performed to obtain the fused image.

[0147] In a possible implementation manner of the embodiment of the present application, when the weight allocation module 1004 determines the first weight corresponding to the first image information and the second weight corresponding to the second image information based on the clarity information, it is specifically used for, including: determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship.

[0148] In a possible implementation manner of the embodiment of the present application, when the first analysis module 1003 determines the second low-frequency coefficient based on the clarity information and determines the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship, it is specifically used for:

[0149] The second low-frequency coefficient is: (k + h) / 2,

[0150] The first low-frequency coefficient is: 1 - (k + h) / 2;

[0151] Wherein, h is a noise compensation value, and both k and h are greater than 0 and less than 1.

[0152] In a possible implementation manner of the embodiment of the present application, when the first analysis module 1003 determines the noise compensation value h, it is specifically used for:

[0153] Determine the noise signal based on the visible light image information obtained in the historical time period;

[0154] Determine the noise intensity threshold of the noise signal;

[0155] Determine the noise compensation value h based on the noise threshold.

[0156] An embodiment of the present application provides an electronic device, such as Figure 3 shown Figure 3 The electronic device 1100 shown includes: a processor 1101 and a memory 1103. Among them, the processor 1101 and the memory 1103 are connected, such as connected through a bus 1102. Optionally, the electronic device 1100 may further include a transceiver 1104. It should be noted that in practical applications, the transceiver 1104 is not limited to one, and the structure of the electronic device 1100 does not constitute a limitation to the embodiments of the present application.

[0157] The processor 1101 may be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in connection with the disclosure of the present application. The processor 1101 may also be a combination for implementing computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0158] The bus 1102 may include a path for transmitting information between the above components. The bus 1102 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard structure) bus, etc. The bus 1102 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 3 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0159] The memory 1103 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0160] The memory 1103 is used to store the application program code for executing the solution of this application, and is controlled by the processor 1101 to execute. The processor 1101 is used to execute the application program code stored in the memory 1103 to implement the content shown in the foregoing method embodiments.

[0161] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The shown electronic device is only an example and should not bring any restrictions to the functions and usage scope of the embodiments of the present disclosure.

[0162] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments.

[0163] It should be understood that although the steps in the flowchart of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0164] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An infrared image and visible light image fusion method, characterized in that, it includes: Obtain the infrared image and visible light image corresponding to the target area; Perform multi-scale decomposition on the infrared image to obtain first image information, and perform multi-scale decomposition on the visible light image to obtain second image information; Determine the clarity information of the visible light image; Based on the clarity information, determine the first weight corresponding to the first image information and the second weight corresponding to the second image information; Based on the first image information, the second image information, the first weight, and the second weight, perform fusion to obtain a fused image; The first image information includes a first low-frequency component and a first high-frequency component obtained by wavelet transform of the infrared image, and the second image information includes a second low-frequency component and a second high-frequency component obtained by wavelet transform of the visible light image; the first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the second weight includes a second low-frequency coefficient corresponding to the second low-frequency component; Based on the clarity information, determining the first weight corresponding to the first image information and the second weight corresponding to the second image information includes: determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship; Determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship includes: The second low-frequency coefficient is: (k + h) / 2, The first low-frequency coefficient is: 1 - (k + h) / 2; where k is the clarity information, h is the noise compensation value, and both k and h are greater than 0 and less than 1; The method for determining the noise compensation value h includes: Determine the noise signal based on the visible light image information obtained in the historical time period; Determine the noise intensity threshold of the noise signal; Determine the noise compensation value h based on the noise intensity threshold.

2. The method according to claim 1, characterized in that, Determining the clarity information of the visible light image includes: determining the average gray information, contrast information, and information entropy corresponding to the visible light image; Based on the average gray information, contrast information, information entropy, and a preset corresponding relationship, determine the clarity information of the visible light image.

3. The method according to claim 1, characterized in that, Determining the clarity information of the visible light image includes: extracting the first edge information corresponding to the infrared image and the second edge information corresponding to the visible light image; Compare the first edge information and the second edge information to determine the difference information; Based on the difference information, determine the clarity information.

4. The method according to claim 1, characterized in that, Performing fusion based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image includes: An infrared image is obtained based on the first low-frequency component, the second low-frequency component, the first low-frequency coefficient, and the second low-frequency coefficient, and a high-frequency image is obtained based on the first high-frequency component and the second high-frequency component; Based on the infrared image and the high-frequency image, an inverse wavelet transform is performed to obtain the fused image.

5. An infrared image and visible light image fusion device, Characterized in that, Comprising: An acquisition module for acquiring an infrared image and a visible light image corresponding to a target area; A decomposition module for performing multi-scale decomposition on the infrared image to obtain first image information, and performing multi-scale decomposition on the visible light image to obtain second image information; A first analysis module for determining the clarity information of the visible light image; A weight allocation module for determining a first weight corresponding to the first image information and a second weight corresponding to the second image information based on the clarity information; An inverse transform module for performing fusion based on the first image information, the second image information, the first weight, and the second weight to obtain a fused image; The first image information includes a first low-frequency component and a first high-frequency component obtained by wavelet transform of the infrared image, and the second image information includes a second low-frequency component and a second high-frequency component obtained by wavelet transform of the visible light image; the first weight includes a first low-frequency coefficient corresponding to the first low-frequency component, and the second weight includes a second low-frequency coefficient corresponding to the second low-frequency component; Determining the first weight corresponding to the first image information and the second weight corresponding to the second image information based on the clarity information includes: determining a second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship; Determining the second low-frequency coefficient based on the clarity information, and determining the first low-frequency coefficient based on the second low-frequency coefficient and a preset allocation relationship includes: The second low-frequency coefficient is: (k + h) / 2, The first low-frequency coefficient is: 1 - (k + h) / 2; Wherein, k is the clarity information, h is the noise compensation value, and both k and h are greater than 0 and less than 1; The method for determining the noise compensation value h includes: Determining a noise signal based on visible light image information obtained in a historical time period; Determining a noise intensity threshold of the noise signal; Determining the noise compensation value h based on the noise intensity threshold.

6. An electronic device, Characterized in that, The electronic device includes: One or more processors; A memory; One or more applications, wherein one or more applications are stored in the memory and are configured to be executed by one or more processors, and one or more applications are configured to: execute the method according to any one of claims 1-4.

7. A computer-readable storage medium, Characterized in that, Comprising: A computer program stored with the ability to be loaded and executed by a processor to execute any one of the methods in claims 1-4.

Images